2024-08-20T22:05:15.1150070Z Current runner version: '2.319.1' 2024-08-20T22:05:15.1156514Z Runner name: 'i-0b0df095cbe3f37a6' 2024-08-20T22:05:15.1157395Z Runner group name: 'Default' 2024-08-20T22:05:15.1158408Z Machine name: 'ip-10-0-48-129' 2024-08-20T22:05:15.1175650Z Testing runner upgrade compatibility 2024-08-20T22:05:15.3001144Z ##[group]GITHUB_TOKEN Permissions 2024-08-20T22:05:15.3003128Z Actions: read 2024-08-20T22:05:15.3003668Z Attestations: read 2024-08-20T22:05:15.3004324Z Checks: read 2024-08-20T22:05:15.3004843Z Contents: read 2024-08-20T22:05:15.3005362Z Deployments: read 2024-08-20T22:05:15.3005953Z Discussions: read 2024-08-20T22:05:15.3006455Z Issues: read 2024-08-20T22:05:15.3006952Z Metadata: read 2024-08-20T22:05:15.3007536Z Packages: read 2024-08-20T22:05:15.3008007Z Pages: read 2024-08-20T22:05:15.3008495Z PullRequests: read 2024-08-20T22:05:15.3009134Z RepositoryProjects: read 2024-08-20T22:05:15.3009735Z SecurityEvents: read 2024-08-20T22:05:15.3010245Z Statuses: read 2024-08-20T22:05:15.3011259Z ##[endgroup] 2024-08-20T22:05:15.3014543Z Secret source: Actions 2024-08-20T22:05:15.3015279Z Prepare workflow directory 2024-08-20T22:05:15.4388524Z Prepare all required actions 2024-08-20T22:05:15.4560319Z Getting action download info 2024-08-20T22:05:15.6158569Z Download action repository 'pytorch/test-infra@main' (SHA:0c3a2634aaa2f638c8f640e743f03d696ce1191f) 2024-08-20T22:05:16.1705727Z Download action repository 'pytorch/pytorch@main' (SHA:1ae5d5bb62141d7c9b1b0b66c66a462b4e10b1f2) 2024-08-20T22:05:20.1839028Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-20T22:05:20.3714659Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-20T22:05:20.6086133Z Getting action download info 2024-08-20T22:05:20.7106069Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-20T22:05:20.9131756Z Getting action download info 2024-08-20T22:05:21.0385429Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-20T22:05:21.1794484Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/133712 (40ec5f6ddd9787aca0449b24128343ff4c4a88b3) 2024-08-20T22:05:21.1796801Z ##[group] Inputs 2024-08-20T22:05:21.1797278Z build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:05:21.1804314Z test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:05:21.1812397Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:05:21.1813576Z sync-tag: 2024-08-20T22:05:21.1814437Z timeout-minutes: 240 2024-08-20T22:05:21.1814787Z use-gha: 2024-08-20T22:05:21.1815079Z dashboard-tag: 2024-08-20T22:05:21.1815415Z s3-bucket: gha-artifacts 2024-08-20T22:05:21.1815771Z aws-role-to-assume: 2024-08-20T22:05:21.1816113Z ##[endgroup] 2024-08-20T22:05:21.1817148Z Complete job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:05:21.2503007Z A job started hook has been configured by the self-hosted runner administrator 2024-08-20T22:05:21.2645697Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-20T22:05:21.2657631Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:05:21.2658178Z ##[endgroup] 2024-08-20T22:05:22.4084826Z Runner Type: linux.g5.4xlarge.nvidia.gpu 2024-08-20T22:05:22.4085363Z Instance Type: g5.4xlarge 2024-08-20T22:05:22.4086059Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-20T22:05:22.4086614Z AMI ID: ami-06c68f701d8090592 2024-08-20T22:05:28.1944850Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-20T22:05:28.1945412Z with: 2024-08-20T22:05:28.1946102Z github-secret: *** 2024-08-20T22:05:28.1947046Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:05:28.1948035Z activate-with-label: false 2024-08-20T22:05:28.1948402Z label: with-ssh 2024-08-20T22:05:28.1948725Z remove-existing-keys: true 2024-08-20T22:05:28.1949090Z fail-silently: true 2024-08-20T22:05:28.1949411Z env: 2024-08-20T22:05:28.1949686Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:05:28.1950050Z ##[endgroup] 2024-08-20T22:05:28.2856466Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-20T22:05:28.2859256Z ciflow reference detected, attempting to extract PR number 2024-08-20T22:05:28.5107161Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-08-20T22:05:28.6080739Z No SSH keys found for user pytorch-bot[bot] 2024-08-20T22:05:28.6081363Z Grabbing public ssh keys from https://github.com/XuehaiPan.keys 2024-08-20T22:05:28.6773190Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-20T22:05:28.6789661Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-20T22:05:28.6814169Z Login using: ssh ec2-user@ec2-54-234-174-51.compute-1.amazonaws.com 2024-08-20T22:05:28.6814958Z All testing is done inside the container, to start an interactive session run: 2024-08-20T22:05:28.6815926Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T22:05:28.6948326Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-20T22:05:28.6948879Z with: 2024-08-20T22:05:28.6949166Z submodules: recursive 2024-08-20T22:05:28.6949505Z fetch-depth: 0 2024-08-20T22:05:28.6949802Z env: 2024-08-20T22:05:28.6950081Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:05:28.6950420Z ##[endgroup] 2024-08-20T22:05:28.7144039Z ##[group]Run retry () { 2024-08-20T22:05:28.7144439Z retry () { 2024-08-20T22:05:28.7144960Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-20T22:05:28.7145521Z } 2024-08-20T22:05:28.7145836Z echo "${GITHUB_WORKSPACE}" 2024-08-20T22:05:28.7146353Z if [ -z "${NO_SUDO}" ]; then 2024-08-20T22:05:28.7146988Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:05:28.7147617Z else 2024-08-20T22:05:28.7148083Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T22:05:28.7148682Z fi 2024-08-20T22:05:28.7149444Z mkdir "${GITHUB_WORKSPACE}" 2024-08-20T22:05:28.7158803Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:05:28.7159311Z env: 2024-08-20T22:05:28.7159592Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:05:28.7159933Z NO_SUDO: 2024-08-20T22:05:28.7160207Z ##[endgroup] 2024-08-20T22:05:28.7191246Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:05:28.8567707Z ##[group]Run malfet/checkout@silent-checkout 2024-08-20T22:05:28.8568138Z with: 2024-08-20T22:05:28.8568473Z ref: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:05:28.8568921Z fetch-depth: 0 2024-08-20T22:05:28.8569234Z submodules: recursive 2024-08-20T22:05:28.8569576Z quiet-checkout: true 2024-08-20T22:05:28.8569926Z repository: pytorch/pytorch 2024-08-20T22:05:28.8570499Z token: *** 2024-08-20T22:05:28.8570792Z ssh-strict: true 2024-08-20T22:05:28.8571119Z persist-credentials: true 2024-08-20T22:05:28.8571475Z clean: true 2024-08-20T22:05:28.8571808Z sparse-checkout-cone-mode: true 2024-08-20T22:05:28.8572210Z lfs: false 2024-08-20T22:05:28.8572505Z set-safe-directory: true 2024-08-20T22:05:28.8572842Z env: 2024-08-20T22:05:28.8573116Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:05:28.8573448Z ##[endgroup] 2024-08-20T22:05:28.9553522Z Syncing repository: pytorch/pytorch 2024-08-20T22:05:28.9555080Z ##[group]Getting Git version info 2024-08-20T22:05:28.9555869Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:05:28.9556712Z [command]/usr/bin/git version 2024-08-20T22:05:28.9557083Z git version 2.40.1 2024-08-20T22:05:28.9562179Z ##[endgroup] 2024-08-20T22:05:28.9575699Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/f08a93e1-5076-441b-828b-4af6a13ea4bf/.gitconfig' 2024-08-20T22:05:28.9582861Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/f08a93e1-5076-441b-828b-4af6a13ea4bf' before making global git config changes 2024-08-20T22:05:28.9584093Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T22:05:28.9587434Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:05:28.9645937Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T22:05:28.9649676Z ##[group]Initializing the repository 2024-08-20T22:05:28.9652657Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:05:28.9697766Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-20T22:05:28.9698758Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-20T22:05:28.9699561Z hint: of your new repositories, which will suppress this warning, call: 2024-08-20T22:05:28.9700304Z hint: 2024-08-20T22:05:28.9700771Z hint: git config --global init.defaultBranch 2024-08-20T22:05:28.9701244Z hint: 2024-08-20T22:05:28.9702067Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-20T22:05:28.9702936Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-20T22:05:28.9703511Z hint: 2024-08-20T22:05:28.9703836Z hint: git branch -m 2024-08-20T22:05:28.9704588Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-20T22:05:28.9711312Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-20T22:05:28.9757183Z ##[endgroup] 2024-08-20T22:05:28.9757776Z ##[group]Disabling automatic garbage collection 2024-08-20T22:05:28.9760077Z [command]/usr/bin/git config --local gc.auto 0 2024-08-20T22:05:28.9801543Z ##[endgroup] 2024-08-20T22:05:28.9802071Z ##[group]Setting up auth 2024-08-20T22:05:28.9806789Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T22:05:28.9849755Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T22:05:29.0204088Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T22:05:29.0247278Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T22:05:29.0598282Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:05:29.0657835Z ##[endgroup] 2024-08-20T22:05:29.0658428Z ##[group]Fetching the repository 2024-08-20T22:05:29.0663906Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-20T22:05:31.8464280Z remote: Enumerating objects: 1008518 2024-08-20T22:05:31.8465275Z remote: Enumerating objects: 1010208, done. 2024-08-20T22:05:31.8466480Z remote: Counting objects: 0% (1/1690) 2024-08-20T22:05:31.8468475Z remote: Counting objects: 1% (17/1690) 2024-08-20T22:05:31.8469291Z remote: Counting objects: 2% (34/1690) 2024-08-20T22:05:31.8469951Z remote: Counting objects: 3% (51/1690) 2024-08-20T22:05:31.8470614Z remote: Counting objects: 4% (68/1690) 2024-08-20T22:05:31.8471279Z remote: Counting objects: 5% (85/1690) 2024-08-20T22:05:31.8471936Z remote: Counting objects: 6% (102/1690) 2024-08-20T22:05:31.8472628Z remote: Counting objects: 7% (119/1690) 2024-08-20T22:05:31.8473271Z remote: Counting objects: 8% (136/1690) 2024-08-20T22:05:31.8473874Z remote: Counting objects: 9% (153/1690) 2024-08-20T22:05:31.8474562Z remote: Counting objects: 10% (169/1690) 2024-08-20T22:05:31.8475090Z remote: Counting objects: 11% (186/1690) 2024-08-20T22:05:31.8475601Z remote: Counting objects: 12% (203/1690) 2024-08-20T22:05:31.8476120Z remote: Counting objects: 13% (220/1690) 2024-08-20T22:05:31.8476637Z remote: Counting objects: 14% (237/1690) 2024-08-20T22:05:31.8477143Z remote: Counting objects: 15% (254/1690) 2024-08-20T22:05:31.8477650Z remote: Counting objects: 16% (271/1690) 2024-08-20T22:05:31.8478151Z remote: Counting objects: 17% (288/1690) 2024-08-20T22:05:31.8478646Z remote: Counting objects: 18% (305/1690) 2024-08-20T22:05:31.8479148Z remote: Counting objects: 19% (322/1690) 2024-08-20T22:05:31.8479657Z remote: Counting objects: 20% (338/1690) 2024-08-20T22:05:31.8480157Z remote: Counting objects: 21% (355/1690) 2024-08-20T22:05:31.8480757Z remote: Counting objects: 22% (372/1690) 2024-08-20T22:05:31.8481264Z remote: Counting objects: 23% (389/1690) 2024-08-20T22:05:31.8481777Z remote: Counting objects: 24% (406/1690) 2024-08-20T22:05:31.8482290Z remote: Counting objects: 25% (423/1690) 2024-08-20T22:05:31.8483228Z remote: Counting objects: 26% (440/1690) 2024-08-20T22:05:31.8483925Z remote: Counting objects: 27% (457/1690) 2024-08-20T22:05:31.8484588Z remote: Counting objects: 28% (474/1690) 2024-08-20T22:05:31.8485147Z remote: Counting objects: 29% (491/1690) 2024-08-20T22:05:31.8485647Z remote: Counting objects: 30% (507/1690) 2024-08-20T22:05:31.8486144Z remote: Counting objects: 31% (524/1690) 2024-08-20T22:05:31.8486656Z remote: Counting objects: 32% (541/1690) 2024-08-20T22:05:31.8487160Z remote: Counting objects: 33% (558/1690) 2024-08-20T22:05:31.8487664Z remote: Counting objects: 34% (575/1690) 2024-08-20T22:05:31.8488299Z remote: Counting objects: 35% (592/1690) 2024-08-20T22:05:31.8488983Z remote: Counting objects: 36% (609/1690) 2024-08-20T22:05:31.8489653Z remote: Counting objects: 37% (626/1690) 2024-08-20T22:05:31.8490462Z remote: Counting objects: 38% (643/1690) 2024-08-20T22:05:31.8491299Z remote: Counting objects: 39% (660/1690) 2024-08-20T22:05:31.8491804Z remote: Counting objects: 40% (676/1690) 2024-08-20T22:05:31.8492309Z remote: Counting objects: 41% (693/1690) 2024-08-20T22:05:31.8492808Z remote: Counting objects: 42% (710/1690) 2024-08-20T22:05:31.8493303Z remote: Counting objects: 43% (727/1690) 2024-08-20T22:05:31.8493807Z remote: Counting objects: 44% (744/1690) 2024-08-20T22:05:31.8494310Z remote: Counting objects: 45% (761/1690) 2024-08-20T22:05:31.8494812Z remote: Counting objects: 46% (778/1690) 2024-08-20T22:05:31.8495307Z remote: Counting objects: 47% (795/1690) 2024-08-20T22:05:31.8495816Z remote: Counting objects: 48% (812/1690) 2024-08-20T22:05:31.8496323Z remote: Counting objects: 49% (829/1690) 2024-08-20T22:05:31.8496827Z remote: Counting objects: 50% (845/1690) 2024-08-20T22:05:31.8497328Z remote: Counting objects: 51% (862/1690) 2024-08-20T22:05:31.8497837Z remote: Counting objects: 52% (879/1690) 2024-08-20T22:05:31.8498344Z remote: Counting objects: 53% (896/1690) 2024-08-20T22:05:31.8498844Z remote: Counting objects: 54% (913/1690) 2024-08-20T22:05:31.8499347Z remote: Counting objects: 55% (930/1690) 2024-08-20T22:05:31.8499851Z remote: Counting objects: 56% (947/1690) 2024-08-20T22:05:31.8500356Z remote: Counting objects: 57% (964/1690) 2024-08-20T22:05:31.8500856Z remote: Counting objects: 58% (981/1690) 2024-08-20T22:05:31.8501363Z remote: Counting objects: 59% (998/1690) 2024-08-20T22:05:31.8501879Z remote: Counting objects: 60% (1014/1690) 2024-08-20T22:05:31.8502412Z remote: Counting objects: 61% (1031/1690) 2024-08-20T22:05:31.8502934Z remote: Counting objects: 62% (1048/1690) 2024-08-20T22:05:31.8503467Z remote: Counting objects: 63% (1065/1690) 2024-08-20T22:05:31.8504001Z remote: Counting objects: 64% (1082/1690) 2024-08-20T22:05:31.8504547Z remote: Counting objects: 65% (1099/1690) 2024-08-20T22:05:31.8505065Z remote: Counting objects: 66% (1116/1690) 2024-08-20T22:05:31.8505588Z remote: Counting objects: 67% (1133/1690) 2024-08-20T22:05:31.8506106Z remote: Counting objects: 68% (1150/1690) 2024-08-20T22:05:31.8506628Z remote: Counting objects: 69% (1167/1690) 2024-08-20T22:05:31.8507145Z remote: Counting objects: 70% (1183/1690) 2024-08-20T22:05:31.8507670Z remote: Counting objects: 71% (1200/1690) 2024-08-20T22:05:31.8508186Z remote: Counting objects: 72% (1217/1690) 2024-08-20T22:05:31.8508703Z remote: Counting objects: 73% (1234/1690) 2024-08-20T22:05:31.8509222Z remote: Counting objects: 74% (1251/1690) 2024-08-20T22:05:31.8509742Z remote: Counting objects: 75% (1268/1690) 2024-08-20T22:05:31.8510251Z remote: Counting objects: 76% (1285/1690) 2024-08-20T22:05:31.8510868Z remote: Counting objects: 77% (1302/1690) 2024-08-20T22:05:31.8511396Z remote: Counting objects: 78% (1319/1690) 2024-08-20T22:05:31.8511911Z remote: Counting objects: 79% (1336/1690) 2024-08-20T22:05:31.8512421Z remote: Counting objects: 80% (1352/1690) 2024-08-20T22:05:31.8512939Z remote: Counting objects: 81% (1369/1690) 2024-08-20T22:05:31.8513453Z remote: Counting objects: 82% (1386/1690) 2024-08-20T22:05:31.8513970Z remote: Counting objects: 83% (1403/1690) 2024-08-20T22:05:31.8514483Z remote: Counting objects: 84% (1420/1690) 2024-08-20T22:05:31.8514997Z remote: Counting objects: 85% (1437/1690) 2024-08-20T22:05:31.8515514Z remote: Counting objects: 86% (1454/1690) 2024-08-20T22:05:31.8516031Z remote: Counting objects: 87% (1471/1690) 2024-08-20T22:05:31.8516546Z remote: Counting objects: 88% (1488/1690) 2024-08-20T22:05:31.8517063Z remote: Counting objects: 89% (1505/1690) 2024-08-20T22:05:31.8517684Z remote: Counting objects: 90% (1521/1690) 2024-08-20T22:05:31.8518206Z remote: Counting objects: 91% (1538/1690) 2024-08-20T22:05:31.8518715Z remote: Counting objects: 92% (1555/1690) 2024-08-20T22:05:31.8519235Z remote: Counting objects: 93% (1572/1690) 2024-08-20T22:05:31.8519752Z remote: Counting objects: 94% (1589/1690) 2024-08-20T22:05:31.8520269Z remote: Counting objects: 95% (1606/1690) 2024-08-20T22:05:31.8520779Z remote: Counting objects: 96% (1623/1690) 2024-08-20T22:05:31.8521297Z remote: Counting objects: 97% (1640/1690) 2024-08-20T22:05:31.8521814Z remote: Counting objects: 98% (1657/1690) 2024-08-20T22:05:31.8522330Z remote: Counting objects: 99% (1674/1690) 2024-08-20T22:05:31.8522846Z remote: Counting objects: 100% (1690/1690) 2024-08-20T22:05:31.8523403Z remote: Counting objects: 100% (1690/1690), done. 2024-08-20T22:05:31.8596756Z remote: Compressing objects: 0% (1/875) 2024-08-20T22:05:31.8912551Z remote: Compressing objects: 1% (9/875) 2024-08-20T22:05:31.9174726Z remote: Compressing objects: 2% (18/875) 2024-08-20T22:05:31.9681267Z remote: Compressing objects: 3% (27/875) 2024-08-20T22:05:32.0440574Z remote: Compressing objects: 4% (35/875) 2024-08-20T22:05:32.0998292Z remote: Compressing objects: 5% (44/875) 2024-08-20T22:05:32.1431533Z remote: Compressing objects: 6% (53/875) 2024-08-20T22:05:32.1857455Z remote: Compressing objects: 7% (62/875) 2024-08-20T22:05:32.2306585Z remote: Compressing objects: 8% (70/875) 2024-08-20T22:05:32.2609628Z remote: Compressing objects: 9% (79/875) 2024-08-20T22:05:32.2841709Z remote: Compressing objects: 10% (88/875) 2024-08-20T22:05:32.3128646Z remote: Compressing objects: 11% (97/875) 2024-08-20T22:05:32.3344573Z remote: Compressing objects: 12% (105/875) 2024-08-20T22:05:32.3540757Z remote: Compressing objects: 13% (114/875) 2024-08-20T22:05:32.3692748Z remote: Compressing objects: 14% (123/875) 2024-08-20T22:05:32.3820058Z remote: Compressing objects: 15% (132/875) 2024-08-20T22:05:32.3904626Z remote: Compressing objects: 16% (140/875) 2024-08-20T22:05:32.3998556Z remote: Compressing objects: 17% (149/875) 2024-08-20T22:05:32.4061348Z remote: Compressing objects: 18% (158/875) 2024-08-20T22:05:32.4111685Z remote: Compressing objects: 19% (167/875) 2024-08-20T22:05:32.4155356Z remote: Compressing objects: 20% (175/875) 2024-08-20T22:05:32.4182424Z remote: Compressing objects: 21% (184/875) 2024-08-20T22:05:32.4200695Z remote: Compressing objects: 22% (193/875) 2024-08-20T22:05:32.4203900Z remote: Compressing objects: 23% (202/875) 2024-08-20T22:05:32.4212175Z remote: Compressing objects: 24% (210/875) 2024-08-20T22:05:32.4220583Z remote: Compressing objects: 25% (219/875) 2024-08-20T22:05:32.4227684Z remote: Compressing objects: 26% (228/875) 2024-08-20T22:05:32.4235199Z remote: Compressing objects: 27% (237/875) 2024-08-20T22:05:32.4258087Z remote: Compressing objects: 28% (245/875) 2024-08-20T22:05:32.4273381Z remote: Compressing objects: 29% (254/875) 2024-08-20T22:05:32.4288333Z remote: Compressing objects: 30% (263/875) 2024-08-20T22:05:32.4292086Z remote: Compressing objects: 31% (272/875) 2024-08-20T22:05:32.4303193Z remote: Compressing objects: 32% (280/875) 2024-08-20T22:05:32.4309282Z remote: Compressing objects: 33% (289/875) 2024-08-20T22:05:32.4316958Z remote: Compressing objects: 34% (298/875) 2024-08-20T22:05:32.4322018Z remote: Compressing objects: 35% (307/875) 2024-08-20T22:05:32.4332269Z remote: Compressing objects: 36% (315/875) 2024-08-20T22:05:32.4339991Z remote: Compressing objects: 37% (324/875) 2024-08-20T22:05:32.4347055Z remote: Compressing objects: 38% (333/875) 2024-08-20T22:05:32.4352344Z remote: Compressing objects: 39% (342/875) 2024-08-20T22:05:32.4359025Z remote: Compressing objects: 40% (350/875) 2024-08-20T22:05:32.4367892Z remote: Compressing objects: 41% (359/875) 2024-08-20T22:05:32.4371663Z remote: Compressing objects: 42% (368/875) 2024-08-20T22:05:32.4376993Z remote: Compressing objects: 43% (377/875) 2024-08-20T22:05:32.4384177Z remote: Compressing objects: 44% (385/875) 2024-08-20T22:05:32.4390227Z remote: Compressing objects: 45% (394/875) 2024-08-20T22:05:32.4395144Z remote: Compressing objects: 46% (403/875) 2024-08-20T22:05:32.4397707Z remote: Compressing objects: 47% (412/875) 2024-08-20T22:05:32.4402172Z remote: Compressing objects: 48% (420/875) 2024-08-20T22:05:32.4404965Z remote: Compressing objects: 49% (429/875) 2024-08-20T22:05:32.4407452Z remote: Compressing objects: 50% (438/875) 2024-08-20T22:05:32.4411146Z remote: Compressing objects: 51% (447/875) 2024-08-20T22:05:32.4413812Z remote: Compressing objects: 52% (455/875) 2024-08-20T22:05:32.4416300Z remote: Compressing objects: 53% (464/875) 2024-08-20T22:05:32.4418896Z remote: Compressing objects: 54% (473/875) 2024-08-20T22:05:32.4421456Z remote: Compressing objects: 55% (482/875) 2024-08-20T22:05:32.4421979Z remote: Compressing objects: 56% (490/875) 2024-08-20T22:05:32.4424059Z remote: Compressing objects: 57% (499/875) 2024-08-20T22:05:32.4424589Z remote: Compressing objects: 58% (508/875) 2024-08-20T22:05:32.4426599Z remote: Compressing objects: 59% (517/875) 2024-08-20T22:05:32.4427118Z remote: Compressing objects: 60% (525/875) 2024-08-20T22:05:32.4429273Z remote: Compressing objects: 61% (534/875) 2024-08-20T22:05:32.4429800Z remote: Compressing objects: 62% (543/875) 2024-08-20T22:05:32.4430321Z remote: Compressing objects: 63% (552/875) 2024-08-20T22:05:32.4432033Z remote: Compressing objects: 64% (560/875) 2024-08-20T22:05:32.4439235Z remote: Compressing objects: 65% (569/875) 2024-08-20T22:05:32.4445576Z remote: Compressing objects: 66% (578/875) 2024-08-20T22:05:32.4449432Z remote: Compressing objects: 67% (587/875) 2024-08-20T22:05:32.4456029Z remote: Compressing objects: 68% (595/875) 2024-08-20T22:05:32.4461348Z remote: Compressing objects: 69% (604/875) 2024-08-20T22:05:32.4463882Z remote: Compressing objects: 70% (613/875) 2024-08-20T22:05:32.4467418Z remote: Compressing objects: 71% (622/875) 2024-08-20T22:05:32.4471238Z remote: Compressing objects: 72% (630/875) 2024-08-20T22:05:32.4475121Z remote: Compressing objects: 73% (639/875) 2024-08-20T22:05:32.4478150Z remote: Compressing objects: 74% (648/875) 2024-08-20T22:05:32.4481866Z remote: Compressing objects: 75% (657/875) 2024-08-20T22:05:32.4484527Z remote: Compressing objects: 76% (665/875) 2024-08-20T22:05:32.4487554Z remote: Compressing objects: 77% (674/875) 2024-08-20T22:05:32.4490033Z remote: Compressing objects: 78% (683/875) 2024-08-20T22:05:32.4492613Z remote: Compressing objects: 79% (692/875) 2024-08-20T22:05:32.4495059Z remote: Compressing objects: 80% (700/875) 2024-08-20T22:05:32.4497580Z remote: Compressing objects: 81% (709/875) 2024-08-20T22:05:32.4500133Z remote: Compressing objects: 82% (718/875) 2024-08-20T22:05:32.4500656Z remote: Compressing objects: 83% (727/875) 2024-08-20T22:05:32.4502646Z remote: Compressing objects: 84% (735/875) 2024-08-20T22:05:32.4505268Z remote: Compressing objects: 85% (744/875) 2024-08-20T22:05:32.4507813Z remote: Compressing objects: 86% (753/875) 2024-08-20T22:05:32.4508338Z remote: Compressing objects: 87% (762/875) 2024-08-20T22:05:32.4510469Z remote: Compressing objects: 88% (770/875) 2024-08-20T22:05:32.4512977Z remote: Compressing objects: 89% (779/875) 2024-08-20T22:05:32.4515751Z remote: Compressing objects: 90% (788/875) 2024-08-20T22:05:32.4516283Z remote: Compressing objects: 91% (797/875) 2024-08-20T22:05:32.4518198Z remote: Compressing objects: 92% (805/875) 2024-08-20T22:05:32.4518732Z remote: Compressing objects: 93% (814/875) 2024-08-20T22:05:32.4519261Z remote: Compressing objects: 94% (823/875) 2024-08-20T22:05:32.4520619Z remote: Compressing objects: 95% (832/875) 2024-08-20T22:05:32.4523354Z remote: Compressing objects: 96% (840/875) 2024-08-20T22:05:32.4523872Z remote: Compressing objects: 97% (849/875) 2024-08-20T22:05:32.4524392Z remote: Compressing objects: 98% (858/875) 2024-08-20T22:05:32.4525972Z remote: Compressing objects: 99% (867/875) 2024-08-20T22:05:32.4526502Z remote: Compressing objects: 100% (875/875) 2024-08-20T22:05:32.4527067Z remote: Compressing objects: 100% (875/875), done. 2024-08-20T22:05:54.6787066Z remote: Total 1010208 (delta 1086), reused 1298 (delta 812), pack-reused 1008518 (from 1) 2024-08-20T22:06:17.7796920Z [command]/usr/bin/git rev-parse --verify --quiet 40ec5f6ddd9787aca0449b24128343ff4c4a88b3^{object} 2024-08-20T22:06:17.7833786Z 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:06:17.7838977Z ##[endgroup] 2024-08-20T22:06:17.7839541Z ##[group]Determining the checkout info 2024-08-20T22:06:17.7840289Z ##[endgroup] 2024-08-20T22:06:17.7842293Z ##[group]Checking out the ref 2024-08-20T22:06:17.7848066Z [command]/usr/bin/git checkout --quiet --force 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:06:19.5193796Z ##[endgroup] 2024-08-20T22:06:19.5194412Z ##[group]Setting up auth for fetching submodules 2024-08-20T22:06:19.5197581Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T22:06:19.5280519Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-20T22:06:19.5321828Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-20T22:06:19.5365615Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-20T22:06:19.5404875Z ##[endgroup] 2024-08-20T22:06:19.5405683Z ##[group]Fetching submodules 2024-08-20T22:06:19.5409394Z [command]/usr/bin/git submodule sync --recursive 2024-08-20T22:06:19.5791772Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-20T22:06:19.6157168Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-20T22:06:19.6158881Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-20T22:06:19.6161839Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-20T22:06:19.6165307Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-20T22:06:19.6168742Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-20T22:06:19.6173116Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-20T22:06:19.6176772Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-20T22:06:19.6180424Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-20T22:06:19.6184043Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-20T22:06:19.6187877Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-20T22:06:19.6191558Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-20T22:06:19.6195332Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-20T22:06:19.6199227Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-20T22:06:19.6203298Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-20T22:06:19.6207274Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-20T22:06:19.6212992Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:06:19.6220607Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-20T22:06:19.6224951Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-20T22:06:19.6229152Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-20T22:06:19.6233827Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-20T22:06:19.6238296Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-20T22:06:19.6243365Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-20T22:06:19.6248191Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-20T22:06:19.6252998Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-20T22:06:19.6257695Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-20T22:06:19.6262825Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-20T22:06:19.6267509Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-20T22:06:19.6272492Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-20T22:06:19.6277510Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-20T22:06:19.6282693Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-20T22:06:19.6287960Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-20T22:06:19.6293397Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-20T22:06:19.6298533Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-20T22:06:19.6307376Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-20T22:06:19.6342893Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-20T22:06:19.9922458Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-20T22:06:20.1703356Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-20T22:06:20.3698086Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-20T22:06:20.6080226Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-20T22:06:22.5760196Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-20T22:06:34.0453409Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-20T22:06:34.4833061Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-20T22:06:34.9179496Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-20T22:06:35.5228604Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-20T22:06:36.7963425Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-20T22:06:38.9659562Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-20T22:06:44.8364291Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-20T22:06:46.2672683Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-20T22:06:48.0555338Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-20T22:06:49.3308467Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-20T22:06:49.7516442Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-20T22:06:50.0784053Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-20T22:06:51.2127452Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-20T22:06:51.6093746Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-20T22:06:51.8555763Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-20T22:06:53.5520163Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-20T22:06:54.3410002Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-20T22:06:55.0196980Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-20T22:07:01.2155079Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-20T22:07:03.2438871Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-20T22:07:07.5725792Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-20T22:07:07.8166000Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-20T22:07:17.3759738Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-20T22:07:17.5570982Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-20T22:07:17.7595132Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-20T22:07:18.7365067Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-20T22:07:19.0136323Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-20T22:07:19.7086427Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-20T22:07:20.1122727Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-20T22:07:20.1277744Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-20T22:07:20.1397332Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-20T22:07:20.1712045Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-20T22:07:20.2147097Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-20T22:07:21.3573356Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-20T22:07:21.3861679Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-20T22:07:21.4387603Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-20T22:07:21.5521646Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-20T22:07:21.5929160Z Submodule path 'third_party/cudnn_frontend': checked out '23511ba176243f27b3b275da1fb3814ea805a171' 2024-08-20T22:07:22.2236934Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-20T22:07:22.5117793Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-20T22:07:22.6065682Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-20T22:07:22.6088832Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:07:22.6092530Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:07:22.6095992Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:07:22.6099533Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:07:22.6103313Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:07:22.6135436Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-20T22:07:23.6899059Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-20T22:07:24.2867702Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-20T22:07:26.4517092Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-20T22:07:27.4903305Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-20T22:07:27.8269106Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-20T22:07:27.9387739Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-20T22:07:28.4519094Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-20T22:07:28.5228777Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-20T22:07:28.5378420Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-20T22:07:28.6909652Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-20T22:07:28.7367219Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-20T22:07:28.7820834Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-20T22:07:28.8145699Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-20T22:07:28.8666774Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:07:28.8825879Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-20T22:07:28.8844961Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-20T22:07:28.8874239Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-20T22:07:42.3001854Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-20T22:07:42.3222749Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-20T22:07:42.4221322Z Submodule path 'third_party/kineto': checked out 'd9753139d181b9ff42872465aac0e5d3018be415' 2024-08-20T22:07:42.4246244Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:07:42.4249511Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:07:42.4253375Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:07:42.4284050Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-20T22:07:42.9810300Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-20T22:07:44.2436319Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-20T22:07:45.3528998Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-20T22:07:45.3548950Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:07:45.3552323Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:07:45.3555785Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:07:45.3559413Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:07:45.3563087Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:07:45.3566861Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:07:45.3570539Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:07:45.3574770Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:07:45.3606343Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-20T22:07:46.2010001Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-20T22:07:46.5718992Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-20T22:07:47.8376054Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-20T22:07:48.1275593Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-20T22:07:48.6491940Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-20T22:07:49.7041510Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-20T22:07:56.4613767Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-20T22:07:56.8614332Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-20T22:07:56.8848537Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-20T22:07:56.9281677Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-20T22:07:56.9449876Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-20T22:07:56.9469957Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:07:56.9501794Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-20T22:07:57.2413378Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-20T22:07:57.2647472Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-20T22:07:57.3129797Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-20T22:07:57.4371960Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-20T22:07:57.4581408Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-20T22:07:57.5020330Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-20T22:07:57.5685524Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-20T22:07:57.6136292Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-20T22:07:57.6439287Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-20T22:07:57.7698638Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-20T22:07:58.2873204Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-20T22:07:58.2911425Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-20T22:07:58.2914943Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-20T22:07:58.2949108Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-20T22:07:58.7247944Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-20T22:07:59.6430440Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-20T22:07:59.6843578Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-20T22:07:59.7737908Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-20T22:07:59.7761861Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:07:59.7765253Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:07:59.7768681Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:07:59.7772305Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:07:59.7776264Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:07:59.7779840Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:07:59.7783492Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:07:59.7786999Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:07:59.7823223Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-20T22:08:00.2263790Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-20T22:08:01.3040044Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-20T22:08:01.6371700Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-20T22:08:07.8938108Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-20T22:08:08.1501697Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-20T22:08:08.3504869Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-20T22:08:08.6429072Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-20T22:08:15.3360157Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-20T22:08:15.3823237Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-20T22:08:15.4008489Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-20T22:08:15.5211430Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-20T22:08:15.5373798Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-20T22:08:15.5558582Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-20T22:08:15.5763646Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-20T22:08:15.5782671Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:15.5786849Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:15.5816992Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-20T22:08:17.4727871Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-20T22:08:18.9698176Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-20T22:08:19.0223511Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T22:08:19.6936119Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-20T22:08:19.7079536Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-20T22:08:20.0264505Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-20T22:08:20.0291076Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:20.0294838Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:20.0328467Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-20T22:08:20.4920574Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-20T22:08:21.5568722Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-20T22:08:21.6388945Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-20T22:08:21.6510289Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-20T22:08:21.6663846Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-20T22:08:21.7097035Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-20T22:08:21.7433301Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-20T22:08:21.7917904Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-20T22:08:21.8261540Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-20T22:08:21.8283332Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:21.8286682Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:21.8289926Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:21.8293702Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:21.8326081Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-20T22:08:22.9033320Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-20T22:08:23.1741861Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-20T22:08:24.4235691Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-20T22:08:25.3984573Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-20T22:08:25.4178893Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-20T22:08:25.4973427Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-20T22:08:25.5322919Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-20T22:08:25.5343223Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:25.5375080Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-20T22:08:25.7644385Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-20T22:08:25.7689309Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-20T22:08:25.8053866Z Entering 'android/libs/fbjni' 2024-08-20T22:08:25.8106237Z Entering 'third_party/FP16' 2024-08-20T22:08:25.8158334Z Entering 'third_party/FXdiv' 2024-08-20T22:08:25.8210103Z Entering 'third_party/NNPACK' 2024-08-20T22:08:25.8262196Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:08:25.8313170Z Entering 'third_party/XNNPACK' 2024-08-20T22:08:25.8387482Z Entering 'third_party/benchmark' 2024-08-20T22:08:25.8438343Z Entering 'third_party/cpp-httplib' 2024-08-20T22:08:25.8491096Z Entering 'third_party/cpuinfo' 2024-08-20T22:08:25.8543303Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:08:25.8594476Z Entering 'third_party/cutlass' 2024-08-20T22:08:25.8654572Z Entering 'third_party/eigen' 2024-08-20T22:08:25.8706928Z Entering 'third_party/fbgemm' 2024-08-20T22:08:25.8758070Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:08:25.8807440Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:08:25.8858794Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:08:25.8914942Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:08:25.8964786Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:08:25.9017663Z Entering 'third_party/flatbuffers' 2024-08-20T22:08:25.9072909Z Entering 'third_party/fmt' 2024-08-20T22:08:25.9124762Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:08:25.9182583Z Entering 'third_party/gloo' 2024-08-20T22:08:25.9237774Z Entering 'third_party/googletest' 2024-08-20T22:08:25.9289914Z Entering 'third_party/ideep' 2024-08-20T22:08:25.9339396Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:08:25.9401421Z Entering 'third_party/ittapi' 2024-08-20T22:08:25.9459578Z Entering 'third_party/kineto' 2024-08-20T22:08:25.9514898Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:08:25.9565401Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:08:25.9621110Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:08:25.9673320Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:08:25.9729009Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:08:25.9779281Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:08:25.9834391Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:08:25.9886884Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:08:25.9940599Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:08:25.9994926Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:08:26.0053366Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:08:26.0104477Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:08:26.0159466Z Entering 'third_party/mimalloc' 2024-08-20T22:08:26.0213181Z Entering 'third_party/nccl/nccl' 2024-08-20T22:08:26.0266487Z Entering 'third_party/nlohmann' 2024-08-20T22:08:26.0319436Z Entering 'third_party/onnx' 2024-08-20T22:08:26.0386644Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:08:26.0437838Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:08:26.0494710Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:08:26.0550384Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:08:26.0600523Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:08:26.0651127Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:08:26.0700992Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:08:26.0753136Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:08:26.0802663Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:08:26.0852534Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:08:26.0900983Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:26.0955074Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:26.1010751Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:08:26.1089718Z Entering 'third_party/pocketfft' 2024-08-20T22:08:26.1143574Z Entering 'third_party/protobuf' 2024-08-20T22:08:26.1197698Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:26.1251891Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:26.1305591Z Entering 'third_party/psimd' 2024-08-20T22:08:26.1359622Z Entering 'third_party/pthreadpool' 2024-08-20T22:08:26.1411418Z Entering 'third_party/pybind11' 2024-08-20T22:08:26.1469307Z Entering 'third_party/python-peachpy' 2024-08-20T22:08:26.1521460Z Entering 'third_party/sleef' 2024-08-20T22:08:26.1575412Z Entering 'third_party/tensorpipe' 2024-08-20T22:08:26.1628325Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:26.1679684Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:26.1729199Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:26.1782513Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:26.1831178Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:26.1906536Z ##[endgroup] 2024-08-20T22:08:26.1909336Z ##[group]Persisting credentials for submodules 2024-08-20T22:08:26.1911687Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-20T22:08:26.2274083Z Entering 'android/libs/fbjni' 2024-08-20T22:08:26.2339489Z Entering 'third_party/FP16' 2024-08-20T22:08:26.2408069Z Entering 'third_party/FXdiv' 2024-08-20T22:08:26.2475106Z Entering 'third_party/NNPACK' 2024-08-20T22:08:26.2546430Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:08:26.2612968Z Entering 'third_party/XNNPACK' 2024-08-20T22:08:26.2697449Z Entering 'third_party/benchmark' 2024-08-20T22:08:26.2763677Z Entering 'third_party/cpp-httplib' 2024-08-20T22:08:26.2829960Z Entering 'third_party/cpuinfo' 2024-08-20T22:08:26.2898163Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:08:26.2968032Z Entering 'third_party/cutlass' 2024-08-20T22:08:26.3046135Z Entering 'third_party/eigen' 2024-08-20T22:08:26.3113812Z Entering 'third_party/fbgemm' 2024-08-20T22:08:26.3180829Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:08:26.3247869Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:08:26.3314377Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:08:26.3386564Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:08:26.3452514Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:08:26.3519448Z Entering 'third_party/flatbuffers' 2024-08-20T22:08:26.3588890Z Entering 'third_party/fmt' 2024-08-20T22:08:26.3656678Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:08:26.3724966Z Entering 'third_party/gloo' 2024-08-20T22:08:26.3797485Z Entering 'third_party/googletest' 2024-08-20T22:08:26.3866198Z Entering 'third_party/ideep' 2024-08-20T22:08:26.3929948Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:08:26.4003702Z Entering 'third_party/ittapi' 2024-08-20T22:08:26.4070287Z Entering 'third_party/kineto' 2024-08-20T22:08:26.4138102Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:08:26.4202727Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:08:26.4276157Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:08:26.4342892Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:08:26.4410105Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:08:26.4475045Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:08:26.4546322Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:08:26.4613317Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:08:26.4680386Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:08:26.4749584Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:08:26.4820617Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:08:26.4887774Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:08:26.4957441Z Entering 'third_party/mimalloc' 2024-08-20T22:08:26.5025436Z Entering 'third_party/nccl/nccl' 2024-08-20T22:08:26.5096103Z Entering 'third_party/nlohmann' 2024-08-20T22:08:26.5167963Z Entering 'third_party/onnx' 2024-08-20T22:08:26.5248412Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:08:26.5315946Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:08:26.5388029Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:08:26.5455258Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:08:26.5521911Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:08:26.5589443Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:08:26.5656240Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:08:26.5723710Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:08:26.5790363Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:08:26.5857992Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:08:26.5922587Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:26.5991660Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:26.6063382Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:08:26.6151903Z Entering 'third_party/pocketfft' 2024-08-20T22:08:26.6220654Z Entering 'third_party/protobuf' 2024-08-20T22:08:26.6290916Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:26.6356718Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:26.6425933Z Entering 'third_party/psimd' 2024-08-20T22:08:26.6492747Z Entering 'third_party/pthreadpool' 2024-08-20T22:08:26.6559669Z Entering 'third_party/pybind11' 2024-08-20T22:08:26.6626201Z Entering 'third_party/python-peachpy' 2024-08-20T22:08:26.6692717Z Entering 'third_party/sleef' 2024-08-20T22:08:26.6759268Z Entering 'third_party/tensorpipe' 2024-08-20T22:08:26.6824346Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:26.6891277Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:26.6957480Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:26.7023542Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:26.7086759Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:26.7181501Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-20T22:08:26.7544021Z Entering 'android/libs/fbjni' 2024-08-20T22:08:26.7605005Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-20T22:08:26.7626119Z Entering 'third_party/FP16' 2024-08-20T22:08:26.7686800Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-20T22:08:26.7709761Z Entering 'third_party/FXdiv' 2024-08-20T22:08:26.7770996Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-20T22:08:26.7791866Z Entering 'third_party/NNPACK' 2024-08-20T22:08:26.7859159Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-20T22:08:26.7880415Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:08:26.7941857Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-20T22:08:26.7963736Z Entering 'third_party/XNNPACK' 2024-08-20T22:08:26.8029507Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-20T22:08:26.8066722Z Entering 'third_party/benchmark' 2024-08-20T22:08:26.8126577Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:08:26.8148139Z Entering 'third_party/cpp-httplib' 2024-08-20T22:08:26.8209219Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-20T22:08:26.8230698Z Entering 'third_party/cpuinfo' 2024-08-20T22:08:26.8296193Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:08:26.8317914Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:08:26.8379451Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-20T22:08:26.8400641Z Entering 'third_party/cutlass' 2024-08-20T22:08:26.8471859Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:08:26.8500358Z Entering 'third_party/eigen' 2024-08-20T22:08:26.8562923Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-20T22:08:26.8586445Z Entering 'third_party/fbgemm' 2024-08-20T22:08:26.8648382Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-20T22:08:26.8668422Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:08:26.8729295Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-20T22:08:26.8750438Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:08:26.8812448Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T22:08:26.8832155Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:08:26.8894006Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-20T22:08:26.8921356Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:08:26.8983672Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:26.9004682Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:08:26.9067104Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-20T22:08:26.9090597Z Entering 'third_party/flatbuffers' 2024-08-20T22:08:26.9152268Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-20T22:08:26.9176189Z Entering 'third_party/fmt' 2024-08-20T22:08:26.9241428Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:08:26.9267646Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:08:26.9328492Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-20T22:08:26.9350079Z Entering 'third_party/gloo' 2024-08-20T22:08:26.9410892Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-20T22:08:26.9431849Z Entering 'third_party/googletest' 2024-08-20T22:08:26.9493170Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:26.9514501Z Entering 'third_party/ideep' 2024-08-20T22:08:26.9577183Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-20T22:08:26.9595973Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:08:26.9657146Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-20T22:08:26.9686218Z Entering 'third_party/ittapi' 2024-08-20T22:08:26.9747309Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-20T22:08:26.9768138Z Entering 'third_party/kineto' 2024-08-20T22:08:26.9829718Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-20T22:08:26.9849312Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:08:26.9913336Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-20T22:08:26.9932856Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:08:26.9995613Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-20T22:08:27.0017676Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:08:27.0085083Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-20T22:08:27.0105490Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:08:27.0168256Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-20T22:08:27.0188812Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:08:27.0251444Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-20T22:08:27.0269910Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:08:27.0332762Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-20T22:08:27.0356629Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:08:27.0418139Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-20T22:08:27.0438849Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:08:27.0501244Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:27.0521990Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:08:27.0589876Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-20T22:08:27.0611247Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:08:27.0673147Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-20T22:08:27.0696219Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:08:27.0757984Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-20T22:08:27.0778230Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:08:27.0838230Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-20T22:08:27.0863753Z Entering 'third_party/mimalloc' 2024-08-20T22:08:27.0925099Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-20T22:08:27.0946834Z Entering 'third_party/nccl/nccl' 2024-08-20T22:08:27.1009595Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-20T22:08:27.1031013Z Entering 'third_party/nlohmann' 2024-08-20T22:08:27.1097468Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-20T22:08:27.1120085Z Entering 'third_party/onnx' 2024-08-20T22:08:27.1182509Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-20T22:08:27.1216714Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:08:27.1279042Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:08:27.1300787Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:08:27.1365186Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:08:27.1390859Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:08:27.1453462Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-20T22:08:27.1474774Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:08:27.1536654Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:08:27.1556869Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:08:27.1618124Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:27.1638260Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:08:27.1699624Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-20T22:08:27.1720073Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:08:27.1782199Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-20T22:08:27.1804864Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:08:27.1866060Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-20T22:08:27.1886291Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:08:27.1947613Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-20T22:08:27.1968456Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:08:27.2032472Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-20T22:08:27.2051996Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:27.2114004Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-20T22:08:27.2136312Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:27.2200382Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-20T22:08:27.2222200Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:08:27.2286812Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-20T22:08:27.2328978Z Entering 'third_party/pocketfft' 2024-08-20T22:08:27.2390470Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-20T22:08:27.2411847Z Entering 'third_party/protobuf' 2024-08-20T22:08:27.2473346Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-20T22:08:27.2495960Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:27.2557493Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-20T22:08:27.2578200Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:27.2639448Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:27.2663777Z Entering 'third_party/psimd' 2024-08-20T22:08:27.2725247Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-20T22:08:27.2748572Z Entering 'third_party/pthreadpool' 2024-08-20T22:08:27.2809299Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-20T22:08:27.2830360Z Entering 'third_party/pybind11' 2024-08-20T22:08:27.2892050Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:08:27.2913249Z Entering 'third_party/python-peachpy' 2024-08-20T22:08:27.2974124Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-20T22:08:27.2995018Z Entering 'third_party/sleef' 2024-08-20T22:08:27.3063500Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-20T22:08:27.3084612Z Entering 'third_party/tensorpipe' 2024-08-20T22:08:27.3151547Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-20T22:08:27.3171208Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:27.3232558Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-20T22:08:27.3254740Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:27.3316513Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-20T22:08:27.3335788Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:27.3398015Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-20T22:08:27.3418552Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:27.3480574Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-20T22:08:27.3499004Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:27.3561302Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-20T22:08:27.4512315Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-20T22:08:27.4883912Z Entering 'android/libs/fbjni' 2024-08-20T22:08:27.4935148Z Entering 'third_party/FP16' 2024-08-20T22:08:27.4987933Z Entering 'third_party/FXdiv' 2024-08-20T22:08:27.5039992Z Entering 'third_party/NNPACK' 2024-08-20T22:08:27.5092737Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:08:27.5145300Z Entering 'third_party/XNNPACK' 2024-08-20T22:08:27.5213032Z Entering 'third_party/benchmark' 2024-08-20T22:08:27.5266065Z Entering 'third_party/cpp-httplib' 2024-08-20T22:08:27.5319202Z Entering 'third_party/cpuinfo' 2024-08-20T22:08:27.5372370Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:08:27.5424879Z Entering 'third_party/cutlass' 2024-08-20T22:08:27.5486498Z Entering 'third_party/eigen' 2024-08-20T22:08:27.5540510Z Entering 'third_party/fbgemm' 2024-08-20T22:08:27.5594158Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:08:27.5647887Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:08:27.5699996Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:08:27.5756103Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:08:27.5805605Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:08:27.5859013Z Entering 'third_party/flatbuffers' 2024-08-20T22:08:27.5915888Z Entering 'third_party/fmt' 2024-08-20T22:08:27.5968597Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:08:27.6021832Z Entering 'third_party/gloo' 2024-08-20T22:08:27.6076484Z Entering 'third_party/googletest' 2024-08-20T22:08:27.6128334Z Entering 'third_party/ideep' 2024-08-20T22:08:27.6181862Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:08:27.6239755Z Entering 'third_party/ittapi' 2024-08-20T22:08:27.6292137Z Entering 'third_party/kineto' 2024-08-20T22:08:27.6343657Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:08:27.6398341Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:08:27.6450192Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:08:27.6501819Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:08:27.6554362Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:08:27.6603950Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:08:27.6658370Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:08:27.6709249Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:08:27.6760905Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:08:27.6813216Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:08:27.6867488Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:08:27.6920768Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:08:27.6975057Z Entering 'third_party/mimalloc' 2024-08-20T22:08:27.7028247Z Entering 'third_party/nccl/nccl' 2024-08-20T22:08:27.7083267Z Entering 'third_party/nlohmann' 2024-08-20T22:08:27.7137592Z Entering 'third_party/onnx' 2024-08-20T22:08:27.7210196Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:08:27.7261353Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:08:27.7317565Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:08:27.7374450Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:08:27.7429904Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:08:27.7480757Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:08:27.7531021Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:08:27.7588855Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:08:27.7638546Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:08:27.7688304Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:08:27.7738163Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:27.7791344Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:27.7846053Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:08:27.7919989Z Entering 'third_party/pocketfft' 2024-08-20T22:08:27.7973802Z Entering 'third_party/protobuf' 2024-08-20T22:08:27.8027780Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:27.8077721Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:27.8133244Z Entering 'third_party/psimd' 2024-08-20T22:08:27.8190195Z Entering 'third_party/pthreadpool' 2024-08-20T22:08:27.8243620Z Entering 'third_party/pybind11' 2024-08-20T22:08:27.8297883Z Entering 'third_party/python-peachpy' 2024-08-20T22:08:27.8351297Z Entering 'third_party/sleef' 2024-08-20T22:08:27.8405775Z Entering 'third_party/tensorpipe' 2024-08-20T22:08:27.8457181Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:27.8506937Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:27.8556199Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:27.8605652Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:27.8654155Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:27.8729544Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-20T22:08:27.9090774Z Entering 'android/libs/fbjni' 2024-08-20T22:08:27.9143915Z Entering 'third_party/FP16' 2024-08-20T22:08:27.9195412Z Entering 'third_party/FXdiv' 2024-08-20T22:08:27.9248810Z Entering 'third_party/NNPACK' 2024-08-20T22:08:27.9300009Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:08:27.9351424Z Entering 'third_party/XNNPACK' 2024-08-20T22:08:27.9417237Z Entering 'third_party/benchmark' 2024-08-20T22:08:27.9468866Z Entering 'third_party/cpp-httplib' 2024-08-20T22:08:27.9520430Z Entering 'third_party/cpuinfo' 2024-08-20T22:08:27.9572976Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:08:27.9626460Z Entering 'third_party/cutlass' 2024-08-20T22:08:27.9686017Z Entering 'third_party/eigen' 2024-08-20T22:08:27.9739633Z Entering 'third_party/fbgemm' 2024-08-20T22:08:27.9791902Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:08:27.9841407Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:08:27.9894082Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:08:27.9955732Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:08:28.0005066Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:08:28.0059080Z Entering 'third_party/flatbuffers' 2024-08-20T22:08:28.0112954Z Entering 'third_party/fmt' 2024-08-20T22:08:28.0165315Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:08:28.0219653Z Entering 'third_party/gloo' 2024-08-20T22:08:28.0272689Z Entering 'third_party/googletest' 2024-08-20T22:08:28.0327693Z Entering 'third_party/ideep' 2024-08-20T22:08:28.0381903Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:08:28.0442779Z Entering 'third_party/ittapi' 2024-08-20T22:08:28.0493487Z Entering 'third_party/kineto' 2024-08-20T22:08:28.0543209Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:08:28.0594835Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:08:28.0647383Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:08:28.0696160Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:08:28.0747319Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:08:28.0795582Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:08:28.0850603Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:08:28.0900763Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:08:28.0950891Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:08:28.1001668Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:08:28.1056449Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:08:28.1106257Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:08:28.1160165Z Entering 'third_party/mimalloc' 2024-08-20T22:08:28.1210875Z Entering 'third_party/nccl/nccl' 2024-08-20T22:08:28.1261482Z Entering 'third_party/nlohmann' 2024-08-20T22:08:28.1312836Z Entering 'third_party/onnx' 2024-08-20T22:08:28.1378416Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:08:28.1429812Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:08:28.1487004Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:08:28.1538488Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:08:28.1588471Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:08:28.1638838Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:08:28.1689541Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:08:28.1741933Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:08:28.1791931Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:08:28.1841221Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:08:28.1892483Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:08:28.1944224Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:08:28.2001069Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:08:28.2073356Z Entering 'third_party/pocketfft' 2024-08-20T22:08:28.2123804Z Entering 'third_party/protobuf' 2024-08-20T22:08:28.2176617Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:08:28.2227564Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:08:28.2282406Z Entering 'third_party/psimd' 2024-08-20T22:08:28.2333196Z Entering 'third_party/pthreadpool' 2024-08-20T22:08:28.2387694Z Entering 'third_party/pybind11' 2024-08-20T22:08:28.2438810Z Entering 'third_party/python-peachpy' 2024-08-20T22:08:28.2489399Z Entering 'third_party/sleef' 2024-08-20T22:08:28.2539699Z Entering 'third_party/tensorpipe' 2024-08-20T22:08:28.2589642Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:08:28.2639253Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:08:28.2688970Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:08:28.2741027Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:08:28.2790691Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:08:28.2865007Z ##[endgroup] 2024-08-20T22:08:28.2914853Z [command]/usr/bin/git log -1 --format='%H' 2024-08-20T22:08:28.2955112Z '40ec5f6ddd9787aca0449b24128343ff4c4a88b3' 2024-08-20T22:08:28.3171565Z Prepare all required actions 2024-08-20T22:08:28.3172023Z Getting action download info 2024-08-20T22:08:28.4554919Z ##[group]Run ./.github/actions/setup-linux 2024-08-20T22:08:28.4555344Z env: 2024-08-20T22:08:28.4555619Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:28.4555953Z ##[endgroup] 2024-08-20T22:08:28.4618236Z ##[group]Run set -euo pipefail 2024-08-20T22:08:28.4618680Z set -euo pipefail 2024-08-20T22:08:28.4619071Z function get_ec2_metadata() { 2024-08-20T22:08:28.4619608Z  # Pulled from instance metadata endpoint for EC2 2024-08-20T22:08:28.4620491Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-20T22:08:28.4621258Z  category=$1 2024-08-20T22:08:28.4621786Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-20T22:08:28.4622430Z  runner_name_str=i-0b0df095cbe3f37a6 2024-08-20T22:08:28.4622936Z  if [[ -f /.inarc ]]; then 2024-08-20T22:08:28.4623431Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-20T22:08:28.4624001Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-20T22:08:28.4624702Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-20T22:08:28.4625309Z  else 2024-08-20T22:08:28.4625800Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-20T22:08:28.4626380Z  fi 2024-08-20T22:08:28.4626670Z } 2024-08-20T22:08:28.4627034Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-20T22:08:28.4627612Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-20T22:08:28.4628263Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-20T22:08:28.4628833Z echo "system info $(uname -a)" 2024-08-20T22:08:28.4638971Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:28.4639462Z env: 2024-08-20T22:08:28.4639751Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:28.4640104Z ##[endgroup] 2024-08-20T22:08:28.4744432Z ami-id: ami-06c68f701d8090592 2024-08-20T22:08:28.4808837Z instance-id: i-0b0df095cbe3f37a6 2024-08-20T22:08:28.4867473Z instance-type: g5.4xlarge 2024-08-20T22:08:28.4881281Z system info Linux ip-10-0-48-129.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-20T22:08:28.4904525Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:08:28.4905439Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T22:08:28.4914567Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:28.4915094Z env: 2024-08-20T22:08:28.4915391Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:28.4915739Z ##[endgroup] 2024-08-20T22:08:28.4994611Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-20T22:08:28.4995253Z if systemctl is-active --quiet docker; then 2024-08-20T22:08:28.4995808Z  echo "Docker daemon is running..."; 2024-08-20T22:08:28.4996423Z else 2024-08-20T22:08:28.4996925Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-20T22:08:28.4997504Z fi 2024-08-20T22:08:28.5006259Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:28.5006759Z env: 2024-08-20T22:08:28.5007038Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:28.5007382Z ##[endgroup] 2024-08-20T22:08:28.5099612Z Docker daemon is running... 2024-08-20T22:08:28.5180860Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:08:28.5181424Z with: 2024-08-20T22:08:28.5181695Z shell: bash 2024-08-20T22:08:28.5181990Z timeout_minutes: 5 2024-08-20T22:08:28.5182322Z max_attempts: 3 2024-08-20T22:08:28.5182639Z retry_wait_seconds: 30 2024-08-20T22:08:28.5185732Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2024-08-20T22:08:28.5188678Z polling_interval_seconds: 1 2024-08-20T22:08:28.5189057Z warning_on_retry: true 2024-08-20T22:08:28.5189404Z continue_on_error: false 2024-08-20T22:08:28.5189740Z env: 2024-08-20T22:08:28.5190015Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:28.5190378Z AWS_RETRY_MODE: standard 2024-08-20T22:08:28.5190726Z AWS_MAX_ATTEMPTS: 5 2024-08-20T22:08:28.5191072Z AWS_DEFAULT_REGION: us-east-1 2024-08-20T22:08:28.5191449Z ##[endgroup] 2024-08-20T22:08:29.8464607Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:08:29.8465593Z Configure a credential helper to remove this warning. See 2024-08-20T22:08:29.8466501Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:08:29.8467013Z 2024-08-20T22:08:29.8467136Z Login Succeeded 2024-08-20T22:08:30.5759411Z Command completed after 1 attempt(s). 2024-08-20T22:08:30.5825732Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:08:30.5826456Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:08:30.5827110Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T22:08:30.5836061Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:30.5836563Z env: 2024-08-20T22:08:30.5836844Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:30.5837192Z ##[endgroup] 2024-08-20T22:08:30.5931990Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:08:30.5932757Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:08:30.5933356Z # shellcheck disable=SC2046 2024-08-20T22:08:30.5933815Z docker stop $(docker ps -q) || true 2024-08-20T22:08:30.5934287Z # Prune all of the docker images 2024-08-20T22:08:30.5934736Z docker system prune -af 2024-08-20T22:08:30.5944310Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:30.5944810Z env: 2024-08-20T22:08:30.5945084Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:30.5945430Z ##[endgroup] 2024-08-20T22:08:30.6261140Z "docker stop" requires at least 1 argument. 2024-08-20T22:08:30.6261879Z See 'docker stop --help'. 2024-08-20T22:08:30.6262119Z 2024-08-20T22:08:30.6262340Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-20T22:08:30.6262718Z 2024-08-20T22:08:30.6265381Z Stop one or more running containers 2024-08-20T22:08:30.6462747Z Total reclaimed space: 0B 2024-08-20T22:08:30.6508199Z ##[group]Run set +e 2024-08-20T22:08:30.6508550Z set +e 2024-08-20T22:08:30.6509017Z set -x 2024-08-20T22:08:30.6509320Z  2024-08-20T22:08:30.6509639Z PT_DOMAIN=download.pytorch.org 2024-08-20T22:08:30.6510490Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-20T22:08:30.6511563Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-20T22:08:30.6512311Z # one is returned at random 2024-08-20T22:08:30.6512856Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-20T22:08:30.6513361Z  2024-08-20T22:08:30.6513679Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:08:30.6514298Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-20T22:08:30.6515237Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-20T22:08:30.6515788Z  2024-08-20T22:08:30.6516117Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T22:08:30.6516679Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-20T22:08:30.6517185Z  exit 1 2024-08-20T22:08:30.6517492Z  fi 2024-08-20T22:08:30.6517777Z fi 2024-08-20T22:08:30.6518056Z  2024-08-20T22:08:30.6518417Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-20T22:08:30.6518929Z  # Clean up any old records first 2024-08-20T22:08:30.6519438Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-20T22:08:30.6519894Z fi 2024-08-20T22:08:30.6520165Z  2024-08-20T22:08:30.6520596Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-20T22:08:30.6521139Z cat /etc/hosts 2024-08-20T22:08:30.6529869Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:30.6530360Z env: 2024-08-20T22:08:30.6530803Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:30.6531197Z ##[endgroup] 2024-08-20T22:08:30.6561884Z + PT_DOMAIN=download.pytorch.org 2024-08-20T22:08:30.6568213Z ++ dig -4 +short download.pytorch.org 2024-08-20T22:08:30.6568693Z ++ tail -n1 2024-08-20T22:08:30.7001453Z + RESOLVED_IP=18.160.10.36 2024-08-20T22:08:30.7001920Z + '[' -z 18.160.10.36 ']' 2024-08-20T22:08:30.7002369Z + grep -r download.pytorch.org /etc/hosts 2024-08-20T22:08:30.7019755Z + echo '18.160.10.36 download.pytorch.org' 2024-08-20T22:08:30.7020353Z + sudo tee -a /etc/hosts 2024-08-20T22:08:30.8454422Z 18.160.10.36 download.pytorch.org 2024-08-20T22:08:30.8477886Z + cat /etc/hosts 2024-08-20T22:08:30.8486879Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-20T22:08:30.8500012Z ::1 localhost6 localhost6.localdomain6 2024-08-20T22:08:30.8500736Z 18.160.10.36 download.pytorch.org 2024-08-20T22:08:30.8652905Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-20T22:08:30.8653499Z with: 2024-08-20T22:08:30.8654738Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:30.8656151Z docker-build-dir: .ci/docker 2024-08-20T22:08:30.8656558Z working-directory: . 2024-08-20T22:08:30.8657065Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:30.8657642Z force-push: false 2024-08-20T22:08:30.8657959Z env: 2024-08-20T22:08:30.8658241Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:30.8658593Z ##[endgroup] 2024-08-20T22:08:30.8677199Z ##[group]Run set -ex 2024-08-20T22:08:30.8677567Z set -ex 2024-08-20T22:08:30.8677858Z  2024-08-20T22:08:30.8678418Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-20T22:08:30.8679390Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-20T22:08:30.8680185Z # job could then download the pre-built image as usual 2024-08-20T22:08:30.8680928Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-20T22:08:30.8681769Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:30.8682400Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:30.8682957Z  2024-08-20T22:08:30.8683463Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-20T22:08:30.8684087Z  exit 0 2024-08-20T22:08:30.8684396Z else 2024-08-20T22:08:30.8684761Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:30.8685216Z fi 2024-08-20T22:08:30.8685494Z  2024-08-20T22:08:30.8685965Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-20T22:08:30.8686808Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-20T22:08:30.8687580Z  # use it as it is, but first let's extract the tag 2024-08-20T22:08:30.8688289Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-20T22:08:30.8689016Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:30.8689714Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:30.8690271Z else 2024-08-20T22:08:30.8690823Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-20T22:08:30.8691489Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:30.8692375Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:30.8693111Z fi 2024-08-20T22:08:30.8701789Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:30.8702295Z env: 2024-08-20T22:08:30.8702579Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:30.8702934Z REPO_NAME: pytorch 2024-08-20T22:08:30.8704072Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:30.8705294Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:08:30.8705794Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:30.8706310Z ##[endgroup] 2024-08-20T22:08:30.8737354Z + [[ ! -d .ci/docker ]] 2024-08-20T22:08:30.8737775Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-20T22:08:30.8738147Z + echo skip=false 2024-08-20T22:08:30.8740112Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-20T22:08:30.8746446Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:30.8747772Z ++ awk -F '[:,]' '{print $2}' 2024-08-20T22:08:30.8772213Z + DOCKER_TAG=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:30.8773170Z + echo docker-tag=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:30.8774666Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:30.8808596Z ##[group]Run set +e 2024-08-20T22:08:30.8809131Z set +e 2024-08-20T22:08:30.8809557Z set -x 2024-08-20T22:08:30.8809961Z  2024-08-20T22:08:30.8810361Z login() { 2024-08-20T22:08:30.8811261Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:08:30.8812114Z } 2024-08-20T22:08:30.8812480Z  2024-08-20T22:08:30.8812906Z retry () { 2024-08-20T22:08:30.8813418Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:08:30.8814145Z } 2024-08-20T22:08:30.8814518Z  2024-08-20T22:08:30.8815013Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:08:30.8815487Z  2024-08-20T22:08:30.8816056Z # Check if image already exists, if it does then skip building it 2024-08-20T22:08:30.8816922Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-20T22:08:30.8817520Z  exit 0 2024-08-20T22:08:30.8817885Z fi 2024-08-20T22:08:30.8818330Z  2024-08-20T22:08:30.8818932Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-20T22:08:30.8819931Z # be empty. The default action would be to continue rebuild the image 2024-08-20T22:08:30.8820741Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-20T22:08:30.8821509Z  # if we're on the base branch then use the parent commit 2024-08-20T22:08:30.8822273Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-20T22:08:30.8822811Z else 2024-08-20T22:08:30.8823355Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-20T22:08:30.8824222Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-20T22:08:30.8824822Z fi 2024-08-20T22:08:30.8825229Z  2024-08-20T22:08:30.8825660Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-20T22:08:30.8826270Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:30.8826849Z  2024-08-20T22:08:30.8827639Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-20T22:08:30.8828453Z  exit 0 2024-08-20T22:08:30.8828878Z fi 2024-08-20T22:08:30.8829303Z  2024-08-20T22:08:30.8829864Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-20T22:08:30.8830887Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-20T22:08:30.8831832Z  exit 1 2024-08-20T22:08:30.8832262Z fi 2024-08-20T22:08:30.8832647Z  2024-08-20T22:08:30.8833236Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-20T22:08:30.8834297Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-20T22:08:30.8835317Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-20T22:08:30.8836410Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-20T22:08:30.8837583Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-20T22:08:30.8838486Z fi 2024-08-20T22:08:30.8838951Z  2024-08-20T22:08:30.8839401Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T22:08:30.8848461Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:30.8849061Z env: 2024-08-20T22:08:30.8849526Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:30.8849995Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T22:08:30.8850690Z BASE_REVISION: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:08:30.8852135Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:30.8853473Z DOCKER_TAG: f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:30.8854173Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:30.8854862Z ##[endgroup] 2024-08-20T22:08:30.8886286Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:30.8887194Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:30.8889488Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:08:30.8890802Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:31.4303234Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:08:31.4304425Z Configure a credential helper to remove this warning. See 2024-08-20T22:08:31.4305569Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:08:31.4306122Z 2024-08-20T22:08:31.4306382Z Login Succeeded 2024-08-20T22:08:31.4328906Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:31.6580952Z { 2024-08-20T22:08:31.6581971Z "schemaVersion": 2, 2024-08-20T22:08:31.6582816Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-20T22:08:31.6584453Z "config": { 2024-08-20T22:08:31.6585750Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-20T22:08:31.6586502Z "size": 48711, 2024-08-20T22:08:31.6587173Z "digest": "sha256:8fef7805cec6e829013a3d62f158b75bdc2f02cdfe3428c032ea017b488473bb" 2024-08-20T22:08:31.6588028Z }, 2024-08-20T22:08:31.6589799Z "layers": [ 2024-08-20T22:08:31.6590652Z { 2024-08-20T22:08:31.6591432Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6592382Z "size": 28580681, 2024-08-20T22:08:31.6593298Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-08-20T22:08:31.6594332Z }, 2024-08-20T22:08:31.6595156Z { 2024-08-20T22:08:31.6596564Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6597511Z "size": 7943451, 2024-08-20T22:08:31.6598433Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-08-20T22:08:31.6599533Z }, 2024-08-20T22:08:31.6600171Z { 2024-08-20T22:08:31.6600868Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6601587Z "size": 55728572, 2024-08-20T22:08:31.6602291Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-08-20T22:08:31.6602954Z }, 2024-08-20T22:08:31.6603310Z { 2024-08-20T22:08:31.6603875Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6604476Z "size": 186, 2024-08-20T22:08:31.6605129Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-08-20T22:08:31.6605890Z }, 2024-08-20T22:08:31.6606226Z { 2024-08-20T22:08:31.6607091Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6607777Z "size": 6886, 2024-08-20T22:08:31.6608427Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-08-20T22:08:31.6609261Z }, 2024-08-20T22:08:31.6609889Z { 2024-08-20T22:08:31.6610469Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6611361Z "size": 1329001756, 2024-08-20T22:08:31.6611998Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-08-20T22:08:31.6612909Z + exit 0 2024-08-20T22:08:31.6613434Z }, 2024-08-20T22:08:31.6613945Z { 2024-08-20T22:08:31.6614683Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6615573Z "size": 62501, 2024-08-20T22:08:31.6616779Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-08-20T22:08:31.6617479Z }, 2024-08-20T22:08:31.6617803Z { 2024-08-20T22:08:31.6618375Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6619018Z "size": 1684, 2024-08-20T22:08:31.6619649Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-08-20T22:08:31.6620350Z }, 2024-08-20T22:08:31.6620690Z { 2024-08-20T22:08:31.6621245Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6621891Z "size": 1523, 2024-08-20T22:08:31.6622531Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-08-20T22:08:31.6623461Z }, 2024-08-20T22:08:31.6623883Z { 2024-08-20T22:08:31.6624392Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6625042Z "size": 2528295403, 2024-08-20T22:08:31.6625746Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-08-20T22:08:31.6626465Z }, 2024-08-20T22:08:31.6626768Z { 2024-08-20T22:08:31.6627393Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6628062Z "size": 86016, 2024-08-20T22:08:31.6628702Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-08-20T22:08:31.6629449Z }, 2024-08-20T22:08:31.6629790Z { 2024-08-20T22:08:31.6630350Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6631027Z "size": 1822, 2024-08-20T22:08:31.6631666Z "digest": "sha256:ac23c5fcc1ef310e9cd0d138b180281188e20a3e28c51cd14751622e3c7d92ea" 2024-08-20T22:08:31.6632379Z }, 2024-08-20T22:08:31.6632791Z { 2024-08-20T22:08:31.6633278Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6633945Z "size": 246761563, 2024-08-20T22:08:31.6634657Z "digest": "sha256:ca426ffc9d2bd051ad1ff5c4f02fd5d9be161e70953cc37882c815080c708f4d" 2024-08-20T22:08:31.6635384Z }, 2024-08-20T22:08:31.6635710Z { 2024-08-20T22:08:31.6636282Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6636920Z "size": 544, 2024-08-20T22:08:31.6637638Z "digest": "sha256:4373aa743caf6660c671295b837048c8350a2165f2336dc3b7b6c96462579e50" 2024-08-20T22:08:31.6638292Z }, 2024-08-20T22:08:31.6638633Z { 2024-08-20T22:08:31.6639237Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6639840Z "size": 1258, 2024-08-20T22:08:31.6640485Z "digest": "sha256:75b2abb16c685854c017d0cf75c626526c956734c656d9e78fdd4520317e9b3c" 2024-08-20T22:08:31.6641236Z }, 2024-08-20T22:08:31.6641581Z { 2024-08-20T22:08:31.6642092Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6643175Z "size": 484, 2024-08-20T22:08:31.6643802Z "digest": "sha256:ef1e3ef1e85f0124270b8506721219150e230184181f8207d64463bf5c67e656" 2024-08-20T22:08:31.6644542Z }, 2024-08-20T22:08:31.6644895Z { 2024-08-20T22:08:31.6645421Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6646086Z "size": 91711766, 2024-08-20T22:08:31.6646758Z "digest": "sha256:0bf61114d37b2d09065aa6852b1e78a32580a1ff011dd5d23f03b0f5c4e710af" 2024-08-20T22:08:31.6647478Z }, 2024-08-20T22:08:31.6647820Z { 2024-08-20T22:08:31.6648397Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6649192Z "size": 3390, 2024-08-20T22:08:31.6649834Z "digest": "sha256:2df849f72ba4258df10c9c45ace21a66d93b73196e78e96f12c13098822f60e2" 2024-08-20T22:08:31.6650725Z }, 2024-08-20T22:08:31.6651087Z { 2024-08-20T22:08:31.6651580Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6652303Z "size": 1909, 2024-08-20T22:08:31.6652932Z "digest": "sha256:34c47d63c5de27418739aa048b4dd781c3006d65691e9b903c95304d0509d89c" 2024-08-20T22:08:31.6653626Z }, 2024-08-20T22:08:31.6653997Z { 2024-08-20T22:08:31.6654525Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6655170Z "size": 701, 2024-08-20T22:08:31.6655831Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:08:31.6656531Z }, 2024-08-20T22:08:31.6656892Z { 2024-08-20T22:08:31.6657489Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6658146Z "size": 2864254879, 2024-08-20T22:08:31.6658844Z "digest": "sha256:6efb96fc87ea1bca54876d85331e5585e28f325a1d60ee1aef7811eabb74f933" 2024-08-20T22:08:31.6659616Z }, 2024-08-20T22:08:31.6659954Z { 2024-08-20T22:08:31.6660461Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6661289Z "size": 380, 2024-08-20T22:08:31.6661933Z "digest": "sha256:b119c4fc6c58d9e72744201ff4850b34184a94eb3566031b7d36e87ecce950a6" 2024-08-20T22:08:31.6662599Z }, 2024-08-20T22:08:31.6662994Z { 2024-08-20T22:08:31.6663540Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6664235Z "size": 12900, 2024-08-20T22:08:31.6664872Z "digest": "sha256:33b639226eeefa41d7bb26a8ecc20d271ceef45571dabfd80aaee3634926381b" 2024-08-20T22:08:31.6665595Z }, 2024-08-20T22:08:31.6666003Z { 2024-08-20T22:08:31.6666550Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6667158Z "size": 803, 2024-08-20T22:08:31.6667895Z "digest": "sha256:d3547bd43146930aa830d54562c98aa678e355cf5f84527e9717eb72fffe8ca3" 2024-08-20T22:08:31.6668615Z }, 2024-08-20T22:08:31.6668960Z { 2024-08-20T22:08:31.6669488Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6670179Z "size": 106, 2024-08-20T22:08:31.6670804Z "digest": "sha256:5645520256fa6d7f2270608eaed53849a131d03fbf4c5d4472c9cfde1a439e47" 2024-08-20T22:08:31.6671508Z }, 2024-08-20T22:08:31.6671890Z { 2024-08-20T22:08:31.6672417Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6673131Z "size": 503, 2024-08-20T22:08:31.6673715Z "digest": "sha256:829ed7d91e3818d0e11834c71551117c7aeec0a51d798e93d641faa0c02582cb" 2024-08-20T22:08:31.6674411Z }, 2024-08-20T22:08:31.6674817Z { 2024-08-20T22:08:31.6675332Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6675935Z "size": 121477267, 2024-08-20T22:08:31.6676647Z "digest": "sha256:385d0421d67c533d628985c453a186028a61f6ad4209ae519ec3d4ecea661d3e" 2024-08-20T22:08:31.6677339Z }, 2024-08-20T22:08:31.6677688Z { 2024-08-20T22:08:31.6678301Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6678937Z "size": 109, 2024-08-20T22:08:31.6679596Z "digest": "sha256:7cbfce999d76851e07dd71056a6d852629d789ccfd46be44cc405b451fb8bb58" 2024-08-20T22:08:31.6680304Z }, 2024-08-20T22:08:31.6680643Z { 2024-08-20T22:08:31.6681186Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6681870Z "size": 490, 2024-08-20T22:08:31.6682450Z "digest": "sha256:6658e558ebe496f0d628947534b0ea1e44808f84f2dfd8a9a704e8aa5329d791" 2024-08-20T22:08:31.6683164Z }, 2024-08-20T22:08:31.6683545Z { 2024-08-20T22:08:31.6684063Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6684688Z "size": 297, 2024-08-20T22:08:31.6685383Z "digest": "sha256:33444a5a7e01a3010fd1cc16b32a66892a08600d7f700747d23faf3c36624f3a" 2024-08-20T22:08:31.6686196Z }, 2024-08-20T22:08:31.6686497Z { 2024-08-20T22:08:31.6687075Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6687782Z "size": 103, 2024-08-20T22:08:31.6688472Z "digest": "sha256:7c47122f4e09b47e891b4ae9e499a35a4eb46faacae87d0e946615adda6e0706" 2024-08-20T22:08:31.6689140Z }, 2024-08-20T22:08:31.6689489Z { 2024-08-20T22:08:31.6690057Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6690826Z "size": 1473, 2024-08-20T22:08:31.6691432Z "digest": "sha256:1811ff4d3556ac255604f346d879528aff2aadd7652c2d39baf4fe6eaa60aec3" 2024-08-20T22:08:31.6692215Z }, 2024-08-20T22:08:31.6692570Z { 2024-08-20T22:08:31.6693056Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6693747Z "size": 594729516, 2024-08-20T22:08:31.6694419Z "digest": "sha256:7754ee068b33f760d9c320c26a1a3935247ace0c8c463a976105c1586a43fb1c" 2024-08-20T22:08:31.6695117Z }, 2024-08-20T22:08:31.6695468Z { 2024-08-20T22:08:31.6696021Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6696662Z "size": 164, 2024-08-20T22:08:31.6697354Z "digest": "sha256:0479fbac3361713c075bf788e778117917dcacdd1ff71ed79a789dd2016a1714" 2024-08-20T22:08:31.6698121Z }, 2024-08-20T22:08:31.6698587Z { 2024-08-20T22:08:31.6699159Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6699802Z "size": 564, 2024-08-20T22:08:31.6700405Z "digest": "sha256:f1d6874dee9c34d154278d2b6935d04447f07fa89d25d4a09398234040d8a7e8" 2024-08-20T22:08:31.6701140Z }, 2024-08-20T22:08:31.6701476Z { 2024-08-20T22:08:31.6701983Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6702682Z "size": 35874203, 2024-08-20T22:08:31.6703337Z "digest": "sha256:afc429e87b593a64f3e0a1be51e09e966ea1aa71e6e8a0bec4edc9f2f7bc1c05" 2024-08-20T22:08:31.6704068Z }, 2024-08-20T22:08:31.6704417Z { 2024-08-20T22:08:31.6705008Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6705639Z "size": 104, 2024-08-20T22:08:31.6706331Z "digest": "sha256:019620471c2592cd22437fe8ed1a94c784114cf7445c5c2ab36dd0bae727fc66" 2024-08-20T22:08:31.6707027Z }, 2024-08-20T22:08:31.6707394Z { 2024-08-20T22:08:31.6707989Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6708618Z "size": 425, 2024-08-20T22:08:31.6709271Z "digest": "sha256:27aeb1b2cd5e28d8f0e54fc1090baf88abc29ebf5fce088b493fcae99c9550d1" 2024-08-20T22:08:31.6710041Z }, 2024-08-20T22:08:31.6710400Z { 2024-08-20T22:08:31.6710884Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6711598Z "size": 20262183, 2024-08-20T22:08:31.6712279Z "digest": "sha256:829c0e19f4b7797e9120a86172fabfd710f9b3971ffd714a616931a3fadc7192" 2024-08-20T22:08:31.6713013Z }, 2024-08-20T22:08:31.6713340Z { 2024-08-20T22:08:31.6713864Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6714551Z "size": 438, 2024-08-20T22:08:31.6715207Z "digest": "sha256:560cd6122a3868e7e82e75eb103d182edf6512114d61ccdfae099af00cb16b4c" 2024-08-20T22:08:31.6715878Z }, 2024-08-20T22:08:31.6716268Z { 2024-08-20T22:08:31.6716804Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6717405Z "size": 701, 2024-08-20T22:08:31.6718101Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:08:31.6718836Z }, 2024-08-20T22:08:31.6719169Z { 2024-08-20T22:08:31.6719724Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6720358Z "size": 141, 2024-08-20T22:08:31.6720987Z "digest": "sha256:5a4eb19d72198a01f3babd4711c40fcf7c6e5cb9d5cfedc507b0061408d47097" 2024-08-20T22:08:31.6721756Z }, 2024-08-20T22:08:31.6722050Z { 2024-08-20T22:08:31.6722577Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6723386Z "size": 135, 2024-08-20T22:08:31.6723973Z "digest": "sha256:da6a838a66c72b5de9a5590c233779880ea901c1264940a1c618825d2501ffd1" 2024-08-20T22:08:31.6724689Z }, 2024-08-20T22:08:31.6725085Z { 2024-08-20T22:08:31.6725646Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6726280Z "size": 4838068375, 2024-08-20T22:08:31.6726986Z "digest": "sha256:cbb79a7053537bed2d80b77198ea552b985563b11d3f8616593c6ab575a3a090" 2024-08-20T22:08:31.6727755Z }, 2024-08-20T22:08:31.6728094Z { 2024-08-20T22:08:31.6728636Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6729305Z "size": 194, 2024-08-20T22:08:31.6729935Z "digest": "sha256:4d4fa6802ab3e6b36d8d4adb650046e9370c4a16a19876970559ef8a312ca855" 2024-08-20T22:08:31.6730772Z }, 2024-08-20T22:08:31.6731115Z { 2024-08-20T22:08:31.6731633Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6732379Z "size": 564, 2024-08-20T22:08:31.6732998Z "digest": "sha256:0f085d4fa279366fcd52d303b7ad8abddc5962a8872174b8109f3f5d5ca5f2cf" 2024-08-20T22:08:31.6733689Z }, 2024-08-20T22:08:31.6734097Z { 2024-08-20T22:08:31.6734615Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6735312Z "size": 43162921, 2024-08-20T22:08:31.6736035Z "digest": "sha256:4a0412373305517489d891a4749e6d2b80b4aeb7df3b8adca0ed0abb2f3e5300" 2024-08-20T22:08:31.6736728Z }, 2024-08-20T22:08:31.6737128Z { 2024-08-20T22:08:31.6737613Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6738262Z "size": 106, 2024-08-20T22:08:31.6738979Z "digest": "sha256:302c8c1779444e54919f83e91c6d5e1fdea653fc899be802b3a2a34d34d99d5d" 2024-08-20T22:08:31.6739668Z }, 2024-08-20T22:08:31.6739969Z { 2024-08-20T22:08:31.6740562Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6741198Z "size": 1380, 2024-08-20T22:08:31.6741777Z "digest": "sha256:02970347dff622ee45629bc28788716d00528464f71f88ec7b74ae5f44a486e2" 2024-08-20T22:08:31.6742750Z }, 2024-08-20T22:08:31.6743171Z { 2024-08-20T22:08:31.6743727Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6744486Z "size": 701, 2024-08-20T22:08:31.6745123Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:08:31.6745874Z }, 2024-08-20T22:08:31.6746287Z { 2024-08-20T22:08:31.6746781Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6747487Z "size": 139, 2024-08-20T22:08:31.6748154Z "digest": "sha256:916c5e8580164926a3aec87e88a5d1c2d02756e49fdeed4198e1e6304efa2fd3" 2024-08-20T22:08:31.6748879Z }, 2024-08-20T22:08:31.6749190Z { 2024-08-20T22:08:31.6749769Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6750426Z "size": 118, 2024-08-20T22:08:31.6751026Z "digest": "sha256:c42b3602a1fb4df5111224650f15adffb025979ae9c614807379ea84e189533b" 2024-08-20T22:08:31.6751767Z }, 2024-08-20T22:08:31.6752127Z { 2024-08-20T22:08:31.6752738Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6753345Z "size": 4324672728, 2024-08-20T22:08:31.6754037Z "digest": "sha256:11fad972ef6cb9d2dc236504b0ee5b6e5ea996e4576ece0ed3262fbb9ee2de60" 2024-08-20T22:08:31.6754793Z }, 2024-08-20T22:08:31.6755151Z { 2024-08-20T22:08:31.6755637Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6756337Z "size": 174, 2024-08-20T22:08:31.6756972Z "digest": "sha256:f957c16244819132b478272aaf24231699c4fcbd495c229a574f94daab8d6b86" 2024-08-20T22:08:31.6757616Z }, 2024-08-20T22:08:31.6758019Z { 2024-08-20T22:08:31.6758559Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6759252Z "size": 907, 2024-08-20T22:08:31.6759902Z "digest": "sha256:768c31dca016b80bbdc973c8ee818034bdbd17b7ea9bdadbf0ec45ea51005720" 2024-08-20T22:08:31.6760795Z }, 2024-08-20T22:08:31.6761142Z { 2024-08-20T22:08:31.6761750Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6762358Z "size": 701, 2024-08-20T22:08:31.6762986Z "digest": "sha256:1fef4bd19792cbf9d3aea12795089ad41164bb6d607c339b7484e1899825b1d2" 2024-08-20T22:08:31.6763774Z }, 2024-08-20T22:08:31.6764077Z { 2024-08-20T22:08:31.6764596Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6765302Z "size": 134, 2024-08-20T22:08:31.6765938Z "digest": "sha256:8c32d0ecebbe2054c96a3a5c1f3d6c8b87c2e826686b369e5242cd55d0f01c2d" 2024-08-20T22:08:31.6766671Z }, 2024-08-20T22:08:31.6767060Z { 2024-08-20T22:08:31.6767591Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6768263Z "size": 32, 2024-08-20T22:08:31.6768910Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:08:31.6769628Z }, 2024-08-20T22:08:31.6770006Z { 2024-08-20T22:08:31.6770578Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6771284Z "size": 158, 2024-08-20T22:08:31.6771941Z "digest": "sha256:78ee77eddef23f20afc78e2924f4467041cadb754634ee3df431c68211cf676d" 2024-08-20T22:08:31.6772868Z }, 2024-08-20T22:08:31.6773238Z { 2024-08-20T22:08:31.6773773Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6774471Z "size": 1841, 2024-08-20T22:08:31.6775127Z "digest": "sha256:37513e9b43d7123e7ddee09100f62a72b57af7b3833eaecd515e247e2f63e13f" 2024-08-20T22:08:31.6775787Z }, 2024-08-20T22:08:31.6776198Z { 2024-08-20T22:08:31.6776726Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6777420Z "size": 7529776, 2024-08-20T22:08:31.6778052Z "digest": "sha256:a7a5392e2bf6b73d9be292cc22a99ad4637636c15733beb7afbe6403b5ba21a8" 2024-08-20T22:08:31.6778757Z }, 2024-08-20T22:08:31.6779139Z { 2024-08-20T22:08:31.6779660Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6780351Z "size": 163, 2024-08-20T22:08:31.6781024Z "digest": "sha256:b10de02255c021b2366b85bf12ddf593aac89f88964201c455b71a145bc5ca2e" 2024-08-20T22:08:31.6781748Z }, 2024-08-20T22:08:31.6782049Z { 2024-08-20T22:08:31.6782638Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6783279Z "size": 7943, 2024-08-20T22:08:31.6783914Z "digest": "sha256:e56eeff2db93ffe5002a2bb764795b7333321c6802cc28bef0801a619fe6a07e" 2024-08-20T22:08:31.6784643Z }, 2024-08-20T22:08:31.6784985Z { 2024-08-20T22:08:31.6785510Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6786213Z "size": 8068, 2024-08-20T22:08:31.6786858Z "digest": "sha256:70ac3e38d0fd1f0f46ad710c3fc485fd01406606054e8cf84a63af1570bc9f3a" 2024-08-20T22:08:31.6787572Z }, 2024-08-20T22:08:31.6787980Z { 2024-08-20T22:08:31.6788470Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6789131Z "size": 302, 2024-08-20T22:08:31.6789804Z "digest": "sha256:eb09cbf3e4168749cc8605e3188ed35070f6037d13930e443c38d8941734488f" 2024-08-20T22:08:31.6790504Z }, 2024-08-20T22:08:31.6790841Z { 2024-08-20T22:08:31.6791422Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6792073Z "size": 7629638, 2024-08-20T22:08:31.6792730Z "digest": "sha256:ee1273153675e6543b58f5be50321abee5e91f7351701d5d501a6837eaf676ef" 2024-08-20T22:08:31.6793428Z }, 2024-08-20T22:08:31.6793808Z { 2024-08-20T22:08:31.6794351Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6795034Z "size": 108, 2024-08-20T22:08:31.6795687Z "digest": "sha256:e22979f9e0eedfa16f98eb4fa26f30a50c237f5ffd2f6e5b898f2b908058496d" 2024-08-20T22:08:31.6796401Z }, 2024-08-20T22:08:31.6796795Z { 2024-08-20T22:08:31.6797302Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6798052Z "size": 54145661, 2024-08-20T22:08:31.6798794Z "digest": "sha256:6c1f0aabce7c84f9b0b9ab3b96325778dba1c1465b7463c5b4c6be1cf52b74db" 2024-08-20T22:08:31.6799510Z }, 2024-08-20T22:08:31.6799820Z { 2024-08-20T22:08:31.6800468Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6801122Z "size": 473, 2024-08-20T22:08:31.6801794Z "digest": "sha256:7ebe49c3a23e71b289ce012b48dad35791a2f2aa7fcee2326a64d763b0d14df8" 2024-08-20T22:08:31.6802484Z }, 2024-08-20T22:08:31.6802816Z { 2024-08-20T22:08:31.6803380Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6804015Z "size": 1374858766, 2024-08-20T22:08:31.6804658Z "digest": "sha256:5d115b29b5d53e5418762bc41ae975dd245bcffc3d058b8902c58146d505cb37" 2024-08-20T22:08:31.6805418Z }, 2024-08-20T22:08:31.6805754Z { 2024-08-20T22:08:31.6806241Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6806987Z "size": 106, 2024-08-20T22:08:31.6807638Z "digest": "sha256:7f2d7a04fbe65bbe1e895138e0cfeeef53ad926c890bab8e7e132ad1e655e48f" 2024-08-20T22:08:31.6808349Z }, 2024-08-20T22:08:31.6808711Z { 2024-08-20T22:08:31.6809229Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6809982Z "size": 559, 2024-08-20T22:08:31.6810606Z "digest": "sha256:d79fd7d526989e554455eda7d459f2c88444030f61710101e52ec8c7a49b7b79" 2024-08-20T22:08:31.6811421Z }, 2024-08-20T22:08:31.6811781Z { 2024-08-20T22:08:31.6812358Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6812962Z "size": 46248488, 2024-08-20T22:08:31.6813681Z "digest": "sha256:9752d6fbbcab4e7403f26777c5b65911320de6389c81b833a47ba45c4f3354b4" 2024-08-20T22:08:31.6814440Z }, 2024-08-20T22:08:31.6814801Z { 2024-08-20T22:08:31.6815283Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6815986Z "size": 111, 2024-08-20T22:08:31.6816631Z "digest": "sha256:83808b2eff4d71ed46db7c29f66fe918f328405c25b314225dfad01b4868d1f4" 2024-08-20T22:08:31.6817330Z }, 2024-08-20T22:08:31.6817707Z { 2024-08-20T22:08:31.6818256Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6818892Z "size": 32, 2024-08-20T22:08:31.6819525Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:08:31.6820244Z }, 2024-08-20T22:08:31.6820642Z { 2024-08-20T22:08:31.6821226Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6821823Z "size": 32, 2024-08-20T22:08:31.6822442Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:08:31.6823201Z }, 2024-08-20T22:08:31.6823531Z { 2024-08-20T22:08:31.6824008Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6824713Z "size": 32, 2024-08-20T22:08:31.6825343Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:08:31.6826082Z }, 2024-08-20T22:08:31.6826404Z { 2024-08-20T22:08:31.6826990Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T22:08:31.6827693Z "size": 32, 2024-08-20T22:08:31.6828280Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T22:08:31.6828974Z } 2024-08-20T22:08:31.6829378Z ] 2024-08-20T22:08:31.6829712Z } 2024-08-20T22:08:31.7125262Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:08:31.7125764Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T22:08:31.7126349Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-20T22:08:31.7135839Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:31.7136347Z env: 2024-08-20T22:08:31.7136626Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:31.7137786Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:31.7138954Z ##[endgroup] 2024-08-20T22:08:31.7172734Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks-f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:31.7225842Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-20T22:08:31.7226427Z with: 2024-08-20T22:08:31.7227548Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:31.7228852Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:31.7229368Z env: 2024-08-20T22:08:31.7229647Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:31.7229997Z ##[endgroup] 2024-08-20T22:08:31.7249554Z ##[group]Run set -x 2024-08-20T22:08:31.7249896Z set -x 2024-08-20T22:08:31.7250207Z set +e 2024-08-20T22:08:31.7250503Z  2024-08-20T22:08:31.7250901Z login() { 2024-08-20T22:08:31.7251560Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T22:08:31.7252271Z } 2024-08-20T22:08:31.7252551Z  2024-08-20T22:08:31.7253017Z retry () { 2024-08-20T22:08:31.7253411Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T22:08:31.7253856Z } 2024-08-20T22:08:31.7254134Z  2024-08-20T22:08:31.7254454Z retry login "${DOCKER_REGISTRY}" 2024-08-20T22:08:31.7254866Z  2024-08-20T22:08:31.7255140Z set -e 2024-08-20T22:08:31.7255616Z # ignore output since only exit code is used for conditional 2024-08-20T22:08:31.7256315Z # only pull docker image if it's not available locally 2024-08-20T22:08:31.7257070Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-20T22:08:31.7257758Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-20T22:08:31.7258197Z fi 2024-08-20T22:08:31.7266284Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:08:31.7266776Z env: 2024-08-20T22:08:31.7267057Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:08:31.7268202Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:31.7269511Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:31.7270028Z ##[endgroup] 2024-08-20T22:08:31.7297883Z + set +e 2024-08-20T22:08:31.7298526Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:31.7299172Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:31.7302446Z + aws ecr get-login-password --region us-east-1 2024-08-20T22:08:31.7303219Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T22:08:32.2692345Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T22:08:32.2693965Z Configure a credential helper to remove this warning. See 2024-08-20T22:08:32.2695627Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T22:08:32.2696699Z 2024-08-20T22:08:32.2696937Z Login Succeeded 2024-08-20T22:08:32.2713979Z + set -e 2024-08-20T22:08:32.2715328Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:32.2878949Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:32.2881075Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:08:32.5682708Z f6d216893d65c7b8ae43df4daaf247db808378e9: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks 2024-08-20T22:08:32.5684015Z 7a2c55901189: Pulling fs layer 2024-08-20T22:08:32.5684560Z 224fe954d725: Pulling fs layer 2024-08-20T22:08:32.5685402Z 75722010b82e: Pulling fs layer 2024-08-20T22:08:32.5685940Z d527cbbb87e3: Pulling fs layer 2024-08-20T22:08:32.5686419Z b57676e46aee: Pulling fs layer 2024-08-20T22:08:32.5686924Z a8c1e85b5e14: Pulling fs layer 2024-08-20T22:08:32.5687434Z a41a8d1c11c8: Pulling fs layer 2024-08-20T22:08:32.5687959Z 0c1227890755: Pulling fs layer 2024-08-20T22:08:32.5688448Z d8d1234baab3: Pulling fs layer 2024-08-20T22:08:32.5688953Z 7ed32bc8e469: Pulling fs layer 2024-08-20T22:08:32.5689453Z ec1e7978c1fe: Pulling fs layer 2024-08-20T22:08:32.5689956Z ac23c5fcc1ef: Pulling fs layer 2024-08-20T22:08:32.5690458Z ca426ffc9d2b: Pulling fs layer 2024-08-20T22:08:32.5691131Z 4373aa743caf: Pulling fs layer 2024-08-20T22:08:32.5691642Z 75b2abb16c68: Pulling fs layer 2024-08-20T22:08:32.5693653Z ef1e3ef1e85f: Pulling fs layer 2024-08-20T22:08:32.5694127Z d527cbbb87e3: Waiting 2024-08-20T22:08:32.5694558Z ec1e7978c1fe: Waiting 2024-08-20T22:08:32.5694981Z a8c1e85b5e14: Waiting 2024-08-20T22:08:32.5695398Z a41a8d1c11c8: Waiting 2024-08-20T22:08:32.5696068Z 0c1227890755: Waiting 2024-08-20T22:08:32.5696505Z ac23c5fcc1ef: Waiting 2024-08-20T22:08:32.5696966Z d8d1234baab3: Waiting 2024-08-20T22:08:32.5699785Z 7ed32bc8e469: Waiting 2024-08-20T22:08:32.5700462Z 0bf61114d37b: Pulling fs layer 2024-08-20T22:08:32.5700989Z 2df849f72ba4: Pulling fs layer 2024-08-20T22:08:32.5701568Z 34c47d63c5de: Pulling fs layer 2024-08-20T22:08:32.5702068Z 1fef4bd19792: Pulling fs layer 2024-08-20T22:08:32.5702598Z 6efb96fc87ea: Pulling fs layer 2024-08-20T22:08:32.5703083Z 75b2abb16c68: Waiting 2024-08-20T22:08:32.5703509Z 1fef4bd19792: Waiting 2024-08-20T22:08:32.5703947Z b119c4fc6c58: Pulling fs layer 2024-08-20T22:08:32.5704449Z 33b639226eee: Pulling fs layer 2024-08-20T22:08:32.5704944Z d3547bd43146: Pulling fs layer 2024-08-20T22:08:32.5705420Z 34c47d63c5de: Waiting 2024-08-20T22:08:32.5705894Z 5645520256fa: Pulling fs layer 2024-08-20T22:08:32.5706403Z 6efb96fc87ea: Waiting 2024-08-20T22:08:32.5706860Z 829ed7d91e38: Pulling fs layer 2024-08-20T22:08:32.5707354Z d3547bd43146: Waiting 2024-08-20T22:08:32.5707776Z 5645520256fa: Waiting 2024-08-20T22:08:32.5708201Z 2df849f72ba4: Waiting 2024-08-20T22:08:32.5708629Z 33b639226eee: Waiting 2024-08-20T22:08:32.5709064Z 385d0421d67c: Pulling fs layer 2024-08-20T22:08:32.5709590Z 7cbfce999d76: Pulling fs layer 2024-08-20T22:08:32.5710115Z 6658e558ebe4: Pulling fs layer 2024-08-20T22:08:32.5710611Z 33444a5a7e01: Pulling fs layer 2024-08-20T22:08:32.5711112Z 385d0421d67c: Waiting 2024-08-20T22:08:32.5711563Z 7cbfce999d76: Waiting 2024-08-20T22:08:32.5712016Z 6658e558ebe4: Waiting 2024-08-20T22:08:32.5712451Z 7c47122f4e09: Pulling fs layer 2024-08-20T22:08:32.5712927Z 33444a5a7e01: Waiting 2024-08-20T22:08:32.5713359Z 1811ff4d3556: Pulling fs layer 2024-08-20T22:08:32.5713827Z 7c47122f4e09: Waiting 2024-08-20T22:08:32.5714266Z 7754ee068b33: Pulling fs layer 2024-08-20T22:08:32.5714759Z 0479fbac3361: Pulling fs layer 2024-08-20T22:08:32.5715235Z 7754ee068b33: Waiting 2024-08-20T22:08:32.5715722Z f1d6874dee9c: Pulling fs layer 2024-08-20T22:08:32.5716281Z afc429e87b59: Pulling fs layer 2024-08-20T22:08:32.5716811Z 019620471c25: Pulling fs layer 2024-08-20T22:08:32.5717341Z 27aeb1b2cd5e: Pulling fs layer 2024-08-20T22:08:32.5717852Z 829c0e19f4b7: Pulling fs layer 2024-08-20T22:08:32.5718379Z 019620471c25: Waiting 2024-08-20T22:08:32.5718831Z f1d6874dee9c: Waiting 2024-08-20T22:08:32.5719352Z 560cd6122a38: Pulling fs layer 2024-08-20T22:08:32.5719864Z 0479fbac3361: Waiting 2024-08-20T22:08:32.5720325Z afc429e87b59: Waiting 2024-08-20T22:08:32.5720765Z 829c0e19f4b7: Waiting 2024-08-20T22:08:32.5721121Z 27aeb1b2cd5e: Waiting 2024-08-20T22:08:32.5721430Z 560cd6122a38: Waiting 2024-08-20T22:08:32.5721759Z 5a4eb19d7219: Pulling fs layer 2024-08-20T22:08:32.5722134Z da6a838a66c7: Pulling fs layer 2024-08-20T22:08:32.5722503Z cbb79a705353: Pulling fs layer 2024-08-20T22:08:32.5722879Z 4d4fa6802ab3: Pulling fs layer 2024-08-20T22:08:32.5723253Z 0f085d4fa279: Pulling fs layer 2024-08-20T22:08:32.5723842Z 4a0412373305: Pulling fs layer 2024-08-20T22:08:32.5724200Z 5a4eb19d7219: Waiting 2024-08-20T22:08:32.5724529Z 302c8c177944: Pulling fs layer 2024-08-20T22:08:32.5724879Z 4a0412373305: Waiting 2024-08-20T22:08:32.5725197Z 02970347dff6: Pulling fs layer 2024-08-20T22:08:32.5725556Z cbb79a705353: Waiting 2024-08-20T22:08:32.5725884Z 916c5e858016: Pulling fs layer 2024-08-20T22:08:32.5726258Z c42b3602a1fb: Pulling fs layer 2024-08-20T22:08:32.5726603Z 302c8c177944: Waiting 2024-08-20T22:08:32.5726935Z 11fad972ef6c: Pulling fs layer 2024-08-20T22:08:32.5727290Z 02970347dff6: Waiting 2024-08-20T22:08:32.5727598Z c42b3602a1fb: Waiting 2024-08-20T22:08:32.5727905Z 0f085d4fa279: Waiting 2024-08-20T22:08:32.5728267Z 4d4fa6802ab3: Waiting 2024-08-20T22:08:32.5728583Z 11fad972ef6c: Waiting 2024-08-20T22:08:32.5728889Z da6a838a66c7: Waiting 2024-08-20T22:08:32.5729197Z 916c5e858016: Waiting 2024-08-20T22:08:32.5729519Z f957c1624481: Pulling fs layer 2024-08-20T22:08:32.5729890Z 768c31dca016: Pulling fs layer 2024-08-20T22:08:32.5730242Z f957c1624481: Waiting 2024-08-20T22:08:32.5730812Z 8c32d0ecebbe: Pulling fs layer 2024-08-20T22:08:32.5731204Z 4f4fb700ef54: Pulling fs layer 2024-08-20T22:08:32.5731562Z 8c32d0ecebbe: Waiting 2024-08-20T22:08:32.5731888Z 78ee77eddef2: Pulling fs layer 2024-08-20T22:08:32.5732264Z 37513e9b43d7: Pulling fs layer 2024-08-20T22:08:32.5732632Z a7a5392e2bf6: Pulling fs layer 2024-08-20T22:08:32.5732977Z 4f4fb700ef54: Waiting 2024-08-20T22:08:32.5733308Z b10de02255c0: Pulling fs layer 2024-08-20T22:08:32.5733680Z e56eeff2db93: Pulling fs layer 2024-08-20T22:08:32.5734048Z 70ac3e38d0fd: Pulling fs layer 2024-08-20T22:08:32.5734415Z eb09cbf3e416: Pulling fs layer 2024-08-20T22:08:32.5734787Z ee1273153675: Pulling fs layer 2024-08-20T22:08:32.5735155Z e22979f9e0ee: Pulling fs layer 2024-08-20T22:08:32.5735527Z 6c1f0aabce7c: Pulling fs layer 2024-08-20T22:08:32.5735877Z 78ee77eddef2: Waiting 2024-08-20T22:08:32.5736190Z e56eeff2db93: Waiting 2024-08-20T22:08:32.5736500Z b10de02255c0: Waiting 2024-08-20T22:08:32.5736822Z 70ac3e38d0fd: Waiting 2024-08-20T22:08:32.5737144Z eb09cbf3e416: Waiting 2024-08-20T22:08:32.5737452Z ee1273153675: Waiting 2024-08-20T22:08:32.5737757Z e22979f9e0ee: Waiting 2024-08-20T22:08:32.5738083Z 7ebe49c3a23e: Pulling fs layer 2024-08-20T22:08:32.5738454Z 5d115b29b5d5: Pulling fs layer 2024-08-20T22:08:32.5738829Z 7f2d7a04fbe6: Pulling fs layer 2024-08-20T22:08:32.5739201Z d79fd7d52698: Pulling fs layer 2024-08-20T22:08:32.5739554Z 6c1f0aabce7c: Waiting 2024-08-20T22:08:32.5739869Z 7ebe49c3a23e: Waiting 2024-08-20T22:08:32.5740181Z d79fd7d52698: Waiting 2024-08-20T22:08:32.5740506Z 9752d6fbbcab: Pulling fs layer 2024-08-20T22:08:32.5740878Z 83808b2eff4d: Pulling fs layer 2024-08-20T22:08:32.5741234Z 9752d6fbbcab: Waiting 2024-08-20T22:08:32.5741556Z 7f2d7a04fbe6: Waiting 2024-08-20T22:08:32.5741860Z 83808b2eff4d: Waiting 2024-08-20T22:08:32.7047457Z 224fe954d725: Verifying Checksum 2024-08-20T22:08:32.7047901Z 224fe954d725: Download complete 2024-08-20T22:08:32.7868139Z d527cbbb87e3: Verifying Checksum 2024-08-20T22:08:32.7868798Z d527cbbb87e3: Download complete 2024-08-20T22:08:32.8579576Z b57676e46aee: Verifying Checksum 2024-08-20T22:08:32.8580026Z b57676e46aee: Download complete 2024-08-20T22:08:32.9031168Z 7a2c55901189: Verifying Checksum 2024-08-20T22:08:32.9031677Z 7a2c55901189: Download complete 2024-08-20T22:08:32.9833769Z a41a8d1c11c8: Download complete 2024-08-20T22:08:33.0771120Z 0c1227890755: Verifying Checksum 2024-08-20T22:08:33.0771698Z 0c1227890755: Download complete 2024-08-20T22:08:33.1463629Z d8d1234baab3: Download complete 2024-08-20T22:08:33.1852021Z 75722010b82e: Verifying Checksum 2024-08-20T22:08:33.1852495Z 75722010b82e: Download complete 2024-08-20T22:08:33.2604573Z ec1e7978c1fe: Verifying Checksum 2024-08-20T22:08:33.2605084Z ec1e7978c1fe: Download complete 2024-08-20T22:08:33.3383639Z ac23c5fcc1ef: Verifying Checksum 2024-08-20T22:08:33.3384209Z ac23c5fcc1ef: Download complete 2024-08-20T22:08:34.0792583Z 7a2c55901189: Pull complete 2024-08-20T22:08:34.3929486Z 224fe954d725: Pull complete 2024-08-20T22:08:35.3162922Z 75722010b82e: Pull complete 2024-08-20T22:08:35.3347428Z d527cbbb87e3: Pull complete 2024-08-20T22:08:35.3526138Z b57676e46aee: Pull complete 2024-08-20T22:08:35.8684811Z ca426ffc9d2b: Verifying Checksum 2024-08-20T22:08:35.8685415Z ca426ffc9d2b: Download complete 2024-08-20T22:08:35.9385583Z 4373aa743caf: Verifying Checksum 2024-08-20T22:08:35.9386175Z 4373aa743caf: Download complete 2024-08-20T22:08:36.0237074Z 75b2abb16c68: Verifying Checksum 2024-08-20T22:08:36.0237531Z 75b2abb16c68: Download complete 2024-08-20T22:08:36.0962938Z ef1e3ef1e85f: Verifying Checksum 2024-08-20T22:08:36.0963375Z ef1e3ef1e85f: Download complete 2024-08-20T22:08:37.0653667Z 0bf61114d37b: Verifying Checksum 2024-08-20T22:08:37.0654134Z 0bf61114d37b: Download complete 2024-08-20T22:08:37.1462344Z 2df849f72ba4: Download complete 2024-08-20T22:08:37.2448384Z 34c47d63c5de: Verifying Checksum 2024-08-20T22:08:37.2448889Z 34c47d63c5de: Download complete 2024-08-20T22:08:37.3213458Z 1fef4bd19792: Verifying Checksum 2024-08-20T22:08:37.3214219Z 1fef4bd19792: Download complete 2024-08-20T22:08:46.2040684Z a8c1e85b5e14: Verifying Checksum 2024-08-20T22:08:46.2041163Z a8c1e85b5e14: Download complete 2024-08-20T22:08:46.2952025Z b119c4fc6c58: Verifying Checksum 2024-08-20T22:08:46.2952575Z b119c4fc6c58: Download complete 2024-08-20T22:08:46.3796666Z 33b639226eee: Verifying Checksum 2024-08-20T22:08:46.3797125Z 33b639226eee: Download complete 2024-08-20T22:08:46.4537679Z d3547bd43146: Verifying Checksum 2024-08-20T22:08:46.4538464Z d3547bd43146: Download complete 2024-08-20T22:08:46.5307140Z 5645520256fa: Verifying Checksum 2024-08-20T22:08:46.5308158Z 5645520256fa: Download complete 2024-08-20T22:08:46.6035732Z 829ed7d91e38: Verifying Checksum 2024-08-20T22:08:46.6036316Z 829ed7d91e38: Download complete 2024-08-20T22:08:47.8923385Z 385d0421d67c: Verifying Checksum 2024-08-20T22:08:47.8924058Z 385d0421d67c: Download complete 2024-08-20T22:08:47.9664306Z 7cbfce999d76: Verifying Checksum 2024-08-20T22:08:47.9665025Z 7cbfce999d76: Download complete 2024-08-20T22:08:48.0302498Z 6658e558ebe4: Verifying Checksum 2024-08-20T22:08:48.0303000Z 6658e558ebe4: Download complete 2024-08-20T22:08:48.1140309Z 33444a5a7e01: Verifying Checksum 2024-08-20T22:08:48.1140866Z 33444a5a7e01: Download complete 2024-08-20T22:08:48.1983522Z 7c47122f4e09: Download complete 2024-08-20T22:08:48.2673134Z 1811ff4d3556: Verifying Checksum 2024-08-20T22:08:48.2673797Z 1811ff4d3556: Download complete 2024-08-20T22:08:54.2852561Z 7754ee068b33: Verifying Checksum 2024-08-20T22:08:54.2853236Z 7754ee068b33: Download complete 2024-08-20T22:08:54.3470735Z 0479fbac3361: Verifying Checksum 2024-08-20T22:08:54.3471290Z 0479fbac3361: Download complete 2024-08-20T22:08:54.4249389Z f1d6874dee9c: Verifying Checksum 2024-08-20T22:08:54.4249944Z f1d6874dee9c: Download complete 2024-08-20T22:08:54.8418463Z afc429e87b59: Verifying Checksum 2024-08-20T22:08:54.8419012Z afc429e87b59: Download complete 2024-08-20T22:08:54.9209544Z 019620471c25: Verifying Checksum 2024-08-20T22:08:54.9210226Z 019620471c25: Download complete 2024-08-20T22:08:54.9934178Z 27aeb1b2cd5e: Verifying Checksum 2024-08-20T22:08:54.9934769Z 27aeb1b2cd5e: Download complete 2024-08-20T22:08:55.2449267Z 829c0e19f4b7: Verifying Checksum 2024-08-20T22:08:55.2449820Z 829c0e19f4b7: Download complete 2024-08-20T22:08:55.3320455Z 560cd6122a38: Verifying Checksum 2024-08-20T22:08:55.3320965Z 560cd6122a38: Download complete 2024-08-20T22:08:55.4014907Z 5a4eb19d7219: Download complete 2024-08-20T22:08:55.4907711Z da6a838a66c7: Verifying Checksum 2024-08-20T22:08:55.4908425Z da6a838a66c7: Download complete 2024-08-20T22:08:58.4876890Z 7ed32bc8e469: Verifying Checksum 2024-08-20T22:08:58.4877447Z 7ed32bc8e469: Download complete 2024-08-20T22:08:58.5552665Z 4d4fa6802ab3: Verifying Checksum 2024-08-20T22:08:58.5553119Z 4d4fa6802ab3: Download complete 2024-08-20T22:08:58.6257989Z 0f085d4fa279: Verifying Checksum 2024-08-20T22:08:58.6258410Z 0f085d4fa279: Download complete 2024-08-20T22:08:59.1497497Z 4a0412373305: Verifying Checksum 2024-08-20T22:08:59.1497997Z 4a0412373305: Download complete 2024-08-20T22:08:59.2201832Z 302c8c177944: Verifying Checksum 2024-08-20T22:08:59.2202278Z 302c8c177944: Download complete 2024-08-20T22:08:59.2895582Z 02970347dff6: Verifying Checksum 2024-08-20T22:08:59.2896012Z 02970347dff6: Download complete 2024-08-20T22:08:59.3577289Z 916c5e858016: Verifying Checksum 2024-08-20T22:08:59.3577707Z 916c5e858016: Download complete 2024-08-20T22:08:59.3857166Z a8c1e85b5e14: Pull complete 2024-08-20T22:08:59.4302342Z c42b3602a1fb: Download complete 2024-08-20T22:08:59.5244231Z a41a8d1c11c8: Pull complete 2024-08-20T22:08:59.6554332Z 0c1227890755: Pull complete 2024-08-20T22:08:59.7936856Z d8d1234baab3: Pull complete 2024-08-20T22:09:06.0564327Z 6efb96fc87ea: Verifying Checksum 2024-08-20T22:09:06.0564890Z 6efb96fc87ea: Download complete 2024-08-20T22:09:06.1298240Z f957c1624481: Download complete 2024-08-20T22:09:06.2072650Z 768c31dca016: Verifying Checksum 2024-08-20T22:09:06.2073137Z 768c31dca016: Download complete 2024-08-20T22:09:06.3013509Z 8c32d0ecebbe: Verifying Checksum 2024-08-20T22:09:06.3014075Z 8c32d0ecebbe: Download complete 2024-08-20T22:09:06.3109174Z 4f4fb700ef54: Verifying Checksum 2024-08-20T22:09:06.3109744Z 4f4fb700ef54: Download complete 2024-08-20T22:09:06.4073499Z 78ee77eddef2: Verifying Checksum 2024-08-20T22:09:06.4073964Z 78ee77eddef2: Download complete 2024-08-20T22:09:06.4819408Z 37513e9b43d7: Verifying Checksum 2024-08-20T22:09:06.4819958Z 37513e9b43d7: Download complete 2024-08-20T22:09:06.6105748Z a7a5392e2bf6: Verifying Checksum 2024-08-20T22:09:06.6106228Z a7a5392e2bf6: Download complete 2024-08-20T22:09:06.7031774Z b10de02255c0: Download complete 2024-08-20T22:09:06.7756303Z e56eeff2db93: Verifying Checksum 2024-08-20T22:09:06.7756803Z e56eeff2db93: Download complete 2024-08-20T22:09:06.8523582Z 70ac3e38d0fd: Verifying Checksum 2024-08-20T22:09:06.9368731Z 70ac3e38d0fd: Download complete 2024-08-20T22:09:06.9369198Z eb09cbf3e416: Download complete 2024-08-20T22:09:07.0690574Z ee1273153675: Verifying Checksum 2024-08-20T22:09:07.0691303Z ee1273153675: Download complete 2024-08-20T22:09:07.1436777Z e22979f9e0ee: Download complete 2024-08-20T22:09:07.7379628Z 6c1f0aabce7c: Verifying Checksum 2024-08-20T22:09:07.7380111Z 6c1f0aabce7c: Download complete 2024-08-20T22:09:07.8070681Z 7ebe49c3a23e: Verifying Checksum 2024-08-20T22:09:07.8071333Z 7ebe49c3a23e: Download complete 2024-08-20T22:09:34.6072179Z 5d115b29b5d5: Verifying Checksum 2024-08-20T22:09:34.6072636Z 5d115b29b5d5: Download complete 2024-08-20T22:09:34.6762317Z 7f2d7a04fbe6: Verifying Checksum 2024-08-20T22:09:34.6762877Z 7f2d7a04fbe6: Download complete 2024-08-20T22:09:34.7650359Z d79fd7d52698: Verifying Checksum 2024-08-20T22:09:34.7651050Z d79fd7d52698: Download complete 2024-08-20T22:09:36.2479641Z 9752d6fbbcab: Verifying Checksum 2024-08-20T22:09:36.2480126Z 9752d6fbbcab: Download complete 2024-08-20T22:09:36.3309078Z 83808b2eff4d: Download complete 2024-08-20T22:10:13.9555342Z 11fad972ef6c: Verifying Checksum 2024-08-20T22:10:13.9557615Z 11fad972ef6c: Download complete 2024-08-20T22:10:15.7349384Z cbb79a705353: Download complete 2024-08-20T22:10:30.2272617Z 7ed32bc8e469: Pull complete 2024-08-20T22:10:30.4476595Z ec1e7978c1fe: Pull complete 2024-08-20T22:10:30.6596571Z ac23c5fcc1ef: Pull complete 2024-08-20T22:10:39.2074021Z ca426ffc9d2b: Pull complete 2024-08-20T22:10:39.4345796Z 4373aa743caf: Pull complete 2024-08-20T22:10:39.6679922Z 75b2abb16c68: Pull complete 2024-08-20T22:10:39.8852165Z ef1e3ef1e85f: Pull complete 2024-08-20T22:10:42.5326416Z 0bf61114d37b: Pull complete 2024-08-20T22:10:42.7530246Z 2df849f72ba4: Pull complete 2024-08-20T22:10:42.7812162Z 34c47d63c5de: Pull complete 2024-08-20T22:10:42.8387193Z 1fef4bd19792: Pull complete 2024-08-20T22:11:41.3502245Z 6efb96fc87ea: Pull complete 2024-08-20T22:11:41.5912557Z b119c4fc6c58: Pull complete 2024-08-20T22:11:41.8326904Z 33b639226eee: Pull complete 2024-08-20T22:11:42.0537873Z d3547bd43146: Pull complete 2024-08-20T22:11:42.2731116Z 5645520256fa: Pull complete 2024-08-20T22:11:42.4999069Z 829ed7d91e38: Pull complete 2024-08-20T22:11:46.0206961Z 385d0421d67c: Pull complete 2024-08-20T22:11:46.2420950Z 7cbfce999d76: Pull complete 2024-08-20T22:11:46.4600063Z 6658e558ebe4: Pull complete 2024-08-20T22:11:46.6239461Z 33444a5a7e01: Pull complete 2024-08-20T22:11:46.7961536Z 7c47122f4e09: Pull complete 2024-08-20T22:11:46.9370857Z 1811ff4d3556: Pull complete 2024-08-20T22:11:57.6377951Z 7754ee068b33: Pull complete 2024-08-20T22:11:57.8437305Z 0479fbac3361: Pull complete 2024-08-20T22:11:58.0451158Z f1d6874dee9c: Pull complete 2024-08-20T22:11:59.0199553Z afc429e87b59: Pull complete 2024-08-20T22:11:59.2120232Z 019620471c25: Pull complete 2024-08-20T22:11:59.3944952Z 27aeb1b2cd5e: Pull complete 2024-08-20T22:11:59.8711306Z 829c0e19f4b7: Pull complete 2024-08-20T22:12:00.0500662Z 560cd6122a38: Pull complete 2024-08-20T22:12:00.3675098Z 5a4eb19d7219: Pull complete 2024-08-20T22:12:00.5329524Z da6a838a66c7: Pull complete 2024-08-20T22:12:39.2540034Z cbb79a705353: Pull complete 2024-08-20T22:12:39.4070869Z 4d4fa6802ab3: Pull complete 2024-08-20T22:12:39.6144177Z 0f085d4fa279: Pull complete 2024-08-20T22:12:41.9432867Z 4a0412373305: Pull complete 2024-08-20T22:12:41.9648843Z 302c8c177944: Pull complete 2024-08-20T22:12:41.9807241Z 02970347dff6: Pull complete 2024-08-20T22:12:42.0163624Z 916c5e858016: Pull complete 2024-08-20T22:12:42.0347066Z c42b3602a1fb: Pull complete 2024-08-20T22:14:04.3950134Z 11fad972ef6c: Pull complete 2024-08-20T22:14:04.5740570Z f957c1624481: Pull complete 2024-08-20T22:14:04.7708882Z 768c31dca016: Pull complete 2024-08-20T22:14:05.1450689Z 8c32d0ecebbe: Pull complete 2024-08-20T22:14:05.3329457Z 4f4fb700ef54: Pull complete 2024-08-20T22:14:05.4728122Z 78ee77eddef2: Pull complete 2024-08-20T22:14:05.6600815Z 37513e9b43d7: Pull complete 2024-08-20T22:14:06.1113519Z a7a5392e2bf6: Pull complete 2024-08-20T22:14:06.2210221Z b10de02255c0: Pull complete 2024-08-20T22:14:06.2947927Z e56eeff2db93: Pull complete 2024-08-20T22:14:06.4459536Z 70ac3e38d0fd: Pull complete 2024-08-20T22:14:06.6317420Z eb09cbf3e416: Pull complete 2024-08-20T22:14:08.0570879Z ee1273153675: Pull complete 2024-08-20T22:14:08.2304400Z e22979f9e0ee: Pull complete 2024-08-20T22:14:10.9452854Z 6c1f0aabce7c: Pull complete 2024-08-20T22:14:11.1172090Z 7ebe49c3a23e: Pull complete 2024-08-20T22:14:40.3750168Z 5d115b29b5d5: Pull complete 2024-08-20T22:14:40.5879864Z 7f2d7a04fbe6: Pull complete 2024-08-20T22:14:40.8041751Z d79fd7d52698: Pull complete 2024-08-20T22:14:41.6782616Z 9752d6fbbcab: Pull complete 2024-08-20T22:14:41.9017264Z 83808b2eff4d: Pull complete 2024-08-20T22:14:42.7146483Z Digest: sha256:3a95d3a177465127ef6cd3b50787d6a050367070fd9063dbb688fad07a96b402 2024-08-20T22:14:42.7601019Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:14:42.7824011Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:14:42.7949670Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:14:42.7950599Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T22:14:42.7962554Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:14:42.7963060Z env: 2024-08-20T22:14:42.7963345Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:14:42.7963696Z ##[endgroup] 2024-08-20T22:14:42.8166175Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-08-20T22:14:42.8166726Z with: 2024-08-20T22:14:42.8167008Z driver-version: 550.54.15 2024-08-20T22:14:42.8167341Z env: 2024-08-20T22:14:42.8167613Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:14:42.8167957Z ##[endgroup] 2024-08-20T22:14:42.8258128Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:14:42.8258736Z with: 2024-08-20T22:14:42.8259031Z timeout_minutes: 10 2024-08-20T22:14:42.8259360Z max_attempts: 3 2024-08-20T22:14:42.8291447Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2024-08-20T22:14:42.8324031Z retry_wait_seconds: 10 2024-08-20T22:14:42.8324397Z polling_interval_seconds: 1 2024-08-20T22:14:42.8324772Z warning_on_retry: true 2024-08-20T22:14:42.8325126Z continue_on_error: false 2024-08-20T22:14:42.8325465Z env: 2024-08-20T22:14:42.8325739Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:14:42.8326094Z DRIVER_VERSION: 550.54.15 2024-08-20T22:14:42.8326447Z ##[endgroup] 2024-08-20T22:14:43.2538823Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-08-20T22:14:43.2539596Z + pre_install_nvidia_driver_amzn2 2024-08-20T22:14:43.2541461Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-08-20T22:14:43.8794307Z No match for argument: nvidia-driver-latest-dkms 2024-08-20T22:14:43.8794869Z No packages marked for removal. 2024-08-20T22:14:43.9161931Z Dependencies resolved. 2024-08-20T22:14:43.9171852Z Nothing to do. 2024-08-20T22:14:43.9172175Z Complete! 2024-08-20T22:14:43.9992008Z + install_nvidia_driver_common 2024-08-20T22:14:43.9994192Z + echo 'Before installing NVIDIA driver' 2024-08-20T22:14:43.9994654Z Before installing NVIDIA driver 2024-08-20T22:14:43.9995368Z + lspci 2024-08-20T22:14:44.0108953Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:14:44.0110882Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:14:44.0113383Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:14:44.0114994Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:14:44.0116619Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:14:44.0118177Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:14:44.0119210Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:14:44.0120015Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:14:44.0120596Z + lsmod 2024-08-20T22:14:44.0211895Z Module Size Used by 2024-08-20T22:14:44.0212328Z nvidia_modeset 1351680 0 2024-08-20T22:14:44.0214133Z video 65536 1 nvidia_modeset 2024-08-20T22:14:44.0214698Z wmi 36864 1 video 2024-08-20T22:14:44.0215100Z nvidia_uvm 4706304 0 2024-08-20T22:14:44.0215567Z nvidia 54071296 7 nvidia_uvm,nvidia_modeset 2024-08-20T22:14:44.0216050Z drm 602112 1 nvidia 2024-08-20T22:14:44.0216497Z drm_panel_orientation_quirks 28672 1 drm 2024-08-20T22:14:44.0217042Z backlight 24576 3 video,drm,nvidia_modeset 2024-08-20T22:14:44.0217544Z i2c_core 106496 2 nvidia,drm 2024-08-20T22:14:44.0217971Z xt_conntrack 16384 1 2024-08-20T22:14:44.0218359Z nft_chain_nat 16384 3 2024-08-20T22:14:44.0218750Z xt_MASQUERADE 20480 1 2024-08-20T22:14:44.0219272Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-20T22:14:44.0219875Z nf_conntrack_netlink 57344 0 2024-08-20T22:14:44.0220489Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:14:44.0221141Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:14:44.0221622Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:14:44.0222054Z xfrm_user 57344 1 2024-08-20T22:14:44.0222565Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:14:44.0223131Z xt_addrtype 16384 2 2024-08-20T22:14:44.0223541Z nft_compat 20480 4 2024-08-20T22:14:44.0224210Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:14:44.0224840Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:14:44.0225388Z br_netfilter 36864 0 2024-08-20T22:14:44.0225805Z bridge 307200 1 br_netfilter 2024-08-20T22:14:44.0226249Z stp 16384 1 bridge 2024-08-20T22:14:44.0226682Z llc 16384 2 bridge,stp 2024-08-20T22:14:44.0227113Z overlay 167936 0 2024-08-20T22:14:44.0227484Z tls 114688 0 2024-08-20T22:14:44.0227882Z nls_ascii 16384 1 2024-08-20T22:14:44.0228403Z nls_cp437 20480 1 2024-08-20T22:14:44.0228942Z vfat 24576 1 2024-08-20T22:14:44.0229494Z fat 86016 1 vfat 2024-08-20T22:14:44.0230036Z sunrpc 692224 1 2024-08-20T22:14:44.0230479Z ena 167936 0 2024-08-20T22:14:44.0230861Z ghash_clmulni_intel 16384 0 2024-08-20T22:14:44.0231255Z ptp 36864 1 ena 2024-08-20T22:14:44.0231660Z pps_core 24576 1 ptp 2024-08-20T22:14:44.0232065Z aesni_intel 393216 0 2024-08-20T22:14:44.0232430Z i8042 45056 0 2024-08-20T22:14:44.0232808Z serio 28672 3 i8042 2024-08-20T22:14:44.0233248Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:14:44.0233773Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:14:44.0234274Z button 24576 0 2024-08-20T22:14:44.0234655Z sch_fq_codel 20480 17 2024-08-20T22:14:44.0235043Z dm_mod 188416 0 2024-08-20T22:14:44.0235415Z fuse 163840 1 2024-08-20T22:14:44.0235788Z dax 45056 1 dm_mod 2024-08-20T22:14:44.0236197Z configfs 57344 1 2024-08-20T22:14:44.0236695Z loop 36864 0 2024-08-20T22:14:44.0237180Z dmi_sysfs 20480 0 2024-08-20T22:14:44.0237709Z crc32_pclmul 16384 0 2024-08-20T22:14:44.0238233Z crc32c_intel 24576 0 2024-08-20T22:14:44.0238699Z efivarfs 24576 1 2024-08-20T22:14:44.0239057Z + modinfo nvidia 2024-08-20T22:14:44.0239936Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:14:44.0240765Z alias: char-major-195-* 2024-08-20T22:14:44.0241155Z version: 550.54.15 2024-08-20T22:14:44.0241507Z supported: external 2024-08-20T22:14:44.0241844Z license: NVIDIA 2024-08-20T22:14:44.0242363Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:14:44.0243171Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:14:44.0243756Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:14:44.0244232Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:14:44.0244742Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:14:44.0245235Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:14:44.0245758Z depends: i2c-core,drm 2024-08-20T22:14:44.0246120Z retpoline: Y 2024-08-20T22:14:44.0246474Z name: nvidia 2024-08-20T22:14:44.0247228Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:14:44.0247933Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:14:44.0248575Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:14:44.0249168Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:14:44.0249627Z parm: NVreg_RmLogonRC:int 2024-08-20T22:14:44.0250067Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:14:44.0250633Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:14:44.0251079Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:14:44.0251623Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:14:44.0252147Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:14:44.0252712Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:14:44.0253362Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:14:44.0253797Z parm: NVreg_EnableMSI:int 2024-08-20T22:14:44.0254208Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:14:44.0254667Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:14:44.0255201Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:14:44.0255777Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:14:44.0256342Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:14:44.0256958Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:14:44.0257555Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:14:44.0258162Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:14:44.0258758Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:14:44.0259250Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:14:44.0259787Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:14:44.0260337Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:14:44.0260836Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:14:44.0261300Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:14:44.0261786Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:14:44.0262255Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:14:44.0262709Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:14:44.0263221Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:14:44.0263751Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:14:44.0264225Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:14:44.0264715Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:14:44.0265218Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:14:44.0265704Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:14:44.0266199Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:14:44.0266678Z parm: NVreg_RmMsg:charp 2024-08-20T22:14:44.0267103Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:14:44.0267567Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:14:44.0268041Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:14:44.0268499Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:14:44.0268981Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:14:44.0269455Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:14:44.0269912Z parm: rm_firmware_active:charp 2024-08-20T22:14:44.0270470Z + HAS_NVIDIA_DRIVER=0 2024-08-20T22:14:44.0270881Z ++ command -v nvidia-smi 2024-08-20T22:14:44.0271284Z + '[' -x /usr/bin/nvidia-smi ']' 2024-08-20T22:14:44.0271671Z + set +e 2024-08-20T22:14:44.0272222Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-08-20T22:14:44.0523288Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-08-20T22:14:44.0523872Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:14:44.0524323Z + '[' 0 -ne 0 ']' 2024-08-20T22:14:44.0524745Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-08-20T22:14:44.0525146Z + HAS_NVIDIA_DRIVER=1 2024-08-20T22:14:44.0525875Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-08-20T22:14:44.0526589Z + set -e 2024-08-20T22:14:44.0526959Z + '[' 1 -eq 0 ']' 2024-08-20T22:14:44.0527677Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-08-20T22:14:44.0528406Z + post_install_nvidia_driver_common 2024-08-20T22:14:44.0531792Z + sudo modprobe nvidia 2024-08-20T22:14:44.1836739Z + echo 'After installing NVIDIA driver' 2024-08-20T22:14:44.1837184Z + lspci 2024-08-20T22:14:44.1837490Z After installing NVIDIA driver 2024-08-20T22:14:44.1942143Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-20T22:14:44.1943158Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-20T22:14:44.1944147Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-20T22:14:44.1945463Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-20T22:14:44.1946296Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-20T22:14:44.1947077Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-20T22:14:44.1947804Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-20T22:14:44.1948607Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-20T22:14:44.1949198Z + lsmod 2024-08-20T22:14:44.1976430Z Module Size Used by 2024-08-20T22:14:44.1977003Z nvidia_modeset 1351680 0 2024-08-20T22:14:44.1977558Z video 65536 1 nvidia_modeset 2024-08-20T22:14:44.1978167Z wmi 36864 1 video 2024-08-20T22:14:44.1978706Z nvidia_uvm 4706304 0 2024-08-20T22:14:44.1979328Z nvidia 54071296 7 nvidia_uvm,nvidia_modeset 2024-08-20T22:14:44.1979901Z drm 602112 1 nvidia 2024-08-20T22:14:44.1980357Z drm_panel_orientation_quirks 28672 1 drm 2024-08-20T22:14:44.1980888Z backlight 24576 3 video,drm,nvidia_modeset 2024-08-20T22:14:44.1981392Z i2c_core 106496 2 nvidia,drm 2024-08-20T22:14:44.1981817Z xt_conntrack 16384 1 2024-08-20T22:14:44.1982201Z nft_chain_nat 16384 3 2024-08-20T22:14:44.1982576Z xt_MASQUERADE 20480 1 2024-08-20T22:14:44.1983031Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-20T22:14:44.1983526Z nf_conntrack_netlink 57344 0 2024-08-20T22:14:44.1984110Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-20T22:14:44.1984751Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-20T22:14:44.1985218Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-20T22:14:44.1985654Z xfrm_user 57344 1 2024-08-20T22:14:44.1986052Z xfrm_algo 16384 1 xfrm_user 2024-08-20T22:14:44.1986466Z xt_addrtype 16384 2 2024-08-20T22:14:44.1986848Z nft_compat 20480 4 2024-08-20T22:14:44.1987305Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-20T22:14:44.1987919Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-20T22:14:44.1988457Z br_netfilter 36864 0 2024-08-20T22:14:44.1988871Z bridge 307200 1 br_netfilter 2024-08-20T22:14:44.1989315Z stp 16384 1 bridge 2024-08-20T22:14:44.1989744Z llc 16384 2 bridge,stp 2024-08-20T22:14:44.1990365Z overlay 167936 0 2024-08-20T22:14:44.1990743Z tls 114688 0 2024-08-20T22:14:44.1991115Z nls_ascii 16384 1 2024-08-20T22:14:44.1991481Z nls_cp437 20480 1 2024-08-20T22:14:44.1991853Z vfat 24576 1 2024-08-20T22:14:44.1992224Z fat 86016 1 vfat 2024-08-20T22:14:44.1992617Z sunrpc 692224 1 2024-08-20T22:14:44.1992978Z ena 167936 0 2024-08-20T22:14:44.1993359Z ghash_clmulni_intel 16384 0 2024-08-20T22:14:44.1993748Z ptp 36864 1 ena 2024-08-20T22:14:44.1994149Z pps_core 24576 1 ptp 2024-08-20T22:14:44.1994662Z aesni_intel 393216 0 2024-08-20T22:14:44.1995177Z i8042 45056 0 2024-08-20T22:14:44.1995698Z serio 28672 3 i8042 2024-08-20T22:14:44.1996233Z crypto_simd 16384 1 aesni_intel 2024-08-20T22:14:44.1996866Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-20T22:14:44.1997554Z button 24576 0 2024-08-20T22:14:44.1998068Z sch_fq_codel 20480 17 2024-08-20T22:14:44.1998516Z dm_mod 188416 0 2024-08-20T22:14:44.1998885Z fuse 163840 1 2024-08-20T22:14:44.1999271Z dax 45056 1 dm_mod 2024-08-20T22:14:44.1999671Z configfs 57344 1 2024-08-20T22:14:44.2000037Z loop 36864 0 2024-08-20T22:14:44.2000412Z dmi_sysfs 20480 0 2024-08-20T22:14:44.2001087Z crc32_pclmul 16384 0 2024-08-20T22:14:44.2001469Z crc32c_intel 24576 0 2024-08-20T22:14:44.2001838Z efivarfs 24576 1 2024-08-20T22:14:44.2002199Z + modinfo nvidia 2024-08-20T22:14:44.2002915Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-20T22:14:44.2003624Z alias: char-major-195-* 2024-08-20T22:14:44.2004007Z version: 550.54.15 2024-08-20T22:14:44.2004357Z supported: external 2024-08-20T22:14:44.2004710Z license: NVIDIA 2024-08-20T22:14:44.2005099Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-20T22:14:44.2005584Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-20T22:14:44.2006056Z srcversion: 833721318DA517F0C2FEC97 2024-08-20T22:14:44.2006533Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-20T22:14:44.2007034Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-20T22:14:44.2007536Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-20T22:14:44.2008051Z depends: i2c-core,drm 2024-08-20T22:14:44.2008415Z retpoline: Y 2024-08-20T22:14:44.2008722Z name: nvidia 2024-08-20T22:14:44.2009393Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-20T22:14:44.2010114Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-20T22:14:44.2010772Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-20T22:14:44.2011506Z parm: NVreg_ResmanDebugLevel:int 2024-08-20T22:14:44.2011964Z parm: NVreg_RmLogonRC:int 2024-08-20T22:14:44.2012406Z parm: NVreg_ModifyDeviceFiles:int 2024-08-20T22:14:44.2012871Z parm: NVreg_DeviceFileUID:int 2024-08-20T22:14:44.2013307Z parm: NVreg_DeviceFileGID:int 2024-08-20T22:14:44.2013759Z parm: NVreg_DeviceFileMode:int 2024-08-20T22:14:44.2014293Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-20T22:14:44.2014872Z parm: NVreg_UsePageAttributeTable:int 2024-08-20T22:14:44.2015364Z parm: NVreg_EnablePCIeGen3:int 2024-08-20T22:14:44.2015803Z parm: NVreg_EnableMSI:int 2024-08-20T22:14:44.2016230Z parm: NVreg_TCEBypassMode:int 2024-08-20T22:14:44.2016691Z parm: NVreg_EnableStreamMemOPs:int 2024-08-20T22:14:44.2017227Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-20T22:14:44.2017815Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-20T22:14:44.2018496Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-20T22:14:44.2019109Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-20T22:14:44.2019713Z parm: NVreg_DynamicPowerManagement:int 2024-08-20T22:14:44.2020331Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-20T22:14:44.2020928Z parm: NVreg_EnableGpuFirmware:int 2024-08-20T22:14:44.2021422Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-20T22:14:44.2021963Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-20T22:14:44.2022529Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-20T22:14:44.2023033Z parm: NVreg_MemoryPoolSize:int 2024-08-20T22:14:44.2023497Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-20T22:14:44.2023987Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-20T22:14:44.2024468Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-20T22:14:44.2024924Z parm: NVreg_NvLinkDisable:int 2024-08-20T22:14:44.2025430Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-20T22:14:44.2025967Z parm: NVreg_RegisterPCIDriver:int 2024-08-20T22:14:44.2026452Z parm: NVreg_EnableResizableBar:int 2024-08-20T22:14:44.2026940Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-20T22:14:44.2027441Z parm: NVreg_EnableNonblockingOpen:int 2024-08-20T22:14:44.2027934Z parm: NVreg_RegistryDwords:charp 2024-08-20T22:14:44.2028449Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-20T22:14:44.2029028Z parm: NVreg_RmMsg:charp 2024-08-20T22:14:44.2029492Z parm: NVreg_GpuBlacklist:charp 2024-08-20T22:14:44.2029967Z parm: NVreg_TemporaryFilePath:charp 2024-08-20T22:14:44.2030443Z parm: NVreg_ExcludedGpus:charp 2024-08-20T22:14:44.2030901Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-20T22:14:44.2031382Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-20T22:14:44.2031863Z parm: NVreg_ImexChannelCount:int 2024-08-20T22:14:44.2032334Z parm: rm_firmware_active:charp 2024-08-20T22:14:44.2032734Z + set +e 2024-08-20T22:14:44.2033055Z + nvidia-smi 2024-08-20T22:14:44.2203213Z Tue Aug 20 22:14:44 2024 2024-08-20T22:14:44.2203967Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:14:44.2204786Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:14:44.2205574Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:14:44.2206391Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:14:44.2207288Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:14:44.2207987Z | | | MIG M. | 2024-08-20T22:14:44.2208542Z |=========================================+========================+======================| 2024-08-20T22:14:44.2495557Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:14:44.2496350Z | 0% 27C P8 23W / 300W | 0MiB / 23028MiB | 0% Default | 2024-08-20T22:14:44.2496988Z | | | N/A | 2024-08-20T22:14:44.2497689Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:14:44.2499963Z 2024-08-20T22:14:44.2500787Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:14:44.2501423Z | Processes: | 2024-08-20T22:14:44.2502128Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:14:44.2503004Z | ID ID Usage | 2024-08-20T22:14:44.2503579Z |=========================================================================================| 2024-08-20T22:14:44.2584618Z | No running processes found | 2024-08-20T22:14:44.2585431Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:14:44.5347358Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-20T22:14:44.5527010Z NVIDIA A10G 2024-08-20T22:14:44.5586801Z + NVIDIA_SMI_STATUS=0 2024-08-20T22:14:44.5587381Z + '[' 0 -eq 0 ']' 2024-08-20T22:14:44.5587836Z + echo 'INFO: Ignoring allowed status 0' 2024-08-20T22:14:44.5588260Z + set -e 2024-08-20T22:14:44.5588561Z INFO: Ignoring allowed status 0 2024-08-20T22:14:44.5596748Z == Installing nvidia container toolkit for amzn2023 == 2024-08-20T22:14:44.5601126Z + sudo yum install -y yum-utils 2024-08-20T22:14:44.9561656Z Last metadata expiration check: 0:34:02 ago on Tue Aug 20 21:40:42 2024. 2024-08-20T22:14:44.9798945Z Package dnf-utils-4.3.0-13.amzn2023.0.4.noarch is already installed. 2024-08-20T22:14:45.0100904Z Dependencies resolved. 2024-08-20T22:14:45.0225150Z Nothing to do. 2024-08-20T22:14:45.0225576Z Complete! 2024-08-20T22:14:45.0948421Z + [[ amzn2023 == \a\m\z\n\2\0\2\3 ]] 2024-08-20T22:14:45.0949542Z + YUM_REPO_URL=https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:14:45.0951227Z + sudo yum-config-manager --add-repo https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:14:45.3679583Z Adding repo from: https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-20T22:14:45.4373374Z + sudo yum install -y nvidia-docker2 2024-08-20T22:14:45.9163765Z nvidia-container-toolkit 11 kB/s | 833 B 00:00 2024-08-20T22:14:45.9393621Z Package nvidia-docker2-2.14.0-1.noarch is already installed. 2024-08-20T22:14:45.9689801Z Dependencies resolved. 2024-08-20T22:14:45.9815840Z Nothing to do. 2024-08-20T22:14:45.9816408Z Complete! 2024-08-20T22:14:46.0724362Z + sudo systemctl restart docker 2024-08-20T22:15:33.4111861Z nvidia-persistenced failed to initialize. Check syslog for more details. 2024-08-20T22:15:33.4348681Z Tue Aug 20 22:15:33 2024 2024-08-20T22:15:33.4349409Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:15:33.4350294Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-20T22:15:33.4351330Z |-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:15:33.4353029Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-20T22:15:33.4353978Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-20T22:15:33.4354696Z | | | MIG M. | 2024-08-20T22:15:33.4355311Z |=========================================+========================+======================| 2024-08-20T22:15:33.4582058Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2024-08-20T22:15:33.4582802Z | 0% 26C P8 23W / 300W | 0MiB / 23028MiB | 0% Default | 2024-08-20T22:15:33.4583451Z | | | N/A | 2024-08-20T22:15:33.4584185Z +-----------------------------------------+------------------------+----------------------+ 2024-08-20T22:15:33.4586202Z 2024-08-20T22:15:33.4586909Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:15:33.4587546Z | Processes: | 2024-08-20T22:15:33.4588483Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-20T22:15:33.4589202Z | ID ID Usage | 2024-08-20T22:15:33.4589778Z |=========================================================================================| 2024-08-20T22:15:33.4593339Z | No running processes found | 2024-08-20T22:15:33.4594148Z +-----------------------------------------------------------------------------------------+ 2024-08-20T22:15:33.9666760Z Command completed after 1 attempt(s). 2024-08-20T22:15:33.9743026Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:15:33.9743813Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T22:15:33.9744510Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-20T22:15:33.9745166Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-20T22:15:33.9756754Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:33.9757255Z env: 2024-08-20T22:15:33.9757533Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:33.9757987Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:33.9758462Z ##[endgroup] 2024-08-20T22:15:34.4676887Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T22:15:34.9044257Z Collecting psutil==5.9.1 2024-08-20T22:15:34.9377690Z Downloading psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB) 2024-08-20T22:15:34.9840547Z Collecting nvidia-ml-py==11.525.84 2024-08-20T22:15:34.9886024Z Downloading nvidia_ml_py-11.525.84-py3-none-any.whl (34 kB) 2024-08-20T22:15:35.0757511Z Installing collected packages: psutil, nvidia-ml-py 2024-08-20T22:15:35.2381132Z Successfully installed nvidia-ml-py-11.525.84 psutil-5.9.1 2024-08-20T22:15:35.4382214Z Prepare all required actions 2024-08-20T22:15:35.4382882Z Getting action download info 2024-08-20T22:15:35.5427651Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-20T22:15:35.7199573Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-20T22:15:35.8561759Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-20T22:15:35.8562233Z with: 2024-08-20T22:15:35.8562582Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:15:35.8563037Z s3-bucket: gha-artifacts 2024-08-20T22:15:35.8563380Z env: 2024-08-20T22:15:35.8563648Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:35.8564101Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:35.8564580Z ##[endgroup] 2024-08-20T22:15:35.8598188Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:15:35.8598638Z with: 2024-08-20T22:15:35.8599002Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:15:35.8599469Z s3-bucket: gha-artifacts 2024-08-20T22:15:35.8599827Z region: us-east-1 2024-08-20T22:15:35.8600134Z env: 2024-08-20T22:15:35.8600403Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:35.8600860Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:35.8601344Z ##[endgroup] 2024-08-20T22:15:36.3310973Z (node:111014) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:15:36.3311680Z 2024-08-20T22:15:36.3311943Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:15:36.3312668Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:15:36.3313544Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:15:36.3989070Z Found 1 objects with prefix pytorch/pytorch/10479310721/linux-focal-cuda12.1-py3.10-gcc9-sm86/ 2024-08-20T22:15:36.3990154Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:15:43.9311329Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T22:15:43.9320970Z Artifact download has finished successfully 2024-08-20T22:15:43.9668004Z ##[group]Run unzip -o artifacts.zip 2024-08-20T22:15:43.9668440Z unzip -o artifacts.zip 2024-08-20T22:15:43.9677311Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:43.9677809Z env: 2024-08-20T22:15:43.9678094Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:43.9678565Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:43.9679051Z ##[endgroup] 2024-08-20T22:15:43.9775064Z Archive: artifacts.zip 2024-08-20T22:15:43.9776909Z creating: dist/ 2024-08-20T22:15:46.1236242Z inflating: dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:15:46.1237972Z creating: build/custom_test_artifacts/ 2024-08-20T22:15:46.1239221Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-20T22:15:46.1240598Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-20T22:15:46.1241438Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:15:46.1245872Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:15:46.1247179Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-20T22:15:46.1248436Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:15:46.1249724Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:15:46.1250930Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:15:46.1252329Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:15:46.1254097Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:15:46.1255135Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:15:46.1256203Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:15:46.1257640Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:15:46.1259119Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:15:46.1260725Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:15:46.1262221Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:15:46.1263927Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:15:46.1265652Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:15:46.1266978Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:15:46.1267986Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:15:46.1307640Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:15:46.1349482Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:15:46.1351106Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:15:46.1396966Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:15:46.1398810Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:15:46.1400741Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:15:46.1402739Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:15:46.1404534Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:15:46.1406445Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:15:46.1408265Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:15:46.1410125Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:15:46.1411922Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:15:46.1413342Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:15:46.1414604Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:15:46.1415835Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:15:46.1417265Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:15:46.1418467Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:15:46.1419687Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:15:46.1474918Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:15:46.1537421Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:15:46.1540524Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:15:46.1541852Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:15:46.1543233Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:15:46.1544377Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-20T22:15:46.1545621Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-20T22:15:46.1546959Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-20T22:15:46.1548447Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-20T22:15:46.1549585Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-20T22:15:46.1550628Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-20T22:15:46.1551706Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-20T22:15:46.1552797Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-20T22:15:46.1553880Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-20T22:15:46.1554960Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-20T22:15:46.1556021Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-20T22:15:46.1569058Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-20T22:15:46.1713189Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-20T22:15:46.1714542Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-20T22:15:46.1716014Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-20T22:15:46.1717630Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-20T22:15:46.1719121Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-20T22:15:46.1720574Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-20T22:15:46.1721907Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-20T22:15:46.1723083Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-20T22:15:46.1724226Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-20T22:15:46.1725381Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-20T22:15:46.1726515Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-20T22:15:46.1743044Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-20T22:15:46.1827192Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-20T22:15:46.1828809Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:15:46.1830191Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:15:46.1831651Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-20T22:15:46.1832895Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-20T22:15:46.1833928Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-20T22:15:46.1834838Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-08-20T22:15:46.1836063Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-20T22:15:46.1837366Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-20T22:15:46.1838489Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-20T22:15:46.1961011Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-20T22:15:46.2024255Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-20T22:15:46.2025266Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-20T22:15:46.2026062Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-20T22:15:46.2026884Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:15:46.2033670Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:15:46.2034911Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-20T22:15:46.2036105Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:15:46.2037175Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:15:46.2038353Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:15:46.2039598Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:15:46.2041077Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:15:46.2042291Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:15:46.2043717Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:15:46.2045262Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:15:46.2046721Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:15:46.2048611Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:15:46.2050106Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:15:46.2051802Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:15:46.2053366Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:15:46.2054739Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:15:46.2055750Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:15:46.2095818Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:15:46.2135921Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:15:46.2137776Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:15:46.2185042Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:15:46.2186856Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:15:46.2189107Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:15:46.2191075Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:15:46.2192835Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:15:46.2194704Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:15:46.2196537Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:15:46.2198568Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:15:46.2200231Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:15:46.2201730Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:15:46.2202971Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:15:46.2204184Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:15:46.2205399Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:15:46.2206594Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:15:46.2207812Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:15:46.2261350Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:15:46.2321797Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:15:46.2323380Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:15:46.2324696Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:15:46.2325878Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:15:46.2327079Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-20T22:15:46.2328340Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-20T22:15:46.2329687Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-20T22:15:46.2331464Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-20T22:15:46.2332994Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-20T22:15:46.2334293Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-20T22:15:46.2335391Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-20T22:15:46.2336504Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-20T22:15:46.2337617Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-20T22:15:46.2338916Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-20T22:15:46.2340018Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-20T22:15:46.2355470Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-20T22:15:46.2420023Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-20T22:15:46.2421647Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:15:46.2423040Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:15:46.2424252Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-20T22:15:46.2425449Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-20T22:15:46.2426474Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-20T22:15:46.2427371Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-08-20T22:15:46.2428578Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-20T22:15:46.2429723Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-20T22:15:46.2431129Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-20T22:15:46.2482612Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-20T22:15:46.2483654Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-20T22:15:46.2484544Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-20T22:15:46.2485434Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-20T22:15:46.2491678Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T22:15:46.2493013Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-20T22:15:46.2494262Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T22:15:46.2495324Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T22:15:46.2496620Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T22:15:46.2497922Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T22:15:46.2499484Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T22:15:46.2500571Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T22:15:46.2501792Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T22:15:46.2503214Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T22:15:46.2504731Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T22:15:46.2506245Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T22:15:46.2507677Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T22:15:46.2509195Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T22:15:46.2510720Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T22:15:46.2512058Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-20T22:15:46.2513355Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-20T22:15:46.2554209Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-20T22:15:46.2594029Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-20T22:15:46.2595873Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-20T22:15:46.2641538Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-20T22:15:46.2643773Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-20T22:15:46.2645770Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-20T22:15:46.2647820Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-20T22:15:46.2649660Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-20T22:15:46.2651740Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-20T22:15:46.2653653Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-20T22:15:46.2655435Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-20T22:15:46.2657142Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-20T22:15:46.2658645Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-20T22:15:46.2659941Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-20T22:15:46.2661221Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-20T22:15:46.2662522Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-20T22:15:46.2663794Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-20T22:15:46.2665076Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-20T22:15:46.2718638Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-20T22:15:46.2779445Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-20T22:15:46.2781192Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-20T22:15:46.2782633Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-20T22:15:46.2783890Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-08-20T22:15:46.2785228Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-20T22:15:46.2786594Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-20T22:15:46.2788068Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-20T22:15:46.2789785Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-20T22:15:46.2791534Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-20T22:15:46.2792694Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-20T22:15:46.2793869Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-20T22:15:46.2795370Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-20T22:15:46.2796697Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-20T22:15:46.2797885Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-20T22:15:46.2799058Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-20T22:15:46.2800301Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-20T22:15:46.2922335Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-20T22:15:46.2923900Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-20T22:15:46.2925506Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-20T22:15:46.2927198Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-20T22:15:46.2928922Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-20T22:15:46.2930552Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-20T22:15:46.2931937Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-20T22:15:46.2933189Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-20T22:15:46.2934427Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-20T22:15:46.2935665Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-20T22:15:46.2936888Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-20T22:15:46.2952992Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-20T22:15:46.3008525Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-20T22:15:46.3010294Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T22:15:46.3013370Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-20T22:15:46.3015948Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-20T22:15:46.3018399Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-20T22:15:46.3020703Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-20T22:15:46.3021697Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-08-20T22:15:46.3022619Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-20T22:15:46.3023463Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-20T22:15:46.3024319Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-20T22:15:46.3123737Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-20T22:15:46.3167376Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-20T22:15:46.3168165Z creating: build/lib/ 2024-08-20T22:15:46.3256651Z inflating: build/lib/libprotobuf-lite.a 2024-08-20T22:15:46.3718079Z inflating: build/lib/libprotobuf.a 2024-08-20T22:15:46.3728808Z inflating: build/lib/libpthreadpool.a 2024-08-20T22:15:46.3736799Z inflating: build/lib/libcpuinfo.a 2024-08-20T22:15:46.3745642Z inflating: build/lib/libcpuinfo_internals.a 2024-08-20T22:15:46.3746524Z inflating: build/lib/libclog.a 2024-08-20T22:15:46.3765421Z inflating: build/lib/libnnpack.a 2024-08-20T22:15:46.3768024Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-20T22:15:46.3833232Z inflating: build/lib/libgtest.a 2024-08-20T22:15:46.3908667Z inflating: build/lib/libbenchmark.a 2024-08-20T22:15:46.3973212Z inflating: build/lib/libasmjit.a 2024-08-20T22:15:46.3980729Z inflating: build/lib/libittnotify.a 2024-08-20T22:15:46.4008528Z inflating: build/lib/libtensorpipe_uv.a 2024-08-20T22:15:46.4136016Z inflating: build/lib/libgloo.a 2024-08-20T22:15:46.4157110Z inflating: build/lib/libfmt.a 2024-08-20T22:15:46.4252454Z inflating: build/lib/libc10.so 2024-08-20T22:15:46.4254200Z inflating: build/lib/libcaffe2_nvrtc.so 2024-08-20T22:15:46.4255590Z inflating: build/lib/libtorch_global_deps.so 2024-08-20T22:15:46.4763042Z inflating: build/lib/libprotoc.a 2024-08-20T22:15:46.4782597Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-20T22:15:46.4800439Z inflating: build/lib/libgmock.a 2024-08-20T22:15:46.4801090Z inflating: build/lib/libgtest_main.a 2024-08-20T22:15:46.4802165Z inflating: build/lib/libbenchmark_main.a 2024-08-20T22:15:46.5194204Z inflating: build/lib/libgloo_cuda.a 2024-08-20T22:15:47.5468676Z inflating: build/lib/libdnnl.a 2024-08-20T22:15:47.6049607Z inflating: build/lib/libtensorpipe.a 2024-08-20T22:15:47.6106893Z inflating: build/lib/libc10_cuda.so 2024-08-20T22:15:47.6107529Z inflating: build/lib/libgmock_main.a 2024-08-20T22:15:47.7378491Z inflating: build/lib/libfbgemm.a 2024-08-20T22:15:47.7662729Z inflating: build/lib/libtensorpipe_cuda.a 2024-08-20T22:15:47.8177605Z inflating: build/lib/libkineto.a 2024-08-20T22:15:47.8363145Z inflating: build/lib/libXNNPACK.a 2024-08-20T22:15:47.8406406Z inflating: build/lib/libonnx_proto.a 2024-08-20T22:15:47.9102182Z inflating: build/lib/libonnx.a 2024-08-20T22:15:50.3590024Z inflating: build/lib/libtorch_cpu.so 2024-08-20T22:15:50.3594879Z inflating: build/lib/libunbox_lib.a 2024-08-20T22:15:50.3599143Z inflating: build/lib/libshm.so 2024-08-20T22:15:52.4252229Z inflating: build/lib/libtorch_cuda.so 2024-08-20T22:15:52.4253232Z inflating: build/lib/libtorch.so 2024-08-20T22:15:52.4256231Z inflating: build/lib/libc10d_cuda_test.so 2024-08-20T22:15:53.2429556Z inflating: build/lib/libtorch_cuda_linalg.so 2024-08-20T22:15:53.4425016Z inflating: build/lib/libtorch_python.so 2024-08-20T22:15:53.4497349Z inflating: build/lib/libtorchbind_test.so 2024-08-20T22:15:53.4518073Z inflating: build/lib/libjitbackend_test.so 2024-08-20T22:15:53.4543839Z inflating: build/lib/libbackend_with_compiler.so 2024-08-20T22:15:53.4569800Z inflating: build/lib/libaoti_custom_ops.so 2024-08-20T22:15:53.4604403Z inflating: build/lib/libnnapi_backend.so 2024-08-20T22:15:53.4605049Z creating: build/bin/ 2024-08-20T22:15:53.4655597Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-20T22:15:53.4707181Z inflating: build/bin/c10_DeviceGuard_test 2024-08-20T22:15:53.4758445Z inflating: build/bin/c10_Device_test 2024-08-20T22:15:53.4817677Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-20T22:15:53.4871349Z inflating: build/bin/c10_Scalar_test 2024-08-20T22:15:53.4920943Z inflating: build/bin/c10_StreamGuard_test 2024-08-20T22:15:53.4973116Z inflating: build/bin/c10_SymInt_test 2024-08-20T22:15:53.5027123Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-20T22:15:53.5083225Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-20T22:15:53.5139660Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-20T22:15:53.5211180Z inflating: build/bin/c10_cow_test 2024-08-20T22:15:53.5264374Z inflating: build/bin/c10_Bitset_test 2024-08-20T22:15:53.5313583Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-20T22:15:53.5363826Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-20T22:15:53.5414657Z inflating: build/bin/c10_Half_test 2024-08-20T22:15:53.5471219Z inflating: build/bin/c10_LeftRight_test 2024-08-20T22:15:53.5526498Z inflating: build/bin/c10_Metaprogramming_test 2024-08-20T22:15:53.5577213Z inflating: build/bin/c10_Synchronized_test 2024-08-20T22:15:53.5632941Z inflating: build/bin/c10_ThreadLocal_test 2024-08-20T22:15:53.5685116Z inflating: build/bin/c10_TypeIndex_test 2024-08-20T22:15:53.5736502Z inflating: build/bin/c10_TypeList_test 2024-08-20T22:15:53.5785726Z inflating: build/bin/c10_TypeTraits_test 2024-08-20T22:15:53.5838331Z inflating: build/bin/c10_accumulate_test 2024-08-20T22:15:53.5893351Z inflating: build/bin/c10_bfloat16_test 2024-08-20T22:15:53.5944188Z inflating: build/bin/c10_bit_cast_test 2024-08-20T22:15:53.6001656Z inflating: build/bin/c10_complex_math_test 2024-08-20T22:15:53.6057013Z inflating: build/bin/c10_complex_test 2024-08-20T22:15:53.6110822Z inflating: build/bin/c10_exception_test 2024-08-20T22:15:53.6161451Z inflating: build/bin/c10_flags_test 2024-08-20T22:15:53.6211788Z inflating: build/bin/c10_generic_math_test 2024-08-20T22:15:53.6377036Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-20T22:15:53.6427732Z inflating: build/bin/c10_irange_test 2024-08-20T22:15:53.6482424Z inflating: build/bin/c10_lazy_test 2024-08-20T22:15:53.6538994Z inflating: build/bin/c10_logging_test 2024-08-20T22:15:53.6615900Z inflating: build/bin/c10_optional_test 2024-08-20T22:15:53.6678860Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-20T22:15:53.6734690Z inflating: build/bin/c10_registry_test 2024-08-20T22:15:53.6886773Z inflating: build/bin/c10_small_vector_test 2024-08-20T22:15:53.6938275Z inflating: build/bin/c10_ssize_test 2024-08-20T22:15:53.6990632Z inflating: build/bin/c10_string_util_test 2024-08-20T22:15:53.7041076Z inflating: build/bin/c10_tempfile_test 2024-08-20T22:15:53.7100338Z inflating: build/bin/c10_string_view_test 2024-08-20T22:15:53.7149207Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-20T22:15:53.7205879Z inflating: build/bin/c10_typeid_test 2024-08-20T22:15:53.7653071Z inflating: build/bin/protoc-3.13.0.0 2024-08-20T22:15:53.8101189Z inflating: build/bin/protoc 2024-08-20T22:15:53.8154627Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-08-20T22:15:53.8207652Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-08-20T22:15:53.8260907Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-08-20T22:15:53.8312831Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-08-20T22:15:53.8366331Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-08-20T22:15:53.8419440Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-08-20T22:15:53.8468739Z inflating: build/bin/c10_cuda_CUDATest 2024-08-20T22:15:53.8521651Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-08-20T22:15:53.8852477Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-20T22:15:53.9198770Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-20T22:15:53.9558956Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-20T22:15:53.9612381Z inflating: build/bin/BackoffTest 2024-08-20T22:15:53.9665573Z inflating: build/bin/FileStoreTest 2024-08-20T22:15:53.9721696Z inflating: build/bin/TCPStoreTest 2024-08-20T22:15:53.9775867Z inflating: build/bin/HashStoreTest 2024-08-20T22:15:53.9789174Z inflating: build/bin/ProcessGroupMPITest 2024-08-20T22:15:53.9844258Z inflating: build/bin/test_edge_op_registration 2024-08-20T22:15:53.9847786Z inflating: build/bin/torch_shm_manager 2024-08-20T22:15:53.9850749Z inflating: build/bin/example_allreduce 2024-08-20T22:15:53.9906536Z inflating: build/bin/test_dist_autograd 2024-08-20T22:15:53.9975358Z inflating: build/bin/test_cpp_rpc 2024-08-20T22:15:53.9978028Z inflating: build/bin/parallel_benchmark 2024-08-20T22:15:54.0045393Z inflating: build/bin/test_mobile_nnc 2024-08-20T22:15:54.0054653Z inflating: build/bin/aot_model_compiler_test 2024-08-20T22:15:54.0402137Z inflating: build/bin/test_lazy 2024-08-20T22:15:54.1585841Z inflating: build/bin/test_api 2024-08-20T22:15:54.1659438Z inflating: build/bin/Dict_test 2024-08-20T22:15:54.1711619Z inflating: build/bin/Dimname_test 2024-08-20T22:15:54.1776580Z inflating: build/bin/MaybeOwned_test 2024-08-20T22:15:54.1833901Z inflating: build/bin/NamedTensor_test 2024-08-20T22:15:54.1894213Z inflating: build/bin/apply_utils_test 2024-08-20T22:15:54.1952854Z inflating: build/bin/atest 2024-08-20T22:15:54.2016311Z inflating: build/bin/basic 2024-08-20T22:15:54.2071482Z inflating: build/bin/broadcast_test 2024-08-20T22:15:54.2122491Z inflating: build/bin/cpu_allocator_test 2024-08-20T22:15:54.2181064Z inflating: build/bin/cpu_generator_test 2024-08-20T22:15:54.2234513Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-20T22:15:54.2328054Z inflating: build/bin/cpu_rng_test 2024-08-20T22:15:54.2378490Z inflating: build/bin/dispatch_key_set_test 2024-08-20T22:15:54.2429059Z inflating: build/bin/dlconvertor_test 2024-08-20T22:15:54.2489816Z inflating: build/bin/extension_backend_test 2024-08-20T22:15:54.2544782Z inflating: build/bin/half_test 2024-08-20T22:15:54.2641366Z inflating: build/bin/ivalue_test 2024-08-20T22:15:54.2690715Z inflating: build/bin/lazy_tensor_test 2024-08-20T22:15:54.2745598Z inflating: build/bin/math_kernel_test 2024-08-20T22:15:54.2801028Z inflating: build/bin/memory_format_test 2024-08-20T22:15:54.2853858Z inflating: build/bin/memory_overlapping_test 2024-08-20T22:15:54.2906919Z inflating: build/bin/mobile_memory_cleanup 2024-08-20T22:15:54.2964122Z inflating: build/bin/native_test 2024-08-20T22:15:54.3014990Z inflating: build/bin/operator_name_test 2024-08-20T22:15:54.3066689Z inflating: build/bin/operators_test 2024-08-20T22:15:54.3118510Z inflating: build/bin/packedtensoraccessor_test 2024-08-20T22:15:54.3186052Z inflating: build/bin/pow_test 2024-08-20T22:15:54.3243726Z inflating: build/bin/quantized_test 2024-08-20T22:15:54.3294422Z inflating: build/bin/reduce_ops_test 2024-08-20T22:15:54.3345228Z inflating: build/bin/reportMemoryUsage_test 2024-08-20T22:15:54.3402424Z inflating: build/bin/scalar_tensor_test 2024-08-20T22:15:54.3461410Z inflating: build/bin/scalar_test 2024-08-20T22:15:54.3513521Z inflating: build/bin/StorageUtils_test 2024-08-20T22:15:54.3566755Z inflating: build/bin/stride_properties_test 2024-08-20T22:15:54.3645780Z inflating: build/bin/tensor_iterator_test 2024-08-20T22:15:54.3700336Z inflating: build/bin/test_parallel 2024-08-20T22:15:54.3703364Z inflating: build/bin/thread_init_test 2024-08-20T22:15:54.3759702Z inflating: build/bin/type_ptr_test 2024-08-20T22:15:54.3820598Z inflating: build/bin/type_test 2024-08-20T22:15:54.3873367Z inflating: build/bin/undefined_tensor_test 2024-08-20T22:15:54.3874743Z inflating: build/bin/verify_api_visibility 2024-08-20T22:15:54.3944131Z inflating: build/bin/legacy_vmap_test 2024-08-20T22:15:54.3996300Z inflating: build/bin/weakref_test 2024-08-20T22:15:54.4047991Z inflating: build/bin/wrapdim_test 2024-08-20T22:15:54.4099685Z inflating: build/bin/xla_tensor_test 2024-08-20T22:15:54.4160212Z inflating: build/bin/IListRef_test 2024-08-20T22:15:54.4267570Z inflating: build/bin/List_test 2024-08-20T22:15:54.4333880Z inflating: build/bin/KernelFunction_test 2024-08-20T22:15:54.4456594Z inflating: build/bin/kernel_function_legacy_test 2024-08-20T22:15:54.4553451Z inflating: build/bin/kernel_function_test 2024-08-20T22:15:54.4683444Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-20T22:15:54.4787321Z inflating: build/bin/kernel_lambda_test 2024-08-20T22:15:54.4848714Z inflating: build/bin/kernel_stackbased_test 2024-08-20T22:15:54.4945500Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-20T22:15:54.4997143Z inflating: build/bin/CppSignature_test 2024-08-20T22:15:54.5046742Z inflating: build/bin/op_allowlist_test 2024-08-20T22:15:54.5102381Z inflating: build/bin/backend_fallback_test 2024-08-20T22:15:54.5405704Z inflating: build/bin/op_registration_test 2024-08-20T22:15:54.5469862Z inflating: build/bin/inline_container_test 2024-08-20T22:15:54.5523378Z inflating: build/bin/cuda_apply_test 2024-08-20T22:15:54.5575638Z inflating: build/bin/cuda_allocator_test 2024-08-20T22:15:54.5634277Z inflating: build/bin/cuda_atomic_ops_test 2024-08-20T22:15:54.5690165Z inflating: build/bin/cuda_caching_host_allocator_test 2024-08-20T22:15:54.5759601Z inflating: build/bin/cuda_complex_math_test 2024-08-20T22:15:54.5818899Z inflating: build/bin/cuda_complex_test 2024-08-20T22:15:54.5868694Z inflating: build/bin/cuda_device_test 2024-08-20T22:15:54.5926241Z inflating: build/bin/cuda_cub_test 2024-08-20T22:15:54.5978089Z inflating: build/bin/cuda_dlconvertor_test 2024-08-20T22:15:54.6043226Z inflating: build/bin/cuda_distributions_test 2024-08-20T22:15:54.6099942Z inflating: build/bin/cuda_generator_test 2024-08-20T22:15:54.6150111Z inflating: build/bin/cuda_half_test 2024-08-20T22:15:54.6202099Z inflating: build/bin/cuda_integer_divider_test 2024-08-20T22:15:54.6251741Z inflating: build/bin/cuda_optional_test 2024-08-20T22:15:54.6303542Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-08-20T22:15:54.6356772Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-08-20T22:15:54.6406901Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-08-20T22:15:54.6467999Z inflating: build/bin/cuda_stream_test 2024-08-20T22:15:54.6517538Z inflating: build/bin/cuda_cudnn_test 2024-08-20T22:15:54.6570002Z inflating: build/bin/cuda_vectorized_test 2024-08-20T22:15:54.6584707Z inflating: build/bin/tutorial_tensorexpr 2024-08-20T22:15:54.6650588Z inflating: build/bin/ProcessGroupGlooTest 2024-08-20T22:15:54.6708214Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-08-20T22:15:54.6772204Z inflating: build/bin/ProcessGroupNCCLTest 2024-08-20T22:15:54.6836297Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-08-20T22:15:54.7664680Z inflating: build/bin/test_tensorexpr 2024-08-20T22:15:54.8247439Z inflating: build/bin/test_jit 2024-08-20T22:15:54.8248044Z creating: .additional_ci_files/ 2024-08-20T22:15:54.8309292Z inflating: .additional_ci_files/test-times.json 2024-08-20T22:15:54.8548288Z inflating: .additional_ci_files/test-class-times.json 2024-08-20T22:15:54.8584423Z ##[group]Run rm artifacts.zip 2024-08-20T22:15:54.8584822Z rm artifacts.zip 2024-08-20T22:15:54.8593740Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:54.8594246Z env: 2024-08-20T22:15:54.8594532Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:54.8594996Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:54.8595486Z ##[endgroup] 2024-08-20T22:15:54.9742782Z ##[group]Run df -H 2024-08-20T22:15:54.9743227Z df -H 2024-08-20T22:15:54.9752300Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:54.9752792Z env: 2024-08-20T22:15:54.9753073Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:54.9753699Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:54.9754181Z ##[endgroup] 2024-08-20T22:15:54.9829595Z Filesystem Size Used Avail Use% Mounted on 2024-08-20T22:15:54.9830252Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-20T22:15:54.9830786Z tmpfs 34G 0 34G 0% /dev/shm 2024-08-20T22:15:54.9831258Z tmpfs 14G 553k 14G 1% /run 2024-08-20T22:15:54.9831711Z /dev/nvme0n1p1 161G 75G 87G 47% / 2024-08-20T22:15:54.9832386Z tmpfs 34G 959k 34G 1% /tmp 2024-08-20T22:15:54.9832881Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-20T22:15:54.9833388Z tmpfs 6.7G 0 6.7G 0% /run/user/0 2024-08-20T22:15:54.9893712Z Prepare all required actions 2024-08-20T22:15:54.9894355Z Getting action download info 2024-08-20T22:15:55.0977184Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-20T22:15:55.0977646Z with: 2024-08-20T22:15:55.0977909Z env: 2024-08-20T22:15:55.0978195Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:55.0978661Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:55.0979146Z ##[endgroup] 2024-08-20T22:15:55.1023190Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T22:15:55.1023634Z with: 2024-08-20T22:15:55.1023904Z name: td_results 2024-08-20T22:15:55.1024237Z s3-bucket: gha-artifacts 2024-08-20T22:15:55.1024596Z region: us-east-1 2024-08-20T22:15:55.1024903Z env: 2024-08-20T22:15:55.1025184Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:55.1025641Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:55.1026124Z ##[endgroup] 2024-08-20T22:15:55.5756561Z (node:111033) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T22:15:55.5757446Z 2024-08-20T22:15:55.5757769Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T22:15:55.5758601Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T22:15:55.5759474Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T22:15:55.6366438Z Found 0 objects with prefix pytorch/pytorch/10479310721/td_results/ 2024-08-20T22:15:55.6373946Z Artifact download has finished successfully 2024-08-20T22:15:55.6697216Z ##[group]Run mkdir -p .additional_ci_files 2024-08-20T22:15:55.6697713Z mkdir -p .additional_ci_files 2024-08-20T22:15:55.6698282Z mv td_results.json .additional_ci_files/td_results.json 2024-08-20T22:15:55.6707129Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:55.6707625Z env: 2024-08-20T22:15:55.6707908Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:55.6708379Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:55.6708864Z ##[endgroup] 2024-08-20T22:15:55.6793532Z mv: cannot stat 'td_results.json': No such file or directory 2024-08-20T22:15:55.6806799Z ##[error]Process completed with exit code 1. 2024-08-20T22:15:55.6841214Z ##[group]Run .github/scripts/parse_ref.py 2024-08-20T22:15:55.6841718Z .github/scripts/parse_ref.py 2024-08-20T22:15:55.6850891Z shell: /usr/bin/bash -e {0} 2024-08-20T22:15:55.6851338Z env: 2024-08-20T22:15:55.6851620Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:55.6852077Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:55.6852556Z ##[endgroup] 2024-08-20T22:15:55.7152649Z Prepare all required actions 2024-08-20T22:15:55.7191121Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-20T22:15:55.7191580Z with: 2024-08-20T22:15:55.7192152Z github-token: *** 2024-08-20T22:15:55.7192464Z env: 2024-08-20T22:15:55.7192744Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:55.7193201Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:55.7193685Z ##[endgroup] 2024-08-20T22:15:55.7223150Z ##[group]Run set -eux 2024-08-20T22:15:55.7223489Z set -eux 2024-08-20T22:15:55.7224075Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-20T22:15:55.7234344Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:55.7234877Z env: 2024-08-20T22:15:55.7235167Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:55.7235647Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:55.7236342Z GITHUB_TOKEN: *** 2024-08-20T22:15:55.7236666Z ##[endgroup] 2024-08-20T22:15:55.7265747Z + python3 .github/scripts/get_workflow_job_id.py 10479310721 i-0b0df095cbe3f37a6 2024-08-20T22:15:56.3763325Z setting job-id=29026298493 2024-08-20T22:15:56.3765347Z setting job-name=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:56.3980481Z Prepare all required actions 2024-08-20T22:15:56.3980926Z Getting action download info 2024-08-20T22:15:56.5081829Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-20T22:15:56.5082286Z with: 2024-08-20T22:15:56.5082751Z github-token: *** 2024-08-20T22:15:56.5089297Z test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-20T22:15:56.5096735Z job-name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:56.5097610Z env: 2024-08-20T22:15:56.5097891Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:56.5098351Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:56.5098836Z ##[endgroup] 2024-08-20T22:15:56.5141020Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T22:15:56.5141584Z with: 2024-08-20T22:15:56.5141852Z shell: bash 2024-08-20T22:15:56.5142153Z timeout_minutes: 10 2024-08-20T22:15:56.5142473Z max_attempts: 5 2024-08-20T22:15:56.5143009Z retry_wait_seconds: 30 2024-08-20T22:15:56.5144126Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:15:56.5145281Z polling_interval_seconds: 1 2024-08-20T22:15:56.5145656Z warning_on_retry: true 2024-08-20T22:15:56.5146002Z continue_on_error: false 2024-08-20T22:15:56.5146347Z env: 2024-08-20T22:15:56.5146626Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:56.5147070Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:56.5147714Z GITHUB_TOKEN: *** 2024-08-20T22:15:56.5148026Z ##[endgroup] 2024-08-20T22:15:56.7531416Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T22:15:57.3398918Z Collecting requests==2.27.1 2024-08-20T22:15:57.3787868Z Downloading requests-2.27.1-py2.py3-none-any.whl.metadata (5.0 kB) 2024-08-20T22:15:57.4252499Z Collecting pyyaml==6.0.1 2024-08-20T22:15:57.4297908Z Downloading PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB) 2024-08-20T22:15:57.4730313Z Collecting urllib3<1.27,>=1.21.1 (from requests==2.27.1) 2024-08-20T22:15:57.4774267Z Downloading urllib3-1.26.19-py2.py3-none-any.whl.metadata (49 kB) 2024-08-20T22:15:57.4965424Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/miniconda/lib/python3.12/site-packages (from requests==2.27.1) (2024.7.4) 2024-08-20T22:15:57.5607460Z Collecting charset-normalizer~=2.0.0 (from requests==2.27.1) 2024-08-20T22:15:57.5653143Z Downloading charset_normalizer-2.0.12-py3-none-any.whl.metadata (11 kB) 2024-08-20T22:15:57.5722823Z Requirement already satisfied: idna<4,>=2.5 in /home/ec2-user/miniconda/lib/python3.12/site-packages (from requests==2.27.1) (3.7) 2024-08-20T22:15:57.5813923Z Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB) 2024-08-20T22:15:57.5962486Z Downloading PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (724 kB) 2024-08-20T22:15:57.6361127Z ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 725.0/725.0 kB 18.5 MB/s eta 0:00:00 2024-08-20T22:15:57.6407273Z Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB) 2024-08-20T22:15:57.6495352Z Downloading urllib3-1.26.19-py2.py3-none-any.whl (143 kB) 2024-08-20T22:15:57.7231504Z Installing collected packages: urllib3, pyyaml, charset-normalizer, requests 2024-08-20T22:15:57.7232782Z Attempting uninstall: urllib3 2024-08-20T22:15:57.7253546Z Found existing installation: urllib3 2.2.2 2024-08-20T22:15:57.7297219Z Uninstalling urllib3-2.2.2: 2024-08-20T22:15:57.7305648Z Successfully uninstalled urllib3-2.2.2 2024-08-20T22:15:57.8481810Z Attempting uninstall: charset-normalizer 2024-08-20T22:15:57.8515944Z Found existing installation: charset-normalizer 3.3.2 2024-08-20T22:15:57.8548346Z Uninstalling charset-normalizer-3.3.2: 2024-08-20T22:15:57.8561781Z Successfully uninstalled charset-normalizer-3.3.2 2024-08-20T22:15:57.8832001Z Attempting uninstall: requests 2024-08-20T22:15:57.8857456Z Found existing installation: requests 2.32.3 2024-08-20T22:15:57.8886038Z Uninstalling requests-2.32.3: 2024-08-20T22:15:57.8893035Z Successfully uninstalled requests-2.32.3 2024-08-20T22:15:57.9266053Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-08-20T22:15:57.9268557Z conda 24.7.1 requires requests<3,>=2.28.0, but you have requests 2.27.1 which is incompatible. 2024-08-20T22:15:57.9269625Z Successfully installed charset-normalizer-2.0.12 pyyaml-6.0.1 requests-2.27.1 urllib3-1.26.19 2024-08-20T22:15:58.5694113Z Command completed after 1 attempt(s). 2024-08-20T22:15:58.5757653Z ##[group]Run set -x 2024-08-20T22:15:58.5758005Z set -x 2024-08-20T22:15:58.5758304Z  2024-08-20T22:15:58.5758857Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:15:58.5759529Z # in runner workspace 2024-08-20T22:15:58.5760060Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-20T22:15:58.5769614Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:58.5770107Z env: 2024-08-20T22:15:58.5770392Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:58.5770851Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:58.5771445Z ##[endgroup] 2024-08-20T22:15:58.5801546Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-20T22:15:58.6044451Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:15:58.6045011Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T22:15:58.6045495Z echo "Job name: ${JOB_NAME}" 2024-08-20T22:15:58.6046095Z  2024-08-20T22:15:58.6046663Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T22:15:58.6047365Z # in runner workspace 2024-08-20T22:15:58.6047967Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-20T22:15:58.6048628Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-20T22:15:58.6049105Z  --job-name "${JOB_NAME}" \ 2024-08-20T22:15:58.6056172Z  --test-matrix "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" \ 2024-08-20T22:15:58.6063073Z  --selected-test-configs "" \ 2024-08-20T22:15:58.6063543Z  --pr-number "${PR_NUMBER}" \ 2024-08-20T22:15:58.6063977Z  --tag "${TAG}" \ 2024-08-20T22:15:58.6064385Z  --event-name "${EVENT_NAME}" \ 2024-08-20T22:15:58.6064841Z  --schedule "${SCHEDULE}" \ 2024-08-20T22:15:58.6065282Z  --branch "${HEAD_BRANCH}" 2024-08-20T22:15:58.6074272Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:58.6074781Z env: 2024-08-20T22:15:58.6075072Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:58.6075536Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:58.6076215Z GITHUB_TOKEN: *** 2024-08-20T22:15:58.6077037Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:58.6077917Z PR_NUMBER: 2024-08-20T22:15:58.6078237Z TAG: ciflow/inductor/133712 2024-08-20T22:15:58.6078613Z EVENT_NAME: push 2024-08-20T22:15:58.6078932Z SCHEDULE: 2024-08-20T22:15:58.6079232Z HEAD_BRANCH: 2024-08-20T22:15:58.6079533Z ##[endgroup] 2024-08-20T22:15:58.6109378Z Workflow: inductor-periodic 2024-08-20T22:15:58.6110525Z Job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:58.8419965Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-20T22:15:58.9906859Z ##[group]Run echo "Filtered matrix:" 2024-08-20T22:15:58.9907336Z echo "Filtered matrix:" 2024-08-20T22:15:58.9914182Z echo "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" 2024-08-20T22:15:58.9921208Z  2024-08-20T22:15:58.9921491Z echo 2024-08-20T22:15:58.9921877Z echo "Is the current job unstable? False" 2024-08-20T22:15:58.9922334Z  2024-08-20T22:15:58.9922612Z echo 2024-08-20T22:15:58.9922987Z echo "Is keep-going label set? False" 2024-08-20T22:15:58.9923430Z  2024-08-20T22:15:58.9923705Z echo 2024-08-20T22:15:58.9924028Z echo "Renabled issues? " 2024-08-20T22:15:58.9933067Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:58.9933564Z env: 2024-08-20T22:15:58.9933851Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:58.9934323Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:58.9934811Z ##[endgroup] 2024-08-20T22:15:58.9963372Z Filtered matrix: 2024-08-20T22:15:58.9970746Z {include: [{config: dynamo_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}]} 2024-08-20T22:15:58.9977405Z 2024-08-20T22:15:58.9977572Z Is the current job unstable? False 2024-08-20T22:15:58.9977863Z 2024-08-20T22:15:58.9978230Z Is keep-going label set? False 2024-08-20T22:15:58.9978491Z 2024-08-20T22:15:58.9978629Z Renabled issues? 2024-08-20T22:15:59.0021221Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:15:59.0021933Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T22:15:59.0030293Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:15:59.0030788Z env: 2024-08-20T22:15:59.0031082Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:59.0031542Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:59.0032032Z JOB_TIMEOUT: 240 2024-08-20T22:15:59.0032338Z ##[endgroup] 2024-08-20T22:15:59.0114033Z ##[group]Run set -x 2024-08-20T22:15:59.0114434Z set -x 2024-08-20T22:15:59.0114730Z  2024-08-20T22:15:59.0115084Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-20T22:15:59.0115621Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-20T22:15:59.0116184Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-20T22:15:59.0116692Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-20T22:15:59.0117108Z else 2024-08-20T22:15:59.0117446Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:15:59.0117864Z fi 2024-08-20T22:15:59.0118154Z  2024-08-20T22:15:59.0118624Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-20T22:15:59.0119380Z # TODO: Stop building test binaries as part of the build phase 2024-08-20T22:15:59.0120058Z # Used for GPU_FLAG since that doesn't play nice 2024-08-20T22:15:59.0120638Z # shellcheck disable=SC2086,SC2090 2024-08-20T22:15:59.0121096Z container_name=$(docker run \ 2024-08-20T22:15:59.0121516Z  ${GPU_FLAG:-} \ 2024-08-20T22:15:59.0121891Z  -e BUILD_ENVIRONMENT \ 2024-08-20T22:15:59.0122290Z  -e PR_NUMBER \ 2024-08-20T22:15:59.0122649Z  -e GITHUB_ACTIONS \ 2024-08-20T22:15:59.0123041Z  -e GITHUB_REPOSITORY \ 2024-08-20T22:15:59.0123447Z  -e GITHUB_WORKFLOW \ 2024-08-20T22:15:59.0123834Z  -e GITHUB_JOB \ 2024-08-20T22:15:59.0124190Z  -e GITHUB_RUN_ID \ 2024-08-20T22:15:59.0124579Z  -e GITHUB_RUN_NUMBER \ 2024-08-20T22:15:59.0124992Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-20T22:15:59.0125389Z  -e JOB_ID \ 2024-08-20T22:15:59.0125722Z  -e JOB_NAME \ 2024-08-20T22:15:59.0126069Z  -e BASE_SHA \ 2024-08-20T22:15:59.0126414Z  -e BRANCH \ 2024-08-20T22:15:59.0126750Z  -e SHA1 \ 2024-08-20T22:15:59.0127095Z  -e AWS_DEFAULT_REGION \ 2024-08-20T22:15:59.0127501Z  -e IN_WHEEL_TEST \ 2024-08-20T22:15:59.0127884Z  -e SHARD_NUMBER \ 2024-08-20T22:15:59.0128255Z  -e TEST_CONFIG \ 2024-08-20T22:15:59.0128634Z  -e NUM_TEST_SHARDS \ 2024-08-20T22:15:59.0129033Z  -e REENABLED_ISSUES \ 2024-08-20T22:15:59.0129450Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-20T22:15:59.0129873Z  -e VERBOSE_TEST_LOGS \ 2024-08-20T22:15:59.0130274Z  -e TEST_SHOWLOCALS \ 2024-08-20T22:15:59.0130664Z  -e NO_TEST_TIMEOUT \ 2024-08-20T22:15:59.0131119Z  -e NO_TD \ 2024-08-20T22:15:59.0131466Z  -e TD_DISTRIBUTED \ 2024-08-20T22:15:59.0131849Z  -e PR_LABELS \ 2024-08-20T22:15:59.0132256Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-20T22:15:59.0132710Z  -e SCCACHE_BUCKET \ 2024-08-20T22:15:59.0133108Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-20T22:15:59.0133515Z  -e XLA_CUDA \ 2024-08-20T22:15:59.0133926Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-20T22:15:59.0134433Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-20T22:15:59.0134962Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-20T22:15:59.0135477Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-20T22:15:59.0135950Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-20T22:15:59.0136403Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2024-08-20T22:15:59.0136844Z  -e DASHBOARD_TAG \ 2024-08-20T22:15:59.0137322Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-20T22:15:59.0138058Z  --security-opt seccomp=unconfined \ 2024-08-20T22:15:59.0138518Z  --cap-add=SYS_PTRACE \ 2024-08-20T22:15:59.0138914Z  --ipc=host \ 2024-08-20T22:15:59.0139276Z  --shm-size="${SHM_SIZE}" \ 2024-08-20T22:15:59.0139678Z  --tty \ 2024-08-20T22:15:59.0139989Z  --detach \ 2024-08-20T22:15:59.0140467Z  --name="${container_name}" \ 2024-08-20T22:15:59.0140890Z  --user jenkins \ 2024-08-20T22:15:59.0141377Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-20T22:15:59.0141933Z  -w /var/lib/jenkins/workspace \ 2024-08-20T22:15:59.0142370Z  "${DOCKER_IMAGE}" 2024-08-20T22:15:59.0143111Z ) 2024-08-20T22:15:59.0143524Z # Propagate download.pytorch.org IP to container 2024-08-20T22:15:59.0144438Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-20T22:15:59.0145404Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-20T22:15:59.0146325Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-20T22:15:59.0155299Z shell: /usr/bin/bash -e {0} 2024-08-20T22:15:59.0155646Z env: 2024-08-20T22:15:59.0155927Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:15:59.0156389Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:15:59.0156989Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:15:59.0157485Z PR_NUMBER: 2024-08-20T22:15:59.0157808Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-20T22:15:59.0158238Z GITHUB_WORKFLOW: inductor-periodic 2024-08-20T22:15:59.0158640Z GITHUB_JOB: test 2024-08-20T22:15:59.0158955Z GITHUB_RUN_ID: 10479310721 2024-08-20T22:15:59.0159366Z GITHUB_RUN_NUMBER: 54565 2024-08-20T22:15:59.0159720Z GITHUB_RUN_ATTEMPT: 1 2024-08-20T22:15:59.0160054Z JOB_ID: 29026298493 2024-08-20T22:15:59.0160854Z JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:15:59.0161719Z BRANCH: 2024-08-20T22:15:59.0162059Z SHA1: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:59.0162581Z BASE_SHA: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:15:59.0163054Z TEST_CONFIG: dynamo_eager_timm 2024-08-20T22:15:59.0163434Z SHARD_NUMBER: 1 2024-08-20T22:15:59.0163740Z NUM_TEST_SHARDS: 2 2024-08-20T22:15:59.0164059Z REENABLED_ISSUES: 2024-08-20T22:15:59.0164389Z CONTINUE_THROUGH_ERROR: False 2024-08-20T22:15:59.0164773Z VERBOSE_TEST_LOGS: False 2024-08-20T22:15:59.0165131Z TEST_SHOWLOCALS: False 2024-08-20T22:15:59.0165470Z NO_TEST_TIMEOUT: False 2024-08-20T22:15:59.0165802Z NO_TD: False 2024-08-20T22:15:59.0166102Z TD_DISTRIBUTED: False 2024-08-20T22:15:59.0166517Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-20T22:15:59.0167023Z SCCACHE_S3_KEY_PREFIX: inductor-periodic 2024-08-20T22:15:59.0167445Z SHM_SIZE: 2g 2024-08-20T22:15:59.0168522Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:15:59.0169682Z XLA_CUDA: 2024-08-20T22:15:59.0170157Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:15:59.0170769Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-20T22:15:59.0171273Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-20T22:15:59.0171682Z DASHBOARD_TAG: 2024-08-20T22:15:59.0172185Z HUGGING_FACE_HUB_TOKEN: *** 2024-08-20T22:15:59.0172560Z SCRIBE_GRAPHQL_ACCESS_TOKEN: 2024-08-20T22:15:59.0172934Z ##[endgroup] 2024-08-20T22:15:59.0205554Z + [[ dynamo_eager_timm == \m\u\l\t\i\g\p\u ]] 2024-08-20T22:15:59.0206299Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *onnx* ]] 2024-08-20T22:15:59.0206820Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T22:15:59.0212477Z +++ nproc --ignore=2 2024-08-20T22:15:59.0256461Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10479310721 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:16:10.8261965Z + container_name=e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:16:10.8267170Z + grep download.pytorch.org /etc/hosts 2024-08-20T22:16:10.8268836Z + docker exec -i e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-20T22:16:11.0287594Z + echo DOCKER_CONTAINER_ID=e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:16:11.0291552Z ++ echo dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl 2024-08-20T22:16:11.0294462Z + docker exec -t e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 sh -c 'pip install dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-20T22:16:11.4380462Z Processing ./dist/torch-2.5.0a0+git40ec5f6-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+git40ec5f6) 2024-08-20T22:16:11.7612171Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.13.1) 2024-08-20T22:16:11.7614519Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (4.12.2) 2024-08-20T22:16:11.7617996Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.8.8) 2024-08-20T22:16:11.7620002Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.1.4) 2024-08-20T22:16:11.7624206Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2024.6.1) 2024-08-20T22:16:11.7628741Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.13.1) 2024-08-20T22:16:11.7645178Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.3.0) 2024-08-20T22:16:11.7658608Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.3.0) 2024-08-20T22:16:11.7675646Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.22.4) 2024-08-20T22:16:11.8064684Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.1.5) 2024-08-20T22:16:12.3882982Z Installing collected packages: torch 2024-08-20T22:16:23.0905406Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-08-20T22:16:23.0907179Z timm 1.0.9.dev0 requires torchvision, which is not installed. 2024-08-20T22:16:23.0907907Z Successfully installed torch-2.5.0a0+git40ec5f6 2024-08-20T22:16:23.1732442Z + export TERM=vt100 2024-08-20T22:16:23.1732776Z + TERM=vt100 2024-08-20T22:16:23.1735533Z ++ dirname .ci/pytorch/test.sh 2024-08-20T22:16:23.1751410Z + source .ci/pytorch/common.sh 2024-08-20T22:16:23.1754878Z +++ dirname .ci/pytorch/common.sh 2024-08-20T22:16:23.1763113Z ++ source .ci/pytorch/common_utils.sh 2024-08-20T22:16:23.1765133Z +++ declare -f -t trap_add 2024-08-20T22:16:23.1773351Z ++ set -ex 2024-08-20T22:16:23.1773924Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:16:23.1774549Z ++ BUILD_TEST_LIBTORCH=0 2024-08-20T22:16:23.1775121Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *rocm* ]] 2024-08-20T22:16:23.1777676Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-20T22:16:23.1831439Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-20T22:16:23.1831874Z + trap_add cleanup_workspace EXIT 2024-08-20T22:16:23.1832392Z + trap_add_cmd=cleanup_workspace 2024-08-20T22:16:23.1833854Z + shift 2024-08-20T22:16:23.1834149Z + for trap_add_name in "$@" 2024-08-20T22:16:23.1839720Z +++ trap -p EXIT 2024-08-20T22:16:23.1842421Z ++ eval 'extract_trap_cmd ' 2024-08-20T22:16:23.1843100Z +++ extract_trap_cmd 2024-08-20T22:16:23.1843500Z +++ printf '%s\n' '' 2024-08-20T22:16:23.1843900Z ++ printf '%s\n' cleanup_workspace 2024-08-20T22:16:23.1846152Z + trap -- ' 2024-08-20T22:16:23.1846502Z cleanup_workspace' EXIT 2024-08-20T22:16:23.1846981Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-20T22:16:23.9050312Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-20T22:16:23.9071923Z + echo 'Environment variables:' 2024-08-20T22:16:23.9072337Z Environment variables: 2024-08-20T22:16:23.9072661Z + env 2024-08-20T22:16:23.9092563Z INSTALLED_DB=yes 2024-08-20T22:16:23.9093154Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-20T22:16:23.9093723Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:16:23.9094283Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-20T22:16:23.9095132Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:16:23.9095882Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:16:23.9096610Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-20T22:16:23.9097284Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:16:23.9097999Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:16:23.9098545Z HOSTNAME=e132a93b3633 2024-08-20T22:16:23.9099486Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_1944508d-7ce3-472d-a817-adf612d79223 2024-08-20T22:16:23.9100376Z GITHUB_ACTION=__self 2024-08-20T22:16:23.9100888Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:16:23.9108276Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-20T22:16:23.9112174Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-20T22:16:23.9112735Z NV_NVTX_VERSION=12.1.105-1 2024-08-20T22:16:23.9114208Z GITHUB_RUN_NUMBER=54565 2024-08-20T22:16:23.9114554Z TEST_CONFIG=dynamo_eager_timm 2024-08-20T22:16:23.9114938Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:16:23.9115416Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:16:23.9115888Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-20T22:16:23.9116377Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-20T22:16:23.9116940Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:16:23.9117338Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-20T22:16:23.9117791Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:16:23.9118279Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:16:23.9118733Z GITHUB_REF_TYPE=tag 2024-08-20T22:16:23.9119060Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:16:23.9119439Z NCCL_VERSION=2.17.1-1 2024-08-20T22:16:23.9119828Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:23.9120267Z XLA_CUDA= 2024-08-20T22:16:23.9120762Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:16:23.9122896Z *** 2024-08-20T22:16:23.9123196Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:16:23.9123609Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:16:23.9123965Z GITHUB_ACTIONS=true 2024-08-20T22:16:23.9124297Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:16:23.9124823Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-20T22:16:23.9125360Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-20T22:16:23.9125838Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:23.9126356Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:16:23.9126838Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:23.9127764Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:16:23.9128523Z UCC_HOME=/usr 2024-08-20T22:16:23.9128861Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-20T22:16:23.9129259Z VERBOSE_TEST_LOGS=False 2024-08-20T22:16:23.9129598Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:16:23.9130065Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-20T22:16:23.9130563Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:16:23.9131183Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-20T22:16:23.9131558Z SHARD_NUMBER=1 2024-08-20T22:16:23.9131867Z GITHUB_REF_PROTECTED=false 2024-08-20T22:16:23.9132215Z HOME=/var/lib/jenkins 2024-08-20T22:16:23.9132572Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:16:23.9133015Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:16:23.9133495Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:16:23.9134059Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-08-20T22:16:23.9134467Z CUDA_VERSION=12.1.1 2024-08-20T22:16:23.9134893Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-20T22:16:23.9135330Z NUM_TEST_SHARDS=2 2024-08-20T22:16:23.9135632Z UCX_HOME=/usr 2024-08-20T22:16:23.9136149Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-20T22:16:23.9137230Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_1944508d-7ce3-472d-a817-adf612d79223 2024-08-20T22:16:23.9138628Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:16:23.9139988Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_1944508d-7ce3-472d-a817-adf612d79223 2024-08-20T22:16:23.9141096Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:16:23.9141753Z GITHUB_EVENT_NAME=push 2024-08-20T22:16:23.9142081Z DASHBOARD_TAG= 2024-08-20T22:16:23.9142386Z GITHUB_RUN_ID=10479310721 2024-08-20T22:16:23.9143218Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-20T22:16:23.9143772Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-20T22:16:23.9144767Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_1944508d-7ce3-472d-a817-adf612d79223 2024-08-20T22:16:23.9145674Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:16:23.9146077Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-20T22:16:23.9146452Z PR_NUMBER= 2024-08-20T22:16:23.9146919Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:16:23.9147250Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:16:23.9147693Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:16:23.9148151Z TERM=vt100 2024-08-20T22:16:23.9148492Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-20T22:16:23.9148905Z INSTALLED_VISION=yes 2024-08-20T22:16:23.9149219Z BRANCH= 2024-08-20T22:16:23.9149632Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:16:23.9150037Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:16:23.9150452Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:16:23.9151214Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:16:23.9151961Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:16:23.9152445Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:16:23.9152906Z REENABLED_ISSUES= 2024-08-20T22:16:23.9153203Z SHLVL=1 2024-08-20T22:16:23.9153463Z MAX_JOBS=14 2024-08-20T22:16:23.9153774Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-20T22:16:23.9154134Z NVARCH=x86_64 2024-08-20T22:16:23.9154428Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:16:23.9154885Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:23.9155414Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:16:23.9155909Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-20T22:16:23.9156582Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:16:23.9157149Z GITHUB_JOB=test 2024-08-20T22:16:23.9157545Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-20T22:16:23.9158098Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:16:23.9158612Z NO_TEST_TIMEOUT=False 2024-08-20T22:16:23.9158934Z TD_DISTRIBUTED=False 2024-08-20T22:16:23.9159314Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-20T22:16:23.9159755Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:16:23.9160177Z NV_NVPROF_VERSION=12.1.105-1 2024-08-20T22:16:23.9160536Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:16:23.9160883Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:16:23.9161239Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:16:23.9162266Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:16:23.9163317Z GITHUB_BASE_REF= 2024-08-20T22:16:23.9163648Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:16:23.9164025Z CI=true 2024-08-20T22:16:23.9164341Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:16:23.9164746Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:16:23.9165120Z JOB_ID=29026298493 2024-08-20T22:16:23.9165432Z INSTALLED_PROTOBUF=yes 2024-08-20T22:16:23.9165754Z GITHUB_HEAD_REF= 2024-08-20T22:16:23.9166058Z GITHUB_ACTION_REF= 2024-08-20T22:16:23.9166498Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:16:23.9166960Z TEST_SHOWLOCALS=False 2024-08-20T22:16:23.9167349Z GITHUB_WORKFLOW=inductor-periodic 2024-08-20T22:16:23.9167752Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:16:23.9168655Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_1944508d-7ce3-472d-a817-adf612d79223 2024-08-20T22:16:23.9169459Z NO_TD=False 2024-08-20T22:16:23.9169755Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:16:23.9170116Z _=/usr/bin/env 2024-08-20T22:16:23.9170602Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-20T22:16:23.9325767Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-08-20T22:16:23.9327048Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-20T22:16:23.9328053Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-08-20T22:16:23.9329060Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-08-20T22:16:23.9329771Z + BUILD_DIR=build 2024-08-20T22:16:23.9330174Z + BUILD_RENAMED_DIR=build_renamed 2024-08-20T22:16:23.9330567Z + BUILD_BIN_DIR=build/bin 2024-08-20T22:16:23.9331100Z + SHARD_NUMBER=1 2024-08-20T22:16:23.9331421Z + NUM_TEST_SHARDS=2 2024-08-20T22:16:23.9331997Z + export VALGRIND=ON 2024-08-20T22:16:23.9332317Z + VALGRIND=ON 2024-08-20T22:16:23.9332900Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *clang9* ]] 2024-08-20T22:16:23.9333435Z + [[ 0 == \1 ]] 2024-08-20T22:16:23.9333775Z + [[ False == \1 ]] 2024-08-20T22:16:23.9334309Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *bazel* ]] 2024-08-20T22:16:23.9335071Z ++ realpath build/custom_test_artifacts 2024-08-20T22:16:23.9357367Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-20T22:16:23.9358319Z + [[ -n '' ]] 2024-08-20T22:16:23.9358697Z + echo 'Environment variables' 2024-08-20T22:16:23.9359079Z Environment variables 2024-08-20T22:16:23.9359390Z + env 2024-08-20T22:16:23.9366086Z INSTALLED_DB=yes 2024-08-20T22:16:23.9366625Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-20T22:16:23.9367191Z NVIDIA_VISIBLE_DEVICES=all 2024-08-20T22:16:23.9367736Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-20T22:16:23.9368587Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:16:23.9369363Z CONTINUE_THROUGH_ERROR=False 2024-08-20T22:16:23.9369897Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-20T22:16:23.9370427Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:16:23.9371378Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-20T22:16:23.9372056Z HOSTNAME=e132a93b3633 2024-08-20T22:16:23.9373182Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_1944508d-7ce3-472d-a817-adf612d79223 2024-08-20T22:16:23.9374082Z GITHUB_ACTION=__self 2024-08-20T22:16:23.9374466Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T22:16:23.9378186Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-20T22:16:23.9382043Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-20T22:16:23.9382562Z NV_NVTX_VERSION=12.1.105-1 2024-08-20T22:16:23.9382921Z GITHUB_RUN_NUMBER=54565 2024-08-20T22:16:23.9383270Z TEST_CONFIG=dynamo_eager_timm 2024-08-20T22:16:23.9383665Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T22:16:23.9384154Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T22:16:23.9384623Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-20T22:16:23.9385072Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-20T22:16:23.9385487Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T22:16:23.9385895Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-20T22:16:23.9386400Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-08-20T22:16:23.9386898Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-20T22:16:23.9387372Z GITHUB_REF_TYPE=tag 2024-08-20T22:16:23.9387704Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T22:16:23.9388086Z NCCL_VERSION=2.17.1-1 2024-08-20T22:16:23.9388528Z BASE_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:23.9388980Z XLA_CUDA= 2024-08-20T22:16:23.9389432Z HUGGING_FACE_HUB_TOKEN=*** 2024-08-20T22:16:23.9389822Z *** 2024-08-20T22:16:23.9390100Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-20T22:16:23.9390503Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T22:16:23.9390871Z GITHUB_ACTIONS=true 2024-08-20T22:16:23.9391210Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:16:23.9391710Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-20T22:16:23.9392259Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-20T22:16:23.9392738Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:23.9393445Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-20T22:16:23.9393926Z GITHUB_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:23.9394871Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/133712 2024-08-20T22:16:23.9395640Z UCC_HOME=/usr 2024-08-20T22:16:23.9396112Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-20T22:16:23.9396512Z VERBOSE_TEST_LOGS=False 2024-08-20T22:16:23.9396865Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-20T22:16:23.9397343Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-20T22:16:23.9397848Z GITHUB_REF=refs/tags/ciflow/inductor/133712 2024-08-20T22:16:23.9398321Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-20T22:16:23.9398705Z SHARD_NUMBER=1 2024-08-20T22:16:23.9399020Z GITHUB_REF_PROTECTED=false 2024-08-20T22:16:23.9399373Z HOME=/var/lib/jenkins 2024-08-20T22:16:23.9399743Z GITHUB_API_URL=https://api.github.com 2024-08-20T22:16:23.9400190Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T22:16:23.9400682Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-20T22:16:23.9401244Z SCCACHE_S3_KEY_PREFIX=inductor-periodic 2024-08-20T22:16:23.9401668Z CUDA_VERSION=12.1.1 2024-08-20T22:16:23.9402105Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-20T22:16:23.9402554Z NUM_TEST_SHARDS=2 2024-08-20T22:16:23.9402856Z UCX_HOME=/usr 2024-08-20T22:16:23.9403389Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-20T22:16:23.9404479Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_1944508d-7ce3-472d-a817-adf612d79223 2024-08-20T22:16:23.9405895Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamo_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-08-20T22:16:23.9407325Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_1944508d-7ce3-472d-a817-adf612d79223 2024-08-20T22:16:23.9408452Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T22:16:23.9409116Z GITHUB_EVENT_NAME=push 2024-08-20T22:16:23.9409448Z DASHBOARD_TAG= 2024-08-20T22:16:23.9409750Z GITHUB_RUN_ID=10479310721 2024-08-20T22:16:23.9410223Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-20T22:16:23.9410772Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-20T22:16:23.9411877Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_1944508d-7ce3-472d-a817-adf612d79223 2024-08-20T22:16:23.9412791Z GITHUB_ACTOR=pytorch-bot[bot] 2024-08-20T22:16:23.9413206Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-20T22:16:23.9413589Z PR_NUMBER= 2024-08-20T22:16:23.9413869Z GITHUB_RUN_ATTEMPT=1 2024-08-20T22:16:23.9414189Z VALGRIND=ON 2024-08-20T22:16:23.9414489Z ANACONDA_PYTHON_VERSION=3.10 2024-08-20T22:16:23.9414936Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T22:16:23.9415391Z TERM=vt100 2024-08-20T22:16:23.9415732Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-20T22:16:23.9416150Z INSTALLED_VISION=yes 2024-08-20T22:16:23.9416466Z BRANCH= 2024-08-20T22:16:23.9416749Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T22:16:23.9417154Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-20T22:16:23.9417580Z CUDA_PATH=/usr/local/cuda 2024-08-20T22:16:23.9418344Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T22:16:23.9419106Z GITHUB_SERVER_URL=https://github.com 2024-08-20T22:16:23.9419599Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-20T22:16:23.9420073Z REENABLED_ISSUES= 2024-08-20T22:16:23.9420366Z SHLVL=1 2024-08-20T22:16:23.9420630Z MAX_JOBS=14 2024-08-20T22:16:23.9420948Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-20T22:16:23.9421308Z NVARCH=x86_64 2024-08-20T22:16:23.9421602Z GITHUB_ACTOR_ID=54816060 2024-08-20T22:16:23.9422065Z GITHUB_WORKFLOW_SHA=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T22:16:23.9422612Z GITHUB_REF_NAME=ciflow/inductor/133712 2024-08-20T22:16:23.9423105Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-20T22:16:23.9423780Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T22:16:23.9424458Z GITHUB_JOB=test 2024-08-20T22:16:23.9424873Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-20T22:16:23.9425426Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-20T22:16:23.9425942Z NO_TEST_TIMEOUT=False 2024-08-20T22:16:23.9426271Z TD_DISTRIBUTED=False 2024-08-20T22:16:23.9426750Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-20T22:16:23.9427192Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T22:16:23.9427630Z NV_NVPROF_VERSION=12.1.105-1 2024-08-20T22:16:23.9428001Z GITHUB_RETENTION_DAYS=90 2024-08-20T22:16:23.9428358Z OPENSSL_DIR=/opt/openssl 2024-08-20T22:16:23.9428711Z GITHUB_ACTION_REPOSITORY= 2024-08-20T22:16:23.9429759Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:16:23.9430829Z GITHUB_BASE_REF= 2024-08-20T22:16:23.9431163Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-20T22:16:23.9431539Z CI=true 2024-08-20T22:16:23.9431867Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-20T22:16:23.9432289Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T22:16:23.9432662Z JOB_ID=29026298493 2024-08-20T22:16:23.9432985Z INSTALLED_PROTOBUF=yes 2024-08-20T22:16:23.9433324Z GITHUB_HEAD_REF= 2024-08-20T22:16:23.9433633Z GITHUB_ACTION_REF= 2024-08-20T22:16:23.9434080Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T22:16:23.9434553Z TEST_SHOWLOCALS=False 2024-08-20T22:16:23.9434953Z GITHUB_WORKFLOW=inductor-periodic 2024-08-20T22:16:23.9435363Z DEBIAN_FRONTEND=noninteractive 2024-08-20T22:16:23.9436286Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_1944508d-7ce3-472d-a817-adf612d79223 2024-08-20T22:16:23.9437104Z NO_TD=False 2024-08-20T22:16:23.9437414Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T22:16:23.9437774Z _=/usr/bin/env 2024-08-20T22:16:23.9438124Z + echo 'Testing pytorch' 2024-08-20T22:16:23.9438474Z Testing pytorch 2024-08-20T22:16:23.9438814Z + export LANG=C.UTF-8 2024-08-20T22:16:23.9439154Z + LANG=C.UTF-8 2024-08-20T22:16:23.9446757Z + PR_NUMBER= 2024-08-20T22:16:23.9447109Z + [[ dynamo_eager_timm == \d\e\f\a\u\l\t ]] 2024-08-20T22:16:23.9447702Z + [[ dynamo_eager_timm == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-20T22:16:23.9448245Z + [[ dynamo_eager_timm == \s\l\o\w ]] 2024-08-20T22:16:23.9448907Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2024-08-20T22:16:23.9449669Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *cuda* ]] 2024-08-20T22:16:23.9450261Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:16:23.9450731Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-20T22:16:23.9451266Z + [[ dynamo_eager_timm == *crossref* ]] 2024-08-20T22:16:23.9451850Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-20T22:16:23.9452487Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *xpu* ]] 2024-08-20T22:16:23.9453131Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:16:23.9453693Z + pip_install --user ninja==1.10.2 2024-08-20T22:16:23.9454237Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-20T22:16:24.3769373Z Collecting ninja==1.10.2 2024-08-20T22:16:24.3944009Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-20T22:16:24.4044755Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-20T22:16:24.9659753Z Installing collected packages: ninja 2024-08-20T22:16:24.9738645Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-20T22:16:24.9739918Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-20T22:16:24.9783192Z Successfully installed ninja-1.10.2 2024-08-20T22:16:25.0571699Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:16:25.0574194Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:16:25.0575805Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:16:25.0576486Z + install_tlparse 2024-08-20T22:16:25.0576890Z + pip_install --user tlparse==0.3.25 2024-08-20T22:16:25.0577453Z + pip install --progress-bar off --user tlparse==0.3.25 2024-08-20T22:16:25.4411537Z Collecting tlparse==0.3.25 2024-08-20T22:16:25.4567057Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB) 2024-08-20T22:16:25.4658374Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-20T22:16:26.0435486Z Installing collected packages: tlparse 2024-08-20T22:16:26.0776878Z Successfully installed tlparse-0.3.25 2024-08-20T22:16:26.1544678Z ++ python -m site --user-base 2024-08-20T22:16:26.1783400Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T22:16:26.1785006Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *asan* ]] 2024-08-20T22:16:26.1785706Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-debug* ]] 2024-08-20T22:16:26.1786373Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-20T22:16:26.1787331Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass' 2024-08-20T22:16:26.1788678Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass 2024-08-20T22:16:26.1789383Z + cd test 2024-08-20T22:16:26.1789914Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-20T22:16:28.5395775Z + [[ dynamo_eager_timm == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-20T22:16:28.5396407Z + [[ dynamo_eager_timm == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-20T22:16:28.5399307Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-20T22:16:28.5399793Z + [[ dynamo_eager_timm == *pr_time_benchmarks* ]] 2024-08-20T22:16:28.5400287Z + [[ dynamo_eager_timm == *dynamo_eager* ]] 2024-08-20T22:16:28.5401026Z + DYNAMO_BENCHMARK_FLAGS+=(--backend eager) 2024-08-20T22:16:28.5401509Z + [[ dynamo_eager_timm == *dynamic* ]] 2024-08-20T22:16:28.5401940Z + [[ dynamo_eager_timm == *cpu* ]] 2024-08-20T22:16:28.5402419Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-20T22:16:28.5431933Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:16:28.5432634Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-bazel-* ]] 2024-08-20T22:16:28.5434690Z + cd test 2024-08-20T22:16:28.5435235Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-20T22:16:29.9912978Z PyTorch built with: 2024-08-20T22:16:29.9913691Z - GCC 9.4 2024-08-20T22:16:29.9914156Z - C++ Version: 201703 2024-08-20T22:16:29.9915198Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:16:29.9916286Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:16:29.9916966Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:16:29.9917503Z - LAPACK is enabled (usually provided by MKL) 2024-08-20T22:16:29.9917991Z - NNPACK is enabled 2024-08-20T22:16:29.9918380Z - CPU capability usage: AVX2 2024-08-20T22:16:29.9918798Z - CUDA Runtime 12.1 2024-08-20T22:16:29.9919324Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2024-08-20T22:16:29.9919941Z - CuDNN 90.1 (built against CUDA 12.4) 2024-08-20T22:16:29.9920391Z - Magma 2.6.1 2024-08-20T22:16:29.9927606Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-20T22:16:29.9934183Z 2024-08-20T22:16:30.3048129Z + cd test 2024-08-20T22:16:30.3048901Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-20T22:16:31.6412477Z ATen/Parallel: 2024-08-20T22:16:31.6412931Z at::get_num_threads() : 8 2024-08-20T22:16:31.6413424Z at::get_num_interop_threads() : 16 2024-08-20T22:16:31.6413874Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T22:16:31.6414277Z omp_get_max_threads() : 8 2024-08-20T22:16:31.6415339Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T22:16:31.6416142Z mkl_get_max_threads() : 8 2024-08-20T22:16:31.6416777Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T22:16:31.6417417Z std::thread::hardware_concurrency() : 16 2024-08-20T22:16:31.6417849Z Environment variables: 2024-08-20T22:16:31.6418203Z OMP_NUM_THREADS : [not set] 2024-08-20T22:16:31.6418579Z MKL_NUM_THREADS : [not set] 2024-08-20T22:16:31.6418980Z ATen parallel backend: OpenMP 2024-08-20T22:16:31.6419233Z 2024-08-20T22:16:31.9158707Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-20T22:16:31.9159350Z + [[ dynamo_eager_timm == *backward* ]] 2024-08-20T22:16:31.9159787Z + [[ dynamo_eager_timm == *xla* ]] 2024-08-20T22:16:31.9160217Z + [[ dynamo_eager_timm == *executorch* ]] 2024-08-20T22:16:31.9160719Z + [[ dynamo_eager_timm == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-20T22:16:31.9161477Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-20T22:16:31.9162088Z + [[ dynamo_eager_timm == distributed ]] 2024-08-20T22:16:31.9162643Z + [[ dynamo_eager_timm == *inductor_distributed* ]] 2024-08-20T22:16:31.9163347Z + [[ dynamo_eager_timm == *inductor-halide* ]] 2024-08-20T22:16:31.9163935Z + [[ dynamo_eager_timm == *inductor-micro-benchmark* ]] 2024-08-20T22:16:31.9164461Z + [[ dynamo_eager_timm == *huggingface* ]] 2024-08-20T22:16:31.9164907Z + [[ dynamo_eager_timm == *timm* ]] 2024-08-20T22:16:31.9165299Z + install_torchvision 2024-08-20T22:16:31.9165648Z + local orig_preload 2024-08-20T22:16:31.9165971Z + local commit 2024-08-20T22:16:31.9166288Z ++ get_pinned_commit vision 2024-08-20T22:16:31.9166675Z ++ cat .github/ci_commit_pins/vision.txt 2024-08-20T22:16:31.9188591Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:16:31.9189052Z + orig_preload= 2024-08-20T22:16:31.9189421Z + '[' -n '' ']' 2024-08-20T22:16:31.9190263Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:16:31.9191744Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:16:32.2535541Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:16:32.2540451Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-dfsmjyxe 2024-08-20T22:16:32.2570818Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-dfsmjyxe 2024-08-20T22:16:33.8203050Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-08-20T22:16:33.8231832Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:16:35.1418330Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:16:35.4536266Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-08-20T22:16:37.9914900Z Preparing metadata (setup.py) ... [?25l- \ | done 2024-08-20T22:16:37.9947956Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.22.4) 2024-08-20T22:16:37.9954722Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+git40ec5f6) 2024-08-20T22:16:37.9958987Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-08-20T22:16:38.0030392Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-08-20T22:16:38.0034950Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-08-20T22:16:38.0038761Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-08-20T22:16:38.0041816Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-08-20T22:16:38.0045551Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.6.1) 2024-08-20T22:16:38.0051446Z Requirement already satisfied: sympy==1.13.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.13.1) 2024-08-20T22:16:38.0065856Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy==1.13.1->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-08-20T22:16:38.0545370Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-08-20T22:16:38.0608143Z Building wheels for collected packages: torchvision 2024-08-20T22:17:54.9981801Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / done 2024-08-20T22:17:55.0016682Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2115751 sha256=87238e1d6ed3159d07545bf772a94e568aa6e0ec13a99fe69dcb20e08165e053 2024-08-20T22:17:55.0018376Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-08-20T22:17:55.0051912Z Successfully built torchvision 2024-08-20T22:17:55.4875670Z Installing collected packages: torchvision 2024-08-20T22:17:55.9032070Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-08-20T22:17:56.0409277Z + '[' -n '' ']' 2024-08-20T22:17:56.0409627Z + id=0 2024-08-20T22:17:56.0412467Z + test_dynamo_benchmark timm_models 0 2024-08-20T22:17:56.0412898Z ++ pwd 2024-08-20T22:17:56.0415929Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:17:56.0416658Z + local suite=timm_models 2024-08-20T22:17:56.0417090Z + shift 2024-08-20T22:17:56.0417455Z + local shard_id=0 2024-08-20T22:17:56.0417902Z + shift 2024-08-20T22:17:56.0418685Z + [[ dynamo_eager_timm == *perf_compare* ]] 2024-08-20T22:17:56.0419305Z + [[ dynamo_eager_timm == *perf* ]] 2024-08-20T22:17:56.0419871Z + [[ dynamo_eager_timm == *cpu* ]] 2024-08-20T22:17:56.0428211Z + [[ dynamo_eager_timm == *aot_inductor* ]] 2024-08-20T22:17:56.0429172Z + test_single_dynamo_benchmark inference timm_models 0 --inference --bfloat16 2024-08-20T22:17:56.0430807Z ++ pwd 2024-08-20T22:17:56.0431295Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:17:56.0431970Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-08-20T22:17:56.0466384Z + local name=inference 2024-08-20T22:17:56.0466892Z + shift 2024-08-20T22:17:56.0467310Z + local suite=timm_models 2024-08-20T22:17:56.0467772Z + shift 2024-08-20T22:17:56.0468118Z + local shard_id=0 2024-08-20T22:17:56.0468423Z + shift 2024-08-20T22:17:56.0468698Z + partition_flags=() 2024-08-20T22:17:56.0469034Z + local partition_flags 2024-08-20T22:17:56.0469447Z + [[ -n 2 ]] 2024-08-20T22:17:56.0469754Z + [[ -n 0 ]] 2024-08-20T22:17:56.0470378Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-08-20T22:17:56.0471060Z + [[ dynamo_eager_timm == *perf_compare* ]] 2024-08-20T22:17:56.0471521Z + [[ dynamo_eager_timm == *perf* ]] 2024-08-20T22:17:56.0471951Z + [[ dynamo_eager_timm == *aot_inductor* ]] 2024-08-20T22:17:56.0472412Z + [[ dynamo_eager_timm == *_avx2* ]] 2024-08-20T22:17:56.0472850Z + [[ dynamo_eager_timm == *_avx512* ]] 2024-08-20T22:17:56.0474466Z + python benchmarks/dynamo/timm_models.py --ci --accuracy --timing --explain --backend eager --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 0 --output /var/lib/jenkins/workspace/test/test-reports/inference_timm_models.csv 2024-08-20T22:18:03.4016382Z 2024-08-20T22:18:03.8036687Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:18:03.8037015Z 2024-08-20T22:18:03.9146877Z model.safetensors: 0% 0.00/95.5M [00:00 will be ignored 2024-08-20T22:26:02.8377702Z pass 2024-08-20T22:26:02.8979158Z TIMING: entire_frame_compile:12.49545 backend_compile:0.00233 total_wall_time:12.49545 2024-08-20T22:26:02.8980337Z STATS: call_* op count: 989 | FakeTensorMode.__torch_dispatch__:5262 | FakeTensor.__torch_dispatch__:3698 2024-08-20T22:26:02.8981253Z Dynamo produced 3 graphs covering 989 ops with 7 graph breaks (5 unique) 2024-08-20T22:26:07.0690767Z 2024-08-20T22:26:08.0116882Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:26:08.0117498Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:26:08.0117939Z cuda train botnet26t_256 2024-08-20T22:26:14.7070168Z pass 2024-08-20T22:26:14.7253474Z TIMING: entire_frame_compile:5.27932 backend_compile:0.00049 total_wall_time:5.27932 2024-08-20T22:26:14.7254967Z STATS: call_* op count: 264 | FakeTensorMode.__torch_dispatch__:4695 | FakeTensor.__torch_dispatch__:36 2024-08-20T22:26:14.7255878Z Dynamo produced 2 graphs covering 264 ops with 6 graph breaks (5 unique) 2024-08-20T22:26:18.7616374Z 2024-08-20T22:26:22.7142071Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:26:22.7142801Z loading model: 0it [00:03, ?it/s] 2024-08-20T22:26:22.7143249Z cuda train cait_m36_384 2024-08-20T22:26:22.7149323Z Traceback (most recent call last): 2024-08-20T22:26:22.7150383Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2507, in validate_model 2024-08-20T22:26:22.7151255Z self.model_iter_fn(model, example_inputs) 2024-08-20T22:26:22.7152178Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 404, in forward_and_backward_pass 2024-08-20T22:26:22.7153210Z pred = mod(*cloned_inputs) 2024-08-20T22:26:22.7154518Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:26:22.7155710Z return self._call_impl(*args, **kwargs) 2024-08-20T22:26:22.7156875Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:26:22.7157848Z return forward_call(*args, **kwargs) 2024-08-20T22:26:22.7159020Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 438, in forward 2024-08-20T22:26:22.7159904Z x = self.forward_features(x) 2024-08-20T22:26:22.7160898Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 423, in forward_features 2024-08-20T22:26:22.7161902Z x = self.blocks(x) 2024-08-20T22:26:22.7163029Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:26:22.7164213Z return self._call_impl(*args, **kwargs) 2024-08-20T22:26:22.7165182Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:26:22.7166009Z return forward_call(*args, **kwargs) 2024-08-20T22:26:22.7166920Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/container.py", line 250, in forward 2024-08-20T22:26:22.7167727Z input = module(input) 2024-08-20T22:26:22.7168638Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:26:22.7169522Z return self._call_impl(*args, **kwargs) 2024-08-20T22:26:22.7170456Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:26:22.7171417Z return forward_call(*args, **kwargs) 2024-08-20T22:26:22.7172266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 199, in forward 2024-08-20T22:26:22.7173138Z x = x + self.drop_path(self.gamma_1 * self.attn(self.norm1(x))) 2024-08-20T22:26:22.7174231Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl 2024-08-20T22:26:22.7175125Z return self._call_impl(*args, **kwargs) 2024-08-20T22:26:22.7176068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl 2024-08-20T22:26:22.7176942Z return forward_call(*args, **kwargs) 2024-08-20T22:26:22.7177795Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 153, in forward 2024-08-20T22:26:22.7178592Z x = (attn @ v).transpose(1, 2).reshape(B, N, C) 2024-08-20T22:26:22.7181850Z torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 21.98 GiB of which 16.44 MiB is free. Process 114891 has 21.95 GiB memory in use. Of the allocated memory 21.21 GiB is allocated by PyTorch, and 430.66 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables) 2024-08-20T22:26:22.7184757Z 2024-08-20T22:26:22.7185053Z The above exception was the direct cause of the following exception: 2024-08-20T22:26:22.7185631Z 2024-08-20T22:26:22.7185778Z Traceback (most recent call last): 2024-08-20T22:26:22.7186429Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4607, in run 2024-08-20T22:26:22.7187075Z ) = runner.load_model( 2024-08-20T22:26:22.7187736Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 328, in load_model 2024-08-20T22:26:22.7188495Z self.validate_model(model, example_inputs) 2024-08-20T22:26:22.7189247Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2509, in validate_model 2024-08-20T22:26:22.7190004Z raise RuntimeError("Eager run failed") from e 2024-08-20T22:26:22.7190485Z RuntimeError: Eager run failed 2024-08-20T22:26:22.7190745Z 2024-08-20T22:26:22.7190865Z eager_fail_to_run 2024-08-20T22:26:26.5891169Z 2024-08-20T22:26:27.5148546Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:26:27.5149181Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:26:27.5149693Z cuda train coat_lite_mini 2024-08-20T22:26:33.5692630Z pass 2024-08-20T22:26:33.5874373Z TIMING: entire_frame_compile:4.74072 backend_compile:0.00044 total_wall_time:4.74072 2024-08-20T22:26:33.5875407Z STATS: call_* op count: 456 | FakeTensorMode.__torch_dispatch__:6700 | FakeTensor.__torch_dispatch__:28 2024-08-20T22:26:33.5876309Z Dynamo produced 2 graphs covering 456 ops with 6 graph breaks (5 unique) 2024-08-20T22:26:37.6758922Z 2024-08-20T22:26:39.3029694Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:26:39.3030178Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:26:39.3030633Z cuda train convit_base 2024-08-20T22:26:39.3346124Z WARNING:common:fp64 golden ref were not generated for convit_base. Setting accuracy check to cosine 2024-08-20T22:26:50.3497646Z pass 2024-08-20T22:26:50.3993551Z TIMING: entire_frame_compile:10.12177 backend_compile:0.00105 total_wall_time:10.12177 2024-08-20T22:26:50.3994753Z STATS: call_* op count: 1142 | FakeTensorMode.__torch_dispatch__:14306 | FakeTensor.__torch_dispatch__:100 2024-08-20T22:26:50.3995786Z Dynamo produced 3 graphs covering 1142 ops with 7 graph breaks (5 unique) 2024-08-20T22:26:54.4843996Z 2024-08-20T22:26:55.3201157Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:26:55.3201739Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:26:55.3202280Z cuda train convmixer_768_32 2024-08-20T22:27:10.9934463Z pass 2024-08-20T22:27:11.0622627Z TIMING: entire_frame_compile:9.58746 backend_compile:0.00093 total_wall_time:9.58746 2024-08-20T22:27:11.0623724Z STATS: call_* op count: 300 | FakeTensorMode.__torch_dispatch__:7264 | FakeTensor.__torch_dispatch__:22 2024-08-20T22:27:11.0624636Z Dynamo produced 2 graphs covering 300 ops with 5 graph breaks (4 unique) 2024-08-20T22:27:15.1168625Z 2024-08-20T22:27:16.8312249Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:27:16.8312858Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:27:16.8314726Z cuda train convnext_base 2024-08-20T22:27:30.2591439Z W0820 22:27:30.258000 3285 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:27:45.0873137Z pass 2024-08-20T22:27:45.1581736Z TIMING: entire_frame_compile:23.37833 backend_compile:0.00451 total_wall_time:23.37833 2024-08-20T22:27:45.1582888Z STATS: call_* op count: 1508 | FakeTensorMode.__torch_dispatch__:7447 | FakeTensor.__torch_dispatch__:6214 2024-08-20T22:27:45.1585307Z Dynamo produced 3 graphs covering 1508 ops with 7 graph breaks (5 unique) 2024-08-20T22:27:49.5388406Z 2024-08-20T22:27:50.3172197Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:27:50.3172692Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:27:50.3173677Z cuda train crossvit_9_240 2024-08-20T22:27:58.7349021Z W0820 22:27:58.734000 3328 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:28:09.5111854Z pass 2024-08-20T22:28:09.5257705Z TIMING: entire_frame_compile:17.56421 backend_compile:0.00324 total_wall_time:17.56421 2024-08-20T22:28:09.5258886Z STATS: call_* op count: 1312 | FakeTensorMode.__torch_dispatch__:8518 | FakeTensor.__torch_dispatch__:4928 2024-08-20T22:28:09.5259808Z Dynamo produced 3 graphs covering 1312 ops with 7 graph breaks (5 unique) 2024-08-20T22:28:13.8465266Z 2024-08-20T22:28:15.0286871Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:28:15.0287433Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:28:15.0289607Z cuda train cspdarknet53 2024-08-20T22:28:28.0728904Z W0820 22:28:28.072000 3371 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:28:38.0279996Z pass 2024-08-20T22:28:38.0568370Z TIMING: entire_frame_compile:20.00584 backend_compile:0.00281 total_wall_time:20.00584 2024-08-20T22:28:38.0569446Z STATS: call_* op count: 950 | FakeTensorMode.__torch_dispatch__:7049 | FakeTensor.__torch_dispatch__:3710 2024-08-20T22:28:38.0570735Z Dynamo produced 3 graphs covering 950 ops with 7 graph breaks (5 unique) 2024-08-20T22:28:42.3762064Z 2024-08-20T22:28:43.9113090Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:28:43.9113698Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:28:43.9114162Z cuda train deit_base_distilled_patch16_224 2024-08-20T22:28:52.1694726Z W0820 22:28:52.168000 3414 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:28:58.5267978Z pass 2024-08-20T22:28:58.5902774Z TIMING: entire_frame_compile:9.84582 backend_compile:0.00208 total_wall_time:9.84582 2024-08-20T22:28:58.5903776Z STATS: call_* op count: 753 | FakeTensorMode.__torch_dispatch__:4200 | FakeTensor.__torch_dispatch__:2882 2024-08-20T22:28:58.5904695Z Dynamo produced 3 graphs covering 753 ops with 7 graph breaks (5 unique) 2024-08-20T22:29:02.7201204Z 2024-08-20T22:29:04.0103565Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:29:04.0104151Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:29:04.0106404Z cuda train dla102 2024-08-20T22:29:21.6234339Z W0820 22:29:21.622000 3457 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:29:37.2384195Z pass 2024-08-20T22:29:37.2687028Z TIMING: entire_frame_compile:29.86062 backend_compile:0.00459 total_wall_time:29.86062 2024-08-20T22:29:37.2690772Z STATS: call_* op count: 1457 | FakeTensorMode.__torch_dispatch__:10779 | FakeTensor.__torch_dispatch__:5746 2024-08-20T22:29:37.2691861Z Dynamo produced 3 graphs covering 1457 ops with 7 graph breaks (5 unique) 2024-08-20T22:29:41.8101531Z 2024-08-20T22:29:43.5290546Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:29:43.5291128Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:29:43.5295280Z cuda train dm_nfnet_f0 2024-08-20T22:29:55.3353443Z pass 2024-08-20T22:29:55.3877622Z TIMING: entire_frame_compile:8.72033 backend_compile:0.00062 total_wall_time:8.72033 2024-08-20T22:29:55.3880058Z STATS: call_* op count: 594 | FakeTensorMode.__torch_dispatch__:6964 | FakeTensor.__torch_dispatch__:20 2024-08-20T22:29:55.3881156Z Dynamo produced 2 graphs covering 594 ops with 6 graph breaks (5 unique) 2024-08-20T22:29:59.4580902Z 2024-08-20T22:30:01.4031447Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:30:01.4032058Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:30:01.4032573Z cuda train dpn107 2024-08-20T22:30:25.0501953Z pass 2024-08-20T22:30:25.1324281Z TIMING: entire_frame_compile:16.93817 backend_compile:0.00139 total_wall_time:16.93817 2024-08-20T22:30:25.1328028Z STATS: call_* op count: 634 | FakeTensorMode.__torch_dispatch__:13964 | FakeTensor.__torch_dispatch__:102 2024-08-20T22:30:25.1329985Z Dynamo produced 2 graphs covering 634 ops with 6 graph breaks (5 unique) 2024-08-20T22:30:29.3239246Z 2024-08-20T22:30:30.2382061Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:30:30.2382914Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:30:30.2383978Z cuda train eca_botnext26ts_256 2024-08-20T22:30:37.0441013Z W0820 22:30:37.043000 3586 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:30:42.3073459Z pass 2024-08-20T22:30:42.3240582Z TIMING: entire_frame_compile:10.6857 backend_compile:0.00146 total_wall_time:10.6857 2024-08-20T22:30:42.3241563Z STATS: call_* op count: 634 | FakeTensorMode.__torch_dispatch__:5011 | FakeTensor.__torch_dispatch__:1972 2024-08-20T22:30:42.3242472Z Dynamo produced 3 graphs covering 634 ops with 7 graph breaks (5 unique) 2024-08-20T22:30:46.4704086Z 2024-08-20T22:30:47.4626425Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:30:47.4626976Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:30:47.4627433Z cuda train eca_halonext26ts 2024-08-20T22:30:54.5105792Z W0820 22:30:54.509000 3629 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:31:00.0402218Z pass 2024-08-20T22:31:00.0553280Z TIMING: entire_frame_compile:11.17458 backend_compile:0.00149 total_wall_time:11.17458 2024-08-20T22:31:00.0554257Z STATS: call_* op count: 666 | FakeTensorMode.__torch_dispatch__:5152 | FakeTensor.__torch_dispatch__:2026 2024-08-20T22:31:00.0555169Z Dynamo produced 3 graphs covering 666 ops with 7 graph breaks (5 unique) 2024-08-20T22:31:04.2180207Z 2024-08-20T22:31:05.0069121Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:31:05.0072916Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:31:05.0073400Z cuda train ese_vovnet19b_dw 2024-08-20T22:31:10.2691584Z W0820 22:31:10.268000 3672 torch/_logging/_internal.py:1058] [7/0] Profiler function will be ignored 2024-08-20T22:31:15.4452359Z pass 2024-08-20T22:31:15.4531330Z TIMING: entire_frame_compile:9.5739 backend_compile:0.00128 total_wall_time:9.5739 2024-08-20T22:31:15.4532615Z STATS: call_* op count: 434 | FakeTensorMode.__torch_dispatch__:2976 | FakeTensor.__torch_dispatch__:1716 2024-08-20T22:31:15.4533538Z Dynamo produced 4 graphs covering 434 ops with 7 graph breaks (5 unique) 2024-08-20T22:31:19.5849655Z 2024-08-20T22:31:20.4369398Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:31:20.4370453Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:31:20.4371327Z cuda train fbnetc_100 2024-08-20T22:31:30.5124198Z W0820 22:31:30.511000 3715 torch/_logging/_internal.py:1058] [7/0] Profiler function will be ignored 2024-08-20T22:31:40.1677811Z pass 2024-08-20T22:31:40.1747532Z TIMING: entire_frame_compile:18.71166 backend_compile:0.0027 total_wall_time:18.71166 2024-08-20T22:31:40.1750200Z STATS: call_* op count: 870 | FakeTensorMode.__torch_dispatch__:6677 | FakeTensor.__torch_dispatch__:3600 2024-08-20T22:31:40.1751227Z Dynamo produced 4 graphs covering 870 ops with 7 graph breaks (5 unique) 2024-08-20T22:31:44.4527814Z 2024-08-20T22:31:45.5143695Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:31:45.5144173Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:31:45.5147241Z cuda train fbnetv3_b 2024-08-20T22:32:01.3201179Z pass 2024-08-20T22:32:01.3338134Z TIMING: entire_frame_compile:14.51955 backend_compile:0.00126 total_wall_time:14.51955 2024-08-20T22:32:01.3340351Z STATS: call_* op count: 459 | FakeTensorMode.__torch_dispatch__:10352 | FakeTensor.__torch_dispatch__:94 2024-08-20T22:32:01.3342412Z Dynamo produced 2 graphs covering 459 ops with 6 graph breaks (5 unique) 2024-08-20T22:32:05.5051738Z 2024-08-20T22:32:06.6703512Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:32:06.6704502Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:32:06.6705063Z cuda train gernet_l 2024-08-20T22:32:17.4112493Z pass 2024-08-20T22:32:17.4343314Z TIMING: entire_frame_compile:8.73897 backend_compile:0.00074 total_wall_time:8.73897 2024-08-20T22:32:17.4344909Z STATS: call_* op count: 250 | FakeTensorMode.__torch_dispatch__:5846 | FakeTensor.__torch_dispatch__:34 2024-08-20T22:32:17.4345830Z Dynamo produced 2 graphs covering 250 ops with 6 graph breaks (5 unique) 2024-08-20T22:32:21.5447455Z 2024-08-20T22:32:22.4743586Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:32:22.4744062Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:32:22.4744515Z cuda train ghostnet_100 2024-08-20T22:32:36.4861684Z pass 2024-08-20T22:32:36.4932143Z TIMING: entire_frame_compile:12.96021 backend_compile:0.00108 total_wall_time:12.96021 2024-08-20T22:32:36.4933311Z STATS: call_* op count: 405 | FakeTensorMode.__torch_dispatch__:9920 | FakeTensor.__torch_dispatch__:64 2024-08-20T22:32:36.4934242Z Dynamo produced 2 graphs covering 405 ops with 6 graph breaks (5 unique) 2024-08-20T22:32:40.6649829Z 2024-08-20T22:32:41.8945748Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:32:41.8946341Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:32:41.8946917Z cuda train gluon_inception_v3 2024-08-20T22:32:57.4959993Z W0820 22:32:57.495000 3887 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:33:11.5827664Z pass 2024-08-20T22:33:11.6036376Z TIMING: entire_frame_compile:27.07549 backend_compile:0.00398 total_wall_time:27.07549 2024-08-20T22:33:11.6037480Z STATS: call_* op count: 1281 | FakeTensorMode.__torch_dispatch__:9679 | FakeTensor.__torch_dispatch__:5154 2024-08-20T22:33:11.6038498Z Dynamo produced 3 graphs covering 1281 ops with 7 graph breaks (5 unique) 2024-08-20T22:33:16.0507619Z 2024-08-20T22:33:16.9428441Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:33:16.9428995Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:33:16.9431154Z cuda train gmixer_24_224 2024-08-20T22:33:26.3563221Z pass 2024-08-20T22:33:26.3760373Z TIMING: entire_frame_compile:7.15675 backend_compile:0.00072 total_wall_time:7.15675 2024-08-20T22:33:26.3761613Z STATS: call_* op count: 587 | FakeTensorMode.__torch_dispatch__:8422 | FakeTensor.__torch_dispatch__:118 2024-08-20T22:33:26.3762671Z Dynamo produced 2 graphs covering 587 ops with 6 graph breaks (5 unique) 2024-08-20T22:33:30.4444845Z 2024-08-20T22:33:31.2885530Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:33:31.2886030Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:33:31.2886472Z cuda train gmlp_s16_224 2024-08-20T22:33:40.5510101Z W0820 22:33:40.550000 3973 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:33:52.5923866Z pass 2024-08-20T22:33:52.6098652Z TIMING: entire_frame_compile:18.89476 backend_compile:0.00372 total_wall_time:18.89476 2024-08-20T22:33:52.6099657Z STATS: call_* op count: 1396 | FakeTensorMode.__torch_dispatch__:7797 | FakeTensor.__torch_dispatch__:5590 2024-08-20T22:33:52.6100574Z Dynamo produced 3 graphs covering 1396 ops with 7 graph breaks (5 unique) 2024-08-20T22:33:56.9329046Z 2024-08-20T22:33:58.5825991Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:33:58.5826516Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:33:58.5826952Z cuda train hrnet_w18 2024-08-20T22:34:50.2894363Z pass 2024-08-20T22:34:50.3325152Z TIMING: entire_frame_compile:46.7739 backend_compile:0.00478 total_wall_time:46.7739 2024-08-20T22:34:50.3328640Z STATS: call_* op count: 1475 | FakeTensorMode.__torch_dispatch__:33913 | FakeTensor.__torch_dispatch__:42 2024-08-20T22:34:50.3329557Z Dynamo produced 2 graphs covering 1475 ops with 5 graph breaks (4 unique) 2024-08-20T22:34:55.0036384Z 2024-08-20T22:34:56.2298025Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:34:56.2299045Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:34:56.2299498Z cuda train inception_v3 2024-08-20T22:35:11.9324403Z pass 2024-08-20T22:35:11.9576050Z TIMING: entire_frame_compile:13.26191 backend_compile:0.00116 total_wall_time:13.26191 2024-08-20T22:35:11.9578503Z STATS: call_* op count: 412 | FakeTensorMode.__torch_dispatch__:9662 | FakeTensor.__torch_dispatch__:42 2024-08-20T22:35:11.9580296Z Dynamo produced 2 graphs covering 412 ops with 6 graph breaks (5 unique) 2024-08-20T22:35:16.0842201Z 2024-08-20T22:35:17.6082529Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:35:17.6083015Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:35:17.6085833Z cuda train jx_nest_base 2024-08-20T22:35:32.0785377Z W0820 22:35:32.077000 4102 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:35:44.2732500Z pass 2024-08-20T22:35:44.3389822Z TIMING: entire_frame_compile:20.84221 backend_compile:0.00384 total_wall_time:20.84221 2024-08-20T22:35:44.3391793Z STATS: call_* op count: 1671 | FakeTensorMode.__torch_dispatch__:12027 | FakeTensor.__torch_dispatch__:5513 2024-08-20T22:35:44.3393314Z Dynamo produced 3 graphs covering 1671 ops with 7 graph breaks (5 unique) 2024-08-20T22:35:48.7051873Z 2024-08-20T22:35:49.4101496Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:35:49.4102059Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:35:49.4102500Z cuda train lcnet_050 2024-08-20T22:35:54.0001575Z pass 2024-08-20T22:35:54.0020784Z TIMING: entire_frame_compile:4.1281 backend_compile:0.00042 total_wall_time:4.1281 2024-08-20T22:35:54.0022005Z STATS: call_* op count: 129 | FakeTensorMode.__torch_dispatch__:3148 | FakeTensor.__torch_dispatch__:54 2024-08-20T22:35:54.0022978Z Dynamo produced 2 graphs covering 129 ops with 6 graph breaks (5 unique) 2024-08-20T22:35:58.0494902Z 2024-08-20T22:35:59.0630519Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:35:59.0631097Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:35:59.0632747Z cuda train levit_128 2024-08-20T22:36:09.9766568Z W0820 22:36:09.975000 4188 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:36:18.5584077Z pass 2024-08-20T22:36:18.5631449Z TIMING: entire_frame_compile:18.29037 backend_compile:0.00296 total_wall_time:18.29037 2024-08-20T22:36:18.5632434Z STATS: call_* op count: 1261 | FakeTensorMode.__torch_dispatch__:11144 | FakeTensor.__torch_dispatch__:3784 2024-08-20T22:36:18.5633355Z Dynamo produced 3 graphs covering 1261 ops with 7 graph breaks (5 unique) 2024-08-20T22:36:22.9125757Z 2024-08-20T22:36:24.0967879Z loading model: 0it [00:00, ?it/s] 2024-08-20T22:36:24.0968345Z loading model: 0it [00:01, ?it/s] 2024-08-20T22:36:24.0970986Z cuda train mixer_b16_224 2024-08-20T22:36:30.7152675Z W0820 22:36:30.714000 4231 torch/_logging/_internal.py:1058] [6/0] Profiler function will be ignored 2024-08-20T22:36:36.7582697Z pass 2024-08-20T22:36:36.8120341Z TIMING: entire_frame_compile:9.01942 backend_compile:0.00188 total_wall_time:9.01942 2024-08-20T22:36:36.8121569Z STATS: call_* op count: 670 | FakeTensorMode.__torch_dispatch__:3713 | FakeTensor.__torch_dispatch__:2770 2024-08-20T22:36:36.8122512Z Dynamo produced 3 graphs covering 670 ops with 7 graph breaks (5 unique) 2024-08-20T22:36:37.8019525Z accuracy pass_rate=96.67% 2024-08-20T22:36:37.8022909Z calls_captured gmean=0.00x mean=779.033x 2024-08-20T22:36:37.8026460Z unique_graphs gmean=0.00x mean=2.533x 2024-08-20T22:36:37.8030105Z graph_breaks gmean=0.00x mean=6.267x 2024-08-20T22:36:37.8034334Z unique_graph_breaks gmean=0.00x mean=4.767x 2024-08-20T22:36:37.8038106Z autograd_captures gmean=0.00x mean=0.000x 2024-08-20T22:36:37.8041827Z autograd_compiles gmean=0.00x mean=0.000x 2024-08-20T22:36:37.8045887Z cudagraph_skips gmean=0.00x mean=0.000x 2024-08-20T22:36:38.4501841Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy/dynamo_eager_timm_training.csv 2024-08-20T22:36:38.7142198Z adv_inception_v3 PASS 2024-08-20T22:36:38.7147848Z beit_base_patch16_224 PASS 2024-08-20T22:36:38.7152856Z botnet26t_256 PASS 2024-08-20T22:36:38.7157191Z cait_m36_384 XFAIL 2024-08-20T22:36:38.7161913Z coat_lite_mini PASS 2024-08-20T22:36:38.7166481Z convit_base PASS 2024-08-20T22:36:38.7171257Z convmixer_768_32 PASS 2024-08-20T22:36:38.7175752Z convnext_base PASS 2024-08-20T22:36:38.7180050Z crossvit_9_240 PASS 2024-08-20T22:36:38.7184620Z cspdarknet53 PASS 2024-08-20T22:36:38.7189213Z deit_base_distilled_patch16_224 PASS 2024-08-20T22:36:38.7193539Z dla102 PASS 2024-08-20T22:36:38.7198203Z dm_nfnet_f0 PASS 2024-08-20T22:36:38.7202675Z dpn107 PASS 2024-08-20T22:36:38.7207240Z eca_botnext26ts_256 PASS 2024-08-20T22:36:38.7211884Z eca_halonext26ts PASS 2024-08-20T22:36:38.7216584Z ese_vovnet19b_dw PASS 2024-08-20T22:36:38.7220959Z fbnetc_100 PASS 2024-08-20T22:36:38.7225547Z fbnetv3_b PASS 2024-08-20T22:36:38.7229928Z gernet_l PASS 2024-08-20T22:36:38.7234582Z ghostnet_100 PASS 2024-08-20T22:36:38.7239105Z gluon_inception_v3 PASS 2024-08-20T22:36:38.7244042Z gmixer_24_224 PASS 2024-08-20T22:36:38.7248379Z gmlp_s16_224 PASS 2024-08-20T22:36:38.7253144Z hrnet_w18 PASS 2024-08-20T22:36:38.7257680Z inception_v3 PASS 2024-08-20T22:36:38.7262478Z jx_nest_base PASS 2024-08-20T22:36:38.7266810Z lcnet_050 PASS 2024-08-20T22:36:38.7271321Z levit_128 PASS 2024-08-20T22:36:38.7275883Z mixer_b16_224 PASS 2024-08-20T22:36:38.7723101Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy/dynamo_eager_timm_training.csv 2024-08-20T22:36:39.0362429Z adv_inception_v3 PASS 2024-08-20T22:36:39.0366478Z beit_base_patch16_224 PASS 2024-08-20T22:36:39.0371165Z botnet26t_256 PASS 2024-08-20T22:36:39.0375843Z cait_m36_384 PASS 2024-08-20T22:36:39.0380346Z coat_lite_mini PASS 2024-08-20T22:36:39.0385011Z convit_base PASS 2024-08-20T22:36:39.0389533Z convmixer_768_32 PASS 2024-08-20T22:36:39.0394242Z convnext_base PASS 2024-08-20T22:36:39.0398746Z crossvit_9_240 PASS 2024-08-20T22:36:39.0403305Z cspdarknet53 PASS 2024-08-20T22:36:39.0407887Z deit_base_distilled_patch16_224 PASS 2024-08-20T22:36:39.0412570Z dla102 PASS 2024-08-20T22:36:39.0417174Z dm_nfnet_f0 PASS 2024-08-20T22:36:39.0421714Z dpn107 PASS 2024-08-20T22:36:39.0426346Z eca_botnext26ts_256 PASS 2024-08-20T22:36:39.0430874Z eca_halonext26ts PASS 2024-08-20T22:36:39.0435615Z ese_vovnet19b_dw PASS 2024-08-20T22:36:39.0440160Z fbnetc_100 PASS 2024-08-20T22:36:39.0444977Z fbnetv3_b PASS 2024-08-20T22:36:39.0450153Z gernet_l PASS 2024-08-20T22:36:39.0455242Z ghostnet_100 PASS 2024-08-20T22:36:39.0459589Z gluon_inception_v3 PASS 2024-08-20T22:36:39.0464136Z gmixer_24_224 PASS 2024-08-20T22:36:39.0468707Z gmlp_s16_224 PASS 2024-08-20T22:36:39.0473282Z hrnet_w18 PASS 2024-08-20T22:36:39.0477864Z inception_v3 PASS 2024-08-20T22:36:39.0482374Z jx_nest_base PASS 2024-08-20T22:36:39.0487063Z lcnet_050 PASS 2024-08-20T22:36:39.0491668Z levit_128 PASS 2024-08-20T22:36:39.0496264Z mixer_b16_224 PASS 2024-08-20T22:36:39.0928482Z + cleanup_workspace 2024-08-20T22:36:39.0929638Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-20T22:36:39.0930763Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-20T22:36:39.0932016Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-20T22:36:39.0932818Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-20T22:36:39.0933697Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-20T22:36:39.0934686Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-20T22:36:39.0935452Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-20T22:36:39.8300448Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-20T22:36:39.8301019Z cat test/**/*_toprint.log || true 2024-08-20T22:36:39.8314481Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:36:39.8314993Z env: 2024-08-20T22:36:39.8315278Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:36:39.8315732Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:36:39.8316463Z DOCKER_CONTAINER_ID: e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:39.8317098Z ##[endgroup] 2024-08-20T22:36:39.8411915Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-20T22:36:39.8445264Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-20T22:36:39.8445740Z kill "$MONITOR_SCRIPT_PID" 2024-08-20T22:36:39.8454079Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:36:39.8454576Z env: 2024-08-20T22:36:39.8454848Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:36:39.8455309Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:36:39.8456027Z DOCKER_CONTAINER_ID: e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:39.8456668Z MONITOR_SCRIPT_PID: 110993 2024-08-20T22:36:39.8457018Z ##[endgroup] 2024-08-20T22:36:39.8654905Z Prepare all required actions 2024-08-20T22:36:39.8655363Z Getting action download info 2024-08-20T22:36:40.0618585Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-20T22:36:40.2043080Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-20T22:36:40.2043555Z with: 2024-08-20T22:36:40.2044069Z file-suffix: test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026298493 2024-08-20T22:36:40.2044726Z s3-bucket: gha-artifacts 2024-08-20T22:36:40.2045072Z env: 2024-08-20T22:36:40.2045341Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:36:40.2045796Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:36:40.2046520Z DOCKER_CONTAINER_ID: e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:40.2047145Z ##[endgroup] 2024-08-20T22:36:40.2073727Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-20T22:36:40.2074371Z # Remove any previous test jsons if they exist 2024-08-20T22:36:40.2074883Z rm -f test-jsons-*.zip 2024-08-20T22:36:40.2075404Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-20T22:36:40.2084339Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:36:40.2084879Z env: 2024-08-20T22:36:40.2085171Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:36:40.2085650Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:36:40.2086548Z DOCKER_CONTAINER_ID: e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:40.2087410Z FILE_SUFFIX: test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026298493 2024-08-20T22:36:40.2088035Z ##[endgroup] 2024-08-20T22:36:40.2426851Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-20T22:36:40.2458400Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-20T22:36:40.2459097Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-20T22:36:40.2465267Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-20T22:36:40.2470901Z adding: test/slow_tests.json (deflated 82%) 2024-08-20T22:36:40.2503081Z ##[group]Run # Remove any previous test reports if they exist 2024-08-20T22:36:40.2503729Z # Remove any previous test reports if they exist 2024-08-20T22:36:40.2504260Z rm -f test-reports-*.zip 2024-08-20T22:36:40.2504854Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-20T22:36:40.2513702Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:36:40.2514212Z env: 2024-08-20T22:36:40.2514500Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:36:40.2514961Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:36:40.2515681Z DOCKER_CONTAINER_ID: e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:40.2516545Z FILE_SUFFIX: test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026298493 2024-08-20T22:36:40.2517177Z ##[endgroup] 2024-08-20T22:36:40.2776410Z adding: test/test-reports/inference_timm_models.csv (deflated 68%) 2024-08-20T22:36:40.2777323Z adding: test/test-reports/training_timm_models.csv (deflated 66%) 2024-08-20T22:36:40.2781175Z adding: test/test-reports/training_timm_model_graph_breaks.csv (deflated 97%) 2024-08-20T22:36:40.2782291Z adding: test/test-reports/training_timm_model_graph_break_deduped.csv (deflated 73%) 2024-08-20T22:36:40.2814219Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-20T22:36:40.2814855Z # Remove any previous usage logs if they exist 2024-08-20T22:36:40.2815379Z rm -f logs-*.zip 2024-08-20T22:36:40.2816058Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-20T22:36:40.2816857Z # so check to see if the file exists first 2024-08-20T22:36:40.2817366Z if [ -f 'usage_log.txt' ]; then 2024-08-20T22:36:40.2818115Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-20T22:36:40.2818623Z fi 2024-08-20T22:36:40.2819003Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-20T22:36:40.2819585Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-20T22:36:40.2820096Z fi 2024-08-20T22:36:40.2828795Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:36:40.2829299Z env: 2024-08-20T22:36:40.2829576Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:36:40.2830046Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:36:40.2830771Z DOCKER_CONTAINER_ID: e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:40.2831644Z FILE_SUFFIX: test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026298493 2024-08-20T22:36:40.2832267Z ##[endgroup] 2024-08-20T22:36:40.2894407Z adding: usage_log.txt (deflated 92%) 2024-08-20T22:36:40.2982389Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-20T22:36:40.2983113Z # Remove any previous debugging artifacts if they exist 2024-08-20T22:36:40.2983670Z rm -f debug-*.zip 2024-08-20T22:36:40.2984068Z if [ -d 'test/debug' ]; then 2024-08-20T22:36:40.2984578Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-20T22:36:40.2985051Z fi 2024-08-20T22:36:40.2993193Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:36:40.2993699Z env: 2024-08-20T22:36:40.2994141Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:36:40.2994596Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:36:40.2995316Z DOCKER_CONTAINER_ID: e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:40.2996180Z FILE_SUFFIX: test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026298493 2024-08-20T22:36:40.2996801Z ##[endgroup] 2024-08-20T22:36:40.3092314Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:36:40.3092756Z with: 2024-08-20T22:36:40.3093051Z s3-bucket: gha-artifacts 2024-08-20T22:36:40.3093505Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T22:36:40.3093994Z retention-days: 14 2024-08-20T22:36:40.3094335Z if-no-files-found: warn 2024-08-20T22:36:40.3094705Z path: test-jsons-*.zip 2024-08-20T22:36:40.3095059Z name: artifact 2024-08-20T22:36:40.3095369Z region: us-east-1 2024-08-20T22:36:40.3095679Z env: 2024-08-20T22:36:40.3095965Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:36:40.3096438Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:36:40.3097163Z DOCKER_CONTAINER_ID: e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:40.3097798Z ##[endgroup] 2024-08-20T22:36:40.6645984Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T22:36:40.6646664Z With the provided path, there will be 1 file uploaded 2024-08-20T22:36:40.6647375Z Uploading to s3 prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T22:36:40.9243963Z Starting upload of test-jsons-test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026298493.zip 2024-08-20T22:36:41.0266174Z Finished upload of test-jsons-test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026298493.zip 2024-08-20T22:36:41.0552294Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:36:41.0552755Z with: 2024-08-20T22:36:41.0553042Z s3-bucket: gha-artifacts 2024-08-20T22:36:41.0553491Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T22:36:41.0553974Z retention-days: 14 2024-08-20T22:36:41.0554344Z if-no-files-found: error 2024-08-20T22:36:41.0554719Z path: test-reports-*.zip 2024-08-20T22:36:41.0555085Z name: artifact 2024-08-20T22:36:41.0555405Z region: us-east-1 2024-08-20T22:36:41.0555713Z env: 2024-08-20T22:36:41.0555998Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:36:41.0556458Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:36:41.0557191Z DOCKER_CONTAINER_ID: e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:41.0557821Z ##[endgroup] 2024-08-20T22:36:41.4136981Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T22:36:41.4137633Z With the provided path, there will be 1 file uploaded 2024-08-20T22:36:41.4138262Z Uploading to s3 prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T22:36:41.4195537Z Starting upload of test-reports-test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026298493.zip 2024-08-20T22:36:41.5902658Z Finished upload of test-reports-test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026298493.zip 2024-08-20T22:36:41.6192133Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:36:41.6192591Z with: 2024-08-20T22:36:41.6192875Z s3-bucket: gha-artifacts 2024-08-20T22:36:41.6193318Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T22:36:41.6193801Z retention-days: 14 2024-08-20T22:36:41.6194151Z if-no-files-found: ignore 2024-08-20T22:36:41.6194511Z path: logs-*.zip 2024-08-20T22:36:41.6194829Z name: artifact 2024-08-20T22:36:41.6195140Z region: us-east-1 2024-08-20T22:36:41.6195447Z env: 2024-08-20T22:36:41.6195743Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:36:41.6196203Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:36:41.6196928Z DOCKER_CONTAINER_ID: e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:41.6197558Z ##[endgroup] 2024-08-20T22:36:41.9823610Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T22:36:41.9824284Z With the provided path, there will be 1 file uploaded 2024-08-20T22:36:41.9825189Z Uploading to s3 prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T22:36:41.9881541Z Starting upload of logs-test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026298493.zip 2024-08-20T22:36:42.1210636Z Finished upload of logs-test-dynamo_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_29026298493.zip 2024-08-20T22:36:42.1497901Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T22:36:42.1498357Z with: 2024-08-20T22:36:42.1498644Z s3-bucket: gha-artifacts 2024-08-20T22:36:42.1499110Z s3-prefix: pytorch/pytorch/10479310721/1/artifact 2024-08-20T22:36:42.1499597Z retention-days: 14 2024-08-20T22:36:42.1500329Z if-no-files-found: ignore 2024-08-20T22:36:42.1500700Z path: debug-*.zip 2024-08-20T22:36:42.1501025Z name: artifact 2024-08-20T22:36:42.1501336Z region: us-east-1 2024-08-20T22:36:42.1501645Z env: 2024-08-20T22:36:42.1501926Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:36:42.1502400Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:36:42.1503145Z DOCKER_CONTAINER_ID: e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:42.1503780Z ##[endgroup] 2024-08-20T22:36:42.5064458Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-20T22:36:42.5351421Z ##[group]Run # shellcheck disable=SC2156 2024-08-20T22:36:42.5351912Z # shellcheck disable=SC2156 2024-08-20T22:36:42.5352703Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-20T22:36:42.5361889Z shell: /usr/bin/bash -e {0} 2024-08-20T22:36:42.5362242Z env: 2024-08-20T22:36:42.5362534Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:36:42.5362995Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:36:42.5363721Z DOCKER_CONTAINER_ID: e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:42.5364345Z ##[endgroup] 2024-08-20T22:36:42.7977889Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-20T22:36:42.7978459Z with: 2024-08-20T22:36:42.7978726Z env: 2024-08-20T22:36:42.7979004Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:36:42.7979463Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:36:42.7980183Z DOCKER_CONTAINER_ID: e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:42.7980810Z ##[endgroup] 2024-08-20T22:36:42.8000218Z ##[group]Run set -eou pipefail 2024-08-20T22:36:42.8000638Z set -eou pipefail 2024-08-20T22:36:42.8000990Z  2024-08-20T22:36:42.8001501Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-20T22:36:42.8002131Z for _ in $(seq 1440); do 2024-08-20T22:36:42.8002609Z  # Break if no ssh session exists anymore 2024-08-20T22:36:42.8003107Z  if [ "$(who)" = "" ]; then 2024-08-20T22:36:42.8003520Z  break 2024-08-20T22:36:42.8003862Z  fi 2024-08-20T22:36:42.8004170Z  echo "." 2024-08-20T22:36:42.8004503Z  sleep 5 2024-08-20T22:36:42.8004821Z done 2024-08-20T22:36:42.8013668Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:36:42.8014175Z env: 2024-08-20T22:36:42.8014459Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:36:42.8014920Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:36:42.8015639Z DOCKER_CONTAINER_ID: e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:42.8016267Z ##[endgroup] 2024-08-20T22:36:42.8048683Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-20T22:36:42.8121804Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:36:42.8122588Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T22:36:42.8123188Z # shellcheck disable=SC2046 2024-08-20T22:36:42.8123671Z docker stop $(docker ps -q) || true 2024-08-20T22:36:42.8124321Z # Prune all of the docker images 2024-08-20T22:36:42.8124781Z docker system prune -af 2024-08-20T22:36:42.8133719Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:36:42.8134223Z env: 2024-08-20T22:36:42.8134508Z GIT_DEFAULT_BRANCH: main 2024-08-20T22:36:42.8134973Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-20T22:36:42.8135705Z DOCKER_CONTAINER_ID: e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:42.8136333Z ##[endgroup] 2024-08-20T22:36:43.4703305Z e132a93b3633 2024-08-20T22:36:44.9322930Z Deleted Containers: 2024-08-20T22:36:44.9323480Z e132a93b363309c0a4c6925986906eb75c947719f74af8178fa92d1f9ff8c703 2024-08-20T22:36:44.9323901Z 2024-08-20T22:36:54.1862251Z Deleted Images: 2024-08-20T22:36:54.1864144Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T22:36:54.1867025Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks@sha256:3a95d3a177465127ef6cd3b50787d6a050367070fd9063dbb688fad07a96b402 2024-08-20T22:36:54.1868951Z deleted: sha256:8fef7805cec6e829013a3d62f158b75bdc2f02cdfe3428c032ea017b488473bb 2024-08-20T22:36:54.1869861Z deleted: sha256:c2ce10014dd98f6abbfbf2265f9780a8204c2a7a27463e61ee565573a23d141d 2024-08-20T22:36:54.1870720Z deleted: sha256:ae249c0e6e6013bd6ffea2ca0b0160c015c9ebf7c03070bc2f983a9ff2415bf5 2024-08-20T22:36:54.1871547Z deleted: sha256:8a73883c1067c0e29ff128f553518682b47ed9707d2f35e7d5138e2caf029d61 2024-08-20T22:36:54.1872362Z deleted: sha256:9c040db729fa084584aaf5573082408a2196bc64e5952270c79dc610ed4680b6 2024-08-20T22:36:54.1873204Z deleted: sha256:17311eeeb8b71b16699880c0f646e185107496c4d41da5d23370603768ec030c 2024-08-20T22:36:54.1874030Z deleted: sha256:07cc10bece8501b1c8035d4385ad9e9867dd6e92fa32218a4dcb3e42af7e0411 2024-08-20T22:36:54.1875217Z deleted: sha256:f3e32d05099d1a3115a4722bc2e94a48752c3e965446773673808252671bb967 2024-08-20T22:36:54.1876050Z deleted: sha256:be7c3f41475e3e72ffe3ac2f92ab2f4cf49d5496b2aae738942ac3bb687d742c 2024-08-20T22:36:54.1876895Z deleted: sha256:4fcac490be825129b468cf458089f1fcf8fd56afc44ef4ab27c2eff41ab1ac90 2024-08-20T22:36:54.1877729Z deleted: sha256:31c66d80a2a4792897f267d363c0cf84fb821c5d4a275ab1636f778fd0284c08 2024-08-20T22:36:54.1878547Z deleted: sha256:8a8bcb6085e7e438e5533e8a5613e2ae5888543853568d023af3e2623a1593a9 2024-08-20T22:36:54.1879367Z deleted: sha256:2f6cb60029c79d11567b738f5bdbfb9cb1838cc0f194c4a5e6ae685178d2ed34 2024-08-20T22:36:54.1880188Z deleted: sha256:e546391804d3313f558e1ac16e93cb29a417a92e4eccaf0d4bd12af3bbcf39e3 2024-08-20T22:36:54.1881011Z deleted: sha256:1623b2aa16f2bdc6b572f6799498d9a92fa8114d07c440599b17aea72e22f8b1 2024-08-20T22:36:54.1881845Z deleted: sha256:c0386ca58b7cb7afee015b7e58d2f70525eaf73d6da2cc63868b20e14b1f9497 2024-08-20T22:36:54.1882678Z deleted: sha256:dcbf534ff289186e246531925fed90f72516eb3d63663bad234d611cdc025d62 2024-08-20T22:36:54.1883505Z deleted: sha256:0bab537af579f56c9cd51c7073fd53635fffb424b363236e0de38a7dd0e60ae8 2024-08-20T22:36:54.1884328Z deleted: sha256:d85a852ae1ee027e2fc17240ed68b87957b0a6d210c9119752d8979a0dcd55cf 2024-08-20T22:36:54.1885147Z deleted: sha256:f3821d761b555362e44cf9a1447d8fd2d95b37d54aea67c7519cd9e5db0c4d7e 2024-08-20T22:36:54.1885972Z deleted: sha256:c05e84a545db5dfcb0847c73d2b0f4749fa67aabce05705632ff7f76c84d4688 2024-08-20T22:36:54.1886798Z deleted: sha256:27054eae7ca67b2bd5ca0fd3f4488bf5c5f78e95487d8c45616f3d753d48a8b5 2024-08-20T22:36:54.1887626Z deleted: sha256:9804139f6ae97796aaa993fe4b89ea0cf111eb4d30a79f4c87c339c2e1e04fa0 2024-08-20T22:36:54.1888444Z deleted: sha256:ad6ae2fc594aaa359ff4e287323836570454eb24dd162930655e414d1d77c6dd 2024-08-20T22:36:54.1889361Z deleted: sha256:9cfbb36e328af52a248853a46aab88e266c09e2b44a03e0a42194a031b9b9453 2024-08-20T22:36:54.1890327Z deleted: sha256:a8a842bd7f37b8d50b6d12d6f95f46bbba57953db466fcbde9ba522625f024ef 2024-08-20T22:36:54.1891597Z deleted: sha256:da9738654b3c86f89ebeed2560f1bae093f1219f1eff90b0394b7f211ecabd04 2024-08-20T22:36:54.1892444Z deleted: sha256:cc683bf0cb245394e0f568984b2dd4fce1b6a3a7fe7c44734bae1aa1caaa3251 2024-08-20T22:36:54.1893283Z deleted: sha256:965f220e71c7d1ec19aaf41b1529ab287beb9d86eb8482c4c14094485ab34475 2024-08-20T22:36:54.1894104Z deleted: sha256:da32ecec41a73ce7dd29464f90c7951510a18cd378753659a78cd7fabd10dc36 2024-08-20T22:36:54.1894933Z deleted: sha256:d828d187cee9ad48dbc3702fb61d359619dd8a1ed582878680b51686bf117e5a 2024-08-20T22:36:54.1895757Z deleted: sha256:712111f4b12ff8e2787597d5fdcb043bd0824e55ddd97f262ef550506ca195f0 2024-08-20T22:36:54.1896577Z deleted: sha256:a064eea405958ad14dc6c794e5df9fd1d4515424272801ec559d45a59317309a 2024-08-20T22:36:54.1897401Z deleted: sha256:e124382d136bd5e9a8b409ca79c1f1ef76458deaf444b1b2060d3fa99dbeb506 2024-08-20T22:36:54.1898226Z deleted: sha256:6e10fe757cc190a1b31b47010f4f7610e02cb1b4b438f190dcd301aa4fe2278e 2024-08-20T22:36:54.1899046Z deleted: sha256:4cea352e10f46ca4729f96ac6c4600fc53d4c36e7658c2e0f19f4f309cd4a2a7 2024-08-20T22:36:54.1899876Z deleted: sha256:518bba00fe1d0933053066d21cac0235c9fb485abbc41557ec184b293095c924 2024-08-20T22:36:54.1900697Z deleted: sha256:a64f9ca42e9f27ff2690c15a5c84312e6efaccad45668d9b496cb542165cfb5a 2024-08-20T22:36:54.1901518Z deleted: sha256:096ea0fa14a6c420658f5276b8a8f5d264c0de7e18161ceda2cfc2f4a92cd9bf 2024-08-20T22:36:54.1902342Z deleted: sha256:6f91c94ce6e3f8034c91c66ee25c66a3d85dba9d813b749bd6d2000942a99edc 2024-08-20T22:36:54.1903178Z deleted: sha256:8f8d255eeae546cead8f9a94d237fa57a99940c0f5d51bc29a9d41c7df9cffd9 2024-08-20T22:36:54.1904021Z deleted: sha256:fe05276f4a49f91eef56bfbc3da87cfff5eb116a827fa2d30efe42ce1c134b40 2024-08-20T22:36:54.1904900Z deleted: sha256:5f83cd75d0669c813d4f58978c92e14305cd8f6a07eb9a60665a583460fa7b59 2024-08-20T22:36:54.1905704Z deleted: sha256:8f7ab6366169d39d8953129f650f1927d2fb1328b3f11e2fd9076adb5f46b93a 2024-08-20T22:36:54.1906652Z deleted: sha256:4146cfd5dba0f5308954749f98596b14a1d8f6d23bd93f7ad86c6c3d72239328 2024-08-20T22:36:54.1907496Z deleted: sha256:c6d83cb9ccc606ff63bb3c4d5a5e1ff1450c619b5aae494303a94ae25971cca0 2024-08-20T22:36:54.1908316Z deleted: sha256:71e037e0a3e0963c4565534a7387c082afcb04475d3ac629acf7a0ab2f49fadc 2024-08-20T22:36:54.1909118Z deleted: sha256:4b31135c564686bdaf23262b901237880aea635655721cc7e8c95dc42b05e06d 2024-08-20T22:36:54.1909930Z deleted: sha256:d09bd5c2f35cedcb846543941ee64ce1f684b52e4285696c93b3fb2032da8579 2024-08-20T22:36:54.1910767Z deleted: sha256:4de7fb7d4fc22212aae5fc1ceb637526916d0cced240d7c93d5edd6b0ea28013 2024-08-20T22:36:54.1911593Z deleted: sha256:69890e3e82746e10aff2f1084f11afb4b95a4a6c26b30f64b7001bc0a5fe32bf 2024-08-20T22:36:54.1912395Z deleted: sha256:0156df53b3224e645336ea52b4bb816f1e8d8c9b591748cd645e5e5649fa507b 2024-08-20T22:36:54.1913202Z deleted: sha256:0664624cb9cc4a386f31bf49919a8d8a11d9fea428d05585d9e6228f754bc7cd 2024-08-20T22:36:54.1914038Z deleted: sha256:8c2ddd5fa535c768c058a23f5cc4b6c64cdd71bb7458491282aabc7f7bb94d69 2024-08-20T22:36:54.1914877Z deleted: sha256:8dd03dbe0e1c5f47dd333074dff4bd00415f2f63727a65bf17866774e2ebbd0d 2024-08-20T22:36:54.1915694Z deleted: sha256:3c56b63e094bd18fad9da7a541d537210bca3b730c590d3641213259f6908d6f 2024-08-20T22:36:54.1916516Z deleted: sha256:51484fef07bd1266d4d6ff3d767751172660fc4cc6a1ffe883523a63c985e31e 2024-08-20T22:36:54.1917336Z deleted: sha256:03e83c94023982ab7dd8dc65a6df2024e342f1b760c91ec7a680ee45824869c7 2024-08-20T22:36:54.1918150Z deleted: sha256:58440ab3e8175b5de94a76485c8689a6dc186015475ebf4b386ebba309c5da34 2024-08-20T22:36:54.1918961Z deleted: sha256:9a980f3383e8aa511f0ef68982cb6dfc53b38d1ac8c3d673cb3ba3d45158c91e 2024-08-20T22:36:54.1919795Z deleted: sha256:cb0269c9cf1c47b6c4188ca7be4db261b5d3fc4b4a5153629e222dba7b1f9f98 2024-08-20T22:36:54.1920634Z deleted: sha256:41f1ba870cdffc05e0bf8d87b640fb9d4c153d7f4f3b775f37f94e9f3d584aad 2024-08-20T22:36:54.1921464Z deleted: sha256:7b502ab352f48e8b6eb5e6824a29f308952a5ee7c7d987562e1df7e47639d7f1 2024-08-20T22:36:54.1922270Z deleted: sha256:6ba5217e18738660e6ca8b7a7ec88845e70b0fd781301520bf46c71d99df3144 2024-08-20T22:36:54.1923179Z deleted: sha256:dd5a1c233594780163726bdbbf5de00f7b6b8dca1dace8a001340714c54baedb 2024-08-20T22:36:54.1924073Z deleted: sha256:c27f0db27da73e01f67f6cb78d10f355e350d829479ba4351d82055bb29f6597 2024-08-20T22:36:54.1924890Z deleted: sha256:4e3ce3c854f35c08be3dd95e0d6e718512a885e4614291d7d8b7cd23f48b7dd8 2024-08-20T22:36:54.1925715Z deleted: sha256:c2d6fb088bbec16e60cf0ad74b016f7287c92e659ec2db01271bc309b4dc6d83 2024-08-20T22:36:54.1926547Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-08-20T22:36:54.1927366Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-08-20T22:36:54.1928169Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-08-20T22:36:54.1928978Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-08-20T22:36:54.1929800Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-08-20T22:36:54.1930623Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-08-20T22:36:54.1931563Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-08-20T22:36:54.1932421Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-08-20T22:36:54.1933265Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-08-20T22:36:54.1934110Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-08-20T22:36:54.1934933Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-08-20T22:36:54.1935428Z 2024-08-20T22:36:54.1935589Z Total reclaimed space: 44.92GB 2024-08-20T22:36:54.2001878Z Post job cleanup. 2024-08-20T22:36:54.2052992Z Post job cleanup. 2024-08-20T22:36:54.2942233Z [command]/usr/bin/git version 2024-08-20T22:36:54.2991951Z git version 2.40.1 2024-08-20T22:36:54.3023050Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/84cb4a82-97a1-4efb-8151-a2fcd35f74b3/.gitconfig' 2024-08-20T22:36:54.3033801Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/84cb4a82-97a1-4efb-8151-a2fcd35f74b3' before making global git config changes 2024-08-20T22:36:54.3035203Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T22:36:54.3038522Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T22:36:54.3098076Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T22:36:54.3143399Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T22:36:54.3521451Z Entering 'android/libs/fbjni' 2024-08-20T22:36:54.3591593Z Entering 'third_party/FP16' 2024-08-20T22:36:54.3661332Z Entering 'third_party/FXdiv' 2024-08-20T22:36:54.3731686Z Entering 'third_party/NNPACK' 2024-08-20T22:36:54.3802214Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:36:54.3871408Z Entering 'third_party/XNNPACK' 2024-08-20T22:36:54.3958752Z Entering 'third_party/benchmark' 2024-08-20T22:36:54.4025307Z Entering 'third_party/cpp-httplib' 2024-08-20T22:36:54.4094735Z Entering 'third_party/cpuinfo' 2024-08-20T22:36:54.4164953Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:36:54.4234204Z Entering 'third_party/cutlass' 2024-08-20T22:36:54.4309957Z Entering 'third_party/eigen' 2024-08-20T22:36:54.4380673Z Entering 'third_party/fbgemm' 2024-08-20T22:36:54.4447793Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:36:54.4513642Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:36:54.4582258Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:36:54.4655594Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:36:54.4723532Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:36:54.4792442Z Entering 'third_party/flatbuffers' 2024-08-20T22:36:54.4863403Z Entering 'third_party/fmt' 2024-08-20T22:36:54.4932394Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:36:54.5000821Z Entering 'third_party/gloo' 2024-08-20T22:36:54.5071092Z Entering 'third_party/googletest' 2024-08-20T22:36:54.5138618Z Entering 'third_party/ideep' 2024-08-20T22:36:54.5204194Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:36:54.5277764Z Entering 'third_party/ittapi' 2024-08-20T22:36:54.5345882Z Entering 'third_party/kineto' 2024-08-20T22:36:54.5412453Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:36:54.5478224Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:36:54.5546612Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:36:54.5613066Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:36:54.5680659Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:36:54.5745412Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:36:54.5816193Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:36:54.5883319Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:36:54.5956465Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:36:54.6023905Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:36:54.6095113Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:36:54.6158790Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:36:54.6226403Z Entering 'third_party/mimalloc' 2024-08-20T22:36:54.6295616Z Entering 'third_party/nccl/nccl' 2024-08-20T22:36:54.6364271Z Entering 'third_party/nlohmann' 2024-08-20T22:36:54.6433190Z Entering 'third_party/onnx' 2024-08-20T22:36:54.6515422Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:36:54.6584156Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:36:54.6658032Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:36:54.6726305Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:36:54.6790494Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:36:54.6854526Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:36:54.6917760Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:36:54.6982275Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:36:54.7045053Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:36:54.7108198Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:36:54.7172117Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:36:54.7239657Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:36:54.7307365Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:36:54.7393182Z Entering 'third_party/pocketfft' 2024-08-20T22:36:54.7463714Z Entering 'third_party/protobuf' 2024-08-20T22:36:54.7535053Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:36:54.7599695Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:36:54.7668496Z Entering 'third_party/psimd' 2024-08-20T22:36:54.7737086Z Entering 'third_party/pthreadpool' 2024-08-20T22:36:54.7809170Z Entering 'third_party/pybind11' 2024-08-20T22:36:54.7877816Z Entering 'third_party/python-peachpy' 2024-08-20T22:36:54.7945957Z Entering 'third_party/sleef' 2024-08-20T22:36:54.8013553Z Entering 'third_party/tensorpipe' 2024-08-20T22:36:54.8082244Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:36:54.8152061Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:36:54.8217215Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:36:54.8285770Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:36:54.8350223Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:36:54.8447263Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T22:36:54.8483213Z http.https://github.com/.extraheader 2024-08-20T22:36:54.8493134Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-20T22:36:54.9034942Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T22:36:54.9406869Z Entering 'android/libs/fbjni' 2024-08-20T22:36:54.9451564Z http.https://github.com/.extraheader 2024-08-20T22:36:54.9495738Z Entering 'third_party/FP16' 2024-08-20T22:36:54.9539283Z http.https://github.com/.extraheader 2024-08-20T22:36:54.9582895Z Entering 'third_party/FXdiv' 2024-08-20T22:36:54.9626560Z http.https://github.com/.extraheader 2024-08-20T22:36:54.9672143Z Entering 'third_party/NNPACK' 2024-08-20T22:36:54.9715520Z http.https://github.com/.extraheader 2024-08-20T22:36:54.9759230Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T22:36:54.9806653Z http.https://github.com/.extraheader 2024-08-20T22:36:54.9851977Z Entering 'third_party/XNNPACK' 2024-08-20T22:36:54.9894981Z http.https://github.com/.extraheader 2024-08-20T22:36:54.9954191Z Entering 'third_party/benchmark' 2024-08-20T22:36:54.9996969Z http.https://github.com/.extraheader 2024-08-20T22:36:55.0040295Z Entering 'third_party/cpp-httplib' 2024-08-20T22:36:55.0084416Z http.https://github.com/.extraheader 2024-08-20T22:36:55.0127282Z Entering 'third_party/cpuinfo' 2024-08-20T22:36:55.0170522Z http.https://github.com/.extraheader 2024-08-20T22:36:55.0214269Z Entering 'third_party/cudnn_frontend' 2024-08-20T22:36:55.0258887Z http.https://github.com/.extraheader 2024-08-20T22:36:55.0302643Z Entering 'third_party/cutlass' 2024-08-20T22:36:55.0346333Z http.https://github.com/.extraheader 2024-08-20T22:36:55.0396861Z Entering 'third_party/eigen' 2024-08-20T22:36:55.0440469Z http.https://github.com/.extraheader 2024-08-20T22:36:55.0490831Z Entering 'third_party/fbgemm' 2024-08-20T22:36:55.0533817Z http.https://github.com/.extraheader 2024-08-20T22:36:55.0577126Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T22:36:55.0620020Z http.https://github.com/.extraheader 2024-08-20T22:36:55.0662706Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T22:36:55.0709552Z http.https://github.com/.extraheader 2024-08-20T22:36:55.0753270Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T22:36:55.0795929Z http.https://github.com/.extraheader 2024-08-20T22:36:55.0846682Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T22:36:55.0889559Z http.https://github.com/.extraheader 2024-08-20T22:36:55.0933629Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T22:36:55.0976874Z http.https://github.com/.extraheader 2024-08-20T22:36:55.1022191Z Entering 'third_party/flatbuffers' 2024-08-20T22:36:55.1067558Z http.https://github.com/.extraheader 2024-08-20T22:36:55.1113845Z Entering 'third_party/fmt' 2024-08-20T22:36:55.1158000Z http.https://github.com/.extraheader 2024-08-20T22:36:55.1202522Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T22:36:55.1248821Z http.https://github.com/.extraheader 2024-08-20T22:36:55.1294375Z Entering 'third_party/gloo' 2024-08-20T22:36:55.1338394Z http.https://github.com/.extraheader 2024-08-20T22:36:55.1384135Z Entering 'third_party/googletest' 2024-08-20T22:36:55.1428516Z http.https://github.com/.extraheader 2024-08-20T22:36:55.1474983Z Entering 'third_party/ideep' 2024-08-20T22:36:55.1518784Z http.https://github.com/.extraheader 2024-08-20T22:36:55.1561506Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T22:36:55.1608154Z http.https://github.com/.extraheader 2024-08-20T22:36:55.1660118Z Entering 'third_party/ittapi' 2024-08-20T22:36:55.1703977Z http.https://github.com/.extraheader 2024-08-20T22:36:55.1747262Z Entering 'third_party/kineto' 2024-08-20T22:36:55.1790781Z http.https://github.com/.extraheader 2024-08-20T22:36:55.1835948Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T22:36:55.1881542Z http.https://github.com/.extraheader 2024-08-20T22:36:55.1923736Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T22:36:55.1967694Z http.https://github.com/.extraheader 2024-08-20T22:36:55.2014931Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T22:36:55.2059591Z http.https://github.com/.extraheader 2024-08-20T22:36:55.2104458Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T22:36:55.2148727Z http.https://github.com/.extraheader 2024-08-20T22:36:55.2192669Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T22:36:55.2236029Z http.https://github.com/.extraheader 2024-08-20T22:36:55.2280294Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T22:36:55.2329367Z http.https://github.com/.extraheader 2024-08-20T22:36:55.2378559Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T22:36:55.2421860Z http.https://github.com/.extraheader 2024-08-20T22:36:55.2469799Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T22:36:55.2512641Z http.https://github.com/.extraheader 2024-08-20T22:36:55.2557333Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T22:36:55.2599959Z http.https://github.com/.extraheader 2024-08-20T22:36:55.2646456Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T22:36:55.2689678Z http.https://github.com/.extraheader 2024-08-20T22:36:55.2737839Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T22:36:55.2782984Z http.https://github.com/.extraheader 2024-08-20T22:36:55.2827126Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T22:36:55.2870178Z http.https://github.com/.extraheader 2024-08-20T22:36:55.2916436Z Entering 'third_party/mimalloc' 2024-08-20T22:36:55.2963412Z http.https://github.com/.extraheader 2024-08-20T22:36:55.3007642Z Entering 'third_party/nccl/nccl' 2024-08-20T22:36:55.3052837Z http.https://github.com/.extraheader 2024-08-20T22:36:55.3097087Z Entering 'third_party/nlohmann' 2024-08-20T22:36:55.3141302Z http.https://github.com/.extraheader 2024-08-20T22:36:55.3187092Z Entering 'third_party/onnx' 2024-08-20T22:36:55.3231926Z http.https://github.com/.extraheader 2024-08-20T22:36:55.3293363Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T22:36:55.3339363Z http.https://github.com/.extraheader 2024-08-20T22:36:55.3384797Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T22:36:55.3428209Z http.https://github.com/.extraheader 2024-08-20T22:36:55.3476923Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T22:36:55.3520964Z http.https://github.com/.extraheader 2024-08-20T22:36:55.3565401Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T22:36:55.3607576Z http.https://github.com/.extraheader 2024-08-20T22:36:55.3650327Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T22:36:55.3692854Z http.https://github.com/.extraheader 2024-08-20T22:36:55.3736225Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T22:36:55.3780516Z http.https://github.com/.extraheader 2024-08-20T22:36:55.3825211Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T22:36:55.3870780Z http.https://github.com/.extraheader 2024-08-20T22:36:55.3918202Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T22:36:55.3961257Z http.https://github.com/.extraheader 2024-08-20T22:36:55.4004095Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T22:36:55.4047931Z http.https://github.com/.extraheader 2024-08-20T22:36:55.4090466Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T22:36:55.4137283Z http.https://github.com/.extraheader 2024-08-20T22:36:55.4180573Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T22:36:55.4224098Z http.https://github.com/.extraheader 2024-08-20T22:36:55.4270702Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T22:36:55.4313450Z http.https://github.com/.extraheader 2024-08-20T22:36:55.4360981Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T22:36:55.4403674Z http.https://github.com/.extraheader 2024-08-20T22:36:55.4468179Z Entering 'third_party/pocketfft' 2024-08-20T22:36:55.4512682Z http.https://github.com/.extraheader 2024-08-20T22:36:55.4557159Z Entering 'third_party/protobuf' 2024-08-20T22:36:55.4601013Z http.https://github.com/.extraheader 2024-08-20T22:36:55.4648898Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T22:36:55.4694118Z http.https://github.com/.extraheader 2024-08-20T22:36:55.4737126Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T22:36:55.4780153Z http.https://github.com/.extraheader 2024-08-20T22:36:55.4828932Z Entering 'third_party/psimd' 2024-08-20T22:36:55.4873570Z http.https://github.com/.extraheader 2024-08-20T22:36:55.4918414Z Entering 'third_party/pthreadpool' 2024-08-20T22:36:55.4964792Z http.https://github.com/.extraheader 2024-08-20T22:36:55.5008464Z Entering 'third_party/pybind11' 2024-08-20T22:36:55.5053163Z http.https://github.com/.extraheader 2024-08-20T22:36:55.5096895Z Entering 'third_party/python-peachpy' 2024-08-20T22:36:55.5140776Z http.https://github.com/.extraheader 2024-08-20T22:36:55.5185655Z Entering 'third_party/sleef' 2024-08-20T22:36:55.5229268Z http.https://github.com/.extraheader 2024-08-20T22:36:55.5274576Z Entering 'third_party/tensorpipe' 2024-08-20T22:36:55.5317819Z http.https://github.com/.extraheader 2024-08-20T22:36:55.5360776Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T22:36:55.5408185Z http.https://github.com/.extraheader 2024-08-20T22:36:55.5452804Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T22:36:55.5494751Z http.https://github.com/.extraheader 2024-08-20T22:36:55.5536646Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T22:36:55.5579455Z http.https://github.com/.extraheader 2024-08-20T22:36:55.5621785Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T22:36:55.5665121Z http.https://github.com/.extraheader 2024-08-20T22:36:55.5707130Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T22:36:55.5750197Z http.https://github.com/.extraheader 2024-08-20T22:36:55.5916708Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-20T22:36:55.5936447Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-20T22:36:55.5944747Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T22:36:55.5945271Z ##[endgroup] 2024-08-20T22:37:02.4852005Z Cleaning up orphan processes