2024-06-26T07:03:37.6535297Z Current runner version: '2.317.0'
2024-06-26T07:03:37.6541299Z Runner name: 'i-0f811bba69cb4bebd'
2024-06-26T07:03:37.6542037Z Runner group name: 'Default'
2024-06-26T07:03:37.6542873Z Machine name: 'ip-10-0-58-106'
2024-06-26T07:03:37.6547563Z ##[group]GITHUB_TOKEN Permissions
2024-06-26T07:03:37.6549441Z Actions: read
2024-06-26T07:03:37.6549922Z Attestations: read
2024-06-26T07:03:37.6550413Z Checks: read
2024-06-26T07:03:37.6550937Z Contents: read
2024-06-26T07:03:37.6551382Z Deployments: read
2024-06-26T07:03:37.6551857Z Discussions: read
2024-06-26T07:03:37.6552340Z Issues: read
2024-06-26T07:03:37.6552767Z Metadata: read
2024-06-26T07:03:37.6553221Z Packages: read
2024-06-26T07:03:37.6553737Z Pages: read
2024-06-26T07:03:37.6554183Z PullRequests: read
2024-06-26T07:03:37.6554673Z RepositoryProjects: read
2024-06-26T07:03:37.6555244Z SecurityEvents: read
2024-06-26T07:03:37.6555737Z Statuses: read
2024-06-26T07:03:37.6556165Z ##[endgroup]
2024-06-26T07:03:37.6559464Z Secret source: Actions
2024-06-26T07:03:37.6560213Z Prepare workflow directory
2024-06-26T07:03:38.0257932Z Prepare all required actions
2024-06-26T07:03:38.0459949Z Getting action download info
2024-06-26T07:03:38.2028459Z Download action repository 'pytorch/test-infra@main' (SHA:43a2ce341cc31288e9a38b65ce600a7f43021bd5)
2024-06-26T07:03:38.4853767Z Download action repository 'pytorch/pytorch@main' (SHA:c422a9549d2b320b47a9391f6cbcf592b2876c88)
2024-06-26T07:03:41.2994135Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0)
2024-06-26T07:03:41.4381176Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a)
2024-06-26T07:03:41.7045549Z Getting action download info
2024-06-26T07:03:41.7914269Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c)
2024-06-26T07:03:41.9695850Z Getting action download info
2024-06-26T07:03:42.0599465Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482)
2024-06-26T07:03:42.1862752Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/129470 (b8c4c54d347aa776934c60784e35936878ef18dc)
2024-06-26T07:03:42.1864975Z ##[group] Inputs
2024-06-26T07:03:42.1865477Z   build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm86
2024-06-26T07:03:42.1872345Z   test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}
2024-06-26T07:03:42.1880005Z   docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:03:42.1881184Z   sync-tag: 
2024-06-26T07:03:42.1882069Z   timeout-minutes: 240
2024-06-26T07:03:42.1882442Z   use-gha: 
2024-06-26T07:03:42.1882743Z   dashboard-tag: 
2024-06-26T07:03:42.1883071Z   s3-bucket: gha-artifacts
2024-06-26T07:03:42.1883432Z   aws-role-to-assume: 
2024-06-26T07:03:42.1883775Z ##[endgroup]
2024-06-26T07:03:42.1885161Z Complete job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamic_aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu)
2024-06-26T07:03:42.2475341Z A job started hook has been configured by the self-hosted runner administrator
2024-06-26T07:03:42.2624219Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh'
2024-06-26T07:03:42.2634806Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:03:42.2635339Z ##[endgroup]
2024-06-26T07:03:43.3250652Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main
2024-06-26T07:03:43.3251210Z with:
2024-06-26T07:03:43.3251895Z   github-secret: ***
2024-06-26T07:03:43.3252844Z   instructions: All testing is done inside the container, to start an interactive session run:
  docker exec -it $(docker container ps --format '{{.ID}}') bash

2024-06-26T07:03:43.3253893Z   activate-with-label: false
2024-06-26T07:03:43.3254267Z   label: with-ssh
2024-06-26T07:03:43.3254601Z   remove-existing-keys: true
2024-06-26T07:03:43.3254972Z   fail-silently: true
2024-06-26T07:03:43.3255297Z env:
2024-06-26T07:03:43.3255581Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:03:43.3255932Z ##[endgroup]
2024-06-26T07:03:43.4107486Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info.
2024-06-26T07:03:43.4108514Z ciflow reference detected, attempting to extract PR number
2024-06-26T07:03:43.7288307Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys
2024-06-26T07:03:43.8045333Z No SSH keys found for user pytorch-bot[bot]
2024-06-26T07:03:43.8046080Z Grabbing public ssh keys from https://github.com/leslie-fang-intel.keys
2024-06-26T07:03:43.8738917Z No SSH keys found for user leslie-fang-intel
2024-06-26T07:03:43.8842053Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main
2024-06-26T07:03:43.8842620Z with:
2024-06-26T07:03:43.8842909Z   submodules: recursive
2024-06-26T07:03:43.8843243Z   fetch-depth: 0
2024-06-26T07:03:43.8843541Z env:
2024-06-26T07:03:43.8843842Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:03:43.8844205Z ##[endgroup]
2024-06-26T07:03:43.9038233Z ##[group]Run retry () {
2024-06-26T07:03:43.9038671Z retry () {
2024-06-26T07:03:43.9039253Z   $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*)
2024-06-26T07:03:43.9040050Z }
2024-06-26T07:03:43.9040417Z echo "${GITHUB_WORKSPACE}"
2024-06-26T07:03:43.9040899Z if [ -z "${NO_SUDO}" ]; then
2024-06-26T07:03:43.9041463Z   retry sudo rm -rf "${GITHUB_WORKSPACE}"
2024-06-26T07:03:43.9041940Z else
2024-06-26T07:03:43.9042338Z   retry rm -rf "${GITHUB_WORKSPACE}"
2024-06-26T07:03:43.9042813Z fi
2024-06-26T07:03:43.9043205Z mkdir "${GITHUB_WORKSPACE}"
2024-06-26T07:03:43.9051370Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:03:43.9051961Z env:
2024-06-26T07:03:43.9052282Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:03:43.9052651Z   NO_SUDO: 
2024-06-26T07:03:43.9082128Z ##[endgroup]
2024-06-26T07:03:43.9105066Z /home/ec2-user/actions-runner/_work/pytorch/pytorch
2024-06-26T07:03:46.8987597Z ##[group]Run malfet/checkout@silent-checkout
2024-06-26T07:03:46.8988040Z with:
2024-06-26T07:03:46.8988363Z   ref: b8c4c54d347aa776934c60784e35936878ef18dc
2024-06-26T07:03:46.8988804Z   fetch-depth: 0
2024-06-26T07:03:46.8989118Z   submodules: recursive
2024-06-26T07:03:46.8989459Z   quiet-checkout: true
2024-06-26T07:03:46.8989933Z   repository: pytorch/pytorch
2024-06-26T07:03:46.8990410Z   token: ***
2024-06-26T07:03:46.8990702Z   ssh-strict: true
2024-06-26T07:03:46.8991034Z   persist-credentials: true
2024-06-26T07:03:46.8991383Z   clean: true
2024-06-26T07:03:46.8991720Z   sparse-checkout-cone-mode: true
2024-06-26T07:03:46.8992110Z   lfs: false
2024-06-26T07:03:46.8992406Z   set-safe-directory: true
2024-06-26T07:03:46.8992744Z env:
2024-06-26T07:03:46.8993020Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:03:46.8993363Z ##[endgroup]
2024-06-26T07:03:46.9923220Z Syncing repository: pytorch/pytorch
2024-06-26T07:03:46.9925095Z ##[group]Getting Git version info
2024-06-26T07:03:46.9925855Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch'
2024-06-26T07:03:46.9926709Z [command]/usr/bin/git version
2024-06-26T07:03:46.9927075Z git version 2.40.1
2024-06-26T07:03:46.9928404Z ##[endgroup]
2024-06-26T07:03:46.9941030Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/8d616d13-65fa-495e-bad5-f3ffbe57cdd2' before making global git config changes
2024-06-26T07:03:46.9942746Z Adding repository directory to the temporary git global config as a safe directory
2024-06-26T07:03:46.9944382Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch
2024-06-26T07:03:46.9961704Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch'
2024-06-26T07:03:46.9965804Z ##[group]Initializing the repository
2024-06-26T07:03:46.9968512Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch
2024-06-26T07:03:46.9987441Z hint: Using 'master' as the name for the initial branch. This default branch name
2024-06-26T07:03:46.9988649Z hint: is subject to change. To configure the initial branch name to use in all
2024-06-26T07:03:46.9989816Z hint: of your new repositories, which will suppress this warning, call:
2024-06-26T07:03:46.9990470Z hint: 
2024-06-26T07:03:46.9991131Z hint: 	git config --global init.defaultBranch <name>
2024-06-26T07:03:46.9991625Z hint: 
2024-06-26T07:03:46.9992145Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
2024-06-26T07:03:46.9993090Z hint: 'development'. The just-created branch can be renamed via this command:
2024-06-26T07:03:46.9993930Z hint: 
2024-06-26T07:03:46.9994272Z hint: 	git branch -m <name>
2024-06-26T07:03:46.9995026Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/
2024-06-26T07:03:46.9996887Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch
2024-06-26T07:03:47.0017375Z ##[endgroup]
2024-06-26T07:03:47.0018277Z ##[group]Disabling automatic garbage collection
2024-06-26T07:03:47.0020816Z [command]/usr/bin/git config --local gc.auto 0
2024-06-26T07:03:47.0040265Z ##[endgroup]
2024-06-26T07:03:47.0041050Z ##[group]Setting up auth
2024-06-26T07:03:47.0047205Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2024-06-26T07:03:47.0068197Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2024-06-26T07:03:47.0257475Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2024-06-26T07:03:47.0278719Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2024-06-26T07:03:47.0471651Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***
2024-06-26T07:03:47.0511477Z ##[endgroup]
2024-06-26T07:03:47.0512305Z ##[group]Fetching the repository
2024-06-26T07:03:47.0518487Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/*
2024-06-26T07:03:49.2302583Z remote: Enumerating objects: 987392        
2024-06-26T07:03:49.2303510Z remote: Enumerating objects: 990209, done.        
2024-06-26T07:03:49.2306557Z remote: Counting objects:   0% (1/2817)        
2024-06-26T07:03:49.2310785Z remote: Counting objects:   1% (29/2817)        
2024-06-26T07:03:49.2311365Z remote: Counting objects:   2% (57/2817)        
2024-06-26T07:03:49.2312015Z remote: Counting objects:   3% (85/2817)        
2024-06-26T07:03:49.2312634Z remote: Counting objects:   4% (113/2817)        
2024-06-26T07:03:49.2313323Z remote: Counting objects:   5% (141/2817)        
2024-06-26T07:03:49.2313832Z remote: Counting objects:   6% (170/2817)        
2024-06-26T07:03:49.2314394Z remote: Counting objects:   7% (198/2817)        
2024-06-26T07:03:49.2315050Z remote: Counting objects:   8% (226/2817)        
2024-06-26T07:03:49.2315683Z remote: Counting objects:   9% (254/2817)        
2024-06-26T07:03:49.2316252Z remote: Counting objects:  10% (282/2817)        
2024-06-26T07:03:49.2316876Z remote: Counting objects:  11% (310/2817)        
2024-06-26T07:03:49.2317376Z remote: Counting objects:  12% (339/2817)        
2024-06-26T07:03:49.2317926Z remote: Counting objects:  13% (367/2817)        
2024-06-26T07:03:49.2318591Z remote: Counting objects:  14% (395/2817)        
2024-06-26T07:03:49.2319114Z remote: Counting objects:  15% (423/2817)        
2024-06-26T07:03:49.2319609Z remote: Counting objects:  16% (451/2817)        
2024-06-26T07:03:49.2320113Z remote: Counting objects:  17% (479/2817)        
2024-06-26T07:03:49.2320624Z remote: Counting objects:  18% (508/2817)        
2024-06-26T07:03:49.2321222Z remote: Counting objects:  19% (536/2817)        
2024-06-26T07:03:49.2321764Z remote: Counting objects:  20% (564/2817)        
2024-06-26T07:03:49.2322449Z remote: Counting objects:  21% (592/2817)        
2024-06-26T07:03:49.2322980Z remote: Counting objects:  22% (620/2817)        
2024-06-26T07:03:49.2323480Z remote: Counting objects:  23% (648/2817)        
2024-06-26T07:03:49.2323988Z remote: Counting objects:  24% (677/2817)        
2024-06-26T07:03:49.2324495Z remote: Counting objects:  25% (705/2817)        
2024-06-26T07:03:49.2325288Z remote: Counting objects:  26% (733/2817)        
2024-06-26T07:03:49.2325785Z remote: Counting objects:  27% (761/2817)        
2024-06-26T07:03:49.2326687Z remote: Counting objects:  28% (789/2817)        
2024-06-26T07:03:49.2327277Z remote: Counting objects:  29% (817/2817)        
2024-06-26T07:03:49.2327776Z remote: Counting objects:  30% (846/2817)        
2024-06-26T07:03:49.2328271Z remote: Counting objects:  31% (874/2817)        
2024-06-26T07:03:49.2328762Z remote: Counting objects:  32% (902/2817)        
2024-06-26T07:03:49.2329393Z remote: Counting objects:  33% (930/2817)        
2024-06-26T07:03:49.2329941Z remote: Counting objects:  34% (958/2817)        
2024-06-26T07:03:49.2330755Z remote: Counting objects:  35% (986/2817)        
2024-06-26T07:03:49.2331364Z remote: Counting objects:  36% (1015/2817)        
2024-06-26T07:03:49.2331895Z remote: Counting objects:  37% (1043/2817)        
2024-06-26T07:03:49.2332429Z remote: Counting objects:  38% (1071/2817)        
2024-06-26T07:03:49.2332943Z remote: Counting objects:  39% (1099/2817)        
2024-06-26T07:03:49.2333477Z remote: Counting objects:  40% (1127/2817)        
2024-06-26T07:03:49.2334139Z remote: Counting objects:  41% (1155/2817)        
2024-06-26T07:03:49.2334738Z remote: Counting objects:  42% (1184/2817)        
2024-06-26T07:03:49.2335257Z remote: Counting objects:  43% (1212/2817)        
2024-06-26T07:03:49.2335760Z remote: Counting objects:  44% (1240/2817)        
2024-06-26T07:03:49.2336273Z remote: Counting objects:  45% (1268/2817)        
2024-06-26T07:03:49.2336785Z remote: Counting objects:  46% (1296/2817)        
2024-06-26T07:03:49.2337299Z remote: Counting objects:  47% (1324/2817)        
2024-06-26T07:03:49.2337862Z remote: Counting objects:  48% (1353/2817)        
2024-06-26T07:03:49.2338542Z remote: Counting objects:  49% (1381/2817)        
2024-06-26T07:03:49.2339272Z remote: Counting objects:  50% (1409/2817)        
2024-06-26T07:03:49.2339788Z remote: Counting objects:  51% (1437/2817)        
2024-06-26T07:03:49.2340344Z remote: Counting objects:  52% (1465/2817)        
2024-06-26T07:03:49.2341044Z remote: Counting objects:  53% (1494/2817)        
2024-06-26T07:03:49.2341632Z remote: Counting objects:  54% (1522/2817)        
2024-06-26T07:03:49.2342143Z remote: Counting objects:  55% (1550/2817)        
2024-06-26T07:03:49.2342650Z remote: Counting objects:  56% (1578/2817)        
2024-06-26T07:03:49.2343157Z remote: Counting objects:  57% (1606/2817)        
2024-06-26T07:03:49.2343674Z remote: Counting objects:  58% (1634/2817)        
2024-06-26T07:03:49.2344177Z remote: Counting objects:  59% (1663/2817)        
2024-06-26T07:03:49.2344677Z remote: Counting objects:  60% (1691/2817)        
2024-06-26T07:03:49.2345182Z remote: Counting objects:  61% (1719/2817)        
2024-06-26T07:03:49.2345689Z remote: Counting objects:  62% (1747/2817)        
2024-06-26T07:03:49.2346235Z remote: Counting objects:  63% (1775/2817)        
2024-06-26T07:03:49.2346912Z remote: Counting objects:  64% (1803/2817)        
2024-06-26T07:03:49.2347415Z remote: Counting objects:  65% (1832/2817)        
2024-06-26T07:03:49.2347927Z remote: Counting objects:  66% (1860/2817)        
2024-06-26T07:03:49.2348434Z remote: Counting objects:  67% (1888/2817)        
2024-06-26T07:03:49.2348938Z remote: Counting objects:  68% (1916/2817)        
2024-06-26T07:03:49.2349439Z remote: Counting objects:  69% (1944/2817)        
2024-06-26T07:03:49.2349951Z remote: Counting objects:  70% (1972/2817)        
2024-06-26T07:03:49.2350455Z remote: Counting objects:  71% (2001/2817)        
2024-06-26T07:03:49.2350956Z remote: Counting objects:  72% (2029/2817)        
2024-06-26T07:03:49.2351451Z remote: Counting objects:  73% (2057/2817)        
2024-06-26T07:03:49.2351970Z remote: Counting objects:  74% (2085/2817)        
2024-06-26T07:03:49.2352666Z remote: Counting objects:  75% (2113/2817)        
2024-06-26T07:03:49.2353354Z remote: Counting objects:  76% (2141/2817)        
2024-06-26T07:03:49.2353896Z remote: Counting objects:  77% (2170/2817)        
2024-06-26T07:03:49.2354416Z remote: Counting objects:  78% (2198/2817)        
2024-06-26T07:03:49.2355089Z remote: Counting objects:  79% (2226/2817)        
2024-06-26T07:03:49.2355602Z remote: Counting objects:  80% (2254/2817)        
2024-06-26T07:03:49.2356101Z remote: Counting objects:  81% (2282/2817)        
2024-06-26T07:03:49.2356612Z remote: Counting objects:  82% (2310/2817)        
2024-06-26T07:03:49.2357116Z remote: Counting objects:  83% (2339/2817)        
2024-06-26T07:03:49.2363189Z remote: Counting objects:  84% (2367/2817)        
2024-06-26T07:03:49.2363890Z remote: Counting objects:  85% (2395/2817)        
2024-06-26T07:03:49.2364447Z remote: Counting objects:  86% (2423/2817)        
2024-06-26T07:03:49.2365186Z remote: Counting objects:  87% (2451/2817)        
2024-06-26T07:03:49.2365738Z remote: Counting objects:  88% (2479/2817)        
2024-06-26T07:03:49.2366436Z remote: Counting objects:  89% (2508/2817)        
2024-06-26T07:03:49.2367028Z remote: Counting objects:  90% (2536/2817)        
2024-06-26T07:03:49.2367541Z remote: Counting objects:  91% (2564/2817)        
2024-06-26T07:03:49.2368054Z remote: Counting objects:  92% (2592/2817)        
2024-06-26T07:03:49.2368559Z remote: Counting objects:  93% (2620/2817)        
2024-06-26T07:03:49.2369055Z remote: Counting objects:  94% (2648/2817)        
2024-06-26T07:03:49.2369566Z remote: Counting objects:  95% (2677/2817)        
2024-06-26T07:03:49.2370078Z remote: Counting objects:  96% (2705/2817)        
2024-06-26T07:03:49.2370684Z remote: Counting objects:  97% (2733/2817)        
2024-06-26T07:03:49.2371189Z remote: Counting objects:  98% (2761/2817)        
2024-06-26T07:03:49.2371701Z remote: Counting objects:  99% (2789/2817)        
2024-06-26T07:03:49.2372204Z remote: Counting objects: 100% (2817/2817)        
2024-06-26T07:03:49.2372738Z remote: Counting objects: 100% (2817/2817), done.        
2024-06-26T07:03:49.2526148Z remote: Compressing objects:   0% (1/1140)        
2024-06-26T07:03:49.3105579Z remote: Compressing objects:   1% (12/1140)        
2024-06-26T07:03:49.3510445Z remote: Compressing objects:   2% (23/1140)        
2024-06-26T07:03:49.3814714Z remote: Compressing objects:   3% (35/1140)        
2024-06-26T07:03:49.4005674Z remote: Compressing objects:   4% (46/1140)        
2024-06-26T07:03:49.4303168Z remote: Compressing objects:   5% (57/1140)        
2024-06-26T07:03:49.4522005Z remote: Compressing objects:   6% (69/1140)        
2024-06-26T07:03:49.5454824Z remote: Compressing objects:   7% (80/1140)        
2024-06-26T07:03:49.6080045Z remote: Compressing objects:   8% (92/1140)        
2024-06-26T07:03:49.6547804Z remote: Compressing objects:   9% (103/1140)        
2024-06-26T07:03:49.6704734Z remote: Compressing objects:  10% (114/1140)        
2024-06-26T07:03:49.6799582Z remote: Compressing objects:  11% (126/1140)        
2024-06-26T07:03:49.6867885Z remote: Compressing objects:  12% (137/1140)        
2024-06-26T07:03:49.6908654Z remote: Compressing objects:  13% (149/1140)        
2024-06-26T07:03:49.6909741Z remote: Compressing objects:  14% (160/1140)        
2024-06-26T07:03:49.6969793Z remote: Compressing objects:  15% (171/1140)        
2024-06-26T07:03:49.6970338Z remote: Compressing objects:  16% (183/1140)        
2024-06-26T07:03:49.6973726Z remote: Compressing objects:  17% (194/1140)        
2024-06-26T07:03:49.6996525Z remote: Compressing objects:  18% (206/1140)        
2024-06-26T07:03:49.7015351Z remote: Compressing objects:  19% (217/1140)        
2024-06-26T07:03:49.7038093Z remote: Compressing objects:  20% (228/1140)        
2024-06-26T07:03:49.7056334Z remote: Compressing objects:  21% (240/1140)        
2024-06-26T07:03:49.7078105Z remote: Compressing objects:  22% (251/1140)        
2024-06-26T07:03:49.7089054Z remote: Compressing objects:  23% (263/1140)        
2024-06-26T07:03:49.7096202Z remote: Compressing objects:  24% (274/1140)        
2024-06-26T07:03:49.7103314Z remote: Compressing objects:  25% (285/1140)        
2024-06-26T07:03:49.7108796Z remote: Compressing objects:  26% (297/1140)        
2024-06-26T07:03:49.7115208Z remote: Compressing objects:  27% (308/1140)        
2024-06-26T07:03:49.7122866Z remote: Compressing objects:  28% (320/1140)        
2024-06-26T07:03:49.7129967Z remote: Compressing objects:  29% (331/1140)        
2024-06-26T07:03:49.7139498Z remote: Compressing objects:  30% (342/1140)        
2024-06-26T07:03:49.7141562Z remote: Compressing objects:  31% (354/1140)        
2024-06-26T07:03:49.7159914Z remote: Compressing objects:  32% (365/1140)        
2024-06-26T07:03:49.7172447Z remote: Compressing objects:  33% (377/1140)        
2024-06-26T07:03:49.7187450Z remote: Compressing objects:  34% (388/1140)        
2024-06-26T07:03:49.7198843Z remote: Compressing objects:  35% (399/1140)        
2024-06-26T07:03:49.7220259Z remote: Compressing objects:  36% (411/1140)        
2024-06-26T07:03:49.7238660Z remote: Compressing objects:  37% (422/1140)        
2024-06-26T07:03:49.7253403Z remote: Compressing objects:  38% (434/1140)        
2024-06-26T07:03:49.7264088Z remote: Compressing objects:  39% (445/1140)        
2024-06-26T07:03:49.7290641Z remote: Compressing objects:  40% (456/1140)        
2024-06-26T07:03:49.7311788Z remote: Compressing objects:  41% (468/1140)        
2024-06-26T07:03:49.7327065Z remote: Compressing objects:  42% (479/1140)        
2024-06-26T07:03:49.7335940Z remote: Compressing objects:  43% (491/1140)        
2024-06-26T07:03:49.7339246Z remote: Compressing objects:  44% (502/1140)        
2024-06-26T07:03:49.7344841Z remote: Compressing objects:  45% (513/1140)        
2024-06-26T07:03:49.7355390Z remote: Compressing objects:  46% (525/1140)        
2024-06-26T07:03:49.7369204Z remote: Compressing objects:  47% (536/1140)        
2024-06-26T07:03:49.7376496Z remote: Compressing objects:  48% (548/1140)        
2024-06-26T07:03:49.7379354Z remote: Compressing objects:  49% (559/1140)        
2024-06-26T07:03:49.7385442Z remote: Compressing objects:  50% (570/1140)        
2024-06-26T07:03:49.7397280Z remote: Compressing objects:  51% (582/1140)        
2024-06-26T07:03:49.7417269Z remote: Compressing objects:  52% (593/1140)        
2024-06-26T07:03:49.7438567Z remote: Compressing objects:  53% (605/1140)        
2024-06-26T07:03:49.7450237Z remote: Compressing objects:  54% (616/1140)        
2024-06-26T07:03:49.7460998Z remote: Compressing objects:  55% (627/1140)        
2024-06-26T07:03:49.7470785Z remote: Compressing objects:  56% (639/1140)        
2024-06-26T07:03:49.7477756Z remote: Compressing objects:  57% (650/1140)        
2024-06-26T07:03:49.7489855Z remote: Compressing objects:  58% (662/1140)        
2024-06-26T07:03:49.7501060Z remote: Compressing objects:  59% (673/1140)        
2024-06-26T07:03:49.7520707Z remote: Compressing objects:  60% (684/1140)        
2024-06-26T07:03:49.7523393Z remote: Compressing objects:  61% (696/1140)        
2024-06-26T07:03:49.7535076Z remote: Compressing objects:  62% (707/1140)        
2024-06-26T07:03:49.7547512Z remote: Compressing objects:  63% (719/1140)        
2024-06-26T07:03:49.7555020Z remote: Compressing objects:  64% (730/1140)        
2024-06-26T07:03:49.7561642Z remote: Compressing objects:  65% (741/1140)        
2024-06-26T07:03:49.7571470Z remote: Compressing objects:  66% (753/1140)        
2024-06-26T07:03:49.7580309Z remote: Compressing objects:  67% (764/1140)        
2024-06-26T07:03:49.7583438Z remote: Compressing objects:  68% (776/1140)        
2024-06-26T07:03:49.7593393Z remote: Compressing objects:  69% (787/1140)        
2024-06-26T07:03:49.7602929Z remote: Compressing objects:  70% (798/1140)        
2024-06-26T07:03:49.7604972Z remote: Compressing objects:  71% (810/1140)        
2024-06-26T07:03:49.7606313Z remote: Compressing objects:  72% (821/1140)        
2024-06-26T07:03:49.7606947Z remote: Compressing objects:  73% (833/1140)        
2024-06-26T07:03:49.7607625Z remote: Compressing objects:  74% (844/1140)        
2024-06-26T07:03:49.7608171Z remote: Compressing objects:  75% (855/1140)        
2024-06-26T07:03:49.7608704Z remote: Compressing objects:  76% (867/1140)        
2024-06-26T07:03:49.7609336Z remote: Compressing objects:  77% (878/1140)        
2024-06-26T07:03:49.7609879Z remote: Compressing objects:  78% (890/1140)        
2024-06-26T07:03:49.7610410Z remote: Compressing objects:  79% (901/1140)        
2024-06-26T07:03:49.7611066Z remote: Compressing objects:  80% (912/1140)        
2024-06-26T07:03:49.7611593Z remote: Compressing objects:  81% (924/1140)        
2024-06-26T07:03:49.7613174Z remote: Compressing objects:  82% (935/1140)        
2024-06-26T07:03:49.7615949Z remote: Compressing objects:  83% (947/1140)        
2024-06-26T07:03:49.7616667Z remote: Compressing objects:  84% (958/1140)        
2024-06-26T07:03:49.7618411Z remote: Compressing objects:  85% (969/1140)        
2024-06-26T07:03:49.7620424Z remote: Compressing objects:  86% (981/1140)        
2024-06-26T07:03:49.7622111Z remote: Compressing objects:  87% (992/1140)        
2024-06-26T07:03:49.7624521Z remote: Compressing objects:  88% (1004/1140)        
2024-06-26T07:03:49.7625085Z remote: Compressing objects:  89% (1015/1140)        
2024-06-26T07:03:49.7628579Z remote: Compressing objects:  90% (1026/1140)        
2024-06-26T07:03:49.7629493Z remote: Compressing objects:  91% (1038/1140)        
2024-06-26T07:03:49.7630141Z remote: Compressing objects:  92% (1049/1140)        
2024-06-26T07:03:49.7630693Z remote: Compressing objects:  93% (1061/1140)        
2024-06-26T07:03:49.7632163Z remote: Compressing objects:  94% (1072/1140)        
2024-06-26T07:03:49.7634516Z remote: Compressing objects:  95% (1083/1140)        
2024-06-26T07:03:49.7635513Z remote: Compressing objects:  96% (1095/1140)        
2024-06-26T07:03:49.7637719Z remote: Compressing objects:  97% (1106/1140)        
2024-06-26T07:03:49.7638494Z remote: Compressing objects:  98% (1118/1140)        
2024-06-26T07:03:49.7639075Z remote: Compressing objects:  99% (1129/1140)        
2024-06-26T07:03:49.7639766Z remote: Compressing objects: 100% (1140/1140)        
2024-06-26T07:03:49.7640341Z remote: Compressing objects: 100% (1140/1140), done.        
2024-06-26T07:04:09.5982136Z remote: Total 990209 (delta 2280), reused 2016 (delta 1676), pack-reused 987392        
2024-06-26T07:04:32.0313359Z [command]/usr/bin/git rev-parse --verify --quiet b8c4c54d347aa776934c60784e35936878ef18dc^{object}
2024-06-26T07:04:32.0333228Z b8c4c54d347aa776934c60784e35936878ef18dc
2024-06-26T07:04:32.0336964Z ##[endgroup]
2024-06-26T07:04:32.0337526Z ##[group]Determining the checkout info
2024-06-26T07:04:32.0338288Z ##[endgroup]
2024-06-26T07:04:32.0338819Z ##[group]Checking out the ref
2024-06-26T07:04:32.0341026Z [command]/usr/bin/git checkout --quiet --force b8c4c54d347aa776934c60784e35936878ef18dc
2024-06-26T07:04:33.2608717Z ##[endgroup]
2024-06-26T07:04:33.2609363Z ##[group]Setting up auth for fetching submodules
2024-06-26T07:04:33.2613568Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic ***
2024-06-26T07:04:33.2656311Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf
2024-06-26T07:04:33.2676209Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com:
2024-06-26T07:04:33.2698277Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com:
2024-06-26T07:04:33.2721159Z ##[endgroup]
2024-06-26T07:04:33.2721712Z ##[group]Fetching submodules
2024-06-26T07:04:33.2725185Z [command]/usr/bin/git submodule sync --recursive
2024-06-26T07:04:33.2929678Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive
2024-06-26T07:04:33.3136004Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni'
2024-06-26T07:04:33.3137698Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16'
2024-06-26T07:04:33.3139407Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv'
2024-06-26T07:04:33.3141315Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK'
2024-06-26T07:04:33.3143278Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator'
2024-06-26T07:04:33.3145139Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK'
2024-06-26T07:04:33.3146626Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark'
2024-06-26T07:04:33.3148114Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib'
2024-06-26T07:04:33.3149804Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo'
2024-06-26T07:04:33.3151983Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend'
2024-06-26T07:04:33.3154182Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass'
2024-06-26T07:04:33.3156620Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen'
2024-06-26T07:04:33.3159182Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm'
2024-06-26T07:04:33.3161710Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers'
2024-06-26T07:04:33.3164022Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt'
2024-06-26T07:04:33.3167001Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi'
2024-06-26T07:04:33.3170004Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp'
2024-06-26T07:04:33.3172651Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo'
2024-06-26T07:04:33.3175393Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest'
2024-06-26T07:04:33.3178057Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep'
2024-06-26T07:04:33.3181019Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi'
2024-06-26T07:04:33.3184015Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto'
2024-06-26T07:04:33.3187256Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc'
2024-06-26T07:04:33.3190316Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl'
2024-06-26T07:04:33.3193570Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann'
2024-06-26T07:04:33.3196714Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx'
2024-06-26T07:04:33.3200378Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp'
2024-06-26T07:04:33.3203553Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft'
2024-06-26T07:04:33.3207439Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf'
2024-06-26T07:04:33.3210978Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd'
2024-06-26T07:04:33.3214918Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool'
2024-06-26T07:04:33.3218285Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11'
2024-06-26T07:04:33.3222055Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy'
2024-06-26T07:04:33.3225669Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef'
2024-06-26T07:04:33.3229678Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe'
2024-06-26T07:04:33.3248450Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'...
2024-06-26T07:04:33.5890301Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'...
2024-06-26T07:04:33.7863192Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'...
2024-06-26T07:04:33.9446060Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'...
2024-06-26T07:04:34.1721865Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'...
2024-06-26T07:04:36.1636503Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'...
2024-06-26T07:04:46.0372148Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'...
2024-06-26T07:04:46.4235160Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'...
2024-06-26T07:04:46.9085953Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'...
2024-06-26T07:04:47.4914529Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'...
2024-06-26T07:04:48.6355023Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'...
2024-06-26T07:04:50.3095052Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'...
2024-06-26T07:04:56.1510420Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'...
2024-06-26T07:04:57.4029077Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'...
2024-06-26T07:04:59.1211775Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'...
2024-06-26T07:05:00.3028644Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'...
2024-06-26T07:05:00.4662114Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'...
2024-06-26T07:05:00.9171827Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'...
2024-06-26T07:05:01.2422692Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'...
2024-06-26T07:05:02.3080047Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'...
2024-06-26T07:05:02.6552779Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'...
2024-06-26T07:05:02.9069246Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'...
2024-06-26T07:05:04.4309374Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'...
2024-06-26T07:05:05.1379136Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'...
2024-06-26T07:05:05.7802354Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'...
2024-06-26T07:05:11.4118076Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'...
2024-06-26T07:05:13.8292595Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'...
2024-06-26T07:05:19.4099795Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'...
2024-06-26T07:05:19.5871687Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'...
2024-06-26T07:05:27.9348748Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'...
2024-06-26T07:05:28.0867450Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'...
2024-06-26T07:05:28.3196289Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'...
2024-06-26T07:05:29.3470509Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'...
2024-06-26T07:05:29.6049725Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'...
2024-06-26T07:05:30.2487494Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'...
2024-06-26T07:05:30.6349315Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f'
2024-06-26T07:05:30.6432204Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3'
2024-06-26T07:05:30.6491779Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1'
2024-06-26T07:05:30.6672230Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73'
2024-06-26T07:05:30.6983259Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191'
2024-06-26T07:05:31.4738157Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2'
2024-06-26T07:05:31.4911014Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415'
2024-06-26T07:05:31.5255676Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006'
2024-06-26T07:05:31.6099808Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5'
2024-06-26T07:05:31.6369308Z Submodule path 'third_party/cudnn_frontend': checked out 'aa3abd4bc689d6412979c7f55f9cd132848c9c6a'
2024-06-26T07:05:32.0662386Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49'
2024-06-26T07:05:32.2784774Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c'
2024-06-26T07:05:32.3419447Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e'
2024-06-26T07:05:32.3429688Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit'
2024-06-26T07:05:32.3432574Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo'
2024-06-26T07:05:32.3434012Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass'
2024-06-26T07:05:32.3435498Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest'
2024-06-26T07:05:32.3437415Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch'
2024-06-26T07:05:32.3454502Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'...
2024-06-26T07:05:33.2766332Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'...
2024-06-26T07:05:33.8469499Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'...
2024-06-26T07:05:35.6339794Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'...
2024-06-26T07:05:36.6064929Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'...
2024-06-26T07:05:36.9619589Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81'
2024-06-26T07:05:37.0454557Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3'
2024-06-26T07:05:37.3884091Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6'
2024-06-26T07:05:37.4422278Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796'
2024-06-26T07:05:37.4513009Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1'
2024-06-26T07:05:37.5421265Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b'
2024-06-26T07:05:37.5707996Z Submodule path 'third_party/fmt': checked out 'e69e5f977d458f2650bb346dadf2ad30c5320281'
2024-06-26T07:05:37.5772023Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad'
2024-06-26T07:05:37.6079876Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350'
2024-06-26T07:05:37.6276333Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509'
2024-06-26T07:05:37.6641138Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929'
2024-06-26T07:05:37.6750969Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b'
2024-06-26T07:05:37.6761318Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn'
2024-06-26T07:05:37.6780015Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'...
2024-06-26T07:05:49.4937585Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67'
2024-06-26T07:05:49.5075193Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42'
2024-06-26T07:05:49.5847987Z Submodule path 'third_party/kineto': checked out '8681ff11e1fa54da39023076c5c43eddd87b7a8a'
2024-06-26T07:05:49.5859206Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog'
2024-06-26T07:05:49.5860892Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt'
2024-06-26T07:05:49.5862931Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest'
2024-06-26T07:05:49.5881685Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'...
2024-06-26T07:05:50.0992586Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'...
2024-06-26T07:05:51.3267905Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'...
2024-06-26T07:05:52.3249161Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e'
2024-06-26T07:05:52.3258799Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2024-06-26T07:05:52.3260546Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2024-06-26T07:05:52.3262228Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2024-06-26T07:05:52.3264161Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2024-06-26T07:05:52.3266007Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2024-06-26T07:05:52.3268291Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2024-06-26T07:05:52.3270329Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2024-06-26T07:05:52.3272383Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2024-06-26T07:05:52.3291026Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'...
2024-06-26T07:05:54.3397127Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'...
2024-06-26T07:05:54.7164997Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'...
2024-06-26T07:05:55.9495086Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'...
2024-06-26T07:05:56.1970956Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'...
2024-06-26T07:05:56.6884142Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'...
2024-06-26T07:05:57.6520601Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'...
2024-06-26T07:06:03.2648402Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'...
2024-06-26T07:06:03.6231170Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9'
2024-06-26T07:06:03.6366474Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400'
2024-06-26T07:06:03.6658294Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05'
2024-06-26T07:06:03.6755369Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067'
2024-06-26T07:06:03.6765964Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2024-06-26T07:06:03.6784163Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'...
2024-06-26T07:06:04.0240268Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4'
2024-06-26T07:06:04.0379425Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446'
2024-06-26T07:06:04.0713197Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850'
2024-06-26T07:06:04.1541213Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5'
2024-06-26T07:06:04.1657502Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150'
2024-06-26T07:06:04.1946417Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out 'a33701196adfad74917046096bf5a2aa0ab0bb50'
2024-06-26T07:06:04.2436154Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347'
2024-06-26T07:06:04.2739225Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78'
2024-06-26T07:06:04.2931529Z Submodule path 'third_party/nccl/nccl': checked out '48bb7fec7953112ff37499a272317f6663f8f600'
2024-06-26T07:06:04.3792913Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806'
2024-06-26T07:06:04.6571232Z Submodule path 'third_party/onnx': checked out '990217f043af7222348ca8f0301e17fa7b841781'
2024-06-26T07:06:04.6624135Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark'
2024-06-26T07:06:04.6625907Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11'
2024-06-26T07:06:04.6627657Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'...
2024-06-26T07:06:05.1193536Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'...
2024-06-26T07:06:06.2427392Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6'
2024-06-26T07:06:06.2706581Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242'
2024-06-26T07:06:06.3243352Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878'
2024-06-26T07:06:06.3255863Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark'
2024-06-26T07:06:06.3261362Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest'
2024-06-26T07:06:06.3263142Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2024-06-26T07:06:06.3264733Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2024-06-26T07:06:06.3266592Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2024-06-26T07:06:06.3268585Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2024-06-26T07:06:06.3270443Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2024-06-26T07:06:06.3272014Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg'
2024-06-26T07:06:06.3288225Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'...
2024-06-26T07:06:06.7058996Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'...
2024-06-26T07:06:07.8104437Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'...
2024-06-26T07:06:08.1445842Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'...
2024-06-26T07:06:13.6784608Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'...
2024-06-26T07:06:13.9369240Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'...
2024-06-26T07:06:14.1193934Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'...
2024-06-26T07:06:14.4367396Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'...
2024-06-26T07:06:21.8985780Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2'
2024-06-26T07:06:21.9318402Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1'
2024-06-26T07:06:21.9445908Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa'
2024-06-26T07:06:22.0299272Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d'
2024-06-26T07:06:22.0391812Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce'
2024-06-26T07:06:22.0497266Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5'
2024-06-26T07:06:22.0609548Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d'
2024-06-26T07:06:22.0620159Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2024-06-26T07:06:22.0622310Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2024-06-26T07:06:22.0637949Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'...
2024-06-26T07:06:23.8622931Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'...
2024-06-26T07:06:25.0474654Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4'
2024-06-26T07:06:25.0844666Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929'
2024-06-26T07:06:25.4530840Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50'
2024-06-26T07:06:25.4610151Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe'
2024-06-26T07:06:25.6843207Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a'
2024-06-26T07:06:25.6857590Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark'
2024-06-26T07:06:25.6859623Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest'
2024-06-26T07:06:25.6877071Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'...
2024-06-26T07:06:26.0798565Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'...
2024-06-26T07:06:27.1092486Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8'
2024-06-26T07:06:27.1717293Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081'
2024-06-26T07:06:27.1779969Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900'
2024-06-26T07:06:27.1869706Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8'
2024-06-26T07:06:27.2148697Z Submodule path 'third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b'
2024-06-26T07:06:27.2372187Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67'
2024-06-26T07:06:27.2704691Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b'
2024-06-26T07:06:27.2905935Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e'
2024-06-26T07:06:27.2917279Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest'
2024-06-26T07:06:27.2918930Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop'
2024-06-26T07:06:27.2920549Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv'
2024-06-26T07:06:27.2922465Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11'
2024-06-26T07:06:27.2940211Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'...
2024-06-26T07:06:28.2237380Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'...
2024-06-26T07:06:28.4420696Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'...
2024-06-26T07:06:29.4966141Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'...
2024-06-26T07:06:30.6801433Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e'
2024-06-26T07:06:30.6914519Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281'
2024-06-26T07:06:30.7475768Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242'
2024-06-26T07:06:30.7700350Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef'
2024-06-26T07:06:30.7710898Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2024-06-26T07:06:30.7726083Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'...
2024-06-26T07:06:30.9546692Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5'
2024-06-26T07:06:30.9571196Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0
2024-06-26T07:06:30.9769126Z Entering 'android/libs/fbjni'
2024-06-26T07:06:30.9797882Z Entering 'third_party/FP16'
2024-06-26T07:06:30.9825037Z Entering 'third_party/FXdiv'
2024-06-26T07:06:30.9849998Z Entering 'third_party/NNPACK'
2024-06-26T07:06:30.9878629Z Entering 'third_party/VulkanMemoryAllocator'
2024-06-26T07:06:30.9905607Z Entering 'third_party/XNNPACK'
2024-06-26T07:06:30.9946073Z Entering 'third_party/benchmark'
2024-06-26T07:06:30.9975233Z Entering 'third_party/cpp-httplib'
2024-06-26T07:06:31.0000703Z Entering 'third_party/cpuinfo'
2024-06-26T07:06:31.0029755Z Entering 'third_party/cudnn_frontend'
2024-06-26T07:06:31.0055436Z Entering 'third_party/cutlass'
2024-06-26T07:06:31.0090835Z Entering 'third_party/eigen'
2024-06-26T07:06:31.0119993Z Entering 'third_party/fbgemm'
2024-06-26T07:06:31.0147206Z Entering 'third_party/fbgemm/third_party/asmjit'
2024-06-26T07:06:31.0173473Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2024-06-26T07:06:31.0200071Z Entering 'third_party/fbgemm/third_party/cutlass'
2024-06-26T07:06:31.0230909Z Entering 'third_party/fbgemm/third_party/googletest'
2024-06-26T07:06:31.0257714Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2024-06-26T07:06:31.0287916Z Entering 'third_party/flatbuffers'
2024-06-26T07:06:31.0315989Z Entering 'third_party/fmt'
2024-06-26T07:06:31.0342659Z Entering 'third_party/foxi'
2024-06-26T07:06:31.0370128Z Entering 'third_party/gemmlowp/gemmlowp'
2024-06-26T07:06:31.0399606Z Entering 'third_party/gloo'
2024-06-26T07:06:31.0429320Z Entering 'third_party/googletest'
2024-06-26T07:06:31.0455954Z Entering 'third_party/ideep'
2024-06-26T07:06:31.0481217Z Entering 'third_party/ideep/mkl-dnn'
2024-06-26T07:06:31.0514880Z Entering 'third_party/ittapi'
2024-06-26T07:06:31.0543996Z Entering 'third_party/kineto'
2024-06-26T07:06:31.0571088Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2024-06-26T07:06:31.0598904Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2024-06-26T07:06:31.0627207Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2024-06-26T07:06:31.0653007Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2024-06-26T07:06:31.0679339Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2024-06-26T07:06:31.0705249Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2024-06-26T07:06:31.0733748Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2024-06-26T07:06:31.0763275Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2024-06-26T07:06:31.0790650Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2024-06-26T07:06:31.0820556Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2024-06-26T07:06:31.0848645Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2024-06-26T07:06:31.0875871Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2024-06-26T07:06:31.0903547Z Entering 'third_party/mimalloc'
2024-06-26T07:06:31.0932596Z Entering 'third_party/nccl/nccl'
2024-06-26T07:06:31.0962092Z Entering 'third_party/nlohmann'
2024-06-26T07:06:31.0991477Z Entering 'third_party/onnx'
2024-06-26T07:06:31.1030930Z Entering 'third_party/onnx/third_party/benchmark'
2024-06-26T07:06:31.1059423Z Entering 'third_party/onnx/third_party/pybind11'
2024-06-26T07:06:31.1091110Z Entering 'third_party/opentelemetry-cpp'
2024-06-26T07:06:31.1118476Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2024-06-26T07:06:31.1144813Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2024-06-26T07:06:31.1172620Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2024-06-26T07:06:31.1200244Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2024-06-26T07:06:31.1227791Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2024-06-26T07:06:31.1255236Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2024-06-26T07:06:31.1281843Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2024-06-26T07:06:31.1310271Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2024-06-26T07:06:31.1339057Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2024-06-26T07:06:31.1368998Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2024-06-26T07:06:31.1414150Z Entering 'third_party/pocketfft'
2024-06-26T07:06:31.1440944Z Entering 'third_party/protobuf'
2024-06-26T07:06:31.1473292Z Entering 'third_party/protobuf/third_party/benchmark'
2024-06-26T07:06:31.1501043Z Entering 'third_party/protobuf/third_party/googletest'
2024-06-26T07:06:31.1530286Z Entering 'third_party/psimd'
2024-06-26T07:06:31.1557017Z Entering 'third_party/pthreadpool'
2024-06-26T07:06:31.1583137Z Entering 'third_party/pybind11'
2024-06-26T07:06:31.1613495Z Entering 'third_party/python-peachpy'
2024-06-26T07:06:31.1641245Z Entering 'third_party/sleef'
2024-06-26T07:06:31.1671133Z Entering 'third_party/tensorpipe'
2024-06-26T07:06:31.1699120Z Entering 'third_party/tensorpipe/third_party/googletest'
2024-06-26T07:06:31.1725732Z Entering 'third_party/tensorpipe/third_party/libnop'
2024-06-26T07:06:31.1752077Z Entering 'third_party/tensorpipe/third_party/libuv'
2024-06-26T07:06:31.1779079Z Entering 'third_party/tensorpipe/third_party/pybind11'
2024-06-26T07:06:31.1806918Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2024-06-26T07:06:31.1846248Z ##[endgroup]
2024-06-26T07:06:31.1847132Z ##[group]Persisting credentials for submodules
2024-06-26T07:06:31.1852717Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :"
2024-06-26T07:06:31.2059633Z Entering 'android/libs/fbjni'
2024-06-26T07:06:31.2094660Z Entering 'third_party/FP16'
2024-06-26T07:06:31.2127638Z Entering 'third_party/FXdiv'
2024-06-26T07:06:31.2159964Z Entering 'third_party/NNPACK'
2024-06-26T07:06:31.2192387Z Entering 'third_party/VulkanMemoryAllocator'
2024-06-26T07:06:31.2227042Z Entering 'third_party/XNNPACK'
2024-06-26T07:06:31.2272941Z Entering 'third_party/benchmark'
2024-06-26T07:06:31.2309009Z Entering 'third_party/cpp-httplib'
2024-06-26T07:06:31.2344706Z Entering 'third_party/cpuinfo'
2024-06-26T07:06:31.2380484Z Entering 'third_party/cudnn_frontend'
2024-06-26T07:06:31.2416118Z Entering 'third_party/cutlass'
2024-06-26T07:06:31.2457074Z Entering 'third_party/eigen'
2024-06-26T07:06:31.2493463Z Entering 'third_party/fbgemm'
2024-06-26T07:06:31.2531502Z Entering 'third_party/fbgemm/third_party/asmjit'
2024-06-26T07:06:31.2566071Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2024-06-26T07:06:31.2599216Z Entering 'third_party/fbgemm/third_party/cutlass'
2024-06-26T07:06:31.2638421Z Entering 'third_party/fbgemm/third_party/googletest'
2024-06-26T07:06:31.2673019Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2024-06-26T07:06:31.2708189Z Entering 'third_party/flatbuffers'
2024-06-26T07:06:31.2746033Z Entering 'third_party/fmt'
2024-06-26T07:06:31.2781077Z Entering 'third_party/foxi'
2024-06-26T07:06:31.2816046Z Entering 'third_party/gemmlowp/gemmlowp'
2024-06-26T07:06:31.2852824Z Entering 'third_party/gloo'
2024-06-26T07:06:31.2889216Z Entering 'third_party/googletest'
2024-06-26T07:06:31.2924552Z Entering 'third_party/ideep'
2024-06-26T07:06:31.2958224Z Entering 'third_party/ideep/mkl-dnn'
2024-06-26T07:06:31.2999876Z Entering 'third_party/ittapi'
2024-06-26T07:06:31.3035712Z Entering 'third_party/kineto'
2024-06-26T07:06:31.3071576Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2024-06-26T07:06:31.3106274Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2024-06-26T07:06:31.3141583Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2024-06-26T07:06:31.3173420Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2024-06-26T07:06:31.3209653Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2024-06-26T07:06:31.3240285Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2024-06-26T07:06:31.3276022Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2024-06-26T07:06:31.3307062Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2024-06-26T07:06:31.3341803Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2024-06-26T07:06:31.3375849Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2024-06-26T07:06:31.3411564Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2024-06-26T07:06:31.3445632Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2024-06-26T07:06:31.3480405Z Entering 'third_party/mimalloc'
2024-06-26T07:06:31.3517690Z Entering 'third_party/nccl/nccl'
2024-06-26T07:06:31.3553570Z Entering 'third_party/nlohmann'
2024-06-26T07:06:31.3587739Z Entering 'third_party/onnx'
2024-06-26T07:06:31.3633487Z Entering 'third_party/onnx/third_party/benchmark'
2024-06-26T07:06:31.3668499Z Entering 'third_party/onnx/third_party/pybind11'
2024-06-26T07:06:31.3706077Z Entering 'third_party/opentelemetry-cpp'
2024-06-26T07:06:31.3742723Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2024-06-26T07:06:31.3777574Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2024-06-26T07:06:31.3809194Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2024-06-26T07:06:31.3842799Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2024-06-26T07:06:31.3878917Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2024-06-26T07:06:31.3911195Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2024-06-26T07:06:31.3946538Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2024-06-26T07:06:31.3978282Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2024-06-26T07:06:31.4012381Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2024-06-26T07:06:31.4047786Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2024-06-26T07:06:31.4094624Z Entering 'third_party/pocketfft'
2024-06-26T07:06:31.4127620Z Entering 'third_party/protobuf'
2024-06-26T07:06:31.4166091Z Entering 'third_party/protobuf/third_party/benchmark'
2024-06-26T07:06:31.4199182Z Entering 'third_party/protobuf/third_party/googletest'
2024-06-26T07:06:31.4235363Z Entering 'third_party/psimd'
2024-06-26T07:06:31.4267819Z Entering 'third_party/pthreadpool'
2024-06-26T07:06:31.4304438Z Entering 'third_party/pybind11'
2024-06-26T07:06:31.4339163Z Entering 'third_party/python-peachpy'
2024-06-26T07:06:31.4373006Z Entering 'third_party/sleef'
2024-06-26T07:06:31.4405246Z Entering 'third_party/tensorpipe'
2024-06-26T07:06:31.4441676Z Entering 'third_party/tensorpipe/third_party/googletest'
2024-06-26T07:06:31.4476636Z Entering 'third_party/tensorpipe/third_party/libnop'
2024-06-26T07:06:31.4511523Z Entering 'third_party/tensorpipe/third_party/libuv'
2024-06-26T07:06:31.4545911Z Entering 'third_party/tensorpipe/third_party/pybind11'
2024-06-26T07:06:31.4579902Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2024-06-26T07:06:31.4626625Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url"
2024-06-26T07:06:31.4832374Z Entering 'android/libs/fbjni'
2024-06-26T07:06:31.4864731Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config	remote.origin.url
2024-06-26T07:06:31.4877489Z Entering 'third_party/FP16'
2024-06-26T07:06:31.4908886Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config	remote.origin.url
2024-06-26T07:06:31.4922101Z Entering 'third_party/FXdiv'
2024-06-26T07:06:31.4954215Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config	remote.origin.url
2024-06-26T07:06:31.4966972Z Entering 'third_party/NNPACK'
2024-06-26T07:06:31.4998205Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config	remote.origin.url
2024-06-26T07:06:31.5013125Z Entering 'third_party/VulkanMemoryAllocator'
2024-06-26T07:06:31.5042659Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config	remote.origin.url
2024-06-26T07:06:31.5056683Z Entering 'third_party/XNNPACK'
2024-06-26T07:06:31.5087663Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config	remote.origin.url
2024-06-26T07:06:31.5112095Z Entering 'third_party/benchmark'
2024-06-26T07:06:31.5144576Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config	remote.origin.url
2024-06-26T07:06:31.5156177Z Entering 'third_party/cpp-httplib'
2024-06-26T07:06:31.5188225Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config	remote.origin.url
2024-06-26T07:06:31.5200829Z Entering 'third_party/cpuinfo'
2024-06-26T07:06:31.5233074Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config	remote.origin.url
2024-06-26T07:06:31.5246370Z Entering 'third_party/cudnn_frontend'
2024-06-26T07:06:31.5278194Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config	remote.origin.url
2024-06-26T07:06:31.5292584Z Entering 'third_party/cutlass'
2024-06-26T07:06:31.5323849Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config	remote.origin.url
2024-06-26T07:06:31.5343686Z Entering 'third_party/eigen'
2024-06-26T07:06:31.5375034Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config	remote.origin.url
2024-06-26T07:06:31.5390439Z Entering 'third_party/fbgemm'
2024-06-26T07:06:31.5422133Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config	remote.origin.url
2024-06-26T07:06:31.5433709Z Entering 'third_party/fbgemm/third_party/asmjit'
2024-06-26T07:06:31.5465091Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config	remote.origin.url
2024-06-26T07:06:31.5477747Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2024-06-26T07:06:31.5509384Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config	remote.origin.url
2024-06-26T07:06:31.5521686Z Entering 'third_party/fbgemm/third_party/cutlass'
2024-06-26T07:06:31.5552556Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config	remote.origin.url
2024-06-26T07:06:31.5570342Z Entering 'third_party/fbgemm/third_party/googletest'
2024-06-26T07:06:31.5601556Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config	remote.origin.url
2024-06-26T07:06:31.5614866Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2024-06-26T07:06:31.5648056Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config	remote.origin.url
2024-06-26T07:06:31.5662186Z Entering 'third_party/flatbuffers'
2024-06-26T07:06:31.5694491Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config	remote.origin.url
2024-06-26T07:06:31.5709797Z Entering 'third_party/fmt'
2024-06-26T07:06:31.5740872Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config	remote.origin.url
2024-06-26T07:06:31.5754494Z Entering 'third_party/foxi'
2024-06-26T07:06:31.5786229Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config	remote.origin.url
2024-06-26T07:06:31.5799228Z Entering 'third_party/gemmlowp/gemmlowp'
2024-06-26T07:06:31.5831666Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config	remote.origin.url
2024-06-26T07:06:31.5845466Z Entering 'third_party/gloo'
2024-06-26T07:06:31.5877019Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config	remote.origin.url
2024-06-26T07:06:31.5889725Z Entering 'third_party/googletest'
2024-06-26T07:06:31.5920019Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config	remote.origin.url
2024-06-26T07:06:31.5932684Z Entering 'third_party/ideep'
2024-06-26T07:06:31.5963108Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config	remote.origin.url
2024-06-26T07:06:31.5975029Z Entering 'third_party/ideep/mkl-dnn'
2024-06-26T07:06:31.6004711Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config	remote.origin.url
2024-06-26T07:06:31.6023009Z Entering 'third_party/ittapi'
2024-06-26T07:06:31.6055229Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config	remote.origin.url
2024-06-26T07:06:31.6068642Z Entering 'third_party/kineto'
2024-06-26T07:06:31.6099795Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config	remote.origin.url
2024-06-26T07:06:31.6113529Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2024-06-26T07:06:31.6145360Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config	remote.origin.url
2024-06-26T07:06:31.6157622Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2024-06-26T07:06:31.6190263Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config	remote.origin.url
2024-06-26T07:06:31.6204219Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2024-06-26T07:06:31.6237810Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config	remote.origin.url
2024-06-26T07:06:31.6249026Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2024-06-26T07:06:31.6280152Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config	remote.origin.url
2024-06-26T07:06:31.6292664Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2024-06-26T07:06:31.6325029Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config	remote.origin.url
2024-06-26T07:06:31.6337198Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2024-06-26T07:06:31.6369144Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config	remote.origin.url
2024-06-26T07:06:31.6382554Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2024-06-26T07:06:31.6414319Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config	remote.origin.url
2024-06-26T07:06:31.6427239Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2024-06-26T07:06:31.6457901Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config	remote.origin.url
2024-06-26T07:06:31.6471093Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2024-06-26T07:06:31.6503626Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config	remote.origin.url
2024-06-26T07:06:31.6516325Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2024-06-26T07:06:31.6548432Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config	remote.origin.url
2024-06-26T07:06:31.6562780Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2024-06-26T07:06:31.6594001Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config	remote.origin.url
2024-06-26T07:06:31.6604415Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2024-06-26T07:06:31.6635142Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config	remote.origin.url
2024-06-26T07:06:31.6648901Z Entering 'third_party/mimalloc'
2024-06-26T07:06:31.6680750Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config	remote.origin.url
2024-06-26T07:06:31.6694374Z Entering 'third_party/nccl/nccl'
2024-06-26T07:06:31.6726290Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config	remote.origin.url
2024-06-26T07:06:31.6740245Z Entering 'third_party/nlohmann'
2024-06-26T07:06:31.6771480Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config	remote.origin.url
2024-06-26T07:06:31.6785154Z Entering 'third_party/onnx'
2024-06-26T07:06:31.6816390Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config	remote.origin.url
2024-06-26T07:06:31.6840715Z Entering 'third_party/onnx/third_party/benchmark'
2024-06-26T07:06:31.6873261Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config	remote.origin.url
2024-06-26T07:06:31.6885892Z Entering 'third_party/onnx/third_party/pybind11'
2024-06-26T07:06:31.6917240Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config	remote.origin.url
2024-06-26T07:06:31.6933141Z Entering 'third_party/opentelemetry-cpp'
2024-06-26T07:06:31.6966431Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config	remote.origin.url
2024-06-26T07:06:31.6980122Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2024-06-26T07:06:31.7011397Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config	remote.origin.url
2024-06-26T07:06:31.7024871Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2024-06-26T07:06:31.7056609Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config	remote.origin.url
2024-06-26T07:06:31.7069224Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2024-06-26T07:06:31.7100609Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config	remote.origin.url
2024-06-26T07:06:31.7112616Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2024-06-26T07:06:31.7143275Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config	remote.origin.url
2024-06-26T07:06:31.7155056Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2024-06-26T07:06:31.7188377Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config	remote.origin.url
2024-06-26T07:06:31.7199632Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2024-06-26T07:06:31.7229946Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config	remote.origin.url
2024-06-26T07:06:31.7242692Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2024-06-26T07:06:31.7275568Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config	remote.origin.url
2024-06-26T07:06:31.7286760Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2024-06-26T07:06:31.7318041Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config	remote.origin.url
2024-06-26T07:06:31.7332445Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2024-06-26T07:06:31.7363113Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config	remote.origin.url
2024-06-26T07:06:31.7376800Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2024-06-26T07:06:31.7407816Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config	remote.origin.url
2024-06-26T07:06:31.7436538Z Entering 'third_party/pocketfft'
2024-06-26T07:06:31.7468691Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config	remote.origin.url
2024-06-26T07:06:31.7482362Z Entering 'third_party/protobuf'
2024-06-26T07:06:31.7514202Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config	remote.origin.url
2024-06-26T07:06:31.7528848Z Entering 'third_party/protobuf/third_party/benchmark'
2024-06-26T07:06:31.7558525Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config	remote.origin.url
2024-06-26T07:06:31.7571296Z Entering 'third_party/protobuf/third_party/googletest'
2024-06-26T07:06:31.7602192Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config	remote.origin.url
2024-06-26T07:06:31.7616873Z Entering 'third_party/psimd'
2024-06-26T07:06:31.7649086Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config	remote.origin.url
2024-06-26T07:06:31.7662002Z Entering 'third_party/pthreadpool'
2024-06-26T07:06:31.7693439Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config	remote.origin.url
2024-06-26T07:06:31.7706889Z Entering 'third_party/pybind11'
2024-06-26T07:06:31.7738622Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config	remote.origin.url
2024-06-26T07:06:31.7751427Z Entering 'third_party/python-peachpy'
2024-06-26T07:06:31.7783363Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config	remote.origin.url
2024-06-26T07:06:31.7795619Z Entering 'third_party/sleef'
2024-06-26T07:06:31.7825972Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config	remote.origin.url
2024-06-26T07:06:31.7836499Z Entering 'third_party/tensorpipe'
2024-06-26T07:06:31.7867369Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config	remote.origin.url
2024-06-26T07:06:31.7881049Z Entering 'third_party/tensorpipe/third_party/googletest'
2024-06-26T07:06:31.7913599Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config	remote.origin.url
2024-06-26T07:06:31.7926907Z Entering 'third_party/tensorpipe/third_party/libnop'
2024-06-26T07:06:31.7957543Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config	remote.origin.url
2024-06-26T07:06:31.7969948Z Entering 'third_party/tensorpipe/third_party/libuv'
2024-06-26T07:06:31.8000903Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config	remote.origin.url
2024-06-26T07:06:31.8014272Z Entering 'third_party/tensorpipe/third_party/pybind11'
2024-06-26T07:06:31.8045450Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config	remote.origin.url
2024-06-26T07:06:31.8057632Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2024-06-26T07:06:31.8088585Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config	remote.origin.url
2024-06-26T07:06:31.9044160Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:'
2024-06-26T07:06:31.9248877Z Entering 'android/libs/fbjni'
2024-06-26T07:06:31.9275454Z Entering 'third_party/FP16'
2024-06-26T07:06:31.9301686Z Entering 'third_party/FXdiv'
2024-06-26T07:06:31.9329892Z Entering 'third_party/NNPACK'
2024-06-26T07:06:31.9357686Z Entering 'third_party/VulkanMemoryAllocator'
2024-06-26T07:06:31.9383384Z Entering 'third_party/XNNPACK'
2024-06-26T07:06:31.9419994Z Entering 'third_party/benchmark'
2024-06-26T07:06:31.9446258Z Entering 'third_party/cpp-httplib'
2024-06-26T07:06:31.9473167Z Entering 'third_party/cpuinfo'
2024-06-26T07:06:31.9501452Z Entering 'third_party/cudnn_frontend'
2024-06-26T07:06:31.9526924Z Entering 'third_party/cutlass'
2024-06-26T07:06:31.9557081Z Entering 'third_party/eigen'
2024-06-26T07:06:31.9584544Z Entering 'third_party/fbgemm'
2024-06-26T07:06:31.9612720Z Entering 'third_party/fbgemm/third_party/asmjit'
2024-06-26T07:06:31.9637517Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2024-06-26T07:06:31.9664481Z Entering 'third_party/fbgemm/third_party/cutlass'
2024-06-26T07:06:31.9695644Z Entering 'third_party/fbgemm/third_party/googletest'
2024-06-26T07:06:31.9723028Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2024-06-26T07:06:31.9752203Z Entering 'third_party/flatbuffers'
2024-06-26T07:06:31.9781353Z Entering 'third_party/fmt'
2024-06-26T07:06:31.9808398Z Entering 'third_party/foxi'
2024-06-26T07:06:31.9837393Z Entering 'third_party/gemmlowp/gemmlowp'
2024-06-26T07:06:31.9866336Z Entering 'third_party/gloo'
2024-06-26T07:06:31.9894891Z Entering 'third_party/googletest'
2024-06-26T07:06:31.9922385Z Entering 'third_party/ideep'
2024-06-26T07:06:31.9949690Z Entering 'third_party/ideep/mkl-dnn'
2024-06-26T07:06:31.9983725Z Entering 'third_party/ittapi'
2024-06-26T07:06:32.0010831Z Entering 'third_party/kineto'
2024-06-26T07:06:32.0039398Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2024-06-26T07:06:32.0066182Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2024-06-26T07:06:32.0094344Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2024-06-26T07:06:32.0120606Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2024-06-26T07:06:32.0148655Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2024-06-26T07:06:32.0177492Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2024-06-26T07:06:32.0204546Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2024-06-26T07:06:32.0232811Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2024-06-26T07:06:32.0262587Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2024-06-26T07:06:32.0291159Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2024-06-26T07:06:32.0317576Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2024-06-26T07:06:32.0343579Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2024-06-26T07:06:32.0373579Z Entering 'third_party/mimalloc'
2024-06-26T07:06:32.0401751Z Entering 'third_party/nccl/nccl'
2024-06-26T07:06:32.0430887Z Entering 'third_party/nlohmann'
2024-06-26T07:06:32.0460117Z Entering 'third_party/onnx'
2024-06-26T07:06:32.0500848Z Entering 'third_party/onnx/third_party/benchmark'
2024-06-26T07:06:32.0530633Z Entering 'third_party/onnx/third_party/pybind11'
2024-06-26T07:06:32.0559588Z Entering 'third_party/opentelemetry-cpp'
2024-06-26T07:06:32.0590418Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2024-06-26T07:06:32.0617418Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2024-06-26T07:06:32.0644414Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2024-06-26T07:06:32.0672269Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2024-06-26T07:06:32.0699331Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2024-06-26T07:06:32.0723883Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2024-06-26T07:06:32.0750732Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2024-06-26T07:06:32.0778255Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2024-06-26T07:06:32.0808118Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2024-06-26T07:06:32.0836939Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2024-06-26T07:06:32.0879424Z Entering 'third_party/pocketfft'
2024-06-26T07:06:32.0909850Z Entering 'third_party/protobuf'
2024-06-26T07:06:32.0939860Z Entering 'third_party/protobuf/third_party/benchmark'
2024-06-26T07:06:32.0968079Z Entering 'third_party/protobuf/third_party/googletest'
2024-06-26T07:06:32.0995940Z Entering 'third_party/psimd'
2024-06-26T07:06:32.1023501Z Entering 'third_party/pthreadpool'
2024-06-26T07:06:32.1052304Z Entering 'third_party/pybind11'
2024-06-26T07:06:32.1079136Z Entering 'third_party/python-peachpy'
2024-06-26T07:06:32.1107969Z Entering 'third_party/sleef'
2024-06-26T07:06:32.1134111Z Entering 'third_party/tensorpipe'
2024-06-26T07:06:32.1160941Z Entering 'third_party/tensorpipe/third_party/googletest'
2024-06-26T07:06:32.1189563Z Entering 'third_party/tensorpipe/third_party/libnop'
2024-06-26T07:06:32.1215912Z Entering 'third_party/tensorpipe/third_party/libuv'
2024-06-26T07:06:32.1244661Z Entering 'third_party/tensorpipe/third_party/pybind11'
2024-06-26T07:06:32.1272362Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2024-06-26T07:06:32.1312539Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:'
2024-06-26T07:06:32.1518610Z Entering 'android/libs/fbjni'
2024-06-26T07:06:32.1547828Z Entering 'third_party/FP16'
2024-06-26T07:06:32.1576943Z Entering 'third_party/FXdiv'
2024-06-26T07:06:32.1602824Z Entering 'third_party/NNPACK'
2024-06-26T07:06:32.1632735Z Entering 'third_party/VulkanMemoryAllocator'
2024-06-26T07:06:32.1659860Z Entering 'third_party/XNNPACK'
2024-06-26T07:06:32.1698792Z Entering 'third_party/benchmark'
2024-06-26T07:06:32.1726843Z Entering 'third_party/cpp-httplib'
2024-06-26T07:06:32.1753216Z Entering 'third_party/cpuinfo'
2024-06-26T07:06:32.1781930Z Entering 'third_party/cudnn_frontend'
2024-06-26T07:06:32.1809441Z Entering 'third_party/cutlass'
2024-06-26T07:06:32.1842717Z Entering 'third_party/eigen'
2024-06-26T07:06:32.1873359Z Entering 'third_party/fbgemm'
2024-06-26T07:06:32.1901242Z Entering 'third_party/fbgemm/third_party/asmjit'
2024-06-26T07:06:32.1928856Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2024-06-26T07:06:32.1956287Z Entering 'third_party/fbgemm/third_party/cutlass'
2024-06-26T07:06:32.1986677Z Entering 'third_party/fbgemm/third_party/googletest'
2024-06-26T07:06:32.2013750Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2024-06-26T07:06:32.2040591Z Entering 'third_party/flatbuffers'
2024-06-26T07:06:32.2071435Z Entering 'third_party/fmt'
2024-06-26T07:06:32.2098562Z Entering 'third_party/foxi'
2024-06-26T07:06:32.2126767Z Entering 'third_party/gemmlowp/gemmlowp'
2024-06-26T07:06:32.2154878Z Entering 'third_party/gloo'
2024-06-26T07:06:32.2184246Z Entering 'third_party/googletest'
2024-06-26T07:06:32.2211374Z Entering 'third_party/ideep'
2024-06-26T07:06:32.2239466Z Entering 'third_party/ideep/mkl-dnn'
2024-06-26T07:06:32.2271979Z Entering 'third_party/ittapi'
2024-06-26T07:06:32.2300518Z Entering 'third_party/kineto'
2024-06-26T07:06:32.2328929Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2024-06-26T07:06:32.2356910Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2024-06-26T07:06:32.2386674Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2024-06-26T07:06:32.2414374Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2024-06-26T07:06:32.2441475Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2024-06-26T07:06:32.2469630Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2024-06-26T07:06:32.2497981Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2024-06-26T07:06:32.2525726Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2024-06-26T07:06:32.2554255Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2024-06-26T07:06:32.2581589Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2024-06-26T07:06:32.2611316Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2024-06-26T07:06:32.2637444Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2024-06-26T07:06:32.2665034Z Entering 'third_party/mimalloc'
2024-06-26T07:06:32.2693131Z Entering 'third_party/nccl/nccl'
2024-06-26T07:06:32.2720301Z Entering 'third_party/nlohmann'
2024-06-26T07:06:32.2749866Z Entering 'third_party/onnx'
2024-06-26T07:06:32.2788757Z Entering 'third_party/onnx/third_party/benchmark'
2024-06-26T07:06:32.2814755Z Entering 'third_party/onnx/third_party/pybind11'
2024-06-26T07:06:32.2845989Z Entering 'third_party/opentelemetry-cpp'
2024-06-26T07:06:32.2874412Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2024-06-26T07:06:32.2900996Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2024-06-26T07:06:32.2929193Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2024-06-26T07:06:32.2956405Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2024-06-26T07:06:32.2985824Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2024-06-26T07:06:32.3011948Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2024-06-26T07:06:32.3036693Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2024-06-26T07:06:32.3064293Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2024-06-26T07:06:32.3092167Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2024-06-26T07:06:32.3121245Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2024-06-26T07:06:32.3165373Z Entering 'third_party/pocketfft'
2024-06-26T07:06:32.3193447Z Entering 'third_party/protobuf'
2024-06-26T07:06:32.3223085Z Entering 'third_party/protobuf/third_party/benchmark'
2024-06-26T07:06:32.3248491Z Entering 'third_party/protobuf/third_party/googletest'
2024-06-26T07:06:32.3278303Z Entering 'third_party/psimd'
2024-06-26T07:06:32.3307214Z Entering 'third_party/pthreadpool'
2024-06-26T07:06:32.3334707Z Entering 'third_party/pybind11'
2024-06-26T07:06:32.3361459Z Entering 'third_party/python-peachpy'
2024-06-26T07:06:32.3389331Z Entering 'third_party/sleef'
2024-06-26T07:06:32.3417926Z Entering 'third_party/tensorpipe'
2024-06-26T07:06:32.3445191Z Entering 'third_party/tensorpipe/third_party/googletest'
2024-06-26T07:06:32.3473487Z Entering 'third_party/tensorpipe/third_party/libnop'
2024-06-26T07:06:32.3499825Z Entering 'third_party/tensorpipe/third_party/libuv'
2024-06-26T07:06:32.3526774Z Entering 'third_party/tensorpipe/third_party/pybind11'
2024-06-26T07:06:32.3552207Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2024-06-26T07:06:32.3590109Z ##[endgroup]
2024-06-26T07:06:32.3620576Z [command]/usr/bin/git log -1 --format='%H'
2024-06-26T07:06:32.3641187Z 'b8c4c54d347aa776934c60784e35936878ef18dc'
2024-06-26T07:06:32.3813242Z Prepare all required actions
2024-06-26T07:06:32.3813708Z Getting action download info
2024-06-26T07:06:32.5420011Z ##[group]Run ./.github/actions/setup-linux
2024-06-26T07:06:32.5420431Z env:
2024-06-26T07:06:32.5420708Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:06:32.5421253Z ##[endgroup]
2024-06-26T07:06:32.5479902Z ##[group]Run set -euo pipefail
2024-06-26T07:06:32.5480599Z set -euo pipefail
2024-06-26T07:06:32.5480983Z function get_ec2_metadata() {
2024-06-26T07:06:32.5481518Z   # Pulled from instance metadata endpoint for EC2
2024-06-26T07:06:32.5482397Z   # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html
2024-06-26T07:06:32.5483151Z   category=$1
2024-06-26T07:06:32.5483669Z   # If it is GCP runner (runner name contains gcp), do not run this
2024-06-26T07:06:32.5484303Z   runner_name_str=i-0f811bba69cb4bebd
2024-06-26T07:06:32.5484921Z   if [[ -f /.inarc ]]; then
2024-06-26T07:06:32.5485427Z     echo "ARC Runner, no info on ec2 metadata"
2024-06-26T07:06:32.5485993Z   elif [[ $runner_name_str == *"gcp"* ]]; then
2024-06-26T07:06:32.5486681Z     echo "Runner is from Google Cloud Platform, No info on ec2 metadata"
2024-06-26T07:06:32.5487279Z   else
2024-06-26T07:06:32.5487772Z     curl -fsSL "http://169.254.169.254/latest/meta-data/${category}"
2024-06-26T07:06:32.5488332Z   fi
2024-06-26T07:06:32.5488630Z }
2024-06-26T07:06:32.5488995Z echo "ami-id: $(get_ec2_metadata ami-id)"
2024-06-26T07:06:32.5489582Z echo "instance-id: $(get_ec2_metadata instance-id)"
2024-06-26T07:06:32.5490229Z echo "instance-type: $(get_ec2_metadata instance-type)"
2024-06-26T07:06:32.5490855Z echo "system info $(uname -a)"
2024-06-26T07:06:32.5498512Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:06:32.5499003Z env:
2024-06-26T07:06:32.5499277Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:06:32.5499623Z ##[endgroup]
2024-06-26T07:06:32.5568343Z ami-id: ami-0ce0c36d7a00b20e2
2024-06-26T07:06:32.5614611Z instance-id: i-0f811bba69cb4bebd
2024-06-26T07:06:32.5657268Z instance-type: g5.4xlarge
2024-06-26T07:06:32.5663906Z system info Linux ip-10-0-58-106.ec2.internal 4.14.336-257.562.amzn2.x86_64 #1 SMP Sat Feb 24 09:50:35 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
2024-06-26T07:06:32.5685380Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)"  >> $GITHUB_OUTPUT
2024-06-26T07:06:32.5686302Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)"  >> $GITHUB_OUTPUT
2024-06-26T07:06:32.5694260Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:06:32.5694757Z env:
2024-06-26T07:06:32.5695040Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:06:32.5695442Z ##[endgroup]
2024-06-26T07:06:32.5757439Z ##[group]Run if systemctl is-active --quiet docker; then
2024-06-26T07:06:32.5758053Z if systemctl is-active --quiet docker; then
2024-06-26T07:06:32.5758606Z     echo "Docker daemon is running...";
2024-06-26T07:06:32.5759190Z else
2024-06-26T07:06:32.5759697Z     echo "Starting docker deamon..." && sudo systemctl start docker;
2024-06-26T07:06:32.5760277Z fi
2024-06-26T07:06:32.5767266Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:06:32.5767767Z env:
2024-06-26T07:06:32.5768045Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:06:32.5768388Z ##[endgroup]
2024-06-26T07:06:32.5805801Z Docker daemon is running...
2024-06-26T07:06:32.5847515Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482
2024-06-26T07:06:32.5848069Z with:
2024-06-26T07:06:32.5848337Z   shell: bash
2024-06-26T07:06:32.5848626Z   timeout_minutes: 5
2024-06-26T07:06:32.5848948Z   max_attempts: 3
2024-06-26T07:06:32.5849267Z   retry_wait_seconds: 30
2024-06-26T07:06:32.5850819Z   command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\")
aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \
    --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com"

2024-06-26T07:06:32.5852463Z   polling_interval_seconds: 1
2024-06-26T07:06:32.5852831Z   warning_on_retry: true
2024-06-26T07:06:32.5853185Z   continue_on_error: false
2024-06-26T07:06:32.5853524Z env:
2024-06-26T07:06:32.5853790Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:06:32.5854146Z   AWS_RETRY_MODE: standard
2024-06-26T07:06:32.5854511Z   AWS_MAX_ATTEMPTS: 5
2024-06-26T07:06:32.5854860Z   AWS_DEFAULT_REGION: us-east-1
2024-06-26T07:06:32.5855246Z ##[endgroup]
2024-06-26T07:06:33.4352291Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json.
2024-06-26T07:06:33.4353342Z Configure a credential helper to remove this warning. See
2024-06-26T07:06:33.4354340Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store
2024-06-26T07:06:33.4354855Z 
2024-06-26T07:06:33.4355008Z Login Succeeded
2024-06-26T07:06:33.6346683Z Command completed after 1 attempt(s).
2024-06-26T07:06:33.6387032Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}"
2024-06-26T07:06:33.6387751Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}"
2024-06-26T07:06:33.6388397Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}"
2024-06-26T07:06:33.6396213Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:06:33.6396695Z env:
2024-06-26T07:06:33.6396971Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:06:33.6397317Z ##[endgroup]
2024-06-26T07:06:33.6468266Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty
2024-06-26T07:06:33.6469379Z # ignore expansion of "docker ps -q" since it could be empty
2024-06-26T07:06:33.6470271Z # shellcheck disable=SC2046
2024-06-26T07:06:33.6470966Z docker stop $(docker ps -q) || true
2024-06-26T07:06:33.6471705Z # Prune all of the docker images
2024-06-26T07:06:33.6472361Z docker system prune -af
2024-06-26T07:06:33.6481072Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:06:33.6481582Z env:
2024-06-26T07:06:33.6481853Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:06:33.6482191Z ##[endgroup]
2024-06-26T07:06:33.6730722Z "docker stop" requires at least 1 argument.
2024-06-26T07:06:33.6731486Z See 'docker stop --help'.
2024-06-26T07:06:33.6731824Z 
2024-06-26T07:06:33.6732281Z Usage:  docker stop [OPTIONS] CONTAINER [CONTAINER...]
2024-06-26T07:06:33.6732948Z 
2024-06-26T07:06:33.6733248Z Stop one or more running containers
2024-06-26T07:06:33.6879175Z Total reclaimed space: 0B
2024-06-26T07:06:33.6907603Z ##[group]Run set +e
2024-06-26T07:06:33.6907972Z set +e
2024-06-26T07:06:33.6908268Z set -x
2024-06-26T07:06:33.6908567Z 
2024-06-26T07:06:33.6908895Z PT_DOMAIN=download.pytorch.org
2024-06-26T07:06:33.6909683Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400,
2024-06-26T07:06:33.6910749Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last
2024-06-26T07:06:33.6911630Z # one is returned at random
2024-06-26T07:06:33.6912180Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1)
2024-06-26T07:06:33.6912686Z 
2024-06-26T07:06:33.6912997Z if [ -z "${RESOLVED_IP}" ]; then
2024-06-26T07:06:33.6913614Z   echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..."
2024-06-26T07:06:33.6914375Z   RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1)
2024-06-26T07:06:33.6914920Z 
2024-06-26T07:06:33.6915237Z   if [ -z "${RESOLVED_IP}" ]; then
2024-06-26T07:06:33.6915788Z     echo "Couldn't resolve ${PT_DOMAIN}, exiting..."
2024-06-26T07:06:33.6916289Z     exit 1
2024-06-26T07:06:33.6916601Z   fi
2024-06-26T07:06:33.6916876Z fi
2024-06-26T07:06:33.6917297Z 
2024-06-26T07:06:33.6917663Z if grep -r "${PT_DOMAIN}" /etc/hosts; then
2024-06-26T07:06:33.6918178Z   # Clean up any old records first
2024-06-26T07:06:33.6918684Z   sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts
2024-06-26T07:06:33.6919138Z fi
2024-06-26T07:06:33.6919411Z 
2024-06-26T07:06:33.6919835Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts
2024-06-26T07:06:33.6920385Z cat /etc/hosts
2024-06-26T07:06:33.6928490Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:06:33.6928987Z env:
2024-06-26T07:06:33.6929257Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:06:33.6929601Z ##[endgroup]
2024-06-26T07:06:33.6948639Z + PT_DOMAIN=download.pytorch.org
2024-06-26T07:06:33.6952877Z ++ dig -4 +short download.pytorch.org
2024-06-26T07:06:33.6953325Z ++ tail -n1
2024-06-26T07:06:33.7271052Z + RESOLVED_IP=18.160.10.22
2024-06-26T07:06:33.7271524Z + '[' -z 18.160.10.22 ']'
2024-06-26T07:06:33.7271996Z + grep -r download.pytorch.org /etc/hosts
2024-06-26T07:06:33.7276618Z 18.160.10.76 download.pytorch.org
2024-06-26T07:06:33.7277492Z + sudo sed -i /download.pytorch.org/d /etc/hosts
2024-06-26T07:06:33.7377307Z + echo '18.160.10.22 download.pytorch.org'
2024-06-26T07:06:33.7377871Z + sudo tee -a /etc/hosts
2024-06-26T07:06:33.7641509Z 18.160.10.22 download.pytorch.org
2024-06-26T07:06:33.7655995Z + cat /etc/hosts
2024-06-26T07:06:33.7660426Z 127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
2024-06-26T07:06:33.7672118Z ::1         localhost6 localhost6.localdomain6
2024-06-26T07:06:33.7672638Z 18.160.10.22 download.pytorch.org
2024-06-26T07:06:33.7875809Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main
2024-06-26T07:06:33.7876416Z with:
2024-06-26T07:06:33.7877505Z   docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:33.7878713Z   docker-build-dir: .ci/docker
2024-06-26T07:06:33.7879085Z   working-directory: .
2024-06-26T07:06:33.7879556Z   docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2024-06-26T07:06:33.7880070Z   force-push: false
2024-06-26T07:06:33.7880365Z env:
2024-06-26T07:06:33.7880633Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:06:33.7880970Z ##[endgroup]
2024-06-26T07:06:33.7899490Z ##[group]Run set -ex
2024-06-26T07:06:33.7899845Z set -ex
2024-06-26T07:06:33.7900133Z 
2024-06-26T07:06:33.7900676Z # If the docker build directory or the build script doesn't exist, the action will
2024-06-26T07:06:33.7901634Z # gracefully return the docker image name as it is.  Pulling docker image in Linux
2024-06-26T07:06:33.7902424Z # job could then download the pre-built image as usual
2024-06-26T07:06:33.7903152Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then
2024-06-26T07:06:33.7903808Z   echo "skip=true" >> "${GITHUB_OUTPUT}"
2024-06-26T07:06:33.7904432Z   echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}"
2024-06-26T07:06:33.7905083Z 
2024-06-26T07:06:33.7905579Z   echo "There is no Docker build script in ${REPO_NAME} repo, skipping..."
2024-06-26T07:06:33.7906184Z   exit 0
2024-06-26T07:06:33.7906476Z else
2024-06-26T07:06:33.7906834Z   echo "skip=false" >> "${GITHUB_OUTPUT}"
2024-06-26T07:06:33.7907269Z fi
2024-06-26T07:06:33.7907534Z 
2024-06-26T07:06:33.7907996Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then
2024-06-26T07:06:33.7908838Z   # The docker image name already includes the ECR prefix and tag, so we can just
2024-06-26T07:06:33.7909606Z   # use it as it is, but first let's extract the tag
2024-06-26T07:06:33.7910302Z   DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}')
2024-06-26T07:06:33.7911011Z   echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}"
2024-06-26T07:06:33.7911706Z   echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}"
2024-06-26T07:06:33.7912252Z else
2024-06-26T07:06:33.7912685Z   DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}")
2024-06-26T07:06:33.7913336Z   echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}"
2024-06-26T07:06:33.7914215Z   echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}"
2024-06-26T07:06:33.7914943Z fi
2024-06-26T07:06:33.7922541Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:06:33.7923023Z env:
2024-06-26T07:06:33.7923293Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:06:33.7923643Z   REPO_NAME: pytorch
2024-06-26T07:06:33.7924901Z   DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:33.7926074Z   DOCKER_BUILD_DIR: .ci/docker
2024-06-26T07:06:33.7926581Z   DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2024-06-26T07:06:33.7927084Z ##[endgroup]
2024-06-26T07:06:33.7947517Z + [[ ! -d .ci/docker ]]
2024-06-26T07:06:33.7947925Z + [[ ! -f .ci/docker/build.sh ]]
2024-06-26T07:06:33.7948303Z + echo skip=false
2024-06-26T07:06:33.7950181Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]]
2024-06-26T07:06:33.7952681Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:33.7953841Z ++ awk -F '[:,]' '{print $2}'
2024-06-26T07:06:33.7958760Z + DOCKER_TAG=91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:33.7959405Z + echo docker-tag=91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:33.7961065Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:33.7985687Z ##[group]Run set +e
2024-06-26T07:06:33.7986056Z set +e
2024-06-26T07:06:33.7986365Z set -x
2024-06-26T07:06:33.7986656Z 
2024-06-26T07:06:33.7986941Z login() {
2024-06-26T07:06:33.7987611Z   aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1"
2024-06-26T07:06:33.7988324Z }
2024-06-26T07:06:33.7988601Z 
2024-06-26T07:06:33.7988890Z retry () {
2024-06-26T07:06:33.7989290Z   $*  || (sleep 1 && $*) || (sleep 2 && $*)
2024-06-26T07:06:33.7989747Z }
2024-06-26T07:06:33.7990022Z 
2024-06-26T07:06:33.7990348Z retry login "${DOCKER_REGISTRY}"
2024-06-26T07:06:33.7990768Z 
2024-06-26T07:06:33.7991241Z # Check if image already exists, if it does then skip building it
2024-06-26T07:06:33.7992046Z if docker manifest inspect "${DOCKER_IMAGE}"; then
2024-06-26T07:06:33.7992549Z   exit 0
2024-06-26T07:06:33.7992856Z fi
2024-06-26T07:06:33.7993133Z 
2024-06-26T07:06:33.7993639Z # NB: This part requires a full checkout. Otherwise, the merge base will
2024-06-26T07:06:33.7994824Z # be empty.  The default action would be to continue rebuild the image
2024-06-26T07:06:33.7995571Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then
2024-06-26T07:06:33.7996373Z   # if we're on the base branch then use the parent commit
2024-06-26T07:06:33.7996965Z   MERGE_BASE=$(git rev-parse HEAD~)
2024-06-26T07:06:33.7997436Z else
2024-06-26T07:06:33.7997909Z   # otherwise we're on a PR, so use the most recent base commit
2024-06-26T07:06:33.7998593Z   MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION")
2024-06-26T07:06:33.7999098Z fi
2024-06-26T07:06:33.7999383Z 
2024-06-26T07:06:33.7999701Z if [[ -z "${MERGE_BASE}" ]]; then
2024-06-26T07:06:33.8000206Z   echo "rebuild=true" >> "${GITHUB_OUTPUT}"
2024-06-26T07:06:33.8000657Z 
2024-06-26T07:06:33.8001305Z   echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..."
2024-06-26T07:06:33.8002053Z   exit 0
2024-06-26T07:06:33.8002353Z fi
2024-06-26T07:06:33.8002626Z 
2024-06-26T07:06:33.8003065Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then
2024-06-26T07:06:33.8004021Z   echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit"
2024-06-26T07:06:33.8004931Z   exit 1
2024-06-26T07:06:33.8005228Z fi
2024-06-26T07:06:33.8005507Z 
2024-06-26T07:06:33.8005994Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}")
2024-06-26T07:06:33.8006940Z # If no image exists but the hash is the same as the previous hash then we should error out here
2024-06-26T07:06:33.8007794Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then
2024-06-26T07:06:33.8008745Z   echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch"
2024-06-26T07:06:33.8009824Z   echo "         Will re-build docker image to store in local cache, TTS may be longer"
2024-06-26T07:06:33.8010628Z fi
2024-06-26T07:06:33.8010920Z 
2024-06-26T07:06:33.8011270Z echo "rebuild=true" >> "${GITHUB_OUTPUT}"
2024-06-26T07:06:33.8018133Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:06:33.8018626Z env:
2024-06-26T07:06:33.8018897Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:06:33.8019266Z   DOCKER_BUILD_DIR: .ci/docker
2024-06-26T07:06:33.8019731Z   BASE_REVISION: b8c4c54d347aa776934c60784e35936878ef18dc
2024-06-26T07:06:33.8020988Z   DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:33.8022213Z   DOCKER_TAG: 91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:33.8022802Z   DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2024-06-26T07:06:33.8023317Z ##[endgroup]
2024-06-26T07:06:33.8042066Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2024-06-26T07:06:33.8042750Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2024-06-26T07:06:33.8043636Z + aws ecr get-login-password --region us-east-1
2024-06-26T07:06:33.8044406Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com
2024-06-26T07:06:34.2140556Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json.
2024-06-26T07:06:34.2141357Z Configure a credential helper to remove this warning. See
2024-06-26T07:06:34.2142431Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store
2024-06-26T07:06:34.2142939Z 
2024-06-26T07:06:34.2143074Z Login Succeeded
2024-06-26T07:06:34.2152439Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:34.4199872Z {
2024-06-26T07:06:34.4200240Z 	"schemaVersion": 2,
2024-06-26T07:06:34.4200897Z 	"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
2024-06-26T07:06:34.4201664Z 	"config": {
2024-06-26T07:06:34.4202212Z 		"mediaType": "application/vnd.docker.container.image.v1+json",
2024-06-26T07:06:34.4202938Z 		"size": 48681,
2024-06-26T07:06:34.4203493Z 		"digest": "sha256:0058b15e517a024817bccd921dc9af264c964f500b3cb9b3f4649a8ac22cc2dd"
2024-06-26T07:06:34.4204435Z 	},
2024-06-26T07:06:34.4205118Z 	"layers": [
2024-06-26T07:06:34.4205521Z 		{
2024-06-26T07:06:34.4206124Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4206881Z 			"size": 28580681,
2024-06-26T07:06:34.4207530Z 			"digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a"
2024-06-26T07:06:34.4208345Z 		},
2024-06-26T07:06:34.4208629Z 		{
2024-06-26T07:06:34.4209144Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4209802Z 			"size": 7943451,
2024-06-26T07:06:34.4210339Z 			"digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d"
2024-06-26T07:06:34.4211057Z 		},
2024-06-26T07:06:34.4211295Z 		{
2024-06-26T07:06:34.4211756Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4212573Z 			"size": 55728572,
2024-06-26T07:06:34.4213298Z 			"digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450"
2024-06-26T07:06:34.4214131Z 		},
2024-06-26T07:06:34.4214474Z 		{
2024-06-26T07:06:34.4215069Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4215904Z + exit 0
2024-06-26T07:06:34.4216272Z 			"size": 186,
2024-06-26T07:06:34.4217052Z 			"digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195"
2024-06-26T07:06:34.4217690Z 		},
2024-06-26T07:06:34.4217951Z 		{
2024-06-26T07:06:34.4218383Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4218937Z 			"size": 6886,
2024-06-26T07:06:34.4219486Z 			"digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80"
2024-06-26T07:06:34.4220431Z 		},
2024-06-26T07:06:34.4220731Z 		{
2024-06-26T07:06:34.4221201Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4221901Z 			"size": 1329001756,
2024-06-26T07:06:34.4222654Z 			"digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108"
2024-06-26T07:06:34.4223472Z 		},
2024-06-26T07:06:34.4223722Z 		{
2024-06-26T07:06:34.4224148Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4224692Z 			"size": 62501,
2024-06-26T07:06:34.4225343Z 			"digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198"
2024-06-26T07:06:34.4226153Z 		},
2024-06-26T07:06:34.4226474Z 		{
2024-06-26T07:06:34.4227001Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4227725Z 			"size": 1684,
2024-06-26T07:06:34.4228370Z 			"digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a"
2024-06-26T07:06:34.4228967Z 		},
2024-06-26T07:06:34.4229218Z 		{
2024-06-26T07:06:34.4229707Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4230339Z 			"size": 1523,
2024-06-26T07:06:34.4231042Z 			"digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1"
2024-06-26T07:06:34.4231842Z 		},
2024-06-26T07:06:34.4232174Z 		{
2024-06-26T07:06:34.4232742Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4233288Z 			"size": 2528295403,
2024-06-26T07:06:34.4233996Z 			"digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5"
2024-06-26T07:06:34.4234601Z 		},
2024-06-26T07:06:34.4234841Z 		{
2024-06-26T07:06:34.4235269Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4235889Z 			"size": 86016,
2024-06-26T07:06:34.4236468Z 			"digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b"
2024-06-26T07:06:34.4237064Z 		},
2024-06-26T07:06:34.4237308Z 		{
2024-06-26T07:06:34.4237736Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4238410Z 			"size": 1823,
2024-06-26T07:06:34.4238926Z 			"digest": "sha256:309b872da805fcdf74137a887464876fa0615dc8b24199307119eb65fc2bfb49"
2024-06-26T07:06:34.4239517Z 		},
2024-06-26T07:06:34.4239758Z 		{
2024-06-26T07:06:34.4240184Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4240720Z 			"size": 246737542,
2024-06-26T07:06:34.4241271Z 			"digest": "sha256:e82932cb002c06a782077b8e5965f8733eb8563406a23cab260d9d8c4a994f7d"
2024-06-26T07:06:34.4241866Z 		},
2024-06-26T07:06:34.4242104Z 		{
2024-06-26T07:06:34.4242521Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4243053Z 			"size": 545,
2024-06-26T07:06:34.4243594Z 			"digest": "sha256:ca9dd73e3e9378225ebdcfd12199ef44649e34fe55003cfbc8afe2e8a4527c5b"
2024-06-26T07:06:34.4244196Z 		},
2024-06-26T07:06:34.4244436Z 		{
2024-06-26T07:06:34.4245115Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4245666Z 			"size": 1283,
2024-06-26T07:06:34.4246179Z 			"digest": "sha256:16d76a79d5d6616e228537785ec83a869f68a402520bbc6e826201621ca6cb98"
2024-06-26T07:06:34.4246766Z 		},
2024-06-26T07:06:34.4247005Z 		{
2024-06-26T07:06:34.4247432Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4247959Z 			"size": 485,
2024-06-26T07:06:34.4248492Z 			"digest": "sha256:4bdf41c9ecbe1651e0986ccbea4663a0556af27b7145a679cfa45c35dc96b891"
2024-06-26T07:06:34.4249116Z 		},
2024-06-26T07:06:34.4249350Z 		{
2024-06-26T07:06:34.4249772Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4250314Z 			"size": 91710937,
2024-06-26T07:06:34.4250975Z 			"digest": "sha256:f4863801e30cdbcd1d4e6cb1cbbcde7dbcaf6ae317ca80429295741ce60ac31d"
2024-06-26T07:06:34.4251588Z 		},
2024-06-26T07:06:34.4251828Z 		{
2024-06-26T07:06:34.4252252Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4252896Z 			"size": 3233,
2024-06-26T07:06:34.4253440Z 			"digest": "sha256:d37dd9ed8948a0d02e1198bbedadbca36477a0980dd553196385aab0c025581d"
2024-06-26T07:06:34.4254046Z 		},
2024-06-26T07:06:34.4254286Z 		{
2024-06-26T07:06:34.4254710Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4255245Z 			"size": 1905,
2024-06-26T07:06:34.4255785Z 			"digest": "sha256:cf2f34f225dbed17c1a9a8b9ea4ad030aba797774a716dc0a82fe212324aa751"
2024-06-26T07:06:34.4256418Z 		},
2024-06-26T07:06:34.4256676Z 		{
2024-06-26T07:06:34.4257097Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4257645Z 			"size": 702,
2024-06-26T07:06:34.4258176Z 			"digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4"
2024-06-26T07:06:34.4258770Z 		},
2024-06-26T07:06:34.4259009Z 		{
2024-06-26T07:06:34.4259430Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4259973Z 			"size": 2784789002,
2024-06-26T07:06:34.4260524Z 			"digest": "sha256:3d914950f8addfc4d657a2365c76e1143eb80b710cd3f6e9ae05640217fc6347"
2024-06-26T07:06:34.4261134Z 		},
2024-06-26T07:06:34.4261383Z 		{
2024-06-26T07:06:34.4261797Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4262338Z 			"size": 381,
2024-06-26T07:06:34.4262863Z 			"digest": "sha256:96d0e2032184c57e62a652aaffcf0241f0d0c7a23e894ca0ad192c4a64a36583"
2024-06-26T07:06:34.4263544Z 		},
2024-06-26T07:06:34.4263780Z 		{
2024-06-26T07:06:34.4264202Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4264738Z 			"size": 11879,
2024-06-26T07:06:34.4265264Z 			"digest": "sha256:06e12ce59790cb8b091e5d619c2a62a28329008ee184a8b9a562e3db12471f30"
2024-06-26T07:06:34.4265849Z 		},
2024-06-26T07:06:34.4266090Z 		{
2024-06-26T07:06:34.4266512Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4267043Z 			"size": 803,
2024-06-26T07:06:34.4267572Z 			"digest": "sha256:ed712f63cde7070bdec2a6179c4c35e752c6991ddbc69fe9f2724a9903250ceb"
2024-06-26T07:06:34.4268177Z 		},
2024-06-26T07:06:34.4268418Z 		{
2024-06-26T07:06:34.4268831Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4269368Z 			"size": 106,
2024-06-26T07:06:34.4269886Z 			"digest": "sha256:a97a71ef8a4e66234dfc2098834f8395ac5630240e60164d1a251fdb0115ba85"
2024-06-26T07:06:34.4270479Z 		},
2024-06-26T07:06:34.4270712Z 		{
2024-06-26T07:06:34.4271138Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4271673Z 			"size": 503,
2024-06-26T07:06:34.4272208Z 			"digest": "sha256:25cd0c482ad612f936b35b4f220c580e16be982226aac8801a6a849a6bfdc221"
2024-06-26T07:06:34.4272796Z 		},
2024-06-26T07:06:34.4273038Z 		{
2024-06-26T07:06:34.4273458Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4273994Z 			"size": 121477177,
2024-06-26T07:06:34.4274550Z 			"digest": "sha256:9bf94bc456584365fda21d0ed83b7cc690b08a71727b71c4c1e667e15028a386"
2024-06-26T07:06:34.4275145Z 		},
2024-06-26T07:06:34.4275385Z 		{
2024-06-26T07:06:34.4275801Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4276338Z 			"size": 109,
2024-06-26T07:06:34.4276865Z 			"digest": "sha256:de45a14c9b227e4b0c9d45d42223b50a71c03522ddd9e2ca0d9517835206c823"
2024-06-26T07:06:34.4277461Z 		},
2024-06-26T07:06:34.4277695Z 		{
2024-06-26T07:06:34.4278114Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4278652Z 			"size": 491,
2024-06-26T07:06:34.4279169Z 			"digest": "sha256:e99241e77209b0b35a0fb59a61c0821fc161190178af7e4af45e6eeaa043375e"
2024-06-26T07:06:34.4279752Z 		},
2024-06-26T07:06:34.4279992Z 		{
2024-06-26T07:06:34.4280415Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4280943Z 			"size": 291,
2024-06-26T07:06:34.4281482Z 			"digest": "sha256:35e8f4e2a42f9df0c969dedd2199cf71719cf5ecfa676dfa014bbdeb4fdb295f"
2024-06-26T07:06:34.4282163Z 		},
2024-06-26T07:06:34.4282407Z 		{
2024-06-26T07:06:34.4282820Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4283361Z 			"size": 101,
2024-06-26T07:06:34.4283883Z 			"digest": "sha256:a823ffa4c541394b52953391dc0a6867e26c37fe4a71fa022a2cc3f07f0ba70d"
2024-06-26T07:06:34.4284482Z 		},
2024-06-26T07:06:34.4284716Z 		{
2024-06-26T07:06:34.4285364Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4285908Z 			"size": 1473,
2024-06-26T07:06:34.4286427Z 			"digest": "sha256:44ba272eae2064603605a6238b3f35cc6578859d660068e702a047a9cd8ab2fd"
2024-06-26T07:06:34.4287060Z 		},
2024-06-26T07:06:34.4287302Z 		{
2024-06-26T07:06:34.4287723Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4288256Z 			"size": 446710781,
2024-06-26T07:06:34.4288804Z 			"digest": "sha256:da02d278e5240c285bc855ce4e42cd6121d5b903787f5c14f03e575ad1268650"
2024-06-26T07:06:34.4289402Z 		},
2024-06-26T07:06:34.4289651Z 		{
2024-06-26T07:06:34.4290067Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4290676Z 			"size": 161,
2024-06-26T07:06:34.4291189Z 			"digest": "sha256:73a0ce681c295135456e190578733273276b38b4dd5babbb39298f384d43b1c6"
2024-06-26T07:06:34.4291776Z 		},
2024-06-26T07:06:34.4292009Z 		{
2024-06-26T07:06:34.4292429Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4293084Z 			"size": 564,
2024-06-26T07:06:34.4293606Z 			"digest": "sha256:49737bd5ae709dbf7956e21fc8b05ba009e9e6c9b130b0126a73bd3d3f5efc78"
2024-06-26T07:06:34.4294204Z 		},
2024-06-26T07:06:34.4294446Z 		{
2024-06-26T07:06:34.4294868Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4295395Z 			"size": 35874600,
2024-06-26T07:06:34.4295940Z 			"digest": "sha256:9cd4c745f8e4f9d5c9bb8050923575afad21f5b7852975a4a1981bc9eff282c5"
2024-06-26T07:06:34.4296585Z 		},
2024-06-26T07:06:34.4296829Z 		{
2024-06-26T07:06:34.4297247Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4297782Z 			"size": 104,
2024-06-26T07:06:34.4298292Z 			"digest": "sha256:b2950a0365345d4c17a48627c905fd8930b3b3154c727f0526eb9ca547102bb7"
2024-06-26T07:06:34.4298876Z 		},
2024-06-26T07:06:34.4299109Z 		{
2024-06-26T07:06:34.4299528Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4300065Z 			"size": 425,
2024-06-26T07:06:34.4300587Z 			"digest": "sha256:25e19d81d9fc081962ff4cbc09e098f4035419822712c19a410f5282300dfb9e"
2024-06-26T07:06:34.4301176Z 		},
2024-06-26T07:06:34.4301416Z 		{
2024-06-26T07:06:34.4301837Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4302368Z 			"size": 20262084,
2024-06-26T07:06:34.4302919Z 			"digest": "sha256:60156b40fcbec2c25f0170b0a82fa8cd9303e0daae8bb1811fd0fc6562e88e8e"
2024-06-26T07:06:34.4303521Z 		},
2024-06-26T07:06:34.4303764Z 		{
2024-06-26T07:06:34.4304186Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4304717Z 			"size": 439,
2024-06-26T07:06:34.4305228Z 			"digest": "sha256:7a868239227fd19b30a19927bbc8f058d247403fe5b343c55f4b4b8987918911"
2024-06-26T07:06:34.4305808Z 		},
2024-06-26T07:06:34.4306048Z 		{
2024-06-26T07:06:34.4306489Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4307050Z 			"size": 702,
2024-06-26T07:06:34.4307574Z 			"digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4"
2024-06-26T07:06:34.4308185Z 		},
2024-06-26T07:06:34.4308437Z 		{
2024-06-26T07:06:34.4308859Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4309386Z 			"size": 143,
2024-06-26T07:06:34.4309902Z 			"digest": "sha256:83911d7c7ecf11b5932bb7449975c52e7f737a5b867691935e1ccb85ceb25b52"
2024-06-26T07:06:34.4310490Z 		},
2024-06-26T07:06:34.4310724Z 		{
2024-06-26T07:06:34.4311147Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4311776Z 			"size": 135,
2024-06-26T07:06:34.4312323Z 			"digest": "sha256:b3c5d66c679f58c71febdd502889bd8cd3d36e5001754cdf1e30eabe21135cce"
2024-06-26T07:06:34.4312922Z 		},
2024-06-26T07:06:34.4313166Z 		{
2024-06-26T07:06:34.4313590Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4314138Z 			"size": 4911495247,
2024-06-26T07:06:34.4314679Z 			"digest": "sha256:84b2c7a899820f6acdb22148c5e8346d8b26c33269fd01842c9573515251e1e1"
2024-06-26T07:06:34.4315276Z 		},
2024-06-26T07:06:34.4315523Z 		{
2024-06-26T07:06:34.4315946Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4316478Z 			"size": 190,
2024-06-26T07:06:34.4317019Z 			"digest": "sha256:c1068d03a811d8dead5da9e82fdc8869205fc4ee9cb92d4f7a4acfbb5ba83950"
2024-06-26T07:06:34.4317631Z 		},
2024-06-26T07:06:34.4317864Z 		{
2024-06-26T07:06:34.4318284Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4318817Z 			"size": 565,
2024-06-26T07:06:34.4319344Z 			"digest": "sha256:671b4a301d81074fca17d36851f81801552c4ecb2f90ff3412aa9ab6a1a91371"
2024-06-26T07:06:34.4319926Z 		},
2024-06-26T07:06:34.4320175Z 		{
2024-06-26T07:06:34.4320595Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4321132Z 			"size": 43163095,
2024-06-26T07:06:34.4321673Z 			"digest": "sha256:67760cf5ec46ab58d3ca98637fd9b26cb47a67446bfd0db176fe5e023606aa0d"
2024-06-26T07:06:34.4322348Z 		},
2024-06-26T07:06:34.4322589Z 		{
2024-06-26T07:06:34.4323003Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4323539Z 			"size": 106,
2024-06-26T07:06:34.4324068Z 			"digest": "sha256:ddc9740bcea2b301e99b50525059aea8d9c7121a1c8eab566561a8968438a8c2"
2024-06-26T07:06:34.4324673Z 		},
2024-06-26T07:06:34.4325091Z 		{
2024-06-26T07:06:34.4325516Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4326050Z 			"size": 1213,
2024-06-26T07:06:34.4326607Z 			"digest": "sha256:fb2e977685609fcc01de170ba7f50a573e73e6520e47253717e92a28952c5ee8"
2024-06-26T07:06:34.4327215Z 		},
2024-06-26T07:06:34.4327457Z 		{
2024-06-26T07:06:34.4327880Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4328435Z 			"size": 702,
2024-06-26T07:06:34.4328960Z 			"digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4"
2024-06-26T07:06:34.4329562Z 		},
2024-06-26T07:06:34.4329800Z 		{
2024-06-26T07:06:34.4330221Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4330815Z 			"size": 138,
2024-06-26T07:06:34.4331350Z 			"digest": "sha256:b599c5a8c7cb461cbd7b3c98b99960b8eb04506e05e79db1cc0cfadb14a47e91"
2024-06-26T07:06:34.4331954Z 		},
2024-06-26T07:06:34.4332190Z 		{
2024-06-26T07:06:34.4332610Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4333141Z 			"size": 120,
2024-06-26T07:06:34.4333679Z 			"digest": "sha256:e08e88c1366827455ee7ddbf92b6ae615ceb3cbd3fb11cd03ddd31bd3fa94a9c"
2024-06-26T07:06:34.4334279Z 		},
2024-06-26T07:06:34.4334523Z 		{
2024-06-26T07:06:34.4334941Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4335477Z 			"size": 1904157495,
2024-06-26T07:06:34.4336028Z 			"digest": "sha256:12ba7431ff38de8cf6af3052a2a52e79f4e712b8c99f5bafde3c20e63b1f570f"
2024-06-26T07:06:34.4336634Z 		},
2024-06-26T07:06:34.4336876Z 		{
2024-06-26T07:06:34.4337290Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4337831Z 			"size": 172,
2024-06-26T07:06:34.4338363Z 			"digest": "sha256:cf7c2cb896cba6a594028cc0fd246e127a2f4481d48905a3aa12c5f5872332d4"
2024-06-26T07:06:34.4338966Z 		},
2024-06-26T07:06:34.4339200Z 		{
2024-06-26T07:06:34.4339619Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4340164Z 			"size": 908,
2024-06-26T07:06:34.4340705Z 			"digest": "sha256:3a778ddfac2af5b3a11f49ddf2f3f1ad1731f74ba691f434ab69966d1e2a189f"
2024-06-26T07:06:34.4341412Z 		},
2024-06-26T07:06:34.4341662Z 		{
2024-06-26T07:06:34.4342087Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4342622Z 			"size": 702,
2024-06-26T07:06:34.4343151Z 			"digest": "sha256:df546d7b3e34396a6f2b51d8c88be976eca017bc99168d838c2c7bacabab9ec4"
2024-06-26T07:06:34.4343757Z 		},
2024-06-26T07:06:34.4343996Z 		{
2024-06-26T07:06:34.4344411Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4344950Z 			"size": 134,
2024-06-26T07:06:34.4345474Z 			"digest": "sha256:236c8f0245f4b40616a4322eaf1e1b3e178d5990db1fcaba39622ea4e79c06a1"
2024-06-26T07:06:34.4346066Z 		},
2024-06-26T07:06:34.4346299Z 		{
2024-06-26T07:06:34.4346776Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4347310Z 			"size": 32,
2024-06-26T07:06:34.4347837Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2024-06-26T07:06:34.4348432Z 		},
2024-06-26T07:06:34.4348685Z 		{
2024-06-26T07:06:34.4349105Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4349630Z 			"size": 156,
2024-06-26T07:06:34.4350166Z 			"digest": "sha256:dda1ce302b9a9df0c8f1145d2e3eaa03c96909bb45a792451f1fcb555b06738d"
2024-06-26T07:06:34.4350782Z 		},
2024-06-26T07:06:34.4351024Z 		{
2024-06-26T07:06:34.4351437Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4351976Z 			"size": 1841,
2024-06-26T07:06:34.4352633Z 			"digest": "sha256:e797fe86caef19319e03f8dca92cf96838ab3bb3644f4447c6dfc04230118afa"
2024-06-26T07:06:34.4353238Z 		},
2024-06-26T07:06:34.4353473Z 		{
2024-06-26T07:06:34.4353893Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4354432Z 			"size": 7529784,
2024-06-26T07:06:34.4354968Z 			"digest": "sha256:967190076fc0ab2bb2fc0d74e58edd3d662a537d702ff3f8b6e876230154a16f"
2024-06-26T07:06:34.4355562Z 		},
2024-06-26T07:06:34.4355810Z 		{
2024-06-26T07:06:34.4356234Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4356815Z 			"size": 165,
2024-06-26T07:06:34.4357342Z 			"digest": "sha256:4fb0a83f06855534ef82b80dcf7c685303edb2bcbd446196066dc2ad2dd71281"
2024-06-26T07:06:34.4357951Z 		},
2024-06-26T07:06:34.4358195Z 		{
2024-06-26T07:06:34.4358608Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4359143Z 			"size": 7944,
2024-06-26T07:06:34.4359676Z 			"digest": "sha256:68af2ec4e30a33f26947ece9f3a31b7d3b04c2bca5865d4b48618457832a1da2"
2024-06-26T07:06:34.4360282Z 		},
2024-06-26T07:06:34.4360517Z 		{
2024-06-26T07:06:34.4360940Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4361480Z 			"size": 8070,
2024-06-26T07:06:34.4362003Z 			"digest": "sha256:cc91933e985a044c6df53caa15488d2285896f5e991f38ec068c253ceec53f14"
2024-06-26T07:06:34.4362596Z 		},
2024-06-26T07:06:34.4362838Z 		{
2024-06-26T07:06:34.4363264Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4363793Z 			"size": 303,
2024-06-26T07:06:34.4364323Z 			"digest": "sha256:9f3187916e7dcad33cc572f8f0e949d2f507b887f99cfd6acefd14826b44960d"
2024-06-26T07:06:34.4365107Z 		},
2024-06-26T07:06:34.4365354Z 		{
2024-06-26T07:06:34.4365768Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4366305Z 			"size": 7629071,
2024-06-26T07:06:34.4366874Z 			"digest": "sha256:52608399485738d10d779254946c9e1d536010dd45593c69bf1eb9f62f85dc24"
2024-06-26T07:06:34.4367455Z 		},
2024-06-26T07:06:34.4367687Z 		{
2024-06-26T07:06:34.4368114Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4368654Z 			"size": 108,
2024-06-26T07:06:34.4369180Z 			"digest": "sha256:05a6f6dcc1c00bac6660923a22adc7d7c34416718f8c89daf9b3351d9365f694"
2024-06-26T07:06:34.4369793Z 		},
2024-06-26T07:06:34.4370033Z 		{
2024-06-26T07:06:34.4370525Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4371170Z 			"size": 54145775,
2024-06-26T07:06:34.4371728Z 			"digest": "sha256:6bc9dc8315b09b27c976bfba9700767ba412d72fd0528b18bbe8f8e62005ec22"
2024-06-26T07:06:34.4372331Z 		},
2024-06-26T07:06:34.4372575Z 		{
2024-06-26T07:06:34.4373002Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4373536Z 			"size": 474,
2024-06-26T07:06:34.4374069Z 			"digest": "sha256:1b0e27c35f07b3fbf0aeb548d1949bf1d60a420136da21789d94d196c891d981"
2024-06-26T07:06:34.4374670Z 		},
2024-06-26T07:06:34.4374912Z 		{
2024-06-26T07:06:34.4375344Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4375886Z 			"size": 1374858747,
2024-06-26T07:06:34.4376418Z 			"digest": "sha256:95237e7311a1c5ce6937db7196c866632f44e8788514a5c0eb4a3f04da4017e6"
2024-06-26T07:06:34.4377007Z 		},
2024-06-26T07:06:34.4377247Z 		{
2024-06-26T07:06:34.4377668Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4378198Z 			"size": 106,
2024-06-26T07:06:34.4378744Z 			"digest": "sha256:789eff6c51aada70bf75eb1acafed63c0d839b768638001500616ff6e53b854a"
2024-06-26T07:06:34.4379353Z 		},
2024-06-26T07:06:34.4379596Z 		{
2024-06-26T07:06:34.4380012Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4380551Z 			"size": 558,
2024-06-26T07:06:34.4381090Z 			"digest": "sha256:de0f8a8c45c6077b3c5ba3b84d7f07de98353fd618dad9a1e2f368ffe00493c7"
2024-06-26T07:06:34.4381690Z 		},
2024-06-26T07:06:34.4382008Z 		{
2024-06-26T07:06:34.4382428Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4382968Z 			"size": 46248513,
2024-06-26T07:06:34.4383499Z 			"digest": "sha256:2060028f74888d218bebdd10d723f66c33ee1c708935c4ffd4654d34912ac4dc"
2024-06-26T07:06:34.4384100Z 		},
2024-06-26T07:06:34.4384346Z 		{
2024-06-26T07:06:34.4384766Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4385298Z 			"size": 111,
2024-06-26T07:06:34.4385829Z 			"digest": "sha256:e412c7f3b2ded440992955bd92512345cbece8a8894cd3592f714ebc758ad1a0"
2024-06-26T07:06:34.4386431Z 		},
2024-06-26T07:06:34.4386711Z 		{
2024-06-26T07:06:34.4387131Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4387667Z 			"size": 32,
2024-06-26T07:06:34.4388198Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2024-06-26T07:06:34.4388791Z 		},
2024-06-26T07:06:34.4389032Z 		{
2024-06-26T07:06:34.4389465Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4390000Z 			"size": 32,
2024-06-26T07:06:34.4390522Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2024-06-26T07:06:34.4391121Z 		},
2024-06-26T07:06:34.4391363Z 		{
2024-06-26T07:06:34.4391776Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4392309Z 			"size": 32,
2024-06-26T07:06:34.4392843Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2024-06-26T07:06:34.4393443Z 		},
2024-06-26T07:06:34.4393675Z 		{
2024-06-26T07:06:34.4394095Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2024-06-26T07:06:34.4394632Z 			"size": 32,
2024-06-26T07:06:34.4395155Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2024-06-26T07:06:34.4395746Z 		}
2024-06-26T07:06:34.4395987Z 	]
2024-06-26T07:06:34.4396228Z }
2024-06-26T07:06:34.4512447Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/}
2024-06-26T07:06:34.4512954Z tag=${ECR_DOCKER_IMAGE##*/}
2024-06-26T07:06:34.4513504Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}"
2024-06-26T07:06:34.4521456Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:06:34.4521956Z env:
2024-06-26T07:06:34.4522239Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:06:34.4523376Z   ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:34.4524524Z ##[endgroup]
2024-06-26T07:06:34.4547013Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks-91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:34.4596328Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main
2024-06-26T07:06:34.4596902Z with:
2024-06-26T07:06:34.4597971Z   docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:34.4599287Z   docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2024-06-26T07:06:34.4599808Z env:
2024-06-26T07:06:34.4600091Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:06:34.4600443Z ##[endgroup]
2024-06-26T07:06:34.4615910Z ##[group]Run set -x
2024-06-26T07:06:34.4616298Z set -x
2024-06-26T07:06:34.4616715Z set +e
2024-06-26T07:06:34.4617018Z 
2024-06-26T07:06:34.4617298Z login() {
2024-06-26T07:06:34.4617961Z   aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1"
2024-06-26T07:06:34.4618677Z }
2024-06-26T07:06:34.4618965Z 
2024-06-26T07:06:34.4619276Z retry () {
2024-06-26T07:06:34.4619679Z   $*  || (sleep 1 && $*) || (sleep 2 && $*)
2024-06-26T07:06:34.4620127Z }
2024-06-26T07:06:34.4620411Z 
2024-06-26T07:06:34.4620856Z retry login "${DOCKER_REGISTRY}"
2024-06-26T07:06:34.4621272Z 
2024-06-26T07:06:34.4621554Z set -e
2024-06-26T07:06:34.4622033Z # ignore output since only exit code is used for conditional
2024-06-26T07:06:34.4622738Z # only pull docker image if it's not available locally
2024-06-26T07:06:34.4623502Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then
2024-06-26T07:06:34.4624198Z   retry docker pull "${DOCKER_IMAGE}"
2024-06-26T07:06:34.4624645Z fi
2024-06-26T07:06:34.4632213Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:06:34.4632707Z env:
2024-06-26T07:06:34.4632994Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:06:34.4634136Z   DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:34.4635435Z   DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2024-06-26T07:06:34.4635968Z ##[endgroup]
2024-06-26T07:06:34.4655902Z + set +e
2024-06-26T07:06:34.4656515Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2024-06-26T07:06:34.4657200Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2024-06-26T07:06:34.4658112Z + aws ecr get-login-password --region us-east-1
2024-06-26T07:06:34.4659181Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com
2024-06-26T07:06:34.8828820Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json.
2024-06-26T07:06:34.8829642Z Configure a credential helper to remove this warning. See
2024-06-26T07:06:34.8830550Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store
2024-06-26T07:06:34.8831158Z 
2024-06-26T07:06:34.8831287Z Login Succeeded
2024-06-26T07:06:34.8840600Z + set -e
2024-06-26T07:06:34.8842079Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:34.8961475Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:34.8963585Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:06:35.1219259Z 91382da70d5719cd7007b6b80b71d2f48398f6b7: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks
2024-06-26T07:06:35.1223066Z 7a2c55901189: Pulling fs layer
2024-06-26T07:06:35.1223842Z 224fe954d725: Pulling fs layer
2024-06-26T07:06:35.1224618Z 75722010b82e: Pulling fs layer
2024-06-26T07:06:35.1225174Z d527cbbb87e3: Pulling fs layer
2024-06-26T07:06:35.1225690Z b57676e46aee: Pulling fs layer
2024-06-26T07:06:35.1226093Z a8c1e85b5e14: Pulling fs layer
2024-06-26T07:06:35.1226477Z a41a8d1c11c8: Pulling fs layer
2024-06-26T07:06:35.1226849Z 0c1227890755: Pulling fs layer
2024-06-26T07:06:35.1227304Z d8d1234baab3: Pulling fs layer
2024-06-26T07:06:35.1227817Z 7ed32bc8e469: Pulling fs layer
2024-06-26T07:06:35.1228321Z ec1e7978c1fe: Pulling fs layer
2024-06-26T07:06:35.1228816Z 309b872da805: Pulling fs layer
2024-06-26T07:06:35.1229306Z e82932cb002c: Pulling fs layer
2024-06-26T07:06:35.1229795Z ca9dd73e3e93: Pulling fs layer
2024-06-26T07:06:35.1230278Z 16d76a79d5d6: Pulling fs layer
2024-06-26T07:06:35.1230762Z 4bdf41c9ecbe: Pulling fs layer
2024-06-26T07:06:35.1231256Z f4863801e30c: Pulling fs layer
2024-06-26T07:06:35.1231719Z d37dd9ed8948: Pulling fs layer
2024-06-26T07:06:35.1232151Z a41a8d1c11c8: Waiting
2024-06-26T07:06:35.1232498Z cf2f34f225db: Pulling fs layer
2024-06-26T07:06:35.1232875Z df546d7b3e34: Pulling fs layer
2024-06-26T07:06:35.1233247Z 3d914950f8ad: Pulling fs layer
2024-06-26T07:06:35.1233639Z 0c1227890755: Waiting
2024-06-26T07:06:35.1233975Z 96d0e2032184: Pulling fs layer
2024-06-26T07:06:35.1234561Z 06e12ce59790: Pulling fs layer
2024-06-26T07:06:35.1234939Z ed712f63cde7: Pulling fs layer
2024-06-26T07:06:35.1235311Z a97a71ef8a4e: Pulling fs layer
2024-06-26T07:06:35.1235674Z b57676e46aee: Waiting
2024-06-26T07:06:35.1236011Z 25cd0c482ad6: Pulling fs layer
2024-06-26T07:06:35.1236386Z 9bf94bc45658: Pulling fs layer
2024-06-26T07:06:35.1236787Z de45a14c9b22: Pulling fs layer
2024-06-26T07:06:35.1237203Z e99241e77209: Pulling fs layer
2024-06-26T07:06:35.1237585Z 35e8f4e2a42f: Pulling fs layer
2024-06-26T07:06:35.1237999Z d527cbbb87e3: Waiting
2024-06-26T07:06:35.1238335Z a823ffa4c541: Pulling fs layer
2024-06-26T07:06:35.1238765Z ca9dd73e3e93: Waiting
2024-06-26T07:06:35.1239095Z e82932cb002c: Waiting
2024-06-26T07:06:35.1239481Z 44ba272eae20: Pulling fs layer
2024-06-26T07:06:35.1239840Z a8c1e85b5e14: Waiting
2024-06-26T07:06:35.1240204Z ec1e7978c1fe: Waiting
2024-06-26T07:06:35.1240531Z 309b872da805: Waiting
2024-06-26T07:06:35.1240860Z da02d278e524: Pulling fs layer
2024-06-26T07:06:35.1241294Z cf2f34f225db: Waiting
2024-06-26T07:06:35.1241608Z 16d76a79d5d6: Waiting
2024-06-26T07:06:35.1241927Z 73a0ce681c29: Pulling fs layer
2024-06-26T07:06:35.1242285Z a97a71ef8a4e: Waiting
2024-06-26T07:06:35.1242600Z df546d7b3e34: Waiting
2024-06-26T07:06:35.1242904Z e99241e77209: Waiting
2024-06-26T07:06:35.1243228Z 49737bd5ae70: Pulling fs layer
2024-06-26T07:06:35.1243586Z 25cd0c482ad6: Waiting
2024-06-26T07:06:35.1243906Z de45a14c9b22: Waiting
2024-06-26T07:06:35.1244279Z 35e8f4e2a42f: Waiting
2024-06-26T07:06:35.1244593Z d37dd9ed8948: Waiting
2024-06-26T07:06:35.1245172Z 9cd4c745f8e4: Pulling fs layer
2024-06-26T07:06:35.1245549Z b2950a036534: Pulling fs layer
2024-06-26T07:06:35.1245914Z 25e19d81d9fc: Pulling fs layer
2024-06-26T07:06:35.1246293Z 60156b40fcbe: Pulling fs layer
2024-06-26T07:06:35.1246660Z 7a868239227f: Pulling fs layer
2024-06-26T07:06:35.1247083Z 83911d7c7ecf: Pulling fs layer
2024-06-26T07:06:35.1247446Z a823ffa4c541: Waiting
2024-06-26T07:06:35.1247815Z b3c5d66c679f: Pulling fs layer
2024-06-26T07:06:35.1248224Z 73a0ce681c29: Waiting
2024-06-26T07:06:35.1248544Z 84b2c7a89982: Pulling fs layer
2024-06-26T07:06:35.1248977Z c1068d03a811: Pulling fs layer
2024-06-26T07:06:35.1249347Z 671b4a301d81: Pulling fs layer
2024-06-26T07:06:35.1249746Z 49737bd5ae70: Waiting
2024-06-26T07:06:35.1250088Z 67760cf5ec46: Pulling fs layer
2024-06-26T07:06:35.1250611Z 84b2c7a89982: Waiting
2024-06-26T07:06:35.1250998Z ddc9740bcea2: Pulling fs layer
2024-06-26T07:06:35.1251378Z fb2e97768560: Pulling fs layer
2024-06-26T07:06:35.1251745Z b3c5d66c679f: Waiting
2024-06-26T07:06:35.1252085Z b599c5a8c7cb: Pulling fs layer
2024-06-26T07:06:35.1252453Z 44ba272eae20: Waiting
2024-06-26T07:06:35.1252768Z b2950a036534: Waiting
2024-06-26T07:06:35.1253255Z e08e88c13668: Pulling fs layer
2024-06-26T07:06:35.1253624Z 25e19d81d9fc: Waiting
2024-06-26T07:06:35.1253954Z c1068d03a811: Waiting
2024-06-26T07:06:35.1254275Z 7a868239227f: Waiting
2024-06-26T07:06:35.1254606Z 12ba7431ff38: Pulling fs layer
2024-06-26T07:06:35.1254993Z cf7c2cb896cb: Pulling fs layer
2024-06-26T07:06:35.1255371Z 3a778ddfac2a: Pulling fs layer
2024-06-26T07:06:35.1255740Z 67760cf5ec46: Waiting
2024-06-26T07:06:35.1256120Z 7ed32bc8e469: Waiting
2024-06-26T07:06:35.1256447Z 236c8f0245f4: Pulling fs layer
2024-06-26T07:06:35.1256845Z 06e12ce59790: Waiting
2024-06-26T07:06:35.1257177Z f4863801e30c: Waiting
2024-06-26T07:06:35.1257510Z 4f4fb700ef54: Pulling fs layer
2024-06-26T07:06:35.1257928Z ed712f63cde7: Waiting
2024-06-26T07:06:35.1258258Z 83911d7c7ecf: Waiting
2024-06-26T07:06:35.1258656Z dda1ce302b9a: Pulling fs layer
2024-06-26T07:06:35.1259019Z 12ba7431ff38: Waiting
2024-06-26T07:06:35.1259367Z e797fe86caef: Pulling fs layer
2024-06-26T07:06:35.1259757Z 96d0e2032184: Waiting
2024-06-26T07:06:35.1260085Z 60156b40fcbe: Waiting
2024-06-26T07:06:35.1260457Z cf7c2cb896cb: Waiting
2024-06-26T07:06:35.1260776Z e797fe86caef: Waiting
2024-06-26T07:06:35.1261134Z 3d914950f8ad: Waiting
2024-06-26T07:06:35.1261601Z b599c5a8c7cb: Waiting
2024-06-26T07:06:35.1261943Z 236c8f0245f4: Waiting
2024-06-26T07:06:35.1262301Z 4f4fb700ef54: Waiting
2024-06-26T07:06:35.1262628Z 967190076fc0: Pulling fs layer
2024-06-26T07:06:35.1263070Z 4fb0a83f0685: Pulling fs layer
2024-06-26T07:06:35.1263438Z 68af2ec4e30a: Pulling fs layer
2024-06-26T07:06:35.1263877Z cc91933e985a: Pulling fs layer
2024-06-26T07:06:35.1264265Z 9f3187916e7d: Pulling fs layer
2024-06-26T07:06:35.1264672Z 3a778ddfac2a: Waiting
2024-06-26T07:06:35.1265004Z 526083994857: Pulling fs layer
2024-06-26T07:06:35.1265386Z 9cd4c745f8e4: Waiting
2024-06-26T07:06:35.1265734Z fb2e97768560: Waiting
2024-06-26T07:06:35.1266069Z 05a6f6dcc1c0: Pulling fs layer
2024-06-26T07:06:35.1266490Z 967190076fc0: Waiting
2024-06-26T07:06:35.1266812Z 4fb0a83f0685: Waiting
2024-06-26T07:06:35.1267258Z 6bc9dc8315b0: Pulling fs layer
2024-06-26T07:06:35.1267612Z cc91933e985a: Waiting
2024-06-26T07:06:35.1267942Z 68af2ec4e30a: Waiting
2024-06-26T07:06:35.1268302Z e08e88c13668: Waiting
2024-06-26T07:06:35.1268623Z 671b4a301d81: Waiting
2024-06-26T07:06:35.1268986Z dda1ce302b9a: Waiting
2024-06-26T07:06:35.1269300Z 9f3187916e7d: Waiting
2024-06-26T07:06:35.1269618Z 6bc9dc8315b0: Waiting
2024-06-26T07:06:35.1269997Z 1b0e27c35f07: Pulling fs layer
2024-06-26T07:06:35.1270376Z 95237e7311a1: Pulling fs layer
2024-06-26T07:06:35.1270791Z 1b0e27c35f07: Waiting
2024-06-26T07:06:35.1271122Z 789eff6c51aa: Pulling fs layer
2024-06-26T07:06:35.1271554Z de0f8a8c45c6: Pulling fs layer
2024-06-26T07:06:35.1271916Z 95237e7311a1: Waiting
2024-06-26T07:06:35.1272265Z 2060028f7488: Pulling fs layer
2024-06-26T07:06:35.1272664Z e412c7f3b2de: Pulling fs layer
2024-06-26T07:06:35.1273021Z 2060028f7488: Waiting
2024-06-26T07:06:35.1273397Z e412c7f3b2de: Waiting
2024-06-26T07:06:35.1273724Z 05a6f6dcc1c0: Waiting
2024-06-26T07:06:35.2819643Z 224fe954d725: Verifying Checksum
2024-06-26T07:06:35.2820114Z 224fe954d725: Download complete
2024-06-26T07:06:35.3944658Z d527cbbb87e3: Download complete
2024-06-26T07:06:35.4673577Z b57676e46aee: Verifying Checksum
2024-06-26T07:06:35.4674188Z b57676e46aee: Download complete
2024-06-26T07:06:35.5075838Z 7a2c55901189: Verifying Checksum
2024-06-26T07:06:35.5076487Z 7a2c55901189: Download complete
2024-06-26T07:06:35.5885345Z a41a8d1c11c8: Verifying Checksum
2024-06-26T07:06:35.5885823Z a41a8d1c11c8: Download complete
2024-06-26T07:06:35.6701616Z 0c1227890755: Download complete
2024-06-26T07:06:35.7391254Z 75722010b82e: Verifying Checksum
2024-06-26T07:06:35.7391877Z 75722010b82e: Download complete
2024-06-26T07:06:35.7434438Z d8d1234baab3: Verifying Checksum
2024-06-26T07:06:35.7434980Z d8d1234baab3: Download complete
2024-06-26T07:06:35.8205100Z ec1e7978c1fe: Download complete
2024-06-26T07:06:35.8890311Z 309b872da805: Download complete
2024-06-26T07:06:36.5468278Z 7a2c55901189: Pull complete
2024-06-26T07:06:36.8334454Z 224fe954d725: Pull complete
2024-06-26T07:06:37.7689001Z 75722010b82e: Pull complete
2024-06-26T07:06:37.8147729Z d527cbbb87e3: Pull complete
2024-06-26T07:06:37.8774092Z b57676e46aee: Pull complete
2024-06-26T07:06:38.4441540Z e82932cb002c: Verifying Checksum
2024-06-26T07:06:38.4442188Z e82932cb002c: Download complete
2024-06-26T07:06:38.5245848Z ca9dd73e3e93: Verifying Checksum
2024-06-26T07:06:38.5246276Z ca9dd73e3e93: Download complete
2024-06-26T07:06:38.6062417Z 16d76a79d5d6: Verifying Checksum
2024-06-26T07:06:38.6063055Z 16d76a79d5d6: Download complete
2024-06-26T07:06:38.6740203Z 4bdf41c9ecbe: Verifying Checksum
2024-06-26T07:06:38.6740829Z 4bdf41c9ecbe: Download complete
2024-06-26T07:06:39.6424147Z f4863801e30c: Verifying Checksum
2024-06-26T07:06:39.6424609Z f4863801e30c: Download complete
2024-06-26T07:06:39.7228154Z d37dd9ed8948: Download complete
2024-06-26T07:06:39.8143256Z cf2f34f225db: Verifying Checksum
2024-06-26T07:06:39.8143727Z cf2f34f225db: Download complete
2024-06-26T07:06:39.8966533Z df546d7b3e34: Verifying Checksum
2024-06-26T07:06:39.8966990Z df546d7b3e34: Download complete
2024-06-26T07:06:48.8226168Z a8c1e85b5e14: Verifying Checksum
2024-06-26T07:06:48.8227179Z a8c1e85b5e14: Download complete
2024-06-26T07:06:48.9104693Z 96d0e2032184: Download complete
2024-06-26T07:06:48.9809016Z 06e12ce59790: Download complete
2024-06-26T07:06:49.0660455Z ed712f63cde7: Verifying Checksum
2024-06-26T07:06:49.0661068Z ed712f63cde7: Download complete
2024-06-26T07:06:49.1451058Z a97a71ef8a4e: Download complete
2024-06-26T07:06:49.2421939Z 25cd0c482ad6: Verifying Checksum
2024-06-26T07:06:49.2422549Z 25cd0c482ad6: Download complete
2024-06-26T07:06:50.5148212Z 9bf94bc45658: Download complete
2024-06-26T07:06:50.6013641Z de45a14c9b22: Download complete
2024-06-26T07:06:50.6881222Z e99241e77209: Verifying Checksum
2024-06-26T07:06:50.6881728Z e99241e77209: Download complete
2024-06-26T07:06:50.7694648Z 35e8f4e2a42f: Download complete
2024-06-26T07:06:50.8403049Z a823ffa4c541: Download complete
2024-06-26T07:06:50.9170068Z 44ba272eae20: Verifying Checksum
2024-06-26T07:06:50.9170757Z 44ba272eae20: Download complete
2024-06-26T07:06:55.4424753Z da02d278e524: Verifying Checksum
2024-06-26T07:06:55.4425251Z da02d278e524: Download complete
2024-06-26T07:06:55.5223665Z 73a0ce681c29: Verifying Checksum
2024-06-26T07:06:55.5224100Z 73a0ce681c29: Download complete
2024-06-26T07:06:55.6070361Z 49737bd5ae70: Verifying Checksum
2024-06-26T07:06:55.6070785Z 49737bd5ae70: Download complete
2024-06-26T07:06:56.0317516Z 9cd4c745f8e4: Verifying Checksum
2024-06-26T07:06:56.0318058Z 9cd4c745f8e4: Download complete
2024-06-26T07:06:56.1019428Z b2950a036534: Verifying Checksum
2024-06-26T07:06:56.1019957Z b2950a036534: Download complete
2024-06-26T07:06:56.1905809Z 25e19d81d9fc: Verifying Checksum
2024-06-26T07:06:56.1906254Z 25e19d81d9fc: Download complete
2024-06-26T07:06:56.5012459Z 60156b40fcbe: Verifying Checksum
2024-06-26T07:06:56.5013077Z 60156b40fcbe: Download complete
2024-06-26T07:06:56.5858678Z 7a868239227f: Verifying Checksum
2024-06-26T07:06:56.5859136Z 7a868239227f: Download complete
2024-06-26T07:06:56.6637210Z 83911d7c7ecf: Verifying Checksum
2024-06-26T07:06:56.6637648Z 83911d7c7ecf: Download complete
2024-06-26T07:06:56.7384479Z b3c5d66c679f: Verifying Checksum
2024-06-26T07:06:56.7385028Z b3c5d66c679f: Download complete
2024-06-26T07:07:01.0766739Z 7ed32bc8e469: Verifying Checksum
2024-06-26T07:07:01.0767213Z 7ed32bc8e469: Download complete
2024-06-26T07:07:01.1491965Z c1068d03a811: Verifying Checksum
2024-06-26T07:07:01.1492434Z c1068d03a811: Download complete
2024-06-26T07:07:01.2359320Z 671b4a301d81: Verifying Checksum
2024-06-26T07:07:01.2359866Z 671b4a301d81: Download complete
2024-06-26T07:07:01.6062654Z a8c1e85b5e14: Pull complete
2024-06-26T07:07:01.7838474Z 67760cf5ec46: Verifying Checksum
2024-06-26T07:07:01.7838923Z 67760cf5ec46: Download complete
2024-06-26T07:07:01.8304514Z a41a8d1c11c8: Pull complete
2024-06-26T07:07:01.8734828Z ddc9740bcea2: Verifying Checksum
2024-06-26T07:07:01.8735314Z ddc9740bcea2: Download complete
2024-06-26T07:07:01.9491648Z fb2e97768560: Verifying Checksum
2024-06-26T07:07:01.9492075Z fb2e97768560: Download complete
2024-06-26T07:07:02.0265547Z b599c5a8c7cb: Download complete
2024-06-26T07:07:02.0572733Z 0c1227890755: Pull complete
2024-06-26T07:07:02.1046931Z e08e88c13668: Verifying Checksum
2024-06-26T07:07:02.1047403Z e08e88c13668: Download complete
2024-06-26T07:07:02.2759107Z d8d1234baab3: Pull complete
2024-06-26T07:07:07.7960552Z 3d914950f8ad: Verifying Checksum
2024-06-26T07:07:07.7961126Z 3d914950f8ad: Download complete
2024-06-26T07:07:07.8815441Z cf7c2cb896cb: Download complete
2024-06-26T07:07:07.9588061Z 3a778ddfac2a: Download complete
2024-06-26T07:07:08.0331987Z 236c8f0245f4: Verifying Checksum
2024-06-26T07:07:08.0332421Z 236c8f0245f4: Download complete
2024-06-26T07:07:08.0449326Z 4f4fb700ef54: Verifying Checksum
2024-06-26T07:07:08.0449784Z 4f4fb700ef54: Download complete
2024-06-26T07:07:08.1602829Z dda1ce302b9a: Verifying Checksum
2024-06-26T07:07:08.1603374Z dda1ce302b9a: Download complete
2024-06-26T07:07:08.2335447Z e797fe86caef: Verifying Checksum
2024-06-26T07:07:08.2335990Z e797fe86caef: Download complete
2024-06-26T07:07:08.4056111Z 967190076fc0: Verifying Checksum
2024-06-26T07:07:08.4056688Z 967190076fc0: Download complete
2024-06-26T07:07:08.4918662Z 4fb0a83f0685: Verifying Checksum
2024-06-26T07:07:08.4919091Z 4fb0a83f0685: Download complete
2024-06-26T07:07:08.5682510Z 68af2ec4e30a: Verifying Checksum
2024-06-26T07:07:08.5682956Z 68af2ec4e30a: Download complete
2024-06-26T07:07:08.6415140Z cc91933e985a: Download complete
2024-06-26T07:07:08.7263745Z 9f3187916e7d: Verifying Checksum
2024-06-26T07:07:08.7264222Z 9f3187916e7d: Download complete
2024-06-26T07:07:08.8666355Z 526083994857: Verifying Checksum
2024-06-26T07:07:08.8666787Z 526083994857: Download complete
2024-06-26T07:07:08.9401335Z 05a6f6dcc1c0: Verifying Checksum
2024-06-26T07:07:08.9401793Z 05a6f6dcc1c0: Download complete
2024-06-26T07:07:09.5526051Z 6bc9dc8315b0: Verifying Checksum
2024-06-26T07:07:09.5526518Z 6bc9dc8315b0: Download complete
2024-06-26T07:07:09.6426773Z 1b0e27c35f07: Download complete
2024-06-26T07:07:32.0690299Z 12ba7431ff38: Verifying Checksum
2024-06-26T07:07:32.0690856Z 12ba7431ff38: Download complete
2024-06-26T07:07:32.1407831Z 789eff6c51aa: Verifying Checksum
2024-06-26T07:07:32.1408530Z 789eff6c51aa: Download complete
2024-06-26T07:07:32.2281912Z de0f8a8c45c6: Download complete
2024-06-26T07:07:33.4232256Z 2060028f7488: Verifying Checksum
2024-06-26T07:07:33.4232807Z 2060028f7488: Download complete
2024-06-26T07:07:33.5092167Z e412c7f3b2de: Verifying Checksum
2024-06-26T07:07:33.5092607Z e412c7f3b2de: Download complete
2024-06-26T07:07:37.1688380Z 95237e7311a1: Verifying Checksum
2024-06-26T07:08:00.5103141Z 84b2c7a89982: Verifying Checksum
2024-06-26T07:08:00.5103623Z 84b2c7a89982: Download complete
2024-06-26T07:08:18.1137435Z 7ed32bc8e469: Pull complete
2024-06-26T07:08:18.3284575Z ec1e7978c1fe: Pull complete
2024-06-26T07:08:18.5327001Z 309b872da805: Pull complete
2024-06-26T07:08:25.8130449Z e82932cb002c: Pull complete
2024-06-26T07:08:26.0194426Z ca9dd73e3e93: Pull complete
2024-06-26T07:08:26.2308569Z 16d76a79d5d6: Pull complete
2024-06-26T07:08:26.4358339Z 4bdf41c9ecbe: Pull complete
2024-06-26T07:08:28.8500398Z f4863801e30c: Pull complete
2024-06-26T07:08:29.0580154Z d37dd9ed8948: Pull complete
2024-06-26T07:08:29.2595933Z cf2f34f225db: Pull complete
2024-06-26T07:08:29.4553114Z df546d7b3e34: Pull complete
2024-06-26T07:09:18.9224008Z 3d914950f8ad: Pull complete
2024-06-26T07:09:19.1182201Z 96d0e2032184: Pull complete
2024-06-26T07:09:19.2572979Z 06e12ce59790: Pull complete
2024-06-26T07:09:19.3653045Z ed712f63cde7: Pull complete
2024-06-26T07:09:19.4941237Z a97a71ef8a4e: Pull complete
2024-06-26T07:09:19.6168655Z 25cd0c482ad6: Pull complete
2024-06-26T07:09:23.1015455Z 9bf94bc45658: Pull complete
2024-06-26T07:09:23.2006021Z de45a14c9b22: Pull complete
2024-06-26T07:09:23.3269763Z e99241e77209: Pull complete
2024-06-26T07:09:23.5244601Z 35e8f4e2a42f: Pull complete
2024-06-26T07:09:23.7567186Z a823ffa4c541: Pull complete
2024-06-26T07:09:23.9694060Z 44ba272eae20: Pull complete
2024-06-26T07:09:32.1955226Z da02d278e524: Pull complete
2024-06-26T07:09:32.3556521Z 73a0ce681c29: Pull complete
2024-06-26T07:09:32.4847618Z 49737bd5ae70: Pull complete
2024-06-26T07:09:33.3855710Z 9cd4c745f8e4: Pull complete
2024-06-26T07:09:33.5150260Z b2950a036534: Pull complete
2024-06-26T07:09:33.6788644Z 25e19d81d9fc: Pull complete
2024-06-26T07:09:34.1687627Z 60156b40fcbe: Pull complete
2024-06-26T07:09:34.3693965Z 7a868239227f: Pull complete
2024-06-26T07:09:34.7196577Z 83911d7c7ecf: Pull complete
2024-06-26T07:09:34.9077793Z b3c5d66c679f: Pull complete
2024-06-26T07:10:13.6099293Z 84b2c7a89982: Pull complete
2024-06-26T07:10:13.7980580Z c1068d03a811: Pull complete
2024-06-26T07:10:13.8367983Z 671b4a301d81: Pull complete
2024-06-26T07:10:15.7988471Z 67760cf5ec46: Pull complete
2024-06-26T07:10:15.8551996Z ddc9740bcea2: Pull complete
2024-06-26T07:10:15.9094526Z fb2e97768560: Pull complete
2024-06-26T07:10:16.0111511Z b599c5a8c7cb: Pull complete
2024-06-26T07:10:16.0711440Z e08e88c13668: Pull complete
2024-06-26T07:10:50.0069373Z 12ba7431ff38: Pull complete
2024-06-26T07:10:50.2077610Z cf7c2cb896cb: Pull complete
2024-06-26T07:10:50.4038023Z 3a778ddfac2a: Pull complete
2024-06-26T07:10:50.8054593Z 236c8f0245f4: Pull complete
2024-06-26T07:10:50.9234787Z 4f4fb700ef54: Pull complete
2024-06-26T07:10:51.1454447Z dda1ce302b9a: Pull complete
2024-06-26T07:10:51.3652848Z e797fe86caef: Pull complete
2024-06-26T07:10:51.7579358Z 967190076fc0: Pull complete
2024-06-26T07:10:51.9742882Z 4fb0a83f0685: Pull complete
2024-06-26T07:10:52.2207528Z 68af2ec4e30a: Pull complete
2024-06-26T07:10:52.4391423Z cc91933e985a: Pull complete
2024-06-26T07:10:52.6637912Z 9f3187916e7d: Pull complete
2024-06-26T07:10:53.7251749Z 526083994857: Pull complete
2024-06-26T07:10:53.9521523Z 05a6f6dcc1c0: Pull complete
2024-06-26T07:10:55.9880547Z 6bc9dc8315b0: Pull complete
2024-06-26T07:10:56.2039520Z 1b0e27c35f07: Pull complete
2024-06-26T07:11:09.6392163Z 95237e7311a1: Pull complete
2024-06-26T07:11:09.7050287Z 789eff6c51aa: Pull complete
2024-06-26T07:11:09.7410208Z de0f8a8c45c6: Pull complete
2024-06-26T07:11:10.4913659Z 2060028f7488: Pull complete
2024-06-26T07:11:10.5978109Z e412c7f3b2de: Pull complete
2024-06-26T07:11:11.3394336Z Digest: sha256:216c88a4f79ebfa71a5dcf4d9a946548184354a3ea1f677f4b1065d28705debb
2024-06-26T07:11:11.3813357Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:11:11.4038218Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:11:11.4089078Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT"
2024-06-26T07:11:11.4089984Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT"
2024-06-26T07:11:11.4098080Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:11:11.4098567Z env:
2024-06-26T07:11:11.4098859Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:11:11.4099203Z ##[endgroup]
2024-06-26T07:11:11.4251397Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main
2024-06-26T07:11:11.4251959Z with:
2024-06-26T07:11:11.4252253Z   driver-version: 550.54.15
2024-06-26T07:11:11.4252605Z env:
2024-06-26T07:11:11.4252883Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:11:11.4253242Z ##[endgroup]
2024-06-26T07:11:11.4371078Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482
2024-06-26T07:11:11.4371640Z with:
2024-06-26T07:11:11.4371925Z   timeout_minutes: 10
2024-06-26T07:11:11.4372254Z   max_attempts: 3
2024-06-26T07:11:11.4402668Z   command: # Is it disgusting to have a full shell script here in this github action? Sure
# But is it the best way to make it so that this action relies on nothing else? Absolutely
set -eou pipefail

DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID)
DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run"

install_nvidia_docker2_amzn2() {
    (
        set -x
        # Needed for yum-config-manager
        sudo yum install -y yum-utils
        if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then
          YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo"
        else
          # Amazon Linux 2
          YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo"
        fi

        sudo yum-config-manager --add-repo "${YUM_REPO_URL}"
        sudo yum install -y nvidia-docker2
        sudo systemctl restart docker
    )
}

install_nvidia_docker2_ubuntu20() {
    (
        set -x
        # Install nvidia-driver package if not installed
        status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)"
        if [ ! $? = 0 ] || [ ! "$status" = installed ]; then
          sudo apt-get install -y nvidia-docker2
          sudo systemctl restart docker
        fi
    )
}

pre_install_nvidia_driver_amzn2() {
    (
        # Purge any nvidia driver installed from RHEL repo
        sudo yum remove -y nvidia-driver-latest-dkms
    )
}

install_nvidia_driver_common() {
    (
        # Try to gather more information about the runner and its existing NVIDIA driver if any
        echo "Before installing NVIDIA driver"
        lspci
        lsmod
        modinfo nvidia || true

        HAS_NVIDIA_DRIVER=0
        # Check if NVIDIA driver has already been installed
        if [ -x "$(command -v nvidia-smi)" ]; then
            set +e
            # The driver exists, check its version next. Also check only the first GPU if there are more than one of them
            # so that the same driver version is not print over multiple lines
            INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0)
            NVIDIA_SMI_STATUS=$?

            if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then
                echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing"
            elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then
                echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing"
            else
                HAS_NVIDIA_DRIVER=1
                echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation"
            fi
            set -e
        fi

        if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then
            # CAUTION: this may need to be updated in future
            if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then
                  sudo yum groupinstall -y "Development Tools"
                  # ensure our kernel install is the same as our underlying kernel,
                  # groupinstall "Development Tools" has a habit of mismatching kernel headers
                  sudo yum install -y "kernel-devel-uname-r == $(uname -r)"
                  sudo modprobe backlight
            fi
            sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN"

            set +e
            sudo /bin/bash /tmp/nvidia_driver -s --no-drm
            NVIDIA_INSTALLATION_STATUS=$?

            RESET_GPU=0
            if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then
                sudo cat /var/log/nvidia-installer.log
                # Fail to install NVIDIA driver, try to reset the GPU
                RESET_GPU=1
            elif [ -x "$(command -v nvidia-smi)" ]; then
                # Check again if nvidia-smi works even if the driver installation completes successfully
                INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0)
                NVIDIA_SMI_STATUS=$?

                if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then
                    RESET_GPU=1
                fi
            fi

            if [ "$RESET_GPU" -eq 1 ]; then
                NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1)
                # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this
                # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388
                for PCI_ID in $NVIDIA_DEVICES; do
                    DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable)

                    echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)"
                    # This requires sudo permission of course
                    echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset
                    sleep 1
                done
            fi

            sudo rm -fv /tmp/nvidia_driver
            set -e
        fi
    )
}

post_install_nvidia_driver_common() {
    (
        sudo modprobe nvidia || true
        echo "After installing NVIDIA driver"
        lspci
        lsmod
        modinfo nvidia || true

        (
            set +e

            nvidia-smi
            # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in
            # the case where the driver has already crashed as it still can get the driver version
            # and some basic information like the bus ID.  However, the rest of the information
            # would be missing (ERR!), for example:
            #
            # +-----------------------------------------------------------------------------+
            # | NVIDIA-SMI 525.89.02    Driver Version: 525.89.02    CUDA Version: 12.0     |
            # |-------------------------------+----------------------+----------------------+
            # | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
            # | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
            # |                               |                      |               MIG M. |
            # |===============================+======================+======================|
            # |   0  ERR!                Off  | 00000000:00:1E.0 Off |                 ERR! |
            # |ERR!  ERR! ERR!    ERR! / ERR! |   4184MiB / 23028MiB |    ERR!      Default |
            # |                               |                      |                 ERR! |
            # +-------------------------------+----------------------+----------------------+
            #
            # +-----------------------------------------------------------------------------+
            # | Processes:                                                                  |
            # |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
            # |        ID   ID                                                   Usage      |
            # |=============================================================================|
            # +-----------------------------------------------------------------------------+
            #
            # This should be reported as a failure instead as it will guarantee to fail when
            # Docker tries to run with --gpus all
            #
            # So, the correct check here is to query one of the missing piece of info like
            # GPU name, so that the command can fail accordingly
            nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0
            NVIDIA_SMI_STATUS=$?

            # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285
            if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then
                echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}"
            else
                echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}"
                exit ${NVIDIA_SMI_STATUS}
            fi
            set -e
        )
    )
}

install_nvidia_driver_amzn2() {
    (
        set -x
        pre_install_nvidia_driver_amzn2
        install_nvidia_driver_common
        post_install_nvidia_driver_common
    )
}

install_nvidia_driver_ubuntu20() {
    (
        set -x
        install_nvidia_driver_common
        post_install_nvidia_driver_common
    )
}

echo "== Installing nvidia driver ${DRIVER_FN} =="
case "${DISTRIBUTION}" in
    amzn*)
        install_nvidia_driver_amzn2
        ;;
    ubuntu20.04)
        install_nvidia_driver_ubuntu20
        ;;
    *)
        echo "ERROR: Unknown distribution ${DISTRIBUTION}"
        exit 1
        ;;
esac

# Install container toolkit based on distribution
echo "== Installing nvidia container toolkit for ${DISTRIBUTION} =="
case "${DISTRIBUTION}" in
    amzn*)
        install_nvidia_docker2_amzn2
        ;;
    ubuntu20.04)
        install_nvidia_docker2_ubuntu20
        ;;
    *)
        echo "ERROR: Unknown distribution ${DISTRIBUTION}"
        exit 1
        ;;
esac
echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}"

2024-06-26T07:11:11.4433465Z   retry_wait_seconds: 10
2024-06-26T07:11:11.4433837Z   polling_interval_seconds: 1
2024-06-26T07:11:11.4434213Z   warning_on_retry: true
2024-06-26T07:11:11.4434578Z   continue_on_error: false
2024-06-26T07:11:11.4434999Z env:
2024-06-26T07:11:11.4435279Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:11:11.4435643Z   DRIVER_VERSION: 550.54.15
2024-06-26T07:11:11.4436011Z ##[endgroup]
2024-06-26T07:11:11.4949658Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run ==
2024-06-26T07:11:11.4950486Z + pre_install_nvidia_driver_amzn2
2024-06-26T07:11:11.4951005Z + sudo yum remove -y nvidia-driver-latest-dkms
2024-06-26T07:11:11.7982337Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
2024-06-26T07:11:11.8373355Z No Match for argument: nvidia-driver-latest-dkms
2024-06-26T07:11:11.8614023Z No Packages marked for removal
2024-06-26T07:11:11.8728737Z + install_nvidia_driver_common
2024-06-26T07:11:11.8732141Z + echo 'Before installing NVIDIA driver'
2024-06-26T07:11:11.8732931Z + lspci
2024-06-26T07:11:11.8735011Z Before installing NVIDIA driver
2024-06-26T07:11:11.8812993Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma]
2024-06-26T07:11:11.8814106Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
2024-06-26T07:11:11.8815488Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
2024-06-26T07:11:11.8816410Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111
2024-06-26T07:11:11.8817426Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. Device 8061
2024-06-26T07:11:11.8818199Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA)
2024-06-26T07:11:11.8818923Z 00:1e.0 3D controller: NVIDIA Corporation Device 2237 (rev a1)
2024-06-26T07:11:11.8819712Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller
2024-06-26T07:11:11.8820308Z + lsmod
2024-06-26T07:11:11.8828585Z Module                  Size  Used by
2024-06-26T07:11:11.8829225Z xt_nat                 16384  0
2024-06-26T07:11:11.8829725Z ib_core               266240  0
2024-06-26T07:11:11.8830112Z nvidia_modeset       1339392  0
2024-06-26T07:11:11.8830490Z veth                   16384  0
2024-06-26T07:11:11.8830871Z nvidia_uvm           4599808  0
2024-06-26T07:11:11.8831512Z nvidia              53989376  2 nvidia_uvm,nvidia_modeset
2024-06-26T07:11:11.8832004Z drm                   421888  1 nvidia
2024-06-26T07:11:11.8832432Z i2c_core               77824  2 nvidia,drm
2024-06-26T07:11:11.8832893Z backlight              16384  1 nvidia_modeset
2024-06-26T07:11:11.8833335Z xt_conntrack           16384  1
2024-06-26T07:11:11.8834872Z ipt_MASQUERADE         16384  1
2024-06-26T07:11:11.8835508Z nf_nat_masquerade_ipv4    16384  1 ipt_MASQUERADE
2024-06-26T07:11:11.8836160Z nf_conntrack_netlink    49152  0
2024-06-26T07:11:11.8836790Z nfnetlink              16384  2 nf_conntrack_netlink
2024-06-26T07:11:11.8837430Z xfrm_user              45056  1
2024-06-26T07:11:11.8837961Z xfrm_algo              16384  1 xfrm_user
2024-06-26T07:11:11.8838509Z iptable_nat            16384  1
2024-06-26T07:11:11.8839001Z nf_conntrack_ipv4      16384  3
2024-06-26T07:11:11.8839440Z nf_defrag_ipv4         16384  1 nf_conntrack_ipv4
2024-06-26T07:11:11.8839961Z nf_nat_ipv4            16384  1 iptable_nat
2024-06-26T07:11:11.8840519Z nf_nat                 32768  3 nf_nat_masquerade_ipv4,nf_nat_ipv4,xt_nat
2024-06-26T07:11:11.8841559Z nf_conntrack          155648  8 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,xt_nat,nf_conntrack_netlink
2024-06-26T07:11:11.8842427Z xt_addrtype            16384  2
2024-06-26T07:11:11.8842854Z iptable_filter         16384  1
2024-06-26T07:11:11.8843274Z br_netfilter           24576  0
2024-06-26T07:11:11.8843682Z bridge                172032  1 br_netfilter
2024-06-26T07:11:11.8844131Z stp                    16384  1 bridge
2024-06-26T07:11:11.8844593Z llc                    16384  2 bridge,stp
2024-06-26T07:11:11.8845386Z overlay                86016  0
2024-06-26T07:11:11.8845759Z sunrpc                393216  1
2024-06-26T07:11:11.8846137Z dm_mirror              28672  0
2024-06-26T07:11:11.8846538Z dm_region_hash         20480  1 dm_mirror
2024-06-26T07:11:11.8847186Z dm_log                 20480  2 dm_region_hash,dm_mirror
2024-06-26T07:11:11.8847700Z dm_mod                143360  2 dm_log,dm_mirror
2024-06-26T07:11:11.8848157Z dax                    69632  1 dm_mod
2024-06-26T07:11:11.8848564Z crc32_pclmul           16384  0
2024-06-26T07:11:11.8848951Z ghash_clmulni_intel    16384  0
2024-06-26T07:11:11.8849331Z pcbc                   16384  0
2024-06-26T07:11:11.8849707Z aesni_intel           188416  0
2024-06-26T07:11:11.8850107Z aes_x86_64             20480  1 aesni_intel
2024-06-26T07:11:11.8850636Z crypto_simd            16384  1 aesni_intel
2024-06-26T07:11:11.8851196Z glue_helper            16384  1 aesni_intel
2024-06-26T07:11:11.8851805Z cryptd                 28672  3 crypto_simd,ghash_clmulni_intel,aesni_intel
2024-06-26T07:11:11.8852351Z mousedev               24576  0
2024-06-26T07:11:11.8852722Z evdev                  20480  3
2024-06-26T07:11:11.8853100Z psmouse                32768  0
2024-06-26T07:11:11.8853478Z button                 16384  0
2024-06-26T07:11:11.8853855Z ena                   139264  0
2024-06-26T07:11:11.8854227Z ptp                    20480  1 ena
2024-06-26T07:11:11.8854627Z pps_core               20480  1 ptp
2024-06-26T07:11:11.8855025Z crc32c_intel           24576  0
2024-06-26T07:11:11.8855401Z autofs4                49152  2
2024-06-26T07:11:11.8855757Z + modinfo nvidia
2024-06-26T07:11:11.8856422Z filename:       /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko
2024-06-26T07:11:11.8857121Z alias:          char-major-195-*
2024-06-26T07:11:11.8857525Z version:        550.54.15
2024-06-26T07:11:11.8857913Z supported:      external
2024-06-26T07:11:11.8858268Z license:        NVIDIA
2024-06-26T07:11:11.8858655Z firmware:       nvidia/550.54.15/gsp_tu10x.bin
2024-06-26T07:11:11.8859133Z firmware:       nvidia/550.54.15/gsp_ga10x.bin
2024-06-26T07:11:11.8859599Z srcversion:     833721318DA517F0C2FEC97
2024-06-26T07:11:11.8860074Z alias:          pci:v000010DEd*sv*sd*bc06sc80i00*
2024-06-26T07:11:11.8860588Z alias:          pci:v000010DEd*sv*sd*bc03sc02i00*
2024-06-26T07:11:11.8861198Z alias:          pci:v000010DEd*sv*sd*bc03sc00i00*
2024-06-26T07:11:11.8861701Z depends:        i2c-core,drm
2024-06-26T07:11:11.8862069Z retpoline:      Y
2024-06-26T07:11:11.8862383Z name:           nvidia
2024-06-26T07:11:11.8862938Z vermagic:       4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 
2024-06-26T07:11:11.8863600Z parm:           NvSwitchRegDwords:NvSwitch regkey (charp)
2024-06-26T07:11:11.8864253Z parm:           NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp)
2024-06-26T07:11:11.8864841Z parm:           NVreg_ResmanDebugLevel:int
2024-06-26T07:11:11.8865289Z parm:           NVreg_RmLogonRC:int
2024-06-26T07:11:11.8865726Z parm:           NVreg_ModifyDeviceFiles:int
2024-06-26T07:11:11.8866185Z parm:           NVreg_DeviceFileUID:int
2024-06-26T07:11:11.8866617Z parm:           NVreg_DeviceFileGID:int
2024-06-26T07:11:11.8867065Z parm:           NVreg_DeviceFileMode:int
2024-06-26T07:11:11.8867608Z parm:           NVreg_InitializeSystemMemoryAllocations:int
2024-06-26T07:11:11.8868189Z parm:           NVreg_UsePageAttributeTable:int
2024-06-26T07:11:11.8868670Z parm:           NVreg_EnablePCIeGen3:int
2024-06-26T07:11:11.8869110Z parm:           NVreg_EnableMSI:int
2024-06-26T07:11:11.8869537Z parm:           NVreg_TCEBypassMode:int
2024-06-26T07:11:11.8870002Z parm:           NVreg_EnableStreamMemOPs:int
2024-06-26T07:11:11.8870527Z parm:           NVreg_RestrictProfilingToAdminUsers:int
2024-06-26T07:11:11.8871113Z parm:           NVreg_PreserveVideoMemoryAllocations:int
2024-06-26T07:11:11.8871683Z parm:           NVreg_EnableS0ixPowerManagement:int
2024-06-26T07:11:11.8872302Z parm:           NVreg_S0ixPowerManagementVideoMemoryThreshold:int
2024-06-26T07:11:11.8872903Z parm:           NVreg_DynamicPowerManagement:int
2024-06-26T07:11:11.8873523Z parm:           NVreg_DynamicPowerManagementVideoMemoryThreshold:int
2024-06-26T07:11:11.8874121Z parm:           NVreg_EnableGpuFirmware:int
2024-06-26T07:11:11.8874702Z parm:           NVreg_EnableGpuFirmwareLogs:int
2024-06-26T07:11:11.8875241Z parm:           NVreg_OpenRmEnableUnsupportedGpus:int
2024-06-26T07:11:11.8875796Z parm:           NVreg_EnableUserNUMAManagement:int
2024-06-26T07:11:11.8876297Z parm:           NVreg_MemoryPoolSize:int
2024-06-26T07:11:11.8876768Z parm:           NVreg_KMallocHeapMaxSize:int
2024-06-26T07:11:11.8877250Z parm:           NVreg_VMallocHeapMaxSize:int
2024-06-26T07:11:11.8877725Z parm:           NVreg_IgnoreMMIOCheck:int
2024-06-26T07:11:11.8878182Z parm:           NVreg_NvLinkDisable:int
2024-06-26T07:11:11.8878683Z parm:           NVreg_EnablePCIERelaxedOrderingMode:int
2024-06-26T07:11:11.8879214Z parm:           NVreg_RegisterPCIDriver:int
2024-06-26T07:11:11.8879700Z parm:           NVreg_EnableResizableBar:int
2024-06-26T07:11:11.8880195Z parm:           NVreg_EnableDbgBreakpoint:int
2024-06-26T07:11:11.8880695Z parm:           NVreg_EnableNonblockingOpen:int
2024-06-26T07:11:11.8881183Z parm:           NVreg_RegistryDwords:charp
2024-06-26T07:11:11.8881973Z parm:           NVreg_RegistryDwordsPerDevice:charp
2024-06-26T07:11:11.8882459Z parm:           NVreg_RmMsg:charp
2024-06-26T07:11:11.8882869Z parm:           NVreg_GpuBlacklist:charp
2024-06-26T07:11:11.8883338Z parm:           NVreg_TemporaryFilePath:charp
2024-06-26T07:11:11.8883813Z parm:           NVreg_ExcludedGpus:charp
2024-06-26T07:11:11.8884269Z parm:           NVreg_DmaRemapPeerMmio:int
2024-06-26T07:11:11.8884896Z parm:           NVreg_RmNvlinkBandwidth:charp
2024-06-26T07:11:11.8885448Z parm:           NVreg_ImexChannelCount:int
2024-06-26T07:11:11.8885904Z parm:           rm_firmware_active:charp
2024-06-26T07:11:11.8886315Z + HAS_NVIDIA_DRIVER=0
2024-06-26T07:11:11.8886731Z ++ command -v nvidia-smi
2024-06-26T07:11:11.8887132Z + '[' -x /usr/bin/nvidia-smi ']'
2024-06-26T07:11:11.8887502Z + set +e
2024-06-26T07:11:11.8888002Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0
2024-06-26T07:11:13.8636933Z + INSTALLED_DRIVER_VERSION=550.54.15
2024-06-26T07:11:13.8637596Z + NVIDIA_SMI_STATUS=0
2024-06-26T07:11:13.8638597Z + '[' 0 -ne 0 ']'
2024-06-26T07:11:13.8639148Z + '[' 550.54.15 '!=' 550.54.15 ']'
2024-06-26T07:11:13.8639689Z + HAS_NVIDIA_DRIVER=1
2024-06-26T07:11:13.8640625Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation'
2024-06-26T07:11:13.8641470Z + set -e
2024-06-26T07:11:13.8641807Z + '[' 1 -eq 0 ']'
2024-06-26T07:11:13.8642380Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation
2024-06-26T07:11:13.8643059Z + post_install_nvidia_driver_common
2024-06-26T07:11:13.8643459Z + sudo modprobe nvidia
2024-06-26T07:11:13.8737940Z + echo 'After installing NVIDIA driver'
2024-06-26T07:11:13.8738409Z + lspci
2024-06-26T07:11:13.8739393Z After installing NVIDIA driver
2024-06-26T07:11:13.8816213Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma]
2024-06-26T07:11:13.8817009Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
2024-06-26T07:11:13.8817966Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
2024-06-26T07:11:13.8818812Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111
2024-06-26T07:11:13.8819562Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. Device 8061
2024-06-26T07:11:13.8820302Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA)
2024-06-26T07:11:13.8821014Z 00:1e.0 3D controller: NVIDIA Corporation Device 2237 (rev a1)
2024-06-26T07:11:13.8821794Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller
2024-06-26T07:11:13.8822377Z + lsmod
2024-06-26T07:11:13.8830022Z Module                  Size  Used by
2024-06-26T07:11:13.8830447Z xt_nat                 16384  0
2024-06-26T07:11:13.8830821Z ib_core               266240  0
2024-06-26T07:11:13.8831197Z nvidia_modeset       1339392  0
2024-06-26T07:11:13.8831566Z veth                   16384  0
2024-06-26T07:11:13.8832178Z nvidia_uvm           4599808  0
2024-06-26T07:11:13.8832634Z nvidia              53989376  2 nvidia_uvm,nvidia_modeset
2024-06-26T07:11:13.8833114Z drm                   421888  1 nvidia
2024-06-26T07:11:13.8833538Z i2c_core               77824  2 nvidia,drm
2024-06-26T07:11:13.8834034Z backlight              16384  1 nvidia_modeset
2024-06-26T07:11:13.8834478Z xt_conntrack           16384  1
2024-06-26T07:11:13.8834856Z ipt_MASQUERADE         16384  1
2024-06-26T07:11:13.8835283Z nf_nat_masquerade_ipv4    16384  1 ipt_MASQUERADE
2024-06-26T07:11:13.8835750Z nf_conntrack_netlink    49152  0
2024-06-26T07:11:13.8836196Z nfnetlink              16384  2 nf_conntrack_netlink
2024-06-26T07:11:13.8836653Z xfrm_user              45056  1
2024-06-26T07:11:13.8837043Z xfrm_algo              16384  1 xfrm_user
2024-06-26T07:11:13.8837458Z iptable_nat            16384  1
2024-06-26T07:11:13.8837836Z nf_conntrack_ipv4      16384  3
2024-06-26T07:11:13.8838259Z nf_defrag_ipv4         16384  1 nf_conntrack_ipv4
2024-06-26T07:11:13.8838796Z nf_nat_ipv4            16384  1 iptable_nat
2024-06-26T07:11:13.8839344Z nf_nat                 32768  3 nf_nat_masquerade_ipv4,nf_nat_ipv4,xt_nat
2024-06-26T07:11:13.8840337Z nf_conntrack          155648  8 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,xt_nat,nf_conntrack_netlink
2024-06-26T07:11:13.8841186Z xt_addrtype            16384  2
2024-06-26T07:11:13.8841563Z iptable_filter         16384  1
2024-06-26T07:11:13.8841943Z br_netfilter           24576  0
2024-06-26T07:11:13.8842357Z bridge                172032  1 br_netfilter
2024-06-26T07:11:13.8842790Z stp                    16384  1 bridge
2024-06-26T07:11:13.8843206Z llc                    16384  2 bridge,stp
2024-06-26T07:11:13.8843622Z overlay                86016  0
2024-06-26T07:11:13.8843994Z sunrpc                393216  1
2024-06-26T07:11:13.8844350Z dm_mirror              28672  0
2024-06-26T07:11:13.8844867Z dm_region_hash         20480  1 dm_mirror
2024-06-26T07:11:13.8845364Z dm_log                 20480  2 dm_region_hash,dm_mirror
2024-06-26T07:11:13.8845993Z dm_mod                143360  2 dm_log,dm_mirror
2024-06-26T07:11:13.8846438Z dax                    69632  1 dm_mod
2024-06-26T07:11:13.8846840Z crc32_pclmul           16384  0
2024-06-26T07:11:13.8847224Z ghash_clmulni_intel    16384  0
2024-06-26T07:11:13.8847666Z pcbc                   16384  0
2024-06-26T07:11:13.8848034Z aesni_intel           188416  0
2024-06-26T07:11:13.8848432Z aes_x86_64             20480  1 aesni_intel
2024-06-26T07:11:13.8849028Z crypto_simd            16384  1 aesni_intel
2024-06-26T07:11:13.8849576Z glue_helper            16384  1 aesni_intel
2024-06-26T07:11:13.8850233Z cryptd                 28672  3 crypto_simd,ghash_clmulni_intel,aesni_intel
2024-06-26T07:11:13.8850867Z mousedev               24576  0
2024-06-26T07:11:13.8851237Z evdev                  20480  3
2024-06-26T07:11:13.8851595Z psmouse                32768  0
2024-06-26T07:11:13.8851959Z button                 16384  0
2024-06-26T07:11:13.8852326Z ena                   139264  0
2024-06-26T07:11:13.8852701Z ptp                    20480  1 ena
2024-06-26T07:11:13.8853095Z pps_core               20480  1 ptp
2024-06-26T07:11:13.8853486Z crc32c_intel           24576  0
2024-06-26T07:11:13.8853851Z autofs4                49152  2
2024-06-26T07:11:13.8854199Z + modinfo nvidia
2024-06-26T07:11:13.8854882Z filename:       /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko
2024-06-26T07:11:13.8855569Z alias:          char-major-195-*
2024-06-26T07:11:13.8855952Z version:        550.54.15
2024-06-26T07:11:13.8856295Z supported:      external
2024-06-26T07:11:13.8856625Z license:        NVIDIA
2024-06-26T07:11:13.8857003Z firmware:       nvidia/550.54.15/gsp_tu10x.bin
2024-06-26T07:11:13.8857487Z firmware:       nvidia/550.54.15/gsp_ga10x.bin
2024-06-26T07:11:13.8857935Z srcversion:     833721318DA517F0C2FEC97
2024-06-26T07:11:13.8858402Z alias:          pci:v000010DEd*sv*sd*bc06sc80i00*
2024-06-26T07:11:13.8858900Z alias:          pci:v000010DEd*sv*sd*bc03sc02i00*
2024-06-26T07:11:13.8859478Z alias:          pci:v000010DEd*sv*sd*bc03sc00i00*
2024-06-26T07:11:13.8859951Z depends:        i2c-core,drm
2024-06-26T07:11:13.8860309Z retpoline:      Y
2024-06-26T07:11:13.8860618Z name:           nvidia
2024-06-26T07:11:13.8861165Z vermagic:       4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 
2024-06-26T07:11:13.8861806Z parm:           NvSwitchRegDwords:NvSwitch regkey (charp)
2024-06-26T07:11:13.8862455Z parm:           NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp)
2024-06-26T07:11:13.8863047Z parm:           NVreg_ResmanDebugLevel:int
2024-06-26T07:11:13.8863484Z parm:           NVreg_RmLogonRC:int
2024-06-26T07:11:13.8863910Z parm:           NVreg_ModifyDeviceFiles:int
2024-06-26T07:11:13.8864364Z parm:           NVreg_DeviceFileUID:int
2024-06-26T07:11:13.8864796Z parm:           NVreg_DeviceFileGID:int
2024-06-26T07:11:13.8865233Z parm:           NVreg_DeviceFileMode:int
2024-06-26T07:11:13.8865760Z parm:           NVreg_InitializeSystemMemoryAllocations:int
2024-06-26T07:11:13.8866328Z parm:           NVreg_UsePageAttributeTable:int
2024-06-26T07:11:13.8866806Z parm:           NVreg_EnablePCIeGen3:int
2024-06-26T07:11:13.8867231Z parm:           NVreg_EnableMSI:int
2024-06-26T07:11:13.8867649Z parm:           NVreg_TCEBypassMode:int
2024-06-26T07:11:13.8868104Z parm:           NVreg_EnableStreamMemOPs:int
2024-06-26T07:11:13.8868630Z parm:           NVreg_RestrictProfilingToAdminUsers:int
2024-06-26T07:11:13.8869203Z parm:           NVreg_PreserveVideoMemoryAllocations:int
2024-06-26T07:11:13.8869759Z parm:           NVreg_EnableS0ixPowerManagement:int
2024-06-26T07:11:13.8870362Z parm:           NVreg_S0ixPowerManagementVideoMemoryThreshold:int
2024-06-26T07:11:13.8870959Z parm:           NVreg_DynamicPowerManagement:int
2024-06-26T07:11:13.8871563Z parm:           NVreg_DynamicPowerManagementVideoMemoryThreshold:int
2024-06-26T07:11:13.8872150Z parm:           NVreg_EnableGpuFirmware:int
2024-06-26T07:11:13.8872648Z parm:           NVreg_EnableGpuFirmwareLogs:int
2024-06-26T07:11:13.8873231Z parm:           NVreg_OpenRmEnableUnsupportedGpus:int
2024-06-26T07:11:13.8873772Z parm:           NVreg_EnableUserNUMAManagement:int
2024-06-26T07:11:13.8874265Z parm:           NVreg_MemoryPoolSize:int
2024-06-26T07:11:13.8874729Z parm:           NVreg_KMallocHeapMaxSize:int
2024-06-26T07:11:13.8875214Z parm:           NVreg_VMallocHeapMaxSize:int
2024-06-26T07:11:13.8875680Z parm:           NVreg_IgnoreMMIOCheck:int
2024-06-26T07:11:13.8876130Z parm:           NVreg_NvLinkDisable:int
2024-06-26T07:11:13.8876631Z parm:           NVreg_EnablePCIERelaxedOrderingMode:int
2024-06-26T07:11:13.8877153Z parm:           NVreg_RegisterPCIDriver:int
2024-06-26T07:11:13.8877619Z parm:           NVreg_EnableResizableBar:int
2024-06-26T07:11:13.8878108Z parm:           NVreg_EnableDbgBreakpoint:int
2024-06-26T07:11:13.8878607Z parm:           NVreg_EnableNonblockingOpen:int
2024-06-26T07:11:13.8879082Z parm:           NVreg_RegistryDwords:charp
2024-06-26T07:11:13.8879594Z parm:           NVreg_RegistryDwordsPerDevice:charp
2024-06-26T07:11:13.8880067Z parm:           NVreg_RmMsg:charp
2024-06-26T07:11:13.8880478Z parm:           NVreg_GpuBlacklist:charp
2024-06-26T07:11:13.8880935Z parm:           NVreg_TemporaryFilePath:charp
2024-06-26T07:11:13.8881402Z parm:           NVreg_ExcludedGpus:charp
2024-06-26T07:11:13.8881855Z parm:           NVreg_DmaRemapPeerMmio:int
2024-06-26T07:11:13.8882332Z parm:           NVreg_RmNvlinkBandwidth:charp
2024-06-26T07:11:13.8882795Z parm:           NVreg_ImexChannelCount:int
2024-06-26T07:11:13.8883249Z parm:           rm_firmware_active:charp
2024-06-26T07:11:13.8883644Z + set +e
2024-06-26T07:11:13.8883940Z + nvidia-smi
2024-06-26T07:11:15.4433792Z Wed Jun 26 07:11:15 2024       
2024-06-26T07:11:15.4434601Z +-----------------------------------------------------------------------------------------+
2024-06-26T07:11:15.4435438Z | NVIDIA-SMI 550.54.15              Driver Version: 550.54.15      CUDA Version: 12.4     |
2024-06-26T07:11:15.4436478Z |-----------------------------------------+------------------------+----------------------+
2024-06-26T07:11:15.4437262Z | GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
2024-06-26T07:11:15.4438132Z | Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
2024-06-26T07:11:15.4438813Z |                                         |                        |               MIG M. |
2024-06-26T07:11:15.4439361Z |=========================================+========================+======================|
2024-06-26T07:11:15.4526404Z |   0  NVIDIA A10G                    Off |   00000000:00:1E.0 Off |                    0 |
2024-06-26T07:11:15.4527145Z |  0%   35C    P0             60W /  300W |       0MiB /  23028MiB |      0%      Default |
2024-06-26T07:11:15.4527766Z |                                         |                        |                  N/A |
2024-06-26T07:11:15.4528425Z +-----------------------------------------+------------------------+----------------------+
2024-06-26T07:11:15.4529029Z                                                                                          
2024-06-26T07:11:15.4529684Z +-----------------------------------------------------------------------------------------+
2024-06-26T07:11:15.4530342Z | Processes:                                                                              |
2024-06-26T07:11:15.4531175Z |  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
2024-06-26T07:11:15.4531867Z |        ID   ID                                                               Usage      |
2024-06-26T07:11:15.4532434Z |=========================================================================================|
2024-06-26T07:11:15.4533087Z |  No running processes found                                                             |
2024-06-26T07:11:15.4533823Z +-----------------------------------------------------------------------------------------+
2024-06-26T07:11:16.0649005Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0
2024-06-26T07:11:17.6252019Z NVIDIA A10G
2024-06-26T07:11:18.0682010Z + NVIDIA_SMI_STATUS=0
2024-06-26T07:11:18.0682658Z + '[' 0 -eq 0 ']'
2024-06-26T07:11:18.0683063Z + echo 'INFO: Ignoring allowed status 0'
2024-06-26T07:11:18.0683505Z + set -e
2024-06-26T07:11:18.0683812Z INFO: Ignoring allowed status 0
2024-06-26T07:11:18.0686063Z == Installing nvidia container toolkit for amzn2 ==
2024-06-26T07:11:18.0688386Z + sudo yum install -y yum-utils
2024-06-26T07:11:18.3690775Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
2024-06-26T07:11:19.8633701Z Package yum-utils-1.1.31-46.amzn2.0.1.noarch already installed and latest version
2024-06-26T07:11:19.8635045Z Nothing to do
2024-06-26T07:11:20.0137088Z + [[ amzn2 == \a\m\z\n\2\0\2\3 ]]
2024-06-26T07:11:20.0138078Z + YUM_REPO_URL=https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo
2024-06-26T07:11:20.0139438Z + sudo yum-config-manager --add-repo https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo
2024-06-26T07:11:20.3332100Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
2024-06-26T07:11:20.3575421Z adding repo from: https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo
2024-06-26T07:11:20.3576699Z grabbing file https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo to /etc/yum.repos.d/nvidia-docker.repo
2024-06-26T07:11:20.3577642Z repo saved to /etc/yum.repos.d/nvidia-docker.repo
2024-06-26T07:11:20.3680689Z + sudo yum install -y nvidia-docker2
2024-06-26T07:11:20.6791585Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
2024-06-26T07:11:22.1339255Z Package nvidia-docker2-2.13.0-1.noarch already installed and latest version
2024-06-26T07:11:22.1340066Z Nothing to do
2024-06-26T07:11:22.2826400Z + sudo systemctl restart docker
2024-06-26T07:11:58.5428210Z Command completed after 1 attempt(s).
2024-06-26T07:11:58.5483956Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84
2024-06-26T07:11:58.5485132Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84
2024-06-26T07:11:58.5485832Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 &
2024-06-26T07:11:58.5486487Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}"
2024-06-26T07:11:58.5494508Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:11:58.5495006Z env:
2024-06-26T07:11:58.5495291Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:11:58.5495750Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:11:58.5496246Z ##[endgroup]
2024-06-26T07:11:58.7656166Z Defaulting to user installation because normal site-packages is not writeable
2024-06-26T07:11:58.7809727Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.7/site-packages (5.9.1)
2024-06-26T07:11:58.7906897Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.7/site-packages (11.525.84)
2024-06-26T07:11:58.8993456Z Prepare all required actions
2024-06-26T07:11:58.8993953Z Getting action download info
2024-06-26T07:11:59.0440372Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6)
2024-06-26T07:11:59.1951770Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a)
2024-06-26T07:11:59.3158384Z ##[group]Run ./.github/actions/download-build-artifacts
2024-06-26T07:11:59.3158875Z with:
2024-06-26T07:11:59.3159221Z   name: linux-focal-cuda12.1-py3.10-gcc9-sm86
2024-06-26T07:11:59.3159677Z   s3-bucket: gha-artifacts
2024-06-26T07:11:59.3160007Z env:
2024-06-26T07:11:59.3160284Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:11:59.3160737Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:11:59.3161221Z ##[endgroup]
2024-06-26T07:11:59.3193327Z ##[group]Run seemethere/download-artifact-s3@v4
2024-06-26T07:11:59.3193771Z with:
2024-06-26T07:11:59.3194139Z   name: linux-focal-cuda12.1-py3.10-gcc9-sm86
2024-06-26T07:11:59.3194606Z   s3-bucket: gha-artifacts
2024-06-26T07:11:59.3194958Z   region: us-east-1
2024-06-26T07:11:59.3195265Z env:
2024-06-26T07:11:59.3195536Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:11:59.3195986Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:11:59.3196465Z ##[endgroup]
2024-06-26T07:11:59.7369057Z (node:27839) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.
2024-06-26T07:11:59.7369757Z 
2024-06-26T07:11:59.7370028Z Please migrate your code to use AWS SDK for JavaScript (v3).
2024-06-26T07:11:59.7370843Z For more information, check the migration guide at https://a.co/7PzMCcy
2024-06-26T07:11:59.7371829Z (Use `node --trace-warnings ...` to show where the warning was created)
2024-06-26T07:11:59.8549038Z Found 1 objects with prefix pytorch/pytorch/9673646806/linux-focal-cuda12.1-py3.10-gcc9-sm86/
2024-06-26T07:11:59.8550131Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip
2024-06-26T07:12:07.4158267Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip
2024-06-26T07:12:07.4164999Z Artifact download has finished successfully
2024-06-26T07:12:07.4314366Z ##[group]Run unzip -o artifacts.zip
2024-06-26T07:12:07.4314811Z unzip -o artifacts.zip
2024-06-26T07:12:07.4322508Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:12:07.4323010Z env:
2024-06-26T07:12:07.4323298Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:07.4323756Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:07.4324238Z ##[endgroup]
2024-06-26T07:12:07.4352495Z Archive:  artifacts.zip
2024-06-26T07:12:07.4354040Z    creating: dist/
2024-06-26T07:12:09.4058425Z   inflating: dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl  
2024-06-26T07:12:09.4059738Z    creating: build/custom_test_artifacts/
2024-06-26T07:12:09.4060897Z    creating: build/custom_test_artifacts/custom-op-build/
2024-06-26T07:12:09.4062659Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/
2024-06-26T07:12:09.4064303Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/
2024-06-26T07:12:09.4067212Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml  
2024-06-26T07:12:09.4069120Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/
2024-06-26T07:12:09.4070329Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake  
2024-06-26T07:12:09.4071578Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/
2024-06-26T07:12:09.4072667Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/
2024-06-26T07:12:09.4073807Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c  
2024-06-26T07:12:09.4075332Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out  
2024-06-26T07:12:09.4076531Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/
2024-06-26T07:12:09.4077853Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/
2024-06-26T07:12:09.4079181Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp  
2024-06-26T07:12:09.4080693Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out  
2024-06-26T07:12:09.4081821Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin  
2024-06-26T07:12:09.4082949Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake  
2024-06-26T07:12:09.4084109Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin  
2024-06-26T07:12:09.4085645Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake  
2024-06-26T07:12:09.4086683Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/
2024-06-26T07:12:09.4087680Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/
2024-06-26T07:12:09.4119476Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii  
2024-06-26T07:12:09.4157711Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp  
2024-06-26T07:12:09.4159148Z  extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id  
2024-06-26T07:12:09.4202974Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii  
2024-06-26T07:12:09.4207197Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c  
2024-06-26T07:12:09.4210089Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu  
2024-06-26T07:12:09.4211638Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c  
2024-06-26T07:12:09.4213030Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx  
2024-06-26T07:12:09.4214406Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin  
2024-06-26T07:12:09.4215787Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin  
2024-06-26T07:12:09.4217156Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c  
2024-06-26T07:12:09.4218929Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o  
2024-06-26T07:12:09.4220200Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin  
2024-06-26T07:12:09.4221438Z  extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c  
2024-06-26T07:12:09.4222648Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin  
2024-06-26T07:12:09.4223860Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c  
2024-06-26T07:12:09.4225047Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o  
2024-06-26T07:12:09.4226248Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu  
2024-06-26T07:12:09.4267841Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out  
2024-06-26T07:12:09.4325754Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin  
2024-06-26T07:12:09.4327413Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake  
2024-06-26T07:12:09.4328785Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/
2024-06-26T07:12:09.4329920Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/
2024-06-26T07:12:09.4331255Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache  
2024-06-26T07:12:09.4332178Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/
2024-06-26T07:12:09.4333195Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts  
2024-06-26T07:12:09.4334349Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make  
2024-06-26T07:12:09.4335445Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make  
2024-06-26T07:12:09.4336472Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt  
2024-06-26T07:12:09.4337525Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake  
2024-06-26T07:12:09.4338586Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make  
2024-06-26T07:12:09.4339656Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake  
2024-06-26T07:12:09.4340719Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make  
2024-06-26T07:12:09.4341766Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make  
2024-06-26T07:12:09.4351074Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d  
2024-06-26T07:12:09.4479169Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o  
2024-06-26T07:12:09.4480577Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/
2024-06-26T07:12:09.4482073Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts  
2024-06-26T07:12:09.4483411Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make  
2024-06-26T07:12:09.4484575Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make  
2024-06-26T07:12:09.4485907Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt  
2024-06-26T07:12:09.4487043Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake  
2024-06-26T07:12:09.4488188Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make  
2024-06-26T07:12:09.4489462Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake  
2024-06-26T07:12:09.4490714Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make  
2024-06-26T07:12:09.4491837Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make  
2024-06-26T07:12:09.4503331Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d  
2024-06-26T07:12:09.4582231Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o  
2024-06-26T07:12:09.4583754Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake  
2024-06-26T07:12:09.4585117Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt  
2024-06-26T07:12:09.4586223Z  extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks  
2024-06-26T07:12:09.4587482Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2  
2024-06-26T07:12:09.4588581Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake  
2024-06-26T07:12:09.4589478Z   inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc  
2024-06-26T07:12:09.4590320Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt  
2024-06-26T07:12:09.4591105Z   inflating: build/custom_test_artifacts/custom-op-build/Makefile  
2024-06-26T07:12:09.4591919Z   inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake  
2024-06-26T07:12:09.4696086Z   inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so  
2024-06-26T07:12:09.4757861Z   inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops  
2024-06-26T07:12:09.4759471Z    creating: build/custom_test_artifacts/jit-hook-build/
2024-06-26T07:12:09.4760176Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/
2024-06-26T07:12:09.4760981Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/
2024-06-26T07:12:09.4764620Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml  
2024-06-26T07:12:09.4766588Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/
2024-06-26T07:12:09.4767540Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake  
2024-06-26T07:12:09.4768512Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/
2024-06-26T07:12:09.4769455Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/
2024-06-26T07:12:09.4770630Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c  
2024-06-26T07:12:09.4771710Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out  
2024-06-26T07:12:09.4772719Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/
2024-06-26T07:12:09.4773688Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/
2024-06-26T07:12:09.4774812Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp  
2024-06-26T07:12:09.4775935Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out  
2024-06-26T07:12:09.4777042Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin  
2024-06-26T07:12:09.4778165Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake  
2024-06-26T07:12:09.4779344Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin  
2024-06-26T07:12:09.4780462Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake  
2024-06-26T07:12:09.4781739Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/
2024-06-26T07:12:09.4782713Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/
2024-06-26T07:12:09.4816610Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii  
2024-06-26T07:12:09.4854193Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp  
2024-06-26T07:12:09.4855605Z  extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id  
2024-06-26T07:12:09.4898287Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii  
2024-06-26T07:12:09.4899678Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c  
2024-06-26T07:12:09.4901235Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu  
2024-06-26T07:12:09.4902680Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c  
2024-06-26T07:12:09.4904054Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx  
2024-06-26T07:12:09.4905422Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin  
2024-06-26T07:12:09.4906787Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin  
2024-06-26T07:12:09.4908141Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c  
2024-06-26T07:12:09.4909514Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o  
2024-06-26T07:12:09.4910768Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin  
2024-06-26T07:12:09.4911973Z  extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c  
2024-06-26T07:12:09.4913157Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin  
2024-06-26T07:12:09.4914351Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c  
2024-06-26T07:12:09.4915517Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o  
2024-06-26T07:12:09.4916700Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu  
2024-06-26T07:12:09.4964054Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out  
2024-06-26T07:12:09.5021961Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin  
2024-06-26T07:12:09.5023130Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake  
2024-06-26T07:12:09.5024091Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/
2024-06-26T07:12:09.5024924Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/
2024-06-26T07:12:09.5025795Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache  
2024-06-26T07:12:09.5026723Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/
2024-06-26T07:12:09.5027776Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts  
2024-06-26T07:12:09.5028991Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make  
2024-06-26T07:12:09.5030115Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make  
2024-06-26T07:12:09.5031294Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt  
2024-06-26T07:12:09.5032381Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake  
2024-06-26T07:12:09.5033468Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make  
2024-06-26T07:12:09.5034550Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake  
2024-06-26T07:12:09.5035631Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make  
2024-06-26T07:12:09.5036708Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make  
2024-06-26T07:12:09.5047084Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d  
2024-06-26T07:12:09.5108569Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o  
2024-06-26T07:12:09.5109741Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake  
2024-06-26T07:12:09.5110800Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt  
2024-06-26T07:12:09.5111747Z  extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks  
2024-06-26T07:12:09.5112631Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2  
2024-06-26T07:12:09.5113514Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake  
2024-06-26T07:12:09.5114389Z   inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc  
2024-06-26T07:12:09.5115213Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt  
2024-06-26T07:12:09.5115980Z   inflating: build/custom_test_artifacts/jit-hook-build/Makefile  
2024-06-26T07:12:09.5116777Z   inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake  
2024-06-26T07:12:09.5162038Z   inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks  
2024-06-26T07:12:09.5162789Z    creating: build/custom_test_artifacts/custom-backend-build/
2024-06-26T07:12:09.5163545Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/
2024-06-26T07:12:09.5164412Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/
2024-06-26T07:12:09.5169574Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml  
2024-06-26T07:12:09.5170609Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/
2024-06-26T07:12:09.5171590Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake  
2024-06-26T07:12:09.5172627Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/
2024-06-26T07:12:09.5173646Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/
2024-06-26T07:12:09.5174787Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c  
2024-06-26T07:12:09.5175928Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out  
2024-06-26T07:12:09.5176969Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/
2024-06-26T07:12:09.5178005Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/
2024-06-26T07:12:09.5179247Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp  
2024-06-26T07:12:09.5180431Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out  
2024-06-26T07:12:09.5181613Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin  
2024-06-26T07:12:09.5182934Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake  
2024-06-26T07:12:09.5199779Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin  
2024-06-26T07:12:09.5201084Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake  
2024-06-26T07:12:09.5202181Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/
2024-06-26T07:12:09.5203248Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/
2024-06-26T07:12:09.5221605Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii  
2024-06-26T07:12:09.5259123Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp  
2024-06-26T07:12:09.5260749Z  extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id  
2024-06-26T07:12:09.5303185Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii  
2024-06-26T07:12:09.5304639Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c  
2024-06-26T07:12:09.5306107Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu  
2024-06-26T07:12:09.5307602Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c  
2024-06-26T07:12:09.5309089Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx  
2024-06-26T07:12:09.5310531Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin  
2024-06-26T07:12:09.5311958Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin  
2024-06-26T07:12:09.5313371Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c  
2024-06-26T07:12:09.5314748Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o  
2024-06-26T07:12:09.5316057Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin  
2024-06-26T07:12:09.5317324Z  extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c  
2024-06-26T07:12:09.5318560Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin  
2024-06-26T07:12:09.5319824Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c  
2024-06-26T07:12:09.5321053Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o  
2024-06-26T07:12:09.5322308Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu  
2024-06-26T07:12:09.5369941Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out  
2024-06-26T07:12:09.5427622Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin  
2024-06-26T07:12:09.5429277Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake  
2024-06-26T07:12:09.5431351Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/
2024-06-26T07:12:09.5433164Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/
2024-06-26T07:12:09.5435299Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache  
2024-06-26T07:12:09.5437276Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/
2024-06-26T07:12:09.5439055Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts  
2024-06-26T07:12:09.5440295Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make  
2024-06-26T07:12:09.5441490Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make  
2024-06-26T07:12:09.5442616Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt  
2024-06-26T07:12:09.5443774Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake  
2024-06-26T07:12:09.5445203Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make  
2024-06-26T07:12:09.5446394Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake  
2024-06-26T07:12:09.5447559Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make  
2024-06-26T07:12:09.5448743Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make  
2024-06-26T07:12:09.5449991Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d  
2024-06-26T07:12:09.5558420Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o  
2024-06-26T07:12:09.5559757Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/
2024-06-26T07:12:09.5560928Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts  
2024-06-26T07:12:09.5562226Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make  
2024-06-26T07:12:09.5563468Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make  
2024-06-26T07:12:09.5564650Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt  
2024-06-26T07:12:09.5565994Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake  
2024-06-26T07:12:09.5567220Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make  
2024-06-26T07:12:09.5568445Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake  
2024-06-26T07:12:09.5569707Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make  
2024-06-26T07:12:09.5570983Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make  
2024-06-26T07:12:09.5581746Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d  
2024-06-26T07:12:09.5635373Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o  
2024-06-26T07:12:09.5637907Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake  
2024-06-26T07:12:09.5639495Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt  
2024-06-26T07:12:09.5640508Z  extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks  
2024-06-26T07:12:09.5641447Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2  
2024-06-26T07:12:09.5642390Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake  
2024-06-26T07:12:09.5643444Z   inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc  
2024-06-26T07:12:09.5644347Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt  
2024-06-26T07:12:09.5645302Z   inflating: build/custom_test_artifacts/custom-backend-build/Makefile  
2024-06-26T07:12:09.5646155Z   inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake  
2024-06-26T07:12:09.5740175Z   inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so  
2024-06-26T07:12:09.5780559Z   inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend  
2024-06-26T07:12:09.5781194Z    creating: build/lib/
2024-06-26T07:12:09.5865599Z   inflating: build/lib/libprotobuf-lite.a  
2024-06-26T07:12:09.6307932Z   inflating: build/lib/libprotobuf.a  
2024-06-26T07:12:09.6317258Z   inflating: build/lib/libpthreadpool.a  
2024-06-26T07:12:09.6325220Z   inflating: build/lib/libcpuinfo.a  
2024-06-26T07:12:09.6333250Z   inflating: build/lib/libcpuinfo_internals.a  
2024-06-26T07:12:09.6334334Z   inflating: build/lib/libclog.a     
2024-06-26T07:12:09.6351220Z   inflating: build/lib/libnnpack.a   
2024-06-26T07:12:09.6353213Z   inflating: build/lib/libnnpack_reference_layers.a  
2024-06-26T07:12:09.6415452Z   inflating: build/lib/libgtest.a    
2024-06-26T07:12:09.6487384Z   inflating: build/lib/libbenchmark.a  
2024-06-26T07:12:09.6549083Z   inflating: build/lib/libasmjit.a   
2024-06-26T07:12:09.6555776Z   inflating: build/lib/libittnotify.a  
2024-06-26T07:12:09.6582519Z   inflating: build/lib/libtensorpipe_uv.a  
2024-06-26T07:12:09.6704600Z   inflating: build/lib/libgloo.a     
2024-06-26T07:12:09.6725598Z   inflating: build/lib/libfmt.a      
2024-06-26T07:12:09.6814405Z   inflating: build/lib/libc10.so     
2024-06-26T07:12:09.6815819Z   inflating: build/lib/libcaffe2_nvrtc.so  
2024-06-26T07:12:09.6816302Z   inflating: build/lib/libfoxi_loader.a  
2024-06-26T07:12:09.6817331Z   inflating: build/lib/libtorch_global_deps.so  
2024-06-26T07:12:09.6836133Z   inflating: build/lib/libpytorch_qnnpack.a  
2024-06-26T07:12:09.6853466Z   inflating: build/lib/libgmock.a    
2024-06-26T07:12:09.6853919Z   inflating: build/lib/libgtest_main.a  
2024-06-26T07:12:09.6854398Z   inflating: build/lib/libbenchmark_main.a  
2024-06-26T07:12:09.7339618Z   inflating: build/lib/libprotoc.a   
2024-06-26T07:12:09.7709858Z   inflating: build/lib/libgloo_cuda.a  
2024-06-26T07:12:10.7438268Z   inflating: build/lib/libdnnl.a     
2024-06-26T07:12:10.7986264Z   inflating: build/lib/libtensorpipe.a  
2024-06-26T07:12:10.8039148Z   inflating: build/lib/libc10_cuda.so  
2024-06-26T07:12:10.8039616Z   inflating: build/lib/libgmock_main.a  
2024-06-26T07:12:10.9241045Z   inflating: build/lib/libfbgemm.a   
2024-06-26T07:12:10.9485902Z   inflating: build/lib/libtensorpipe_cuda.a  
2024-06-26T07:12:10.9982833Z   inflating: build/lib/libkineto.a   
2024-06-26T07:12:11.0160151Z   inflating: build/lib/libXNNPACK.a  
2024-06-26T07:12:11.0200744Z   inflating: build/lib/libonnx_proto.a  
2024-06-26T07:12:11.0863941Z   inflating: build/lib/libonnx.a     
2024-06-26T07:12:13.4114333Z   inflating: build/lib/libtorch_cpu.so  
2024-06-26T07:12:13.4118590Z   inflating: build/lib/libunbox_lib.a  
2024-06-26T07:12:13.4122235Z   inflating: build/lib/libshm.so     
2024-06-26T07:12:15.3962912Z   inflating: build/lib/libtorch_cuda.so  
2024-06-26T07:12:15.3964158Z   inflating: build/lib/libtorch.so   
2024-06-26T07:12:16.1890804Z   inflating: build/lib/libtorch_cuda_linalg.so  
2024-06-26T07:12:16.1892821Z   inflating: build/lib/libc10d_cuda_test.so  
2024-06-26T07:12:16.3782424Z   inflating: build/lib/libtorch_python.so  
2024-06-26T07:12:16.3849032Z   inflating: build/lib/libtorchbind_test.so  
2024-06-26T07:12:16.3868872Z   inflating: build/lib/libjitbackend_test.so  
2024-06-26T07:12:16.3893397Z   inflating: build/lib/libbackend_with_compiler.so  
2024-06-26T07:12:16.3927254Z   inflating: build/lib/libnnapi_backend.so  
2024-06-26T07:12:16.3928440Z    creating: build/bin/
2024-06-26T07:12:16.3975714Z   inflating: build/bin/c10_CompileTimeFunctionPointer_test  
2024-06-26T07:12:16.4025321Z   inflating: build/bin/c10_DeviceGuard_test  
2024-06-26T07:12:16.4074806Z   inflating: build/bin/c10_Device_test  
2024-06-26T07:12:16.4132063Z   inflating: build/bin/c10_DispatchKeySet_test  
2024-06-26T07:12:16.4183361Z   inflating: build/bin/c10_Scalar_test  
2024-06-26T07:12:16.4230510Z   inflating: build/bin/c10_StreamGuard_test  
2024-06-26T07:12:16.4283485Z   inflating: build/bin/c10_InlineDeviceGuard_test  
2024-06-26T07:12:16.4332593Z   inflating: build/bin/c10_SymInt_test  
2024-06-26T07:12:16.4386403Z   inflating: build/bin/c10_InlineStreamGuard_test  
2024-06-26T07:12:16.4441232Z   inflating: build/bin/c10_SizesAndStrides_test  
2024-06-26T07:12:16.4491871Z   inflating: build/bin/c10_Bitset_test  
2024-06-26T07:12:16.4563069Z   inflating: build/bin/c10_cow_test  
2024-06-26T07:12:16.4610591Z   inflating: build/bin/c10_ConstexprCrc_test  
2024-06-26T07:12:16.4658300Z   inflating: build/bin/c10_DeadlockDetection_test  
2024-06-26T07:12:16.4707133Z   inflating: build/bin/c10_Half_test  
2024-06-26T07:12:16.4761768Z   inflating: build/bin/c10_LeftRight_test  
2024-06-26T07:12:16.4814485Z   inflating: build/bin/c10_Metaprogramming_test  
2024-06-26T07:12:16.4862678Z   inflating: build/bin/c10_Synchronized_test  
2024-06-26T07:12:16.4916878Z   inflating: build/bin/c10_ThreadLocal_test  
2024-06-26T07:12:16.4966700Z   inflating: build/bin/c10_TypeIndex_test  
2024-06-26T07:12:16.5015535Z   inflating: build/bin/c10_TypeList_test  
2024-06-26T07:12:16.5063518Z   inflating: build/bin/c10_TypeTraits_test  
2024-06-26T07:12:16.5113592Z   inflating: build/bin/c10_accumulate_test  
2024-06-26T07:12:16.5167157Z   inflating: build/bin/c10_bfloat16_test  
2024-06-26T07:12:16.5216395Z   inflating: build/bin/c10_bit_cast_test  
2024-06-26T07:12:16.5270660Z   inflating: build/bin/c10_complex_math_test  
2024-06-26T07:12:16.5324577Z   inflating: build/bin/c10_complex_test  
2024-06-26T07:12:16.5375512Z   inflating: build/bin/c10_exception_test  
2024-06-26T07:12:16.5424226Z   inflating: build/bin/c10_flags_test  
2024-06-26T07:12:16.5472813Z   inflating: build/bin/c10_generic_math_test  
2024-06-26T07:12:16.5631553Z   inflating: build/bin/c10_intrusive_ptr_test  
2024-06-26T07:12:16.5680558Z   inflating: build/bin/c10_irange_test  
2024-06-26T07:12:16.5732099Z   inflating: build/bin/c10_lazy_test  
2024-06-26T07:12:16.5787225Z   inflating: build/bin/c10_logging_test  
2024-06-26T07:12:16.5859584Z   inflating: build/bin/c10_optional_test  
2024-06-26T07:12:16.5919872Z   inflating: build/bin/c10_ordered_preserving_dict_test  
2024-06-26T07:12:16.5972023Z   inflating: build/bin/c10_registry_test  
2024-06-26T07:12:16.6117678Z   inflating: build/bin/c10_small_vector_test  
2024-06-26T07:12:16.6167632Z   inflating: build/bin/c10_ssize_test  
2024-06-26T07:12:16.6217332Z   inflating: build/bin/c10_string_util_test  
2024-06-26T07:12:16.6274153Z   inflating: build/bin/c10_string_view_test  
2024-06-26T07:12:16.6322888Z   inflating: build/bin/c10_tempfile_test  
2024-06-26T07:12:16.6369842Z   inflating: build/bin/c10_intrusive_ptr_benchmark  
2024-06-26T07:12:16.6424242Z   inflating: build/bin/c10_typeid_test  
2024-06-26T07:12:16.6855687Z   inflating: build/bin/protoc-3.13.0.0  
2024-06-26T07:12:16.7287870Z   inflating: build/bin/protoc        
2024-06-26T07:12:16.7339136Z   inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test  
2024-06-26T07:12:16.7390259Z   inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream  
2024-06-26T07:12:16.7441933Z   inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device  
2024-06-26T07:12:16.7491558Z   inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes  
2024-06-26T07:12:16.7542450Z   inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads  
2024-06-26T07:12:16.7593299Z   inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks  
2024-06-26T07:12:16.7640688Z   inflating: build/bin/c10_cuda_CUDATest  
2024-06-26T07:12:16.7692278Z   inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block  
2024-06-26T07:12:16.8009976Z   inflating: build/bin/vec_test_all_types_DEFAULT  
2024-06-26T07:12:16.8341401Z   inflating: build/bin/vec_test_all_types_AVX512  
2024-06-26T07:12:16.8687650Z   inflating: build/bin/vec_test_all_types_AVX2  
2024-06-26T07:12:16.8739071Z   inflating: build/bin/FileStoreTest  
2024-06-26T07:12:16.8793429Z   inflating: build/bin/TCPStoreTest  
2024-06-26T07:12:16.8845412Z   inflating: build/bin/HashStoreTest  
2024-06-26T07:12:16.8858186Z   inflating: build/bin/ProcessGroupMPITest  
2024-06-26T07:12:16.8910175Z   inflating: build/bin/test_edge_op_registration  
2024-06-26T07:12:16.8914111Z   inflating: build/bin/torch_shm_manager  
2024-06-26T07:12:16.8916924Z   inflating: build/bin/example_allreduce  
2024-06-26T07:12:16.8970816Z   inflating: build/bin/test_dist_autograd  
2024-06-26T07:12:16.9037968Z   inflating: build/bin/test_cpp_rpc  
2024-06-26T07:12:16.9039763Z   inflating: build/bin/parallel_benchmark  
2024-06-26T07:12:16.9104850Z   inflating: build/bin/test_mobile_nnc  
2024-06-26T07:12:16.9113691Z   inflating: build/bin/aot_model_compiler_test  
2024-06-26T07:12:16.9445130Z   inflating: build/bin/test_lazy     
2024-06-26T07:12:17.0585211Z   inflating: build/bin/test_api      
2024-06-26T07:12:17.0656192Z   inflating: build/bin/Dict_test     
2024-06-26T07:12:17.0706753Z   inflating: build/bin/Dimname_test  
2024-06-26T07:12:17.0769720Z   inflating: build/bin/MaybeOwned_test  
2024-06-26T07:12:17.0824672Z   inflating: build/bin/NamedTensor_test  
2024-06-26T07:12:17.0882546Z   inflating: build/bin/apply_utils_test  
2024-06-26T07:12:17.0939602Z   inflating: build/bin/atest         
2024-06-26T07:12:17.1000823Z   inflating: build/bin/basic         
2024-06-26T07:12:17.1053905Z   inflating: build/bin/broadcast_test  
2024-06-26T07:12:17.1102922Z   inflating: build/bin/cpu_allocator_test  
2024-06-26T07:12:17.1159130Z   inflating: build/bin/cpu_generator_test  
2024-06-26T07:12:17.1210695Z   inflating: build/bin/cpu_profiling_allocator_test  
2024-06-26T07:12:17.1300963Z   inflating: build/bin/cpu_rng_test  
2024-06-26T07:12:17.1349217Z   inflating: build/bin/dispatch_key_set_test  
2024-06-26T07:12:17.1399241Z   inflating: build/bin/dlconvertor_test  
2024-06-26T07:12:17.1455461Z   inflating: build/bin/extension_backend_test  
2024-06-26T07:12:17.1508671Z   inflating: build/bin/half_test     
2024-06-26T07:12:17.1602041Z   inflating: build/bin/ivalue_test   
2024-06-26T07:12:17.1649515Z   inflating: build/bin/lazy_tensor_test  
2024-06-26T07:12:17.1702126Z   inflating: build/bin/math_kernel_test  
2024-06-26T07:12:17.1754451Z   inflating: build/bin/memory_format_test  
2024-06-26T07:12:17.1805616Z   inflating: build/bin/memory_overlapping_test  
2024-06-26T07:12:17.1856993Z   inflating: build/bin/mobile_memory_cleanup  
2024-06-26T07:12:17.1911142Z   inflating: build/bin/native_test   
2024-06-26T07:12:17.1960091Z   inflating: build/bin/operator_name_test  
2024-06-26T07:12:17.2009883Z   inflating: build/bin/operators_test  
2024-06-26T07:12:17.2060276Z   inflating: build/bin/packedtensoraccessor_test  
2024-06-26T07:12:17.2125622Z   inflating: build/bin/pow_test      
2024-06-26T07:12:17.2180542Z   inflating: build/bin/quantized_test  
2024-06-26T07:12:17.2229432Z   inflating: build/bin/reduce_ops_test  
2024-06-26T07:12:17.2278345Z   inflating: build/bin/reportMemoryUsage_test  
2024-06-26T07:12:17.2333223Z   inflating: build/bin/scalar_tensor_test  
2024-06-26T07:12:17.2389656Z   inflating: build/bin/scalar_test   
2024-06-26T07:12:17.2439656Z   inflating: build/bin/StorageUtils_test  
2024-06-26T07:12:17.2490449Z   inflating: build/bin/stride_properties_test  
2024-06-26T07:12:17.2566642Z   inflating: build/bin/tensor_iterator_test  
2024-06-26T07:12:17.2619144Z   inflating: build/bin/test_parallel  
2024-06-26T07:12:17.2622103Z   inflating: build/bin/thread_init_test  
2024-06-26T07:12:17.2676566Z   inflating: build/bin/type_ptr_test  
2024-06-26T07:12:17.2734705Z   inflating: build/bin/type_test     
2024-06-26T07:12:17.2785125Z   inflating: build/bin/undefined_tensor_test  
2024-06-26T07:12:17.2786263Z   inflating: build/bin/verify_api_visibility  
2024-06-26T07:12:17.2853760Z   inflating: build/bin/legacy_vmap_test  
2024-06-26T07:12:17.2904272Z   inflating: build/bin/weakref_test  
2024-06-26T07:12:17.2954249Z   inflating: build/bin/wrapdim_test  
2024-06-26T07:12:17.3004191Z   inflating: build/bin/xla_tensor_test  
2024-06-26T07:12:17.3062423Z   inflating: build/bin/IListRef_test  
2024-06-26T07:12:17.3165300Z   inflating: build/bin/List_test     
2024-06-26T07:12:17.3229495Z   inflating: build/bin/KernelFunction_test  
2024-06-26T07:12:17.3346950Z   inflating: build/bin/kernel_function_legacy_test  
2024-06-26T07:12:17.3440371Z   inflating: build/bin/kernel_function_test  
2024-06-26T07:12:17.3564995Z   inflating: build/bin/kernel_lambda_legacy_test  
2024-06-26T07:12:17.3666764Z   inflating: build/bin/kernel_lambda_test  
2024-06-26T07:12:17.3725705Z   inflating: build/bin/kernel_stackbased_test  
2024-06-26T07:12:17.3819806Z   inflating: build/bin/make_boxed_from_unboxed_functor_test  
2024-06-26T07:12:17.3869732Z   inflating: build/bin/CppSignature_test  
2024-06-26T07:12:17.3923034Z   inflating: build/bin/backend_fallback_test  
2024-06-26T07:12:17.3970365Z   inflating: build/bin/op_allowlist_test  
2024-06-26T07:12:17.4261862Z   inflating: build/bin/op_registration_test  
2024-06-26T07:12:17.4323581Z   inflating: build/bin/inline_container_test  
2024-06-26T07:12:17.4375578Z   inflating: build/bin/cuda_apply_test  
2024-06-26T07:12:17.4424612Z   inflating: build/bin/cuda_allocator_test  
2024-06-26T07:12:17.4482622Z   inflating: build/bin/cuda_atomic_ops_test  
2024-06-26T07:12:17.4534732Z   inflating: build/bin/cuda_caching_host_allocator_test  
2024-06-26T07:12:17.4603251Z   inflating: build/bin/cuda_complex_math_test  
2024-06-26T07:12:17.4659938Z   inflating: build/bin/cuda_complex_test  
2024-06-26T07:12:17.4708004Z   inflating: build/bin/cuda_device_test  
2024-06-26T07:12:17.4763480Z   inflating: build/bin/cuda_cub_test  
2024-06-26T07:12:17.4812779Z   inflating: build/bin/cuda_dlconvertor_test  
2024-06-26T07:12:17.4876237Z   inflating: build/bin/cuda_distributions_test  
2024-06-26T07:12:17.4930445Z   inflating: build/bin/cuda_generator_test  
2024-06-26T07:12:17.4978831Z   inflating: build/bin/cuda_half_test  
2024-06-26T07:12:17.5028380Z   inflating: build/bin/cuda_integer_divider_test  
2024-06-26T07:12:17.5075938Z   inflating: build/bin/cuda_optional_test  
2024-06-26T07:12:17.5125954Z   inflating: build/bin/cuda_packedtensoraccessor_test  
2024-06-26T07:12:17.5177007Z   inflating: build/bin/cuda_reportMemoryUsage_test  
2024-06-26T07:12:17.5226083Z   inflating: build/bin/cuda_allocatorTraceTracker_test  
2024-06-26T07:12:17.5285028Z   inflating: build/bin/cuda_stream_test  
2024-06-26T07:12:17.5332100Z   inflating: build/bin/cuda_cudnn_test  
2024-06-26T07:12:17.5383003Z   inflating: build/bin/cuda_vectorized_test  
2024-06-26T07:12:17.5397743Z   inflating: build/bin/tutorial_tensorexpr  
2024-06-26T07:12:17.5461523Z   inflating: build/bin/ProcessGroupGlooTest  
2024-06-26T07:12:17.5517935Z   inflating: build/bin/ProcessGroupGlooAsyncTest  
2024-06-26T07:12:17.5578578Z   inflating: build/bin/ProcessGroupNCCLTest  
2024-06-26T07:12:17.5638562Z   inflating: build/bin/ProcessGroupNCCLErrorsTest  
2024-06-26T07:12:17.6442879Z   inflating: build/bin/test_tensorexpr  
2024-06-26T07:12:17.7002432Z   inflating: build/bin/test_jit      
2024-06-26T07:12:17.7002986Z    creating: .additional_ci_files/
2024-06-26T07:12:17.7056235Z   inflating: .additional_ci_files/test-times.json  
2024-06-26T07:12:17.7267769Z   inflating: .additional_ci_files/test-class-times.json  
2024-06-26T07:12:17.7293753Z ##[group]Run rm artifacts.zip
2024-06-26T07:12:17.7294143Z rm artifacts.zip
2024-06-26T07:12:17.7303156Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:12:17.7303654Z env:
2024-06-26T07:12:17.7304073Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:17.7304526Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:17.7305008Z ##[endgroup]
2024-06-26T07:12:17.8019406Z ##[group]Run df -H
2024-06-26T07:12:17.8019731Z df -H
2024-06-26T07:12:17.8027426Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:12:17.8027916Z env:
2024-06-26T07:12:17.8028196Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:17.8028648Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:17.8029134Z ##[endgroup]
2024-06-26T07:12:17.8055216Z Filesystem      Size  Used Avail Use% Mounted on
2024-06-26T07:12:17.8055734Z devtmpfs         34G     0   34G   0% /dev
2024-06-26T07:12:17.8056202Z tmpfs            34G     0   34G   0% /dev/shm
2024-06-26T07:12:17.8056913Z tmpfs            34G  455k   34G   1% /run
2024-06-26T07:12:17.8057399Z tmpfs            34G     0   34G   0% /sys/fs/cgroup
2024-06-26T07:12:17.8057876Z /dev/nvme0n1p1  162G   44G  118G  28% /
2024-06-26T07:12:17.8094860Z Prepare all required actions
2024-06-26T07:12:17.8095330Z Getting action download info
2024-06-26T07:12:17.9210898Z ##[group]Run ./.github/actions/download-td-artifacts
2024-06-26T07:12:17.9211371Z with:
2024-06-26T07:12:17.9211624Z env:
2024-06-26T07:12:17.9211902Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:17.9212364Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:17.9212851Z ##[endgroup]
2024-06-26T07:12:17.9243220Z ##[group]Run seemethere/download-artifact-s3@v4
2024-06-26T07:12:17.9243666Z with:
2024-06-26T07:12:17.9243939Z   name: td_results
2024-06-26T07:12:17.9244259Z   s3-bucket: gha-artifacts
2024-06-26T07:12:17.9244614Z   region: us-east-1
2024-06-26T07:12:17.9245113Z env:
2024-06-26T07:12:17.9245383Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:17.9245836Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:17.9246318Z ##[endgroup]
2024-06-26T07:12:18.3268704Z (node:27862) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.
2024-06-26T07:12:18.3270343Z 
2024-06-26T07:12:18.3270869Z Please migrate your code to use AWS SDK for JavaScript (v3).
2024-06-26T07:12:18.3272279Z For more information, check the migration guide at https://a.co/7PzMCcy
2024-06-26T07:12:18.3273913Z (Use `node --trace-warnings ...` to show where the warning was created)
2024-06-26T07:12:18.4003526Z Found 0 objects with prefix pytorch/pytorch/9673646806/td_results/
2024-06-26T07:12:18.4010151Z Artifact download has finished successfully
2024-06-26T07:12:18.4136985Z ##[group]Run mkdir -p .additional_ci_files
2024-06-26T07:12:18.4137478Z mkdir -p .additional_ci_files
2024-06-26T07:12:18.4138043Z mv td_results.json .additional_ci_files/td_results.json
2024-06-26T07:12:18.4145820Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:12:18.4146313Z env:
2024-06-26T07:12:18.4146592Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:18.4147061Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:18.4147563Z ##[endgroup]
2024-06-26T07:12:18.4181494Z mv: cannot stat 'td_results.json': No such file or directory
2024-06-26T07:12:18.4191907Z ##[error]Process completed with exit code 1.
2024-06-26T07:12:18.4239186Z ##[group]Run .github/scripts/parse_ref.py
2024-06-26T07:12:18.4239670Z .github/scripts/parse_ref.py
2024-06-26T07:12:18.4247657Z shell: /usr/bin/bash -e {0}
2024-06-26T07:12:18.4248013Z env:
2024-06-26T07:12:18.4248296Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:18.4248762Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:18.4249247Z ##[endgroup]
2024-06-26T07:12:18.4476733Z Prepare all required actions
2024-06-26T07:12:18.4512521Z ##[group]Run ./.github/actions/get-workflow-job-id
2024-06-26T07:12:18.4512971Z with:
2024-06-26T07:12:18.4513524Z   github-token: ***
2024-06-26T07:12:18.4513836Z env:
2024-06-26T07:12:18.4514103Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:18.4514550Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:18.4515158Z ##[endgroup]
2024-06-26T07:12:18.4529898Z ##[group]Run set -eux
2024-06-26T07:12:18.4530234Z set -eux
2024-06-26T07:12:18.4530982Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}"
2024-06-26T07:12:18.4538506Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:12:18.4538999Z env:
2024-06-26T07:12:18.4539277Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:18.4539729Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:18.4540369Z   GITHUB_TOKEN: ***
2024-06-26T07:12:18.4540678Z ##[endgroup]
2024-06-26T07:12:18.4559835Z + python3 .github/scripts/get_workflow_job_id.py 9673646806 i-0f811bba69cb4bebd
2024-06-26T07:12:19.0757161Z setting job-id=26688917313
2024-06-26T07:12:19.0758237Z setting job-name=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamic_aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu)
2024-06-26T07:12:19.0956522Z Prepare all required actions
2024-06-26T07:12:19.0956981Z Getting action download info
2024-06-26T07:12:19.2057296Z ##[group]Run ./.github/actions/filter-test-configs
2024-06-26T07:12:19.2057744Z with:
2024-06-26T07:12:19.2058198Z   github-token: ***
2024-06-26T07:12:19.2064665Z   test-matrix: {"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}
2024-06-26T07:12:19.2071956Z   job-name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamic_aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu)
2024-06-26T07:12:19.2072844Z env:
2024-06-26T07:12:19.2073110Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:19.2073568Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:19.2074102Z ##[endgroup]
2024-06-26T07:12:19.2114107Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482
2024-06-26T07:12:19.2114654Z with:
2024-06-26T07:12:19.2114926Z   shell: bash
2024-06-26T07:12:19.2115223Z   timeout_minutes: 10
2024-06-26T07:12:19.2115554Z   max_attempts: 5
2024-06-26T07:12:19.2115860Z   retry_wait_seconds: 30
2024-06-26T07:12:19.2116966Z   command: set -eux
# PyYAML 6.0 doesn't work with MacOS x86 anymore
# This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2
python3 -m pip install requests==2.27.1 pyyaml==6.0.1

2024-06-26T07:12:19.2118120Z   polling_interval_seconds: 1
2024-06-26T07:12:19.2118493Z   warning_on_retry: true
2024-06-26T07:12:19.2118835Z   continue_on_error: false
2024-06-26T07:12:19.2119182Z env:
2024-06-26T07:12:19.2119455Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:19.2120038Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:19.2120673Z   GITHUB_TOKEN: ***
2024-06-26T07:12:19.2120988Z ##[endgroup]
2024-06-26T07:12:19.2608681Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1
2024-06-26T07:12:19.4763219Z Defaulting to user installation because normal site-packages is not writeable
2024-06-26T07:12:19.4924140Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.7/site-packages (2.27.1)
2024-06-26T07:12:19.5068916Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.7/site-packages (6.0.1)
2024-06-26T07:12:19.5077828Z Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (3.7)
2024-06-26T07:12:19.5091121Z Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2.0.12)
2024-06-26T07:12:19.5111085Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2024.6.2)
2024-06-26T07:12:19.5119202Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (1.26.19)
2024-06-26T07:12:20.2613367Z Command completed after 1 attempt(s).
2024-06-26T07:12:20.2653626Z ##[group]Run set -x
2024-06-26T07:12:20.2653965Z set -x
2024-06-26T07:12:20.2654259Z 
2024-06-26T07:12:20.2654810Z # Use relative path here as this could be checked out anywhere, not necessarily
2024-06-26T07:12:20.2655476Z # in runner workspace
2024-06-26T07:12:20.2656007Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py"
2024-06-26T07:12:20.2663748Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:12:20.2664247Z env:
2024-06-26T07:12:20.2664532Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:20.2665212Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:20.2665697Z ##[endgroup]
2024-06-26T07:12:20.2687151Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py
2024-06-26T07:12:20.2883570Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}"
2024-06-26T07:12:20.2884132Z echo "Workflow: ${GITHUB_WORKFLOW}"
2024-06-26T07:12:20.2884606Z echo "Job name: ${JOB_NAME}"
2024-06-26T07:12:20.2885234Z 
2024-06-26T07:12:20.2885778Z # Use relative path here as this could be checked out anywhere, not necessarily
2024-06-26T07:12:20.2886463Z # in runner workspace
2024-06-26T07:12:20.2887098Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \
2024-06-26T07:12:20.2887740Z   --workflow "${GITHUB_WORKFLOW}" \
2024-06-26T07:12:20.2888215Z   --job-name "${JOB_NAME}" \
2024-06-26T07:12:20.2895780Z   --test-matrix "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" \
2024-06-26T07:12:20.2902779Z   --selected-test-configs "" \
2024-06-26T07:12:20.2903249Z   --pr-number "${PR_NUMBER}" \
2024-06-26T07:12:20.2903676Z   --tag "${TAG}" \
2024-06-26T07:12:20.2904082Z   --event-name "${EVENT_NAME}" \
2024-06-26T07:12:20.2904526Z   --schedule "${SCHEDULE}" \
2024-06-26T07:12:20.2905056Z   --branch "${HEAD_BRANCH}"
2024-06-26T07:12:20.2912310Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:12:20.2912817Z env:
2024-06-26T07:12:20.2913094Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:20.2913560Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:20.2914247Z   GITHUB_TOKEN: ***
2024-06-26T07:12:20.2915078Z   JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamic_aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu)
2024-06-26T07:12:20.2915965Z   PR_NUMBER: 
2024-06-26T07:12:20.2916288Z   TAG: ciflow/inductor/129470
2024-06-26T07:12:20.2932346Z   EVENT_NAME: push
2024-06-26T07:12:20.2932711Z   SCHEDULE: 
2024-06-26T07:12:20.2933040Z   HEAD_BRANCH: 
2024-06-26T07:12:20.2933354Z ##[endgroup]
2024-06-26T07:12:20.2953186Z Workflow: inductor-periodic
2024-06-26T07:12:20.2954321Z Job name: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamic_aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu)
2024-06-26T07:12:20.5184994Z INFO:root:Found no test-config label on the PR, so all test configs are included
2024-06-26T07:12:20.6631181Z ##[group]Run echo "Filtered matrix:"
2024-06-26T07:12:20.6631637Z echo "Filtered matrix:"
2024-06-26T07:12:20.6638169Z echo "{"include": [{"config": "dynamo_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamo_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_torchbench", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_huggingface", "shard": 1, "num_shards": 1, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "dynamic_aot_eager_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}"
2024-06-26T07:12:20.6645283Z 
2024-06-26T07:12:20.6645559Z echo
2024-06-26T07:12:20.6645945Z echo "Is the current job unstable? False"
2024-06-26T07:12:20.6646549Z 
2024-06-26T07:12:20.6646816Z echo
2024-06-26T07:12:20.6647224Z echo "Is keep-going label set? False"
2024-06-26T07:12:20.6647661Z 
2024-06-26T07:12:20.6647922Z echo
2024-06-26T07:12:20.6648244Z echo "Renabled issues? "
2024-06-26T07:12:20.6656028Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:12:20.6656527Z env:
2024-06-26T07:12:20.6656798Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:20.6657252Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:20.6657731Z ##[endgroup]
2024-06-26T07:12:20.6679986Z Filtered matrix:
2024-06-26T07:12:20.6688217Z {include: [{config: dynamo_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamo_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_torchbench, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_huggingface, shard: 1, num_shards: 1, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: dynamic_aot_eager_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}]}
2024-06-26T07:12:20.6694870Z 
2024-06-26T07:12:20.6695020Z Is the current job unstable? False
2024-06-26T07:12:20.6695312Z 
2024-06-26T07:12:20.6695524Z Is keep-going label set? False
2024-06-26T07:12:20.6695779Z 
2024-06-26T07:12:20.6695910Z Renabled issues? 
2024-06-26T07:12:20.6736623Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}"
2024-06-26T07:12:20.6737542Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}"
2024-06-26T07:12:20.6745229Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:12:20.6745720Z env:
2024-06-26T07:12:20.6745999Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:20.6746461Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:20.6746997Z   JOB_TIMEOUT: 240
2024-06-26T07:12:20.6747306Z ##[endgroup]
2024-06-26T07:12:20.6819036Z ##[group]Run set -x
2024-06-26T07:12:20.6819428Z set -x
2024-06-26T07:12:20.6819728Z 
2024-06-26T07:12:20.6820095Z if [[ $TEST_CONFIG == 'multigpu' ]]; then
2024-06-26T07:12:20.6820630Z   TEST_COMMAND=.ci/pytorch/multigpu-test.sh
2024-06-26T07:12:20.6821193Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then
2024-06-26T07:12:20.6821711Z   TEST_COMMAND=.ci/onnx/test.sh
2024-06-26T07:12:20.6822121Z else
2024-06-26T07:12:20.6822458Z   TEST_COMMAND=.ci/pytorch/test.sh
2024-06-26T07:12:20.6822878Z fi
2024-06-26T07:12:20.6823156Z 
2024-06-26T07:12:20.6823619Z # detached container should get cleaned up by teardown_ec2_linux
2024-06-26T07:12:20.6824368Z # TODO: Stop building test binaries as part of the build phase
2024-06-26T07:12:20.6825041Z # Used for GPU_FLAG since that doesn't play nice
2024-06-26T07:12:20.6825625Z # shellcheck disable=SC2086,SC2090
2024-06-26T07:12:20.6826081Z container_name=$(docker run \
2024-06-26T07:12:20.6826500Z   ${GPU_FLAG:-} \
2024-06-26T07:12:20.6827004Z   -e BUILD_ENVIRONMENT \
2024-06-26T07:12:20.6827400Z   -e PR_NUMBER \
2024-06-26T07:12:20.6827755Z   -e GITHUB_ACTIONS \
2024-06-26T07:12:20.6828146Z   -e GITHUB_REPOSITORY \
2024-06-26T07:12:20.6828548Z   -e GITHUB_WORKFLOW \
2024-06-26T07:12:20.6828934Z   -e GITHUB_JOB \
2024-06-26T07:12:20.6829292Z   -e GITHUB_RUN_ID \
2024-06-26T07:12:20.6829673Z   -e GITHUB_RUN_NUMBER \
2024-06-26T07:12:20.6830083Z   -e GITHUB_RUN_ATTEMPT \
2024-06-26T07:12:20.6830472Z   -e JOB_ID \
2024-06-26T07:12:20.6830809Z   -e JOB_NAME \
2024-06-26T07:12:20.6831159Z   -e BASE_SHA \
2024-06-26T07:12:20.6831501Z   -e BRANCH \
2024-06-26T07:12:20.6831827Z   -e SHA1 \
2024-06-26T07:12:20.6832171Z   -e AWS_DEFAULT_REGION \
2024-06-26T07:12:20.6832577Z   -e IN_WHEEL_TEST \
2024-06-26T07:12:20.6832951Z   -e SHARD_NUMBER \
2024-06-26T07:12:20.6833320Z   -e TEST_CONFIG \
2024-06-26T07:12:20.6833689Z   -e NUM_TEST_SHARDS \
2024-06-26T07:12:20.6834081Z   -e REENABLED_ISSUES \
2024-06-26T07:12:20.6834493Z   -e CONTINUE_THROUGH_ERROR \
2024-06-26T07:12:20.6834908Z   -e VERBOSE_TEST_LOGS \
2024-06-26T07:12:20.6835310Z   -e NO_TEST_TIMEOUT \
2024-06-26T07:12:20.6835687Z   -e NO_TD \
2024-06-26T07:12:20.6836023Z   -e TD_DISTRIBUTED \
2024-06-26T07:12:20.6836403Z   -e PR_LABELS \
2024-06-26T07:12:20.6836809Z   -e MAX_JOBS="$(nproc --ignore=2)" \
2024-06-26T07:12:20.6837265Z   -e SCCACHE_BUCKET \
2024-06-26T07:12:20.6837653Z   -e SCCACHE_S3_KEY_PREFIX \
2024-06-26T07:12:20.6838058Z   -e XLA_CUDA \
2024-06-26T07:12:20.6838462Z   -e XLA_CLANG_CACHE_S3_BUCKET_NAME \
2024-06-26T07:12:20.6838969Z   -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \
2024-06-26T07:12:20.6839483Z   -e PYTORCH_TEST_RERUN_DISABLED_TESTS \
2024-06-26T07:12:20.6839998Z   -e SKIP_SCCACHE_INITIALIZATION=1 \
2024-06-26T07:12:20.6840462Z   -e HUGGING_FACE_HUB_TOKEN \
2024-06-26T07:12:20.6840879Z   -e DASHBOARD_TAG \
2024-06-26T07:12:20.6841347Z   --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \
2024-06-26T07:12:20.6841908Z   --security-opt seccomp=unconfined \
2024-06-26T07:12:20.6842371Z   --cap-add=SYS_PTRACE \
2024-06-26T07:12:20.6842762Z   --ipc=host \
2024-06-26T07:12:20.6843113Z   --shm-size="${SHM_SIZE}" \
2024-06-26T07:12:20.6843502Z   --tty \
2024-06-26T07:12:20.6843815Z   --detach \
2024-06-26T07:12:20.6844170Z   --name="${container_name}" \
2024-06-26T07:12:20.6844576Z   --user jenkins \
2024-06-26T07:12:20.6845398Z   -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \
2024-06-26T07:12:20.6846000Z   -w /var/lib/jenkins/workspace \
2024-06-26T07:12:20.6846439Z   "${DOCKER_IMAGE}"
2024-06-26T07:12:20.6846780Z )
2024-06-26T07:12:20.6847180Z # Propagate download.pytorch.org IP to container
2024-06-26T07:12:20.6848137Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts"
2024-06-26T07:12:20.6849088Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}"
2024-06-26T07:12:20.6849976Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}"
2024-06-26T07:12:20.6857431Z shell: /usr/bin/bash -e {0}
2024-06-26T07:12:20.6857807Z env:
2024-06-26T07:12:20.6858088Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:12:20.6858539Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:20.6859150Z   BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm86
2024-06-26T07:12:20.6859646Z   PR_NUMBER: 
2024-06-26T07:12:20.6859971Z   GITHUB_REPOSITORY: pytorch/pytorch
2024-06-26T07:12:20.6860493Z   GITHUB_WORKFLOW: inductor-periodic
2024-06-26T07:12:20.6860903Z   GITHUB_JOB: test
2024-06-26T07:12:20.6861222Z   GITHUB_RUN_ID: 9673646806
2024-06-26T07:12:20.6861583Z   GITHUB_RUN_NUMBER: 45567
2024-06-26T07:12:20.6861926Z   GITHUB_RUN_ATTEMPT: 1
2024-06-26T07:12:20.6862257Z   JOB_ID: 26688917313
2024-06-26T07:12:20.6863089Z   JOB_NAME: cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamic_aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu)
2024-06-26T07:12:20.6863963Z   BRANCH: 
2024-06-26T07:12:20.6864298Z   SHA1: b8c4c54d347aa776934c60784e35936878ef18dc
2024-06-26T07:12:20.6864809Z   BASE_SHA: b8c4c54d347aa776934c60784e35936878ef18dc
2024-06-26T07:12:20.6865293Z   TEST_CONFIG: dynamic_aot_eager_timm
2024-06-26T07:12:20.6865693Z   SHARD_NUMBER: 1
2024-06-26T07:12:20.6865992Z   NUM_TEST_SHARDS: 2
2024-06-26T07:12:20.6866315Z   REENABLED_ISSUES: 
2024-06-26T07:12:20.6866658Z   CONTINUE_THROUGH_ERROR: False
2024-06-26T07:12:20.6867037Z   VERBOSE_TEST_LOGS: False
2024-06-26T07:12:20.6867405Z   NO_TEST_TIMEOUT: False
2024-06-26T07:12:20.6867737Z   NO_TD: False
2024-06-26T07:12:20.6868035Z   TD_DISTRIBUTED: False
2024-06-26T07:12:20.6868445Z   SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2
2024-06-26T07:12:20.6868959Z   SCCACHE_S3_KEY_PREFIX: inductor-periodic
2024-06-26T07:12:20.6869379Z   SHM_SIZE: 2g
2024-06-26T07:12:20.6870445Z   DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:12:20.6871577Z   XLA_CUDA: 
2024-06-26T07:12:20.6872063Z   XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla
2024-06-26T07:12:20.6872673Z   PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0
2024-06-26T07:12:20.6873106Z   PYTORCH_TEST_RERUN_DISABLED_TESTS: 0
2024-06-26T07:12:20.6873508Z   DASHBOARD_TAG: 
2024-06-26T07:12:20.6874036Z   HUGGING_FACE_HUB_TOKEN: ***
2024-06-26T07:12:20.6874401Z ##[endgroup]
2024-06-26T07:12:20.6894248Z + [[ dynamic_aot_eager_timm == \m\u\l\t\i\g\p\u ]]
2024-06-26T07:12:20.6894969Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *onnx* ]]
2024-06-26T07:12:20.6895494Z + TEST_COMMAND=.ci/pytorch/test.sh
2024-06-26T07:12:20.6901101Z +++ nproc --ignore=2
2024-06-26T07:12:20.6914630Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_9673646806 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:12:31.8956340Z + container_name=e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:12:31.8957523Z + grep download.pytorch.org /etc/hosts
2024-06-26T07:12:31.8958619Z + docker exec -i e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106 sudo bash -c '/bin/cat >> /etc/hosts'
2024-06-26T07:12:31.9566406Z + echo DOCKER_CONTAINER_ID=e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:12:31.9568425Z ++ echo dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl
2024-06-26T07:12:31.9570249Z + docker exec -t e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106 sh -c 'pip install dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh'
2024-06-26T07:12:32.3426035Z Processing ./dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+gitb8c4c54)
2024-06-26T07:12:32.6563367Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.13.1)
2024-06-26T07:12:32.6566694Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (4.12.2)
2024-06-26T07:12:32.6568911Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.12.1)
2024-06-26T07:12:32.6571948Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.8.8)
2024-06-26T07:12:32.6574877Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.1.4)
2024-06-26T07:12:32.6577568Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2024.2.0)
2024-06-26T07:12:32.6603067Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.3.0)
2024-06-26T07:12:32.6661760Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.21.2)
2024-06-26T07:12:32.7095436Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.1.5)
2024-06-26T07:12:32.7254906Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.3.0)
2024-06-26T07:12:34.0517271Z Installing collected packages: torch
2024-06-26T07:12:42.7153169Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
2024-06-26T07:12:42.7154636Z timm 0.9.7 requires torchvision, which is not installed.
2024-06-26T07:12:42.7155389Z Successfully installed torch-2.5.0a0+gitb8c4c54
2024-06-26T07:12:42.8116658Z ++ dirname .ci/pytorch/test.sh
2024-06-26T07:12:42.8121789Z + source .ci/pytorch/common.sh
2024-06-26T07:12:42.8123723Z +++ dirname .ci/pytorch/common.sh
2024-06-26T07:12:42.8129542Z ++ source .ci/pytorch/common_utils.sh
2024-06-26T07:12:42.8130342Z +++ declare -f -t trap_add
2024-06-26T07:12:42.8136417Z ++ set -ex
2024-06-26T07:12:42.8137002Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]]
2024-06-26T07:12:42.8137571Z ++ BUILD_TEST_LIBTORCH=0
2024-06-26T07:12:42.8138791Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *rocm* ]]
2024-06-26T07:12:42.8140692Z ++ stat -c %u /var/lib/jenkins/workspace
2024-06-26T07:12:42.8149306Z + WORKSPACE_ORIGINAL_OWNER_ID=1000
2024-06-26T07:12:42.8149759Z + trap_add cleanup_workspace EXIT
2024-06-26T07:12:42.8150236Z + trap_add_cmd=cleanup_workspace
2024-06-26T07:12:42.8150620Z + shift
2024-06-26T07:12:42.8151263Z + for trap_add_name in "$@"
2024-06-26T07:12:42.8155090Z +++ trap -p EXIT
2024-06-26T07:12:42.8157504Z ++ eval 'extract_trap_cmd '
2024-06-26T07:12:42.8157951Z +++ extract_trap_cmd
2024-06-26T07:12:42.8158304Z +++ printf '%s\n' ''
2024-06-26T07:12:42.8158781Z ++ printf '%s\n' cleanup_workspace
2024-06-26T07:12:42.8159944Z + trap -- '
2024-06-26T07:12:42.8160409Z cleanup_workspace' EXIT
2024-06-26T07:12:42.8161039Z + sudo chown -R jenkins /var/lib/jenkins/workspace
2024-06-26T07:12:43.2864591Z + git config --global --add safe.directory /var/lib/jenkins/workspace
2024-06-26T07:12:43.2877155Z + echo 'Environment variables:'
2024-06-26T07:12:43.2877740Z Environment variables:
2024-06-26T07:12:43.2878168Z + env
2024-06-26T07:12:43.2883508Z INSTALLED_DB=yes
2024-06-26T07:12:43.2884063Z NV_LIBCUBLAS_VERSION=12.1.3.1-1
2024-06-26T07:12:43.2884611Z NVIDIA_VISIBLE_DEVICES=all
2024-06-26T07:12:43.2885406Z NV_NVML_DEV_VERSION=12.1.105-1
2024-06-26T07:12:43.2886263Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch
2024-06-26T07:12:43.2887038Z CONTINUE_THROUGH_ERROR=False
2024-06-26T07:12:43.2887725Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1
2024-06-26T07:12:43.2888367Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1
2024-06-26T07:12:43.2888972Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86
2024-06-26T07:12:43.2889673Z HOSTNAME=e827c3f8b855
2024-06-26T07:12:43.2890891Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_e39de6d8-b310-450f-a17d-90a80fbdfbdf
2024-06-26T07:12:43.2891953Z GITHUB_ACTION=__self
2024-06-26T07:12:43.2892421Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0
2024-06-26T07:12:43.2896442Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526
2024-06-26T07:12:43.2900230Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1
2024-06-26T07:12:43.2900736Z NV_NVTX_VERSION=12.1.105-1
2024-06-26T07:12:43.2901083Z GITHUB_RUN_NUMBER=45567
2024-06-26T07:12:43.2901437Z TEST_CONFIG=dynamic_aot_eager_timm
2024-06-26T07:12:43.2901846Z GITHUB_REPOSITORY_OWNER_ID=21003710
2024-06-26T07:12:43.2902313Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all
2024-06-26T07:12:43.2902767Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1
2024-06-26T07:12:43.2903228Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1
2024-06-26T07:12:43.2903681Z NV_LIBNPP_VERSION=12.1.0.40-1
2024-06-26T07:12:43.2904125Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot]
2024-06-26T07:12:43.2904608Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache
2024-06-26T07:12:43.2905072Z GITHUB_REF_TYPE=tag
2024-06-26T07:12:43.2905397Z TORCH_CUDA_ARCH_LIST=Maxwell
2024-06-26T07:12:43.2905766Z NCCL_VERSION=2.17.1-1
2024-06-26T07:12:43.2906344Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc
2024-06-26T07:12:43.2906793Z XLA_CUDA=
2024-06-26T07:12:43.2907297Z HUGGING_FACE_HUB_TOKEN=***
2024-06-26T07:12:43.2910328Z ***
2024-06-26T07:12:43.2910716Z CARGO_NET_GIT_FETCH_WITH_CLI=true
2024-06-26T07:12:43.2911127Z GITHUB_REPOSITORY_ID=65600975
2024-06-26T07:12:43.2911491Z GITHUB_ACTIONS=true
2024-06-26T07:12:43.2911814Z NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:43.2912334Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1
2024-06-26T07:12:43.2912861Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1
2024-06-26T07:12:43.2913375Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc
2024-06-26T07:12:43.2913866Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev
2024-06-26T07:12:43.2914343Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc
2024-06-26T07:12:43.2915257Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/129470
2024-06-26T07:12:43.2916015Z UCC_HOME=/usr
2024-06-26T07:12:43.2916340Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1
2024-06-26T07:12:43.2916746Z VERBOSE_TEST_LOGS=False
2024-06-26T07:12:43.2917085Z NVIDIA_PRODUCT_NAME=CUDA
2024-06-26T07:12:43.2917535Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1
2024-06-26T07:12:43.2918179Z GITHUB_REF=refs/tags/ciflow/inductor/129470
2024-06-26T07:12:43.2918642Z NV_CUDA_CUDART_VERSION=12.1.105-1
2024-06-26T07:12:43.2919019Z SHARD_NUMBER=1
2024-06-26T07:12:43.2919327Z GITHUB_REF_PROTECTED=false
2024-06-26T07:12:43.2921192Z HOME=/var/lib/jenkins
2024-06-26T07:12:43.2921736Z GITHUB_API_URL=https://api.github.com
2024-06-26T07:12:43.2922330Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0
2024-06-26T07:12:43.2922929Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb
2024-06-26T07:12:43.2923561Z SCCACHE_S3_KEY_PREFIX=inductor-periodic
2024-06-26T07:12:43.2924073Z CUDA_VERSION=12.1.1
2024-06-26T07:12:43.2924609Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1
2024-06-26T07:12:43.2925448Z NUM_TEST_SHARDS=2
2024-06-26T07:12:43.2925769Z UCX_HOME=/usr
2024-06-26T07:12:43.2926309Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1
2024-06-26T07:12:43.2927361Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_e39de6d8-b310-450f-a17d-90a80fbdfbdf
2024-06-26T07:12:43.2928770Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamic_aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu)
2024-06-26T07:12:43.2930133Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_e39de6d8-b310-450f-a17d-90a80fbdfbdf
2024-06-26T07:12:43.2931360Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json
2024-06-26T07:12:43.2932005Z GITHUB_EVENT_NAME=push
2024-06-26T07:12:43.2932327Z DASHBOARD_TAG=
2024-06-26T07:12:43.2932629Z GITHUB_RUN_ID=9673646806
2024-06-26T07:12:43.2933085Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1
2024-06-26T07:12:43.2933609Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1
2024-06-26T07:12:43.2934588Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_e39de6d8-b310-450f-a17d-90a80fbdfbdf
2024-06-26T07:12:43.2935477Z GITHUB_ACTOR=pytorch-bot[bot]
2024-06-26T07:12:43.2935882Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1
2024-06-26T07:12:43.2936297Z PR_NUMBER=
2024-06-26T07:12:43.2936572Z GITHUB_RUN_ATTEMPT=1
2024-06-26T07:12:43.2936909Z ANACONDA_PYTHON_VERSION=3.10
2024-06-26T07:12:43.2937348Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql
2024-06-26T07:12:43.2937799Z TERM=xterm
2024-06-26T07:12:43.2938123Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1
2024-06-26T07:12:43.2938530Z INSTALLED_VISION=yes
2024-06-26T07:12:43.2938846Z BRANCH=
2024-06-26T07:12:43.2939123Z OPENSSL_ROOT_DIR=/opt/openssl
2024-06-26T07:12:43.2939518Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs
2024-06-26T07:12:43.2939936Z CUDA_PATH=/usr/local/cuda
2024-06-26T07:12:43.2940679Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux
2024-06-26T07:12:43.2941421Z GITHUB_SERVER_URL=https://github.com
2024-06-26T07:12:43.2942040Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b
2024-06-26T07:12:43.2942515Z REENABLED_ISSUES=
2024-06-26T07:12:43.2942811Z SHLVL=1
2024-06-26T07:12:43.2943074Z MAX_JOBS=14
2024-06-26T07:12:43.2943440Z NV_CUDA_LIB_VERSION=12.1.1-1
2024-06-26T07:12:43.2943801Z NVARCH=x86_64
2024-06-26T07:12:43.2944088Z GITHUB_ACTOR_ID=54816060
2024-06-26T07:12:43.2944536Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc
2024-06-26T07:12:43.2945066Z GITHUB_REF_NAME=ciflow/inductor/129470
2024-06-26T07:12:43.2945549Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1
2024-06-26T07:12:43.2946193Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla
2024-06-26T07:12:43.2946757Z GITHUB_JOB=test
2024-06-26T07:12:43.2947147Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1
2024-06-26T07:12:43.2947703Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
2024-06-26T07:12:43.2948202Z NO_TEST_TIMEOUT=False
2024-06-26T07:12:43.2948530Z TD_DISTRIBUTED=False
2024-06-26T07:12:43.2948902Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1
2024-06-26T07:12:43.2949346Z GITHUB_REPOSITORY=pytorch/pytorch
2024-06-26T07:12:43.2949752Z NV_NVPROF_VERSION=12.1.105-1
2024-06-26T07:12:43.2950110Z GITHUB_RETENTION_DAYS=90
2024-06-26T07:12:43.2950561Z OPENSSL_DIR=/opt/openssl
2024-06-26T07:12:43.2950903Z GITHUB_ACTION_REPOSITORY=
2024-06-26T07:12:43.2951923Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2024-06-26T07:12:43.2952965Z GITHUB_BASE_REF=
2024-06-26T07:12:43.2953296Z NV_LIBNCCL_PACKAGE_NAME=libnccl2
2024-06-26T07:12:43.2953656Z CI=true
2024-06-26T07:12:43.2953979Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1
2024-06-26T07:12:43.2954392Z GITHUB_REPOSITORY_OWNER=pytorch
2024-06-26T07:12:43.2954761Z JOB_ID=26688917313
2024-06-26T07:12:43.2955066Z INSTALLED_PROTOBUF=yes
2024-06-26T07:12:43.2955391Z GITHUB_HEAD_REF=
2024-06-26T07:12:43.2955690Z GITHUB_ACTION_REF=
2024-06-26T07:12:43.2956108Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2
2024-06-26T07:12:43.2956631Z GITHUB_WORKFLOW=inductor-periodic
2024-06-26T07:12:43.2957035Z DEBIAN_FRONTEND=noninteractive
2024-06-26T07:12:43.2957917Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_e39de6d8-b310-450f-a17d-90a80fbdfbdf
2024-06-26T07:12:43.2958717Z NO_TD=False
2024-06-26T07:12:43.2959024Z SKIP_SCCACHE_INITIALIZATION=1
2024-06-26T07:12:43.2959382Z _=/usr/bin/env
2024-06-26T07:12:43.2959851Z ++ python -c 'import site; print(site.getsitepackages()[0])'
2024-06-26T07:12:43.3032886Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch
2024-06-26T07:12:43.3034300Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin
2024-06-26T07:12:43.3035471Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib
2024-06-26T07:12:43.3036348Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test
2024-06-26T07:12:43.3036954Z + BUILD_DIR=build
2024-06-26T07:12:43.3037305Z + BUILD_RENAMED_DIR=build_renamed
2024-06-26T07:12:43.3037706Z + BUILD_BIN_DIR=build/bin
2024-06-26T07:12:43.3038043Z + SHARD_NUMBER=1
2024-06-26T07:12:43.3038336Z + NUM_TEST_SHARDS=2
2024-06-26T07:12:43.3038748Z + export VALGRIND=ON
2024-06-26T07:12:43.3039069Z + VALGRIND=ON
2024-06-26T07:12:43.3039530Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *clang9* ]]
2024-06-26T07:12:43.3040030Z + [[ 0 == \1 ]]
2024-06-26T07:12:43.3040338Z + [[ False == \1 ]]
2024-06-26T07:12:43.3040811Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *bazel* ]]
2024-06-26T07:12:43.3041321Z ++ realpath build/custom_test_artifacts
2024-06-26T07:12:43.3046293Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts
2024-06-26T07:12:43.3047077Z + [[ -n '' ]]
2024-06-26T07:12:43.3047432Z + echo 'Environment variables'
2024-06-26T07:12:43.3047814Z Environment variables
2024-06-26T07:12:43.3048121Z + env
2024-06-26T07:12:43.3052396Z INSTALLED_DB=yes
2024-06-26T07:12:43.3052944Z NV_LIBCUBLAS_VERSION=12.1.3.1-1
2024-06-26T07:12:43.3053703Z NVIDIA_VISIBLE_DEVICES=all
2024-06-26T07:12:43.3054232Z NV_NVML_DEV_VERSION=12.1.105-1
2024-06-26T07:12:43.3055000Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch
2024-06-26T07:12:43.3055721Z CONTINUE_THROUGH_ERROR=False
2024-06-26T07:12:43.3056245Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1
2024-06-26T07:12:43.3056937Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1
2024-06-26T07:12:43.3057703Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86
2024-06-26T07:12:43.3058343Z HOSTNAME=e827c3f8b855
2024-06-26T07:12:43.3059348Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_e39de6d8-b310-450f-a17d-90a80fbdfbdf
2024-06-26T07:12:43.3060319Z GITHUB_ACTION=__self
2024-06-26T07:12:43.3060708Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0
2024-06-26T07:12:43.3065006Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526
2024-06-26T07:12:43.3068987Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1
2024-06-26T07:12:43.3069492Z NV_NVTX_VERSION=12.1.105-1
2024-06-26T07:12:43.3069840Z GITHUB_RUN_NUMBER=45567
2024-06-26T07:12:43.3070197Z TEST_CONFIG=dynamic_aot_eager_timm
2024-06-26T07:12:43.3070597Z GITHUB_REPOSITORY_OWNER_ID=21003710
2024-06-26T07:12:43.3071056Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all
2024-06-26T07:12:43.3071521Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1
2024-06-26T07:12:43.3071952Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1
2024-06-26T07:12:43.3072355Z NV_LIBNPP_VERSION=12.1.0.40-1
2024-06-26T07:12:43.3072796Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot]
2024-06-26T07:12:43.3073313Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache
2024-06-26T07:12:43.3073782Z GITHUB_REF_TYPE=tag
2024-06-26T07:12:43.3074104Z TORCH_CUDA_ARCH_LIST=Maxwell
2024-06-26T07:12:43.3074481Z NCCL_VERSION=2.17.1-1
2024-06-26T07:12:43.3074860Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc
2024-06-26T07:12:43.3075283Z XLA_CUDA=
2024-06-26T07:12:43.3075758Z HUGGING_FACE_HUB_TOKEN=***
2024-06-26T07:12:43.3076147Z ***
2024-06-26T07:12:43.3076423Z CARGO_NET_GIT_FETCH_WITH_CLI=true
2024-06-26T07:12:43.3076809Z GITHUB_REPOSITORY_ID=65600975
2024-06-26T07:12:43.3077169Z GITHUB_ACTIONS=true
2024-06-26T07:12:43.3077499Z NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:12:43.3077983Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1
2024-06-26T07:12:43.3078504Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1
2024-06-26T07:12:43.3078964Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc
2024-06-26T07:12:43.3079459Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev
2024-06-26T07:12:43.3079938Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc
2024-06-26T07:12:43.3080832Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor-periodic.yml@refs/tags/ciflow/inductor/129470
2024-06-26T07:12:43.3081583Z UCC_HOME=/usr
2024-06-26T07:12:43.3081913Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1
2024-06-26T07:12:43.3082308Z VERBOSE_TEST_LOGS=False
2024-06-26T07:12:43.3082646Z NVIDIA_PRODUCT_NAME=CUDA
2024-06-26T07:12:43.3083103Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1
2024-06-26T07:12:43.3083638Z GITHUB_REF=refs/tags/ciflow/inductor/129470
2024-06-26T07:12:43.3084095Z NV_CUDA_CUDART_VERSION=12.1.105-1
2024-06-26T07:12:43.3084469Z SHARD_NUMBER=1
2024-06-26T07:12:43.3085128Z GITHUB_REF_PROTECTED=false
2024-06-26T07:12:43.3085485Z HOME=/var/lib/jenkins
2024-06-26T07:12:43.3085840Z GITHUB_API_URL=https://api.github.com
2024-06-26T07:12:43.3086376Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0
2024-06-26T07:12:43.3086851Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb
2024-06-26T07:12:43.3087406Z SCCACHE_S3_KEY_PREFIX=inductor-periodic
2024-06-26T07:12:43.3087810Z CUDA_VERSION=12.1.1
2024-06-26T07:12:43.3088216Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1
2024-06-26T07:12:43.3088645Z NUM_TEST_SHARDS=2
2024-06-26T07:12:43.3088934Z UCX_HOME=/usr
2024-06-26T07:12:43.3089440Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1
2024-06-26T07:12:43.3090586Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_e39de6d8-b310-450f-a17d-90a80fbdfbdf
2024-06-26T07:12:43.3091982Z JOB_NAME=cuda12.1-py3.10-gcc9-sm86-periodic-dynamo-benchmarks / test (dynamic_aot_eager_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu)
2024-06-26T07:12:43.3093340Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_e39de6d8-b310-450f-a17d-90a80fbdfbdf
2024-06-26T07:12:43.3094429Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json
2024-06-26T07:12:43.3095069Z GITHUB_EVENT_NAME=push
2024-06-26T07:12:43.3095528Z DASHBOARD_TAG=
2024-06-26T07:12:43.3095826Z GITHUB_RUN_ID=9673646806
2024-06-26T07:12:43.3096271Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1
2024-06-26T07:12:43.3096799Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1
2024-06-26T07:12:43.3097771Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_e39de6d8-b310-450f-a17d-90a80fbdfbdf
2024-06-26T07:12:43.3098660Z GITHUB_ACTOR=pytorch-bot[bot]
2024-06-26T07:12:43.3099058Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1
2024-06-26T07:12:43.3099434Z PR_NUMBER=
2024-06-26T07:12:43.3099714Z GITHUB_RUN_ATTEMPT=1
2024-06-26T07:12:43.3100021Z VALGRIND=ON
2024-06-26T07:12:43.3100313Z ANACONDA_PYTHON_VERSION=3.10
2024-06-26T07:12:43.3100749Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql
2024-06-26T07:12:43.3101199Z TERM=xterm
2024-06-26T07:12:43.3101525Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1
2024-06-26T07:12:43.3101934Z INSTALLED_VISION=yes
2024-06-26T07:12:43.3102241Z BRANCH=
2024-06-26T07:12:43.3102524Z OPENSSL_ROOT_DIR=/opt/openssl
2024-06-26T07:12:43.3102924Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs
2024-06-26T07:12:43.3103342Z CUDA_PATH=/usr/local/cuda
2024-06-26T07:12:43.3104077Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux
2024-06-26T07:12:43.3104810Z GITHUB_SERVER_URL=https://github.com
2024-06-26T07:12:43.3105298Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b
2024-06-26T07:12:43.3105758Z REENABLED_ISSUES=
2024-06-26T07:12:43.3106051Z SHLVL=1
2024-06-26T07:12:43.3106301Z MAX_JOBS=14
2024-06-26T07:12:43.3106609Z NV_CUDA_LIB_VERSION=12.1.1-1
2024-06-26T07:12:43.3106958Z NVARCH=x86_64
2024-06-26T07:12:43.3107259Z GITHUB_ACTOR_ID=54816060
2024-06-26T07:12:43.3107700Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc
2024-06-26T07:12:43.3108221Z GITHUB_REF_NAME=ciflow/inductor/129470
2024-06-26T07:12:43.3108703Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1
2024-06-26T07:12:43.3109342Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla
2024-06-26T07:12:43.3109911Z GITHUB_JOB=test
2024-06-26T07:12:43.3110303Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1
2024-06-26T07:12:43.3110858Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
2024-06-26T07:12:43.3128057Z NO_TEST_TIMEOUT=False
2024-06-26T07:12:43.3128411Z TD_DISTRIBUTED=False
2024-06-26T07:12:43.3128816Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1
2024-06-26T07:12:43.3129263Z GITHUB_REPOSITORY=pytorch/pytorch
2024-06-26T07:12:43.3129685Z NV_NVPROF_VERSION=12.1.105-1
2024-06-26T07:12:43.3130052Z GITHUB_RETENTION_DAYS=90
2024-06-26T07:12:43.3130397Z OPENSSL_DIR=/opt/openssl
2024-06-26T07:12:43.3130862Z GITHUB_ACTION_REPOSITORY=
2024-06-26T07:12:43.3131888Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2024-06-26T07:12:43.3133073Z GITHUB_BASE_REF=
2024-06-26T07:12:43.3133403Z NV_LIBNCCL_PACKAGE_NAME=libnccl2
2024-06-26T07:12:43.3133781Z CI=true
2024-06-26T07:12:43.3134121Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1
2024-06-26T07:12:43.3134530Z GITHUB_REPOSITORY_OWNER=pytorch
2024-06-26T07:12:43.3134905Z JOB_ID=26688917313
2024-06-26T07:12:43.3135221Z INSTALLED_PROTOBUF=yes
2024-06-26T07:12:43.3135554Z GITHUB_HEAD_REF=
2024-06-26T07:12:43.3135855Z GITHUB_ACTION_REF=
2024-06-26T07:12:43.3136296Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2
2024-06-26T07:12:43.3136815Z GITHUB_WORKFLOW=inductor-periodic
2024-06-26T07:12:43.3137222Z DEBIAN_FRONTEND=noninteractive
2024-06-26T07:12:43.3138099Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_e39de6d8-b310-450f-a17d-90a80fbdfbdf
2024-06-26T07:12:43.3138910Z NO_TD=False
2024-06-26T07:12:43.3139218Z SKIP_SCCACHE_INITIALIZATION=1
2024-06-26T07:12:43.3139575Z _=/usr/bin/env
2024-06-26T07:12:43.3139913Z + echo 'Testing pytorch'
2024-06-26T07:12:43.3140266Z Testing pytorch
2024-06-26T07:12:43.3140588Z + export LANG=C.UTF-8
2024-06-26T07:12:43.3140919Z + LANG=C.UTF-8
2024-06-26T07:12:43.3141210Z + PR_NUMBER=
2024-06-26T07:12:43.3141660Z + [[ dynamic_aot_eager_timm == \d\e\f\a\u\l\t ]]
2024-06-26T07:12:43.3142191Z + [[ dynamic_aot_eager_timm == \d\i\s\t\r\i\b\u\t\e\d ]]
2024-06-26T07:12:43.3142692Z + [[ dynamic_aot_eager_timm == \s\l\o\w ]]
2024-06-26T07:12:43.3143340Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *slow-gradcheck* ]]
2024-06-26T07:12:43.3144031Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *cuda* ]]
2024-06-26T07:12:43.3144573Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda
2024-06-26T07:12:43.3145041Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda
2024-06-26T07:12:43.3145498Z + [[ dynamic_aot_eager_timm == *crossref* ]]
2024-06-26T07:12:43.3146073Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]]
2024-06-26T07:12:43.3146701Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *xpu* ]]
2024-06-26T07:12:43.3147340Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]]
2024-06-26T07:12:43.3147892Z + pip_install --user ninja==1.10.2
2024-06-26T07:12:43.3148420Z + pip install --progress-bar off --user ninja==1.10.2
2024-06-26T07:12:43.6717279Z Collecting ninja==1.10.2
2024-06-26T07:12:43.6868065Z   Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB)
2024-06-26T07:12:43.6996169Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB)
2024-06-26T07:12:44.9855627Z Installing collected packages: ninja
2024-06-26T07:12:44.9921288Z   WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH.
2024-06-26T07:12:44.9922539Z   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
2024-06-26T07:12:44.9962398Z Successfully installed ninja-1.10.2
2024-06-26T07:12:45.0903873Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2024-06-26T07:12:45.0906011Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2024-06-26T07:12:45.0907411Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]]
2024-06-26T07:12:45.0907916Z + install_tlparse
2024-06-26T07:12:45.0908293Z + pip_install --user tlparse==0.3.7
2024-06-26T07:12:45.0908831Z + pip install --progress-bar off --user tlparse==0.3.7
2024-06-26T07:12:45.4395557Z Collecting tlparse==0.3.7
2024-06-26T07:12:45.4548164Z   Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes)
2024-06-26T07:12:45.4656822Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB)
2024-06-26T07:12:46.7697903Z Installing collected packages: tlparse
2024-06-26T07:12:46.8057769Z Successfully installed tlparse-0.3.7
2024-06-26T07:12:46.8959225Z ++ python -m site --user-base
2024-06-26T07:12:46.9113146Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2024-06-26T07:12:46.9115104Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *asan* ]]
2024-06-26T07:12:46.9115883Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-debug* ]]
2024-06-26T07:12:46.9116553Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]]
2024-06-26T07:12:46.9117604Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass'
2024-06-26T07:12:46.9118733Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass
2024-06-26T07:12:46.9119421Z + cd test
2024-06-26T07:12:46.9119939Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)'
2024-06-26T07:12:48.4062971Z + [[ dynamic_aot_eager_timm == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]]
2024-06-26T07:12:48.4063619Z + [[ dynamic_aot_eager_timm == \n\o\g\p\u\_\A\V\X\5\1\2 ]]
2024-06-26T07:12:48.4064686Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]]
2024-06-26T07:12:48.4065198Z + pushd test
2024-06-26T07:12:48.4065524Z ~/workspace/test ~/workspace
2024-06-26T07:12:48.4066051Z ++ python -c 'import torch; print(torch.version.cuda)'
2024-06-26T07:12:49.8605610Z + CUDA_VERSION=12.1
2024-06-26T07:12:49.8606231Z + '[' 12.1 == 12.4 ']'
2024-06-26T07:12:49.8606582Z + ISCUDA124=
2024-06-26T07:12:49.8606882Z + popd
2024-06-26T07:12:49.8607160Z ~/workspace
2024-06-26T07:12:49.8611364Z + DYNAMO_BENCHMARK_FLAGS=()
2024-06-26T07:12:49.8612036Z + [[ dynamic_aot_eager_timm == *dynamo_eager* ]]
2024-06-26T07:12:49.8612820Z + [[ dynamic_aot_eager_timm == *aot_eager* ]]
2024-06-26T07:12:49.8613698Z + DYNAMO_BENCHMARK_FLAGS+=(--backend aot_eager)
2024-06-26T07:12:49.8614469Z + [[ dynamic_aot_eager_timm == *dynamic* ]]
2024-06-26T07:12:49.8615463Z + DYNAMO_BENCHMARK_FLAGS+=(--dynamic-shapes --dynamic-batch-only)
2024-06-26T07:12:49.8616367Z + [[ dynamic_aot_eager_timm == *cpu_inductor* ]]
2024-06-26T07:12:49.8617103Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda)
2024-06-26T07:12:49.8642131Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]]
2024-06-26T07:12:49.8643032Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-bazel-* ]]
2024-06-26T07:12:49.8643814Z + cd test
2024-06-26T07:12:49.8644558Z + python -c 'import torch; print(torch.__config__.show())'
2024-06-26T07:12:51.2096387Z PyTorch built with:
2024-06-26T07:12:51.2096927Z   - GCC 9.4
2024-06-26T07:12:51.2097303Z   - C++ Version: 201703
2024-06-26T07:12:51.2098142Z   - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
2024-06-26T07:12:51.2099224Z   - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67)
2024-06-26T07:12:51.2099885Z   - OpenMP 201511 (a.k.a. OpenMP 4.5)
2024-06-26T07:12:51.2100420Z   - LAPACK is enabled (usually provided by MKL)
2024-06-26T07:12:51.2100897Z   - NNPACK is enabled
2024-06-26T07:12:51.2101288Z   - CPU capability usage: AVX2
2024-06-26T07:12:51.2101708Z   - CUDA Runtime 12.1
2024-06-26T07:12:51.2102233Z   - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86
2024-06-26T07:12:51.2102823Z   - CuDNN 90.1  (built against CUDA 12.4)
2024-06-26T07:12:51.2103262Z   - Magma 2.6.1
2024-06-26T07:12:51.2110402Z   - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 
2024-06-26T07:12:51.2116823Z 
2024-06-26T07:12:51.4819878Z + cd test
2024-06-26T07:12:51.4820540Z + python -c 'import torch; print(torch.__config__.parallel_info())'
2024-06-26T07:12:52.6989297Z ATen/Parallel:
2024-06-26T07:12:52.6990211Z 	at::get_num_threads() : 8
2024-06-26T07:12:52.6990658Z 	at::get_num_interop_threads() : 16
2024-06-26T07:12:52.6991081Z OpenMP 201511 (a.k.a. OpenMP 4.5)
2024-06-26T07:12:52.6991474Z 	omp_get_max_threads() : 8
2024-06-26T07:12:52.6992634Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
2024-06-26T07:12:52.6993430Z 	mkl_get_max_threads() : 8
2024-06-26T07:12:52.6994046Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67)
2024-06-26T07:12:52.6994674Z std::thread::hardware_concurrency() : 16
2024-06-26T07:12:52.6995094Z Environment variables:
2024-06-26T07:12:52.6995444Z 	OMP_NUM_THREADS : [not set]
2024-06-26T07:12:52.6995813Z 	MKL_NUM_THREADS : [not set]
2024-06-26T07:12:52.6996187Z ATen parallel backend: OpenMP
2024-06-26T07:12:52.6996437Z 
2024-06-26T07:12:52.9411344Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]]
2024-06-26T07:12:52.9411935Z + [[ dynamic_aot_eager_timm == *backward* ]]
2024-06-26T07:12:52.9412400Z + [[ dynamic_aot_eager_timm == *xla* ]]
2024-06-26T07:12:52.9412948Z + [[ dynamic_aot_eager_timm == *executorch* ]]
2024-06-26T07:12:52.9413454Z + [[ dynamic_aot_eager_timm == \j\i\t\_\l\e\g\a\c\y ]]
2024-06-26T07:12:52.9414094Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]]
2024-06-26T07:12:52.9414638Z + [[ dynamic_aot_eager_timm == distributed ]]
2024-06-26T07:12:52.9415172Z + [[ dynamic_aot_eager_timm == *inductor_distributed* ]]
2024-06-26T07:12:52.9415835Z + [[ dynamic_aot_eager_timm == *inductor-halide* ]]
2024-06-26T07:12:52.9416461Z + [[ dynamic_aot_eager_timm == *inductor-micro-benchmark* ]]
2024-06-26T07:12:52.9417011Z + [[ dynamic_aot_eager_timm == *huggingface* ]]
2024-06-26T07:12:52.9417487Z + [[ dynamic_aot_eager_timm == *timm* ]]
2024-06-26T07:12:52.9417906Z + install_torchvision
2024-06-26T07:12:52.9418243Z + local orig_preload
2024-06-26T07:12:52.9418562Z + local commit
2024-06-26T07:12:52.9418876Z ++ get_pinned_commit vision
2024-06-26T07:12:52.9419274Z ++ cat .github/ci_commit_pins/vision.txt
2024-06-26T07:12:52.9426635Z + commit=d23a6e1664d20707c11781299611436e1f0c104f
2024-06-26T07:12:52.9427101Z + orig_preload=
2024-06-26T07:12:52.9427497Z + '[' -n '' ']'
2024-06-26T07:12:52.9428316Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f
2024-06-26T07:12:52.9429719Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f
2024-06-26T07:12:53.2542917Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f
2024-06-26T07:12:53.2546491Z   Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-jcvsdsug
2024-06-26T07:12:53.2565222Z   Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-jcvsdsug
2024-06-26T07:12:54.6252016Z   Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f'
2024-06-26T07:12:54.6265731Z   Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f
2024-06-26T07:12:55.8365353Z   Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f
2024-06-26T07:12:56.0532883Z   Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f
2024-06-26T07:12:58.3422557Z   Preparing metadata (setup.py) ... [?25l- \ done
2024-06-26T07:12:58.3469209Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.21.2)
2024-06-26T07:12:58.3471576Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+gitb8c4c54)
2024-06-26T07:12:58.3477141Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0)
2024-06-26T07:12:58.3678562Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1)
2024-06-26T07:12:58.3685466Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2)
2024-06-26T07:12:58.3687626Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.12.1)
2024-06-26T07:12:58.3690806Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8)
2024-06-26T07:12:58.3693258Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4)
2024-06-26T07:12:58.3696009Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.2.0)
2024-06-26T07:12:58.4123849Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5)
2024-06-26T07:12:58.4275818Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0)
2024-06-26T07:12:58.4349532Z Building wheels for collected packages: torchvision
2024-06-26T07:14:11.3915582Z   Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ done
2024-06-26T07:14:11.3942817Z [?25h  Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2112723 sha256=5ea9a65e9f56fd91e50a225b12b481a7c56321a5aa7af894d1e610e4170a3a49
2024-06-26T07:14:11.3944483Z   Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96
2024-06-26T07:14:11.3980175Z Successfully built torchvision
2024-06-26T07:14:12.5951728Z Installing collected packages: torchvision
2024-06-26T07:14:12.9780922Z Successfully installed torchvision-0.19.0a0+d23a6e1
2024-06-26T07:14:13.0912070Z + '[' -n '' ']'
2024-06-26T07:14:13.0912838Z + id=0
2024-06-26T07:14:13.0915600Z + test_dynamo_benchmark timm_models 0
2024-06-26T07:14:13.0916782Z ++ pwd
2024-06-26T07:14:13.0918038Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports
2024-06-26T07:14:13.0918770Z + local suite=timm_models
2024-06-26T07:14:13.0919109Z + shift
2024-06-26T07:14:13.0919388Z + local shard_id=0
2024-06-26T07:14:13.0921297Z + shift
2024-06-26T07:14:13.0921840Z + [[ dynamic_aot_eager_timm == *perf_compare* ]]
2024-06-26T07:14:13.0922496Z + [[ dynamic_aot_eager_timm == *perf* ]]
2024-06-26T07:14:13.0923107Z + [[ dynamic_aot_eager_timm == *cpu_inductor* ]]
2024-06-26T07:14:13.0923808Z + [[ dynamic_aot_eager_timm == *aot_inductor* ]]
2024-06-26T07:14:13.0925313Z + test_single_dynamo_benchmark inference timm_models 0 --inference --bfloat16
2024-06-26T07:14:13.0925907Z ++ pwd
2024-06-26T07:14:13.0926392Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports
2024-06-26T07:14:13.0927058Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports
2024-06-26T07:14:13.0937332Z + local name=inference
2024-06-26T07:14:13.0938008Z + shift
2024-06-26T07:14:13.0938432Z + local suite=timm_models
2024-06-26T07:14:13.0938847Z + shift
2024-06-26T07:14:13.0939133Z + local shard_id=0
2024-06-26T07:14:13.0939443Z + shift
2024-06-26T07:14:13.0939769Z + partition_flags=()
2024-06-26T07:14:13.0940114Z + local partition_flags
2024-06-26T07:14:13.0940545Z + [[ -n 2 ]]
2024-06-26T07:14:13.0940859Z + [[ -n 0 ]]
2024-06-26T07:14:13.0941500Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id")
2024-06-26T07:14:13.0942251Z + [[ dynamic_aot_eager_timm == *perf_compare* ]]
2024-06-26T07:14:13.0942755Z + [[ dynamic_aot_eager_timm == *perf* ]]
2024-06-26T07:14:13.0943281Z + [[ dynamic_aot_eager_timm == *aot_inductor* ]]
2024-06-26T07:14:13.0945243Z + python benchmarks/dynamo/timm_models.py --ci --accuracy --timing --explain --backend aot_eager --dynamic-shapes --dynamic-batch-only --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 0 --output /var/lib/jenkins/workspace/test/test-reports/inference_timm_models.csv
2024-06-26T07:14:18.8665470Z 
2024-06-26T07:14:20.7716578Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:14:20.7717034Z 
2024-06-26T07:14:20.8976624Z model.safetensors:   0% 0.00/95.5M [00:00<?, ?B/s]
2024-06-26T07:14:20.8977038Z 
2024-06-26T07:14:21.0053064Z model.safetensors:  22% 21.0M/95.5M [00:00<00:00, 167MB/s]
2024-06-26T07:14:21.0053510Z 
2024-06-26T07:14:21.0816817Z model.safetensors:  55% 52.4M/95.5M [00:00<00:00, 236MB/s]
2024-06-26T07:14:21.0817471Z model.safetensors: 100% 95.5M/95.5M [00:00<00:00, 308MB/s]
2024-06-26T07:14:21.6295802Z 
2024-06-26T07:14:21.6296266Z loading model: 0it [00:02, ?it/s]
2024-06-26T07:14:21.6419655Z cuda eval  adv_inception_v3                   
2024-06-26T07:14:33.9376111Z pass
2024-06-26T07:14:33.9400162Z TIMING: backend_compile:5.68583 entire_frame_compile:11.25958
2024-06-26T07:14:33.9401508Z STATS: call_* op count: 816 | FakeTensor.__torch_dispatch__:620 | FakeTensorMode.__torch_dispatch__:35069 | ProxyTorchDispatchMode.__torch_dispatch__:7234
2024-06-26T07:14:33.9402678Z Dynamo produced 1 graphs covering 816 ops with 0 graph breaks (0 unique)
2024-06-26T07:14:37.8333732Z 
2024-06-26T07:14:38.7592950Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:14:38.7593324Z 
2024-06-26T07:14:38.8667317Z model.safetensors:   0% 0.00/350M [00:00<?, ?B/s]
2024-06-26T07:14:38.8667737Z 
2024-06-26T07:14:38.9840192Z model.safetensors:  15% 52.4M/350M [00:00<00:00, 489MB/s]
2024-06-26T07:14:38.9840632Z 
2024-06-26T07:14:39.0889423Z model.safetensors:  33% 115M/350M [00:00<00:00, 518MB/s] 
2024-06-26T07:14:39.0889881Z 
2024-06-26T07:14:39.2219065Z model.safetensors:  48% 168M/350M [00:00<00:00, 510MB/s]
2024-06-26T07:14:39.2219491Z 
2024-06-26T07:14:39.3331803Z model.safetensors:  63% 220M/350M [00:00<00:00, 459MB/s]
2024-06-26T07:14:39.3332235Z 
2024-06-26T07:14:39.6117735Z model.safetensors:  78% 273M/350M [00:00<00:00, 463MB/s]
2024-06-26T07:14:39.6118173Z 
2024-06-26T07:14:39.6730274Z model.safetensors:  93% 325M/350M [00:00<00:00, 311MB/s]
2024-06-26T07:14:39.6730993Z model.safetensors: 100% 350M/350M [00:00<00:00, 383MB/s]
2024-06-26T07:14:40.1381797Z 
2024-06-26T07:14:40.1382392Z loading model: 0it [00:02, ?it/s]
2024-06-26T07:14:40.1440147Z cuda eval  beit_base_patch16_224              
2024-06-26T07:14:47.8110304Z pass
2024-06-26T07:14:47.8120181Z TIMING: backend_compile:3.52436 entire_frame_compile:6.02425
2024-06-26T07:14:47.8121377Z STATS: call_* op count: 818 | FakeTensor.__torch_dispatch__:752 | FakeTensorMode.__torch_dispatch__:19139 | ProxyTorchDispatchMode.__torch_dispatch__:6533
2024-06-26T07:14:47.8122820Z Dynamo produced 1 graphs covering 818 ops with 0 graph breaks (0 unique)
2024-06-26T07:14:51.4422036Z 
2024-06-26T07:14:51.6435749Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:14:51.6436107Z 
2024-06-26T07:14:51.7537608Z model.safetensors:   0% 0.00/50.1M [00:00<?, ?B/s]
2024-06-26T07:14:51.7538007Z 
2024-06-26T07:14:51.7560271Z model.safetensors: 100% 50.1M/50.1M [00:00<00:00, 455MB/s]
2024-06-26T07:14:51.7560892Z model.safetensors: 100% 50.1M/50.1M [00:00<00:00, 446MB/s]
2024-06-26T07:14:52.2166014Z 
2024-06-26T07:14:52.2166854Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:14:52.2209389Z cuda eval  botnet26t_256                      
2024-06-26T07:14:58.1466645Z pass
2024-06-26T07:14:58.1483601Z TIMING: backend_compile:2.77617 entire_frame_compile:5.35029
2024-06-26T07:14:58.1485064Z STATS: call_* op count: 612 | FakeTensor.__torch_dispatch__:223 | FakeTensorMode.__torch_dispatch__:16625 | ProxyTorchDispatchMode.__torch_dispatch__:4131
2024-06-26T07:14:58.1486181Z Dynamo produced 1 graphs covering 612 ops with 0 graph breaks (0 unique)
2024-06-26T07:15:01.7617303Z 
2024-06-26T07:15:04.5245327Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:15:04.5245662Z 
2024-06-26T07:15:04.6449367Z model.safetensors:   0% 0.00/1.08G [00:00<?, ?B/s]
2024-06-26T07:15:04.6449784Z 
2024-06-26T07:15:04.7618485Z model.safetensors:   6% 62.9M/1.08G [00:00<00:01, 524MB/s]
2024-06-26T07:15:04.7618953Z 
2024-06-26T07:15:04.8736013Z model.safetensors:  12% 126M/1.08G [00:00<00:01, 532MB/s] 
2024-06-26T07:15:04.8736437Z 
2024-06-26T07:15:05.0167274Z model.safetensors:  17% 189M/1.08G [00:00<00:01, 546MB/s]
2024-06-26T07:15:05.0167879Z 
2024-06-26T07:15:05.1250272Z model.safetensors:  23% 252M/1.08G [00:00<00:01, 498MB/s]
2024-06-26T07:15:05.1251054Z 
2024-06-26T07:15:05.2362958Z model.safetensors:  29% 315M/1.08G [00:00<00:01, 525MB/s]
2024-06-26T07:15:05.2367112Z 
2024-06-26T07:15:05.3438368Z model.safetensors:  35% 377M/1.08G [00:00<00:01, 538MB/s]
2024-06-26T07:15:05.3438969Z 
2024-06-26T07:15:05.4546883Z model.safetensors:  41% 440M/1.08G [00:00<00:01, 552MB/s]
2024-06-26T07:15:05.4547315Z 
2024-06-26T07:15:05.5712942Z model.safetensors:  46% 503M/1.08G [00:00<00:01, 557MB/s]
2024-06-26T07:15:05.5713363Z 
2024-06-26T07:15:05.6838806Z model.safetensors:  52% 566M/1.08G [00:01<00:00, 551MB/s]
2024-06-26T07:15:05.6839206Z 
2024-06-26T07:15:05.8006787Z model.safetensors:  58% 629M/1.08G [00:01<00:00, 554MB/s]
2024-06-26T07:15:05.8007186Z 
2024-06-26T07:15:05.9312501Z model.safetensors:  64% 692M/1.08G [00:01<00:00, 549MB/s]
2024-06-26T07:15:05.9312955Z 
2024-06-26T07:15:06.0579195Z model.safetensors:  70% 755M/1.08G [00:01<00:00, 527MB/s]
2024-06-26T07:15:06.0579985Z 
2024-06-26T07:15:06.1692073Z model.safetensors:  75% 818M/1.08G [00:01<00:00, 517MB/s]
2024-06-26T07:15:06.1692690Z 
2024-06-26T07:15:06.2897353Z model.safetensors:  80% 870M/1.08G [00:01<00:00, 504MB/s]
2024-06-26T07:15:06.2897838Z 
2024-06-26T07:15:06.4201961Z model.safetensors:  85% 923M/1.08G [00:01<00:00, 483MB/s]
2024-06-26T07:15:06.4202424Z 
2024-06-26T07:15:06.5346975Z model.safetensors:  90% 975M/1.08G [00:01<00:00, 457MB/s]
2024-06-26T07:15:06.5347519Z 
2024-06-26T07:15:06.6778859Z model.safetensors:  95% 1.03G/1.08G [00:02<00:00, 457MB/s]
2024-06-26T07:15:06.6779274Z 
2024-06-26T07:15:06.6898632Z model.safetensors: 100% 1.08G/1.08G [00:02<00:00, 426MB/s]
2024-06-26T07:15:06.6899260Z model.safetensors: 100% 1.08G/1.08G [00:02<00:00, 501MB/s]
2024-06-26T07:15:07.7877806Z 
2024-06-26T07:15:07.7878152Z loading model: 0it [00:06, ?it/s]
2024-06-26T07:15:07.8060314Z cuda eval  cait_m36_384                       
2024-06-26T07:15:44.7136561Z pass
2024-06-26T07:15:44.7193505Z TIMING: backend_compile:12.75612 entire_frame_compile:18.75871
2024-06-26T07:15:44.7194617Z STATS: call_* op count: 2740 | FakeTensor.__torch_dispatch__:4164 | FakeTensorMode.__torch_dispatch__:81032 | ProxyTorchDispatchMode.__torch_dispatch__:27990
2024-06-26T07:15:44.7195739Z Dynamo produced 1 graphs covering 2740 ops with 0 graph breaks (0 unique)
2024-06-26T07:15:48.6696218Z 
2024-06-26T07:15:48.8613913Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:15:48.8614255Z 
2024-06-26T07:15:49.0377412Z pytorch_model.bin:   0% 0.00/44.1M [00:00<?, ?B/s]
2024-06-26T07:15:49.0377837Z 
2024-06-26T07:15:49.1180944Z pytorch_model.bin:  48% 21.0M/44.1M [00:00<00:00, 119MB/s]
2024-06-26T07:15:49.1181591Z pytorch_model.bin: 100% 44.1M/44.1M [00:00<00:00, 172MB/s]
2024-06-26T07:15:49.5107320Z 
2024-06-26T07:15:49.5107876Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:15:49.5154745Z cuda eval  coat_lite_mini                     
2024-06-26T07:15:56.2692625Z pass
2024-06-26T07:15:56.2708484Z TIMING: backend_compile:3.79766 entire_frame_compile:6.24425
2024-06-26T07:15:56.2710063Z STATS: call_* op count: 952 | FakeTensor.__torch_dispatch__:954 | FakeTensorMode.__torch_dispatch__:21933 | ProxyTorchDispatchMode.__torch_dispatch__:7218
2024-06-26T07:15:56.2711667Z Dynamo produced 1 graphs covering 952 ops with 0 graph breaks (0 unique)
2024-06-26T07:15:59.9652641Z 
2024-06-26T07:16:00.8838359Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:16:00.8838693Z 
2024-06-26T07:16:00.9986611Z model.safetensors:   0% 0.00/346M [00:00<?, ?B/s]
2024-06-26T07:16:00.9987140Z 
2024-06-26T07:16:01.1095323Z model.safetensors:  15% 52.4M/346M [00:00<00:00, 457MB/s]
2024-06-26T07:16:01.1095722Z 
2024-06-26T07:16:01.2482736Z model.safetensors:  30% 105M/346M [00:00<00:00, 466MB/s] 
2024-06-26T07:16:01.2483136Z 
2024-06-26T07:16:01.3674231Z model.safetensors:  45% 157M/346M [00:00<00:00, 421MB/s]
2024-06-26T07:16:01.3674625Z 
2024-06-26T07:16:01.4981900Z model.safetensors:  61% 210M/346M [00:00<00:00, 428MB/s]
2024-06-26T07:16:01.4982298Z 
2024-06-26T07:16:01.6951146Z model.safetensors:  76% 262M/346M [00:00<00:00, 418MB/s]
2024-06-26T07:16:01.6951553Z 
2024-06-26T07:16:01.7896438Z model.safetensors:  91% 315M/346M [00:00<00:00, 350MB/s]
2024-06-26T07:16:01.7897129Z model.safetensors: 100% 346M/346M [00:00<00:00, 382MB/s]
2024-06-26T07:16:02.0725006Z WARNING:common:Model convit_base does not support bfloat16, running with amp instead
2024-06-26T07:16:02.2971253Z 
2024-06-26T07:16:02.2971759Z loading model: 0it [00:02, ?it/s]
2024-06-26T07:16:02.2974331Z WARNING:common:Model convit_base does not support bfloat16, running with amp instead
2024-06-26T07:16:02.2975748Z cuda eval  convit_base                        
2024-06-26T07:16:02.3267620Z WARNING:common:fp64 golden ref were not generated for convit_base. Setting accuracy check to cosine
2024-06-26T07:16:02.3870548Z WARNING:common:Model convit_base does not support bfloat16, running with amp instead
2024-06-26T07:16:39.0467729Z pass
2024-06-26T07:16:39.0496424Z TIMING: backend_compile:27.07125 entire_frame_compile:36.27589
2024-06-26T07:16:39.0497864Z STATS: call_* op count: 1306 | FakeTensor.__torch_dispatch__:726 | FakeTensorMode.__torch_dispatch__:54397 | attempt fast:2314 | slow no contiguity match:127 | fast is_contiguous:2187 | ProxyTorchDispatchMode.__torch_dispatch__:12524
2024-06-26T07:16:39.0499321Z Dynamo produced 1 graphs covering 1306 ops with 0 graph breaks (0 unique)
2024-06-26T07:16:44.6030511Z 
2024-06-26T07:16:44.7985096Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:16:44.7985478Z 
2024-06-26T07:16:44.9036426Z model.safetensors:   0% 0.00/84.9M [00:00<?, ?B/s]
2024-06-26T07:16:44.9036830Z 
2024-06-26T07:16:44.9690170Z model.safetensors:  62% 52.4M/84.9M [00:00<00:00, 500MB/s]
2024-06-26T07:16:44.9690904Z model.safetensors: 100% 84.9M/84.9M [00:00<00:00, 499MB/s]
2024-06-26T07:16:45.4086015Z 
2024-06-26T07:16:45.4086523Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:16:45.4195047Z cuda eval  convmixer_768_32                   
2024-06-26T07:16:53.3480140Z pass
2024-06-26T07:16:53.3480789Z TIMING: backend_compile:3.67439 entire_frame_compile:5.87264
2024-06-26T07:16:53.3481944Z STATS: call_* op count: 462 | FakeTensor.__torch_dispatch__:1578 | FakeTensorMode.__torch_dispatch__:28635 | ProxyTorchDispatchMode.__torch_dispatch__:6100
2024-06-26T07:16:53.3483263Z Dynamo produced 1 graphs covering 462 ops with 0 graph breaks (0 unique)
2024-06-26T07:16:56.9443456Z 
2024-06-26T07:16:57.8958096Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:16:57.8958447Z 
2024-06-26T07:16:58.0187089Z model.safetensors:   0% 0.00/354M [00:00<?, ?B/s]
2024-06-26T07:16:58.0187467Z 
2024-06-26T07:16:58.1306894Z model.safetensors:   6% 21.0M/354M [00:00<00:01, 171MB/s]
2024-06-26T07:16:58.1307315Z 
2024-06-26T07:16:58.2497099Z model.safetensors:  15% 52.4M/354M [00:00<00:01, 233MB/s]
2024-06-26T07:16:58.2497513Z 
2024-06-26T07:16:58.3580809Z model.safetensors:  33% 115M/354M [00:00<00:00, 370MB/s] 
2024-06-26T07:16:58.3581249Z 
2024-06-26T07:16:58.4723858Z model.safetensors:  50% 178M/354M [00:00<00:00, 449MB/s]
2024-06-26T07:16:58.4724262Z 
2024-06-26T07:16:58.5854981Z model.safetensors:  68% 241M/354M [00:00<00:00, 486MB/s]
2024-06-26T07:16:58.5855370Z 
2024-06-26T07:16:58.6734299Z model.safetensors:  86% 304M/354M [00:00<00:00, 509MB/s]
2024-06-26T07:16:58.6735490Z model.safetensors: 100% 354M/354M [00:00<00:00, 456MB/s]
2024-06-26T07:16:59.1836265Z 
2024-06-26T07:16:59.1836606Z loading model: 0it [00:02, ?it/s]
2024-06-26T07:16:59.1928502Z cuda eval  convnext_base                      
2024-06-26T07:17:10.0878986Z pass
2024-06-26T07:17:10.0891135Z TIMING: backend_compile:4.21401 entire_frame_compile:8.36949
2024-06-26T07:17:10.0892431Z STATS: call_* op count: 1134 | FakeTensor.__torch_dispatch__:1386 | FakeTensorMode.__torch_dispatch__:24655 | ProxyTorchDispatchMode.__torch_dispatch__:8432
2024-06-26T07:17:10.0893558Z Dynamo produced 1 graphs covering 1134 ops with 0 graph breaks (0 unique)
2024-06-26T07:17:13.7377186Z 
2024-06-26T07:17:13.9044361Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:17:13.9044915Z 
2024-06-26T07:17:14.0089172Z model.safetensors:   0% 0.00/34.2M [00:00<?, ?B/s]
2024-06-26T07:17:14.0089676Z 
2024-06-26T07:17:14.0096831Z model.safetensors: 100% 34.2M/34.2M [00:00<00:00, 328MB/s]
2024-06-26T07:17:14.0097592Z model.safetensors: 100% 34.2M/34.2M [00:00<00:00, 325MB/s]
2024-06-26T07:17:14.4412300Z 
2024-06-26T07:17:14.4412649Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:17:14.4489630Z cuda eval  crossvit_9_240                     
2024-06-26T07:17:23.1623405Z pass
2024-06-26T07:17:23.1638088Z TIMING: backend_compile:5.25644 entire_frame_compile:8.11006
2024-06-26T07:17:23.1639260Z STATS: call_* op count: 1154 | FakeTensor.__torch_dispatch__:1853 | FakeTensorMode.__torch_dispatch__:33060 | ProxyTorchDispatchMode.__torch_dispatch__:11306
2024-06-26T07:17:23.1640382Z Dynamo produced 1 graphs covering 1154 ops with 0 graph breaks (0 unique)
2024-06-26T07:17:26.8511077Z 
2024-06-26T07:17:27.2125324Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:17:27.2126154Z 
2024-06-26T07:17:27.3246640Z model.safetensors:   0% 0.00/111M [00:00<?, ?B/s]
2024-06-26T07:17:27.3247048Z 
2024-06-26T07:17:27.4173414Z model.safetensors:  57% 62.9M/111M [00:00<00:00, 561MB/s]
2024-06-26T07:17:27.4174639Z model.safetensors: 100% 111M/111M [00:00<00:00, 541MB/s] 
2024-06-26T07:17:27.8987137Z 
2024-06-26T07:17:27.8987640Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:17:27.9076859Z cuda eval  cspdarknet53                       
2024-06-26T07:17:38.0878742Z pass
2024-06-26T07:17:38.0911574Z TIMING: backend_compile:4.23004 entire_frame_compile:8.81413
2024-06-26T07:17:38.0912650Z STATS: call_* op count: 744 | FakeTensor.__torch_dispatch__:458 | FakeTensorMode.__torch_dispatch__:25870 | ProxyTorchDispatchMode.__torch_dispatch__:5321
2024-06-26T07:17:38.0913765Z Dynamo produced 1 graphs covering 744 ops with 0 graph breaks (0 unique)
2024-06-26T07:17:41.7599210Z 
2024-06-26T07:17:42.6748864Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:17:42.6749211Z 
2024-06-26T07:17:42.7780531Z model.safetensors:   0% 0.00/349M [00:00<?, ?B/s]
2024-06-26T07:17:42.7780944Z 
2024-06-26T07:17:42.8815067Z model.safetensors:  15% 52.4M/349M [00:00<00:00, 509MB/s]
2024-06-26T07:17:42.8815512Z 
2024-06-26T07:17:42.9852719Z model.safetensors:  30% 105M/349M [00:00<00:00, 508MB/s] 
2024-06-26T07:17:42.9853141Z 
2024-06-26T07:17:43.1017996Z model.safetensors:  45% 157M/349M [00:00<00:00, 507MB/s]
2024-06-26T07:17:43.1018432Z 
2024-06-26T07:17:43.2291682Z model.safetensors:  60% 210M/349M [00:00<00:00, 483MB/s]
2024-06-26T07:17:43.2292131Z 
2024-06-26T07:17:43.3533349Z model.safetensors:  75% 262M/349M [00:00<00:00, 454MB/s]
2024-06-26T07:17:43.3533774Z 
2024-06-26T07:17:43.4065457Z model.safetensors:  93% 325M/349M [00:00<00:00, 473MB/s]
2024-06-26T07:17:43.4066100Z model.safetensors: 100% 349M/349M [00:00<00:00, 478MB/s]
2024-06-26T07:17:43.8653095Z 
2024-06-26T07:17:43.8653908Z loading model: 0it [00:02, ?it/s]
2024-06-26T07:17:43.8698553Z cuda eval  deit_base_distilled_patch16_224    
2024-06-26T07:17:49.7105054Z pass
2024-06-26T07:17:49.7114905Z TIMING: backend_compile:2.57645 entire_frame_compile:4.22427
2024-06-26T07:17:49.7116080Z STATS: call_* op count: 708 | FakeTensor.__torch_dispatch__:1138 | FakeTensorMode.__torch_dispatch__:15435 | ProxyTorchDispatchMode.__torch_dispatch__:5069
2024-06-26T07:17:49.7117226Z Dynamo produced 1 graphs covering 708 ops with 0 graph breaks (0 unique)
2024-06-26T07:17:53.3139668Z 
2024-06-26T07:17:53.7065993Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:17:53.7066635Z 
2024-06-26T07:17:53.8076536Z model.safetensors:   0% 0.00/133M [00:00<?, ?B/s]
2024-06-26T07:17:53.8076954Z 
2024-06-26T07:17:53.9143676Z model.safetensors:  31% 41.9M/133M [00:00<00:00, 416MB/s]
2024-06-26T07:17:53.9144298Z 
2024-06-26T07:17:53.9912736Z model.safetensors:  71% 94.4M/133M [00:00<00:00, 461MB/s]
2024-06-26T07:17:53.9913484Z model.safetensors: 100% 133M/133M [00:00<00:00, 469MB/s] 
2024-06-26T07:17:54.5300476Z 
2024-06-26T07:17:54.5300998Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:17:54.5424223Z cuda eval  dla102                             
2024-06-26T07:18:04.9841009Z pass
2024-06-26T07:18:04.9847844Z TIMING: backend_compile:5.67517 entire_frame_compile:9.00912
2024-06-26T07:18:04.9849117Z STATS: call_* op count: 832 | FakeTensor.__torch_dispatch__:1937 | FakeTensorMode.__torch_dispatch__:42435 | ProxyTorchDispatchMode.__torch_dispatch__:8591
2024-06-26T07:18:04.9850232Z Dynamo produced 1 graphs covering 832 ops with 0 graph breaks (0 unique)
2024-06-26T07:18:08.7051922Z 
2024-06-26T07:18:09.4718609Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:18:09.4719088Z 
2024-06-26T07:18:09.6799782Z model.safetensors:   0% 0.00/286M [00:00<?, ?B/s]
2024-06-26T07:18:09.6800329Z 
2024-06-26T07:18:09.8740977Z model.safetensors:   4% 10.5M/286M [00:00<00:05, 50.4MB/s]
2024-06-26T07:18:09.8741616Z 
2024-06-26T07:18:10.2930600Z model.safetensors:   7% 21.0M/286M [00:00<00:05, 52.5MB/s]
2024-06-26T07:18:10.2931032Z 
2024-06-26T07:18:10.8012708Z model.safetensors:  11% 31.5M/286M [00:00<00:07, 35.0MB/s]
2024-06-26T07:18:11.1080406Z 
2024-06-26T07:18:11.1080829Z model.safetensors:  15% 41.9M/286M [00:01<00:08, 27.4MB/s]
2024-06-26T07:18:11.1081247Z 
2024-06-26T07:18:11.2518031Z model.safetensors:  18% 52.4M/286M [00:01<00:07, 29.5MB/s]
2024-06-26T07:18:11.2518457Z 
2024-06-26T07:18:11.5130865Z model.safetensors:  26% 73.4M/286M [00:01<00:04, 49.6MB/s]
2024-06-26T07:18:11.5131296Z 
2024-06-26T07:18:11.6235395Z model.safetensors:  29% 83.9M/286M [00:02<00:04, 46.7MB/s]
2024-06-26T07:18:11.6235813Z 
2024-06-26T07:18:11.7603357Z model.safetensors:  37% 105M/286M [00:02<00:02, 69.2MB/s] 
2024-06-26T07:18:11.7603970Z 
2024-06-26T07:18:12.0605803Z model.safetensors:  40% 115M/286M [00:02<00:02, 70.8MB/s]
2024-06-26T07:18:12.0606378Z 
2024-06-26T07:18:12.5315571Z model.safetensors:  44% 126M/286M [00:02<00:02, 56.3MB/s]
2024-06-26T07:18:12.5316169Z 
2024-06-26T07:18:12.8585208Z model.safetensors:  48% 136M/286M [00:03<00:03, 40.1MB/s]
2024-06-26T07:18:12.8585792Z 
2024-06-26T07:18:13.2580351Z model.safetensors:  51% 147M/286M [00:03<00:03, 37.5MB/s]
2024-06-26T07:18:13.2580838Z 
2024-06-26T07:18:13.4431078Z model.safetensors:  55% 157M/286M [00:03<00:03, 33.5MB/s]
2024-06-26T07:18:13.4431539Z 
2024-06-26T07:18:13.5762958Z model.safetensors:  59% 168M/286M [00:03<00:03, 37.9MB/s]
2024-06-26T07:18:13.5763714Z 
2024-06-26T07:18:13.9474665Z model.safetensors:  62% 178M/286M [00:04<00:02, 44.7MB/s]
2024-06-26T07:18:13.9475183Z 
2024-06-26T07:18:14.6012825Z model.safetensors:  66% 189M/286M [00:04<00:02, 38.1MB/s]
2024-06-26T07:18:14.6013347Z 
2024-06-26T07:18:15.2368696Z model.safetensors:  70% 199M/286M [00:05<00:03, 27.1MB/s]
2024-06-26T07:18:15.2369161Z 
2024-06-26T07:18:15.7697571Z model.safetensors:  73% 210M/286M [00:05<00:03, 22.8MB/s]
2024-06-26T07:18:15.7698005Z 
2024-06-26T07:18:16.4252371Z model.safetensors:  77% 220M/286M [00:06<00:03, 21.7MB/s]
2024-06-26T07:18:16.4253365Z 
2024-06-26T07:18:16.8329408Z model.safetensors:  81% 231M/286M [00:06<00:02, 19.6MB/s]
2024-06-26T07:18:16.8329834Z 
2024-06-26T07:18:16.9437157Z model.safetensors:  84% 241M/286M [00:07<00:02, 21.1MB/s]
2024-06-26T07:18:16.9437560Z 
2024-06-26T07:18:17.2677373Z model.safetensors:  95% 273M/286M [00:07<00:00, 44.0MB/s]
2024-06-26T07:18:17.2677817Z 
2024-06-26T07:18:17.3871290Z model.safetensors:  99% 283M/286M [00:07<00:00, 40.9MB/s]
2024-06-26T07:18:17.3871923Z model.safetensors: 100% 286M/286M [00:07<00:00, 36.1MB/s]
2024-06-26T07:18:17.9068514Z 
2024-06-26T07:18:17.9069165Z loading model: 0it [00:09, ?it/s]
2024-06-26T07:18:17.9132772Z cuda eval  dm_nfnet_f0                        
2024-06-26T07:18:29.9984019Z pass
2024-06-26T07:18:30.0006845Z TIMING: backend_compile:4.83824 entire_frame_compile:9.92062
2024-06-26T07:18:30.0007939Z STATS: call_* op count: 1232 | FakeTensorMode.__torch_dispatch__:25950 | FakeTensor.__torch_dispatch__:313 | ProxyTorchDispatchMode.__torch_dispatch__:7759
2024-06-26T07:18:30.0009058Z Dynamo produced 1 graphs covering 1232 ops with 0 graph breaks (0 unique)
2024-06-26T07:18:33.7099809Z 
2024-06-26T07:18:34.1979490Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:18:34.1979825Z 
2024-06-26T07:18:34.3140995Z model.safetensors:   0% 0.00/349M [00:00<?, ?B/s]
2024-06-26T07:18:34.3141367Z 
2024-06-26T07:18:34.4295822Z model.safetensors:  15% 52.4M/349M [00:00<00:00, 451MB/s]
2024-06-26T07:18:34.4296515Z 
2024-06-26T07:18:34.5406966Z model.safetensors:  33% 115M/349M [00:00<00:00, 506MB/s] 
2024-06-26T07:18:34.5407392Z 
2024-06-26T07:18:34.6529201Z model.safetensors:  48% 168M/349M [00:00<00:00, 491MB/s]
2024-06-26T07:18:34.6529630Z 
2024-06-26T07:18:34.7658224Z model.safetensors:  63% 220M/349M [00:00<00:00, 482MB/s]
2024-06-26T07:18:34.7658622Z 
2024-06-26T07:18:34.8771140Z model.safetensors:  81% 283M/349M [00:00<00:00, 509MB/s]
2024-06-26T07:18:34.8771642Z 
2024-06-26T07:18:34.9026053Z model.safetensors:  96% 336M/349M [00:00<00:00, 496MB/s]
2024-06-26T07:18:34.9026685Z model.safetensors: 100% 349M/349M [00:00<00:00, 495MB/s]
2024-06-26T07:18:35.5557142Z 
2024-06-26T07:18:35.5557762Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:18:35.5692761Z cuda eval  dpn107                             
2024-06-26T07:18:53.9892399Z pass
2024-06-26T07:18:53.9913694Z TIMING: backend_compile:8.2882 entire_frame_compile:15.86824
2024-06-26T07:18:53.9914867Z STATS: call_* op count: 1262 | FakeTensor.__torch_dispatch__:721 | FakeTensorMode.__torch_dispatch__:50251 | ProxyTorchDispatchMode.__torch_dispatch__:11700
2024-06-26T07:18:53.9916013Z Dynamo produced 1 graphs covering 1262 ops with 0 graph breaks (0 unique)
2024-06-26T07:18:57.8121397Z 
2024-06-26T07:18:58.0258294Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:18:58.0258645Z 
2024-06-26T07:18:58.1120878Z model.safetensors:   0% 0.00/42.5M [00:00<?, ?B/s]
2024-06-26T07:18:58.1121560Z model.safetensors: 100% 42.5M/42.5M [00:00<00:00, 493MB/s]
2024-06-26T07:18:58.5606750Z 
2024-06-26T07:18:58.5607099Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:18:58.5650333Z cuda eval  eca_botnext26ts_256                
2024-06-26T07:19:04.8371435Z pass
2024-06-26T07:19:04.8389484Z TIMING: backend_compile:3.02064 entire_frame_compile:5.75106
2024-06-26T07:19:04.8390720Z STATS: call_* op count: 672 | FakeTensor.__torch_dispatch__:283 | FakeTensorMode.__torch_dispatch__:17665 | ProxyTorchDispatchMode.__torch_dispatch__:4476
2024-06-26T07:19:04.8392212Z Dynamo produced 1 graphs covering 672 ops with 0 graph breaks (0 unique)
2024-06-26T07:19:08.4532418Z 
2024-06-26T07:19:08.6323420Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:19:08.6323816Z 
2024-06-26T07:19:08.7087156Z model.safetensors:   0% 0.00/43.2M [00:00<?, ?B/s]
2024-06-26T07:19:08.7087752Z model.safetensors: 100% 43.2M/43.2M [00:00<00:00, 565MB/s]
2024-06-26T07:19:09.1787138Z 
2024-06-26T07:19:09.1787643Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:19:09.1832461Z cuda eval  eca_halonext26ts                   
2024-06-26T07:19:15.7437705Z pass
2024-06-26T07:19:15.7456357Z TIMING: backend_compile:3.19691 entire_frame_compile:6.02809
2024-06-26T07:19:15.7457432Z STATS: call_* op count: 720 | FakeTensor.__torch_dispatch__:301 | FakeTensorMode.__torch_dispatch__:18229 | ProxyTorchDispatchMode.__torch_dispatch__:4697
2024-06-26T07:19:15.7458544Z Dynamo produced 1 graphs covering 720 ops with 0 graph breaks (0 unique)
2024-06-26T07:19:19.3851451Z 
2024-06-26T07:19:19.5171154Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:19:19.5171489Z 
2024-06-26T07:19:19.6301265Z model.safetensors:   0% 0.00/26.2M [00:00<?, ?B/s]
2024-06-26T07:19:19.6302179Z 
2024-06-26T07:19:19.7402086Z model.safetensors:  40% 10.5M/26.2M [00:00<00:00, 92.8MB/s]
2024-06-26T07:19:19.7402696Z 
2024-06-26T07:19:19.7415810Z model.safetensors: 100% 26.2M/26.2M [00:00<00:00, 122MB/s] 
2024-06-26T07:19:19.7416750Z model.safetensors: 100% 26.2M/26.2M [00:00<00:00, 117MB/s]
2024-06-26T07:19:20.1899331Z 
2024-06-26T07:19:20.1899773Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:19:20.1935872Z cuda eval  ese_vovnet19b_dw                   
2024-06-26T07:19:24.1169104Z pass
2024-06-26T07:19:24.1190549Z TIMING: backend_compile:1.53876 entire_frame_compile:3.47038
2024-06-26T07:19:24.1191795Z STATS: call_* op count: 268 | FakeTensor.__torch_dispatch__:311 | FakeTensorMode.__torch_dispatch__:10136 | ProxyTorchDispatchMode.__torch_dispatch__:2243
2024-06-26T07:19:24.1192909Z Dynamo produced 1 graphs covering 268 ops with 0 graph breaks (0 unique)
2024-06-26T07:19:27.7160898Z 
2024-06-26T07:19:27.8523077Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:19:27.8523444Z 
2024-06-26T07:19:27.8970093Z model.safetensors:   0% 0.00/22.5M [00:00<?, ?B/s]
2024-06-26T07:19:27.8970758Z model.safetensors: 100% 22.5M/22.5M [00:00<00:00, 503MB/s]
2024-06-26T07:19:28.3484823Z 
2024-06-26T07:19:28.3485295Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:19:28.3565907Z cuda eval  fbnetc_100                         
2024-06-26T07:19:36.1110139Z pass
2024-06-26T07:19:36.1117218Z TIMING: backend_compile:3.64375 entire_frame_compile:7.34614
2024-06-26T07:19:36.1118304Z STATS: call_* op count: 628 | FakeTensor.__torch_dispatch__:441 | FakeTensorMode.__torch_dispatch__:23945 | ProxyTorchDispatchMode.__torch_dispatch__:4861
2024-06-26T07:19:36.1119401Z Dynamo produced 1 graphs covering 628 ops with 0 graph breaks (0 unique)
2024-06-26T07:19:39.8062449Z 
2024-06-26T07:19:39.9660723Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:19:39.9661177Z 
2024-06-26T07:19:40.0666080Z model.safetensors:   0% 0.00/34.6M [00:00<?, ?B/s]
2024-06-26T07:19:40.0666822Z model.safetensors: 100% 34.6M/34.6M [00:00<00:00, 345MB/s]
2024-06-26T07:19:40.5689136Z 
2024-06-26T07:19:40.5689695Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:19:40.5816504Z cuda eval  fbnetv3_b                          
2024-06-26T07:19:52.9466825Z pass
2024-06-26T07:19:52.9472810Z TIMING: backend_compile:6.35812 entire_frame_compile:11.80856
2024-06-26T07:19:52.9473914Z STATS: call_* op count: 1038 | FakeTensor.__torch_dispatch__:1061 | FakeTensorMode.__torch_dispatch__:36388 | ProxyTorchDispatchMode.__torch_dispatch__:7868
2024-06-26T07:19:52.9475082Z Dynamo produced 1 graphs covering 1038 ops with 0 graph breaks (0 unique)
2024-06-26T07:19:56.7036635Z 
2024-06-26T07:19:57.0719608Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:19:57.0719937Z 
2024-06-26T07:19:57.1768657Z model.safetensors:   0% 0.00/125M [00:00<?, ?B/s]
2024-06-26T07:19:57.1769092Z 
2024-06-26T07:19:57.2899473Z model.safetensors:  34% 41.9M/125M [00:00<00:00, 401MB/s]
2024-06-26T07:19:57.2899912Z 
2024-06-26T07:19:57.3634823Z model.safetensors:  76% 94.4M/125M [00:00<00:00, 439MB/s]
2024-06-26T07:19:57.3636102Z model.safetensors: 100% 125M/125M [00:00<00:00, 429MB/s] 
2024-06-26T07:19:57.8363917Z 
2024-06-26T07:19:57.8364439Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:19:57.8451375Z cuda eval  gernet_l                           
2024-06-26T07:20:05.9857506Z pass
2024-06-26T07:20:05.9869121Z TIMING: backend_compile:3.24382 entire_frame_compile:7.18209
2024-06-26T07:20:05.9870200Z STATS: call_* op count: 698 | FakeTensor.__torch_dispatch__:373 | FakeTensorMode.__torch_dispatch__:21268 | ProxyTorchDispatchMode.__torch_dispatch__:4385
2024-06-26T07:20:05.9871491Z Dynamo produced 1 graphs covering 698 ops with 0 graph breaks (0 unique)
2024-06-26T07:20:09.6443261Z 
2024-06-26T07:20:09.7335859Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:20:09.7336298Z 
2024-06-26T07:20:09.7856452Z model.safetensors:   0% 0.00/20.9M [00:00<?, ?B/s]
2024-06-26T07:20:09.7857066Z model.safetensors: 100% 20.9M/20.9M [00:00<00:00, 402MB/s]
2024-06-26T07:20:10.2978845Z 
2024-06-26T07:20:10.2979294Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:20:10.3082659Z cuda eval  ghostnet_100                       
2024-06-26T07:20:19.1894160Z pass
2024-06-26T07:20:19.1901444Z TIMING: backend_compile:5.21016 entire_frame_compile:8.41646
2024-06-26T07:20:19.1903050Z STATS: call_* op count: 710 | FakeTensor.__torch_dispatch__:1691 | FakeTensorMode.__torch_dispatch__:36917 | ProxyTorchDispatchMode.__torch_dispatch__:8038
2024-06-26T07:20:19.1904668Z Dynamo produced 1 graphs covering 710 ops with 0 graph breaks (0 unique)
2024-06-26T07:20:22.8671016Z 
2024-06-26T07:20:23.1786418Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:20:23.1786758Z 
2024-06-26T07:20:23.2858017Z model.safetensors:   0% 0.00/95.5M [00:00<?, ?B/s]
2024-06-26T07:20:23.2858404Z 
2024-06-26T07:20:23.3754148Z model.safetensors:  55% 52.4M/95.5M [00:00<00:00, 489MB/s]
2024-06-26T07:20:23.3755363Z model.safetensors: 100% 95.5M/95.5M [00:00<00:00, 486MB/s]
2024-06-26T07:20:23.9106822Z 
2024-06-26T07:20:23.9107152Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:20:23.9221249Z cuda eval  gluon_inception_v3                 
2024-06-26T07:20:36.1336034Z pass
2024-06-26T07:20:36.1360424Z TIMING: backend_compile:5.63145 entire_frame_compile:11.18067
2024-06-26T07:20:36.1361510Z STATS: call_* op count: 816 | FakeTensor.__torch_dispatch__:620 | FakeTensorMode.__torch_dispatch__:35069 | ProxyTorchDispatchMode.__torch_dispatch__:7234
2024-06-26T07:20:36.1362624Z Dynamo produced 1 graphs covering 816 ops with 0 graph breaks (0 unique)
2024-06-26T07:20:39.8578970Z 
2024-06-26T07:20:40.1929743Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:20:40.1930074Z 
2024-06-26T07:20:40.3149711Z model.safetensors:   0% 0.00/98.9M [00:00<?, ?B/s]
2024-06-26T07:20:40.3150128Z 
2024-06-26T07:20:40.5053905Z model.safetensors:  42% 41.9M/98.9M [00:00<00:00, 349MB/s]
2024-06-26T07:20:40.5054345Z 
2024-06-26T07:20:40.5470716Z model.safetensors:  85% 83.9M/98.9M [00:00<00:00, 258MB/s]
2024-06-26T07:20:40.5471386Z model.safetensors: 100% 98.9M/98.9M [00:00<00:00, 280MB/s]
2024-06-26T07:20:40.9979081Z 
2024-06-26T07:20:40.9979749Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:20:41.0061212Z cuda eval  gmixer_24_224                      
2024-06-26T07:20:49.4054651Z pass
2024-06-26T07:20:49.4055276Z TIMING: backend_compile:4.67889 entire_frame_compile:7.56938
2024-06-26T07:20:49.4056416Z STATS: call_* op count: 1360 | FakeTensor.__torch_dispatch__:2068 | FakeTensorMode.__torch_dispatch__:30705 | ProxyTorchDispatchMode.__torch_dispatch__:10012
2024-06-26T07:20:49.4057650Z Dynamo produced 1 graphs covering 1360 ops with 0 graph breaks (0 unique)
2024-06-26T07:20:53.0665655Z 
2024-06-26T07:20:53.3279604Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:20:53.3280029Z 
2024-06-26T07:20:53.4446351Z model.safetensors:   0% 0.00/77.7M [00:00<?, ?B/s]
2024-06-26T07:20:53.4447044Z 
2024-06-26T07:20:53.4942182Z model.safetensors:  67% 52.4M/77.7M [00:00<00:00, 449MB/s]
2024-06-26T07:20:53.4942879Z model.safetensors: 100% 77.7M/77.7M [00:00<00:00, 468MB/s]
2024-06-26T07:20:53.9393592Z 
2024-06-26T07:20:53.9394017Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:20:53.9476385Z cuda eval  gmlp_s16_224                       
2024-06-26T07:21:02.1208062Z pass
2024-06-26T07:21:02.1211548Z TIMING: backend_compile:4.47714 entire_frame_compile:7.37273
2024-06-26T07:21:02.1212924Z STATS: call_* op count: 1036 | FakeTensor.__torch_dispatch__:2212 | FakeTensorMode.__torch_dispatch__:28285 | ProxyTorchDispatchMode.__torch_dispatch__:9244
2024-06-26T07:21:02.1214095Z Dynamo produced 1 graphs covering 1036 ops with 0 graph breaks (0 unique)
2024-06-26T07:21:05.7343131Z 
2024-06-26T07:21:06.1045318Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:21:06.1045921Z 
2024-06-26T07:21:06.6167073Z model.safetensors:   0% 0.00/85.6M [00:00<?, ?B/s]
2024-06-26T07:21:06.6167531Z 
2024-06-26T07:21:06.7705400Z model.safetensors:  12% 10.5M/85.6M [00:00<00:03, 20.5MB/s]
2024-06-26T07:21:06.7706198Z 
2024-06-26T07:21:07.5192313Z model.safetensors:  61% 52.4M/85.6M [00:00<00:00, 96.2MB/s]
2024-06-26T07:21:07.5192977Z 
2024-06-26T07:21:07.6149613Z model.safetensors:  86% 73.4M/85.6M [00:01<00:00, 50.1MB/s]
2024-06-26T07:21:07.6150252Z model.safetensors: 100% 85.6M/85.6M [00:01<00:00, 56.7MB/s]
2024-06-26T07:21:08.6247175Z 
2024-06-26T07:21:08.6247696Z loading model: 0it [00:02, ?it/s]
2024-06-26T07:21:08.6641387Z cuda eval  hrnet_w18                          
2024-06-26T07:21:38.8311457Z pass
2024-06-26T07:21:38.8324400Z TIMING: backend_compile:17.77806 entire_frame_compile:28.03056
2024-06-26T07:21:38.8326050Z STATS: call_* op count: 2780 | FakeTensor.__torch_dispatch__:5940 | FakeTensorMode.__torch_dispatch__:139036 | ProxyTorchDispatchMode.__torch_dispatch__:29949
2024-06-26T07:21:38.8327317Z Dynamo produced 1 graphs covering 2780 ops with 0 graph breaks (0 unique)
2024-06-26T07:21:43.0324244Z 
2024-06-26T07:21:43.3383795Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:21:43.3384133Z 
2024-06-26T07:21:43.4561300Z model.safetensors:   0% 0.00/95.5M [00:00<?, ?B/s]
2024-06-26T07:21:43.4562086Z 
2024-06-26T07:21:43.5717087Z model.safetensors:  44% 41.9M/95.5M [00:00<00:00, 357MB/s]
2024-06-26T07:21:43.5717508Z 
2024-06-26T07:21:43.6008149Z model.safetensors:  88% 83.9M/95.5M [00:00<00:00, 360MB/s]
2024-06-26T07:21:43.6008798Z model.safetensors: 100% 95.5M/95.5M [00:00<00:00, 364MB/s]
2024-06-26T07:21:44.1481687Z 
2024-06-26T07:21:44.1482237Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:21:44.1600890Z cuda eval  inception_v3                       
2024-06-26T07:21:56.4267403Z pass
2024-06-26T07:21:56.4291479Z TIMING: backend_compile:5.67462 entire_frame_compile:11.23147
2024-06-26T07:21:56.4292839Z STATS: call_* op count: 816 | FakeTensor.__torch_dispatch__:620 | FakeTensorMode.__torch_dispatch__:35069 | ProxyTorchDispatchMode.__torch_dispatch__:7234
2024-06-26T07:21:56.4293983Z Dynamo produced 1 graphs covering 816 ops with 0 graph breaks (0 unique)
2024-06-26T07:22:00.1579919Z 
2024-06-26T07:22:00.8797142Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:22:00.8797490Z 
2024-06-26T07:22:00.9886123Z model.safetensors:   0% 0.00/271M [00:00<?, ?B/s]
2024-06-26T07:22:00.9886493Z 
2024-06-26T07:22:01.1028403Z model.safetensors:  19% 52.4M/271M [00:00<00:00, 482MB/s]
2024-06-26T07:22:01.1028817Z 
2024-06-26T07:22:01.2070673Z model.safetensors:  39% 105M/271M [00:00<00:00, 468MB/s] 
2024-06-26T07:22:01.2071087Z 
2024-06-26T07:22:01.3370320Z model.safetensors:  58% 157M/271M [00:00<00:00, 483MB/s]
2024-06-26T07:22:01.3370892Z 
2024-06-26T07:22:01.4463722Z model.safetensors:  77% 210M/271M [00:00<00:00, 448MB/s]
2024-06-26T07:22:01.4464310Z 
2024-06-26T07:22:01.4627024Z model.safetensors:  97% 262M/271M [00:00<00:00, 459MB/s]
2024-06-26T07:22:01.4627946Z model.safetensors: 100% 271M/271M [00:00<00:00, 465MB/s]
2024-06-26T07:22:01.9678345Z 
2024-06-26T07:22:01.9678831Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:22:01.9760549Z cuda eval  jx_nest_base                       
2024-06-26T07:22:15.4467672Z pass
2024-06-26T07:22:15.4499982Z TIMING: backend_compile:6.685 entire_frame_compile:11.3993
2024-06-26T07:22:15.4501093Z STATS: call_* op count: 1158 | FakeTensor.__torch_dispatch__:1225 | FakeTensorMode.__torch_dispatch__:38875 | ProxyTorchDispatchMode.__torch_dispatch__:13525
2024-06-26T07:22:15.4502214Z Dynamo produced 1 graphs covering 1158 ops with 0 graph breaks (0 unique)
2024-06-26T07:22:19.1725117Z 
2024-06-26T07:22:19.2355517Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:22:19.2355842Z 
2024-06-26T07:22:19.2518150Z model.safetensors:   0% 0.00/7.56M [00:00<?, ?B/s]
2024-06-26T07:22:19.2519323Z model.safetensors: 100% 7.56M/7.56M [00:00<00:00, 472MB/s]
2024-06-26T07:22:19.6293787Z 
2024-06-26T07:22:19.6294227Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:22:19.6331764Z cuda eval  lcnet_050                          
2024-06-26T07:22:23.4649920Z pass
2024-06-26T07:22:23.4651215Z TIMING: backend_compile:1.81509 entire_frame_compile:3.5822
2024-06-26T07:22:23.4653175Z STATS: call_* op count: 274 | FakeTensor.__torch_dispatch__:270 | FakeTensorMode.__torch_dispatch__:10710 | ProxyTorchDispatchMode.__torch_dispatch__:2195
2024-06-26T07:22:23.4654712Z Dynamo produced 1 graphs covering 274 ops with 0 graph breaks (0 unique)
2024-06-26T07:22:27.0473454Z 
2024-06-26T07:22:27.1706819Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:22:27.1707165Z 
2024-06-26T07:22:27.2454089Z pytorch_model.bin:   0% 0.00/37.2M [00:00<?, ?B/s]
2024-06-26T07:22:27.2454684Z pytorch_model.bin: 100% 37.2M/37.2M [00:00<00:00, 497MB/s]
2024-06-26T07:22:27.7276282Z 
2024-06-26T07:22:27.7276949Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:22:27.7361021Z cuda eval  levit_128                          
2024-06-26T07:22:38.3056317Z pass
2024-06-26T07:22:38.3058578Z TIMING: backend_compile:6.35757 entire_frame_compile:10.09677
2024-06-26T07:22:38.3059896Z STATS: call_* op count: 1144 | FakeTensor.__torch_dispatch__:1192 | FakeTensorMode.__torch_dispatch__:41064 | ProxyTorchDispatchMode.__torch_dispatch__:11576
2024-06-26T07:22:38.3061019Z Dynamo produced 1 graphs covering 1144 ops with 0 graph breaks (0 unique)
2024-06-26T07:22:42.0432401Z 
2024-06-26T07:22:42.6485422Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:22:42.6486120Z 
2024-06-26T07:22:42.7877093Z model.safetensors:   0% 0.00/240M [00:00<?, ?B/s]
2024-06-26T07:22:42.7877564Z 
2024-06-26T07:22:42.9060175Z model.safetensors:   9% 21.0M/240M [00:00<00:01, 150MB/s]
2024-06-26T07:22:42.9060695Z 
2024-06-26T07:22:43.0129834Z model.safetensors:  26% 62.9M/240M [00:00<00:00, 262MB/s]
2024-06-26T07:22:43.0130350Z 
2024-06-26T07:22:43.1244462Z model.safetensors:  48% 115M/240M [00:00<00:00, 357MB/s] 
2024-06-26T07:22:43.1245016Z 
2024-06-26T07:22:43.2299611Z model.safetensors:  70% 168M/240M [00:00<00:00, 400MB/s]
2024-06-26T07:22:43.2300015Z 
2024-06-26T07:22:43.2474138Z model.safetensors:  96% 231M/240M [00:00<00:00, 467MB/s]
2024-06-26T07:22:43.2475362Z model.safetensors: 100% 240M/240M [00:00<00:00, 400MB/s]
2024-06-26T07:22:43.6762488Z 
2024-06-26T07:22:43.6763221Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:22:43.6804069Z cuda eval  mixer_b16_224                      
2024-06-26T07:22:48.4323281Z pass
2024-06-26T07:22:48.4329313Z TIMING: backend_compile:2.0474 entire_frame_compile:3.49408
2024-06-26T07:22:48.4330469Z STATS: call_* op count: 496 | FakeTensor.__torch_dispatch__:1061 | FakeTensorMode.__torch_dispatch__:13305 | ProxyTorchDispatchMode.__torch_dispatch__:4492
2024-06-26T07:22:48.4331580Z Dynamo produced 1 graphs covering 496 ops with 0 graph breaks (0 unique)
2024-06-26T07:22:49.1937732Z accuracy            pass_rate=100.00%
2024-06-26T07:22:49.1940163Z calls_captured      gmean=861.14x mean=979.533x
2024-06-26T07:22:49.1943330Z unique_graphs       gmean=1.00x mean=1.000x
2024-06-26T07:22:49.1946883Z graph_breaks        gmean=0.00x mean=0.000x
2024-06-26T07:22:49.1950065Z unique_graph_breaks gmean=0.00x mean=0.000x
2024-06-26T07:22:49.1953985Z autograd_captures   gmean=0.00x mean=0.000x
2024-06-26T07:22:49.1956699Z autograd_compiles   gmean=0.00x mean=0.000x
2024-06-26T07:22:49.1960041Z cudagraph_skips     gmean=0.00x mean=0.000x
2024-06-26T07:22:49.7835073Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy//dynamic_aot_eager_timm_inference.csv
2024-06-26T07:22:50.0218833Z adv_inception_v3                    PASS
2024-06-26T07:22:50.0222407Z beit_base_patch16_224               PASS
2024-06-26T07:22:50.0227303Z botnet26t_256                       PASS
2024-06-26T07:22:50.0232311Z cait_m36_384                        PASS
2024-06-26T07:22:50.0237945Z coat_lite_mini                      PASS
2024-06-26T07:22:50.0242892Z convit_base                         PASS
2024-06-26T07:22:50.0248056Z convmixer_768_32                    PASS
2024-06-26T07:22:50.0253302Z convnext_base                       PASS
2024-06-26T07:22:50.0258242Z crossvit_9_240                      PASS
2024-06-26T07:22:50.0263166Z cspdarknet53                        PASS
2024-06-26T07:22:50.0267999Z deit_base_distilled_patch16_224     PASS
2024-06-26T07:22:50.0272936Z dla102                              PASS
2024-06-26T07:22:50.0277747Z dm_nfnet_f0                         PASS
2024-06-26T07:22:50.0282624Z dpn107                              PASS
2024-06-26T07:22:50.0287690Z eca_botnext26ts_256                 PASS
2024-06-26T07:22:50.0293009Z eca_halonext26ts                    PASS
2024-06-26T07:22:50.0297709Z ese_vovnet19b_dw                    PASS
2024-06-26T07:22:50.0302606Z fbnetc_100                          PASS
2024-06-26T07:22:50.0307403Z fbnetv3_b                           PASS
2024-06-26T07:22:50.0312318Z gernet_l                            PASS
2024-06-26T07:22:50.0317226Z ghostnet_100                        PASS
2024-06-26T07:22:50.0322103Z gluon_inception_v3                  PASS
2024-06-26T07:22:50.0327005Z gmixer_24_224                       PASS
2024-06-26T07:22:50.0332214Z gmlp_s16_224                        PASS
2024-06-26T07:22:50.0337224Z hrnet_w18                           PASS
2024-06-26T07:22:50.0342020Z inception_v3                        PASS
2024-06-26T07:22:50.0346835Z jx_nest_base                        PASS
2024-06-26T07:22:50.0351746Z lcnet_050                           PASS
2024-06-26T07:22:50.0356637Z levit_128                           PASS
2024-06-26T07:22:50.0361527Z mixer_b16_224                       PASS
2024-06-26T07:22:50.0744417Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/inference_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy//dynamic_aot_eager_timm_inference.csv
2024-06-26T07:22:50.3354342Z adv_inception_v3                    PASS
2024-06-26T07:22:50.3361131Z beit_base_patch16_224               PASS
2024-06-26T07:22:50.3369211Z botnet26t_256                       PASS
2024-06-26T07:22:50.3376552Z cait_m36_384                        PASS
2024-06-26T07:22:50.3383601Z coat_lite_mini                      PASS
2024-06-26T07:22:50.3389069Z convit_base                         PASS
2024-06-26T07:22:50.3394570Z convmixer_768_32                    PASS
2024-06-26T07:22:50.3400615Z convnext_base                       PASS
2024-06-26T07:22:50.3406321Z crossvit_9_240                      PASS
2024-06-26T07:22:50.3412580Z cspdarknet53                        PASS
2024-06-26T07:22:50.3417381Z deit_base_distilled_patch16_224     PASS
2024-06-26T07:22:50.3421748Z dla102                              PASS
2024-06-26T07:22:50.3426301Z dm_nfnet_f0                         PASS
2024-06-26T07:22:50.3430689Z dpn107                              PASS
2024-06-26T07:22:50.3435194Z eca_botnext26ts_256                 PASS
2024-06-26T07:22:50.3439602Z eca_halonext26ts                    PASS
2024-06-26T07:22:50.3444069Z ese_vovnet19b_dw                    PASS
2024-06-26T07:22:50.3448790Z fbnetc_100                          PASS
2024-06-26T07:22:50.3454068Z fbnetv3_b                           PASS
2024-06-26T07:22:50.3458061Z gernet_l                            PASS
2024-06-26T07:22:50.3462528Z ghostnet_100                        PASS
2024-06-26T07:22:50.3466955Z gluon_inception_v3                  PASS
2024-06-26T07:22:50.3471472Z gmixer_24_224                       PASS
2024-06-26T07:22:50.3475919Z gmlp_s16_224                        PASS
2024-06-26T07:22:50.3480416Z hrnet_w18                           PASS
2024-06-26T07:22:50.3484992Z inception_v3                        PASS
2024-06-26T07:22:50.3489601Z jx_nest_base                        PASS
2024-06-26T07:22:50.3494278Z lcnet_050                           PASS
2024-06-26T07:22:50.3498766Z levit_128                           PASS
2024-06-26T07:22:50.3503150Z mixer_b16_224                       PASS
2024-06-26T07:22:50.3891916Z + test_single_dynamo_benchmark training timm_models 0 --training --amp
2024-06-26T07:22:50.3892885Z ++ pwd
2024-06-26T07:22:50.3895428Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports
2024-06-26T07:22:50.3896116Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports
2024-06-26T07:22:50.3905123Z + local name=training
2024-06-26T07:22:50.3905926Z + shift
2024-06-26T07:22:50.3906263Z + local suite=timm_models
2024-06-26T07:22:50.3906605Z + shift
2024-06-26T07:22:50.3906922Z + local shard_id=0
2024-06-26T07:22:50.3907277Z + shift
2024-06-26T07:22:50.3909238Z + partition_flags=()
2024-06-26T07:22:50.3911784Z + local partition_flags
2024-06-26T07:22:50.3912394Z + [[ -n 2 ]]
2024-06-26T07:22:50.3912737Z + [[ -n 0 ]]
2024-06-26T07:22:50.3913496Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id")
2024-06-26T07:22:50.3914272Z + [[ dynamic_aot_eager_timm == *perf_compare* ]]
2024-06-26T07:22:50.3914761Z + [[ dynamic_aot_eager_timm == *perf* ]]
2024-06-26T07:22:50.3915224Z + [[ dynamic_aot_eager_timm == *aot_inductor* ]]
2024-06-26T07:22:50.3917030Z + python benchmarks/dynamo/timm_models.py --ci --accuracy --timing --explain --backend aot_eager --dynamic-shapes --dynamic-batch-only --device cuda --training --amp --total-partitions 2 --partition-id 0 --output /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv
2024-06-26T07:22:56.1428267Z 
2024-06-26T07:22:57.2458189Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:22:57.2458652Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:22:57.2459102Z cuda train adv_inception_v3                   
2024-06-26T07:23:36.9713950Z pass
2024-06-26T07:23:36.9942251Z TIMING: entire_frame_compile:37.17488 backend_compile:24.91577
2024-06-26T07:23:36.9943870Z STATS: call_* op count: 506 | FakeTensor.__torch_dispatch__:665 | FakeTensorMode.__torch_dispatch__:54976 | attempt fast:2205 | fast is_contiguous:2205 | ProxyTorchDispatchMode.__torch_dispatch__:11234
2024-06-26T07:23:36.9945315Z Dynamo produced 2 graphs covering 506 ops with 6 graph breaks (5 unique)
2024-06-26T07:23:42.4088432Z 
2024-06-26T07:23:43.9283266Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:23:43.9283742Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:23:43.9284215Z cuda train beit_base_patch16_224              
2024-06-26T07:24:11.3512501Z W0626 07:24:11.350000 140460941607552 torch/_logging/_internal.py:1040] [6/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
2024-06-26T07:24:30.5578113Z pass
2024-06-26T07:24:30.6429421Z TIMING: entire_frame_compile:41.54019 backend_compile:26.96251
2024-06-26T07:24:30.6430732Z STATS: call_* op count: 1054 | FakeTensor.__torch_dispatch__:4504 | FakeTensorMode.__torch_dispatch__:65078 | attempt fast:1731 | fast is_contiguous:1731 | ProxyTorchDispatchMode.__torch_dispatch__:18627
2024-06-26T07:24:30.6432055Z Dynamo produced 3 graphs covering 1054 ops with 7 graph breaks (5 unique)
2024-06-26T07:24:35.6838143Z 
2024-06-26T07:24:36.4518893Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:24:36.4519407Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:24:36.4519848Z cuda train botnet26t_256                      
2024-06-26T07:24:57.2759452Z pass
2024-06-26T07:24:57.2933669Z TIMING: entire_frame_compile:19.35104 backend_compile:12.62838
2024-06-26T07:24:57.2935192Z STATS: call_* op count: 374 | FakeTensor.__torch_dispatch__:262 | FakeTensorMode.__torch_dispatch__:25616 | attempt fast:839 | fast is_contiguous:824 | slow no contiguity match:15 | ProxyTorchDispatchMode.__torch_dispatch__:5527
2024-06-26T07:24:57.2936649Z Dynamo produced 2 graphs covering 374 ops with 6 graph breaks (5 unique)
2024-06-26T07:25:01.8561301Z 
2024-06-26T07:25:05.6069158Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:25:05.6069754Z loading model: 0it [00:03, ?it/s]
2024-06-26T07:25:05.6071193Z cuda train cait_m36_384                       
2024-06-26T07:25:05.6075820Z Traceback (most recent call last):
2024-06-26T07:25:05.6076718Z   File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2321, in validate_model
2024-06-26T07:25:05.6077487Z     self.model_iter_fn(model, example_inputs)
2024-06-26T07:25:05.6078326Z   File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 371, in forward_and_backward_pass
2024-06-26T07:25:05.6079455Z     pred = mod(*cloned_inputs)
2024-06-26T07:25:05.6080783Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl
2024-06-26T07:25:05.6082477Z     return self._call_impl(*args, **kwargs)
2024-06-26T07:25:05.6083741Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl
2024-06-26T07:25:05.6085214Z     return forward_call(*args, **kwargs)
2024-06-26T07:25:05.6086107Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 361, in forward
2024-06-26T07:25:05.6086979Z     x = self.forward_features(x)
2024-06-26T07:25:05.6088135Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 346, in forward_features
2024-06-26T07:25:05.6089078Z     x = self.blocks(x)
2024-06-26T07:25:05.6089967Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl
2024-06-26T07:25:05.6090937Z     return self._call_impl(*args, **kwargs)
2024-06-26T07:25:05.6091861Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl
2024-06-26T07:25:05.6092697Z     return forward_call(*args, **kwargs)
2024-06-26T07:25:05.6093617Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/container.py", line 249, in forward
2024-06-26T07:25:05.6094422Z     input = module(input)
2024-06-26T07:25:05.6095319Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl
2024-06-26T07:25:05.6096192Z     return self._call_impl(*args, **kwargs)
2024-06-26T07:25:05.6097105Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl
2024-06-26T07:25:05.6097926Z     return forward_call(*args, **kwargs)
2024-06-26T07:25:05.6098766Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 197, in forward
2024-06-26T07:25:05.6099634Z     x = x + self.drop_path(self.gamma_1 * self.attn(self.norm1(x)))
2024-06-26T07:25:05.6100754Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl
2024-06-26T07:25:05.6101633Z     return self._call_impl(*args, **kwargs)
2024-06-26T07:25:05.6102541Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl
2024-06-26T07:25:05.6103364Z     return forward_call(*args, **kwargs)
2024-06-26T07:25:05.6104178Z   File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 151, in forward
2024-06-26T07:25:05.6104970Z     x = (attn @ v).transpose(1, 2).reshape(B, N, C)
2024-06-26T07:25:05.6108308Z torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 21.98 GiB of which 16.44 MiB is free. Process 879 has 21.95 GiB memory in use. Of the allocated memory 21.21 GiB is allocated by PyTorch, and 430.66 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
2024-06-26T07:25:05.6111055Z 
2024-06-26T07:25:05.6111354Z The above exception was the direct cause of the following exception:
2024-06-26T07:25:05.6111805Z 
2024-06-26T07:25:05.6111951Z Traceback (most recent call last):
2024-06-26T07:25:05.6112595Z   File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4229, in run
2024-06-26T07:25:05.6113246Z     ) = runner.load_model(
2024-06-26T07:25:05.6113897Z   File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 303, in load_model
2024-06-26T07:25:05.6114644Z     self.validate_model(model, example_inputs)
2024-06-26T07:25:05.6115396Z   File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2323, in validate_model
2024-06-26T07:25:05.6116146Z     raise RuntimeError("Eager run failed") from e
2024-06-26T07:25:05.6116609Z RuntimeError: Eager run failed
2024-06-26T07:25:05.6116948Z 
2024-06-26T07:25:05.6117066Z eager_fail_to_run
2024-06-26T07:25:09.0756156Z 
2024-06-26T07:25:09.7936816Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:25:09.7937346Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:25:09.7937914Z cuda train coat_lite_mini                     
2024-06-26T07:25:44.2247583Z pass
2024-06-26T07:25:44.2437109Z TIMING: entire_frame_compile:33.05733 backend_compile:22.6348
2024-06-26T07:25:44.2438822Z STATS: call_* op count: 538 | FakeTensor.__torch_dispatch__:951 | FakeTensorMode.__torch_dispatch__:41353 | attempt fast:1467 | fast is_contiguous:1355 | slow no contiguity match:112 | ProxyTorchDispatchMode.__torch_dispatch__:10627
2024-06-26T07:25:44.2440281Z Dynamo produced 2 graphs covering 538 ops with 6 graph breaks (5 unique)
2024-06-26T07:25:50.0409574Z 
2024-06-26T07:25:51.5054178Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:25:51.5054813Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:25:51.5055257Z cuda train convit_base                        
2024-06-26T07:25:51.5368281Z WARNING:common:fp64 golden ref were not generated for convit_base. Setting accuracy check to cosine
2024-06-26T07:27:04.8357707Z pass
2024-06-26T07:27:04.8833737Z TIMING: entire_frame_compile:72.33715 backend_compile:52.8282
2024-06-26T07:27:04.8836383Z STATS: call_* op count: 1308 | FakeTensor.__torch_dispatch__:1456 | FakeTensorMode.__torch_dispatch__:101336 | attempt fast:4260 | slow no contiguity match:191 | fast is_contiguous:4069 | ProxyTorchDispatchMode.__torch_dispatch__:28022
2024-06-26T07:27:04.8837870Z Dynamo produced 3 graphs covering 1308 ops with 7 graph breaks (5 unique)
2024-06-26T07:27:11.9370277Z 
2024-06-26T07:27:12.6886465Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:27:12.6886930Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:27:12.6888288Z cuda train convmixer_768_32                   
2024-06-26T07:27:41.0201641Z pass
2024-06-26T07:27:41.0786282Z TIMING: entire_frame_compile:22.21955 backend_compile:15.95093
2024-06-26T07:27:41.0787808Z STATS: call_* op count: 235 | FakeTensor.__torch_dispatch__:1603 | FakeTensorMode.__torch_dispatch__:44597 | attempt fast:1663 | fast is_contiguous:1663 | ProxyTorchDispatchMode.__torch_dispatch__:9010
2024-06-26T07:27:41.0789109Z Dynamo produced 2 graphs covering 235 ops with 5 graph breaks (4 unique)
2024-06-26T07:27:45.7619034Z 
2024-06-26T07:27:47.2939030Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:27:47.2939501Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:27:47.2939943Z cuda train convnext_base                      
2024-06-26T07:28:28.8823274Z W0626 07:28:28.881000 139882214544000 torch/_logging/_internal.py:1040] [6/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
2024-06-26T07:29:04.5518814Z pass
2024-06-26T07:29:04.6353219Z TIMING: entire_frame_compile:72.00478 backend_compile:44.90271
2024-06-26T07:29:04.6355403Z STATS: call_* op count: 1621 | FakeTensor.__torch_dispatch__:8036 | FakeTensorMode.__torch_dispatch__:105434 | attempt fast:3151 | fast is_contiguous:2758 | fast channels_last:196 | slow no contiguity match:197 | ProxyTorchDispatchMode.__torch_dispatch__:29746
2024-06-26T07:29:04.6357160Z Dynamo produced 3 graphs covering 1621 ops with 7 graph breaks (5 unique)
2024-06-26T07:29:10.8218269Z 
2024-06-26T07:29:11.4551921Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:29:11.4552406Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:29:11.4552919Z cuda train crossvit_9_240                     
2024-06-26T07:29:53.6727218Z W0626 07:29:53.671000 140405482668672 torch/_logging/_internal.py:1040] [6/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
2024-06-26T07:30:20.3078478Z pass
2024-06-26T07:30:20.3255296Z TIMING: entire_frame_compile:66.87112 backend_compile:43.91917
2024-06-26T07:30:20.3257920Z STATS: call_* op count: 1441 | FakeTensor.__torch_dispatch__:6944 | FakeTensorMode.__torch_dispatch__:99894 | attempt fast:3070 | fast is_contiguous:3046 | slow no contiguity match:24 | ProxyTorchDispatchMode.__torch_dispatch__:27501
2024-06-26T07:30:20.3259747Z Dynamo produced 3 graphs covering 1441 ops with 7 graph breaks (5 unique)
2024-06-26T07:30:26.7829408Z 
2024-06-26T07:30:27.7743981Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:30:27.7744572Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:30:27.7745026Z cuda train cspdarknet53                       
2024-06-26T07:31:02.4182038Z W0626 07:31:02.417000 140559792906880 torch/_logging/_internal.py:1040] [6/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
2024-06-26T07:31:24.8218552Z pass
2024-06-26T07:31:24.8508449Z TIMING: entire_frame_compile:53.80861 backend_compile:33.80866
2024-06-26T07:31:24.8509929Z STATS: call_* op count: 1070 | FakeTensor.__torch_dispatch__:4688 | FakeTensorMode.__torch_dispatch__:73123 | attempt fast:1999 | fast is_contiguous:1979 | slow no contiguity match:20 | ProxyTorchDispatchMode.__torch_dispatch__:17131
2024-06-26T07:31:24.8511404Z Dynamo produced 3 graphs covering 1070 ops with 7 graph breaks (5 unique)
2024-06-26T07:31:30.3445198Z 
2024-06-26T07:31:31.7750939Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:31:31.7751531Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:31:31.7754181Z cuda train deit_base_distilled_patch16_224    
2024-06-26T07:31:55.5769845Z W0626 07:31:55.576000 140424133272192 torch/_logging/_internal.py:1040] [6/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
2024-06-26T07:32:10.9801454Z pass
2024-06-26T07:32:11.0714026Z TIMING: entire_frame_compile:34.31111 backend_compile:22.69986
2024-06-26T07:32:11.0715306Z STATS: call_* op count: 869 | FakeTensor.__torch_dispatch__:4082 | FakeTensorMode.__torch_dispatch__:53911 | attempt fast:1609 | fast is_contiguous:1609 | ProxyTorchDispatchMode.__torch_dispatch__:14631
2024-06-26T07:32:11.0716644Z Dynamo produced 3 graphs covering 869 ops with 7 graph breaks (5 unique)
2024-06-26T07:32:15.8675868Z 
2024-06-26T07:32:16.9868220Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:32:16.9868791Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:32:16.9871412Z cuda train dla102                             
2024-06-26T07:32:55.1173300Z W0626 07:32:55.116000 139752739000960 torch/_logging/_internal.py:1040] [6/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
2024-06-26T07:33:30.5025704Z pass
2024-06-26T07:33:30.5316291Z TIMING: entire_frame_compile:69.69235 backend_compile:47.82858
2024-06-26T07:33:30.5317721Z STATS: call_* op count: 1389 | FakeTensor.__torch_dispatch__:8125 | FakeTensorMode.__torch_dispatch__:110312 | attempt fast:2568 | fast is_contiguous:2529 | ProxyTorchDispatchMode.__torch_dispatch__:26824 | slow no contiguity match:39
2024-06-26T07:33:30.5319174Z Dynamo produced 3 graphs covering 1389 ops with 7 graph breaks (5 unique)
2024-06-26T07:33:36.6267817Z 
2024-06-26T07:33:38.1326024Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:33:38.1326499Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:33:38.1326925Z cuda train dm_nfnet_f0                        
2024-06-26T07:34:08.3527440Z pass
2024-06-26T07:34:08.4032059Z TIMING: entire_frame_compile:27.07783 backend_compile:17.03518
2024-06-26T07:34:08.4033436Z STATS: call_* op count: 620 | FakeTensorMode.__torch_dispatch__:39327 | attempt fast:1872 | fast is_contiguous:1872 | FakeTensor.__torch_dispatch__:334 | ProxyTorchDispatchMode.__torch_dispatch__:10519
2024-06-26T07:34:08.4034726Z Dynamo produced 2 graphs covering 620 ops with 6 graph breaks (5 unique)
2024-06-26T07:34:13.0844246Z 
2024-06-26T07:34:14.4087595Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:34:14.4088099Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:34:14.4088540Z cuda train dpn107                             
2024-06-26T07:35:24.7648777Z pass
2024-06-26T07:35:24.8407538Z TIMING: entire_frame_compile:63.44432 backend_compile:44.84018
2024-06-26T07:35:24.8409333Z STATS: call_* op count: 746 | FakeTensor.__torch_dispatch__:820 | FakeTensorMode.__torch_dispatch__:79398 | attempt fast:2798 | fast is_contiguous:2596 | slow no contiguity match:202 | ProxyTorchDispatchMode.__torch_dispatch__:16915
2024-06-26T07:35:24.8411232Z Dynamo produced 2 graphs covering 746 ops with 6 graph breaks (5 unique)
2024-06-26T07:35:31.7241805Z 
2024-06-26T07:35:32.4695651Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:35:32.4696131Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:35:32.4699855Z cuda train eca_botnext26ts_256                
2024-06-26T07:35:58.5836369Z W0626 07:35:58.582000 140589602300544 torch/_logging/_internal.py:1040] [6/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
2024-06-26T07:36:10.7380179Z pass
2024-06-26T07:36:10.7531650Z TIMING: entire_frame_compile:36.72513 backend_compile:22.78905
2024-06-26T07:36:10.7535905Z STATS: call_* op count: 760 | FakeTensor.__torch_dispatch__:2550 | FakeTensorMode.__torch_dispatch__:46660 | attempt fast:1105 | fast is_contiguous:1065 | slow no contiguity match:40 | ProxyTorchDispatchMode.__torch_dispatch__:11488
2024-06-26T07:36:10.7537399Z Dynamo produced 3 graphs covering 760 ops with 7 graph breaks (5 unique)
2024-06-26T07:36:15.7634603Z 
2024-06-26T07:36:16.5466985Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:36:16.5467444Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:36:16.5469793Z cuda train eca_halonext26ts                   
2024-06-26T07:36:46.1229953Z W0626 07:36:46.122000 139654391009920 torch/_logging/_internal.py:1040] [6/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
2024-06-26T07:36:59.0581209Z pass
2024-06-26T07:36:59.0718484Z TIMING: entire_frame_compile:40.92641 backend_compile:25.32345
2024-06-26T07:36:59.0721691Z STATS: call_* op count: 790 | FakeTensor.__torch_dispatch__:2621 | FakeTensorMode.__torch_dispatch__:48442 | attempt fast:1108 | fast is_contiguous:1068 | slow no contiguity match:40 | ProxyTorchDispatchMode.__torch_dispatch__:11921
2024-06-26T07:36:59.0723240Z Dynamo produced 3 graphs covering 790 ops with 7 graph breaks (5 unique)
2024-06-26T07:37:04.2810241Z 
2024-06-26T07:37:04.9595448Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:37:04.9596356Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:37:04.9600588Z cuda train ese_vovnet19b_dw                   
2024-06-26T07:37:18.2572104Z W0626 07:37:18.256000 140047129391744 torch/_logging/_internal.py:1040] [7/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
2024-06-26T07:37:29.3839907Z pass
2024-06-26T07:37:29.3915406Z TIMING: entire_frame_compile:23.44795 backend_compile:14.10311
2024-06-26T07:37:29.3918011Z STATS: call_* op count: 459 | FakeTensor.__torch_dispatch__:2244 | FakeTensorMode.__torch_dispatch__:31315 | attempt fast:613 | fast is_contiguous:601 | ProxyTorchDispatchMode.__torch_dispatch__:7887 | slow no contiguity match:12
2024-06-26T07:37:29.3919472Z Dynamo produced 4 graphs covering 459 ops with 7 graph breaks (5 unique)
2024-06-26T07:37:33.6688332Z 
2024-06-26T07:37:34.3875469Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:37:34.3875948Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:37:34.3880084Z cuda train fbnetc_100                         
2024-06-26T07:37:58.0028246Z W0626 07:37:58.002000 140412061864576 torch/_logging/_internal.py:1040] [7/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
2024-06-26T07:38:19.9889540Z pass
2024-06-26T07:38:19.9959320Z TIMING: entire_frame_compile:44.33291 backend_compile:27.01446
2024-06-26T07:38:19.9961948Z STATS: call_* op count: 993 | FakeTensor.__torch_dispatch__:4430 | FakeTensorMode.__torch_dispatch__:65282 | attempt fast:1578 | fast is_contiguous:1578 | ProxyTorchDispatchMode.__torch_dispatch__:16037
2024-06-26T07:38:19.9963280Z Dynamo produced 4 graphs covering 993 ops with 7 graph breaks (5 unique)
2024-06-26T07:38:25.1048316Z 
2024-06-26T07:38:26.0149297Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:38:26.0149765Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:38:26.0150207Z cuda train fbnetv3_b                          
2024-06-26T07:39:22.0358062Z pass
2024-06-26T07:39:22.0486985Z TIMING: entire_frame_compile:54.50683 backend_compile:35.0349
2024-06-26T07:39:22.0489710Z STATS: call_* op count: 610 | FakeTensor.__torch_dispatch__:1382 | FakeTensorMode.__torch_dispatch__:71463 | attempt fast:3567 | fast is_contiguous:3567 | ProxyTorchDispatchMode.__torch_dispatch__:11990
2024-06-26T07:39:22.0491124Z Dynamo produced 2 graphs covering 610 ops with 6 graph breaks (5 unique)
2024-06-26T07:39:28.6276035Z 
2024-06-26T07:39:29.6433395Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:39:29.6434312Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:39:29.6435204Z cuda train gernet_l                           
2024-06-26T07:39:53.6342783Z pass
2024-06-26T07:39:53.6565119Z TIMING: entire_frame_compile:21.92716 backend_compile:14.21166
2024-06-26T07:39:53.6567684Z STATS: call_* op count: 410 | FakeTensor.__torch_dispatch__:410 | FakeTensorMode.__torch_dispatch__:32784 | attempt fast:1409 | fast is_contiguous:1409 | ProxyTorchDispatchMode.__torch_dispatch__:6598
2024-06-26T07:39:53.6569005Z Dynamo produced 2 graphs covering 410 ops with 6 graph breaks (5 unique)
2024-06-26T07:39:58.2163955Z 
2024-06-26T07:39:58.9795757Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:39:58.9796226Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:39:58.9799248Z cuda train ghostnet_100                       
2024-06-26T07:40:35.2929609Z pass
2024-06-26T07:40:35.2995338Z TIMING: entire_frame_compile:35.13682 backend_compile:25.29065
2024-06-26T07:40:35.2997013Z STATS: call_* op count: 360 | FakeTensor.__torch_dispatch__:1790 | FakeTensorMode.__torch_dispatch__:56784 | attempt fast:2040 | fast is_contiguous:2008 | ProxyTorchDispatchMode.__torch_dispatch__:11967 | slow no contiguity match:32
2024-06-26T07:40:35.2998444Z Dynamo produced 2 graphs covering 360 ops with 6 graph breaks (5 unique)
2024-06-26T07:40:40.9519291Z 
2024-06-26T07:40:42.0503285Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:40:42.0503871Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:40:42.0504339Z cuda train gluon_inception_v3                 
2024-06-26T07:41:21.2940531Z W0626 07:41:21.293000 140038683169408 torch/_logging/_internal.py:1040] [6/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
2024-06-26T07:41:53.0296342Z pass
2024-06-26T07:41:53.0525746Z TIMING: entire_frame_compile:67.8871 backend_compile:42.1974
2024-06-26T07:41:53.0528223Z STATS: call_* op count: 1376 | FakeTensor.__torch_dispatch__:6261 | FakeTensorMode.__torch_dispatch__:97073 | attempt fast:2205 | fast is_contiguous:2205 | ProxyTorchDispatchMode.__torch_dispatch__:24031
2024-06-26T07:41:53.0529633Z Dynamo produced 3 graphs covering 1376 ops with 7 graph breaks (5 unique)
2024-06-26T07:41:58.9757813Z 
2024-06-26T07:41:59.8027757Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:41:59.8028238Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:41:59.8028680Z cuda train gmixer_24_224                      
2024-06-26T07:42:47.3387468Z pass
2024-06-26T07:42:47.3559317Z TIMING: entire_frame_compile:45.16782 backend_compile:32.42013
2024-06-26T07:42:47.3561602Z STATS: call_* op count: 684 | FakeTensor.__torch_dispatch__:1795 | FakeTensorMode.__torch_dispatch__:69863 | attempt fast:3111 | fast is_contiguous:2631 | slow no contiguity match:480 | ProxyTorchDispatchMode.__torch_dispatch__:16426
2024-06-26T07:42:47.3563094Z Dynamo produced 2 graphs covering 684 ops with 6 graph breaks (5 unique)
2024-06-26T07:42:52.9603573Z 
2024-06-26T07:42:53.7160814Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:42:53.7161874Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:42:53.7162557Z cuda train gmlp_s16_224                       
2024-06-26T07:43:34.1100704Z W0626 07:43:34.109000 140178471117440 torch/_logging/_internal.py:1040] [6/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
2024-06-26T07:44:04.1317579Z pass
2024-06-26T07:44:04.1563257Z TIMING: entire_frame_compile:67.67248 backend_compile:44.76592
2024-06-26T07:44:04.1566420Z STATS: call_* op count: 1458 | FakeTensor.__torch_dispatch__:7723 | FakeTensorMode.__torch_dispatch__:107061 | attempt fast:3579 | fast is_contiguous:3279 | slow no contiguity match:300 | ProxyTorchDispatchMode.__torch_dispatch__:28113
2024-06-26T07:44:04.1568092Z Dynamo produced 3 graphs covering 1458 ops with 7 graph breaks (5 unique)
2024-06-26T07:44:10.2082520Z 
2024-06-26T07:44:11.7911471Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:44:11.7912093Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:44:11.7912584Z cuda train hrnet_w18                          
2024-06-26T07:46:00.7749378Z pass
2024-06-26T07:46:00.7968135Z TIMING: entire_frame_compile:103.71713 backend_compile:79.19339
2024-06-26T07:46:00.7971745Z STATS: call_* op count: 1394 | FakeTensor.__torch_dispatch__:5985 | FakeTensorMode.__torch_dispatch__:196668 | attempt fast:8144 | fast is_contiguous:8144 | ProxyTorchDispatchMode.__torch_dispatch__:39732
2024-06-26T07:46:00.7973126Z Dynamo produced 2 graphs covering 1394 ops with 5 graph breaks (4 unique)
2024-06-26T07:46:10.0371096Z 
2024-06-26T07:46:11.1142555Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:46:11.1143463Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:46:11.1144322Z cuda train inception_v3                       
2024-06-26T07:46:50.6819333Z pass
2024-06-26T07:46:50.7047898Z TIMING: entire_frame_compile:37.02471 backend_compile:24.80571
2024-06-26T07:46:50.7051187Z STATS: call_* op count: 506 | FakeTensor.__torch_dispatch__:665 | FakeTensorMode.__torch_dispatch__:54976 | attempt fast:2205 | fast is_contiguous:2205 | ProxyTorchDispatchMode.__torch_dispatch__:11234
2024-06-26T07:46:50.7052508Z Dynamo produced 2 graphs covering 506 ops with 6 graph breaks (5 unique)
2024-06-26T07:46:56.1217968Z 
2024-06-26T07:46:57.5046991Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:46:57.5047687Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:46:57.5050260Z cuda train jx_nest_base                       
2024-06-26T07:48:07.5762095Z W0626 07:48:07.575000 139859519353472 torch/_logging/_internal.py:1040] [6/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
2024-06-26T07:48:37.1390238Z pass
2024-06-26T07:48:37.1988671Z TIMING: entire_frame_compile:93.45509 backend_compile:62.74036
2024-06-26T07:48:37.1990082Z STATS: call_* op count: 1852 | FakeTensor.__torch_dispatch__:7156 | FakeTensorMode.__torch_dispatch__:127031 | attempt fast:3795 | fast is_contiguous:3795 | ProxyTorchDispatchMode.__torch_dispatch__:33243
2024-06-26T07:48:37.1991569Z Dynamo produced 3 graphs covering 1852 ops with 7 graph breaks (5 unique)
2024-06-26T07:48:44.6792276Z 
2024-06-26T07:48:45.2370869Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:48:45.2371465Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:48:45.2373257Z cuda train lcnet_050                          
2024-06-26T07:49:02.1405936Z E0626 07:49:02.139000 140080960598656 torch/_dynamo/utils.py:1470] RMSE (res-fp64): 0.00769, (ref-fp64): 0.00017 and shape=torch.Size([32]). res.dtype: torch.float32, multiplier: 3.000000, tol: 0.010000
2024-06-26T07:49:02.1407523Z E0626 07:49:02.139000 140080960598656 torch/_dynamo/utils.py:1384] Accuracy failed for key name blocks.1.0.bn2.weight.grad
2024-06-26T07:49:02.1419302Z fail_accuracy
2024-06-26T07:49:02.1439073Z TIMING: entire_frame_compile:16.45984 backend_compile:10.5457
2024-06-26T07:49:02.1440547Z STATS: call_* op count: 168 | FakeTensor.__torch_dispatch__:395 | FakeTensorMode.__torch_dispatch__:21371 | attempt fast:1081 | fast is_contiguous:1081 | ProxyTorchDispatchMode.__torch_dispatch__:3329
2024-06-26T07:49:02.1441854Z Dynamo produced 2 graphs covering 168 ops with 6 graph breaks (5 unique)
2024-06-26T07:49:06.5254004Z 
2024-06-26T07:49:07.3359317Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:49:07.3360207Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:49:07.3361075Z cuda train levit_128                          
2024-06-26T07:49:51.3238325Z W0626 07:49:51.323000 140309604774528 torch/_logging/_internal.py:1040] [6/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
2024-06-26T07:50:12.1871207Z pass
2024-06-26T07:50:12.1917894Z TIMING: entire_frame_compile:63.35204 backend_compile:42.71262
2024-06-26T07:50:12.1921140Z STATS: call_* op count: 1257 | FakeTensor.__torch_dispatch__:5301 | FakeTensorMode.__torch_dispatch__:93739 | attempt fast:2375 | fast is_contiguous:2339 | slow no contiguity match:35 | ProxyTorchDispatchMode.__torch_dispatch__:23437 | fast channels_last:1
2024-06-26T07:50:12.1922720Z Dynamo produced 3 graphs covering 1257 ops with 7 graph breaks (5 unique)
2024-06-26T07:50:18.3608899Z 
2024-06-26T07:50:19.4369559Z loading model: 0it [00:00, ?it/s]
2024-06-26T07:50:19.4370087Z loading model: 0it [00:01, ?it/s]
2024-06-26T07:50:19.4370670Z cuda train mixer_b16_224                      
2024-06-26T07:50:42.1728316Z W0626 07:50:42.172000 140470030426752 torch/_logging/_internal.py:1040] [6/0] Profiler function <class 'torch.autograd.profiler.record_function'> will be ignored
2024-06-26T07:50:56.6075888Z pass
2024-06-26T07:50:56.6746647Z TIMING: entire_frame_compile:33.37465 backend_compile:22.23268
2024-06-26T07:50:56.6749550Z STATS: call_* op count: 720 | FakeTensor.__torch_dispatch__:3828 | FakeTensorMode.__torch_dispatch__:52244 | attempt fast:1839 | fast is_contiguous:1743 | slow no contiguity match:96 | ProxyTorchDispatchMode.__torch_dispatch__:13749
2024-06-26T07:50:56.6751050Z Dynamo produced 3 graphs covering 720 ops with 7 graph breaks (5 unique)
2024-06-26T07:50:58.5668331Z accuracy            pass_rate=93.33%
2024-06-26T07:50:58.5673022Z calls_captured      gmean=0.00x mean=852.267x
2024-06-26T07:50:58.5677101Z unique_graphs       gmean=0.00x mean=2.533x
2024-06-26T07:50:58.5681380Z graph_breaks        gmean=0.00x mean=6.267x
2024-06-26T07:50:58.5685710Z unique_graph_breaks gmean=0.00x mean=4.767x
2024-06-26T07:50:58.5690145Z autograd_captures   gmean=0.00x mean=0.000x
2024-06-26T07:50:58.5694469Z autograd_compiles   gmean=0.00x mean=0.000x
2024-06-26T07:50:58.5698297Z cudagraph_skips     gmean=0.00x mean=0.000x
2024-06-26T07:50:59.1582976Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy//dynamic_aot_eager_timm_training.csv
2024-06-26T07:50:59.3994840Z adv_inception_v3                    PASS
2024-06-26T07:50:59.4000203Z beit_base_patch16_224               PASS
2024-06-26T07:50:59.4004416Z botnet26t_256                       PASS
2024-06-26T07:50:59.4008739Z cait_m36_384                        XFAIL
2024-06-26T07:50:59.4013517Z coat_lite_mini                      PASS
2024-06-26T07:50:59.4018298Z convit_base                         PASS
2024-06-26T07:50:59.4023013Z convmixer_768_32                    PASS
2024-06-26T07:50:59.4027541Z convnext_base                       PASS
2024-06-26T07:50:59.4032122Z crossvit_9_240                      PASS
2024-06-26T07:50:59.4036613Z cspdarknet53                        PASS
2024-06-26T07:50:59.4041091Z deit_base_distilled_patch16_224     PASS
2024-06-26T07:50:59.4046160Z dla102                              PASS
2024-06-26T07:50:59.4050400Z dm_nfnet_f0                         PASS
2024-06-26T07:50:59.4055224Z dpn107                              PASS
2024-06-26T07:50:59.4059610Z eca_botnext26ts_256                 PASS
2024-06-26T07:50:59.4063985Z eca_halonext26ts                    PASS
2024-06-26T07:50:59.4068528Z ese_vovnet19b_dw                    PASS
2024-06-26T07:50:59.4072945Z fbnetc_100                          PASS
2024-06-26T07:50:59.4077447Z fbnetv3_b                           PASS
2024-06-26T07:50:59.4081899Z gernet_l                            PASS
2024-06-26T07:50:59.4087077Z ghostnet_100                        PASS
2024-06-26T07:50:59.4091213Z gluon_inception_v3                  PASS
2024-06-26T07:50:59.4095764Z gmixer_24_224                       PASS
2024-06-26T07:50:59.4100204Z gmlp_s16_224                        PASS
2024-06-26T07:50:59.4104653Z hrnet_w18                           PASS
2024-06-26T07:50:59.4109113Z inception_v3                        PASS
2024-06-26T07:50:59.4113635Z jx_nest_base                        PASS
2024-06-26T07:50:59.4118182Z lcnet_050                           XFAIL
2024-06-26T07:50:59.4122623Z levit_128                           PASS
2024-06-26T07:50:59.4127382Z mixer_b16_224                       PASS
2024-06-26T07:50:59.4504212Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy//dynamic_aot_eager_timm_training.csv
2024-06-26T07:50:59.7055222Z adv_inception_v3                    PASS
2024-06-26T07:50:59.7058481Z beit_base_patch16_224               PASS
2024-06-26T07:50:59.7063622Z botnet26t_256                       PASS
2024-06-26T07:50:59.7068415Z cait_m36_384                        PASS
2024-06-26T07:50:59.7073290Z coat_lite_mini                      PASS
2024-06-26T07:50:59.7078184Z convit_base                         PASS
2024-06-26T07:50:59.7082754Z convmixer_768_32                    PASS
2024-06-26T07:50:59.7087933Z convnext_base                       PASS
2024-06-26T07:50:59.7092714Z crossvit_9_240                      PASS
2024-06-26T07:50:59.7097624Z cspdarknet53                        PASS
2024-06-26T07:50:59.7102318Z deit_base_distilled_patch16_224     PASS
2024-06-26T07:50:59.7107047Z dla102                              PASS
2024-06-26T07:50:59.7111813Z dm_nfnet_f0                         PASS
2024-06-26T07:50:59.7116262Z dpn107                              PASS
2024-06-26T07:50:59.7121154Z eca_botnext26ts_256                 PASS
2024-06-26T07:50:59.7126062Z eca_halonext26ts                    PASS
2024-06-26T07:50:59.7131123Z ese_vovnet19b_dw                    PASS
2024-06-26T07:50:59.7136334Z fbnetc_100                          PASS
2024-06-26T07:50:59.7141044Z fbnetv3_b                           PASS
2024-06-26T07:50:59.7146159Z gernet_l                            PASS
2024-06-26T07:50:59.7151024Z ghostnet_100                        PASS
2024-06-26T07:50:59.7156021Z gluon_inception_v3                  PASS
2024-06-26T07:50:59.7160685Z gmixer_24_224                       PASS
2024-06-26T07:50:59.7166010Z gmlp_s16_224                        PASS
2024-06-26T07:50:59.7170881Z hrnet_w18                           PASS
2024-06-26T07:50:59.7175903Z inception_v3                        PASS
2024-06-26T07:50:59.7180822Z jx_nest_base                        PASS
2024-06-26T07:50:59.7185788Z lcnet_050                           PASS
2024-06-26T07:50:59.7190694Z levit_128                           PASS
2024-06-26T07:50:59.7195667Z mixer_b16_224                       PASS
2024-06-26T07:50:59.7583219Z + cleanup_workspace
2024-06-26T07:50:59.7584249Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.'
2024-06-26T07:50:59.7585643Z sudo may print the following warning message that can be ignored. The chown command will still run.
2024-06-26T07:50:59.7587643Z + echo '    sudo: setrlimit(RLIMIT_STACK): Operation not permitted'
2024-06-26T07:50:59.7588595Z     sudo: setrlimit(RLIMIT_STACK): Operation not permitted
2024-06-26T07:50:59.7589419Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42'
2024-06-26T07:50:59.7590291Z For more details refer to https://github.com/sudo-project/sudo/issues/42
2024-06-26T07:50:59.7590974Z + sudo chown -R 1000 /var/lib/jenkins/workspace
2024-06-26T07:51:00.3685970Z ##[group]Run cat test/**/*_toprint.log || true
2024-06-26T07:51:00.3686504Z cat test/**/*_toprint.log || true
2024-06-26T07:51:00.3694300Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:51:00.3694790Z env:
2024-06-26T07:51:00.3695058Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:51:00.3695506Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:51:00.3696243Z   DOCKER_CONTAINER_ID: e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:00.3696883Z ##[endgroup]
2024-06-26T07:51:00.3763400Z cat: test/**/*_toprint.log: No such file or directory
2024-06-26T07:51:00.3826043Z ##[group]Run kill "$MONITOR_SCRIPT_PID"
2024-06-26T07:51:00.3826510Z kill "$MONITOR_SCRIPT_PID"
2024-06-26T07:51:00.3834213Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:51:00.3834713Z env:
2024-06-26T07:51:00.3834993Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:51:00.3835447Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:51:00.3836183Z   DOCKER_CONTAINER_ID: e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:00.3836846Z   MONITOR_SCRIPT_PID: 27812
2024-06-26T07:51:00.3837200Z ##[endgroup]
2024-06-26T07:51:00.4018109Z Prepare all required actions
2024-06-26T07:51:00.4018554Z Getting action download info
2024-06-26T07:51:00.5895954Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32)
2024-06-26T07:51:00.7587764Z ##[group]Run ./.github/actions/upload-test-artifacts
2024-06-26T07:51:00.7588236Z with:
2024-06-26T07:51:00.7588780Z   file-suffix: test-dynamic_aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688917313
2024-06-26T07:51:00.7589451Z   s3-bucket: gha-artifacts
2024-06-26T07:51:00.7589821Z env:
2024-06-26T07:51:00.7590102Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:51:00.7590568Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:51:00.7591318Z   DOCKER_CONTAINER_ID: e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:00.7591967Z ##[endgroup]
2024-06-26T07:51:00.7624768Z ##[group]Run # Remove any previous test jsons if they exist
2024-06-26T07:51:00.7625419Z # Remove any previous test jsons if they exist
2024-06-26T07:51:00.7625934Z rm -f test-jsons-*.zip
2024-06-26T07:51:00.7626451Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json'
2024-06-26T07:51:00.7634213Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:51:00.7634709Z env:
2024-06-26T07:51:00.7634992Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:51:00.7635441Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:51:00.7636187Z   DOCKER_CONTAINER_ID: e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:00.7637091Z   FILE_SUFFIX: test-dynamic_aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688917313
2024-06-26T07:51:00.7637734Z ##[endgroup]
2024-06-26T07:51:00.7809243Z   adding: test/allowlist_for_publicAPI.json (deflated 79%)
2024-06-26T07:51:00.7838412Z   adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%)
2024-06-26T07:51:00.7839100Z   adding: test/minioptest_failures_dict.json (deflated 70%)
2024-06-26T07:51:00.7845340Z   adding: test/profiler/profiler_utils_mock_events.json (deflated 87%)
2024-06-26T07:51:00.7870739Z ##[group]Run # Remove any previous test reports if they exist
2024-06-26T07:51:00.7871389Z # Remove any previous test reports if they exist
2024-06-26T07:51:00.7871911Z rm -f test-reports-*.zip
2024-06-26T07:51:00.7872495Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv'
2024-06-26T07:51:00.7880158Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:51:00.7880644Z env:
2024-06-26T07:51:00.7880938Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:51:00.7881388Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:51:00.7882126Z   DOCKER_CONTAINER_ID: e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:00.7883018Z   FILE_SUFFIX: test-dynamic_aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688917313
2024-06-26T07:51:00.7883658Z ##[endgroup]
2024-06-26T07:51:00.8034972Z   adding: test/test-reports/inference_timm_models.csv (deflated 69%)
2024-06-26T07:51:00.8035775Z   adding: test/test-reports/training_timm_models.csv (deflated 66%)
2024-06-26T07:51:00.8039274Z   adding: test/test-reports/training_timm_model_graph_breaks.csv (deflated 97%)
2024-06-26T07:51:00.8040213Z   adding: test/test-reports/training_timm_model_graph_break_deduped.csv (deflated 73%)
2024-06-26T07:51:00.8065337Z ##[group]Run # Remove any previous usage logs if they exist
2024-06-26T07:51:00.8065964Z # Remove any previous usage logs if they exist
2024-06-26T07:51:00.8066605Z rm -f logs-*.zip
2024-06-26T07:51:00.8067260Z # this workflow is also run in bazel build test, but we dont generate usage reports for it
2024-06-26T07:51:00.8068034Z # so check to see if the file exists first
2024-06-26T07:51:00.8068533Z if [ -f 'usage_log.txt' ]; then
2024-06-26T07:51:00.8069059Z     zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt'
2024-06-26T07:51:00.8069544Z fi
2024-06-26T07:51:00.8070025Z if ls test/**/*.log 1> /dev/null 2>&1; then
2024-06-26T07:51:00.8070642Z     zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log'
2024-06-26T07:51:00.8071169Z fi
2024-06-26T07:51:00.8078381Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:51:00.8078882Z env:
2024-06-26T07:51:00.8079155Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:51:00.8079611Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:51:00.8080362Z   DOCKER_CONTAINER_ID: e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:00.8081278Z   FILE_SUFFIX: test-dynamic_aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688917313
2024-06-26T07:51:00.8081912Z ##[endgroup]
2024-06-26T07:51:00.8131347Z   adding: usage_log.txt (deflated 92%)
2024-06-26T07:51:00.8194928Z ##[group]Run # Remove any previous debugging artifacts if they exist
2024-06-26T07:51:00.8195643Z # Remove any previous debugging artifacts if they exist
2024-06-26T07:51:00.8196199Z rm -f debug-*.zip
2024-06-26T07:51:00.8196583Z if [ -d 'test/debug' ]; then
2024-06-26T07:51:00.8197080Z   zip -r "debug-${FILE_SUFFIX}.zip" test/debug
2024-06-26T07:51:00.8197552Z fi
2024-06-26T07:51:00.8204122Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:51:00.8204610Z env:
2024-06-26T07:51:00.8205164Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:51:00.8205620Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:51:00.8206363Z   DOCKER_CONTAINER_ID: e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:00.8207260Z   FILE_SUFFIX: test-dynamic_aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688917313
2024-06-26T07:51:00.8207894Z ##[endgroup]
2024-06-26T07:51:00.8298199Z ##[group]Run seemethere/upload-artifact-s3@v5
2024-06-26T07:51:00.8298644Z with:
2024-06-26T07:51:00.8298923Z   s3-bucket: gha-artifacts
2024-06-26T07:51:00.8299357Z   s3-prefix: pytorch/pytorch/9673646806/1/artifact

2024-06-26T07:51:00.8299829Z   retention-days: 14
2024-06-26T07:51:00.8300169Z   if-no-files-found: warn
2024-06-26T07:51:00.8300518Z   path: test-jsons-*.zip
2024-06-26T07:51:00.8300860Z   name: artifact
2024-06-26T07:51:00.8301167Z   region: us-east-1
2024-06-26T07:51:00.8301460Z env:
2024-06-26T07:51:00.8301736Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:51:00.8302187Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:51:00.8302929Z   DOCKER_CONTAINER_ID: e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:00.8303581Z ##[endgroup]
2024-06-26T07:51:01.1350870Z NOTE: s3-prefix specified, ignoring name parameter
2024-06-26T07:51:01.1351564Z With the provided path, there will be 1 file uploaded
2024-06-26T07:51:01.1352176Z Uploading to s3 prefix: pytorch/pytorch/9673646806/1/artifact
2024-06-26T07:51:01.1377897Z Starting upload of test-jsons-test-dynamic_aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688917313.zip
2024-06-26T07:51:01.2401043Z Finished upload of test-jsons-test-dynamic_aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688917313.zip
2024-06-26T07:51:01.2533563Z ##[group]Run seemethere/upload-artifact-s3@v5
2024-06-26T07:51:01.2533996Z with:
2024-06-26T07:51:01.2534280Z   s3-bucket: gha-artifacts
2024-06-26T07:51:01.2534710Z   s3-prefix: pytorch/pytorch/9673646806/1/artifact

2024-06-26T07:51:01.2535173Z   retention-days: 14
2024-06-26T07:51:01.2535501Z   if-no-files-found: error
2024-06-26T07:51:01.2535871Z   path: test-reports-*.zip
2024-06-26T07:51:01.2536216Z   name: artifact
2024-06-26T07:51:01.2536646Z   region: us-east-1
2024-06-26T07:51:01.2536945Z env:
2024-06-26T07:51:01.2537218Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:51:01.2537663Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:51:01.2538393Z   DOCKER_CONTAINER_ID: e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:01.2539034Z ##[endgroup]
2024-06-26T07:51:01.5610732Z NOTE: s3-prefix specified, ignoring name parameter
2024-06-26T07:51:01.5611809Z With the provided path, there will be 1 file uploaded
2024-06-26T07:51:01.5612426Z Uploading to s3 prefix: pytorch/pytorch/9673646806/1/artifact
2024-06-26T07:51:01.5637105Z Starting upload of test-reports-test-dynamic_aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688917313.zip
2024-06-26T07:51:01.6868692Z Finished upload of test-reports-test-dynamic_aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688917313.zip
2024-06-26T07:51:01.6993773Z ##[group]Run seemethere/upload-artifact-s3@v5
2024-06-26T07:51:01.6994216Z with:
2024-06-26T07:51:01.6994522Z   s3-bucket: gha-artifacts
2024-06-26T07:51:01.6994953Z   s3-prefix: pytorch/pytorch/9673646806/1/artifact

2024-06-26T07:51:01.6995424Z   retention-days: 14
2024-06-26T07:51:01.6995757Z   if-no-files-found: ignore
2024-06-26T07:51:01.6996119Z   path: logs-*.zip
2024-06-26T07:51:01.6996435Z   name: artifact
2024-06-26T07:51:01.6996736Z   region: us-east-1
2024-06-26T07:51:01.6997038Z env:
2024-06-26T07:51:01.6997327Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:51:01.6997781Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:51:01.6998520Z   DOCKER_CONTAINER_ID: e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:01.6999168Z ##[endgroup]
2024-06-26T07:51:02.0005855Z NOTE: s3-prefix specified, ignoring name parameter
2024-06-26T07:51:02.0006644Z With the provided path, there will be 1 file uploaded
2024-06-26T07:51:02.0007266Z Uploading to s3 prefix: pytorch/pytorch/9673646806/1/artifact
2024-06-26T07:51:02.0033135Z Starting upload of logs-test-dynamic_aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688917313.zip
2024-06-26T07:51:02.1455005Z Finished upload of logs-test-dynamic_aot_eager_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688917313.zip
2024-06-26T07:51:02.1583076Z ##[group]Run seemethere/upload-artifact-s3@v5
2024-06-26T07:51:02.1583509Z with:
2024-06-26T07:51:02.1583792Z   s3-bucket: gha-artifacts
2024-06-26T07:51:02.1584213Z   s3-prefix: pytorch/pytorch/9673646806/1/artifact

2024-06-26T07:51:02.1584690Z   retention-days: 14
2024-06-26T07:51:02.1585020Z   if-no-files-found: ignore
2024-06-26T07:51:02.1585401Z   path: debug-*.zip
2024-06-26T07:51:02.1585744Z   name: artifact
2024-06-26T07:51:02.1586040Z   region: us-east-1
2024-06-26T07:51:02.1586334Z env:
2024-06-26T07:51:02.1586607Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:51:02.1587056Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:51:02.1587786Z   DOCKER_CONTAINER_ID: e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:02.1588422Z ##[endgroup]
2024-06-26T07:51:02.4527955Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded.
2024-06-26T07:51:02.4654589Z ##[group]Run # shellcheck disable=SC2156
2024-06-26T07:51:02.4655064Z # shellcheck disable=SC2156
2024-06-26T07:51:02.4655903Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \;
2024-06-26T07:51:02.4663914Z shell: /usr/bin/bash -e {0}
2024-06-26T07:51:02.4664262Z env:
2024-06-26T07:51:02.4664537Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:51:02.4664994Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:51:02.4665730Z   DOCKER_CONTAINER_ID: e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:02.4666368Z ##[endgroup]
2024-06-26T07:51:02.6275376Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main
2024-06-26T07:51:02.6275946Z with:
2024-06-26T07:51:02.6276231Z env:
2024-06-26T07:51:02.6276500Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:51:02.6277079Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:51:02.6277812Z   DOCKER_CONTAINER_ID: e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:02.6278442Z ##[endgroup]
2024-06-26T07:51:02.6295768Z ##[group]Run set -eou pipefail
2024-06-26T07:51:02.6296178Z set -eou pipefail
2024-06-26T07:51:02.6296529Z 
2024-06-26T07:51:02.6297038Z echo "Holding runner for 2 hours until all ssh sessions have logged out"
2024-06-26T07:51:02.6297664Z for _ in $(seq 1440); do
2024-06-26T07:51:02.6298143Z     # Break if no ssh session exists anymore
2024-06-26T07:51:02.6298640Z     if [ "$(who)" = "" ]; then
2024-06-26T07:51:02.6299047Z       break
2024-06-26T07:51:02.6299385Z     fi
2024-06-26T07:51:02.6299680Z     echo "."
2024-06-26T07:51:02.6300009Z     sleep 5
2024-06-26T07:51:02.6300321Z done
2024-06-26T07:51:02.6307685Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:51:02.6308189Z env:
2024-06-26T07:51:02.6308464Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:51:02.6308916Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:51:02.6309650Z   DOCKER_CONTAINER_ID: e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:02.6310288Z ##[endgroup]
2024-06-26T07:51:02.6329775Z Holding runner for 2 hours until all ssh sessions have logged out
2024-06-26T07:51:02.6364210Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty
2024-06-26T07:51:02.6365257Z # ignore expansion of "docker ps -q" since it could be empty
2024-06-26T07:51:02.6366030Z # shellcheck disable=SC2046
2024-06-26T07:51:02.6366598Z docker stop $(docker ps -q) || true
2024-06-26T07:51:02.6367074Z # Prune all of the docker images
2024-06-26T07:51:02.6367531Z docker system prune -af
2024-06-26T07:51:02.6374406Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:51:02.6374908Z env:
2024-06-26T07:51:02.6375180Z   GIT_DEFAULT_BRANCH: main
2024-06-26T07:51:02.6375633Z   GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all
2024-06-26T07:51:02.6376371Z   DOCKER_CONTAINER_ID: e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:02.6393333Z ##[endgroup]
2024-06-26T07:51:02.9107878Z e827c3f8b855
2024-06-26T07:51:03.6600468Z Deleted Containers:
2024-06-26T07:51:03.6601033Z e827c3f8b8551cfad298e46b4ae1dc6ce0fd3092b98c1601d0c09c15312a8106
2024-06-26T07:51:03.6601479Z 
2024-06-26T07:51:08.6497881Z Deleted Images:
2024-06-26T07:51:08.6499739Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7
2024-06-26T07:51:08.6502814Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-inductor-benchmarks@sha256:216c88a4f79ebfa71a5dcf4d9a946548184354a3ea1f677f4b1065d28705debb
2024-06-26T07:51:08.6504820Z deleted: sha256:0058b15e517a024817bccd921dc9af264c964f500b3cb9b3f4649a8ac22cc2dd
2024-06-26T07:51:08.6505696Z deleted: sha256:754ead7012f0b7691885a9660f02e80be6cee8c5665486fb995c1fb06adb6e92
2024-06-26T07:51:08.6506527Z deleted: sha256:fbe4496d8830e11be86d8f222522e7956f32f85bfc06b05ea53961f34a58af74
2024-06-26T07:51:08.6507360Z deleted: sha256:1c83814aa1a1a7e0cc998e3479bdab8e8766b975560cbf00690acf39ed899a80
2024-06-26T07:51:08.6508193Z deleted: sha256:e9ea8e61942c7f8ae4aab97b8d102b01d6bb4b6cf2a605808cd34130a17d8a0f
2024-06-26T07:51:08.6509024Z deleted: sha256:acc5fc32588cd95e082a6ce35e48d0f303ae8f96e3ab65c4e375b85e1f3c4a92
2024-06-26T07:51:08.6509847Z deleted: sha256:a93e53400904d29e4185d3be942ea34f19572ea438962788ae4e9b3d9bf4c498
2024-06-26T07:51:08.6510905Z deleted: sha256:c97092b88b5e4c77c68bbc4b51afab64e38719af1ff58013ec49a696cdd40a6e
2024-06-26T07:51:08.6511730Z deleted: sha256:b132889c90d67ffa53655846603e3fcc2372c23f04dbdbc54690a39d304eb89b
2024-06-26T07:51:08.6512638Z deleted: sha256:915d54081ed607710624abe3f2490114c745858831f4fc13396bd3b4b9a121bc
2024-06-26T07:51:08.6513442Z deleted: sha256:bfa8d446c498c87f535aef102695564de1b22c755e6c63944ae2f5073a2b31ec
2024-06-26T07:51:08.6514263Z deleted: sha256:e71460fb363c29e70bbe357d680aaa7ae694f094ef2f2713609158ad90611744
2024-06-26T07:51:08.6515074Z deleted: sha256:84079bb1b4ee1d0119ae3d3e701de460a1324912297c675425c9a36d30fcdb0f
2024-06-26T07:51:08.6516078Z deleted: sha256:d86ac3ca148d4758cb03b493fdd49109d8f1a2c7c85c4b755493c3c4529553e5
2024-06-26T07:51:08.6516986Z deleted: sha256:532b3ab97c44a673a0d5f62ee46fa6a83248a84de8877754a612355be56ed0a3
2024-06-26T07:51:08.6517803Z deleted: sha256:617e2a35f25ae24500c88c0c9b8475f0687fbcbed3d200013b4c99c6d8941aff
2024-06-26T07:51:08.6518633Z deleted: sha256:b88120bcd5a26a660c09c5df13176e0ef098fb2c8cdb57d74ab0796a3d1194d6
2024-06-26T07:51:08.6519445Z deleted: sha256:c016b2e7764735243205fb76fa9a273bb3dd4bf05532288e3a613ce022131ac3
2024-06-26T07:51:08.6520257Z deleted: sha256:bcaca462c390deddd2995de2d245a7495e7d2e1e94f0edad30a573c83d0588fd
2024-06-26T07:51:08.6521117Z deleted: sha256:9e260eba003d5c25d7b410e5c4c3093845d9e46611896840901c689d7fbf8ee6
2024-06-26T07:51:08.6521917Z deleted: sha256:3e62b86f8b911a9b59c62d55f063e71683ae6a01c22418db5d27930712adaad6
2024-06-26T07:51:08.6522723Z deleted: sha256:905920ccb1d88449b8dde686c34a4b2a8b6f59e78ac44916b189ea3593b9fbbb
2024-06-26T07:51:08.6523525Z deleted: sha256:961201237e03acd5061406238dd7a3c1343ac888d7583212cd0ab5ae1b4f0540
2024-06-26T07:51:08.6524337Z deleted: sha256:70047a2aac62159fa28fb4dbd4567ec2ffbab68fe046967e09acb0eb3e5cb9cb
2024-06-26T07:51:08.6525651Z deleted: sha256:1ec680ad781436ae65fa987f5fa1ed032ecd5c82a6e35eea6707de501f62c0f4
2024-06-26T07:51:08.6526678Z deleted: sha256:ef859623a50e85abf6e6c1dbd6188aa765a66cc7b2bd618a1cbe4825b7b2bd91
2024-06-26T07:51:08.6527725Z deleted: sha256:404ade5cacff59773b93ab438bfa64cd3a171d07f6f168bdc4a5d4bfa0b43526
2024-06-26T07:51:08.6528779Z deleted: sha256:5b27de6ee3d9f2f18217bf18dec725f7b4954c0e48c85d35752634c22d62209b
2024-06-26T07:51:08.6529621Z deleted: sha256:498e39c575b3fa069acaac7e46b91b4e54ae3a05f2ada5c4347091f346f25d4c
2024-06-26T07:51:08.6530429Z deleted: sha256:1405919d2694d65669b1f4c842282021afe89208ce86570e96464903733da2d0
2024-06-26T07:51:08.6531337Z deleted: sha256:d5b41e5d70896c0472fc3e6da0516a17d89cd6635dc75010ecddc31ce9690fad
2024-06-26T07:51:08.6532154Z deleted: sha256:18e9379b62b9d0bb40293d28a652eafaee66a499f66f0776bf14e37a513101b7
2024-06-26T07:51:08.6532978Z deleted: sha256:a781f80ed00ff2926a699fd01a309bb0b4fb7d5655eb652c205bb9c7f811035f
2024-06-26T07:51:08.6533798Z deleted: sha256:c0430973b7cf759075330ee29c79bfd1312c91e6033d9dd883b4b1c29743a171
2024-06-26T07:51:08.6534618Z deleted: sha256:e733b3d66efb9342e5a283376c0e1ddffa47a42fecc26e61ecaf0a3472885d97
2024-06-26T07:51:08.6535446Z deleted: sha256:8929c0b2047f74b84c82c39eff9077a92ae1a828cde00c9ab78fb3e0ed5e0283
2024-06-26T07:51:08.6536262Z deleted: sha256:f5b56155fb58a0a60fe6391d3581cf91610ed69d4f0a321257c19944da296a52
2024-06-26T07:51:08.6537075Z deleted: sha256:b54d2cb355238d1aa91e58df93064447b80f53706d1ca62823acc834f5ac54fe
2024-06-26T07:51:08.6537886Z deleted: sha256:e7311285af9eeddb51aa5629aa98386e89f1456ff32843b6aee6443247edf66c
2024-06-26T07:51:08.6538695Z deleted: sha256:67657ceec347da8dc8085d5731633efeac115ba677fa4b7c5966c4260f7abab0
2024-06-26T07:51:08.6539518Z deleted: sha256:b496e50409dbf414930fc9b4437aa0aa4ce27bc7840e050ae459aaffcf9aca86
2024-06-26T07:51:08.6540386Z deleted: sha256:8f5276170cc4a60d36ad7b964a502306b29546254cedcbcb101ea9123497d2c0
2024-06-26T07:51:08.6541191Z deleted: sha256:53d524d1564745e96a7e515462171b73184d9f1ff5454b32fcfbbca39d361467
2024-06-26T07:51:08.6542006Z deleted: sha256:2acbadb00d69b6aca575f04baba4ea26ae2b5c2d85f09945004ad93a78a8eb75
2024-06-26T07:51:08.6542971Z deleted: sha256:02665efeff878aeb603c1edb2e21cb700c171c025d64f5aae1b2b20b12f3964a
2024-06-26T07:51:08.6543815Z deleted: sha256:19bd23e9a51dfe883a8b8d2ff6d66ff1b3476b8912856406bb79b81762aa6d4b
2024-06-26T07:51:08.6544638Z deleted: sha256:73aa8637bf9299d6298dabd858480c274a62117388f2ca1d5e9eb15c6d1e4155
2024-06-26T07:51:08.6545518Z deleted: sha256:5f8968406305f348b1caebc0e035187d6e9965ace7b9d5f2f21114d37c9fe2e1
2024-06-26T07:51:08.6546333Z deleted: sha256:d827104914db0c503cf5c81f13e2456bf4efce42d4cf1f8d5d70838207278bee
2024-06-26T07:51:08.6547158Z deleted: sha256:867b219724f2f1aaf56f42e3e9f03a5d64bdca8a6a72407efd10328aeea5d510
2024-06-26T07:51:08.6547990Z deleted: sha256:b9ff2a7cceb9237b26bcb17728edf8a558eec5a76f63626e84f80541dfc0f1d4
2024-06-26T07:51:08.6548814Z deleted: sha256:be73f509106819007ce2b0f5a1301ac56ae583cb6c1fb69bee93efa0840368f1
2024-06-26T07:51:08.6549642Z deleted: sha256:2a11c07c915c3fdbb4e73096dea73ea950c970d50fe1a5764060bd62b7b7cf2c
2024-06-26T07:51:08.6550470Z deleted: sha256:5e9989622e2abc1c72f8184343507b45d75e4ff971edb68b96cbcd75ae484ff5
2024-06-26T07:51:08.6551303Z deleted: sha256:05ce642bd9cf5a9205bff107df28b1d6cef7d5f6ab3ea944d28c032ec61270df
2024-06-26T07:51:08.6552120Z deleted: sha256:017702617f6c07b73f2088678f4710b794cfe897a213a5cf122a0e2519eae0ac
2024-06-26T07:51:08.6552927Z deleted: sha256:44b3b37bb1ca39dc5cf356de41a23a5c12f605a1992813e408087fb2b0ad6d1c
2024-06-26T07:51:08.6553749Z deleted: sha256:8c387982e9f27db357a1d149dc149cabc0bd04668ad6956387a5072682dada67
2024-06-26T07:51:08.6554572Z deleted: sha256:10d415dca4e3fbb88a747b40317f3c6333c041ba8f73855f67fb9fe981fae5ee
2024-06-26T07:51:08.6555390Z deleted: sha256:6c65469bf0f53dcd625faefe363b8f3136c08388a6cf64d9158e8576e4ee89b1
2024-06-26T07:51:08.6556219Z deleted: sha256:027db5d8b76deeefbd8c34b485ab7ff14ed2a3e6800425c1ce0d77c2e9fdd4ab
2024-06-26T07:51:08.6557045Z deleted: sha256:24941234ad28cf5c8174d7cf18326c10b4e17f2c2137672ea0322df461597dd7
2024-06-26T07:51:08.6557869Z deleted: sha256:240ea2eebfa8519d6c2d88f4c99e66ce4e038d000f93ce8d03726c01d81180ef
2024-06-26T07:51:08.6558695Z deleted: sha256:09e823cb391a69f6a195e3ca8a41624b5f8b34eee14c6eb36277de814a91966e
2024-06-26T07:51:08.6559493Z deleted: sha256:730479d695169a6593bf0ac16b377491dce65175a6d158e012422367d04577fd
2024-06-26T07:51:08.6560310Z deleted: sha256:b0a30f9f7bd5459ff662453b9346a7ba8ea8db1191528345c1953eab4aa2f54e
2024-06-26T07:51:08.6561132Z deleted: sha256:75cefa87effaa3ad361c74642839923101f0f9982f814f3c9e2ed4ad52f2a008
2024-06-26T07:51:08.6561946Z deleted: sha256:6f97db87ba9512e615e871c6982000602982e1e5a5be334a56e7292251e84e00
2024-06-26T07:51:08.6562755Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8
2024-06-26T07:51:08.6563576Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00
2024-06-26T07:51:08.6564384Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f
2024-06-26T07:51:08.6565391Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64
2024-06-26T07:51:08.6566210Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88
2024-06-26T07:51:08.6567019Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d
2024-06-26T07:51:08.6567852Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25
2024-06-26T07:51:08.6568676Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9
2024-06-26T07:51:08.6569507Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29
2024-06-26T07:51:08.6570333Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037
2024-06-26T07:51:08.6571234Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9
2024-06-26T07:51:08.6571726Z 
2024-06-26T07:51:08.6571867Z Total reclaimed space: 36.35GB
2024-06-26T07:51:08.6633995Z Post job cleanup.
2024-06-26T07:51:08.6681358Z Post job cleanup.
2024-06-26T07:51:08.7485689Z [command]/usr/bin/git version
2024-06-26T07:51:08.7519807Z git version 2.40.1
2024-06-26T07:51:08.7557244Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/904bbd55-d947-4c30-8834-10ebc5c49f78' before making global git config changes
2024-06-26T07:51:08.7558662Z Adding repository directory to the temporary git global config as a safe directory
2024-06-26T07:51:08.7562913Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch
2024-06-26T07:51:08.7590047Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2024-06-26T07:51:08.7612702Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2024-06-26T07:51:08.7822854Z Entering 'android/libs/fbjni'
2024-06-26T07:51:08.7859078Z Entering 'third_party/FP16'
2024-06-26T07:51:08.7895804Z Entering 'third_party/FXdiv'
2024-06-26T07:51:08.7931983Z Entering 'third_party/NNPACK'
2024-06-26T07:51:08.7968454Z Entering 'third_party/VulkanMemoryAllocator'
2024-06-26T07:51:08.8006211Z Entering 'third_party/XNNPACK'
2024-06-26T07:51:08.8053567Z Entering 'third_party/benchmark'
2024-06-26T07:51:08.8088790Z Entering 'third_party/cpp-httplib'
2024-06-26T07:51:08.8122935Z Entering 'third_party/cpuinfo'
2024-06-26T07:51:08.8161010Z Entering 'third_party/cudnn_frontend'
2024-06-26T07:51:08.8197327Z Entering 'third_party/cutlass'
2024-06-26T07:51:08.8240106Z Entering 'third_party/eigen'
2024-06-26T07:51:08.8274759Z Entering 'third_party/fbgemm'
2024-06-26T07:51:08.8309996Z Entering 'third_party/fbgemm/third_party/asmjit'
2024-06-26T07:51:08.8345022Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2024-06-26T07:51:08.8379585Z Entering 'third_party/fbgemm/third_party/cutlass'
2024-06-26T07:51:08.8418413Z Entering 'third_party/fbgemm/third_party/googletest'
2024-06-26T07:51:08.8452769Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2024-06-26T07:51:08.8488126Z Entering 'third_party/flatbuffers'
2024-06-26T07:51:08.8525865Z Entering 'third_party/fmt'
2024-06-26T07:51:08.8560631Z Entering 'third_party/foxi'
2024-06-26T07:51:08.8594059Z Entering 'third_party/gemmlowp/gemmlowp'
2024-06-26T07:51:08.8629504Z Entering 'third_party/gloo'
2024-06-26T07:51:08.8664607Z Entering 'third_party/googletest'
2024-06-26T07:51:08.8699685Z Entering 'third_party/ideep'
2024-06-26T07:51:08.8734381Z Entering 'third_party/ideep/mkl-dnn'
2024-06-26T07:51:08.8773974Z Entering 'third_party/ittapi'
2024-06-26T07:51:08.8810993Z Entering 'third_party/kineto'
2024-06-26T07:51:08.8843691Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2024-06-26T07:51:08.8877411Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2024-06-26T07:51:08.8913965Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2024-06-26T07:51:08.8949498Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2024-06-26T07:51:08.8983455Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2024-06-26T07:51:08.9017323Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2024-06-26T07:51:08.9053033Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2024-06-26T07:51:08.9085734Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2024-06-26T07:51:08.9120163Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2024-06-26T07:51:08.9154771Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2024-06-26T07:51:08.9191327Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2024-06-26T07:51:08.9225088Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2024-06-26T07:51:08.9260431Z Entering 'third_party/mimalloc'
2024-06-26T07:51:08.9295388Z Entering 'third_party/nccl/nccl'
2024-06-26T07:51:08.9332173Z Entering 'third_party/nlohmann'
2024-06-26T07:51:08.9367173Z Entering 'third_party/onnx'
2024-06-26T07:51:08.9414300Z Entering 'third_party/onnx/third_party/benchmark'
2024-06-26T07:51:08.9451685Z Entering 'third_party/onnx/third_party/pybind11'
2024-06-26T07:51:08.9488181Z Entering 'third_party/opentelemetry-cpp'
2024-06-26T07:51:08.9522108Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2024-06-26T07:51:08.9556396Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2024-06-26T07:51:08.9591968Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2024-06-26T07:51:08.9625390Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2024-06-26T07:51:08.9660760Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2024-06-26T07:51:08.9696376Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2024-06-26T07:51:08.9730112Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2024-06-26T07:51:08.9763448Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2024-06-26T07:51:08.9799215Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2024-06-26T07:51:08.9835880Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2024-06-26T07:51:08.9883420Z Entering 'third_party/pocketfft'
2024-06-26T07:51:08.9919079Z Entering 'third_party/protobuf'
2024-06-26T07:51:08.9956486Z Entering 'third_party/protobuf/third_party/benchmark'
2024-06-26T07:51:08.9991863Z Entering 'third_party/protobuf/third_party/googletest'
2024-06-26T07:51:09.0026438Z Entering 'third_party/psimd'
2024-06-26T07:51:09.0062188Z Entering 'third_party/pthreadpool'
2024-06-26T07:51:09.0096824Z Entering 'third_party/pybind11'
2024-06-26T07:51:09.0130929Z Entering 'third_party/python-peachpy'
2024-06-26T07:51:09.0166459Z Entering 'third_party/sleef'
2024-06-26T07:51:09.0202077Z Entering 'third_party/tensorpipe'
2024-06-26T07:51:09.0236774Z Entering 'third_party/tensorpipe/third_party/googletest'
2024-06-26T07:51:09.0272509Z Entering 'third_party/tensorpipe/third_party/libnop'
2024-06-26T07:51:09.0304887Z Entering 'third_party/tensorpipe/third_party/libuv'
2024-06-26T07:51:09.0339877Z Entering 'third_party/tensorpipe/third_party/pybind11'
2024-06-26T07:51:09.0372144Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2024-06-26T07:51:09.0423558Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2024-06-26T07:51:09.0440889Z http.https://github.com/.extraheader
2024-06-26T07:51:09.0448789Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
2024-06-26T07:51:09.0476528Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2024-06-26T07:51:09.0681813Z Entering 'android/libs/fbjni'
2024-06-26T07:51:09.0704750Z http.https://github.com/.extraheader
2024-06-26T07:51:09.0728556Z Entering 'third_party/FP16'
2024-06-26T07:51:09.0751018Z http.https://github.com/.extraheader
2024-06-26T07:51:09.0774623Z Entering 'third_party/FXdiv'
2024-06-26T07:51:09.0796136Z http.https://github.com/.extraheader
2024-06-26T07:51:09.0819474Z Entering 'third_party/NNPACK'
2024-06-26T07:51:09.0840545Z http.https://github.com/.extraheader
2024-06-26T07:51:09.0863995Z Entering 'third_party/VulkanMemoryAllocator'
2024-06-26T07:51:09.0885711Z http.https://github.com/.extraheader
2024-06-26T07:51:09.0908591Z Entering 'third_party/XNNPACK'
2024-06-26T07:51:09.0930363Z http.https://github.com/.extraheader
2024-06-26T07:51:09.0965554Z Entering 'third_party/benchmark'
2024-06-26T07:51:09.0988546Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1012122Z Entering 'third_party/cpp-httplib'
2024-06-26T07:51:09.1033968Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1057538Z Entering 'third_party/cpuinfo'
2024-06-26T07:51:09.1079816Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1105121Z Entering 'third_party/cudnn_frontend'
2024-06-26T07:51:09.1128004Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1151637Z Entering 'third_party/cutlass'
2024-06-26T07:51:09.1171968Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1201415Z Entering 'third_party/eigen'
2024-06-26T07:51:09.1223617Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1249187Z Entering 'third_party/fbgemm'
2024-06-26T07:51:09.1270857Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1294580Z Entering 'third_party/fbgemm/third_party/asmjit'
2024-06-26T07:51:09.1315153Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1338029Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2024-06-26T07:51:09.1358769Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1383152Z Entering 'third_party/fbgemm/third_party/cutlass'
2024-06-26T07:51:09.1403944Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1430965Z Entering 'third_party/fbgemm/third_party/googletest'
2024-06-26T07:51:09.1454096Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1477278Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2024-06-26T07:51:09.1499405Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1523698Z Entering 'third_party/flatbuffers'
2024-06-26T07:51:09.1545029Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1569877Z Entering 'third_party/fmt'
2024-06-26T07:51:09.1592254Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1617550Z Entering 'third_party/foxi'
2024-06-26T07:51:09.1638874Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1660788Z Entering 'third_party/gemmlowp/gemmlowp'
2024-06-26T07:51:09.1681616Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1705291Z Entering 'third_party/gloo'
2024-06-26T07:51:09.1727134Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1750868Z Entering 'third_party/googletest'
2024-06-26T07:51:09.1772900Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1795690Z Entering 'third_party/ideep'
2024-06-26T07:51:09.1817190Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1839743Z Entering 'third_party/ideep/mkl-dnn'
2024-06-26T07:51:09.1862031Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1889781Z Entering 'third_party/ittapi'
2024-06-26T07:51:09.1911759Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1934841Z Entering 'third_party/kineto'
2024-06-26T07:51:09.1956119Z http.https://github.com/.extraheader
2024-06-26T07:51:09.1979628Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2024-06-26T07:51:09.2001690Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2024939Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2024-06-26T07:51:09.2045672Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2070468Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2024-06-26T07:51:09.2093303Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2115928Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2024-06-26T07:51:09.2137591Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2160842Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2024-06-26T07:51:09.2181566Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2201914Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2024-06-26T07:51:09.2224303Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2247959Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2024-06-26T07:51:09.2270968Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2294829Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2024-06-26T07:51:09.2316004Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2340666Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2024-06-26T07:51:09.2361581Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2385739Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2024-06-26T07:51:09.2408127Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2433652Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2024-06-26T07:51:09.2456148Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2477366Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2024-06-26T07:51:09.2498908Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2522362Z Entering 'third_party/mimalloc'
2024-06-26T07:51:09.2543116Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2563440Z Entering 'third_party/nccl/nccl'
2024-06-26T07:51:09.2584592Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2606884Z Entering 'third_party/nlohmann'
2024-06-26T07:51:09.2628080Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2651883Z Entering 'third_party/onnx'
2024-06-26T07:51:09.2673515Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2707583Z Entering 'third_party/onnx/third_party/benchmark'
2024-06-26T07:51:09.2729227Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2751780Z Entering 'third_party/onnx/third_party/pybind11'
2024-06-26T07:51:09.2773225Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2798749Z Entering 'third_party/opentelemetry-cpp'
2024-06-26T07:51:09.2822130Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2846304Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2024-06-26T07:51:09.2867477Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2893030Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2024-06-26T07:51:09.2913980Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2935942Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2024-06-26T07:51:09.2956847Z http.https://github.com/.extraheader
2024-06-26T07:51:09.2979891Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2024-06-26T07:51:09.3000862Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3024268Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2024-06-26T07:51:09.3044189Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3066165Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2024-06-26T07:51:09.3087660Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3110725Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2024-06-26T07:51:09.3131952Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3153363Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2024-06-26T07:51:09.3173821Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3197776Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2024-06-26T07:51:09.3219341Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3242306Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2024-06-26T07:51:09.3263721Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3302104Z Entering 'third_party/pocketfft'
2024-06-26T07:51:09.3324468Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3348645Z Entering 'third_party/protobuf'
2024-06-26T07:51:09.3370023Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3396914Z Entering 'third_party/protobuf/third_party/benchmark'
2024-06-26T07:51:09.3419028Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3442589Z Entering 'third_party/protobuf/third_party/googletest'
2024-06-26T07:51:09.3466607Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3492299Z Entering 'third_party/psimd'
2024-06-26T07:51:09.3514266Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3535840Z Entering 'third_party/pthreadpool'
2024-06-26T07:51:09.3556897Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3580878Z Entering 'third_party/pybind11'
2024-06-26T07:51:09.3602584Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3625155Z Entering 'third_party/python-peachpy'
2024-06-26T07:51:09.3647727Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3671055Z Entering 'third_party/sleef'
2024-06-26T07:51:09.3693304Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3716219Z Entering 'third_party/tensorpipe'
2024-06-26T07:51:09.3737816Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3760773Z Entering 'third_party/tensorpipe/third_party/googletest'
2024-06-26T07:51:09.3783282Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3806467Z Entering 'third_party/tensorpipe/third_party/libnop'
2024-06-26T07:51:09.3827595Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3849572Z Entering 'third_party/tensorpipe/third_party/libuv'
2024-06-26T07:51:09.3871105Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3894876Z Entering 'third_party/tensorpipe/third_party/pybind11'
2024-06-26T07:51:09.3915730Z http.https://github.com/.extraheader
2024-06-26T07:51:09.3938182Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2024-06-26T07:51:09.3959455Z http.https://github.com/.extraheader
2024-06-26T07:51:09.4058573Z A job completed hook has been configured by the self-hosted runner administrator
2024-06-26T07:51:09.4075736Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh'
2024-06-26T07:51:09.4082345Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2024-06-26T07:51:09.4082860Z ##[endgroup]
2024-06-26T07:51:10.2888454Z Cleaning up orphan processes