2024-06-26T07:05:01.8977121Z Current runner version: '2.317.0' 2024-06-26T07:05:01.8983165Z Runner name: 'i-044844c31b6a1c4ce' 2024-06-26T07:05:01.8983903Z Runner group name: 'Default' 2024-06-26T07:05:01.8984822Z Machine name: 'ip-10-0-49-197' 2024-06-26T07:05:01.8989552Z ##[group]GITHUB_TOKEN Permissions 2024-06-26T07:05:01.8991483Z Actions: read 2024-06-26T07:05:01.8991950Z Attestations: read 2024-06-26T07:05:01.8992523Z Checks: read 2024-06-26T07:05:01.8992968Z Contents: read 2024-06-26T07:05:01.8993421Z Deployments: read 2024-06-26T07:05:01.8993921Z Discussions: read 2024-06-26T07:05:01.8994405Z Issues: read 2024-06-26T07:05:01.8994909Z Metadata: read 2024-06-26T07:05:01.8995370Z Packages: read 2024-06-26T07:05:01.8995827Z Pages: read 2024-06-26T07:05:01.8996277Z PullRequests: read 2024-06-26T07:05:01.8996787Z RepositoryProjects: read 2024-06-26T07:05:01.8997350Z SecurityEvents: read 2024-06-26T07:05:01.8997898Z Statuses: read 2024-06-26T07:05:01.8998378Z ##[endgroup] 2024-06-26T07:05:01.9001508Z Secret source: Actions 2024-06-26T07:05:01.9002293Z Prepare workflow directory 2024-06-26T07:05:02.2556702Z Prepare all required actions 2024-06-26T07:05:02.2756016Z Getting action download info 2024-06-26T07:05:02.4315396Z Download action repository 'pytorch/test-infra@main' (SHA:43a2ce341cc31288e9a38b65ce600a7f43021bd5) 2024-06-26T07:05:02.7465782Z Download action repository 'pytorch/pytorch@main' (SHA:c422a9549d2b320b47a9391f6cbcf592b2876c88) 2024-06-26T07:05:05.5095854Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-06-26T07:05:05.6523756Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-06-26T07:05:05.9025910Z Getting action download info 2024-06-26T07:05:06.0094858Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-06-26T07:05:06.1840462Z Getting action download info 2024-06-26T07:05:06.2801643Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-06-26T07:05:06.4235586Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/tags/ciflow/inductor/129470 (b8c4c54d347aa776934c60784e35936878ef18dc) 2024-06-26T07:05:06.4237731Z ##[group] Inputs 2024-06-26T07:05:06.4238198Z build-environment: linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-06-26T07:05:06.4239474Z test-matrix: {"include": [{"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-06-26T07:05:06.4241471Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:05:06.4242752Z sync-tag: linux-focal-cuda12_4-py3_10-gcc9-inductor-test 2024-06-26T07:05:06.4243730Z timeout-minutes: 240 2024-06-26T07:05:06.4244083Z use-gha: 2024-06-26T07:05:06.4244382Z dashboard-tag: 2024-06-26T07:05:06.4244699Z s3-bucket: gha-artifacts 2024-06-26T07:05:06.4245044Z aws-role-to-assume: 2024-06-26T07:05:06.4245369Z ##[endgroup] 2024-06-26T07:05:06.4246188Z Complete job name: cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:05:06.4803936Z A job started hook has been configured by the self-hosted runner administrator 2024-06-26T07:05:06.4943279Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T07:05:06.4953948Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:05:06.4954487Z ##[endgroup] 2024-06-26T07:05:07.3668487Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-06-26T07:05:07.3669041Z with: 2024-06-26T07:05:07.3669646Z github-secret: *** 2024-06-26T07:05:07.3670571Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-06-26T07:05:07.3671699Z activate-with-label: false 2024-06-26T07:05:07.3672052Z label: with-ssh 2024-06-26T07:05:07.3672383Z remove-existing-keys: true 2024-06-26T07:05:07.3672775Z fail-silently: true 2024-06-26T07:05:07.3673076Z env: 2024-06-26T07:05:07.3673349Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:07.3673689Z ##[endgroup] 2024-06-26T07:05:07.4543871Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-06-26T07:05:07.4544921Z ciflow reference detected, attempting to extract PR number 2024-06-26T07:05:07.7619474Z Grabbing public ssh keys from https://github.com/pytorch-bot[bot].keys 2024-06-26T07:05:07.8345601Z No SSH keys found for user pytorch-bot[bot] 2024-06-26T07:05:07.8346397Z Grabbing public ssh keys from https://github.com/leslie-fang-intel.keys 2024-06-26T07:05:07.9034982Z No SSH keys found for user leslie-fang-intel 2024-06-26T07:05:07.9142336Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-06-26T07:05:07.9142965Z with: 2024-06-26T07:05:07.9143335Z submodules: recursive 2024-06-26T07:05:07.9143707Z fetch-depth: 0 2024-06-26T07:05:07.9144044Z env: 2024-06-26T07:05:07.9144397Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:07.9144783Z ##[endgroup] 2024-06-26T07:05:07.9371990Z ##[group]Run retry () { 2024-06-26T07:05:07.9372396Z retry () { 2024-06-26T07:05:07.9372911Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-06-26T07:05:07.9373467Z } 2024-06-26T07:05:07.9373778Z echo "${GITHUB_WORKSPACE}" 2024-06-26T07:05:07.9374191Z if [ -z "${NO_SUDO}" ]; then 2024-06-26T07:05:07.9374666Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T07:05:07.9375121Z else 2024-06-26T07:05:07.9375463Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T07:05:07.9375882Z fi 2024-06-26T07:05:07.9376387Z mkdir "${GITHUB_WORKSPACE}" 2024-06-26T07:05:07.9384446Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:05:07.9384959Z env: 2024-06-26T07:05:07.9385244Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:07.9385665Z NO_SUDO: 2024-06-26T07:05:07.9385939Z ##[endgroup] 2024-06-26T07:05:07.9408634Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:05:10.1659355Z ##[group]Run malfet/checkout@silent-checkout 2024-06-26T07:05:10.1659834Z with: 2024-06-26T07:05:10.1660150Z ref: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:05:10.1660586Z fetch-depth: 0 2024-06-26T07:05:10.1660895Z submodules: recursive 2024-06-26T07:05:10.1661235Z quiet-checkout: true 2024-06-26T07:05:10.1661573Z repository: pytorch/pytorch 2024-06-26T07:05:10.1662057Z token: *** 2024-06-26T07:05:10.1662345Z ssh-strict: true 2024-06-26T07:05:10.1662667Z persist-credentials: true 2024-06-26T07:05:10.1663015Z clean: true 2024-06-26T07:05:10.1663348Z sparse-checkout-cone-mode: true 2024-06-26T07:05:10.1663730Z lfs: false 2024-06-26T07:05:10.1664024Z set-safe-directory: true 2024-06-26T07:05:10.1664371Z env: 2024-06-26T07:05:10.1664639Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:05:10.1664979Z ##[endgroup] 2024-06-26T07:05:10.2602375Z Syncing repository: pytorch/pytorch 2024-06-26T07:05:10.2603984Z ##[group]Getting Git version info 2024-06-26T07:05:10.2604764Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T07:05:10.2605619Z [command]/usr/bin/git version 2024-06-26T07:05:10.2605984Z git version 2.40.1 2024-06-26T07:05:10.2607332Z ##[endgroup] 2024-06-26T07:05:10.2618934Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/9a4dc42e-ca00-4f2f-b035-3cfd79bc1a3d/.gitconfig' 2024-06-26T07:05:10.2620552Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/9a4dc42e-ca00-4f2f-b035-3cfd79bc1a3d' before making global git config changes 2024-06-26T07:05:10.2621801Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T07:05:10.2622920Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:05:10.2650709Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T07:05:10.2654987Z ##[group]Initializing the repository 2024-06-26T07:05:10.2658068Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:05:10.2676726Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-06-26T07:05:10.2677718Z hint: is subject to change. To configure the initial branch name to use in all 2024-06-26T07:05:10.2678531Z hint: of your new repositories, which will suppress this warning, call: 2024-06-26T07:05:10.2679310Z hint: 2024-06-26T07:05:10.2679759Z hint: git config --global init.defaultBranch 2024-06-26T07:05:10.2680236Z hint: 2024-06-26T07:05:10.2680927Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-06-26T07:05:10.2681785Z hint: 'development'. The just-created branch can be renamed via this command: 2024-06-26T07:05:10.2682387Z hint: 2024-06-26T07:05:10.2682695Z hint: git branch -m 2024-06-26T07:05:10.2683427Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-06-26T07:05:10.2685755Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-06-26T07:05:10.2712455Z ##[endgroup] 2024-06-26T07:05:10.2713050Z ##[group]Disabling automatic garbage collection 2024-06-26T07:05:10.2715989Z [command]/usr/bin/git config --local gc.auto 0 2024-06-26T07:05:10.2736535Z ##[endgroup] 2024-06-26T07:05:10.2737068Z ##[group]Setting up auth 2024-06-26T07:05:10.2742115Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T07:05:10.2763264Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T07:05:10.2958596Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T07:05:10.2982609Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T07:05:10.3191143Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T07:05:10.3237802Z ##[endgroup] 2024-06-26T07:05:10.3238355Z ##[group]Fetching the repository 2024-06-26T07:05:10.3244076Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-06-26T07:05:12.7121384Z remote: Enumerating objects: 987619 2024-06-26T07:05:12.7122003Z remote: Enumerating objects: 990225, done. 2024-06-26T07:05:12.7124303Z remote: Counting objects: 0% (1/2606) 2024-06-26T07:05:12.7126452Z remote: Counting objects: 1% (27/2606) 2024-06-26T07:05:12.7126993Z remote: Counting objects: 2% (53/2606) 2024-06-26T07:05:12.7128858Z remote: Counting objects: 3% (79/2606) 2024-06-26T07:05:12.7129381Z remote: Counting objects: 4% (105/2606) 2024-06-26T07:05:12.7131182Z remote: Counting objects: 5% (131/2606) 2024-06-26T07:05:12.7131699Z remote: Counting objects: 6% (157/2606) 2024-06-26T07:05:12.7132912Z remote: Counting objects: 7% (183/2606) 2024-06-26T07:05:12.7133609Z remote: Counting objects: 8% (209/2606) 2024-06-26T07:05:12.7134592Z remote: Counting objects: 9% (235/2606) 2024-06-26T07:05:12.7136743Z remote: Counting objects: 10% (261/2606) 2024-06-26T07:05:12.7137503Z remote: Counting objects: 11% (287/2606) 2024-06-26T07:05:12.7138485Z remote: Counting objects: 12% (313/2606) 2024-06-26T07:05:12.7139006Z remote: Counting objects: 13% (339/2606) 2024-06-26T07:05:12.7140408Z remote: Counting objects: 14% (365/2606) 2024-06-26T07:05:12.7141505Z remote: Counting objects: 15% (391/2606) 2024-06-26T07:05:12.7142056Z remote: Counting objects: 16% (417/2606) 2024-06-26T07:05:12.7142559Z remote: Counting objects: 17% (444/2606) 2024-06-26T07:05:12.7143105Z remote: Counting objects: 18% (470/2606) 2024-06-26T07:05:12.7143867Z remote: Counting objects: 19% (496/2606) 2024-06-26T07:05:12.7144620Z remote: Counting objects: 20% (522/2606) 2024-06-26T07:05:12.7145487Z remote: Counting objects: 21% (548/2606) 2024-06-26T07:05:12.7146235Z remote: Counting objects: 22% (574/2606) 2024-06-26T07:05:12.7146980Z remote: Counting objects: 23% (600/2606) 2024-06-26T07:05:12.7147695Z remote: Counting objects: 24% (626/2606) 2024-06-26T07:05:12.7148188Z remote: Counting objects: 25% (652/2606) 2024-06-26T07:05:12.7149839Z remote: Counting objects: 26% (678/2606) 2024-06-26T07:05:12.7150510Z remote: Counting objects: 27% (704/2606) 2024-06-26T07:05:12.7151214Z remote: Counting objects: 28% (730/2606) 2024-06-26T07:05:12.7151709Z remote: Counting objects: 29% (756/2606) 2024-06-26T07:05:12.7152235Z remote: Counting objects: 30% (782/2606) 2024-06-26T07:05:12.7153132Z remote: Counting objects: 31% (808/2606) 2024-06-26T07:05:12.7153645Z remote: Counting objects: 32% (834/2606) 2024-06-26T07:05:12.7155409Z remote: Counting objects: 33% (860/2606) 2024-06-26T07:05:12.7155932Z remote: Counting objects: 34% (887/2606) 2024-06-26T07:05:12.7156444Z remote: Counting objects: 35% (913/2606) 2024-06-26T07:05:12.7157644Z remote: Counting objects: 36% (939/2606) 2024-06-26T07:05:12.7158222Z remote: Counting objects: 37% (965/2606) 2024-06-26T07:05:12.7158716Z remote: Counting objects: 38% (991/2606) 2024-06-26T07:05:12.7160554Z remote: Counting objects: 39% (1017/2606) 2024-06-26T07:05:12.7161093Z remote: Counting objects: 40% (1043/2606) 2024-06-26T07:05:12.7162164Z remote: Counting objects: 41% (1069/2606) 2024-06-26T07:05:12.7162839Z remote: Counting objects: 42% (1095/2606) 2024-06-26T07:05:12.7164964Z remote: Counting objects: 43% (1121/2606) 2024-06-26T07:05:12.7165494Z remote: Counting objects: 44% (1147/2606) 2024-06-26T07:05:12.7166141Z remote: Counting objects: 45% (1173/2606) 2024-06-26T07:05:12.7167139Z remote: Counting objects: 46% (1199/2606) 2024-06-26T07:05:12.7167765Z remote: Counting objects: 47% (1225/2606) 2024-06-26T07:05:12.7168441Z remote: Counting objects: 48% (1251/2606) 2024-06-26T07:05:12.7169045Z remote: Counting objects: 49% (1277/2606) 2024-06-26T07:05:12.7169571Z remote: Counting objects: 50% (1303/2606) 2024-06-26T07:05:12.7170246Z remote: Counting objects: 51% (1330/2606) 2024-06-26T07:05:12.7170870Z remote: Counting objects: 52% (1356/2606) 2024-06-26T07:05:12.7171394Z remote: Counting objects: 53% (1382/2606) 2024-06-26T07:05:12.7171913Z remote: Counting objects: 54% (1408/2606) 2024-06-26T07:05:12.7173184Z remote: Counting objects: 55% (1434/2606) 2024-06-26T07:05:12.7173920Z remote: Counting objects: 56% (1460/2606) 2024-06-26T07:05:12.7174519Z remote: Counting objects: 57% (1486/2606) 2024-06-26T07:05:12.7175206Z remote: Counting objects: 58% (1512/2606) 2024-06-26T07:05:12.7175788Z remote: Counting objects: 59% (1538/2606) 2024-06-26T07:05:12.7176731Z remote: Counting objects: 60% (1564/2606) 2024-06-26T07:05:12.7177381Z remote: Counting objects: 61% (1590/2606) 2024-06-26T07:05:12.7178063Z remote: Counting objects: 62% (1616/2606) 2024-06-26T07:05:12.7178803Z remote: Counting objects: 63% (1642/2606) 2024-06-26T07:05:12.7179663Z remote: Counting objects: 64% (1668/2606) 2024-06-26T07:05:12.7180462Z remote: Counting objects: 65% (1694/2606) 2024-06-26T07:05:12.7181413Z remote: Counting objects: 66% (1720/2606) 2024-06-26T07:05:12.7182144Z remote: Counting objects: 67% (1747/2606) 2024-06-26T07:05:12.7182829Z remote: Counting objects: 68% (1773/2606) 2024-06-26T07:05:12.7183437Z remote: Counting objects: 69% (1799/2606) 2024-06-26T07:05:12.7184010Z remote: Counting objects: 70% (1825/2606) 2024-06-26T07:05:12.7184675Z remote: Counting objects: 71% (1851/2606) 2024-06-26T07:05:12.7185239Z remote: Counting objects: 72% (1877/2606) 2024-06-26T07:05:12.7185988Z remote: Counting objects: 73% (1903/2606) 2024-06-26T07:05:12.7186643Z remote: Counting objects: 74% (1929/2606) 2024-06-26T07:05:12.7187193Z remote: Counting objects: 75% (1955/2606) 2024-06-26T07:05:12.7187818Z remote: Counting objects: 76% (1981/2606) 2024-06-26T07:05:12.7188494Z remote: Counting objects: 77% (2007/2606) 2024-06-26T07:05:12.7189021Z remote: Counting objects: 78% (2033/2606) 2024-06-26T07:05:12.7189536Z remote: Counting objects: 79% (2059/2606) 2024-06-26T07:05:12.7190049Z remote: Counting objects: 80% (2085/2606) 2024-06-26T07:05:12.7190668Z remote: Counting objects: 81% (2111/2606) 2024-06-26T07:05:12.7191189Z remote: Counting objects: 82% (2137/2606) 2024-06-26T07:05:12.7191725Z remote: Counting objects: 83% (2163/2606) 2024-06-26T07:05:12.7192235Z remote: Counting objects: 84% (2190/2606) 2024-06-26T07:05:12.7192755Z remote: Counting objects: 85% (2216/2606) 2024-06-26T07:05:12.7193271Z remote: Counting objects: 86% (2242/2606) 2024-06-26T07:05:12.7193788Z remote: Counting objects: 87% (2268/2606) 2024-06-26T07:05:12.7194298Z remote: Counting objects: 88% (2294/2606) 2024-06-26T07:05:12.7194814Z remote: Counting objects: 89% (2320/2606) 2024-06-26T07:05:12.7195338Z remote: Counting objects: 90% (2346/2606) 2024-06-26T07:05:12.7195845Z remote: Counting objects: 91% (2372/2606) 2024-06-26T07:05:12.7196373Z remote: Counting objects: 92% (2398/2606) 2024-06-26T07:05:12.7196892Z remote: Counting objects: 93% (2424/2606) 2024-06-26T07:05:12.7197401Z remote: Counting objects: 94% (2450/2606) 2024-06-26T07:05:12.7197906Z remote: Counting objects: 95% (2476/2606) 2024-06-26T07:05:12.7198419Z remote: Counting objects: 96% (2502/2606) 2024-06-26T07:05:12.7198928Z remote: Counting objects: 97% (2528/2606) 2024-06-26T07:05:12.7199439Z remote: Counting objects: 98% (2554/2606) 2024-06-26T07:05:12.7199940Z remote: Counting objects: 99% (2580/2606) 2024-06-26T07:05:12.7200453Z remote: Counting objects: 100% (2606/2606) 2024-06-26T07:05:12.7201001Z remote: Counting objects: 100% (2606/2606), done. 2024-06-26T07:05:12.7329162Z remote: Compressing objects: 0% (1/1076) 2024-06-26T07:05:12.7771809Z remote: Compressing objects: 1% (11/1076) 2024-06-26T07:05:12.8211320Z remote: Compressing objects: 2% (22/1076) 2024-06-26T07:05:12.8633346Z remote: Compressing objects: 3% (33/1076) 2024-06-26T07:05:12.8817060Z remote: Compressing objects: 4% (44/1076) 2024-06-26T07:05:12.9084650Z remote: Compressing objects: 5% (54/1076) 2024-06-26T07:05:12.9290862Z remote: Compressing objects: 6% (65/1076) 2024-06-26T07:05:12.9722027Z remote: Compressing objects: 7% (76/1076) 2024-06-26T07:05:13.0531752Z remote: Compressing objects: 8% (87/1076) 2024-06-26T07:05:13.1388221Z remote: Compressing objects: 9% (97/1076) 2024-06-26T07:05:13.1834406Z remote: Compressing objects: 10% (108/1076) 2024-06-26T07:05:13.1865538Z remote: Compressing objects: 11% (119/1076) 2024-06-26T07:05:13.1999169Z remote: Compressing objects: 12% (130/1076) 2024-06-26T07:05:13.2095894Z remote: Compressing objects: 13% (140/1076) 2024-06-26T07:05:13.2156885Z remote: Compressing objects: 14% (151/1076) 2024-06-26T07:05:13.2157559Z remote: Compressing objects: 15% (162/1076) 2024-06-26T07:05:13.2241498Z remote: Compressing objects: 16% (173/1076) 2024-06-26T07:05:13.2242050Z remote: Compressing objects: 17% (183/1076) 2024-06-26T07:05:13.2242590Z remote: Compressing objects: 18% (194/1076) 2024-06-26T07:05:13.2263893Z remote: Compressing objects: 19% (205/1076) 2024-06-26T07:05:13.2282089Z remote: Compressing objects: 20% (216/1076) 2024-06-26T07:05:13.2304792Z remote: Compressing objects: 21% (226/1076) 2024-06-26T07:05:13.2318250Z remote: Compressing objects: 22% (237/1076) 2024-06-26T07:05:13.2345191Z remote: Compressing objects: 23% (248/1076) 2024-06-26T07:05:13.2354866Z remote: Compressing objects: 24% (259/1076) 2024-06-26T07:05:13.2364657Z remote: Compressing objects: 25% (269/1076) 2024-06-26T07:05:13.2374024Z remote: Compressing objects: 26% (280/1076) 2024-06-26T07:05:13.2378160Z remote: Compressing objects: 27% (291/1076) 2024-06-26T07:05:13.2383457Z remote: Compressing objects: 28% (302/1076) 2024-06-26T07:05:13.2392461Z remote: Compressing objects: 29% (313/1076) 2024-06-26T07:05:13.2398329Z remote: Compressing objects: 30% (323/1076) 2024-06-26T07:05:13.2406729Z remote: Compressing objects: 31% (334/1076) 2024-06-26T07:05:13.2409544Z remote: Compressing objects: 32% (345/1076) 2024-06-26T07:05:13.2421173Z remote: Compressing objects: 33% (356/1076) 2024-06-26T07:05:13.2440215Z remote: Compressing objects: 34% (366/1076) 2024-06-26T07:05:13.2452105Z remote: Compressing objects: 35% (377/1076) 2024-06-26T07:05:13.2472791Z remote: Compressing objects: 36% (388/1076) 2024-06-26T07:05:13.2488571Z remote: Compressing objects: 37% (399/1076) 2024-06-26T07:05:13.2508802Z remote: Compressing objects: 38% (409/1076) 2024-06-26T07:05:13.2528625Z remote: Compressing objects: 39% (420/1076) 2024-06-26T07:05:13.2550028Z remote: Compressing objects: 40% (431/1076) 2024-06-26T07:05:13.2568175Z remote: Compressing objects: 41% (442/1076) 2024-06-26T07:05:13.2586774Z remote: Compressing objects: 42% (452/1076) 2024-06-26T07:05:13.2598543Z remote: Compressing objects: 43% (463/1076) 2024-06-26T07:05:13.2620690Z remote: Compressing objects: 44% (474/1076) 2024-06-26T07:05:13.2633479Z remote: Compressing objects: 45% (485/1076) 2024-06-26T07:05:13.2634026Z remote: Compressing objects: 46% (495/1076) 2024-06-26T07:05:13.2647848Z remote: Compressing objects: 47% (506/1076) 2024-06-26T07:05:13.2657723Z remote: Compressing objects: 48% (517/1076) 2024-06-26T07:05:13.2662915Z remote: Compressing objects: 49% (528/1076) 2024-06-26T07:05:13.2671127Z remote: Compressing objects: 50% (538/1076) 2024-06-26T07:05:13.2680906Z remote: Compressing objects: 51% (549/1076) 2024-06-26T07:05:13.2698631Z remote: Compressing objects: 52% (560/1076) 2024-06-26T07:05:13.2716788Z remote: Compressing objects: 53% (571/1076) 2024-06-26T07:05:13.2727496Z remote: Compressing objects: 54% (582/1076) 2024-06-26T07:05:13.2738152Z remote: Compressing objects: 55% (592/1076) 2024-06-26T07:05:13.2750068Z remote: Compressing objects: 56% (603/1076) 2024-06-26T07:05:13.2763022Z remote: Compressing objects: 57% (614/1076) 2024-06-26T07:05:13.2774747Z remote: Compressing objects: 58% (625/1076) 2024-06-26T07:05:13.2782365Z remote: Compressing objects: 59% (635/1076) 2024-06-26T07:05:13.2794794Z remote: Compressing objects: 60% (646/1076) 2024-06-26T07:05:13.2807391Z remote: Compressing objects: 61% (657/1076) 2024-06-26T07:05:13.2813872Z remote: Compressing objects: 62% (668/1076) 2024-06-26T07:05:13.2822894Z remote: Compressing objects: 63% (678/1076) 2024-06-26T07:05:13.2833372Z remote: Compressing objects: 64% (689/1076) 2024-06-26T07:05:13.2835872Z remote: Compressing objects: 65% (700/1076) 2024-06-26T07:05:13.2844544Z remote: Compressing objects: 66% (711/1076) 2024-06-26T07:05:13.2851203Z remote: Compressing objects: 67% (721/1076) 2024-06-26T07:05:13.2855837Z remote: Compressing objects: 68% (732/1076) 2024-06-26T07:05:13.2859917Z remote: Compressing objects: 69% (743/1076) 2024-06-26T07:05:13.2863953Z remote: Compressing objects: 70% (754/1076) 2024-06-26T07:05:13.2866389Z remote: Compressing objects: 71% (764/1076) 2024-06-26T07:05:13.2868531Z remote: Compressing objects: 72% (775/1076) 2024-06-26T07:05:13.2870444Z remote: Compressing objects: 73% (786/1076) 2024-06-26T07:05:13.2870992Z remote: Compressing objects: 74% (797/1076) 2024-06-26T07:05:13.2871633Z remote: Compressing objects: 75% (807/1076) 2024-06-26T07:05:13.2872177Z remote: Compressing objects: 76% (818/1076) 2024-06-26T07:05:13.2872720Z remote: Compressing objects: 77% (829/1076) 2024-06-26T07:05:13.2874723Z remote: Compressing objects: 78% (840/1076) 2024-06-26T07:05:13.2878982Z remote: Compressing objects: 79% (851/1076) 2024-06-26T07:05:13.2879518Z remote: Compressing objects: 80% (861/1076) 2024-06-26T07:05:13.2882735Z remote: Compressing objects: 81% (872/1076) 2024-06-26T07:05:13.2884731Z remote: Compressing objects: 82% (883/1076) 2024-06-26T07:05:13.2885270Z remote: Compressing objects: 83% (894/1076) 2024-06-26T07:05:13.2887783Z remote: Compressing objects: 84% (904/1076) 2024-06-26T07:05:13.2889899Z remote: Compressing objects: 85% (915/1076) 2024-06-26T07:05:13.2891848Z remote: Compressing objects: 86% (926/1076) 2024-06-26T07:05:13.2894128Z remote: Compressing objects: 87% (937/1076) 2024-06-26T07:05:13.2896287Z remote: Compressing objects: 88% (947/1076) 2024-06-26T07:05:13.2898069Z remote: Compressing objects: 89% (958/1076) 2024-06-26T07:05:13.2898618Z remote: Compressing objects: 90% (969/1076) 2024-06-26T07:05:13.2899154Z remote: Compressing objects: 91% (980/1076) 2024-06-26T07:05:13.2900610Z remote: Compressing objects: 92% (990/1076) 2024-06-26T07:05:13.2902662Z remote: Compressing objects: 93% (1001/1076) 2024-06-26T07:05:13.2903225Z remote: Compressing objects: 94% (1012/1076) 2024-06-26T07:05:13.2905188Z remote: Compressing objects: 95% (1023/1076) 2024-06-26T07:05:13.2905863Z remote: Compressing objects: 96% (1033/1076) 2024-06-26T07:05:13.2907645Z remote: Compressing objects: 97% (1044/1076) 2024-06-26T07:05:13.2909566Z remote: Compressing objects: 98% (1055/1076) 2024-06-26T07:05:13.2910119Z remote: Compressing objects: 99% (1066/1076) 2024-06-26T07:05:13.2912011Z remote: Compressing objects: 100% (1076/1076) 2024-06-26T07:05:13.2912605Z remote: Compressing objects: 100% (1076/1076), done. 2024-06-26T07:05:32.5154114Z remote: Total 990225 (delta 2068), reused 1872 (delta 1529), pack-reused 987619 2024-06-26T07:05:58.2413572Z [command]/usr/bin/git rev-parse --verify --quiet b8c4c54d347aa776934c60784e35936878ef18dc^{object} 2024-06-26T07:05:58.2432998Z b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:05:58.2436732Z ##[endgroup] 2024-06-26T07:05:58.2437269Z ##[group]Determining the checkout info 2024-06-26T07:05:58.2439539Z ##[endgroup] 2024-06-26T07:05:58.2440073Z ##[group]Checking out the ref 2024-06-26T07:05:58.2441978Z [command]/usr/bin/git checkout --quiet --force b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:05:59.4624007Z ##[endgroup] 2024-06-26T07:05:59.4628070Z ##[group]Setting up auth for fetching submodules 2024-06-26T07:05:59.4631631Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T07:05:59.4676937Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-06-26T07:05:59.4698596Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-06-26T07:05:59.4719760Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-06-26T07:05:59.4739012Z ##[endgroup] 2024-06-26T07:05:59.4739546Z ##[group]Fetching submodules 2024-06-26T07:05:59.4742436Z [command]/usr/bin/git submodule sync --recursive 2024-06-26T07:05:59.4951302Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-06-26T07:05:59.5157755Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-06-26T07:05:59.5159359Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-06-26T07:05:59.5161226Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-06-26T07:05:59.5162769Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-06-26T07:05:59.5164683Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-06-26T07:05:59.5166569Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-06-26T07:05:59.5168217Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-06-26T07:05:59.5169876Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-06-26T07:05:59.5171886Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-06-26T07:05:59.5174474Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-06-26T07:05:59.5176583Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-06-26T07:05:59.5179488Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-06-26T07:05:59.5181832Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-06-26T07:05:59.5184420Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-06-26T07:05:59.5187125Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-06-26T07:05:59.5189762Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-06-26T07:05:59.5192588Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:05:59.5195210Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-06-26T07:05:59.5198085Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-06-26T07:05:59.5200869Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-06-26T07:05:59.5203866Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-06-26T07:05:59.5206912Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-06-26T07:05:59.5210201Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-06-26T07:05:59.5213295Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-06-26T07:05:59.5216709Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-06-26T07:05:59.5220223Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-06-26T07:05:59.5223808Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-06-26T07:05:59.5227225Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-06-26T07:05:59.5230679Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-06-26T07:05:59.5234267Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-06-26T07:05:59.5238065Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-06-26T07:05:59.5241553Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-06-26T07:05:59.5245308Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-06-26T07:05:59.5248993Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-06-26T07:05:59.5252962Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-06-26T07:05:59.5272746Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-06-26T07:05:59.7614780Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-06-26T07:05:59.9421535Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-06-26T07:06:00.1257620Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-06-26T07:06:00.3579472Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-06-26T07:06:02.4443897Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-06-26T07:06:12.0037225Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-06-26T07:06:12.3709636Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-06-26T07:06:12.9767108Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-06-26T07:06:13.5539614Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-06-26T07:06:14.8089722Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-06-26T07:06:16.5522417Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-06-26T07:06:22.3416819Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-06-26T07:06:23.6443037Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-06-26T07:06:25.2796099Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-06-26T07:06:26.5359565Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-06-26T07:06:26.7134198Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-06-26T07:06:27.0687561Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-06-26T07:06:27.3674350Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-06-26T07:06:28.4029271Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-06-26T07:06:28.7322267Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-06-26T07:06:28.9701375Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-06-26T07:06:30.4546884Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-06-26T07:06:31.2244976Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-06-26T07:06:31.8929990Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-06-26T07:06:37.4466405Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-06-26T07:06:39.8394013Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-06-26T07:06:45.3885688Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-06-26T07:06:45.5917103Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-06-26T07:06:53.6046237Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-06-26T07:06:53.7719221Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-06-26T07:06:53.9844008Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-06-26T07:06:54.9779351Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-06-26T07:06:55.2575448Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-06-26T07:06:55.8744686Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-06-26T07:06:56.3013317Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-06-26T07:06:56.3094795Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-06-26T07:06:56.3156387Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-06-26T07:06:56.3336907Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-06-26T07:06:56.3644148Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-06-26T07:06:57.1373025Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-06-26T07:06:57.1547967Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-06-26T07:06:57.1903158Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-06-26T07:06:57.2747085Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-06-26T07:06:57.3013004Z Submodule path 'third_party/cudnn_frontend': checked out 'aa3abd4bc689d6412979c7f55f9cd132848c9c6a' 2024-06-26T07:06:57.7307700Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-06-26T07:06:57.9422352Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-06-26T07:06:58.0053406Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-06-26T07:06:58.0064324Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:06:58.0066277Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:06:58.0068725Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:06:58.0071094Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:06:58.0073512Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:06:58.0090128Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-06-26T07:06:58.9966777Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-06-26T07:06:59.6194169Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-06-26T07:07:01.3916900Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-06-26T07:07:02.3472576Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-06-26T07:07:02.6676214Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-06-26T07:07:02.7520650Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-06-26T07:07:03.0952368Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-06-26T07:07:03.1489370Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-06-26T07:07:03.1574873Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-06-26T07:07:03.2477064Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-06-26T07:07:03.2766261Z Submodule path 'third_party/fmt': checked out 'e69e5f977d458f2650bb346dadf2ad30c5320281' 2024-06-26T07:07:03.2831523Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-06-26T07:07:03.3134689Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-06-26T07:07:03.3328406Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-06-26T07:07:03.3695937Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T07:07:03.3794863Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-06-26T07:07:03.3806169Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:03.3821190Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-06-26T07:07:15.5480596Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-06-26T07:07:15.5616699Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-06-26T07:07:15.6399997Z Submodule path 'third_party/kineto': checked out '8681ff11e1fa54da39023076c5c43eddd87b7a8a' 2024-06-26T07:07:15.6413283Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:15.6416238Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:15.6418117Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:15.6435457Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-06-26T07:07:16.1418347Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-06-26T07:07:17.4145187Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-06-26T07:07:18.4264383Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-06-26T07:07:18.4274090Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:18.4275862Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:18.4277678Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:18.4279407Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:18.4281558Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:18.4283637Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:18.4285810Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:18.4288127Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:18.4307066Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-06-26T07:07:19.4823001Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-06-26T07:07:19.8178117Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-06-26T07:07:21.0347821Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-06-26T07:07:21.2795701Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-06-26T07:07:21.7388870Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-06-26T07:07:22.7010615Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-06-26T07:07:28.1766066Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-06-26T07:07:28.5451469Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-06-26T07:07:28.5589023Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-06-26T07:07:28.5879623Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-06-26T07:07:28.5978439Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-06-26T07:07:28.5989269Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:28.6006211Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-06-26T07:07:28.8518565Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-06-26T07:07:28.8658433Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-06-26T07:07:28.8999430Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-06-26T07:07:28.9804370Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-06-26T07:07:28.9925405Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-06-26T07:07:29.0224018Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out 'a33701196adfad74917046096bf5a2aa0ab0bb50' 2024-06-26T07:07:29.0717081Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-06-26T07:07:29.1012498Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-06-26T07:07:29.1210883Z Submodule path 'third_party/nccl/nccl': checked out '48bb7fec7953112ff37499a272317f6663f8f600' 2024-06-26T07:07:29.2075275Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-06-26T07:07:29.4817780Z Submodule path 'third_party/onnx': checked out '990217f043af7222348ca8f0301e17fa7b841781' 2024-06-26T07:07:29.4846171Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:29.4847691Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:29.4863391Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-06-26T07:07:29.8701261Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-06-26T07:07:31.0263219Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-06-26T07:07:31.0538112Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-06-26T07:07:31.1058037Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-06-26T07:07:31.1071188Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:31.1077429Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:31.1079150Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:31.1080836Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:31.1082843Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:31.1085076Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:31.1087854Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:31.1089480Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:31.1100492Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-06-26T07:07:31.5708913Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-06-26T07:07:32.6155066Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-06-26T07:07:32.9781679Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-06-26T07:07:38.6575083Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-06-26T07:07:38.9392785Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-06-26T07:07:39.1254981Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-06-26T07:07:39.4105822Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-06-26T07:07:47.0166611Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-06-26T07:07:47.0496994Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-06-26T07:07:47.0622466Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-06-26T07:07:47.1464013Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-06-26T07:07:47.1555578Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-06-26T07:07:47.1663309Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-06-26T07:07:47.1775780Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-06-26T07:07:47.1786490Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:47.1788551Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:47.1805233Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-06-26T07:07:48.9751385Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-06-26T07:07:50.1755525Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-06-26T07:07:50.2120759Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T07:07:50.5692481Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-06-26T07:07:50.5776566Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-06-26T07:07:50.7968266Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-06-26T07:07:50.7984859Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:50.7987016Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:50.8005015Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-06-26T07:07:51.1767470Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-06-26T07:07:52.1562830Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-06-26T07:07:52.2184606Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-06-26T07:07:52.2246208Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-06-26T07:07:52.2332981Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-06-26T07:07:52.2618909Z Submodule path 'third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b' 2024-06-26T07:07:52.2847949Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-06-26T07:07:52.3178647Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-06-26T07:07:52.3380421Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-06-26T07:07:52.3393300Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:52.3395169Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:52.3396901Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:52.3398695Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:52.3414480Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-06-26T07:07:53.3374493Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-06-26T07:07:53.5645287Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-06-26T07:07:54.6225255Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-06-26T07:07:55.6853869Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-06-26T07:07:55.6971328Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-06-26T07:07:55.7537377Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-06-26T07:07:55.7763753Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-06-26T07:07:55.7775007Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:55.7792751Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-06-26T07:07:55.9464484Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-06-26T07:07:55.9489213Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-06-26T07:07:55.9698265Z Entering 'android/libs/fbjni' 2024-06-26T07:07:55.9725133Z Entering 'third_party/FP16' 2024-06-26T07:07:55.9753494Z Entering 'third_party/FXdiv' 2024-06-26T07:07:55.9780602Z Entering 'third_party/NNPACK' 2024-06-26T07:07:55.9809237Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:55.9837158Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:55.9876296Z Entering 'third_party/benchmark' 2024-06-26T07:07:55.9907100Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:55.9934261Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:55.9964679Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:55.9993465Z Entering 'third_party/cutlass' 2024-06-26T07:07:56.0028176Z Entering 'third_party/eigen' 2024-06-26T07:07:56.0059387Z Entering 'third_party/fbgemm' 2024-06-26T07:07:56.0087166Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:56.0116314Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:56.0145633Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:56.0177992Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:56.0206073Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:56.0234345Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:56.0262550Z Entering 'third_party/fmt' 2024-06-26T07:07:56.0291127Z Entering 'third_party/foxi' 2024-06-26T07:07:56.0317776Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:56.0342854Z Entering 'third_party/gloo' 2024-06-26T07:07:56.0370317Z Entering 'third_party/googletest' 2024-06-26T07:07:56.0398598Z Entering 'third_party/ideep' 2024-06-26T07:07:56.0425277Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:56.0457468Z Entering 'third_party/ittapi' 2024-06-26T07:07:56.0483430Z Entering 'third_party/kineto' 2024-06-26T07:07:56.0514111Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:56.0543295Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:56.0572720Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:56.0599714Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:56.0626881Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:56.0653457Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:56.0683220Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:56.0708748Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:56.0734813Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:56.0764197Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:56.0792515Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:56.0820337Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:56.0847946Z Entering 'third_party/mimalloc' 2024-06-26T07:07:56.0876338Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:56.0905219Z Entering 'third_party/nlohmann' 2024-06-26T07:07:56.0932379Z Entering 'third_party/onnx' 2024-06-26T07:07:56.0972087Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:56.1001598Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:56.1031976Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:56.1060685Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:56.1088117Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:56.1114788Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:56.1139821Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:56.1168391Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:56.1194525Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:56.1221367Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:56.1247827Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:56.1277800Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:56.1305857Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:56.1347805Z Entering 'third_party/pocketfft' 2024-06-26T07:07:56.1375855Z Entering 'third_party/protobuf' 2024-06-26T07:07:56.1406631Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:56.1435199Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:56.1463626Z Entering 'third_party/psimd' 2024-06-26T07:07:56.1491617Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:56.1520093Z Entering 'third_party/pybind11' 2024-06-26T07:07:56.1550461Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:56.1576530Z Entering 'third_party/sleef' 2024-06-26T07:07:56.1605503Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:56.1633431Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:56.1658387Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:56.1685632Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:56.1713592Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:56.1738669Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:56.1779466Z ##[endgroup] 2024-06-26T07:07:56.1781426Z ##[group]Persisting credentials for submodules 2024-06-26T07:07:56.1784124Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-06-26T07:07:56.1995732Z Entering 'android/libs/fbjni' 2024-06-26T07:07:56.2031190Z Entering 'third_party/FP16' 2024-06-26T07:07:56.2066958Z Entering 'third_party/FXdiv' 2024-06-26T07:07:56.2100454Z Entering 'third_party/NNPACK' 2024-06-26T07:07:56.2133786Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:56.2167612Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:56.2212564Z Entering 'third_party/benchmark' 2024-06-26T07:07:56.2247806Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:56.2279966Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:56.2313472Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:56.2348537Z Entering 'third_party/cutlass' 2024-06-26T07:07:56.2390413Z Entering 'third_party/eigen' 2024-06-26T07:07:56.2425767Z Entering 'third_party/fbgemm' 2024-06-26T07:07:56.2463233Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:56.2497613Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:56.2532085Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:56.2570100Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:56.2604021Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:56.2639438Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:56.2675448Z Entering 'third_party/fmt' 2024-06-26T07:07:56.2709573Z Entering 'third_party/foxi' 2024-06-26T07:07:56.2744222Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:56.2778950Z Entering 'third_party/gloo' 2024-06-26T07:07:56.2814167Z Entering 'third_party/googletest' 2024-06-26T07:07:56.2849255Z Entering 'third_party/ideep' 2024-06-26T07:07:56.2884075Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:56.2925576Z Entering 'third_party/ittapi' 2024-06-26T07:07:56.2961675Z Entering 'third_party/kineto' 2024-06-26T07:07:56.2994999Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:56.3028958Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:56.3063933Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:56.3099302Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:56.3134384Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:56.3169046Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:56.3207273Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:56.3242566Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:56.3275787Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:56.3313400Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:56.3348820Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:56.3383881Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:56.3420010Z Entering 'third_party/mimalloc' 2024-06-26T07:07:56.3456822Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:56.3491452Z Entering 'third_party/nlohmann' 2024-06-26T07:07:56.3528192Z Entering 'third_party/onnx' 2024-06-26T07:07:56.3576446Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:56.3611161Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:56.3646591Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:56.3683339Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:56.3716217Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:56.3750012Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:56.3782551Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:56.3817114Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:56.3851175Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:56.3886500Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:56.3919626Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:56.3954145Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:56.3989942Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:56.4039334Z Entering 'third_party/pocketfft' 2024-06-26T07:07:56.4075319Z Entering 'third_party/protobuf' 2024-06-26T07:07:56.4110967Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:56.4144745Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:56.4180649Z Entering 'third_party/psimd' 2024-06-26T07:07:56.4214212Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:56.4250472Z Entering 'third_party/pybind11' 2024-06-26T07:07:56.4286183Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:56.4321244Z Entering 'third_party/sleef' 2024-06-26T07:07:56.4355450Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:56.4389772Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:56.4424198Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:56.4455944Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:56.4490835Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:56.4524992Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:56.4573414Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-06-26T07:07:56.4785646Z Entering 'android/libs/fbjni' 2024-06-26T07:07:56.4818362Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-06-26T07:07:56.4831678Z Entering 'third_party/FP16' 2024-06-26T07:07:56.4863404Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-06-26T07:07:56.4876703Z Entering 'third_party/FXdiv' 2024-06-26T07:07:56.4907881Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-06-26T07:07:56.4920792Z Entering 'third_party/NNPACK' 2024-06-26T07:07:56.4951898Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-06-26T07:07:56.4964471Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:56.4994805Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-06-26T07:07:56.5007774Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:56.5040694Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-06-26T07:07:56.5064061Z Entering 'third_party/benchmark' 2024-06-26T07:07:56.5096441Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:56.5109383Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:56.5139830Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-06-26T07:07:56.5153507Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:56.5184088Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T07:07:56.5197134Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:56.5229274Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-06-26T07:07:56.5241979Z Entering 'third_party/cutlass' 2024-06-26T07:07:56.5273113Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-06-26T07:07:56.5292042Z Entering 'third_party/eigen' 2024-06-26T07:07:56.5325455Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-06-26T07:07:56.5339678Z Entering 'third_party/fbgemm' 2024-06-26T07:07:56.5371484Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-06-26T07:07:56.5384695Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:56.5415518Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-06-26T07:07:56.5428512Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:56.5461189Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T07:07:56.5474377Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:56.5506092Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-06-26T07:07:56.5523156Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:56.5554937Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:56.5567107Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:56.5598607Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-06-26T07:07:56.5612173Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:56.5643927Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-06-26T07:07:56.5659900Z Entering 'third_party/fmt' 2024-06-26T07:07:56.5692649Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-06-26T07:07:56.5705055Z Entering 'third_party/foxi' 2024-06-26T07:07:56.5735838Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-06-26T07:07:56.5750897Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:56.5782362Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-06-26T07:07:56.5796578Z Entering 'third_party/gloo' 2024-06-26T07:07:56.5828272Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-06-26T07:07:56.5842434Z Entering 'third_party/googletest' 2024-06-26T07:07:56.5874378Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:56.5888062Z Entering 'third_party/ideep' 2024-06-26T07:07:56.5922586Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-06-26T07:07:56.5934466Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:56.5967698Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-06-26T07:07:56.5985781Z Entering 'third_party/ittapi' 2024-06-26T07:07:56.6017900Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-06-26T07:07:56.6032325Z Entering 'third_party/kineto' 2024-06-26T07:07:56.6063441Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-06-26T07:07:56.6077058Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:56.6109936Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-06-26T07:07:56.6122434Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:56.6154242Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-06-26T07:07:56.6167928Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:56.6200989Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-06-26T07:07:56.6212894Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:56.6245145Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-06-26T07:07:56.6257900Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:56.6289916Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-06-26T07:07:56.6301527Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:56.6332974Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-06-26T07:07:56.6346526Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:56.6379437Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-06-26T07:07:56.6392216Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:56.6423485Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:56.6435587Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:56.6466839Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-06-26T07:07:56.6480120Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:56.6513853Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-06-26T07:07:56.6529661Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:56.6561261Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-06-26T07:07:56.6573447Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:56.6603965Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-06-26T07:07:56.6616683Z Entering 'third_party/mimalloc' 2024-06-26T07:07:56.6648533Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-06-26T07:07:56.6661233Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:56.6690975Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-06-26T07:07:56.6702506Z Entering 'third_party/nlohmann' 2024-06-26T07:07:56.6733006Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-06-26T07:07:56.6747567Z Entering 'third_party/onnx' 2024-06-26T07:07:56.6778756Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-06-26T07:07:56.6803418Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:56.6835033Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:56.6848430Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:56.6879853Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:07:56.6893710Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:56.6927209Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-06-26T07:07:56.6941596Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:56.6972286Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:56.6984529Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:56.7016823Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:56.7030098Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:56.7061954Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-06-26T07:07:56.7074437Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:56.7105848Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-06-26T07:07:56.7119199Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:56.7150289Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-06-26T07:07:56.7162567Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:56.7194977Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-06-26T07:07:56.7207304Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:56.7239686Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-06-26T07:07:56.7251118Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:56.7282900Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-06-26T07:07:56.7297651Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:56.7328216Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-06-26T07:07:56.7342863Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:56.7374022Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-06-26T07:07:56.7400967Z Entering 'third_party/pocketfft' 2024-06-26T07:07:56.7434882Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-06-26T07:07:56.7447258Z Entering 'third_party/protobuf' 2024-06-26T07:07:56.7477275Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-06-26T07:07:56.7493301Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:56.7527439Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-06-26T07:07:56.7540265Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:56.7571147Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:56.7585919Z Entering 'third_party/psimd' 2024-06-26T07:07:56.7618738Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-06-26T07:07:56.7631262Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:56.7663099Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-06-26T07:07:56.7676245Z Entering 'third_party/pybind11' 2024-06-26T07:07:56.7708310Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:07:56.7721646Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:56.7753321Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-06-26T07:07:56.7765803Z Entering 'third_party/sleef' 2024-06-26T07:07:56.7796711Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-06-26T07:07:56.7808084Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:56.7839934Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-06-26T07:07:56.7852452Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:56.7885103Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-06-26T07:07:56.7898738Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:56.7929147Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-06-26T07:07:56.7941263Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:56.7971347Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-06-26T07:07:56.7983170Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:56.8013904Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-06-26T07:07:56.8026399Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:56.8058087Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-06-26T07:07:56.8951320Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-06-26T07:07:56.9151143Z Entering 'android/libs/fbjni' 2024-06-26T07:07:56.9180184Z Entering 'third_party/FP16' 2024-06-26T07:07:56.9208471Z Entering 'third_party/FXdiv' 2024-06-26T07:07:56.9237908Z Entering 'third_party/NNPACK' 2024-06-26T07:07:56.9265458Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:56.9292268Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:56.9328558Z Entering 'third_party/benchmark' 2024-06-26T07:07:56.9355200Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:56.9381751Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:56.9409038Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:56.9438338Z Entering 'third_party/cutlass' 2024-06-26T07:07:56.9472846Z Entering 'third_party/eigen' 2024-06-26T07:07:56.9501646Z Entering 'third_party/fbgemm' 2024-06-26T07:07:56.9531227Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:56.9557491Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:56.9585646Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:56.9616887Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:56.9645527Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:56.9674901Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:56.9706071Z Entering 'third_party/fmt' 2024-06-26T07:07:56.9732673Z Entering 'third_party/foxi' 2024-06-26T07:07:56.9764074Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:56.9792225Z Entering 'third_party/gloo' 2024-06-26T07:07:56.9822260Z Entering 'third_party/googletest' 2024-06-26T07:07:56.9849595Z Entering 'third_party/ideep' 2024-06-26T07:07:56.9876213Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:56.9906756Z Entering 'third_party/ittapi' 2024-06-26T07:07:56.9933544Z Entering 'third_party/kineto' 2024-06-26T07:07:56.9962345Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:56.9990476Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:57.0019875Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:57.0047652Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:57.0076021Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:57.0101799Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:57.0129896Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:57.0157999Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:57.0184427Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:57.0213162Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:57.0242376Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:57.0268000Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:57.0296785Z Entering 'third_party/mimalloc' 2024-06-26T07:07:57.0325869Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:57.0353384Z Entering 'third_party/nlohmann' 2024-06-26T07:07:57.0382220Z Entering 'third_party/onnx' 2024-06-26T07:07:57.0420730Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:57.0447703Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:57.0476141Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:57.0505700Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:57.0532408Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:57.0561085Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:57.0587550Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:57.0614999Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:57.0642734Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:57.0669880Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:57.0694920Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:57.0725101Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:57.0753516Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:57.0796284Z Entering 'third_party/pocketfft' 2024-06-26T07:07:57.0824299Z Entering 'third_party/protobuf' 2024-06-26T07:07:57.0853301Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:57.0880877Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:57.0907540Z Entering 'third_party/psimd' 2024-06-26T07:07:57.0937036Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:57.0963544Z Entering 'third_party/pybind11' 2024-06-26T07:07:57.0990400Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:57.1016423Z Entering 'third_party/sleef' 2024-06-26T07:07:57.1044071Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:57.1071746Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:57.1099641Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:57.1127632Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:57.1155228Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:57.1181976Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:57.1223886Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-06-26T07:07:57.1432693Z Entering 'android/libs/fbjni' 2024-06-26T07:07:57.1462035Z Entering 'third_party/FP16' 2024-06-26T07:07:57.1491260Z Entering 'third_party/FXdiv' 2024-06-26T07:07:57.1520842Z Entering 'third_party/NNPACK' 2024-06-26T07:07:57.1549947Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T07:07:57.1581548Z Entering 'third_party/XNNPACK' 2024-06-26T07:07:57.1619408Z Entering 'third_party/benchmark' 2024-06-26T07:07:57.1647894Z Entering 'third_party/cpp-httplib' 2024-06-26T07:07:57.1676033Z Entering 'third_party/cpuinfo' 2024-06-26T07:07:57.1704672Z Entering 'third_party/cudnn_frontend' 2024-06-26T07:07:57.1730923Z Entering 'third_party/cutlass' 2024-06-26T07:07:57.1764251Z Entering 'third_party/eigen' 2024-06-26T07:07:57.1793798Z Entering 'third_party/fbgemm' 2024-06-26T07:07:57.1819882Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T07:07:57.1845937Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T07:07:57.1874131Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T07:07:57.1904429Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T07:07:57.1932741Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T07:07:57.1960858Z Entering 'third_party/flatbuffers' 2024-06-26T07:07:57.1992609Z Entering 'third_party/fmt' 2024-06-26T07:07:57.2020311Z Entering 'third_party/foxi' 2024-06-26T07:07:57.2048322Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T07:07:57.2076162Z Entering 'third_party/gloo' 2024-06-26T07:07:57.2102624Z Entering 'third_party/googletest' 2024-06-26T07:07:57.2131827Z Entering 'third_party/ideep' 2024-06-26T07:07:57.2160770Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T07:07:57.2193821Z Entering 'third_party/ittapi' 2024-06-26T07:07:57.2222024Z Entering 'third_party/kineto' 2024-06-26T07:07:57.2251300Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T07:07:57.2279693Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T07:07:57.2309711Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T07:07:57.2337350Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T07:07:57.2363287Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T07:07:57.2390756Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T07:07:57.2418810Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T07:07:57.2447786Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T07:07:57.2474427Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T07:07:57.2502428Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T07:07:57.2531514Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T07:07:57.2557225Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T07:07:57.2583905Z Entering 'third_party/mimalloc' 2024-06-26T07:07:57.2614090Z Entering 'third_party/nccl/nccl' 2024-06-26T07:07:57.2644012Z Entering 'third_party/nlohmann' 2024-06-26T07:07:57.2672440Z Entering 'third_party/onnx' 2024-06-26T07:07:57.2712581Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T07:07:57.2741789Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T07:07:57.2769800Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T07:07:57.2800493Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T07:07:57.2827991Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T07:07:57.2855435Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T07:07:57.2883621Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T07:07:57.2910194Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T07:07:57.2936601Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T07:07:57.2963529Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T07:07:57.2991284Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T07:07:57.3020467Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T07:07:57.3049144Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T07:07:57.3090814Z Entering 'third_party/pocketfft' 2024-06-26T07:07:57.3121710Z Entering 'third_party/protobuf' 2024-06-26T07:07:57.3151243Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T07:07:57.3178017Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T07:07:57.3205995Z Entering 'third_party/psimd' 2024-06-26T07:07:57.3234463Z Entering 'third_party/pthreadpool' 2024-06-26T07:07:57.3264006Z Entering 'third_party/pybind11' 2024-06-26T07:07:57.3293233Z Entering 'third_party/python-peachpy' 2024-06-26T07:07:57.3321709Z Entering 'third_party/sleef' 2024-06-26T07:07:57.3350325Z Entering 'third_party/tensorpipe' 2024-06-26T07:07:57.3377225Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T07:07:57.3403958Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T07:07:57.3430531Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T07:07:57.3458518Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T07:07:57.3484309Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T07:07:57.3523730Z ##[endgroup] 2024-06-26T07:07:57.3551956Z [command]/usr/bin/git log -1 --format='%H' 2024-06-26T07:07:57.3572731Z 'b8c4c54d347aa776934c60784e35936878ef18dc' 2024-06-26T07:07:57.3745024Z Prepare all required actions 2024-06-26T07:07:57.3745614Z Getting action download info 2024-06-26T07:07:57.5236056Z ##[group]Run ./.github/actions/setup-linux 2024-06-26T07:07:57.5236481Z env: 2024-06-26T07:07:57.5236756Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:57.5237097Z ##[endgroup] 2024-06-26T07:07:57.5295442Z ##[group]Run set -euo pipefail 2024-06-26T07:07:57.5295989Z set -euo pipefail 2024-06-26T07:07:57.5296539Z function get_ec2_metadata() { 2024-06-26T07:07:57.5297070Z  # Pulled from instance metadata endpoint for EC2 2024-06-26T07:07:57.5297957Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-06-26T07:07:57.5298726Z  category=$1 2024-06-26T07:07:57.5299238Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-06-26T07:07:57.5299872Z  runner_name_str=i-044844c31b6a1c4ce 2024-06-26T07:07:57.5300363Z  if [[ -f /.inarc ]]; then 2024-06-26T07:07:57.5300850Z  echo "ARC Runner, no info on ec2 metadata" 2024-06-26T07:07:57.5301415Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-06-26T07:07:57.5302091Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-06-26T07:07:57.5302694Z  else 2024-06-26T07:07:57.5303185Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-06-26T07:07:57.5303758Z  fi 2024-06-26T07:07:57.5304040Z } 2024-06-26T07:07:57.5304394Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-06-26T07:07:57.5304963Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-06-26T07:07:57.5305672Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-06-26T07:07:57.5306236Z echo "system info $(uname -a)" 2024-06-26T07:07:57.5314087Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:57.5314577Z env: 2024-06-26T07:07:57.5314856Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:57.5315199Z ##[endgroup] 2024-06-26T07:07:57.5388100Z ami-id: ami-0ce0c36d7a00b20e2 2024-06-26T07:07:57.5428423Z instance-id: i-044844c31b6a1c4ce 2024-06-26T07:07:57.5467959Z instance-type: g5.4xlarge 2024-06-26T07:07:57.5474299Z system info Linux ip-10-0-49-197.ec2.internal 4.14.336-257.562.amzn2.x86_64 #1 SMP Sat Feb 24 09:50:35 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-06-26T07:07:57.5493505Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T07:07:57.5494422Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T07:07:57.5502829Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:57.5503322Z env: 2024-06-26T07:07:57.5503596Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:57.5503939Z ##[endgroup] 2024-06-26T07:07:57.5564613Z ##[group]Run if systemctl is-active --quiet docker; then 2024-06-26T07:07:57.5565221Z if systemctl is-active --quiet docker; then 2024-06-26T07:07:57.5565758Z  echo "Docker daemon is running..."; 2024-06-26T07:07:57.5566208Z else 2024-06-26T07:07:57.5566694Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-06-26T07:07:57.5567276Z fi 2024-06-26T07:07:57.5574831Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:57.5575359Z env: 2024-06-26T07:07:57.5575661Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:57.5576006Z ##[endgroup] 2024-06-26T07:07:57.5610841Z Docker daemon is running... 2024-06-26T07:07:57.5654008Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:07:57.5654562Z with: 2024-06-26T07:07:57.5654823Z shell: bash 2024-06-26T07:07:57.5655111Z timeout_minutes: 5 2024-06-26T07:07:57.5655431Z max_attempts: 3 2024-06-26T07:07:57.5655740Z retry_wait_seconds: 30 2024-06-26T07:07:57.5657502Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-06-26T07:07:57.5659123Z polling_interval_seconds: 1 2024-06-26T07:07:57.5659488Z warning_on_retry: true 2024-06-26T07:07:57.5659842Z continue_on_error: false 2024-06-26T07:07:57.5660284Z env: 2024-06-26T07:07:57.5660550Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:57.5660902Z AWS_RETRY_MODE: standard 2024-06-26T07:07:57.5661245Z AWS_MAX_ATTEMPTS: 5 2024-06-26T07:07:57.5661579Z AWS_DEFAULT_REGION: us-east-1 2024-06-26T07:07:57.5661938Z ##[endgroup] 2024-06-26T07:07:58.4171015Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:07:58.4171823Z Configure a credential helper to remove this warning. See 2024-06-26T07:07:58.4173560Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:07:58.4174125Z 2024-06-26T07:07:58.4174281Z Login Succeeded 2024-06-26T07:07:58.6148306Z Command completed after 1 attempt(s). 2024-06-26T07:07:58.6190484Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:07:58.6191195Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:07:58.6191840Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T07:07:58.6201237Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:58.6201725Z env: 2024-06-26T07:07:58.6201996Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:58.6202339Z ##[endgroup] 2024-06-26T07:07:58.6261686Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:07:58.6262462Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T07:07:58.6263046Z # shellcheck disable=SC2046 2024-06-26T07:07:58.6263505Z docker stop $(docker ps -q) || true 2024-06-26T07:07:58.6263985Z # Prune all of the docker images 2024-06-26T07:07:58.6264428Z docker system prune -af 2024-06-26T07:07:58.6272222Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:58.6272715Z env: 2024-06-26T07:07:58.6272991Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:58.6273338Z ##[endgroup] 2024-06-26T07:07:58.6526279Z "docker stop" requires at least 1 argument. 2024-06-26T07:07:58.6527105Z See 'docker stop --help'. 2024-06-26T07:07:58.6527357Z 2024-06-26T07:07:58.6527605Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-06-26T07:07:58.6528100Z 2024-06-26T07:07:58.6528305Z Stop one or more running containers 2024-06-26T07:07:58.6671373Z Total reclaimed space: 0B 2024-06-26T07:07:58.6700833Z ##[group]Run set +e 2024-06-26T07:07:58.6701193Z set +e 2024-06-26T07:07:58.6701488Z set -x 2024-06-26T07:07:58.6701780Z  2024-06-26T07:07:58.6702100Z PT_DOMAIN=download.pytorch.org 2024-06-26T07:07:58.6702883Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-06-26T07:07:58.6703950Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-06-26T07:07:58.6704694Z # one is returned at random 2024-06-26T07:07:58.6705364Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-06-26T07:07:58.6705903Z  2024-06-26T07:07:58.6706239Z if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T07:07:58.6706845Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-06-26T07:07:58.6707598Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-06-26T07:07:58.6708140Z  2024-06-26T07:07:58.6708457Z  if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T07:07:58.6708999Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-06-26T07:07:58.6709502Z  exit 1 2024-06-26T07:07:58.6709802Z  fi 2024-06-26T07:07:58.6710095Z fi 2024-06-26T07:07:58.6710493Z  2024-06-26T07:07:58.6710849Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-06-26T07:07:58.6711353Z  # Clean up any old records first 2024-06-26T07:07:58.6711854Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-06-26T07:07:58.6712310Z fi 2024-06-26T07:07:58.6712575Z  2024-06-26T07:07:58.6713097Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-06-26T07:07:58.6713634Z cat /etc/hosts 2024-06-26T07:07:58.6721340Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:58.6721828Z env: 2024-06-26T07:07:58.6722101Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:58.6722442Z ##[endgroup] 2024-06-26T07:07:58.6742489Z + PT_DOMAIN=download.pytorch.org 2024-06-26T07:07:58.6746944Z ++ dig -4 +short download.pytorch.org 2024-06-26T07:07:58.6747504Z ++ tail -n1 2024-06-26T07:07:58.6832964Z + RESOLVED_IP=18.67.76.42 2024-06-26T07:07:58.6833385Z + '[' -z 18.67.76.42 ']' 2024-06-26T07:07:58.6833829Z + grep -r download.pytorch.org /etc/hosts 2024-06-26T07:07:58.6838842Z 18.67.76.121 download.pytorch.org 2024-06-26T07:07:58.6839429Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-06-26T07:07:58.6935916Z + echo '18.67.76.42 download.pytorch.org' 2024-06-26T07:07:58.6936997Z + sudo tee -a /etc/hosts 2024-06-26T07:07:58.7172783Z 18.67.76.42 download.pytorch.org 2024-06-26T07:07:58.7187227Z + cat /etc/hosts 2024-06-26T07:07:58.7190886Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-06-26T07:07:58.7205996Z ::1 localhost6 localhost6.localdomain6 2024-06-26T07:07:58.7206500Z 18.67.76.42 download.pytorch.org 2024-06-26T07:07:58.7414327Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-06-26T07:07:58.7414925Z with: 2024-06-26T07:07:58.7416007Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:58.7417589Z docker-build-dir: .ci/docker 2024-06-26T07:07:58.7417959Z working-directory: . 2024-06-26T07:07:58.7418413Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:58.7418931Z force-push: false 2024-06-26T07:07:58.7419220Z env: 2024-06-26T07:07:58.7419479Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:58.7419823Z ##[endgroup] 2024-06-26T07:07:58.7436825Z ##[group]Run set -ex 2024-06-26T07:07:58.7437193Z set -ex 2024-06-26T07:07:58.7437478Z  2024-06-26T07:07:58.7438030Z # If the docker build directory or the build script doesn't exist, the action will 2024-06-26T07:07:58.7439023Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-06-26T07:07:58.7439822Z # job could then download the pre-built image as usual 2024-06-26T07:07:58.7440570Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-06-26T07:07:58.7441246Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:58.7441879Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:58.7442440Z  2024-06-26T07:07:58.7442946Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-06-26T07:07:58.7443570Z  exit 0 2024-06-26T07:07:58.7443878Z else 2024-06-26T07:07:58.7444243Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:58.7444691Z fi 2024-06-26T07:07:58.7444971Z  2024-06-26T07:07:58.7445440Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-06-26T07:07:58.7446292Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-06-26T07:07:58.7447060Z  # use it as it is, but first let's extract the tag 2024-06-26T07:07:58.7447765Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-06-26T07:07:58.7448489Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:58.7449189Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:58.7449742Z else 2024-06-26T07:07:58.7450176Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-06-26T07:07:58.7450971Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:58.7451865Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:58.7452608Z fi 2024-06-26T07:07:58.7460720Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:58.7461217Z env: 2024-06-26T07:07:58.7461484Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:58.7461826Z REPO_NAME: pytorch 2024-06-26T07:07:58.7462949Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:58.7464141Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T07:07:58.7464636Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:58.7465237Z ##[endgroup] 2024-06-26T07:07:58.7484568Z + [[ ! -d .ci/docker ]] 2024-06-26T07:07:58.7484986Z + [[ ! -f .ci/docker/build.sh ]] 2024-06-26T07:07:58.7485395Z + echo skip=false 2024-06-26T07:07:58.7487412Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-06-26T07:07:58.7490180Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:58.7491359Z ++ awk -F '[:,]' '{print $2}' 2024-06-26T07:07:58.7498324Z + DOCKER_TAG=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:58.7498981Z + echo docker-tag=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:58.7500678Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:58.7526918Z ##[group]Run set +e 2024-06-26T07:07:58.7527286Z set +e 2024-06-26T07:07:58.7527581Z set -x 2024-06-26T07:07:58.7527858Z  2024-06-26T07:07:58.7528128Z login() { 2024-06-26T07:07:58.7528792Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T07:07:58.7529510Z } 2024-06-26T07:07:58.7529772Z  2024-06-26T07:07:58.7530054Z retry () { 2024-06-26T07:07:58.7530442Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T07:07:58.7530883Z } 2024-06-26T07:07:58.7531152Z  2024-06-26T07:07:58.7531465Z retry login "${DOCKER_REGISTRY}" 2024-06-26T07:07:58.7531876Z  2024-06-26T07:07:58.7532334Z # Check if image already exists, if it does then skip building it 2024-06-26T07:07:58.7533038Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-06-26T07:07:58.7533539Z  exit 0 2024-06-26T07:07:58.7533832Z fi 2024-06-26T07:07:58.7534108Z  2024-06-26T07:07:58.7534603Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-06-26T07:07:58.7535434Z # be empty. The default action would be to continue rebuild the image 2024-06-26T07:07:58.7536567Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-06-26T07:07:58.7537231Z  # if we're on the base branch then use the parent commit 2024-06-26T07:07:58.7537820Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-06-26T07:07:58.7538245Z else 2024-06-26T07:07:58.7538715Z  # otherwise we're on a PR, so use the most recent base commit 2024-06-26T07:07:58.7539395Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-06-26T07:07:58.7539895Z fi 2024-06-26T07:07:58.7540167Z  2024-06-26T07:07:58.7540482Z if [[ -z "${MERGE_BASE}" ]]; then 2024-06-26T07:07:58.7540972Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:58.7541549Z  2024-06-26T07:07:58.7542197Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-06-26T07:07:58.7542957Z  exit 0 2024-06-26T07:07:58.7543246Z fi 2024-06-26T07:07:58.7543520Z  2024-06-26T07:07:58.7543954Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-06-26T07:07:58.7544911Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-06-26T07:07:58.7545817Z  exit 1 2024-06-26T07:07:58.7546120Z fi 2024-06-26T07:07:58.7546395Z  2024-06-26T07:07:58.7546882Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-06-26T07:07:58.7547820Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-06-26T07:07:58.7548683Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-06-26T07:07:58.7549731Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-06-26T07:07:58.7550830Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-06-26T07:07:58.7551594Z fi 2024-06-26T07:07:58.7551869Z  2024-06-26T07:07:58.7552227Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T07:07:58.7559950Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:58.7560442Z env: 2024-06-26T07:07:58.7560704Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:58.7561062Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T07:07:58.7561521Z BASE_REVISION: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:07:58.7562770Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:58.7564014Z DOCKER_TAG: 91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:58.7564598Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:58.7565111Z ##[endgroup] 2024-06-26T07:07:58.7584181Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:58.7584842Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:58.7586011Z + aws ecr get-login-password --region us-east-1 2024-06-26T07:07:58.7586903Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:59.1870594Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:07:59.1871581Z Configure a credential helper to remove this warning. See 2024-06-26T07:07:59.1872414Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:07:59.1872930Z 2024-06-26T07:07:59.1873050Z Login Succeeded 2024-06-26T07:07:59.1882963Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:59.4166051Z { 2024-06-26T07:07:59.4166467Z "schemaVersion": 2, 2024-06-26T07:07:59.4167216Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-06-26T07:07:59.4167970Z "config": { 2024-06-26T07:07:59.4168592Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-06-26T07:07:59.4169331Z "size": 49644, 2024-06-26T07:07:59.4170059Z "digest": "sha256:e7d48c2cbd3d9b87b03afe1b0df8f064acd2699045900da46dadf7d133480a59" 2024-06-26T07:07:59.4170908Z }, 2024-06-26T07:07:59.4171165Z "layers": [ 2024-06-26T07:07:59.4171445Z { 2024-06-26T07:07:59.4171880Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4172478Z "size": 28584317, 2024-06-26T07:07:59.4173253Z "digest": "sha256:63e9bbe323274e77e58d77c6ab6802d247458f784222fbb07a2556d6ec74ee05" 2024-06-26T07:07:59.4174282Z }, 2024-06-26T07:07:59.4174919Z { 2024-06-26T07:07:59.4175503Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4176446Z "size": 7944771, 2024-06-26T07:07:59.4177015Z "digest": "sha256:cfb3d849840ee60cee7b02bad68c1fc3c15928ebcd88f327754766b670578ed6" 2024-06-26T07:07:59.4177639Z }, 2024-06-26T07:07:59.4177883Z { 2024-06-26T07:07:59.4178316Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4178862Z "size": 57593718, 2024-06-26T07:07:59.4179402Z "digest": "sha256:968831e596a6288f0fed9b8612ee4ee8e75511037c4305058805492c5162e481" 2024-06-26T07:07:59.4180217Z }, 2024-06-26T07:07:59.4180465Z { 2024-06-26T07:07:59.4180892Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4181449Z "size": 187, 2024-06-26T07:07:59.4181992Z "digest": "sha256:ea310eb267ca1cab61b6b16f566cd28bfd59a741395a011f5e76716e15ba57c6" 2024-06-26T07:07:59.4182608Z }, 2024-06-26T07:07:59.4182854Z { 2024-06-26T07:07:59.4183284Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4183836Z "size": 6885, 2024-06-26T07:07:59.4184376Z "digest": "sha256:3af11d09e9cd1eb9c379f0a4071231e5a5642eb728b4b33bcb76be291f3c9488" 2024-06-26T07:07:59.4184978Z }, 2024-06-26T07:07:59.4185502Z { 2024-06-26T07:07:59.4185937Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4186481Z "size": 1361380219, 2024-06-26T07:07:59.4187031Z "digest": "sha256:ebfec18059b91e56882881ac34754f917861edb5f732c395d2a1a851bbd6db46" 2024-06-26T07:07:59.4187639Z }, 2024-06-26T07:07:59.4187885Z { 2024-06-26T07:07:59.4188328Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4188923Z "size": 62686, 2024-06-26T07:07:59.4189505Z "digest": "sha256:533b4aebf16914c763b7b0de3ce657590c6f979045e9fdf1f816adaf68d8a4d3" 2024-06-26T07:07:59.4190171Z }, 2024-06-26T07:07:59.4190410Z { 2024-06-26T07:07:59.4190859Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4191451Z "size": 1685, 2024-06-26T07:07:59.4192016Z "digest": "sha256:9dd75d06a0910f28cb1e484b8808724e5a6ee570ecb8fc04631368f546b39ed9" 2024-06-26T07:07:59.4192684Z }, 2024-06-26T07:07:59.4192931Z { 2024-06-26T07:07:59.4193388Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4193973Z "size": 1523, 2024-06-26T07:07:59.4194550Z "digest": "sha256:30bfca4dd3492d60ed8035b0eeb1229897041140db117f5663465a551e25851d" 2024-06-26T07:07:59.4195214Z }, 2024-06-26T07:07:59.4195461Z { 2024-06-26T07:07:59.4195941Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4196556Z "size": 2608186849, 2024-06-26T07:07:59.4197164Z "digest": "sha256:1b57ce94cad9a5097d99b8d6f7bcd51df5a162fc3c5f5686b689b76993724bc8" 2024-06-26T07:07:59.4197834Z }, 2024-06-26T07:07:59.4198076Z { 2024-06-26T07:07:59.4198526Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4199117Z "size": 86631, 2024-06-26T07:07:59.4199697Z "digest": "sha256:9ee6bdb31195dd42fd98147a75d540efe0c5708e0ecda866a0bca060084fddab" 2024-06-26T07:07:59.4200368Z }, 2024-06-26T07:07:59.4200617Z { 2024-06-26T07:07:59.4201074Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4201653Z "size": 1823, 2024-06-26T07:07:59.4202232Z "digest": "sha256:ee3624ae37279188ae1b8234c84dd6ef73e08e0d47f5bcdae71b074e76bc9d3d" 2024-06-26T07:07:59.4202903Z }, 2024-06-26T07:07:59.4203149Z { 2024-06-26T07:07:59.4203593Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4204188Z "size": 245740686, 2024-06-26T07:07:59.4204778Z "digest": "sha256:7813623661508ecef7ea82adbc87a31b2365eab7e0f40f452599dc093335b55c" 2024-06-26T07:07:59.4205443Z }, 2024-06-26T07:07:59.4205685Z { 2024-06-26T07:07:59.4206184Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4206857Z "size": 546, 2024-06-26T07:07:59.4207451Z "digest": "sha256:9b08a0698a8acd9f6f4aca5fcf77b3dfaafdb9acc3b1773ac8bd0ab95f4f559a" 2024-06-26T07:07:59.4208140Z }, 2024-06-26T07:07:59.4208391Z { 2024-06-26T07:07:59.4208841Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4209435Z "size": 1287, 2024-06-26T07:07:59.4210028Z "digest": "sha256:7b7b1bdcbcc9c46ac60fba4fc2466d5c42f5759493cde3bb98c6570d4d4a0fe0" 2024-06-26T07:07:59.4210712Z }, 2024-06-26T07:07:59.4210957Z { 2024-06-26T07:07:59.4211401Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4211994Z "size": 485, 2024-06-26T07:07:59.4212576Z "digest": "sha256:2d176cec9db3304b5a5c4b22717eb363aceec7c6683832ab07fa653bddd056d2" 2024-06-26T07:07:59.4213246Z }, 2024-06-26T07:07:59.4213494Z { 2024-06-26T07:07:59.4213944Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4214538Z "size": 91711722, 2024-06-26T07:07:59.4215141Z "digest": "sha256:cba4658f6a4a3027448a41ab8d0140caf0c7f88e52064c13fddaaa96f57f4c17" 2024-06-26T07:07:59.4215812Z }, 2024-06-26T07:07:59.4216214Z { 2024-06-26T07:07:59.4216709Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4217247Z "size": 3236, 2024-06-26T07:07:59.4217858Z "digest": "sha256:23b5517a7953f885dd26596ad67bd40bf66f7e0de0fd085073129d4797318e93" 2024-06-26T07:07:59.4218469Z }, 2024-06-26T07:07:59.4218714Z { 2024-06-26T07:07:59.4219130Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4219672Z "size": 1906, 2024-06-26T07:07:59.4220196Z "digest": "sha256:df0096280bdbf2bda44279839121c44b92b452c811944842490e27a100d43afa" 2024-06-26T07:07:59.4220784Z }, 2024-06-26T07:07:59.4221081Z { 2024-06-26T07:07:59.4221544Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4222082Z "size": 701, 2024-06-26T07:07:59.4222606Z "digest": "sha256:7454066a3d8e8de9a95e09dcf7ebb617a9211152c7a93c14602c3eaa0bfada5e" 2024-06-26T07:07:59.4223220Z }, 2024-06-26T07:07:59.4223462Z { 2024-06-26T07:07:59.4223891Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4224428Z "size": 2800676235, 2024-06-26T07:07:59.4225002Z "digest": "sha256:6d6f4e10cdaab6e61a20c168c32ce33442679a2904440ee70e5f73bdc7a333f9" 2024-06-26T07:07:59.4225712Z }, 2024-06-26T07:07:59.4225951Z { 2024-06-26T07:07:59.4226371Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4226918Z "size": 381, 2024-06-26T07:07:59.4227446Z "digest": "sha256:cbb2f8884ff2f08c1506a4239f24e669b4d2570b17a6d565dbf4d6ff09967ffe" 2024-06-26T07:07:59.4228040Z }, 2024-06-26T07:07:59.4228283Z { 2024-06-26T07:07:59.4228701Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4229257Z "size": 11884, 2024-06-26T07:07:59.4229795Z "digest": "sha256:ff99b372daac9bbf6d7ed53bf3a13d861e5481fa413509dd07b4eca89d33eb5a" 2024-06-26T07:07:59.4230420Z }, 2024-06-26T07:07:59.4230661Z { 2024-06-26T07:07:59.4231076Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4231615Z "size": 803, 2024-06-26T07:07:59.4232301Z "digest": "sha256:be64fa49eeafe614dfde751dedd7943dacee92c63a14b96f70883921f48ea2ad" 2024-06-26T07:07:59.4232931Z }, 2024-06-26T07:07:59.4233167Z { 2024-06-26T07:07:59.4233593Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4234136Z "size": 106, 2024-06-26T07:07:59.4234658Z "digest": "sha256:13e86101f8af83d45e0a882b25c508c99f1039ea52ec791d0ca0b15075780ba8" 2024-06-26T07:07:59.4235255Z }, 2024-06-26T07:07:59.4235497Z { 2024-06-26T07:07:59.4235971Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4236513Z "size": 504, 2024-06-26T07:07:59.4237036Z "digest": "sha256:6593f0a66be9f407a072fcf27a5e1610f17f11c05d9e321d81f87f2ace448183" 2024-06-26T07:07:59.4237730Z }, 2024-06-26T07:07:59.4237975Z { 2024-06-26T07:07:59.4238393Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4238936Z "size": 121477253, 2024-06-26T07:07:59.4239484Z "digest": "sha256:1b30bd2fbb84b8b9c9a808813226c10322406609ac5f28906d35f32f63e7e533" 2024-06-26T07:07:59.4240096Z }, 2024-06-26T07:07:59.4240332Z { 2024-06-26T07:07:59.4240755Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4241292Z "size": 109, 2024-06-26T07:07:59.4241816Z "digest": "sha256:2192e794bf7429b6bb2d1c0b847843ec7d6ccd461378755d404877e2ea22367b" 2024-06-26T07:07:59.4242412Z }, 2024-06-26T07:07:59.4242654Z { 2024-06-26T07:07:59.4243075Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4243613Z "size": 490, 2024-06-26T07:07:59.4244124Z "digest": "sha256:028168f5e8d46269269d44a154749a92eb9e4db95194c9c7952d75fdf9bf9724" 2024-06-26T07:07:59.4244712Z }, 2024-06-26T07:07:59.4244961Z { 2024-06-26T07:07:59.4245376Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4245916Z "size": 297, 2024-06-26T07:07:59.4246453Z "digest": "sha256:4b2cffe5f919caab7ddc734454591fb4be35295fba19b08792cff995b2f276a4" 2024-06-26T07:07:59.4247067Z }, 2024-06-26T07:07:59.4247365Z { 2024-06-26T07:07:59.4247789Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4248335Z "size": 103, 2024-06-26T07:07:59.4248863Z "digest": "sha256:340ec058739ef8635db4e96bca19a61d3e99728b9cff847c56ff6473d805a50d" 2024-06-26T07:07:59.4249456Z }, 2024-06-26T07:07:59.4249700Z { 2024-06-26T07:07:59.4250124Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4250666Z "size": 1474, 2024-06-26T07:07:59.4251209Z "digest": "sha256:54d3435eb7bb51dde703ed78f15e27c749edcffda9ba59dcb58afae63c21e8c2" 2024-06-26T07:07:59.4251831Z }, 2024-06-26T07:07:59.4252074Z { 2024-06-26T07:07:59.4252486Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4253033Z "size": 446711170, 2024-06-26T07:07:59.4253606Z "digest": "sha256:2a0a83b318ababab37ca3bfcd55c3b18bbd9fa86e1c6aa15cce35ac76fa0d861" 2024-06-26T07:07:59.4254233Z }, 2024-06-26T07:07:59.4254468Z { 2024-06-26T07:07:59.4254894Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4255436Z "size": 158, 2024-06-26T07:07:59.4255967Z "digest": "sha256:4923a3f1456efc88ea7278de228a2bdb1e80ba784c735ad2c7fb83fa30247bbc" 2024-06-26T07:07:59.4256775Z }, 2024-06-26T07:07:59.4257021Z { 2024-06-26T07:07:59.4257443Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4257977Z "size": 564, 2024-06-26T07:07:59.4258507Z "digest": "sha256:a3cf7711a64b8f29c42d779973b3887aa5c018fb3c519cddfe21a7ad4adc5d7a" 2024-06-26T07:07:59.4259114Z }, 2024-06-26T07:07:59.4259357Z { 2024-06-26T07:07:59.4259770Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4260318Z "size": 35866956, 2024-06-26T07:07:59.4260874Z "digest": "sha256:9b330ec9c5f5326d6ca1ad1e8f00551b63f4b33e36c5b6ed08727dba5274e427" 2024-06-26T07:07:59.4261479Z }, 2024-06-26T07:07:59.4261718Z { 2024-06-26T07:07:59.4262146Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4262687Z "size": 104, 2024-06-26T07:07:59.4263205Z "digest": "sha256:578e8684b0dd62cc1bbcfe3773d63c395219efa2882c898c502b73e211f93c1d" 2024-06-26T07:07:59.4263810Z }, 2024-06-26T07:07:59.4264053Z { 2024-06-26T07:07:59.4264473Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4265009Z "size": 426, 2024-06-26T07:07:59.4265659Z "digest": "sha256:0be96edb295ccccd2633f47ae8b9c771196bc6acedfeb98e8a8317194e3d3a69" 2024-06-26T07:07:59.4266299Z }, 2024-06-26T07:07:59.4266569Z { 2024-06-26T07:07:59.4266987Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4267643Z "size": 20262083, 2024-06-26T07:07:59.4269504Z "digest": "sha256:f8cc0024874727e0abb67e208e0b858588da47aa83d759d341b42e69d08c7b8b" 2024-06-26T07:07:59.4270101Z }, 2024-06-26T07:07:59.4270339Z { 2024-06-26T07:07:59.4270761Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4271309Z "size": 439, 2024-06-26T07:07:59.4271831Z "digest": "sha256:0f8c404a49903396bf3074fdabb1d8e580f25f8491adc79c63e2e4b9e1f58c10" 2024-06-26T07:07:59.4272435Z }, 2024-06-26T07:07:59.4272674Z { 2024-06-26T07:07:59.4273094Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4273628Z "size": 701, 2024-06-26T07:07:59.4274152Z "digest": "sha256:7454066a3d8e8de9a95e09dcf7ebb617a9211152c7a93c14602c3eaa0bfada5e" 2024-06-26T07:07:59.4274762Z }, 2024-06-26T07:07:59.4275017Z { 2024-06-26T07:07:59.4275431Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4275970Z "size": 141, 2024-06-26T07:07:59.4276535Z "digest": "sha256:81f06ec3ad2eacdde9ca0fba9edcb5bb4daa0ce53d47a367d5f1c93a0498f47d" 2024-06-26T07:07:59.4277161Z }, 2024-06-26T07:07:59.4277404Z { 2024-06-26T07:07:59.4277828Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4278371Z "size": 136, 2024-06-26T07:07:59.4278956Z "digest": "sha256:15cb896a08c77e954532754de424be80448dbe89f02d2ceee1726dfbb0d13c00" 2024-06-26T07:07:59.4279558Z }, 2024-06-26T07:07:59.4279803Z { 2024-06-26T07:07:59.4280225Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4280761Z "size": 4911495392, 2024-06-26T07:07:59.4281309Z "digest": "sha256:70f24c0071799d23886a2bde3de22579c6ccbeab9f32ad10db4fbb86748359d3" 2024-06-26T07:07:59.4281906Z }, 2024-06-26T07:07:59.4282148Z { 2024-06-26T07:07:59.4282566Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4283110Z "size": 190, 2024-06-26T07:07:59.4283648Z "digest": "sha256:d6e5b2cdad18c8465ac6fbb119197e164a8c97ca1cd2fabcc283d75390c609d7" 2024-06-26T07:07:59.4284264Z }, 2024-06-26T07:07:59.4284506Z { 2024-06-26T07:07:59.4284927Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4285467Z "size": 566, 2024-06-26T07:07:59.4285995Z "digest": "sha256:d613fbc17c49d7880b9bb8ce020ca503055881227ff77daf9931db0e838009cc" 2024-06-26T07:07:59.4286598Z }, 2024-06-26T07:07:59.4286839Z { 2024-06-26T07:07:59.4287261Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4287798Z "size": 43163752, 2024-06-26T07:07:59.4288347Z "digest": "sha256:f60db448a4cf61af4de5135e9d47f077847e92fa7d4369ea7a15d782913ddae0" 2024-06-26T07:07:59.4288952Z }, 2024-06-26T07:07:59.4289188Z { 2024-06-26T07:07:59.4289608Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4290148Z "size": 106, 2024-06-26T07:07:59.4290664Z "digest": "sha256:77d3743428f27e62adbb1755c963bb6050b293260408ff3e35b3cd29416d1ec8" 2024-06-26T07:07:59.4291254Z }, 2024-06-26T07:07:59.4291495Z { 2024-06-26T07:07:59.4291919Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4292461Z "size": 1213, 2024-06-26T07:07:59.4293016Z "digest": "sha256:246f548b4992ec948ddcdaeda2779ece425be96304d241f556691ddcb43081a1" 2024-06-26T07:07:59.4293624Z }, 2024-06-26T07:07:59.4293869Z { 2024-06-26T07:07:59.4294286Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4294821Z "size": 701, 2024-06-26T07:07:59.4295349Z "digest": "sha256:7454066a3d8e8de9a95e09dcf7ebb617a9211152c7a93c14602c3eaa0bfada5e" 2024-06-26T07:07:59.4295954Z }, 2024-06-26T07:07:59.4296366Z { 2024-06-26T07:07:59.4296806Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4297352Z "size": 138, 2024-06-26T07:07:59.4297880Z "digest": "sha256:7347b8d528ae7d715b23105b2c8c1ca9dcef925f0360526a23dca384b89802b2" 2024-06-26T07:07:59.4298559Z }, 2024-06-26T07:07:59.4298802Z { 2024-06-26T07:07:59.4299223Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4299768Z "size": 120, 2024-06-26T07:07:59.4300293Z "digest": "sha256:f5fdfb720d778718fee5f39d48b041e44f7d0a79c85c96bc3135b853b506e339" 2024-06-26T07:07:59.4300911Z }, 2024-06-26T07:07:59.4301152Z { 2024-06-26T07:07:59.4301566Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4302110Z "size": 1904165241, 2024-06-26T07:07:59.4302670Z "digest": "sha256:39eccd4fa4cacc38279baa0f2805d587775ff2e5a6e3e690820106385122b03f" 2024-06-26T07:07:59.4303281Z }, 2024-06-26T07:07:59.4303515Z { 2024-06-26T07:07:59.4303939Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4304480Z "size": 171, 2024-06-26T07:07:59.4305013Z "digest": "sha256:8690fc49f7f82a9b8432517aefe2df3b8744b46901f0994e04cf88a5f94ff72a" 2024-06-26T07:07:59.4305712Z }, 2024-06-26T07:07:59.4305960Z { 2024-06-26T07:07:59.4306381Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4306922Z "size": 908, 2024-06-26T07:07:59.4307440Z "digest": "sha256:8d7518f93a82a0fc5f7025e252a0ad7d852687c5958e337860fdbcb833e94638" 2024-06-26T07:07:59.4308036Z }, 2024-06-26T07:07:59.4308361Z { 2024-06-26T07:07:59.4308776Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4309319Z "size": 701, 2024-06-26T07:07:59.4309843Z "digest": "sha256:7454066a3d8e8de9a95e09dcf7ebb617a9211152c7a93c14602c3eaa0bfada5e" 2024-06-26T07:07:59.4310445Z }, 2024-06-26T07:07:59.4310684Z { 2024-06-26T07:07:59.4311105Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4311646Z "size": 134, 2024-06-26T07:07:59.4312186Z "digest": "sha256:32afbdb75edead6a7949b50e3a9df52a07401168263fc4e3d0a48caf2e6b007b" 2024-06-26T07:07:59.4312793Z }, 2024-06-26T07:07:59.4313038Z { 2024-06-26T07:07:59.4313461Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4314011Z "size": 32, 2024-06-26T07:07:59.4314536Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:59.4315145Z }, 2024-06-26T07:07:59.4315385Z { 2024-06-26T07:07:59.4315805Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4316353Z "size": 156, 2024-06-26T07:07:59.4316885Z "digest": "sha256:29fa11d7e3f843bf8f97f94b44e1ccee56a8a0f757225e61249ec61ea09285a6" 2024-06-26T07:07:59.4317496Z }, 2024-06-26T07:07:59.4317731Z { 2024-06-26T07:07:59.4318155Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4318697Z "size": 1841, 2024-06-26T07:07:59.4319230Z "digest": "sha256:4fad4714694523fb61fdebda512ec545265b8d440e12d7e5cb2e0733bd1ab623" 2024-06-26T07:07:59.4319834Z }, 2024-06-26T07:07:59.4320077Z { 2024-06-26T07:07:59.4320499Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4321041Z "size": 7529784, 2024-06-26T07:07:59.4321590Z "digest": "sha256:64f272eae1dbffd659ddf30f628ade1707991bb8f8955d71793faa1333f0adbd" 2024-06-26T07:07:59.4322201Z }, 2024-06-26T07:07:59.4322444Z { 2024-06-26T07:07:59.4322866Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4323413Z "size": 164, 2024-06-26T07:07:59.4323950Z "digest": "sha256:7e5a573d67ada9f6563a45a648813a80eabdace41a9ddb4f060334e71c23c54b" 2024-06-26T07:07:59.4324560Z }, 2024-06-26T07:07:59.4324798Z { 2024-06-26T07:07:59.4325221Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4325765Z "size": 7944, 2024-06-26T07:07:59.4326314Z "digest": "sha256:bb11a9f8be67176857e7e6d2af7b07b48dec6f95b49db4975fe76db0db3d6f6f" 2024-06-26T07:07:59.4326966Z }, 2024-06-26T07:07:59.4327210Z { 2024-06-26T07:07:59.4327632Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4328223Z "size": 8070, 2024-06-26T07:07:59.4328757Z "digest": "sha256:3ff78522fefe4a25273fa0e89550a748e2d6291be0a078b0a8b6abd554ee985d" 2024-06-26T07:07:59.4329363Z }, 2024-06-26T07:07:59.4329605Z { 2024-06-26T07:07:59.4330020Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4330569Z "size": 303, 2024-06-26T07:07:59.4331098Z "digest": "sha256:79fb75b4561bf33c20e79e5f2c1e9f41ac01ed5e6ca70d927fcc9208735f715e" 2024-06-26T07:07:59.4331706Z }, 2024-06-26T07:07:59.4331943Z { 2024-06-26T07:07:59.4332361Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4332906Z "size": 7629627, 2024-06-26T07:07:59.4333441Z "digest": "sha256:5f47c3db012a3a2c5b18e32a9e1131012cd2d831872c4df937a461a60488a3be" 2024-06-26T07:07:59.4334042Z }, 2024-06-26T07:07:59.4334288Z { 2024-06-26T07:07:59.4334710Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4335245Z "size": 108, 2024-06-26T07:07:59.4335767Z "digest": "sha256:4db47072195da67747e4785caba4c434979990d1c379813a6728ed85ab1d8508" 2024-06-26T07:07:59.4336469Z }, 2024-06-26T07:07:59.4336715Z { 2024-06-26T07:07:59.4337132Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4337695Z + exit 0 2024-06-26T07:07:59.4338038Z "size": 54145777, 2024-06-26T07:07:59.4338582Z "digest": "sha256:2d3f55b15ec02cdcf6276839811c63508418cc99d2c2a9a0845969401fa09469" 2024-06-26T07:07:59.4339168Z }, 2024-06-26T07:07:59.4339407Z { 2024-06-26T07:07:59.4339827Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4340365Z "size": 474, 2024-06-26T07:07:59.4340881Z "digest": "sha256:b948d493b64026cf04be83a2e80f5ee81b10ce8165942a1b747e0d4c60840cfc" 2024-06-26T07:07:59.4341485Z }, 2024-06-26T07:07:59.4341726Z { 2024-06-26T07:07:59.4342141Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4342686Z "size": 1374858502, 2024-06-26T07:07:59.4343243Z "digest": "sha256:a7f6534d479c5ae8e53c7227610f43d9f74a352690f07b8cde3fbefb08e8aadf" 2024-06-26T07:07:59.4343847Z }, 2024-06-26T07:07:59.4344082Z { 2024-06-26T07:07:59.4344505Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4345045Z "size": 106, 2024-06-26T07:07:59.4345702Z "digest": "sha256:5f87f50e4f6cc6e9ab3ab17362c2f2f58d82d93a25c747e48b2fd93f95cbcb05" 2024-06-26T07:07:59.4346305Z }, 2024-06-26T07:07:59.4346548Z { 2024-06-26T07:07:59.4346970Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4347509Z "size": 558, 2024-06-26T07:07:59.4348019Z "digest": "sha256:629b88f68f9b74389f93d83244a3a1b1d019b1be702c153676c6fda5f1d86ff9" 2024-06-26T07:07:59.4348613Z }, 2024-06-26T07:07:59.4348855Z { 2024-06-26T07:07:59.4349267Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4349811Z "size": 46248418, 2024-06-26T07:07:59.4350355Z "digest": "sha256:07e387a2f37267d95a9fe93028134f1b7fb2abd82237aec7114ec03a619cf65a" 2024-06-26T07:07:59.4350958Z }, 2024-06-26T07:07:59.4351194Z { 2024-06-26T07:07:59.4351614Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4352153Z "size": 111, 2024-06-26T07:07:59.4352684Z "digest": "sha256:34d8f3f114f8990e13a79c92c5d7f03cfc2c77e61288fd67cf9feec6d7e5d0c8" 2024-06-26T07:07:59.4353281Z }, 2024-06-26T07:07:59.4353523Z { 2024-06-26T07:07:59.4353942Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4354478Z "size": 32, 2024-06-26T07:07:59.4355001Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:59.4355609Z }, 2024-06-26T07:07:59.4355852Z { 2024-06-26T07:07:59.4356268Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4356812Z "size": 32, 2024-06-26T07:07:59.4357341Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:59.4358032Z }, 2024-06-26T07:07:59.4358269Z { 2024-06-26T07:07:59.4358694Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4359233Z "size": 32, 2024-06-26T07:07:59.4359765Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:59.4360362Z }, 2024-06-26T07:07:59.4360606Z { 2024-06-26T07:07:59.4361034Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T07:07:59.4361571Z "size": 32, 2024-06-26T07:07:59.4362100Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T07:07:59.4362711Z } 2024-06-26T07:07:59.4362952Z ] 2024-06-26T07:07:59.4363186Z } 2024-06-26T07:07:59.4480509Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T07:07:59.4480989Z tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T07:07:59.4481531Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-06-26T07:07:59.4489564Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:59.4490076Z env: 2024-06-26T07:07:59.4490355Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:59.4491499Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:59.4492649Z ##[endgroup] 2024-06-26T07:07:59.4516286Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks-91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:59.4566170Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-06-26T07:07:59.4566740Z with: 2024-06-26T07:07:59.4567796Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:59.4569107Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:59.4569627Z env: 2024-06-26T07:07:59.4569897Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:59.4570236Z ##[endgroup] 2024-06-26T07:07:59.4585343Z ##[group]Run set -x 2024-06-26T07:07:59.4585682Z set -x 2024-06-26T07:07:59.4585983Z set +e 2024-06-26T07:07:59.4586274Z  2024-06-26T07:07:59.4586538Z login() { 2024-06-26T07:07:59.4587186Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T07:07:59.4587898Z } 2024-06-26T07:07:59.4588172Z  2024-06-26T07:07:59.4588470Z retry () { 2024-06-26T07:07:59.4588856Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T07:07:59.4589293Z } 2024-06-26T07:07:59.4589565Z  2024-06-26T07:07:59.4589878Z retry login "${DOCKER_REGISTRY}" 2024-06-26T07:07:59.4590281Z  2024-06-26T07:07:59.4590553Z set -e 2024-06-26T07:07:59.4591018Z # ignore output since only exit code is used for conditional 2024-06-26T07:07:59.4591723Z # only pull docker image if it's not available locally 2024-06-26T07:07:59.4592479Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-06-26T07:07:59.4593171Z  retry docker pull "${DOCKER_IMAGE}" 2024-06-26T07:07:59.4593607Z fi 2024-06-26T07:07:59.4601219Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:07:59.4601712Z env: 2024-06-26T07:07:59.4601990Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:07:59.4603137Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:59.4604428Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:59.4604939Z ##[endgroup] 2024-06-26T07:07:59.4623309Z + set +e 2024-06-26T07:07:59.4624177Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:59.4624844Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:59.4626355Z + aws ecr get-login-password --region us-east-1 2024-06-26T07:07:59.4627128Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T07:07:59.8756642Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T07:07:59.8757641Z Configure a credential helper to remove this warning. See 2024-06-26T07:07:59.8758759Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T07:07:59.8759499Z 2024-06-26T07:07:59.8759669Z Login Succeeded 2024-06-26T07:07:59.8767429Z + set -e 2024-06-26T07:07:59.8768720Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:59.8894943Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:07:59.8897427Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:08:00.1113742Z 91382da70d5719cd7007b6b80b71d2f48398f6b7: Pulling from pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks 2024-06-26T07:08:00.1136545Z 63e9bbe32327: Pulling fs layer 2024-06-26T07:08:00.1136944Z cfb3d849840e: Pulling fs layer 2024-06-26T07:08:00.1137642Z 968831e596a6: Pulling fs layer 2024-06-26T07:08:00.1138179Z ea310eb267ca: Pulling fs layer 2024-06-26T07:08:00.1138701Z 3af11d09e9cd: Pulling fs layer 2024-06-26T07:08:00.1139221Z ebfec18059b9: Pulling fs layer 2024-06-26T07:08:00.1139647Z 533b4aebf169: Pulling fs layer 2024-06-26T07:08:00.1140093Z 9dd75d06a091: Pulling fs layer 2024-06-26T07:08:00.1140576Z 30bfca4dd349: Pulling fs layer 2024-06-26T07:08:00.1141084Z 1b57ce94cad9: Pulling fs layer 2024-06-26T07:08:00.1141588Z 9ee6bdb31195: Pulling fs layer 2024-06-26T07:08:00.1142081Z ee3624ae3727: Pulling fs layer 2024-06-26T07:08:00.1142564Z 781362366150: Pulling fs layer 2024-06-26T07:08:00.1143050Z 9b08a0698a8a: Pulling fs layer 2024-06-26T07:08:00.1143632Z 7b7b1bdcbcc9: Pulling fs layer 2024-06-26T07:08:00.1144032Z 2d176cec9db3: Pulling fs layer 2024-06-26T07:08:00.1144406Z cba4658f6a4a: Pulling fs layer 2024-06-26T07:08:00.1144781Z 23b5517a7953: Pulling fs layer 2024-06-26T07:08:00.1145275Z df0096280bdb: Pulling fs layer 2024-06-26T07:08:00.1145658Z 7454066a3d8e: Pulling fs layer 2024-06-26T07:08:00.1146036Z 6d6f4e10cdaa: Pulling fs layer 2024-06-26T07:08:00.1146408Z cbb2f8884ff2: Pulling fs layer 2024-06-26T07:08:00.1146776Z ff99b372daac: Pulling fs layer 2024-06-26T07:08:00.1147152Z be64fa49eeaf: Pulling fs layer 2024-06-26T07:08:00.1147514Z 30bfca4dd349: Waiting 2024-06-26T07:08:00.1147849Z 13e86101f8af: Pulling fs layer 2024-06-26T07:08:00.1148211Z 6593f0a66be9: Pulling fs layer 2024-06-26T07:08:00.1148591Z 1b30bd2fbb84: Pulling fs layer 2024-06-26T07:08:00.1148949Z ebfec18059b9: Waiting 2024-06-26T07:08:00.1149275Z 2192e794bf74: Pulling fs layer 2024-06-26T07:08:00.1149642Z 028168f5e8d4: Pulling fs layer 2024-06-26T07:08:00.1149996Z 1b57ce94cad9: Waiting 2024-06-26T07:08:00.1150310Z 533b4aebf169: Waiting 2024-06-26T07:08:00.1150636Z 4b2cffe5f919: Pulling fs layer 2024-06-26T07:08:00.1151012Z 340ec058739e: Pulling fs layer 2024-06-26T07:08:00.1151372Z 9ee6bdb31195: Waiting 2024-06-26T07:08:00.1151682Z 9dd75d06a091: Waiting 2024-06-26T07:08:00.1152022Z 54d3435eb7bb: Pulling fs layer 2024-06-26T07:08:00.1152386Z ea310eb267ca: Waiting 2024-06-26T07:08:00.1152721Z 2a0a83b318ab: Pulling fs layer 2024-06-26T07:08:00.1153073Z ee3624ae3727: Waiting 2024-06-26T07:08:00.1153404Z 4923a3f1456e: Pulling fs layer 2024-06-26T07:08:00.1153772Z a3cf7711a64b: Pulling fs layer 2024-06-26T07:08:00.1154147Z 9b330ec9c5f5: Pulling fs layer 2024-06-26T07:08:00.1154494Z 9b08a0698a8a: Waiting 2024-06-26T07:08:00.1154818Z 578e8684b0dd: Pulling fs layer 2024-06-26T07:08:00.1155342Z 3af11d09e9cd: Waiting 2024-06-26T07:08:00.1155678Z 0be96edb295c: Pulling fs layer 2024-06-26T07:08:00.1156103Z f8cc00248747: Pulling fs layer 2024-06-26T07:08:00.1156460Z 7b7b1bdcbcc9: Waiting 2024-06-26T07:08:00.1156791Z 0f8c404a4990: Pulling fs layer 2024-06-26T07:08:00.1157141Z 2d176cec9db3: Waiting 2024-06-26T07:08:00.1157472Z 81f06ec3ad2e: Pulling fs layer 2024-06-26T07:08:00.1157844Z 15cb896a08c7: Pulling fs layer 2024-06-26T07:08:00.1158215Z 70f24c007179: Pulling fs layer 2024-06-26T07:08:00.1158584Z d6e5b2cdad18: Pulling fs layer 2024-06-26T07:08:00.1158958Z d613fbc17c49: Pulling fs layer 2024-06-26T07:08:00.1159315Z 23b5517a7953: Waiting 2024-06-26T07:08:00.1159636Z f60db448a4cf: Pulling fs layer 2024-06-26T07:08:00.1160010Z 77d3743428f2: Pulling fs layer 2024-06-26T07:08:00.1160367Z 6d6f4e10cdaa: Waiting 2024-06-26T07:08:00.1160682Z df0096280bdb: Waiting 2024-06-26T07:08:00.1161001Z 246f548b4992: Pulling fs layer 2024-06-26T07:08:00.1161353Z 7454066a3d8e: Waiting 2024-06-26T07:08:00.1161676Z cbb2f8884ff2: Waiting 2024-06-26T07:08:00.1162008Z 7347b8d528ae: Pulling fs layer 2024-06-26T07:08:00.1162358Z 2192e794bf74: Waiting 2024-06-26T07:08:00.1162670Z 54d3435eb7bb: Waiting 2024-06-26T07:08:00.1162985Z ff99b372daac: Waiting 2024-06-26T07:08:00.1163308Z f5fdfb720d77: Pulling fs layer 2024-06-26T07:08:00.1163668Z 028168f5e8d4: Waiting 2024-06-26T07:08:00.1163977Z 2a0a83b318ab: Waiting 2024-06-26T07:08:00.1164289Z 4b2cffe5f919: Waiting 2024-06-26T07:08:00.1164591Z 4923a3f1456e: Waiting 2024-06-26T07:08:00.1165004Z 39eccd4fa4ca: Pulling fs layer 2024-06-26T07:08:00.1165364Z 340ec058739e: Waiting 2024-06-26T07:08:00.1165667Z a3cf7711a64b: Waiting 2024-06-26T07:08:00.1165986Z be64fa49eeaf: Waiting 2024-06-26T07:08:00.1166302Z 1b30bd2fbb84: Waiting 2024-06-26T07:08:00.1166617Z 9b330ec9c5f5: Waiting 2024-06-26T07:08:00.1166937Z 8690fc49f7f8: Pulling fs layer 2024-06-26T07:08:00.1167299Z 13e86101f8af: Waiting 2024-06-26T07:08:00.1167625Z 8d7518f93a82: Pulling fs layer 2024-06-26T07:08:00.1167983Z 6593f0a66be9: Waiting 2024-06-26T07:08:00.1168289Z 578e8684b0dd: Waiting 2024-06-26T07:08:00.1168615Z 32afbdb75ede: Pulling fs layer 2024-06-26T07:08:00.1168982Z 0be96edb295c: Waiting 2024-06-26T07:08:00.1169305Z 4f4fb700ef54: Pulling fs layer 2024-06-26T07:08:00.1169659Z 0f8c404a4990: Waiting 2024-06-26T07:08:00.1169968Z f8cc00248747: Waiting 2024-06-26T07:08:00.1170279Z 81f06ec3ad2e: Waiting 2024-06-26T07:08:00.1170600Z 29fa11d7e3f8: Pulling fs layer 2024-06-26T07:08:00.1170974Z 4fad47146945: Pulling fs layer 2024-06-26T07:08:00.1171350Z 64f272eae1db: Pulling fs layer 2024-06-26T07:08:00.1171717Z 7e5a573d67ad: Pulling fs layer 2024-06-26T07:08:00.1172071Z 77d3743428f2: Waiting 2024-06-26T07:08:00.1172399Z bb11a9f8be67: Pulling fs layer 2024-06-26T07:08:00.1172771Z 3ff78522fefe: Pulling fs layer 2024-06-26T07:08:00.1173134Z 79fb75b4561b: Pulling fs layer 2024-06-26T07:08:00.1173504Z 5f47c3db012a: Pulling fs layer 2024-06-26T07:08:00.1173857Z 246f548b4992: Waiting 2024-06-26T07:08:00.1174173Z 70f24c007179: Waiting 2024-06-26T07:08:00.1174491Z 4db47072195d: Pulling fs layer 2024-06-26T07:08:00.1174846Z d6e5b2cdad18: Waiting 2024-06-26T07:08:00.1175176Z 2d3f55b15ec0: Pulling fs layer 2024-06-26T07:08:00.1175539Z b948d493b640: Pulling fs layer 2024-06-26T07:08:00.1175908Z a7f6534d479c: Pulling fs layer 2024-06-26T07:08:00.1176527Z 7347b8d528ae: Waiting 2024-06-26T07:08:00.1176844Z 39eccd4fa4ca: Waiting 2024-06-26T07:08:00.1177168Z 5f87f50e4f6c: Pulling fs layer 2024-06-26T07:08:00.1177536Z 629b88f68f9b: Pulling fs layer 2024-06-26T07:08:00.1177900Z f5fdfb720d77: Waiting 2024-06-26T07:08:00.1178228Z 07e387a2f372: Pulling fs layer 2024-06-26T07:08:00.1178588Z 34d8f3f114f8: Pulling fs layer 2024-06-26T07:08:00.1178947Z 4f4fb700ef54: Waiting 2024-06-26T07:08:00.1179258Z 8690fc49f7f8: Waiting 2024-06-26T07:08:00.1179564Z 64f272eae1db: Waiting 2024-06-26T07:08:00.1179876Z 29fa11d7e3f8: Waiting 2024-06-26T07:08:00.1180188Z 7e5a573d67ad: Waiting 2024-06-26T07:08:00.1180501Z 4fad47146945: Waiting 2024-06-26T07:08:00.1180897Z 8d7518f93a82: Waiting 2024-06-26T07:08:00.1181206Z 4db47072195d: Waiting 2024-06-26T07:08:00.1181521Z bb11a9f8be67: Waiting 2024-06-26T07:08:00.1181832Z 32afbdb75ede: Waiting 2024-06-26T07:08:00.1182150Z 2d3f55b15ec0: Waiting 2024-06-26T07:08:00.1182461Z 5f47c3db012a: Waiting 2024-06-26T07:08:00.1182771Z 3ff78522fefe: Waiting 2024-06-26T07:08:00.1183081Z 79fb75b4561b: Waiting 2024-06-26T07:08:00.1183388Z 34d8f3f114f8: Waiting 2024-06-26T07:08:00.1183700Z b948d493b640: Waiting 2024-06-26T07:08:00.1184018Z 629b88f68f9b: Waiting 2024-06-26T07:08:00.1184322Z a7f6534d479c: Waiting 2024-06-26T07:08:00.1184633Z 5f87f50e4f6c: Waiting 2024-06-26T07:08:00.1184943Z 07e387a2f372: Waiting 2024-06-26T07:08:00.1185345Z d613fbc17c49: Waiting 2024-06-26T07:08:00.1185661Z cba4658f6a4a: Waiting 2024-06-26T07:08:00.1185974Z 15cb896a08c7: Waiting 2024-06-26T07:08:00.5401330Z cfb3d849840e: Verifying Checksum 2024-06-26T07:08:00.5401772Z cfb3d849840e: Download complete 2024-06-26T07:08:00.6252804Z ea310eb267ca: Download complete 2024-06-26T07:08:00.6909993Z 3af11d09e9cd: Verifying Checksum 2024-06-26T07:08:00.6910524Z 3af11d09e9cd: Download complete 2024-06-26T07:08:01.0339520Z 63e9bbe32327: Verifying Checksum 2024-06-26T07:08:01.0339967Z 63e9bbe32327: Download complete 2024-06-26T07:08:01.1215138Z 533b4aebf169: Download complete 2024-06-26T07:08:01.2092602Z 9dd75d06a091: Verifying Checksum 2024-06-26T07:08:01.2093181Z 9dd75d06a091: Download complete 2024-06-26T07:08:01.3099379Z 30bfca4dd349: Download complete 2024-06-26T07:08:01.8336772Z 968831e596a6: Verifying Checksum 2024-06-26T07:08:01.8337271Z 968831e596a6: Download complete 2024-06-26T07:08:01.9542974Z 9ee6bdb31195: Download complete 2024-06-26T07:08:02.0564302Z 63e9bbe32327: Pull complete 2024-06-26T07:08:02.1043122Z ee3624ae3727: Verifying Checksum 2024-06-26T07:08:02.1043682Z ee3624ae3727: Download complete 2024-06-26T07:08:02.3382692Z cfb3d849840e: Pull complete 2024-06-26T07:08:03.2932422Z 968831e596a6: Pull complete 2024-06-26T07:08:03.3502312Z ea310eb267ca: Pull complete 2024-06-26T07:08:03.4014512Z 3af11d09e9cd: Pull complete 2024-06-26T07:08:09.2285829Z 781362366150: Verifying Checksum 2024-06-26T07:08:09.2286473Z 781362366150: Download complete 2024-06-26T07:08:09.3100983Z 9b08a0698a8a: Verifying Checksum 2024-06-26T07:08:09.3101613Z 9b08a0698a8a: Download complete 2024-06-26T07:08:09.3946044Z 7b7b1bdcbcc9: Download complete 2024-06-26T07:08:09.4726289Z 2d176cec9db3: Download complete 2024-06-26T07:08:12.2641936Z cba4658f6a4a: Verifying Checksum 2024-06-26T07:08:12.2642550Z cba4658f6a4a: Download complete 2024-06-26T07:08:12.3765759Z 23b5517a7953: Verifying Checksum 2024-06-26T07:08:12.3767415Z 23b5517a7953: Download complete 2024-06-26T07:08:12.4695026Z df0096280bdb: Verifying Checksum 2024-06-26T07:08:12.4695614Z df0096280bdb: Download complete 2024-06-26T07:08:12.5631958Z 7454066a3d8e: Download complete 2024-06-26T07:08:35.7860644Z ebfec18059b9: Download complete 2024-06-26T07:08:35.8641413Z cbb2f8884ff2: Verifying Checksum 2024-06-26T07:08:35.8642094Z cbb2f8884ff2: Download complete 2024-06-26T07:08:35.9475972Z ff99b372daac: Download complete 2024-06-26T07:08:36.0346781Z be64fa49eeaf: Download complete 2024-06-26T07:08:36.1198392Z 13e86101f8af: Verifying Checksum 2024-06-26T07:08:36.1200152Z 13e86101f8af: Download complete 2024-06-26T07:08:36.1924827Z 6593f0a66be9: Verifying Checksum 2024-06-26T07:08:36.1925259Z 6593f0a66be9: Download complete 2024-06-26T07:08:39.4180715Z 1b30bd2fbb84: Verifying Checksum 2024-06-26T07:08:39.4181395Z 1b30bd2fbb84: Download complete 2024-06-26T07:08:39.5218362Z 2192e794bf74: Verifying Checksum 2024-06-26T07:08:39.5219026Z 2192e794bf74: Download complete 2024-06-26T07:08:39.6220811Z 028168f5e8d4: Download complete 2024-06-26T07:08:39.7031586Z 4b2cffe5f919: Verifying Checksum 2024-06-26T07:08:39.7032208Z 4b2cffe5f919: Download complete 2024-06-26T07:08:39.7960943Z 340ec058739e: Verifying Checksum 2024-06-26T07:08:39.7961575Z 340ec058739e: Download complete 2024-06-26T07:08:39.9316624Z 54d3435eb7bb: Verifying Checksum 2024-06-26T07:08:39.9317658Z 54d3435eb7bb: Download complete 2024-06-26T07:08:48.2890124Z ebfec18059b9: Pull complete 2024-06-26T07:08:48.4869699Z 533b4aebf169: Pull complete 2024-06-26T07:08:48.7192107Z 9dd75d06a091: Pull complete 2024-06-26T07:08:48.9302026Z 30bfca4dd349: Pull complete 2024-06-26T07:08:52.4173397Z 2a0a83b318ab: Verifying Checksum 2024-06-26T07:08:52.4173876Z 2a0a83b318ab: Download complete 2024-06-26T07:08:52.5043382Z 4923a3f1456e: Verifying Checksum 2024-06-26T07:08:52.5043787Z 4923a3f1456e: Download complete 2024-06-26T07:08:52.6021729Z a3cf7711a64b: Verifying Checksum 2024-06-26T07:08:52.6022145Z a3cf7711a64b: Download complete 2024-06-26T07:08:54.1304764Z 9b330ec9c5f5: Download complete 2024-06-26T07:08:54.2216575Z 578e8684b0dd: Verifying Checksum 2024-06-26T07:08:54.2217011Z 578e8684b0dd: Download complete 2024-06-26T07:08:54.3300090Z 0be96edb295c: Verifying Checksum 2024-06-26T07:08:54.3300523Z 0be96edb295c: Download complete 2024-06-26T07:08:54.9639239Z f8cc00248747: Verifying Checksum 2024-06-26T07:08:54.9639745Z f8cc00248747: Download complete 2024-06-26T07:08:55.0685433Z 0f8c404a4990: Verifying Checksum 2024-06-26T07:08:55.0685901Z 0f8c404a4990: Download complete 2024-06-26T07:08:55.1321889Z 81f06ec3ad2e: Download complete 2024-06-26T07:08:55.2249824Z 15cb896a08c7: Verifying Checksum 2024-06-26T07:08:55.2250987Z 15cb896a08c7: Download complete 2024-06-26T07:09:07.4708487Z 1b57ce94cad9: Verifying Checksum 2024-06-26T07:09:07.4709086Z 1b57ce94cad9: Download complete 2024-06-26T07:09:07.5673540Z d6e5b2cdad18: Verifying Checksum 2024-06-26T07:09:07.5674136Z d6e5b2cdad18: Download complete 2024-06-26T07:09:07.6592301Z d613fbc17c49: Verifying Checksum 2024-06-26T07:09:07.6592772Z d613fbc17c49: Download complete 2024-06-26T07:09:08.8132194Z f60db448a4cf: Verifying Checksum 2024-06-26T07:09:08.8132681Z f60db448a4cf: Download complete 2024-06-26T07:09:08.8819387Z 77d3743428f2: Download complete 2024-06-26T07:09:08.9573332Z 246f548b4992: Verifying Checksum 2024-06-26T07:09:08.9573804Z 246f548b4992: Download complete 2024-06-26T07:09:09.0371566Z 7347b8d528ae: Verifying Checksum 2024-06-26T07:09:09.0371988Z 7347b8d528ae: Download complete 2024-06-26T07:09:09.1231794Z f5fdfb720d77: Download complete 2024-06-26T07:09:25.0556748Z 6d6f4e10cdaa: Verifying Checksum 2024-06-26T07:09:25.0557232Z 6d6f4e10cdaa: Download complete 2024-06-26T07:09:25.1342874Z 8690fc49f7f8: Download complete 2024-06-26T07:09:25.2103624Z 8d7518f93a82: Download complete 2024-06-26T07:09:25.2889533Z 32afbdb75ede: Verifying Checksum 2024-06-26T07:09:25.2890037Z 32afbdb75ede: Download complete 2024-06-26T07:09:25.2996763Z 4f4fb700ef54: Verifying Checksum 2024-06-26T07:09:25.2997173Z 4f4fb700ef54: Download complete 2024-06-26T07:09:25.3748415Z 29fa11d7e3f8: Download complete 2024-06-26T07:09:25.4611489Z 4fad47146945: Verifying Checksum 2024-06-26T07:09:25.4611953Z 4fad47146945: Download complete 2024-06-26T07:09:25.8223635Z 64f272eae1db: Download complete 2024-06-26T07:09:25.9007805Z 7e5a573d67ad: Verifying Checksum 2024-06-26T07:09:25.9008398Z 7e5a573d67ad: Download complete 2024-06-26T07:09:26.0050280Z bb11a9f8be67: Verifying Checksum 2024-06-26T07:09:26.0050733Z bb11a9f8be67: Download complete 2024-06-26T07:09:26.0831126Z 3ff78522fefe: Download complete 2024-06-26T07:09:26.1642796Z 79fb75b4561b: Download complete 2024-06-26T07:09:26.5725142Z 5f47c3db012a: Verifying Checksum 2024-06-26T07:09:26.5725601Z 5f47c3db012a: Download complete 2024-06-26T07:09:26.6818349Z 4db47072195d: Verifying Checksum 2024-06-26T07:09:26.6818922Z 4db47072195d: Download complete 2024-06-26T07:09:27.7729769Z 2d3f55b15ec0: Verifying Checksum 2024-06-26T07:09:27.7730646Z 2d3f55b15ec0: Download complete 2024-06-26T07:09:27.8610553Z b948d493b640: Download complete 2024-06-26T07:09:38.6228085Z 1b57ce94cad9: Pull complete 2024-06-26T07:09:38.8540760Z 9ee6bdb31195: Pull complete 2024-06-26T07:09:39.0577923Z ee3624ae3727: Pull complete 2024-06-26T07:09:46.5902433Z 781362366150: Pull complete 2024-06-26T07:09:46.8072055Z 9b08a0698a8a: Pull complete 2024-06-26T07:09:47.0541160Z 7b7b1bdcbcc9: Pull complete 2024-06-26T07:09:47.2759202Z 2d176cec9db3: Pull complete 2024-06-26T07:09:49.6575993Z cba4658f6a4a: Pull complete 2024-06-26T07:09:49.8681897Z 23b5517a7953: Pull complete 2024-06-26T07:09:50.0929908Z df0096280bdb: Pull complete 2024-06-26T07:09:50.3168600Z 7454066a3d8e: Pull complete 2024-06-26T07:09:56.1052176Z 39eccd4fa4ca: Verifying Checksum 2024-06-26T07:09:56.1052689Z 39eccd4fa4ca: Download complete 2024-06-26T07:09:56.1796006Z 5f87f50e4f6c: Download complete 2024-06-26T07:09:56.2694380Z 629b88f68f9b: Verifying Checksum 2024-06-26T07:09:56.2695208Z 629b88f68f9b: Download complete 2024-06-26T07:09:57.6562392Z 07e387a2f372: Verifying Checksum 2024-06-26T07:09:57.6563004Z 07e387a2f372: Download complete 2024-06-26T07:09:57.7698459Z 34d8f3f114f8: Download complete 2024-06-26T07:10:02.9169834Z a7f6534d479c: Verifying Checksum 2024-06-26T07:10:02.9170361Z a7f6534d479c: Download complete 2024-06-26T07:10:58.5780243Z 6d6f4e10cdaa: Pull complete 2024-06-26T07:10:58.7999604Z cbb2f8884ff2: Pull complete 2024-06-26T07:10:59.0138293Z ff99b372daac: Pull complete 2024-06-26T07:10:59.1507341Z be64fa49eeaf: Pull complete 2024-06-26T07:10:59.3147638Z 13e86101f8af: Pull complete 2024-06-26T07:10:59.4880679Z 6593f0a66be9: Pull complete 2024-06-26T07:11:02.5202335Z 70f24c007179: Verifying Checksum 2024-06-26T07:11:02.5202812Z 70f24c007179: Download complete 2024-06-26T07:11:03.0853346Z 1b30bd2fbb84: Pull complete 2024-06-26T07:11:03.2984567Z 2192e794bf74: Pull complete 2024-06-26T07:11:03.4968824Z 028168f5e8d4: Pull complete 2024-06-26T07:11:03.6524286Z 4b2cffe5f919: Pull complete 2024-06-26T07:11:03.7221687Z 340ec058739e: Pull complete 2024-06-26T07:11:03.8137495Z 54d3435eb7bb: Pull complete 2024-06-26T07:11:12.0522925Z 2a0a83b318ab: Pull complete 2024-06-26T07:11:12.2765719Z 4923a3f1456e: Pull complete 2024-06-26T07:11:12.4624077Z a3cf7711a64b: Pull complete 2024-06-26T07:11:13.4073047Z 9b330ec9c5f5: Pull complete 2024-06-26T07:11:13.6397526Z 578e8684b0dd: Pull complete 2024-06-26T07:11:13.8630133Z 0be96edb295c: Pull complete 2024-06-26T07:11:14.3786313Z f8cc00248747: Pull complete 2024-06-26T07:11:14.6132766Z 0f8c404a4990: Pull complete 2024-06-26T07:11:14.9282150Z 81f06ec3ad2e: Pull complete 2024-06-26T07:11:15.1523449Z 15cb896a08c7: Pull complete 2024-06-26T07:11:57.7176382Z 70f24c007179: Pull complete 2024-06-26T07:11:58.2268451Z d6e5b2cdad18: Pull complete 2024-06-26T07:11:58.4324188Z d613fbc17c49: Pull complete 2024-06-26T07:12:00.6099523Z f60db448a4cf: Pull complete 2024-06-26T07:12:00.8323195Z 77d3743428f2: Pull complete 2024-06-26T07:12:01.0671111Z 246f548b4992: Pull complete 2024-06-26T07:12:01.5479290Z 7347b8d528ae: Pull complete 2024-06-26T07:12:01.7645010Z f5fdfb720d77: Pull complete 2024-06-26T07:12:35.6325014Z 39eccd4fa4ca: Pull complete 2024-06-26T07:12:35.8681717Z 8690fc49f7f8: Pull complete 2024-06-26T07:12:36.0961250Z 8d7518f93a82: Pull complete 2024-06-26T07:12:36.5910171Z 32afbdb75ede: Pull complete 2024-06-26T07:12:36.8204908Z 4f4fb700ef54: Pull complete 2024-06-26T07:12:37.0590129Z 29fa11d7e3f8: Pull complete 2024-06-26T07:12:37.2833725Z 4fad47146945: Pull complete 2024-06-26T07:12:37.6517960Z 64f272eae1db: Pull complete 2024-06-26T07:12:37.8636892Z 7e5a573d67ad: Pull complete 2024-06-26T07:12:38.1013722Z bb11a9f8be67: Pull complete 2024-06-26T07:12:38.3381156Z 3ff78522fefe: Pull complete 2024-06-26T07:12:38.5625199Z 79fb75b4561b: Pull complete 2024-06-26T07:12:39.5368531Z 5f47c3db012a: Pull complete 2024-06-26T07:12:39.6004431Z 4db47072195d: Pull complete 2024-06-26T07:12:41.7027668Z 2d3f55b15ec0: Pull complete 2024-06-26T07:12:41.9293962Z b948d493b640: Pull complete 2024-06-26T07:12:55.4754766Z a7f6534d479c: Pull complete 2024-06-26T07:12:55.6951915Z 5f87f50e4f6c: Pull complete 2024-06-26T07:12:55.9300964Z 629b88f68f9b: Pull complete 2024-06-26T07:12:56.8106834Z 07e387a2f372: Pull complete 2024-06-26T07:12:57.0434210Z 34d8f3f114f8: Pull complete 2024-06-26T07:12:58.1150833Z Digest: sha256:037160f9b5ec877b659f1e94829492249e0bd2fc1a009ee3abbf0528226f6e1a 2024-06-26T07:12:58.1581202Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:12:58.1784038Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:12:58.1835294Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T07:12:58.1836217Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T07:12:58.1844565Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:12:58.1845292Z env: 2024-06-26T07:12:58.1845569Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:58.1845917Z ##[endgroup] 2024-06-26T07:12:58.1998725Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-06-26T07:12:58.1999262Z with: 2024-06-26T07:12:58.1999561Z driver-version: 550.54.15 2024-06-26T07:12:58.1999901Z env: 2024-06-26T07:12:58.2000170Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:58.2000501Z ##[endgroup] 2024-06-26T07:12:58.2113006Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:12:58.2113558Z with: 2024-06-26T07:12:58.2113825Z timeout_minutes: 10 2024-06-26T07:12:58.2114135Z max_attempts: 3 2024-06-26T07:12:58.2145481Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" 2024-06-26T07:12:58.2177170Z retry_wait_seconds: 10 2024-06-26T07:12:58.2177533Z polling_interval_seconds: 1 2024-06-26T07:12:58.2177898Z warning_on_retry: true 2024-06-26T07:12:58.2178245Z continue_on_error: false 2024-06-26T07:12:58.2178582Z env: 2024-06-26T07:12:58.2178850Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:12:58.2179197Z DRIVER_VERSION: 550.54.15 2024-06-26T07:12:58.2179542Z ##[endgroup] 2024-06-26T07:12:58.2713775Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-06-26T07:12:58.2714694Z + pre_install_nvidia_driver_amzn2 2024-06-26T07:12:58.2715282Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-06-26T07:12:58.5771099Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:12:58.6196813Z No Match for argument: nvidia-driver-latest-dkms 2024-06-26T07:12:58.6440053Z No Packages marked for removal 2024-06-26T07:12:58.6560094Z + install_nvidia_driver_common 2024-06-26T07:12:58.6562508Z + echo 'Before installing NVIDIA driver' 2024-06-26T07:12:58.6563129Z + lspci 2024-06-26T07:12:58.6563559Z Before installing NVIDIA driver 2024-06-26T07:12:58.6641122Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-06-26T07:12:58.6642651Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-06-26T07:12:58.6644755Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-06-26T07:12:58.6646345Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-06-26T07:12:58.6648144Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. Device 8061 2024-06-26T07:12:58.6649627Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-06-26T07:12:58.6651076Z 00:1e.0 3D controller: NVIDIA Corporation Device 2237 (rev a1) 2024-06-26T07:12:58.6651941Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-06-26T07:12:58.6652525Z + lsmod 2024-06-26T07:12:58.6654313Z Module Size Used by 2024-06-26T07:12:58.6654730Z xt_nat 16384 0 2024-06-26T07:12:58.6655103Z ib_core 266240 0 2024-06-26T07:12:58.6655475Z nvidia_modeset 1339392 0 2024-06-26T07:12:58.6655836Z veth 16384 0 2024-06-26T07:12:58.6656460Z nvidia_uvm 4599808 0 2024-06-26T07:12:58.6657098Z nvidia 53989376 2 nvidia_uvm,nvidia_modeset 2024-06-26T07:12:58.6657603Z drm 421888 1 nvidia 2024-06-26T07:12:58.6658022Z i2c_core 77824 2 nvidia,drm 2024-06-26T07:12:58.6658491Z backlight 16384 1 nvidia_modeset 2024-06-26T07:12:58.6658929Z xt_conntrack 16384 1 2024-06-26T07:12:58.6659311Z ipt_MASQUERADE 16384 1 2024-06-26T07:12:58.6659762Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-06-26T07:12:58.6660230Z nf_conntrack_netlink 49152 0 2024-06-26T07:12:58.6660679Z nfnetlink 16384 2 nf_conntrack_netlink 2024-06-26T07:12:58.6661137Z xfrm_user 45056 1 2024-06-26T07:12:58.6661521Z xfrm_algo 16384 1 xfrm_user 2024-06-26T07:12:58.6661933Z iptable_nat 16384 1 2024-06-26T07:12:58.6662310Z nf_conntrack_ipv4 16384 3 2024-06-26T07:12:58.6662740Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-06-26T07:12:58.6663215Z nf_nat_ipv4 16384 1 iptable_nat 2024-06-26T07:12:58.6663768Z nf_nat 32768 3 nf_nat_masquerade_ipv4,nf_nat_ipv4,xt_nat 2024-06-26T07:12:58.6664755Z nf_conntrack 155648 8 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,xt_nat,nf_conntrack_netlink 2024-06-26T07:12:58.6665798Z xt_addrtype 16384 2 2024-06-26T07:12:58.6666166Z iptable_filter 16384 1 2024-06-26T07:12:58.6666544Z br_netfilter 24576 0 2024-06-26T07:12:58.6666949Z bridge 172032 1 br_netfilter 2024-06-26T07:12:58.6667383Z stp 16384 1 bridge 2024-06-26T07:12:58.6667790Z llc 16384 2 bridge,stp 2024-06-26T07:12:58.6668200Z overlay 86016 0 2024-06-26T07:12:58.6668629Z sunrpc 393216 1 2024-06-26T07:12:58.6668992Z dm_mirror 28672 0 2024-06-26T07:12:58.6669383Z dm_region_hash 20480 1 dm_mirror 2024-06-26T07:12:58.6669860Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-06-26T07:12:58.6670365Z dm_mod 143360 2 dm_log,dm_mirror 2024-06-26T07:12:58.6670807Z dax 69632 1 dm_mod 2024-06-26T07:12:58.6671206Z crc32_pclmul 16384 0 2024-06-26T07:12:58.6671572Z ghash_clmulni_intel 16384 0 2024-06-26T07:12:58.6671953Z pcbc 16384 0 2024-06-26T07:12:58.6672317Z aesni_intel 188416 0 2024-06-26T07:12:58.6672695Z aes_x86_64 20480 1 aesni_intel 2024-06-26T07:12:58.6673137Z crypto_simd 16384 1 aesni_intel 2024-06-26T07:12:58.6673582Z glue_helper 16384 1 aesni_intel 2024-06-26T07:12:58.6674215Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-06-26T07:12:58.6674741Z mousedev 24576 0 2024-06-26T07:12:58.6675150Z evdev 20480 3 2024-06-26T07:12:58.6675526Z psmouse 32768 0 2024-06-26T07:12:58.6675893Z button 16384 0 2024-06-26T07:12:58.6676243Z ena 139264 0 2024-06-26T07:12:58.6676615Z ptp 20480 1 ena 2024-06-26T07:12:58.6677008Z pps_core 20480 1 ptp 2024-06-26T07:12:58.6677390Z crc32c_intel 24576 0 2024-06-26T07:12:58.6677868Z autofs4 49152 2 2024-06-26T07:12:58.6678222Z + modinfo nvidia 2024-06-26T07:12:58.6678869Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-06-26T07:12:58.6679557Z alias: char-major-195-* 2024-06-26T07:12:58.6679933Z version: 550.54.15 2024-06-26T07:12:58.6680271Z supported: external 2024-06-26T07:12:58.6680605Z license: NVIDIA 2024-06-26T07:12:58.6680970Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T07:12:58.6681450Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T07:12:58.6681905Z srcversion: 833721318DA517F0C2FEC97 2024-06-26T07:12:58.6682368Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T07:12:58.6682853Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T07:12:58.6683417Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T07:12:58.6683899Z depends: i2c-core,drm 2024-06-26T07:12:58.6684248Z retpoline: Y 2024-06-26T07:12:58.6684559Z name: nvidia 2024-06-26T07:12:58.6685102Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-06-26T07:12:58.6685752Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T07:12:58.6686381Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T07:12:58.6686969Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T07:12:58.6687407Z parm: NVreg_RmLogonRC:int 2024-06-26T07:12:58.6687837Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T07:12:58.6688281Z parm: NVreg_DeviceFileUID:int 2024-06-26T07:12:58.6688709Z parm: NVreg_DeviceFileGID:int 2024-06-26T07:12:58.6689139Z parm: NVreg_DeviceFileMode:int 2024-06-26T07:12:58.6689660Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T07:12:58.6690219Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T07:12:58.6690694Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T07:12:58.6691131Z parm: NVreg_EnableMSI:int 2024-06-26T07:12:58.6691540Z parm: NVreg_TCEBypassMode:int 2024-06-26T07:12:58.6692038Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T07:12:58.6692561Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T07:12:58.6693133Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T07:12:58.6693690Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T07:12:58.6694288Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T07:12:58.6694880Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T07:12:58.6695488Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T07:12:58.6696281Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T07:12:58.6696783Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T07:12:58.6697321Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T07:12:58.6697862Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T07:12:58.6698351Z parm: NVreg_MemoryPoolSize:int 2024-06-26T07:12:58.6698811Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T07:12:58.6699300Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T07:12:58.6699768Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T07:12:58.6700207Z parm: NVreg_NvLinkDisable:int 2024-06-26T07:12:58.6700703Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T07:12:58.6701221Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T07:12:58.6701743Z parm: NVreg_EnableResizableBar:int 2024-06-26T07:12:58.6702214Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T07:12:58.6702705Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T07:12:58.6703180Z parm: NVreg_RegistryDwords:charp 2024-06-26T07:12:58.6703676Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T07:12:58.6704143Z parm: NVreg_RmMsg:charp 2024-06-26T07:12:58.6704555Z parm: NVreg_GpuBlacklist:charp 2024-06-26T07:12:58.6705207Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T07:12:58.6705669Z parm: NVreg_ExcludedGpus:charp 2024-06-26T07:12:58.6706117Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T07:12:58.6706588Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T07:12:58.6707055Z parm: NVreg_ImexChannelCount:int 2024-06-26T07:12:58.6707492Z parm: rm_firmware_active:charp 2024-06-26T07:12:58.6707898Z + HAS_NVIDIA_DRIVER=0 2024-06-26T07:12:58.6708275Z ++ command -v nvidia-smi 2024-06-26T07:12:58.6708662Z + '[' -x /usr/bin/nvidia-smi ']' 2024-06-26T07:12:58.6709017Z + set +e 2024-06-26T07:12:58.6709514Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-06-26T07:13:00.6557035Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-06-26T07:13:00.6557698Z + NVIDIA_SMI_STATUS=0 2024-06-26T07:13:00.6558627Z + '[' 0 -ne 0 ']' 2024-06-26T07:13:00.6559006Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-06-26T07:13:00.6559380Z + HAS_NVIDIA_DRIVER=1 2024-06-26T07:13:00.6560130Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-06-26T07:13:00.6560844Z + set -e 2024-06-26T07:13:00.6561145Z + '[' 1 -eq 0 ']' 2024-06-26T07:13:00.6561706Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-06-26T07:13:00.6562393Z + post_install_nvidia_driver_common 2024-06-26T07:13:00.6563074Z + sudo modprobe nvidia 2024-06-26T07:13:00.6659001Z + echo 'After installing NVIDIA driver' 2024-06-26T07:13:00.6659481Z + lspci 2024-06-26T07:13:00.6659779Z After installing NVIDIA driver 2024-06-26T07:13:00.6737470Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-06-26T07:13:00.6738371Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-06-26T07:13:00.6739319Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-06-26T07:13:00.6740120Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-06-26T07:13:00.6740881Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. Device 8061 2024-06-26T07:13:00.6741631Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-06-26T07:13:00.6742348Z 00:1e.0 3D controller: NVIDIA Corporation Device 2237 (rev a1) 2024-06-26T07:13:00.6743122Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-06-26T07:13:00.6743708Z + lsmod 2024-06-26T07:13:00.6751662Z Module Size Used by 2024-06-26T07:13:00.6752257Z xt_nat 16384 0 2024-06-26T07:13:00.6752762Z ib_core 266240 0 2024-06-26T07:13:00.6753261Z nvidia_modeset 1339392 0 2024-06-26T07:13:00.6753744Z veth 16384 0 2024-06-26T07:13:00.6754111Z nvidia_uvm 4599808 0 2024-06-26T07:13:00.6754566Z nvidia 53989376 2 nvidia_uvm,nvidia_modeset 2024-06-26T07:13:00.6755052Z drm 421888 1 nvidia 2024-06-26T07:13:00.6755491Z i2c_core 77824 2 nvidia,drm 2024-06-26T07:13:00.6755965Z backlight 16384 1 nvidia_modeset 2024-06-26T07:13:00.6756398Z xt_conntrack 16384 1 2024-06-26T07:13:00.6756779Z ipt_MASQUERADE 16384 1 2024-06-26T07:13:00.6757211Z nf_nat_masquerade_ipv4 16384 1 ipt_MASQUERADE 2024-06-26T07:13:00.6757675Z nf_conntrack_netlink 49152 0 2024-06-26T07:13:00.6758106Z nfnetlink 16384 2 nf_conntrack_netlink 2024-06-26T07:13:00.6758563Z xfrm_user 45056 1 2024-06-26T07:13:00.6759034Z xfrm_algo 16384 1 xfrm_user 2024-06-26T07:13:00.6759681Z iptable_nat 16384 1 2024-06-26T07:13:00.6760188Z nf_conntrack_ipv4 16384 3 2024-06-26T07:13:00.6760757Z nf_defrag_ipv4 16384 1 nf_conntrack_ipv4 2024-06-26T07:13:00.6761391Z nf_nat_ipv4 16384 1 iptable_nat 2024-06-26T07:13:00.6762114Z nf_nat 32768 3 nf_nat_masquerade_ipv4,nf_nat_ipv4,xt_nat 2024-06-26T07:13:00.6763105Z nf_conntrack 155648 8 xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4,nf_nat,ipt_MASQUERADE,nf_nat_ipv4,xt_nat,nf_conntrack_netlink 2024-06-26T07:13:00.6764241Z xt_addrtype 16384 2 2024-06-26T07:13:00.6764622Z iptable_filter 16384 1 2024-06-26T07:13:00.6764991Z br_netfilter 24576 0 2024-06-26T07:13:00.6765395Z bridge 172032 1 br_netfilter 2024-06-26T07:13:00.6765833Z stp 16384 1 bridge 2024-06-26T07:13:00.6766255Z llc 16384 2 bridge,stp 2024-06-26T07:13:00.6766663Z overlay 86016 0 2024-06-26T07:13:00.6767124Z sunrpc 393216 1 2024-06-26T07:13:00.6767493Z dm_mirror 28672 0 2024-06-26T07:13:00.6767936Z dm_region_hash 20480 1 dm_mirror 2024-06-26T07:13:00.6768413Z dm_log 20480 2 dm_region_hash,dm_mirror 2024-06-26T07:13:00.6769031Z dm_mod 143360 2 dm_log,dm_mirror 2024-06-26T07:13:00.6769478Z dax 69632 1 dm_mod 2024-06-26T07:13:00.6769876Z crc32_pclmul 16384 0 2024-06-26T07:13:00.6770252Z ghash_clmulni_intel 16384 0 2024-06-26T07:13:00.6770629Z pcbc 16384 0 2024-06-26T07:13:00.6770992Z aesni_intel 188416 0 2024-06-26T07:13:00.6771379Z aes_x86_64 20480 1 aesni_intel 2024-06-26T07:13:00.6771821Z crypto_simd 16384 1 aesni_intel 2024-06-26T07:13:00.6772267Z glue_helper 16384 1 aesni_intel 2024-06-26T07:13:00.6772815Z cryptd 28672 3 crypto_simd,ghash_clmulni_intel,aesni_intel 2024-06-26T07:13:00.6773345Z mousedev 24576 0 2024-06-26T07:13:00.6773713Z evdev 20480 3 2024-06-26T07:13:00.6774076Z psmouse 32768 0 2024-06-26T07:13:00.6774443Z button 16384 0 2024-06-26T07:13:00.6774797Z ena 139264 0 2024-06-26T07:13:00.6775166Z ptp 20480 1 ena 2024-06-26T07:13:00.6775554Z pps_core 20480 1 ptp 2024-06-26T07:13:00.6775935Z crc32c_intel 24576 0 2024-06-26T07:13:00.6776580Z autofs4 49152 2 2024-06-26T07:13:00.6776937Z + modinfo nvidia 2024-06-26T07:13:00.6777623Z filename: /lib/modules/4.14.336-257.562.amzn2.x86_64/kernel/drivers/video/nvidia.ko 2024-06-26T07:13:00.6778308Z alias: char-major-195-* 2024-06-26T07:13:00.6778679Z version: 550.54.15 2024-06-26T07:13:00.6779018Z supported: external 2024-06-26T07:13:00.6779352Z license: NVIDIA 2024-06-26T07:13:00.6779715Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-06-26T07:13:00.6780191Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-06-26T07:13:00.6780642Z srcversion: 833721318DA517F0C2FEC97 2024-06-26T07:13:00.6781106Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-06-26T07:13:00.6781592Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-06-26T07:13:00.6782130Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-06-26T07:13:00.6782607Z depends: i2c-core,drm 2024-06-26T07:13:00.6782963Z retpoline: Y 2024-06-26T07:13:00.6783263Z name: nvidia 2024-06-26T07:13:00.6783806Z vermagic: 4.14.336-257.562.amzn2.x86_64 SMP mod_unload modversions 2024-06-26T07:13:00.6784453Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-06-26T07:13:00.6785223Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-06-26T07:13:00.6785806Z parm: NVreg_ResmanDebugLevel:int 2024-06-26T07:13:00.6786243Z parm: NVreg_RmLogonRC:int 2024-06-26T07:13:00.6786671Z parm: NVreg_ModifyDeviceFiles:int 2024-06-26T07:13:00.6787120Z parm: NVreg_DeviceFileUID:int 2024-06-26T07:13:00.6787544Z parm: NVreg_DeviceFileGID:int 2024-06-26T07:13:00.6787982Z parm: NVreg_DeviceFileMode:int 2024-06-26T07:13:00.6788502Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-06-26T07:13:00.6789064Z parm: NVreg_UsePageAttributeTable:int 2024-06-26T07:13:00.6789538Z parm: NVreg_EnablePCIeGen3:int 2024-06-26T07:13:00.6789964Z parm: NVreg_EnableMSI:int 2024-06-26T07:13:00.6790486Z parm: NVreg_TCEBypassMode:int 2024-06-26T07:13:00.6790934Z parm: NVreg_EnableStreamMemOPs:int 2024-06-26T07:13:00.6791456Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-06-26T07:13:00.6792031Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-06-26T07:13:00.6792583Z parm: NVreg_EnableS0ixPowerManagement:int 2024-06-26T07:13:00.6793178Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-06-26T07:13:00.6793779Z parm: NVreg_DynamicPowerManagement:int 2024-06-26T07:13:00.6794383Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-06-26T07:13:00.6794968Z parm: NVreg_EnableGpuFirmware:int 2024-06-26T07:13:00.6795445Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-06-26T07:13:00.6796033Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-06-26T07:13:00.6796583Z parm: NVreg_EnableUserNUMAManagement:int 2024-06-26T07:13:00.6797100Z parm: NVreg_MemoryPoolSize:int 2024-06-26T07:13:00.6797552Z parm: NVreg_KMallocHeapMaxSize:int 2024-06-26T07:13:00.6798029Z parm: NVreg_VMallocHeapMaxSize:int 2024-06-26T07:13:00.6798496Z parm: NVreg_IgnoreMMIOCheck:int 2024-06-26T07:13:00.6798943Z parm: NVreg_NvLinkDisable:int 2024-06-26T07:13:00.6799436Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-06-26T07:13:00.6799957Z parm: NVreg_RegisterPCIDriver:int 2024-06-26T07:13:00.6800439Z parm: NVreg_EnableResizableBar:int 2024-06-26T07:13:00.6800923Z parm: NVreg_EnableDbgBreakpoint:int 2024-06-26T07:13:00.6801407Z parm: NVreg_EnableNonblockingOpen:int 2024-06-26T07:13:00.6801888Z parm: NVreg_RegistryDwords:charp 2024-06-26T07:13:00.6802393Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-06-26T07:13:00.6802863Z parm: NVreg_RmMsg:charp 2024-06-26T07:13:00.6803281Z parm: NVreg_GpuBlacklist:charp 2024-06-26T07:13:00.6803756Z parm: NVreg_TemporaryFilePath:charp 2024-06-26T07:13:00.6804232Z parm: NVreg_ExcludedGpus:charp 2024-06-26T07:13:00.6804675Z parm: NVreg_DmaRemapPeerMmio:int 2024-06-26T07:13:00.6805155Z parm: NVreg_RmNvlinkBandwidth:charp 2024-06-26T07:13:00.6805623Z parm: NVreg_ImexChannelCount:int 2024-06-26T07:13:00.6806078Z parm: rm_firmware_active:charp 2024-06-26T07:13:00.6806468Z + set +e 2024-06-26T07:13:00.6806769Z + nvidia-smi 2024-06-26T07:13:02.2359649Z Wed Jun 26 07:13:02 2024 2024-06-26T07:13:02.2360418Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:13:02.2361288Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-06-26T07:13:02.2380248Z |-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:13:02.2381130Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-06-26T07:13:02.2382036Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-06-26T07:13:02.2382721Z | | | MIG M. | 2024-06-26T07:13:02.2383257Z |=========================================+========================+======================| 2024-06-26T07:13:02.2499829Z | 0 NVIDIA A10G Off | 00000000:00:1E.0 Off | 0 | 2024-06-26T07:13:02.2500601Z | 0% 32C P0 57W / 300W | 0MiB / 23028MiB | 0% Default | 2024-06-26T07:13:02.2501242Z | | | N/A | 2024-06-26T07:13:02.2501921Z +-----------------------------------------+------------------------+----------------------+ 2024-06-26T07:13:02.2502677Z 2024-06-26T07:13:02.2503343Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:13:02.2504261Z | Processes: | 2024-06-26T07:13:02.2505126Z | GPU GI CI PID Type Process name GPU Memory | 2024-06-26T07:13:02.2505818Z | ID ID Usage | 2024-06-26T07:13:02.2506381Z |=========================================================================================| 2024-06-26T07:13:02.2507026Z | No running processes found | 2024-06-26T07:13:02.2507789Z +-----------------------------------------------------------------------------------------+ 2024-06-26T07:13:02.8770191Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-06-26T07:13:04.4493481Z NVIDIA A10G 2024-06-26T07:13:04.8907674Z + NVIDIA_SMI_STATUS=0 2024-06-26T07:13:04.8908382Z + '[' 0 -eq 0 ']' 2024-06-26T07:13:04.8908853Z + echo 'INFO: Ignoring allowed status 0' 2024-06-26T07:13:04.8909278Z + set -e 2024-06-26T07:13:04.8909575Z INFO: Ignoring allowed status 0 2024-06-26T07:13:04.8911826Z == Installing nvidia container toolkit for amzn2 == 2024-06-26T07:13:04.8913904Z + sudo yum install -y yum-utils 2024-06-26T07:13:05.1961559Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:13:06.6938131Z Package yum-utils-1.1.31-46.amzn2.0.1.noarch already installed and latest version 2024-06-26T07:13:06.6939555Z Nothing to do 2024-06-26T07:13:06.8474311Z + [[ amzn2 == \a\m\z\n\2\0\2\3 ]] 2024-06-26T07:13:06.8475417Z + YUM_REPO_URL=https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:13:06.8476516Z + sudo yum-config-manager --add-repo https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:13:07.1764054Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:13:07.2022371Z adding repo from: https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo 2024-06-26T07:13:07.2023734Z grabbing file https://nvidia.github.io/nvidia-docker/amzn2/nvidia-docker.repo to /etc/yum.repos.d/nvidia-docker.repo 2024-06-26T07:13:07.2024676Z repo saved to /etc/yum.repos.d/nvidia-docker.repo 2024-06-26T07:13:07.2142840Z + sudo yum install -y nvidia-docker2 2024-06-26T07:13:07.5392402Z Loaded plugins: extras_suggestions, langpacks, priorities, update-motd 2024-06-26T07:13:08.9991723Z Package nvidia-docker2-2.13.0-1.noarch already installed and latest version 2024-06-26T07:13:08.9992537Z Nothing to do 2024-06-26T07:13:09.1507033Z + sudo systemctl restart docker 2024-06-26T07:13:44.3137243Z Command completed after 1 attempt(s). 2024-06-26T07:13:44.3195907Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T07:13:44.3196669Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T07:13:44.3197356Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-06-26T07:13:44.3198020Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-06-26T07:13:44.3205825Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:13:44.3206329Z env: 2024-06-26T07:13:44.3206602Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:44.3207041Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:44.3207521Z ##[endgroup] 2024-06-26T07:13:44.5407192Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T07:13:44.5566833Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.7/site-packages (5.9.1) 2024-06-26T07:13:44.5665909Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.7/site-packages (11.525.84) 2024-06-26T07:13:44.6579855Z Prepare all required actions 2024-06-26T07:13:44.6580346Z Getting action download info 2024-06-26T07:13:44.7616466Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-06-26T07:13:44.9510005Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-06-26T07:13:45.0579202Z ##[group]Run ./.github/actions/download-build-artifacts 2024-06-26T07:13:45.0579700Z with: 2024-06-26T07:13:45.0580044Z name: linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-06-26T07:13:45.0580507Z s3-bucket: gha-artifacts 2024-06-26T07:13:45.0580852Z env: 2024-06-26T07:13:45.0581142Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:45.0581602Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:45.0582094Z ##[endgroup] 2024-06-26T07:13:45.0613717Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T07:13:45.0614180Z with: 2024-06-26T07:13:45.0614607Z name: linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-06-26T07:13:45.0615069Z s3-bucket: gha-artifacts 2024-06-26T07:13:45.0615421Z region: us-east-1 2024-06-26T07:13:45.0615731Z env: 2024-06-26T07:13:45.0616015Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:13:45.0616702Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:13:45.0617192Z ##[endgroup] 2024-06-26T07:13:45.4736570Z (node:15798) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T07:13:45.4737298Z 2024-06-26T07:13:45.4737556Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T07:13:45.4738304Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T07:13:45.4739323Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T07:13:45.5594690Z Found 1 objects with prefix pytorch/pytorch/9673646808/linux-focal-cuda12.4-py3.10-gcc9-sm86/ 2024-06-26T07:13:45.5595798Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T07:14:02.4632876Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T07:14:02.4639067Z Artifact download has finished successfully 2024-06-26T07:14:02.4790479Z ##[group]Run unzip -o artifacts.zip 2024-06-26T07:14:02.4790917Z unzip -o artifacts.zip 2024-06-26T07:14:02.4798706Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:14:02.4799204Z env: 2024-06-26T07:14:02.4799476Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:02.4799933Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:02.4800417Z ##[endgroup] 2024-06-26T07:14:02.4829150Z Archive: artifacts.zip 2024-06-26T07:14:02.4831161Z creating: dist/ 2024-06-26T07:14:04.4752361Z inflating: dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T07:14:04.4753223Z creating: build/custom_test_artifacts/ 2024-06-26T07:14:04.4753891Z creating: build/custom_test_artifacts/custom-op-build/ 2024-06-26T07:14:04.4754606Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-06-26T07:14:04.4755423Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:14:04.4759519Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:14:04.4760572Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-06-26T07:14:04.4761499Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:14:04.4762571Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:14:04.4763601Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:14:04.4764890Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:14:04.4766308Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:14:04.4767572Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:14:04.4768775Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:14:04.4770333Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:14:04.4771878Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:14:04.4773109Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:14:04.4774241Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:14:04.4775391Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:14:04.4776973Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:14:04.4777997Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:14:04.4778981Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:14:04.4812966Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:14:04.4852738Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:14:04.4854191Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:14:04.4899295Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:14:04.4902506Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:14:04.4904416Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:14:04.4906421Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:14:04.4907843Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:14:04.4909227Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:14:04.4910608Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:14:04.4911978Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:14:04.4913329Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:14:04.4914599Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:14:04.4915828Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:14:04.4917034Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:14:04.4918246Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:14:04.4919427Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:14:04.4920630Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:14:04.4978240Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:14:04.5049501Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:14:04.5051113Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:14:04.5052595Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:14:04.5053741Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:14:04.5054860Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-06-26T07:14:04.5055771Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-06-26T07:14:04.5057040Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-06-26T07:14:04.5058198Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-06-26T07:14:04.5059295Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-06-26T07:14:04.5060318Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-06-26T07:14:04.5061377Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-06-26T07:14:04.5062446Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-06-26T07:14:04.5063509Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-06-26T07:14:04.5064572Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-06-26T07:14:04.5065716Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-06-26T07:14:04.5075015Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-06-26T07:14:04.5202974Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-06-26T07:14:04.5204327Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-06-26T07:14:04.5205827Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-06-26T07:14:04.5207244Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-06-26T07:14:04.5208404Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-06-26T07:14:04.5209488Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-06-26T07:14:04.5210619Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-06-26T07:14:04.5211756Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-06-26T07:14:04.5212880Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-06-26T07:14:04.5214004Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-06-26T07:14:04.5215117Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-06-26T07:14:04.5227349Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-06-26T07:14:04.5307243Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-06-26T07:14:04.5308664Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:14:04.5309906Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:14:04.5311094Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-06-26T07:14:04.5312052Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-06-26T07:14:04.5313113Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-06-26T07:14:04.5314280Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-06-26T07:14:04.5315120Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-06-26T07:14:04.5315889Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-06-26T07:14:04.5316691Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-06-26T07:14:04.5421931Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-06-26T07:14:04.5483368Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-06-26T07:14:04.5484250Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-06-26T07:14:04.5484950Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-06-26T07:14:04.5485753Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:14:04.5490935Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:14:04.5492061Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-06-26T07:14:04.5492987Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:14:04.5494054Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:14:04.5495086Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:14:04.5496410Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:14:04.5497511Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:14:04.5498492Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:14:04.5499470Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:14:04.5500602Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:14:04.5501734Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:14:04.5502848Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:14:04.5504067Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:14:04.5505379Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:14:04.5506519Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:14:04.5507541Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:14:04.5508528Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:14:04.5543841Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:14:04.5583411Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:14:04.5586197Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:14:04.5630318Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:14:04.5631713Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:14:04.5633276Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:14:04.5634740Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:14:04.5636233Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:14:04.5637599Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:14:04.5638978Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:14:04.5640338Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:14:04.5641675Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:14:04.5642987Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:14:04.5644197Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:14:04.5645401Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:14:04.5646612Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:14:04.5663233Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:14:04.5664452Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:14:04.5710915Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:14:04.5782279Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:14:04.5783742Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:14:04.5785176Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:14:04.5786140Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:14:04.5787122Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-06-26T07:14:04.5788262Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-06-26T07:14:04.5789337Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-06-26T07:14:04.5790503Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-06-26T07:14:04.5791638Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-06-26T07:14:04.5792699Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-06-26T07:14:04.5793807Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-06-26T07:14:04.5794896Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-06-26T07:14:04.5795998Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-06-26T07:14:04.5797096Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-06-26T07:14:04.5798187Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-06-26T07:14:04.5809892Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-06-26T07:14:04.5872088Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-06-26T07:14:04.5873623Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:14:04.5875095Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:14:04.5876062Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-06-26T07:14:04.5877273Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-06-26T07:14:04.5878339Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-06-26T07:14:04.5879228Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-06-26T07:14:04.5880062Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-06-26T07:14:04.5880819Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-06-26T07:14:04.5881613Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-06-26T07:14:04.5925979Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-06-26T07:14:04.5926957Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-06-26T07:14:04.5927730Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-06-26T07:14:04.5928605Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-06-26T07:14:04.5933519Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T07:14:04.5934677Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-06-26T07:14:04.5935688Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T07:14:04.5937381Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T07:14:04.5938689Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T07:14:04.5939842Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T07:14:04.5941292Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T07:14:04.5942596Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T07:14:04.5943971Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T07:14:04.5945506Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T07:14:04.5947106Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T07:14:04.5948295Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T07:14:04.5949493Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T07:14:04.5950699Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T07:14:04.5951906Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T07:14:04.5952984Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-06-26T07:14:04.5954034Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-06-26T07:14:04.5987169Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-06-26T07:14:04.6026358Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-06-26T07:14:04.6027983Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-06-26T07:14:04.6071935Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-06-26T07:14:04.6074094Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-06-26T07:14:04.6076135Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-06-26T07:14:04.6078173Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-06-26T07:14:04.6079972Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-06-26T07:14:04.6081423Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-06-26T07:14:04.6082878Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-06-26T07:14:04.6084326Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-06-26T07:14:04.6085727Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-06-26T07:14:04.6087058Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-06-26T07:14:04.6088335Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-06-26T07:14:04.6089599Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-06-26T07:14:04.6090878Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-06-26T07:14:04.6092119Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-06-26T07:14:04.6093387Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-06-26T07:14:04.6151663Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-06-26T07:14:04.6222669Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-06-26T07:14:04.6224356Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-06-26T07:14:04.6225962Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-06-26T07:14:04.6227115Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-06-26T07:14:04.6228417Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-06-26T07:14:04.6229580Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-06-26T07:14:04.6230714Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-06-26T07:14:04.6231956Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-06-26T07:14:04.6233432Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-06-26T07:14:04.6234572Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-06-26T07:14:04.6235749Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-06-26T07:14:04.6237039Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-06-26T07:14:04.6238221Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-06-26T07:14:04.6239389Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-06-26T07:14:04.6240633Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-06-26T07:14:04.6241873Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-06-26T07:14:04.6354918Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-06-26T07:14:04.6356537Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-06-26T07:14:04.6358061Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-06-26T07:14:04.6359584Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-06-26T07:14:04.6360842Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-06-26T07:14:04.6362046Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-06-26T07:14:04.6363269Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-06-26T07:14:04.6364499Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-06-26T07:14:04.6365727Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-06-26T07:14:04.6366948Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-06-26T07:14:04.6368170Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-06-26T07:14:04.6380455Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-06-26T07:14:04.6433710Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-06-26T07:14:04.6435386Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T07:14:04.6436891Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-06-26T07:14:04.6437931Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-06-26T07:14:04.6439086Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-06-26T07:14:04.6440393Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-06-26T07:14:04.6441360Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-06-26T07:14:04.6442265Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-06-26T07:14:04.6443106Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-06-26T07:14:04.6443952Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-06-26T07:14:04.6540028Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-06-26T07:14:04.6582112Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-06-26T07:14:04.6582846Z creating: build/lib/ 2024-06-26T07:14:04.6667435Z inflating: build/lib/libprotobuf-lite.a 2024-06-26T07:14:04.7109360Z inflating: build/lib/libprotobuf.a 2024-06-26T07:14:04.7118755Z inflating: build/lib/libpthreadpool.a 2024-06-26T07:14:04.7126592Z inflating: build/lib/libcpuinfo.a 2024-06-26T07:14:04.7134414Z inflating: build/lib/libcpuinfo_internals.a 2024-06-26T07:14:04.7135081Z inflating: build/lib/libclog.a 2024-06-26T07:14:04.7153145Z inflating: build/lib/libnnpack.a 2024-06-26T07:14:04.7155305Z inflating: build/lib/libnnpack_reference_layers.a 2024-06-26T07:14:04.7218188Z inflating: build/lib/libgtest.a 2024-06-26T07:14:04.7290802Z inflating: build/lib/libbenchmark.a 2024-06-26T07:14:04.7353180Z inflating: build/lib/libasmjit.a 2024-06-26T07:14:04.7360298Z inflating: build/lib/libittnotify.a 2024-06-26T07:14:04.7388905Z inflating: build/lib/libtensorpipe_uv.a 2024-06-26T07:14:04.7512371Z inflating: build/lib/libgloo.a 2024-06-26T07:14:04.7533525Z inflating: build/lib/libfmt.a 2024-06-26T07:14:04.7535046Z inflating: build/lib/libcaffe2_nvrtc.so 2024-06-26T07:14:04.7535697Z inflating: build/lib/libfoxi_loader.a 2024-06-26T07:14:04.7625834Z inflating: build/lib/libc10.so 2024-06-26T07:14:04.7626954Z inflating: build/lib/libtorch_global_deps.so 2024-06-26T07:14:04.7645868Z inflating: build/lib/libpytorch_qnnpack.a 2024-06-26T07:14:04.7663252Z inflating: build/lib/libgmock.a 2024-06-26T07:14:04.7663853Z inflating: build/lib/libgtest_main.a 2024-06-26T07:14:04.7664689Z inflating: build/lib/libbenchmark_main.a 2024-06-26T07:14:04.8153335Z inflating: build/lib/libprotoc.a 2024-06-26T07:14:04.8525590Z inflating: build/lib/libgloo_cuda.a 2024-06-26T07:14:05.8302177Z inflating: build/lib/libdnnl.a 2024-06-26T07:14:05.8855014Z inflating: build/lib/libtensorpipe.a 2024-06-26T07:14:05.8908665Z inflating: build/lib/libc10_cuda.so 2024-06-26T07:14:05.8909224Z inflating: build/lib/libgmock_main.a 2024-06-26T07:14:06.0117571Z inflating: build/lib/libfbgemm.a 2024-06-26T07:14:06.0617783Z inflating: build/lib/libkineto.a 2024-06-26T07:14:06.0863878Z inflating: build/lib/libtensorpipe_cuda.a 2024-06-26T07:14:06.1043583Z inflating: build/lib/libXNNPACK.a 2024-06-26T07:14:06.1084087Z inflating: build/lib/libonnx_proto.a 2024-06-26T07:14:06.1752767Z inflating: build/lib/libonnx.a 2024-06-26T07:14:08.5182639Z inflating: build/lib/libtorch_cpu.so 2024-06-26T07:14:08.5186912Z inflating: build/lib/libunbox_lib.a 2024-06-26T07:14:08.5190964Z inflating: build/lib/libshm.so 2024-06-26T07:14:10.5158667Z inflating: build/lib/libtorch_cuda.so 2024-06-26T07:14:10.5159583Z inflating: build/lib/libtorch.so 2024-06-26T07:14:10.5161608Z inflating: build/lib/libc10d_cuda_test.so 2024-06-26T07:14:11.3743284Z inflating: build/lib/libtorch_cuda_linalg.so 2024-06-26T07:14:11.5646442Z inflating: build/lib/libtorch_python.so 2024-06-26T07:14:11.5713006Z inflating: build/lib/libtorchbind_test.so 2024-06-26T07:14:11.5732852Z inflating: build/lib/libjitbackend_test.so 2024-06-26T07:14:11.5758156Z inflating: build/lib/libbackend_with_compiler.so 2024-06-26T07:14:11.5792182Z inflating: build/lib/libnnapi_backend.so 2024-06-26T07:14:11.5792765Z creating: build/bin/ 2024-06-26T07:14:11.5841543Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-06-26T07:14:11.5891162Z inflating: build/bin/c10_DeviceGuard_test 2024-06-26T07:14:11.5940957Z inflating: build/bin/c10_Device_test 2024-06-26T07:14:11.5998043Z inflating: build/bin/c10_DispatchKeySet_test 2024-06-26T07:14:11.6050312Z inflating: build/bin/c10_Scalar_test 2024-06-26T07:14:11.6098318Z inflating: build/bin/c10_StreamGuard_test 2024-06-26T07:14:11.6148228Z inflating: build/bin/c10_SymInt_test 2024-06-26T07:14:11.6201433Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-06-26T07:14:11.6255420Z inflating: build/bin/c10_InlineStreamGuard_test 2024-06-26T07:14:11.6311011Z inflating: build/bin/c10_SizesAndStrides_test 2024-06-26T07:14:11.6380726Z inflating: build/bin/c10_cow_test 2024-06-26T07:14:11.6431961Z inflating: build/bin/c10_Bitset_test 2024-06-26T07:14:11.6480172Z inflating: build/bin/c10_ConstexprCrc_test 2024-06-26T07:14:11.6529646Z inflating: build/bin/c10_DeadlockDetection_test 2024-06-26T07:14:11.6578139Z inflating: build/bin/c10_Half_test 2024-06-26T07:14:11.6633270Z inflating: build/bin/c10_LeftRight_test 2024-06-26T07:14:11.6686862Z inflating: build/bin/c10_Metaprogramming_test 2024-06-26T07:14:11.6736465Z inflating: build/bin/c10_Synchronized_test 2024-06-26T07:14:11.6790141Z inflating: build/bin/c10_ThreadLocal_test 2024-06-26T07:14:11.6840724Z inflating: build/bin/c10_TypeIndex_test 2024-06-26T07:14:11.6890142Z inflating: build/bin/c10_TypeList_test 2024-06-26T07:14:11.6938440Z inflating: build/bin/c10_TypeTraits_test 2024-06-26T07:14:11.6989309Z inflating: build/bin/c10_accumulate_test 2024-06-26T07:14:11.7043250Z inflating: build/bin/c10_bfloat16_test 2024-06-26T07:14:11.7093250Z inflating: build/bin/c10_bit_cast_test 2024-06-26T07:14:11.7147746Z inflating: build/bin/c10_complex_math_test 2024-06-26T07:14:11.7201974Z inflating: build/bin/c10_complex_test 2024-06-26T07:14:11.7253271Z inflating: build/bin/c10_exception_test 2024-06-26T07:14:11.7302338Z inflating: build/bin/c10_flags_test 2024-06-26T07:14:11.7351424Z inflating: build/bin/c10_generic_math_test 2024-06-26T07:14:11.7511164Z inflating: build/bin/c10_intrusive_ptr_test 2024-06-26T07:14:11.7560580Z inflating: build/bin/c10_irange_test 2024-06-26T07:14:11.7612906Z inflating: build/bin/c10_lazy_test 2024-06-26T07:14:11.7668604Z inflating: build/bin/c10_logging_test 2024-06-26T07:14:11.7729098Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-06-26T07:14:11.7801842Z inflating: build/bin/c10_optional_test 2024-06-26T07:14:11.7854458Z inflating: build/bin/c10_registry_test 2024-06-26T07:14:11.8001802Z inflating: build/bin/c10_small_vector_test 2024-06-26T07:14:11.8052467Z inflating: build/bin/c10_ssize_test 2024-06-26T07:14:11.8103336Z inflating: build/bin/c10_string_util_test 2024-06-26T07:14:11.8152526Z inflating: build/bin/c10_tempfile_test 2024-06-26T07:14:11.8209986Z inflating: build/bin/c10_string_view_test 2024-06-26T07:14:11.8258268Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-06-26T07:14:11.8312435Z inflating: build/bin/c10_typeid_test 2024-06-26T07:14:11.8747743Z inflating: build/bin/protoc-3.13.0.0 2024-06-26T07:14:11.9182200Z inflating: build/bin/protoc 2024-06-26T07:14:11.9234174Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-06-26T07:14:11.9285805Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-06-26T07:14:11.9338309Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-06-26T07:14:11.9388119Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-06-26T07:14:11.9439598Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-06-26T07:14:11.9491465Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-06-26T07:14:11.9539345Z inflating: build/bin/c10_cuda_CUDATest 2024-06-26T07:14:11.9590867Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-06-26T07:14:11.9910809Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-06-26T07:14:12.0246169Z inflating: build/bin/vec_test_all_types_AVX512 2024-06-26T07:14:12.0593910Z inflating: build/bin/vec_test_all_types_AVX2 2024-06-26T07:14:12.0646027Z inflating: build/bin/FileStoreTest 2024-06-26T07:14:12.0699941Z inflating: build/bin/TCPStoreTest 2024-06-26T07:14:12.0752182Z inflating: build/bin/HashStoreTest 2024-06-26T07:14:12.0765233Z inflating: build/bin/ProcessGroupMPITest 2024-06-26T07:14:12.0818569Z inflating: build/bin/test_edge_op_registration 2024-06-26T07:14:12.0822461Z inflating: build/bin/torch_shm_manager 2024-06-26T07:14:12.0824221Z inflating: build/bin/example_allreduce 2024-06-26T07:14:12.0878088Z inflating: build/bin/test_dist_autograd 2024-06-26T07:14:12.0945900Z inflating: build/bin/test_cpp_rpc 2024-06-26T07:14:12.0949953Z inflating: build/bin/parallel_benchmark 2024-06-26T07:14:12.1015814Z inflating: build/bin/test_mobile_nnc 2024-06-26T07:14:12.1024303Z inflating: build/bin/aot_model_compiler_test 2024-06-26T07:14:12.1359062Z inflating: build/bin/test_lazy 2024-06-26T07:14:12.2509252Z inflating: build/bin/test_api 2024-06-26T07:14:12.2580890Z inflating: build/bin/Dict_test 2024-06-26T07:14:12.2631890Z inflating: build/bin/Dimname_test 2024-06-26T07:14:12.2694903Z inflating: build/bin/MaybeOwned_test 2024-06-26T07:14:12.2750805Z inflating: build/bin/NamedTensor_test 2024-06-26T07:14:12.2808326Z inflating: build/bin/apply_utils_test 2024-06-26T07:14:12.2865845Z inflating: build/bin/atest 2024-06-26T07:14:12.2927365Z inflating: build/bin/basic 2024-06-26T07:14:12.2980639Z inflating: build/bin/broadcast_test 2024-06-26T07:14:12.3030340Z inflating: build/bin/cpu_allocator_test 2024-06-26T07:14:12.3087971Z inflating: build/bin/cpu_generator_test 2024-06-26T07:14:12.3139203Z inflating: build/bin/cpu_profiling_allocator_test 2024-06-26T07:14:12.3230391Z inflating: build/bin/cpu_rng_test 2024-06-26T07:14:12.3279057Z inflating: build/bin/dispatch_key_set_test 2024-06-26T07:14:12.3328071Z inflating: build/bin/dlconvertor_test 2024-06-26T07:14:12.3384895Z inflating: build/bin/extension_backend_test 2024-06-26T07:14:12.3438682Z inflating: build/bin/half_test 2024-06-26T07:14:12.3532741Z inflating: build/bin/ivalue_test 2024-06-26T07:14:12.3580543Z inflating: build/bin/lazy_tensor_test 2024-06-26T07:14:12.3633965Z inflating: build/bin/math_kernel_test 2024-06-26T07:14:12.3687645Z inflating: build/bin/memory_format_test 2024-06-26T07:14:12.3739336Z inflating: build/bin/memory_overlapping_test 2024-06-26T07:14:12.3790705Z inflating: build/bin/mobile_memory_cleanup 2024-06-26T07:14:12.3845749Z inflating: build/bin/native_test 2024-06-26T07:14:12.3894559Z inflating: build/bin/operator_name_test 2024-06-26T07:14:12.3944280Z inflating: build/bin/operators_test 2024-06-26T07:14:12.3995053Z inflating: build/bin/packedtensoraccessor_test 2024-06-26T07:14:12.4060280Z inflating: build/bin/pow_test 2024-06-26T07:14:12.4116492Z inflating: build/bin/quantized_test 2024-06-26T07:14:12.4165588Z inflating: build/bin/reduce_ops_test 2024-06-26T07:14:12.4214749Z inflating: build/bin/reportMemoryUsage_test 2024-06-26T07:14:12.4269879Z inflating: build/bin/scalar_tensor_test 2024-06-26T07:14:12.4327087Z inflating: build/bin/scalar_test 2024-06-26T07:14:12.4377062Z inflating: build/bin/StorageUtils_test 2024-06-26T07:14:12.4428715Z inflating: build/bin/stride_properties_test 2024-06-26T07:14:12.4505077Z inflating: build/bin/tensor_iterator_test 2024-06-26T07:14:12.4558196Z inflating: build/bin/test_parallel 2024-06-26T07:14:12.4560980Z inflating: build/bin/thread_init_test 2024-06-26T07:14:12.4615646Z inflating: build/bin/type_ptr_test 2024-06-26T07:14:12.4674468Z inflating: build/bin/type_test 2024-06-26T07:14:12.4725563Z inflating: build/bin/undefined_tensor_test 2024-06-26T07:14:12.4727105Z inflating: build/bin/verify_api_visibility 2024-06-26T07:14:12.4794523Z inflating: build/bin/legacy_vmap_test 2024-06-26T07:14:12.4845572Z inflating: build/bin/weakref_test 2024-06-26T07:14:12.4895256Z inflating: build/bin/wrapdim_test 2024-06-26T07:14:12.4946369Z inflating: build/bin/xla_tensor_test 2024-06-26T07:14:12.5050591Z inflating: build/bin/List_test 2024-06-26T07:14:12.5109090Z inflating: build/bin/IListRef_test 2024-06-26T07:14:12.5227529Z inflating: build/bin/kernel_function_legacy_test 2024-06-26T07:14:12.5292440Z inflating: build/bin/KernelFunction_test 2024-06-26T07:14:12.5386645Z inflating: build/bin/kernel_function_test 2024-06-26T07:14:12.5511638Z inflating: build/bin/kernel_lambda_legacy_test 2024-06-26T07:14:12.5612583Z inflating: build/bin/kernel_lambda_test 2024-06-26T07:14:12.5672236Z inflating: build/bin/kernel_stackbased_test 2024-06-26T07:14:12.5766785Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-06-26T07:14:12.5815369Z inflating: build/bin/CppSignature_test 2024-06-26T07:14:12.5863083Z inflating: build/bin/op_allowlist_test 2024-06-26T07:14:12.5917603Z inflating: build/bin/backend_fallback_test 2024-06-26T07:14:12.6210118Z inflating: build/bin/op_registration_test 2024-06-26T07:14:12.6273073Z inflating: build/bin/inline_container_test 2024-06-26T07:14:12.6324338Z inflating: build/bin/cuda_apply_test 2024-06-26T07:14:12.6373681Z inflating: build/bin/cuda_allocator_test 2024-06-26T07:14:12.6427258Z inflating: build/bin/cuda_caching_host_allocator_test 2024-06-26T07:14:12.6485648Z inflating: build/bin/cuda_atomic_ops_test 2024-06-26T07:14:12.6553032Z inflating: build/bin/cuda_complex_math_test 2024-06-26T07:14:12.6611371Z inflating: build/bin/cuda_complex_test 2024-06-26T07:14:12.6659128Z inflating: build/bin/cuda_device_test 2024-06-26T07:14:12.6716110Z inflating: build/bin/cuda_cub_test 2024-06-26T07:14:12.6765582Z inflating: build/bin/cuda_dlconvertor_test 2024-06-26T07:14:12.6828597Z inflating: build/bin/cuda_distributions_test 2024-06-26T07:14:12.6883862Z inflating: build/bin/cuda_generator_test 2024-06-26T07:14:12.6932605Z inflating: build/bin/cuda_half_test 2024-06-26T07:14:12.6983249Z inflating: build/bin/cuda_integer_divider_test 2024-06-26T07:14:12.7031024Z inflating: build/bin/cuda_optional_test 2024-06-26T07:14:12.7081351Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-06-26T07:14:12.7133223Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-06-26T07:14:12.7181023Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-06-26T07:14:12.7240535Z inflating: build/bin/cuda_stream_test 2024-06-26T07:14:12.7288551Z inflating: build/bin/cuda_cudnn_test 2024-06-26T07:14:12.7340345Z inflating: build/bin/cuda_vectorized_test 2024-06-26T07:14:12.7354446Z inflating: build/bin/tutorial_tensorexpr 2024-06-26T07:14:12.7418871Z inflating: build/bin/ProcessGroupGlooTest 2024-06-26T07:14:12.7474153Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-06-26T07:14:12.7535550Z inflating: build/bin/ProcessGroupNCCLTest 2024-06-26T07:14:12.7596685Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-06-26T07:14:12.8406842Z inflating: build/bin/test_tensorexpr 2024-06-26T07:14:12.8970666Z inflating: build/bin/test_jit 2024-06-26T07:14:12.8971169Z creating: .additional_ci_files/ 2024-06-26T07:14:12.9024737Z inflating: .additional_ci_files/test-times.json 2024-06-26T07:14:12.9238104Z inflating: .additional_ci_files/test-class-times.json 2024-06-26T07:14:12.9266111Z ##[group]Run rm artifacts.zip 2024-06-26T07:14:12.9266490Z rm artifacts.zip 2024-06-26T07:14:12.9274093Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:14:12.9274590Z env: 2024-06-26T07:14:12.9274885Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:12.9275326Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:12.9275811Z ##[endgroup] 2024-06-26T07:14:12.9960961Z ##[group]Run df -H 2024-06-26T07:14:12.9961296Z df -H 2024-06-26T07:14:12.9969085Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:14:12.9969591Z env: 2024-06-26T07:14:12.9969873Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:12.9970327Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:12.9970805Z ##[endgroup] 2024-06-26T07:14:12.9997558Z Filesystem Size Used Avail Use% Mounted on 2024-06-26T07:14:12.9998068Z devtmpfs 34G 0 34G 0% /dev 2024-06-26T07:14:12.9998531Z tmpfs 34G 1.1M 34G 1% /dev/shm 2024-06-26T07:14:12.9999220Z tmpfs 34G 455k 34G 1% /run 2024-06-26T07:14:12.9999711Z tmpfs 34G 0 34G 0% /sys/fs/cgroup 2024-06-26T07:14:13.0001681Z /dev/nvme0n1p1 162G 67G 96G 42% / 2024-06-26T07:14:13.0039813Z Prepare all required actions 2024-06-26T07:14:13.0040245Z Getting action download info 2024-06-26T07:14:13.1351825Z ##[group]Run ./.github/actions/download-td-artifacts 2024-06-26T07:14:13.1352285Z with: 2024-06-26T07:14:13.1352668Z env: 2024-06-26T07:14:13.1352942Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:13.1353393Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:13.1353867Z ##[endgroup] 2024-06-26T07:14:13.1384362Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T07:14:13.1384912Z with: 2024-06-26T07:14:13.1385177Z name: td_results 2024-06-26T07:14:13.1385492Z s3-bucket: gha-artifacts 2024-06-26T07:14:13.1385839Z region: us-east-1 2024-06-26T07:14:13.1386137Z env: 2024-06-26T07:14:13.1386401Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:13.1386848Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:13.1387323Z ##[endgroup] 2024-06-26T07:14:13.5591053Z (node:15823) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T07:14:13.5591758Z 2024-06-26T07:14:13.5592024Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T07:14:13.5592748Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T07:14:13.5593589Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T07:14:13.6310471Z Found 0 objects with prefix pytorch/pytorch/9673646808/td_results/ 2024-06-26T07:14:13.6318154Z Artifact download has finished successfully 2024-06-26T07:14:13.6447807Z ##[group]Run mkdir -p .additional_ci_files 2024-06-26T07:14:13.6460932Z mkdir -p .additional_ci_files 2024-06-26T07:14:13.6461510Z mv td_results.json .additional_ci_files/td_results.json 2024-06-26T07:14:13.6469585Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:14:13.6470087Z env: 2024-06-26T07:14:13.6470355Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:13.6470812Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:13.6471291Z ##[endgroup] 2024-06-26T07:14:13.6507328Z mv: cannot stat 'td_results.json': No such file or directory 2024-06-26T07:14:13.6521689Z ##[error]Process completed with exit code 1. 2024-06-26T07:14:13.6554403Z ##[group]Run .github/scripts/parse_ref.py 2024-06-26T07:14:13.6554936Z .github/scripts/parse_ref.py 2024-06-26T07:14:13.6562647Z shell: /usr/bin/bash -e {0} 2024-06-26T07:14:13.6563005Z env: 2024-06-26T07:14:13.6563291Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:13.6563755Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:13.6564239Z ##[endgroup] 2024-06-26T07:14:13.6790955Z Prepare all required actions 2024-06-26T07:14:13.6827580Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-06-26T07:14:13.6828032Z with: 2024-06-26T07:14:13.6828581Z github-token: *** 2024-06-26T07:14:13.6828886Z env: 2024-06-26T07:14:13.6829150Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:13.6829597Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:13.6830077Z ##[endgroup] 2024-06-26T07:14:13.6844704Z ##[group]Run set -eux 2024-06-26T07:14:13.6845035Z set -eux 2024-06-26T07:14:13.6845610Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-06-26T07:14:13.6853207Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:14:13.6853696Z env: 2024-06-26T07:14:13.6853970Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:13.6854418Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:13.6855055Z GITHUB_TOKEN: *** 2024-06-26T07:14:13.6855360Z ##[endgroup] 2024-06-26T07:14:13.6874608Z + python3 .github/scripts/get_workflow_job_id.py 9673646808 i-044844c31b6a1c4ce 2024-06-26T07:14:14.9518649Z setting job-id=26688921705 2024-06-26T07:14:14.9519480Z setting job-name=cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:14:14.9716987Z Prepare all required actions 2024-06-26T07:14:14.9717435Z Getting action download info 2024-06-26T07:14:15.1238842Z ##[group]Run ./.github/actions/filter-test-configs 2024-06-26T07:14:15.1239300Z with: 2024-06-26T07:14:15.1239751Z github-token: *** 2024-06-26T07:14:15.1240822Z test-matrix: {"include": [{"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]} 2024-06-26T07:14:15.1242420Z job-name: cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:14:15.1243097Z env: 2024-06-26T07:14:15.1243371Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:15.1243822Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:15.1244293Z ##[endgroup] 2024-06-26T07:14:15.1286567Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T07:14:15.1287383Z with: 2024-06-26T07:14:15.1287904Z shell: bash 2024-06-26T07:14:15.1288325Z timeout_minutes: 10 2024-06-26T07:14:15.1288769Z max_attempts: 5 2024-06-26T07:14:15.1289175Z retry_wait_seconds: 30 2024-06-26T07:14:15.1290297Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T07:14:15.1291478Z polling_interval_seconds: 1 2024-06-26T07:14:15.1291844Z warning_on_retry: true 2024-06-26T07:14:15.1292191Z continue_on_error: false 2024-06-26T07:14:15.1292528Z env: 2024-06-26T07:14:15.1292788Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:15.1293234Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:15.1293889Z GITHUB_TOKEN: *** 2024-06-26T07:14:15.1294200Z ##[endgroup] 2024-06-26T07:14:15.1786798Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T07:14:15.3935998Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T07:14:15.4086246Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.7/site-packages (2.27.1) 2024-06-26T07:14:15.4231196Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.7/site-packages (6.0.1) 2024-06-26T07:14:15.4240322Z Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2.0.12) 2024-06-26T07:14:15.4261257Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2024.6.2) 2024-06-26T07:14:15.4270108Z Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (3.7) 2024-06-26T07:14:15.4281704Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (1.26.19) 2024-06-26T07:14:16.1779920Z Command completed after 1 attempt(s). 2024-06-26T07:14:16.1822109Z ##[group]Run set -x 2024-06-26T07:14:16.1822442Z set -x 2024-06-26T07:14:16.1822737Z  2024-06-26T07:14:16.1823285Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T07:14:16.1823972Z # in runner workspace 2024-06-26T07:14:16.1824496Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-06-26T07:14:16.1832359Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:14:16.1832859Z env: 2024-06-26T07:14:16.1833128Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:16.1833578Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:16.1834058Z ##[endgroup] 2024-06-26T07:14:16.1855668Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-06-26T07:14:16.2069758Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T07:14:16.2070402Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T07:14:16.2070868Z echo "Job name: ${JOB_NAME}" 2024-06-26T07:14:16.2071257Z  2024-06-26T07:14:16.2071804Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T07:14:16.2072482Z # in runner workspace 2024-06-26T07:14:16.2073190Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-06-26T07:14:16.2073822Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-06-26T07:14:16.2074282Z  --job-name "${JOB_NAME}" \ 2024-06-26T07:14:16.2075504Z  --test-matrix "{"include": [{"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" \ 2024-06-26T07:14:16.2076749Z  --selected-test-configs "" \ 2024-06-26T07:14:16.2077206Z  --pr-number "${PR_NUMBER}" \ 2024-06-26T07:14:16.2077622Z  --tag "${TAG}" \ 2024-06-26T07:14:16.2078020Z  --event-name "${EVENT_NAME}" \ 2024-06-26T07:14:16.2078460Z  --schedule "${SCHEDULE}" \ 2024-06-26T07:14:16.2078886Z  --branch "${HEAD_BRANCH}" 2024-06-26T07:14:16.2086412Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:14:16.2086924Z env: 2024-06-26T07:14:16.2087198Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:16.2087651Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:16.2088321Z GITHUB_TOKEN: *** 2024-06-26T07:14:16.2088933Z JOB_NAME: cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:14:16.2089616Z PR_NUMBER: 2024-06-26T07:14:16.2089925Z TAG: ciflow/inductor/129470 2024-06-26T07:14:16.2090278Z EVENT_NAME: push 2024-06-26T07:14:16.2090588Z SCHEDULE: 2024-06-26T07:14:16.2090875Z HEAD_BRANCH: 2024-06-26T07:14:16.2091169Z ##[endgroup] 2024-06-26T07:14:16.2110766Z Workflow: inductor 2024-06-26T07:14:16.2111525Z Job name: cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:14:16.3971766Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-06-26T07:14:16.5316036Z ##[group]Run echo "Filtered matrix:" 2024-06-26T07:14:16.5316498Z echo "Filtered matrix:" 2024-06-26T07:14:16.5317665Z echo "{"include": [{"config": "inductor_timm", "shard": 1, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}, {"config": "inductor_timm", "shard": 2, "num_shards": 2, "runner": "linux.g5.4xlarge.nvidia.gpu"}]}" 2024-06-26T07:14:16.5318808Z  2024-06-26T07:14:16.5319077Z echo 2024-06-26T07:14:16.5319451Z echo "Is the current job unstable? False" 2024-06-26T07:14:16.5319905Z  2024-06-26T07:14:16.5320173Z echo 2024-06-26T07:14:16.5320533Z echo "Is keep-going label set? False" 2024-06-26T07:14:16.5320978Z  2024-06-26T07:14:16.5321253Z echo 2024-06-26T07:14:16.5321568Z echo "Renabled issues? " 2024-06-26T07:14:16.5329205Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:14:16.5329696Z env: 2024-06-26T07:14:16.5329970Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:16.5330418Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:16.5330899Z ##[endgroup] 2024-06-26T07:14:16.5350696Z Filtered matrix: 2024-06-26T07:14:16.5352180Z {include: [{config: inductor_timm, shard: 1, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}, {config: inductor_timm, shard: 2, num_shards: 2, runner: linux.g5.4xlarge.nvidia.gpu}]} 2024-06-26T07:14:16.5353409Z 2024-06-26T07:14:16.5353558Z Is the current job unstable? False 2024-06-26T07:14:16.5353849Z 2024-06-26T07:14:16.5354123Z Is keep-going label set? False 2024-06-26T07:14:16.5354383Z 2024-06-26T07:14:16.5354504Z Renabled issues? 2024-06-26T07:14:16.5392434Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T07:14:16.5393133Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T07:14:16.5399959Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T07:14:16.5400448Z env: 2024-06-26T07:14:16.5400722Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:16.5401168Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:16.5401763Z JOB_TIMEOUT: 240 2024-06-26T07:14:16.5402058Z ##[endgroup] 2024-06-26T07:14:16.5468649Z ##[group]Run set -x 2024-06-26T07:14:16.5469032Z set -x 2024-06-26T07:14:16.5469322Z  2024-06-26T07:14:16.5469670Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-06-26T07:14:16.5470202Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-06-26T07:14:16.5470758Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-06-26T07:14:16.5471270Z  TEST_COMMAND=.ci/onnx/test.sh 2024-06-26T07:14:16.5471684Z else 2024-06-26T07:14:16.5472015Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T07:14:16.5472439Z fi 2024-06-26T07:14:16.5472711Z  2024-06-26T07:14:16.5473171Z # detached container should get cleaned up by teardown_ec2_linux 2024-06-26T07:14:16.5473920Z # TODO: Stop building test binaries as part of the build phase 2024-06-26T07:14:16.5474593Z # Used for GPU_FLAG since that doesn't play nice 2024-06-26T07:14:16.5475186Z # shellcheck disable=SC2086,SC2090 2024-06-26T07:14:16.5475639Z container_name=$(docker run \ 2024-06-26T07:14:16.5476053Z  ${GPU_FLAG:-} \ 2024-06-26T07:14:16.5476422Z  -e BUILD_ENVIRONMENT \ 2024-06-26T07:14:16.5476819Z  -e PR_NUMBER \ 2024-06-26T07:14:16.5477175Z  -e GITHUB_ACTIONS \ 2024-06-26T07:14:16.5477570Z  -e GITHUB_REPOSITORY \ 2024-06-26T07:14:16.5477970Z  -e GITHUB_WORKFLOW \ 2024-06-26T07:14:16.5478360Z  -e GITHUB_JOB \ 2024-06-26T07:14:16.5478713Z  -e GITHUB_RUN_ID \ 2024-06-26T07:14:16.5479093Z  -e GITHUB_RUN_NUMBER \ 2024-06-26T07:14:16.5479493Z  -e GITHUB_RUN_ATTEMPT \ 2024-06-26T07:14:16.5479875Z  -e JOB_ID \ 2024-06-26T07:14:16.5480211Z  -e JOB_NAME \ 2024-06-26T07:14:16.5480557Z  -e BASE_SHA \ 2024-06-26T07:14:16.5480896Z  -e BRANCH \ 2024-06-26T07:14:16.5481235Z  -e SHA1 \ 2024-06-26T07:14:16.5481578Z  -e AWS_DEFAULT_REGION \ 2024-06-26T07:14:16.5481981Z  -e IN_WHEEL_TEST \ 2024-06-26T07:14:16.5482359Z  -e SHARD_NUMBER \ 2024-06-26T07:14:16.5482724Z  -e TEST_CONFIG \ 2024-06-26T07:14:16.5483090Z  -e NUM_TEST_SHARDS \ 2024-06-26T07:14:16.5483479Z  -e REENABLED_ISSUES \ 2024-06-26T07:14:16.5483891Z  -e CONTINUE_THROUGH_ERROR \ 2024-06-26T07:14:16.5484309Z  -e VERBOSE_TEST_LOGS \ 2024-06-26T07:14:16.5484701Z  -e NO_TEST_TIMEOUT \ 2024-06-26T07:14:16.5485082Z  -e NO_TD \ 2024-06-26T07:14:16.5485415Z  -e TD_DISTRIBUTED \ 2024-06-26T07:14:16.5485792Z  -e PR_LABELS \ 2024-06-26T07:14:16.5486200Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-06-26T07:14:16.5486652Z  -e SCCACHE_BUCKET \ 2024-06-26T07:14:16.5487042Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-06-26T07:14:16.5487448Z  -e XLA_CUDA \ 2024-06-26T07:14:16.5487849Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-06-26T07:14:16.5488357Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-06-26T07:14:16.5488868Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-06-26T07:14:16.5489378Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-06-26T07:14:16.5489840Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-06-26T07:14:16.5490257Z  -e DASHBOARD_TAG \ 2024-06-26T07:14:16.5490726Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-06-26T07:14:16.5491277Z  --security-opt seccomp=unconfined \ 2024-06-26T07:14:16.5491734Z  --cap-add=SYS_PTRACE \ 2024-06-26T07:14:16.5492123Z  --ipc=host \ 2024-06-26T07:14:16.5492469Z  --shm-size="${SHM_SIZE}" \ 2024-06-26T07:14:16.5492857Z  --tty \ 2024-06-26T07:14:16.5493176Z  --detach \ 2024-06-26T07:14:16.5493526Z  --name="${container_name}" \ 2024-06-26T07:14:16.5494039Z  --user jenkins \ 2024-06-26T07:14:16.5494579Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-06-26T07:14:16.5495139Z  -w /var/lib/jenkins/workspace \ 2024-06-26T07:14:16.5495604Z  "${DOCKER_IMAGE}" 2024-06-26T07:14:16.5495965Z ) 2024-06-26T07:14:16.5496599Z # Propagate download.pytorch.org IP to container 2024-06-26T07:14:16.5497527Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-06-26T07:14:16.5498493Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-06-26T07:14:16.5499394Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-06-26T07:14:16.5506752Z shell: /usr/bin/bash -e {0} 2024-06-26T07:14:16.5507102Z env: 2024-06-26T07:14:16.5507371Z GIT_DEFAULT_BRANCH: main 2024-06-26T07:14:16.5507816Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:16.5508433Z BUILD_ENVIRONMENT: linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-06-26T07:14:16.5508933Z PR_NUMBER: 2024-06-26T07:14:16.5509248Z GITHUB_REPOSITORY: pytorch/pytorch 2024-06-26T07:14:16.5509650Z GITHUB_WORKFLOW: inductor 2024-06-26T07:14:16.5510003Z GITHUB_JOB: test 2024-06-26T07:14:16.5510315Z GITHUB_RUN_ID: 9673646808 2024-06-26T07:14:16.5510664Z GITHUB_RUN_NUMBER: 76977 2024-06-26T07:14:16.5511000Z GITHUB_RUN_ATTEMPT: 1 2024-06-26T07:14:16.5511329Z JOB_ID: 26688921705 2024-06-26T07:14:16.5511949Z JOB_NAME: cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:14:16.5512624Z BRANCH: 2024-06-26T07:14:16.5512949Z SHA1: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:14:16.5513459Z BASE_SHA: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:14:16.5513924Z TEST_CONFIG: inductor_timm 2024-06-26T07:14:16.5514276Z SHARD_NUMBER: 1 2024-06-26T07:14:16.5514571Z NUM_TEST_SHARDS: 2 2024-06-26T07:14:16.5514899Z REENABLED_ISSUES: 2024-06-26T07:14:16.5515233Z CONTINUE_THROUGH_ERROR: False 2024-06-26T07:14:16.5515608Z VERBOSE_TEST_LOGS: False 2024-06-26T07:14:16.5515955Z NO_TEST_TIMEOUT: False 2024-06-26T07:14:16.5516284Z NO_TD: False 2024-06-26T07:14:16.5516582Z TD_DISTRIBUTED: False 2024-06-26T07:14:16.5516986Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-06-26T07:14:16.5517474Z SCCACHE_S3_KEY_PREFIX: inductor 2024-06-26T07:14:16.5517852Z SHM_SIZE: 2g 2024-06-26T07:14:16.5518930Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:14:16.5520079Z XLA_CUDA: 2024-06-26T07:14:16.5520563Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:14:16.5521171Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-06-26T07:14:16.5521605Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-06-26T07:14:16.5522001Z DASHBOARD_TAG: 2024-06-26T07:14:16.5522515Z HUGGING_FACE_HUB_TOKEN: *** 2024-06-26T07:14:16.5522881Z ##[endgroup] 2024-06-26T07:14:16.5541928Z + [[ inductor_timm == \m\u\l\t\i\g\p\u ]] 2024-06-26T07:14:16.5542626Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *onnx* ]] 2024-06-26T07:14:16.5543140Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T07:14:16.5548425Z +++ nproc --ignore=2 2024-06-26T07:14:16.5562661Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_9673646808 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T07:14:27.3811971Z + container_name=2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T07:14:27.3813051Z + grep download.pytorch.org /etc/hosts 2024-06-26T07:14:27.3815258Z + docker exec -i 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b sudo bash -c '/bin/cat >> /etc/hosts' 2024-06-26T07:14:27.4377675Z + echo DOCKER_CONTAINER_ID=2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T07:14:27.4380314Z ++ echo dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl 2024-06-26T07:14:27.4382472Z + docker exec -t 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b sh -c 'pip install dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-06-26T07:14:27.8208484Z Processing ./dist/torch-2.5.0a0+gitb8c4c54-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+gitb8c4c54) 2024-06-26T07:14:28.1359275Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.13.1) 2024-06-26T07:14:28.1363974Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (4.12.2) 2024-06-26T07:14:28.1366343Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.12.1) 2024-06-26T07:14:28.1370047Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.8.8) 2024-06-26T07:14:28.1372141Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.1.4) 2024-06-26T07:14:28.1375015Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2024.2.0) 2024-06-26T07:14:28.1400988Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.3.0) 2024-06-26T07:14:28.1459371Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.21.2) 2024-06-26T07:14:28.1909930Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.1.5) 2024-06-26T07:14:28.2070105Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.3.0) 2024-06-26T07:14:29.5103174Z Installing collected packages: torch 2024-06-26T07:14:38.2569202Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 2024-06-26T07:14:38.2570548Z timm 0.9.7 requires torchvision, which is not installed. 2024-06-26T07:14:38.2571229Z Successfully installed torch-2.5.0a0+gitb8c4c54 2024-06-26T07:14:38.3464702Z ++ dirname .ci/pytorch/test.sh 2024-06-26T07:14:38.3470709Z + source .ci/pytorch/common.sh 2024-06-26T07:14:38.3473344Z +++ dirname .ci/pytorch/common.sh 2024-06-26T07:14:38.3479303Z ++ source .ci/pytorch/common_utils.sh 2024-06-26T07:14:38.3481525Z +++ declare -f -t trap_add 2024-06-26T07:14:38.3486333Z ++ set -ex 2024-06-26T07:14:38.3486919Z ++ [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *rocm* ]] 2024-06-26T07:14:38.3487486Z ++ BUILD_TEST_LIBTORCH=0 2024-06-26T07:14:38.3488479Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 != *rocm* ]] 2024-06-26T07:14:38.3490850Z ++ stat -c %u /var/lib/jenkins/workspace 2024-06-26T07:14:38.3498280Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-06-26T07:14:38.3498863Z + trap_add cleanup_workspace EXIT 2024-06-26T07:14:38.3499357Z + trap_add_cmd=cleanup_workspace 2024-06-26T07:14:38.3499848Z + shift 2024-06-26T07:14:38.3500174Z + for trap_add_name in "$@" 2024-06-26T07:14:38.3503266Z +++ trap -p EXIT 2024-06-26T07:14:38.3504180Z ++ eval 'extract_trap_cmd ' 2024-06-26T07:14:38.3504666Z +++ extract_trap_cmd 2024-06-26T07:14:38.3505150Z +++ printf '%s\n' '' 2024-06-26T07:14:38.3506893Z ++ printf '%s\n' cleanup_workspace 2024-06-26T07:14:38.3507493Z + trap -- ' 2024-06-26T07:14:38.3507927Z cleanup_workspace' EXIT 2024-06-26T07:14:38.3508459Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-06-26T07:14:38.9591281Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-06-26T07:14:38.9604910Z + echo 'Environment variables:' 2024-06-26T07:14:38.9605447Z Environment variables: 2024-06-26T07:14:38.9605821Z + env 2024-06-26T07:14:38.9610076Z INSTALLED_DB=yes 2024-06-26T07:14:38.9610701Z NV_LIBCUBLAS_VERSION=12.4.2.65-1 2024-06-26T07:14:38.9611352Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T07:14:38.9612087Z NV_NVML_DEV_VERSION=12.4.99-1 2024-06-26T07:14:38.9612904Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:14:38.9613715Z CONTINUE_THROUGH_ERROR=False 2024-06-26T07:14:38.9614412Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.20.5-1+cuda12.4 2024-06-26T07:14:38.9615097Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.20.5-1 2024-06-26T07:14:38.9615753Z BUILD_ENVIRONMENT=linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-06-26T07:14:38.9616654Z HOSTNAME=2f0c6f998c37 2024-06-26T07:14:38.9617783Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_69dce1e2-8cbd-4653-9ace-3b31190a2aa9 2024-06-26T07:14:38.9618663Z GITHUB_ACTION=__self 2024-06-26T07:14:38.9619110Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T07:14:38.9626540Z NVIDIA_REQUIRE_CUDA=cuda>=12.4 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 brand=tesla,driver>=535,driver<536 brand=unknown,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=geforce,driver>=535,driver<536 brand=geforcertx,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=titan,driver>=535,driver<536 brand=titanrtx,driver>=535,driver<536 2024-06-26T07:14:38.9632481Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-4=12.4.2.65-1 2024-06-26T07:14:38.9632986Z NV_NVTX_VERSION=12.4.99-1 2024-06-26T07:14:38.9633332Z GITHUB_RUN_NUMBER=76977 2024-06-26T07:14:38.9633664Z TEST_CONFIG=inductor_timm 2024-06-26T07:14:38.9634024Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T07:14:38.9634485Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T07:14:38.9634933Z NV_CUDA_CUDART_DEV_VERSION=12.4.99-1 2024-06-26T07:14:38.9635353Z NV_LIBCUSPARSE_VERSION=12.3.0.142-1 2024-06-26T07:14:38.9635793Z NV_LIBNPP_VERSION=12.2.5.2-1 2024-06-26T07:14:38.9636463Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-06-26T07:14:38.9637048Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T07:14:38.9637511Z GITHUB_REF_TYPE=tag 2024-06-26T07:14:38.9637824Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T07:14:38.9638196Z NCCL_VERSION=2.20.5-1 2024-06-26T07:14:38.9638583Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:14:38.9639024Z XLA_CUDA= 2024-06-26T07:14:38.9639512Z HUGGING_FACE_HUB_TOKEN=*** 2024-06-26T07:14:38.9643388Z *** 2024-06-26T07:14:38.9643791Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T07:14:38.9644330Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T07:14:38.9644783Z GITHUB_ACTIONS=true 2024-06-26T07:14:38.9645119Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:38.9645620Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-4=12.4.99-1 2024-06-26T07:14:38.9646151Z NV_LIBNPP_PACKAGE=libnpp-12-4=12.2.5.2-1 2024-06-26T07:14:38.9646609Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:14:38.9647109Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T07:14:38.9647603Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:14:38.9648397Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/129470 2024-06-26T07:14:38.9649104Z UCC_HOME=/usr 2024-06-26T07:14:38.9649435Z NV_LIBCUBLAS_DEV_VERSION=12.4.2.65-1 2024-06-26T07:14:38.9649836Z VERBOSE_TEST_LOGS=False 2024-06-26T07:14:38.9650167Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T07:14:38.9650617Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-4 2024-06-26T07:14:38.9651110Z GITHUB_REF=refs/tags/ciflow/inductor/129470 2024-06-26T07:14:38.9651566Z NV_CUDA_CUDART_VERSION=12.4.99-1 2024-06-26T07:14:38.9651931Z SHARD_NUMBER=1 2024-06-26T07:14:38.9652236Z GITHUB_REF_PROTECTED=false 2024-06-26T07:14:38.9652582Z HOME=/var/lib/jenkins 2024-06-26T07:14:38.9652944Z GITHUB_API_URL=https://api.github.com 2024-06-26T07:14:38.9653369Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T07:14:38.9653843Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T07:14:38.9654339Z SCCACHE_S3_KEY_PREFIX=inductor 2024-06-26T07:14:38.9654701Z CUDA_VERSION=12.4.0 2024-06-26T07:14:38.9655113Z NV_LIBCUBLAS_PACKAGE=libcublas-12-4=12.4.2.65-1 2024-06-26T07:14:38.9655548Z NUM_TEST_SHARDS=2 2024-06-26T07:14:38.9655841Z UCX_HOME=/usr 2024-06-26T07:14:38.9656672Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-4=12.4.0-1 2024-06-26T07:14:38.9657754Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_69dce1e2-8cbd-4653-9ace-3b31190a2aa9 2024-06-26T07:14:38.9658951Z JOB_NAME=cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:14:38.9660131Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_69dce1e2-8cbd-4653-9ace-3b31190a2aa9 2024-06-26T07:14:38.9661240Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T07:14:38.9661899Z GITHUB_EVENT_NAME=push 2024-06-26T07:14:38.9662222Z DASHBOARD_TAG= 2024-06-26T07:14:38.9662526Z GITHUB_RUN_ID=9673646808 2024-06-26T07:14:38.9662970Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-4=12.2.5.2-1 2024-06-26T07:14:38.9663498Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-4 2024-06-26T07:14:38.9664496Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_69dce1e2-8cbd-4653-9ace-3b31190a2aa9 2024-06-26T07:14:38.9665414Z GITHUB_ACTOR=pytorch-bot[bot] 2024-06-26T07:14:38.9665923Z NV_LIBNPP_DEV_VERSION=12.2.5.2-1 2024-06-26T07:14:38.9666290Z PR_NUMBER= 2024-06-26T07:14:38.9666567Z GITHUB_RUN_ATTEMPT=1 2024-06-26T07:14:38.9666901Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T07:14:38.9667332Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T07:14:38.9667786Z TERM=xterm 2024-06-26T07:14:38.9668117Z NV_LIBCUSPARSE_DEV_VERSION=12.3.0.142-1 2024-06-26T07:14:38.9668519Z INSTALLED_VISION=yes 2024-06-26T07:14:38.9668825Z BRANCH= 2024-06-26T07:14:38.9669098Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T07:14:38.9669494Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T07:14:38.9670051Z CUDA_PATH=/usr/local/cuda 2024-06-26T07:14:38.9670879Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T07:14:38.9671636Z GITHUB_SERVER_URL=https://github.com 2024-06-26T07:14:38.9672120Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T07:14:38.9672578Z REENABLED_ISSUES= 2024-06-26T07:14:38.9672873Z SHLVL=1 2024-06-26T07:14:38.9673130Z MAX_JOBS=14 2024-06-26T07:14:38.9673431Z NV_CUDA_LIB_VERSION=12.4.0-1 2024-06-26T07:14:38.9673778Z NVARCH=x86_64 2024-06-26T07:14:38.9674068Z GITHUB_ACTOR_ID=54816060 2024-06-26T07:14:38.9674514Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:14:38.9675053Z GITHUB_REF_NAME=ciflow/inductor/129470 2024-06-26T07:14:38.9675533Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-4 2024-06-26T07:14:38.9676187Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:14:38.9676756Z GITHUB_JOB=test 2024-06-26T07:14:38.9677141Z NV_LIBNCCL_PACKAGE=libnccl2=2.20.5-1+cuda12.4 2024-06-26T07:14:38.9677698Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T07:14:38.9678209Z NO_TEST_TIMEOUT=False 2024-06-26T07:14:38.9678532Z TD_DISTRIBUTED=False 2024-06-26T07:14:38.9678890Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.4.0-1 2024-06-26T07:14:38.9679322Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T07:14:38.9679734Z NV_NVPROF_VERSION=12.4.99-1 2024-06-26T07:14:38.9680094Z GITHUB_RETENTION_DAYS=90 2024-06-26T07:14:38.9680442Z OPENSSL_DIR=/opt/openssl 2024-06-26T07:14:38.9680791Z GITHUB_ACTION_REPOSITORY= 2024-06-26T07:14:38.9681844Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:14:38.9682978Z GITHUB_BASE_REF= 2024-06-26T07:14:38.9683294Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T07:14:38.9683657Z CI=true 2024-06-26T07:14:38.9683965Z NV_LIBNCCL_PACKAGE_VERSION=2.20.5-1 2024-06-26T07:14:38.9684371Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T07:14:38.9684739Z JOB_ID=26688921705 2024-06-26T07:14:38.9685058Z INSTALLED_PROTOBUF=yes 2024-06-26T07:14:38.9685383Z GITHUB_HEAD_REF= 2024-06-26T07:14:38.9685676Z GITHUB_ACTION_REF= 2024-06-26T07:14:38.9686101Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T07:14:38.9686567Z GITHUB_WORKFLOW=inductor 2024-06-26T07:14:38.9686912Z DEBIAN_FRONTEND=noninteractive 2024-06-26T07:14:38.9687817Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_69dce1e2-8cbd-4653-9ace-3b31190a2aa9 2024-06-26T07:14:38.9688640Z NO_TD=False 2024-06-26T07:14:38.9688939Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T07:14:38.9689298Z _=/usr/bin/env 2024-06-26T07:14:38.9689762Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-06-26T07:14:38.9759788Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-06-26T07:14:38.9761086Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-06-26T07:14:38.9762383Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-06-26T07:14:38.9763606Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-06-26T07:14:38.9764489Z + BUILD_DIR=build 2024-06-26T07:14:38.9764943Z + BUILD_RENAMED_DIR=build_renamed 2024-06-26T07:14:38.9765501Z + BUILD_BIN_DIR=build/bin 2024-06-26T07:14:38.9765947Z + SHARD_NUMBER=1 2024-06-26T07:14:38.9766361Z + NUM_TEST_SHARDS=2 2024-06-26T07:14:38.9766795Z + export VALGRIND=ON 2024-06-26T07:14:38.9767231Z + VALGRIND=ON 2024-06-26T07:14:38.9767900Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *clang9* ]] 2024-06-26T07:14:38.9768593Z + [[ 0 == \1 ]] 2024-06-26T07:14:38.9769024Z + [[ False == \1 ]] 2024-06-26T07:14:38.9769581Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 != *bazel* ]] 2024-06-26T07:14:38.9770111Z ++ realpath build/custom_test_artifacts 2024-06-26T07:14:38.9772453Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-06-26T07:14:38.9773628Z + [[ -n '' ]] 2024-06-26T07:14:38.9774108Z + echo 'Environment variables' 2024-06-26T07:14:38.9774830Z Environment variables 2024-06-26T07:14:38.9775278Z + env 2024-06-26T07:14:38.9778580Z INSTALLED_DB=yes 2024-06-26T07:14:38.9779120Z NV_LIBCUBLAS_VERSION=12.4.2.65-1 2024-06-26T07:14:38.9779641Z NVIDIA_VISIBLE_DEVICES=all 2024-06-26T07:14:38.9780179Z NV_NVML_DEV_VERSION=12.4.99-1 2024-06-26T07:14:38.9780988Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T07:14:38.9781713Z CONTINUE_THROUGH_ERROR=False 2024-06-26T07:14:38.9782472Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.20.5-1+cuda12.4 2024-06-26T07:14:38.9783223Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.20.5-1 2024-06-26T07:14:38.9784066Z BUILD_ENVIRONMENT=linux-focal-cuda12.4-py3.10-gcc9-sm86 2024-06-26T07:14:38.9784836Z HOSTNAME=2f0c6f998c37 2024-06-26T07:14:38.9786275Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_69dce1e2-8cbd-4653-9ace-3b31190a2aa9 2024-06-26T07:14:38.9787612Z GITHUB_ACTION=__self 2024-06-26T07:14:38.9788139Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T07:14:38.9794994Z NVIDIA_REQUIRE_CUDA=cuda>=12.4 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 brand=tesla,driver>=535,driver<536 brand=unknown,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=geforce,driver>=535,driver<536 brand=geforcertx,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=titan,driver>=535,driver<536 brand=titanrtx,driver>=535,driver<536 2024-06-26T07:14:38.9800819Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-4=12.4.2.65-1 2024-06-26T07:14:38.9801317Z NV_NVTX_VERSION=12.4.99-1 2024-06-26T07:14:38.9801666Z GITHUB_RUN_NUMBER=76977 2024-06-26T07:14:38.9802005Z TEST_CONFIG=inductor_timm 2024-06-26T07:14:38.9802375Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T07:14:38.9802834Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T07:14:38.9803284Z NV_CUDA_CUDART_DEV_VERSION=12.4.99-1 2024-06-26T07:14:38.9803714Z NV_LIBCUSPARSE_VERSION=12.3.0.142-1 2024-06-26T07:14:38.9804129Z NV_LIBNPP_VERSION=12.2.5.2-1 2024-06-26T07:14:38.9804551Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot] 2024-06-26T07:14:38.9805048Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-06-26T07:14:38.9805511Z GITHUB_REF_TYPE=tag 2024-06-26T07:14:38.9805832Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T07:14:38.9806202Z NCCL_VERSION=2.20.5-1 2024-06-26T07:14:38.9806587Z BASE_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:14:38.9807026Z XLA_CUDA= 2024-06-26T07:14:38.9807509Z HUGGING_FACE_HUB_TOKEN=*** 2024-06-26T07:14:38.9807906Z *** 2024-06-26T07:14:38.9808190Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-06-26T07:14:38.9808582Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T07:14:38.9808933Z GITHUB_ACTIONS=true 2024-06-26T07:14:38.9809257Z NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T07:14:38.9809733Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-4=12.4.99-1 2024-06-26T07:14:38.9810251Z NV_LIBNPP_PACKAGE=libnpp-12-4=12.2.5.2-1 2024-06-26T07:14:38.9810702Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:14:38.9811197Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-06-26T07:14:38.9811669Z GITHUB_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:14:38.9812482Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/inductor.yml@refs/tags/ciflow/inductor/129470 2024-06-26T07:14:38.9813359Z UCC_HOME=/usr 2024-06-26T07:14:38.9813769Z NV_LIBCUBLAS_DEV_VERSION=12.4.2.65-1 2024-06-26T07:14:38.9814175Z VERBOSE_TEST_LOGS=False 2024-06-26T07:14:38.9814512Z NVIDIA_PRODUCT_NAME=CUDA 2024-06-26T07:14:38.9814956Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-4 2024-06-26T07:14:38.9815450Z GITHUB_REF=refs/tags/ciflow/inductor/129470 2024-06-26T07:14:38.9815907Z NV_CUDA_CUDART_VERSION=12.4.99-1 2024-06-26T07:14:38.9816593Z SHARD_NUMBER=1 2024-06-26T07:14:38.9816904Z GITHUB_REF_PROTECTED=false 2024-06-26T07:14:38.9817251Z HOME=/var/lib/jenkins 2024-06-26T07:14:38.9817670Z GITHUB_API_URL=https://api.github.com 2024-06-26T07:14:38.9818116Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T07:14:38.9818579Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-06-26T07:14:38.9819053Z SCCACHE_S3_KEY_PREFIX=inductor 2024-06-26T07:14:38.9819414Z CUDA_VERSION=12.4.0 2024-06-26T07:14:38.9833222Z NV_LIBCUBLAS_PACKAGE=libcublas-12-4=12.4.2.65-1 2024-06-26T07:14:38.9833729Z NUM_TEST_SHARDS=2 2024-06-26T07:14:38.9834032Z UCX_HOME=/usr 2024-06-26T07:14:38.9834564Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-4=12.4.0-1 2024-06-26T07:14:38.9835632Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_69dce1e2-8cbd-4653-9ace-3b31190a2aa9 2024-06-26T07:14:38.9836806Z JOB_NAME=cuda12.4-py3.10-gcc9-sm86 / test (inductor_timm, 1, 2, linux.g5.4xlarge.nvidia.gpu) 2024-06-26T07:14:38.9837970Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_69dce1e2-8cbd-4653-9ace-3b31190a2aa9 2024-06-26T07:14:38.9839069Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T07:14:38.9839711Z GITHUB_EVENT_NAME=push 2024-06-26T07:14:38.9840041Z DASHBOARD_TAG= 2024-06-26T07:14:38.9840343Z GITHUB_RUN_ID=9673646808 2024-06-26T07:14:38.9840784Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-4=12.2.5.2-1 2024-06-26T07:14:38.9841296Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-4 2024-06-26T07:14:38.9842283Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_69dce1e2-8cbd-4653-9ace-3b31190a2aa9 2024-06-26T07:14:38.9843186Z GITHUB_ACTOR=pytorch-bot[bot] 2024-06-26T07:14:38.9843587Z NV_LIBNPP_DEV_VERSION=12.2.5.2-1 2024-06-26T07:14:38.9843947Z PR_NUMBER= 2024-06-26T07:14:38.9844226Z GITHUB_RUN_ATTEMPT=1 2024-06-26T07:14:38.9844533Z VALGRIND=ON 2024-06-26T07:14:38.9844822Z ANACONDA_PYTHON_VERSION=3.10 2024-06-26T07:14:38.9845251Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T07:14:38.9845703Z TERM=xterm 2024-06-26T07:14:38.9846028Z NV_LIBCUSPARSE_DEV_VERSION=12.3.0.142-1 2024-06-26T07:14:38.9846425Z INSTALLED_VISION=yes 2024-06-26T07:14:38.9846737Z BRANCH= 2024-06-26T07:14:38.9847013Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T07:14:38.9847404Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-06-26T07:14:38.9847810Z CUDA_PATH=/usr/local/cuda 2024-06-26T07:14:38.9848546Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T07:14:38.9849292Z GITHUB_SERVER_URL=https://github.com 2024-06-26T07:14:38.9849772Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-06-26T07:14:38.9850221Z REENABLED_ISSUES= 2024-06-26T07:14:38.9850510Z SHLVL=1 2024-06-26T07:14:38.9850761Z MAX_JOBS=14 2024-06-26T07:14:38.9851054Z NV_CUDA_LIB_VERSION=12.4.0-1 2024-06-26T07:14:38.9851405Z NVARCH=x86_64 2024-06-26T07:14:38.9851693Z GITHUB_ACTOR_ID=54816060 2024-06-26T07:14:38.9852134Z GITHUB_WORKFLOW_SHA=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T07:14:38.9852708Z GITHUB_REF_NAME=ciflow/inductor/129470 2024-06-26T07:14:38.9853181Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-4 2024-06-26T07:14:38.9853823Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T07:14:38.9854379Z GITHUB_JOB=test 2024-06-26T07:14:38.9854753Z NV_LIBNCCL_PACKAGE=libnccl2=2.20.5-1+cuda12.4 2024-06-26T07:14:38.9855298Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-06-26T07:14:38.9855798Z NO_TEST_TIMEOUT=False 2024-06-26T07:14:38.9856515Z TD_DISTRIBUTED=False 2024-06-26T07:14:38.9856983Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.4.0-1 2024-06-26T07:14:38.9857426Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T07:14:38.9857830Z NV_NVPROF_VERSION=12.4.99-1 2024-06-26T07:14:38.9858173Z GITHUB_RETENTION_DAYS=90 2024-06-26T07:14:38.9858516Z OPENSSL_DIR=/opt/openssl 2024-06-26T07:14:38.9858857Z GITHUB_ACTION_REPOSITORY= 2024-06-26T07:14:38.9859885Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:14:38.9860934Z GITHUB_BASE_REF= 2024-06-26T07:14:38.9861251Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-06-26T07:14:38.9861607Z CI=true 2024-06-26T07:14:38.9861916Z NV_LIBNCCL_PACKAGE_VERSION=2.20.5-1 2024-06-26T07:14:38.9862316Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T07:14:38.9862734Z JOB_ID=26688921705 2024-06-26T07:14:38.9863038Z INSTALLED_PROTOBUF=yes 2024-06-26T07:14:38.9863360Z GITHUB_HEAD_REF= 2024-06-26T07:14:38.9863654Z GITHUB_ACTION_REF= 2024-06-26T07:14:38.9864074Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T07:14:38.9864535Z GITHUB_WORKFLOW=inductor 2024-06-26T07:14:38.9864875Z DEBIAN_FRONTEND=noninteractive 2024-06-26T07:14:38.9865888Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_69dce1e2-8cbd-4653-9ace-3b31190a2aa9 2024-06-26T07:14:38.9866688Z NO_TD=False 2024-06-26T07:14:38.9866980Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T07:14:38.9867333Z _=/usr/bin/env 2024-06-26T07:14:38.9867663Z + echo 'Testing pytorch' 2024-06-26T07:14:38.9867997Z Testing pytorch 2024-06-26T07:14:38.9868308Z + export LANG=C.UTF-8 2024-06-26T07:14:38.9868628Z + LANG=C.UTF-8 2024-06-26T07:14:38.9868907Z + PR_NUMBER= 2024-06-26T07:14:38.9869221Z + [[ inductor_timm == \d\e\f\a\u\l\t ]] 2024-06-26T07:14:38.9869668Z + [[ inductor_timm == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-06-26T07:14:38.9870116Z + [[ inductor_timm == \s\l\o\w ]] 2024-06-26T07:14:38.9870698Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2024-06-26T07:14:38.9871399Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *cuda* ]] 2024-06-26T07:14:38.9871929Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T07:14:38.9872394Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-06-26T07:14:38.9872814Z + [[ inductor_timm == *crossref* ]] 2024-06-26T07:14:38.9873348Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *rocm* ]] 2024-06-26T07:14:38.9873964Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *xpu* ]] 2024-06-26T07:14:38.9874597Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:14:38.9875146Z + pip_install --user ninja==1.10.2 2024-06-26T07:14:38.9875668Z + pip install --progress-bar off --user ninja==1.10.2 2024-06-26T07:14:39.3421169Z Collecting ninja==1.10.2 2024-06-26T07:14:39.3579310Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-06-26T07:14:39.3737654Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-06-26T07:14:40.6777711Z Installing collected packages: ninja 2024-06-26T07:14:40.6843550Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-06-26T07:14:40.6846033Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-06-26T07:14:40.6895160Z Successfully installed ninja-1.10.2 2024-06-26T07:14:40.7837526Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:14:40.7839660Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:14:40.7841088Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *aarch64* ]] 2024-06-26T07:14:40.7841872Z + install_tlparse 2024-06-26T07:14:40.7842248Z + pip_install --user tlparse==0.3.7 2024-06-26T07:14:40.7842881Z + pip install --progress-bar off --user tlparse==0.3.7 2024-06-26T07:14:41.1385718Z Collecting tlparse==0.3.7 2024-06-26T07:14:41.1535220Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-06-26T07:14:41.1658351Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-06-26T07:14:42.4577867Z Installing collected packages: tlparse 2024-06-26T07:14:42.5132951Z Successfully installed tlparse-0.3.7 2024-06-26T07:14:42.6050638Z ++ python -m site --user-base 2024-06-26T07:14:42.6205996Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T07:14:42.6207905Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *asan* ]] 2024-06-26T07:14:42.6208643Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *-debug* ]] 2024-06-26T07:14:42.6209310Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:14:42.6210246Z + echo 'We are not in debug mode: linux-focal-cuda12.4-py3.10-gcc9-sm86. Expect the assertion to pass' 2024-06-26T07:14:42.6211382Z We are not in debug mode: linux-focal-cuda12.4-py3.10-gcc9-sm86. Expect the assertion to pass 2024-06-26T07:14:42.6212076Z + cd test 2024-06-26T07:14:42.6212590Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-06-26T07:14:44.1081671Z + [[ inductor_timm == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-06-26T07:14:44.1082248Z + [[ inductor_timm == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-06-26T07:14:44.1083013Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-06-26T07:14:44.1083517Z + pushd test 2024-06-26T07:14:44.1083847Z ~/workspace/test ~/workspace 2024-06-26T07:14:44.1084619Z ++ python -c 'import torch; print(torch.version.cuda)' 2024-06-26T07:14:45.6004333Z + CUDA_VERSION=12.4 2024-06-26T07:14:45.6004966Z + '[' 12.4 == 12.4 ']' 2024-06-26T07:14:45.6005330Z + ISCUDA124=cu124 2024-06-26T07:14:45.6005647Z + popd 2024-06-26T07:14:45.6005917Z ~/workspace 2024-06-26T07:14:45.6008255Z + DYNAMO_BENCHMARK_FLAGS=() 2024-06-26T07:14:45.6008655Z + [[ inductor_timm == *dynamo_eager* ]] 2024-06-26T07:14:45.6009167Z + [[ inductor_timm == *aot_eager* ]] 2024-06-26T07:14:45.6009721Z + [[ inductor_timm == *aot_inductor* ]] 2024-06-26T07:14:45.6010301Z + [[ inductor_timm == *inductor* ]] 2024-06-26T07:14:45.6010794Z + [[ inductor_timm != *perf* ]] 2024-06-26T07:14:45.6011264Z + DYNAMO_BENCHMARK_FLAGS+=(--inductor) 2024-06-26T07:14:45.6011689Z + [[ inductor_timm == *dynamic* ]] 2024-06-26T07:14:45.6012104Z + [[ inductor_timm == *cpu_inductor* ]] 2024-06-26T07:14:45.6012584Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-06-26T07:14:45.6039262Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *libtorch* ]] 2024-06-26T07:14:45.6039967Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *-bazel-* ]] 2024-06-26T07:14:45.6040621Z + cd test 2024-06-26T07:14:45.6041062Z + python -c 'import torch; print(torch.__config__.show())' 2024-06-26T07:14:46.9554320Z PyTorch built with: 2024-06-26T07:14:46.9554875Z - GCC 9.4 2024-06-26T07:14:46.9555298Z - C++ Version: 201703 2024-06-26T07:14:46.9556238Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T07:14:46.9557400Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T07:14:46.9558142Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T07:14:46.9558705Z - LAPACK is enabled (usually provided by MKL) 2024-06-26T07:14:46.9559209Z - NNPACK is enabled 2024-06-26T07:14:46.9559670Z - CPU capability usage: AVX2 2024-06-26T07:14:46.9560077Z - CUDA Runtime 12.4 2024-06-26T07:14:46.9560664Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2024-06-26T07:14:46.9561239Z - CuDNN 90.1 2024-06-26T07:14:46.9561603Z - Magma 2.6.1 2024-06-26T07:14:46.9569666Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.4, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-06-26T07:14:46.9577141Z 2024-06-26T07:14:47.2330223Z + cd test 2024-06-26T07:14:47.2330930Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-06-26T07:14:48.4378459Z ATen/Parallel: 2024-06-26T07:14:48.4378997Z at::get_num_threads() : 8 2024-06-26T07:14:48.4379538Z at::get_num_interop_threads() : 16 2024-06-26T07:14:48.4380127Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T07:14:48.4380624Z omp_get_max_threads() : 8 2024-06-26T07:14:48.4381544Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T07:14:48.4382346Z mkl_get_max_threads() : 8 2024-06-26T07:14:48.4382973Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T07:14:48.4383623Z std::thread::hardware_concurrency() : 16 2024-06-26T07:14:48.4384057Z Environment variables: 2024-06-26T07:14:48.4384404Z OMP_NUM_THREADS : [not set] 2024-06-26T07:14:48.4384767Z MKL_NUM_THREADS : [not set] 2024-06-26T07:14:48.4385136Z ATen parallel backend: OpenMP 2024-06-26T07:14:48.4385399Z 2024-06-26T07:14:48.6802340Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *aarch64* ]] 2024-06-26T07:14:48.6802987Z + [[ inductor_timm == *backward* ]] 2024-06-26T07:14:48.6803467Z + [[ inductor_timm == *xla* ]] 2024-06-26T07:14:48.6803873Z + [[ inductor_timm == *executorch* ]] 2024-06-26T07:14:48.6804329Z + [[ inductor_timm == \j\i\t\_\l\e\g\a\c\y ]] 2024-06-26T07:14:48.6805032Z + [[ linux-focal-cuda12.4-py3.10-gcc9-sm86 == *libtorch* ]] 2024-06-26T07:14:48.6805563Z + [[ inductor_timm == distributed ]] 2024-06-26T07:14:48.6806027Z + [[ inductor_timm == *inductor_distributed* ]] 2024-06-26T07:14:48.6806550Z + [[ inductor_timm == *inductor-halide* ]] 2024-06-26T07:14:48.6807114Z + [[ inductor_timm == *inductor-micro-benchmark* ]] 2024-06-26T07:14:48.6807613Z + [[ inductor_timm == *huggingface* ]] 2024-06-26T07:14:48.6808038Z + [[ inductor_timm == *timm* ]] 2024-06-26T07:14:48.6808413Z + install_torchvision 2024-06-26T07:14:48.6808741Z + local orig_preload 2024-06-26T07:14:48.6809056Z + local commit 2024-06-26T07:14:48.6809363Z ++ get_pinned_commit vision 2024-06-26T07:14:48.6809743Z ++ cat .github/ci_commit_pins/vision.txt 2024-06-26T07:14:48.6817067Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:14:48.6817657Z + orig_preload= 2024-06-26T07:14:48.6818070Z + '[' -n '' ']' 2024-06-26T07:14:48.6818900Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:14:48.6820337Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:14:48.9972519Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:14:48.9976011Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-sv6serlc 2024-06-26T07:14:48.9989671Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-sv6serlc 2024-06-26T07:14:50.3490387Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-06-26T07:14:50.3505844Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:14:51.5804721Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:14:51.7966944Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-06-26T07:14:54.1163772Z Preparing metadata (setup.py) ... [?25l- \ done 2024-06-26T07:14:54.1212608Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.21.2) 2024-06-26T07:14:54.1215937Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+gitb8c4c54) 2024-06-26T07:14:54.1220386Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-06-26T07:14:54.1419395Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-06-26T07:14:54.1426210Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-06-26T07:14:54.1429014Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.12.1) 2024-06-26T07:14:54.1431864Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-06-26T07:14:54.1435258Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-06-26T07:14:54.1438700Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.2.0) 2024-06-26T07:14:54.1864615Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-06-26T07:14:54.2015799Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-06-26T07:14:54.2087478Z Building wheels for collected packages: torchvision 2024-06-26T07:16:04.6430178Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ done 2024-06-26T07:16:04.6455996Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2023537 sha256=dde0d8b430c44a1c6241e367bf02647cf1e947cfe7e5d8756b97268c36356dc9 2024-06-26T07:16:04.6457901Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-06-26T07:16:04.6491314Z Successfully built torchvision 2024-06-26T07:16:05.8366640Z Installing collected packages: torchvision 2024-06-26T07:16:06.2154884Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-06-26T07:16:06.3295986Z + '[' -n '' ']' 2024-06-26T07:16:06.3296448Z + id=0 2024-06-26T07:16:06.3296763Z + test_dynamo_benchmark timm_models 0 2024-06-26T07:16:06.3298889Z ++ pwd 2024-06-26T07:16:06.3301091Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:16:06.3302217Z + local suite=timm_models 2024-06-26T07:16:06.3302746Z + shift 2024-06-26T07:16:06.3303046Z + local shard_id=0 2024-06-26T07:16:06.3303403Z + shift 2024-06-26T07:16:06.3303781Z + [[ inductor_timm == *perf_compare* ]] 2024-06-26T07:16:06.3304324Z + [[ inductor_timm == *perf* ]] 2024-06-26T07:16:06.3305070Z + [[ inductor_timm == *cpu_inductor* ]] 2024-06-26T07:16:06.3305605Z + [[ inductor_timm == *aot_inductor* ]] 2024-06-26T07:16:06.3310693Z + test_single_dynamo_benchmark inference timm_models 0 --inference --bfloat16 2024-06-26T07:16:06.3311313Z ++ pwd 2024-06-26T07:16:06.3311772Z + TEST_REPORTS_DIR=/var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:16:06.3312427Z + mkdir -p /var/lib/jenkins/workspace/test/test-reports 2024-06-26T07:16:06.3316776Z + local name=inference 2024-06-26T07:16:06.3317228Z + shift 2024-06-26T07:16:06.3317615Z + local suite=timm_models 2024-06-26T07:16:06.3318023Z + shift 2024-06-26T07:16:06.3318286Z + local shard_id=0 2024-06-26T07:16:06.3318609Z + shift 2024-06-26T07:16:06.3318883Z + partition_flags=() 2024-06-26T07:16:06.3319227Z + local partition_flags 2024-06-26T07:16:06.3319605Z + [[ -n 2 ]] 2024-06-26T07:16:06.3319912Z + [[ -n 0 ]] 2024-06-26T07:16:06.3320497Z + partition_flags=(--total-partitions "$NUM_TEST_SHARDS" --partition-id "$shard_id") 2024-06-26T07:16:06.3321150Z + [[ inductor_timm == *perf_compare* ]] 2024-06-26T07:16:06.3321584Z + [[ inductor_timm == *perf* ]] 2024-06-26T07:16:06.3321988Z + [[ inductor_timm == *aot_inductor* ]] 2024-06-26T07:16:06.3323557Z + python benchmarks/dynamo/timm_models.py --ci --accuracy --timing --explain --inductor --device cuda --inference --bfloat16 --total-partitions 2 --partition-id 0 --output /var/lib/jenkins/workspace/test/test-reports/inference_timm_models.csv 2024-06-26T07:16:12.0980890Z 2024-06-26T07:16:12.4100317Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:16:12.4100657Z 2024-06-26T07:16:12.5132383Z model.safetensors: 0% 0.00/95.5M [00:00 will be ignored 2024-06-26T07:34:32.2982356Z E0626 07:34:32.297000 139914609185408 torch/_dynamo/utils.py:1470] RMSE (res-fp64): 0.00937, (ref-fp64): 0.00257 and shape=torch.Size([768]). res.dtype: torch.float32, multiplier: 3.000000, tol: 0.010000 2024-06-26T07:34:32.2984763Z E0626 07:34:32.297000 139914609185408 torch/_dynamo/utils.py:1384] Accuracy failed for key name blocks.0.attn.proj.bias.grad 2024-06-26T07:34:32.3004808Z fail_accuracy 2024-06-26T07:34:32.3502299Z TIMING: entire_frame_compile:63.09836 code_gen:23.8073 inductor_compile:43.5287 backend_compile:52.66707 2024-06-26T07:34:32.3503588Z STATS: call_* op count: 1028 | FakeTensor.__torch_dispatch__:16296 | FakeTensorMode.__torch_dispatch__:94641 | ProxyTorchDispatchMode.__torch_dispatch__:21953 2024-06-26T07:34:32.3504716Z Dynamo produced 3 graphs covering 1028 ops with 7 graph breaks (5 unique) 2024-06-26T07:34:39.2217740Z 2024-06-26T07:34:40.1056344Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:34:40.1057265Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:34:40.1058129Z cuda train botnet26t_256 2024-06-26T07:35:25.5772184Z pass 2024-06-26T07:35:25.5874359Z TIMING: entire_frame_compile:26.48142 code_gen:21.259 inductor_compile:29.85743 backend_compile:23.24058 2024-06-26T07:35:25.5876677Z STATS: call_* op count: 341 | FakeTensor.__torch_dispatch__:5764 | FakeTensorMode.__torch_dispatch__:34145 | ProxyTorchDispatchMode.__torch_dispatch__:9029 2024-06-26T07:35:25.5877823Z Dynamo produced 2 graphs covering 341 ops with 6 graph breaks (5 unique) 2024-06-26T07:35:30.5509997Z 2024-06-26T07:35:34.3592621Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:35:34.3593599Z loading model: 0it [00:03, ?it/s] 2024-06-26T07:35:34.3594781Z cuda train cait_m36_384 2024-06-26T07:35:34.3598336Z Traceback (most recent call last): 2024-06-26T07:35:34.3599239Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2321, in validate_model 2024-06-26T07:35:34.3600430Z self.model_iter_fn(model, example_inputs) 2024-06-26T07:35:34.3601746Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 371, in forward_and_backward_pass 2024-06-26T07:35:34.3602715Z pred = mod(*cloned_inputs) 2024-06-26T07:35:34.3603911Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T07:35:34.3605123Z return self._call_impl(*args, **kwargs) 2024-06-26T07:35:34.3606068Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:35:34.3606892Z return forward_call(*args, **kwargs) 2024-06-26T07:35:34.3607733Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 361, in forward 2024-06-26T07:35:34.3608486Z x = self.forward_features(x) 2024-06-26T07:35:34.3609586Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 346, in forward_features 2024-06-26T07:35:34.3610578Z x = self.blocks(x) 2024-06-26T07:35:34.3611745Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T07:35:34.3612635Z return self._call_impl(*args, **kwargs) 2024-06-26T07:35:34.3613552Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:35:34.3614373Z return forward_call(*args, **kwargs) 2024-06-26T07:35:34.3615281Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/container.py", line 249, in forward 2024-06-26T07:35:34.3616430Z input = module(input) 2024-06-26T07:35:34.3617397Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T07:35:34.3618280Z return self._call_impl(*args, **kwargs) 2024-06-26T07:35:34.3619194Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:35:34.3620028Z return forward_call(*args, **kwargs) 2024-06-26T07:35:34.3620851Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 197, in forward 2024-06-26T07:35:34.3621714Z x = x + self.drop_path(self.gamma_1 * self.attn(self.norm1(x))) 2024-06-26T07:35:34.3622783Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1657, in _wrapped_call_impl 2024-06-26T07:35:34.3623660Z return self._call_impl(*args, **kwargs) 2024-06-26T07:35:34.3624573Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:35:34.3625531Z return forward_call(*args, **kwargs) 2024-06-26T07:35:34.3626352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/cait.py", line 151, in forward 2024-06-26T07:35:34.3627168Z x = (attn @ v).transpose(1, 2).reshape(B, N, C) 2024-06-26T07:35:34.3630406Z torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 21.98 GiB of which 16.44 MiB is free. Process 30126 has 21.95 GiB memory in use. Of the allocated memory 21.21 GiB is allocated by PyTorch, and 430.66 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables) 2024-06-26T07:35:34.3633092Z 2024-06-26T07:35:34.3633387Z The above exception was the direct cause of the following exception: 2024-06-26T07:35:34.3633835Z 2024-06-26T07:35:34.3633981Z Traceback (most recent call last): 2024-06-26T07:35:34.3634626Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 4229, in run 2024-06-26T07:35:34.3635277Z ) = runner.load_model( 2024-06-26T07:35:34.3635923Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 303, in load_model 2024-06-26T07:35:34.3636916Z self.validate_model(model, example_inputs) 2024-06-26T07:35:34.3637673Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/common.py", line 2323, in validate_model 2024-06-26T07:35:34.3638430Z raise RuntimeError("Eager run failed") from e 2024-06-26T07:35:34.3638885Z RuntimeError: Eager run failed 2024-06-26T07:35:34.3639145Z 2024-06-26T07:35:34.3639261Z eager_fail_to_run 2024-06-26T07:35:37.8777534Z 2024-06-26T07:35:38.6333276Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:35:38.6333795Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:35:38.6334239Z cuda train coat_lite_mini 2024-06-26T07:36:26.9824515Z pass 2024-06-26T07:36:26.9938293Z TIMING: entire_frame_compile:24.52786 code_gen:20.65243 inductor_compile:30.65597 backend_compile:21.73954 2024-06-26T07:36:26.9939594Z STATS: call_* op count: 480 | FakeTensor.__torch_dispatch__:6822 | FakeTensorMode.__torch_dispatch__:48449 | ProxyTorchDispatchMode.__torch_dispatch__:14338 2024-06-26T07:36:26.9940734Z Dynamo produced 2 graphs covering 480 ops with 6 graph breaks (5 unique) 2024-06-26T07:36:32.3222881Z 2024-06-26T07:36:33.8457455Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:36:33.8457930Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:36:33.8458373Z cuda train convit_base 2024-06-26T07:36:33.8967486Z WARNING:common:fp64 golden ref were not generated for convit_base. Setting accuracy check to cosine 2024-06-26T07:37:54.5021851Z pass 2024-06-26T07:37:54.5245017Z TIMING: entire_frame_compile:47.44122 code_gen:28.43686 inductor_compile:47.58835 backend_compile:40.38917 2024-06-26T07:37:54.5246440Z STATS: call_* op count: 1216 | FakeTensor.__torch_dispatch__:17010 | FakeTensorMode.__torch_dispatch__:118421 | ProxyTorchDispatchMode.__torch_dispatch__:40189 2024-06-26T07:37:54.5247792Z Dynamo produced 3 graphs covering 1216 ops with 7 graph breaks (5 unique) 2024-06-26T07:38:01.0118672Z 2024-06-26T07:38:01.8099703Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:38:01.8100211Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:38:01.8101733Z cuda train convmixer_768_32 2024-06-26T07:38:49.7176895Z pass 2024-06-26T07:38:49.7326482Z TIMING: entire_frame_compile:26.30923 code_gen:14.3893 inductor_compile:27.60549 backend_compile:23.89928 2024-06-26T07:38:49.7328215Z STATS: call_* op count: 235 | FakeTensor.__torch_dispatch__:12638 | FakeTensorMode.__torch_dispatch__:59863 | ProxyTorchDispatchMode.__torch_dispatch__:15494 2024-06-26T07:38:49.7329378Z Dynamo produced 2 graphs covering 235 ops with 5 graph breaks (4 unique) 2024-06-26T07:38:55.4604996Z 2024-06-26T07:38:57.0766694Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:38:57.0767172Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:38:57.0767611Z cuda train convnext_base 2024-06-26T07:39:50.6165029Z W0626 07:39:50.615000 139878239785600 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:41:20.1651698Z pass 2024-06-26T07:41:20.2145940Z TIMING: entire_frame_compile:108.30344 code_gen:45.51561 inductor_compile:79.58332 backend_compile:89.90128 2024-06-26T07:41:20.2147306Z STATS: call_* op count: 1621 | FakeTensor.__torch_dispatch__:29605 | FakeTensorMode.__torch_dispatch__:156830 | ProxyTorchDispatchMode.__torch_dispatch__:35464 2024-06-26T07:41:20.2148450Z Dynamo produced 3 graphs covering 1621 ops with 7 graph breaks (5 unique) 2024-06-26T07:41:29.7033771Z 2024-06-26T07:41:30.4118808Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:41:30.4119289Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:41:30.4119718Z cuda train crossvit_9_240 2024-06-26T07:42:20.9078911Z W0626 07:42:20.907000 139704375657088 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:43:30.2918628Z pass 2024-06-26T07:43:30.3052446Z TIMING: entire_frame_compile:93.83641 code_gen:39.42053 inductor_compile:68.43018 backend_compile:80.50867 2024-06-26T07:43:30.3056427Z STATS: call_* op count: 1403 | FakeTensor.__torch_dispatch__:25436 | FakeTensorMode.__torch_dispatch__:142104 | ProxyTorchDispatchMode.__torch_dispatch__:34361 2024-06-26T07:43:30.3057643Z Dynamo produced 3 graphs covering 1403 ops with 7 graph breaks (5 unique) 2024-06-26T07:43:38.7300058Z 2024-06-26T07:43:39.8757301Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:43:39.8757772Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:43:39.8758215Z cuda train cspdarknet53 2024-06-26T07:44:36.5605733Z W0626 07:44:36.559000 140529994261120 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:45:32.4243255Z pass 2024-06-26T07:45:32.4389391Z TIMING: entire_frame_compile:85.00239 code_gen:35.75922 inductor_compile:64.0298 backend_compile:70.19488 2024-06-26T07:45:32.4392235Z STATS: call_* op count: 1070 | FakeTensor.__torch_dispatch__:22900 | FakeTensorMode.__torch_dispatch__:115291 | ProxyTorchDispatchMode.__torch_dispatch__:24624 2024-06-26T07:45:32.4393640Z Dynamo produced 3 graphs covering 1070 ops with 7 graph breaks (5 unique) 2024-06-26T07:45:40.8648419Z 2024-06-26T07:45:42.3302641Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:45:42.3303262Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:45:42.3303808Z cuda train deit_base_distilled_patch16_224 2024-06-26T07:46:09.1317308Z W0626 07:46:09.130000 140087559344768 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:46:47.9098874Z pass 2024-06-26T07:46:47.9615361Z TIMING: entire_frame_compile:48.8812 code_gen:19.40464 inductor_compile:35.04724 backend_compile:41.60176 2024-06-26T07:46:47.9618558Z STATS: call_* op count: 841 | FakeTensor.__torch_dispatch__:14206 | FakeTensorMode.__torch_dispatch__:76972 | ProxyTorchDispatchMode.__torch_dispatch__:18136 2024-06-26T07:46:47.9619722Z Dynamo produced 3 graphs covering 841 ops with 7 graph breaks (5 unique) 2024-06-26T07:46:54.0806970Z 2024-06-26T07:46:55.3057227Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:46:55.3058113Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:46:55.3061475Z cuda train dla102 2024-06-26T07:48:09.3199741Z W0626 07:48:09.319000 140339871421056 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:49:38.4339658Z pass 2024-06-26T07:49:38.4552853Z TIMING: entire_frame_compile:127.05092 code_gen:55.53532 inductor_compile:94.8061 backend_compile:108.00944 2024-06-26T07:49:38.4554181Z STATS: call_* op count: 1389 | FakeTensor.__torch_dispatch__:34853 | FakeTensorMode.__torch_dispatch__:173859 | ProxyTorchDispatchMode.__torch_dispatch__:36994 2024-06-26T07:49:38.4555323Z Dynamo produced 3 graphs covering 1389 ops with 7 graph breaks (5 unique) 2024-06-26T07:49:49.4197676Z 2024-06-26T07:49:51.0110219Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:49:51.0110790Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:49:51.0111237Z cuda train dm_nfnet_f0 2024-06-26T07:50:44.6827978Z pass 2024-06-26T07:50:44.7330202Z TIMING: entire_frame_compile:27.73972 code_gen:17.07049 inductor_compile:30.60591 backend_compile:21.08505 2024-06-26T07:50:44.7331621Z STATS: call_* op count: 620 | FakeTensorMode.__torch_dispatch__:57475 | FakeTensor.__torch_dispatch__:11878 | ProxyTorchDispatchMode.__torch_dispatch__:16005 2024-06-26T07:50:44.7332753Z Dynamo produced 2 graphs covering 620 ops with 6 graph breaks (5 unique) 2024-06-26T07:50:50.1775191Z 2024-06-26T07:50:51.5809048Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:50:51.5809919Z loading model: 0it [00:01, ?it/s] 2024-06-26T07:50:51.5810769Z cuda train dpn107 2024-06-26T07:52:28.6853494Z pass 2024-06-26T07:52:28.7495525Z TIMING: entire_frame_compile:59.00725 code_gen:33.3198 inductor_compile:54.66618 backend_compile:48.63578 2024-06-26T07:52:28.7497435Z STATS: call_* op count: 746 | FakeTensor.__torch_dispatch__:18755 | FakeTensorMode.__torch_dispatch__:102809 | ProxyTorchDispatchMode.__torch_dispatch__:28636 2024-06-26T07:52:28.7498581Z Dynamo produced 2 graphs covering 746 ops with 6 graph breaks (5 unique) 2024-06-26T07:52:36.1851574Z 2024-06-26T07:52:37.0334854Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:52:37.0335322Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:52:37.0335763Z cuda train eca_botnext26ts_256 2024-06-26T07:53:22.0635573Z W0626 07:53:22.062000 139933540455040 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:53:52.9469143Z pass 2024-06-26T07:53:52.9583026Z TIMING: entire_frame_compile:53.6582 code_gen:29.0218 inductor_compile:45.99021 backend_compile:45.35613 2024-06-26T07:53:52.9585811Z STATS: call_* op count: 707 | FakeTensor.__torch_dispatch__:12859 | FakeTensorMode.__torch_dispatch__:67191 | ProxyTorchDispatchMode.__torch_dispatch__:15134 2024-06-26T07:53:52.9588121Z Dynamo produced 3 graphs covering 707 ops with 7 graph breaks (5 unique) 2024-06-26T07:53:59.0770994Z 2024-06-26T07:53:59.9462490Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:53:59.9463059Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:53:59.9463507Z cuda train eca_halonext26ts 2024-06-26T07:54:49.6755618Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T07:54:49.6756875Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 371, in torch_dynamo_resume_in_forward_and_backward_pass_at_369 2024-06-26T07:54:49.6757916Z pred = mod(*cloned_inputs) 2024-06-26T07:54:49.6759057Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:54:49.6760108Z return forward_call(*args, **kwargs) 2024-06-26T07:54:49.6761352Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/byobnet.py", line 1257, in forward 2024-06-26T07:54:49.6762282Z x = self.forward_features(x) 2024-06-26T07:54:49.6763414Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/byobnet.py", line 1249, in forward_features 2024-06-26T07:54:49.6764237Z x = self.stages(x) 2024-06-26T07:54:49.6765071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:54:49.6765910Z return forward_call(*args, **kwargs) 2024-06-26T07:54:49.6766768Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/byobnet.py", line 883, in forward 2024-06-26T07:54:49.6767531Z x = self.self_attn(x) 2024-06-26T07:54:49.6768360Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T07:54:49.6769199Z return forward_call(*args, **kwargs) 2024-06-26T07:54:49.6770071Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/layers/halo_attn.py", line 191, in forward 2024-06-26T07:54:49.6771155Z kv = kv.unfold(2, self.win_size, self.block_size).unfold(3, self.win_size, self.block_size).reshape( 2024-06-26T07:54:49.6771731Z 2024-06-26T07:54:49.9328134Z W0626 07:54:49.931000 140412346409600 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T07:55:20.4000097Z pass 2024-06-26T07:55:20.4071678Z TIMING: entire_frame_compile:55.6174 code_gen:32.51142 inductor_compile:50.49489 backend_compile:46.53207 2024-06-26T07:55:20.4072969Z STATS: call_* op count: 740 | FakeTensor.__torch_dispatch__:12680 | FakeTensorMode.__torch_dispatch__:69484 | ProxyTorchDispatchMode.__torch_dispatch__:15740 2024-06-26T07:55:20.4075205Z Dynamo produced 3 graphs covering 740 ops with 7 graph breaks (5 unique) 2024-06-26T07:55:26.5865407Z 2024-06-26T07:55:27.3200572Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:55:27.3201061Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:55:27.3201505Z cuda train ese_vovnet19b_dw 2024-06-26T07:55:58.1258014Z W0626 07:55:58.124000 139683082809984 torch/_logging/_internal.py:1040] [7/0] Profiler function will be ignored 2024-06-26T07:56:27.1793654Z pass 2024-06-26T07:56:27.1870286Z TIMING: entire_frame_compile:41.83577 code_gen:24.57748 inductor_compile:36.49835 backend_compile:34.77394 2024-06-26T07:56:27.1871677Z STATS: call_* op count: 459 | FakeTensor.__torch_dispatch__:9729 | FakeTensorMode.__torch_dispatch__:49643 | ProxyTorchDispatchMode.__torch_dispatch__:10392 2024-06-26T07:56:27.1872800Z Dynamo produced 4 graphs covering 459 ops with 7 graph breaks (5 unique) 2024-06-26T07:56:32.9277653Z 2024-06-26T07:56:33.7231377Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:56:33.7231863Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:56:33.7232284Z cuda train fbnetc_100 2024-06-26T07:57:20.8785974Z W0626 07:57:20.877000 140175650583168 torch/_logging/_internal.py:1040] [7/0] Profiler function will be ignored 2024-06-26T07:58:18.0977128Z pass 2024-06-26T07:58:18.1022682Z TIMING: entire_frame_compile:81.97811 code_gen:33.07573 inductor_compile:58.37261 backend_compile:68.14438 2024-06-26T07:58:18.1024067Z STATS: call_* op count: 993 | FakeTensor.__torch_dispatch__:20728 | FakeTensorMode.__torch_dispatch__:105778 | ProxyTorchDispatchMode.__torch_dispatch__:22083 2024-06-26T07:58:18.1025768Z Dynamo produced 4 graphs covering 993 ops with 7 graph breaks (5 unique) 2024-06-26T07:58:26.3379458Z 2024-06-26T07:58:27.3025014Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:58:27.3025598Z loading model: 0it [00:00, ?it/s] 2024-06-26T07:58:27.3026031Z cuda train fbnetv3_b 2024-06-26T07:59:52.9950402Z pass 2024-06-26T07:59:53.0031822Z TIMING: entire_frame_compile:50.7276 code_gen:31.73706 inductor_compile:55.98787 backend_compile:42.94146 2024-06-26T07:59:53.0033163Z STATS: call_* op count: 610 | FakeTensor.__torch_dispatch__:20660 | FakeTensorMode.__torch_dispatch__:91852 | ProxyTorchDispatchMode.__torch_dispatch__:23345 2024-06-26T07:59:53.0034303Z Dynamo produced 2 graphs covering 610 ops with 6 graph breaks (5 unique) 2024-06-26T08:00:00.7027206Z 2024-06-26T08:00:01.8045322Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:00:01.8045893Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:00:01.8048601Z cuda train gernet_l 2024-06-26T08:00:45.0537088Z pass 2024-06-26T08:00:45.0738503Z TIMING: entire_frame_compile:27.51113 code_gen:13.68481 inductor_compile:24.80245 backend_compile:22.16551 2024-06-26T08:00:45.0739821Z STATS: call_* op count: 410 | FakeTensor.__torch_dispatch__:9258 | FakeTensorMode.__torch_dispatch__:47041 | ProxyTorchDispatchMode.__torch_dispatch__:12116 2024-06-26T08:00:45.0740952Z Dynamo produced 2 graphs covering 410 ops with 6 graph breaks (5 unique) 2024-06-26T08:00:50.5787891Z 2024-06-26T08:00:51.4351198Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:00:51.4351881Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:00:51.4352529Z cuda train ghostnet_100 2024-06-26T08:01:58.5045220Z pass 2024-06-26T08:01:58.5091303Z TIMING: entire_frame_compile:40.99355 code_gen:27.51766 inductor_compile:45.2458 backend_compile:37.22964 2024-06-26T08:01:58.5092699Z STATS: call_* op count: 360 | FakeTensor.__torch_dispatch__:15561 | FakeTensorMode.__torch_dispatch__:77583 | ProxyTorchDispatchMode.__torch_dispatch__:19864 2024-06-26T08:01:58.5093842Z Dynamo produced 2 graphs covering 360 ops with 6 graph breaks (5 unique) 2024-06-26T08:02:05.2336839Z 2024-06-26T08:02:06.4014469Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:02:06.4015102Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:02:06.4015554Z cuda train gluon_inception_v3 2024-06-26T08:02:37.6066974Z W0626 08:02:37.605000 140323276219008 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T08:03:53.5677148Z pass 2024-06-26T08:03:53.5905864Z TIMING: entire_frame_compile:95.17149 inductor_compile:44.26437 backend_compile:74.49126 code_gen:20.76427 2024-06-26T08:03:53.5907188Z STATS: call_* op count: 1376 | FakeTensor.__torch_dispatch__:17669 | FakeTensorMode.__torch_dispatch__:126519 | ProxyTorchDispatchMode.__torch_dispatch__:33213 2024-06-26T08:03:53.5908331Z Dynamo produced 3 graphs covering 1376 ops with 7 graph breaks (5 unique) 2024-06-26T08:04:01.2381347Z 2024-06-26T08:04:02.1254240Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:04:02.1254709Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:04:02.1255154Z cuda train gmixer_24_224 2024-06-26T08:04:50.7789934Z pass 2024-06-26T08:04:50.8035975Z TIMING: entire_frame_compile:26.48005 code_gen:15.8048 inductor_compile:27.89037 backend_compile:23.37555 2024-06-26T08:04:50.8037902Z STATS: call_* op count: 684 | FakeTensor.__torch_dispatch__:12879 | FakeTensorMode.__torch_dispatch__:71357 | ProxyTorchDispatchMode.__torch_dispatch__:23778 2024-06-26T08:04:50.8039597Z Dynamo produced 2 graphs covering 684 ops with 6 graph breaks (5 unique) 2024-06-26T08:04:56.4186084Z 2024-06-26T08:04:57.2272512Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:04:57.2272991Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:04:57.2273430Z cuda train gmlp_s16_224 2024-06-26T08:05:44.8498622Z W0626 08:05:44.848000 140034051084928 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T08:06:59.9953118Z pass 2024-06-26T08:07:00.0154560Z TIMING: entire_frame_compile:94.40858 code_gen:36.00515 inductor_compile:68.33232 backend_compile:79.99911 2024-06-26T08:07:00.0155916Z STATS: call_* op count: 1458 | FakeTensor.__torch_dispatch__:28954 | FakeTensorMode.__torch_dispatch__:150087 | ProxyTorchDispatchMode.__torch_dispatch__:35364 2024-06-26T08:07:00.0157077Z Dynamo produced 3 graphs covering 1458 ops with 7 graph breaks (5 unique) 2024-06-26T08:07:08.9393092Z 2024-06-26T08:07:10.6068016Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:07:10.6068669Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:07:10.6069194Z cuda train hrnet_w18 2024-06-26T08:10:16.1944637Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T08:10:16.1946214Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 371, in torch_dynamo_resume_in_forward_and_backward_pass_at_369 2024-06-26T08:10:16.1947342Z pred = mod(*cloned_inputs) 2024-06-26T08:10:16.1948613Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:10:16.1949562Z return forward_call(*args, **kwargs) 2024-06-26T08:10:16.1950441Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/hrnet.py", line 792, in forward 2024-06-26T08:10:16.1951199Z y = self.forward_features(x) 2024-06-26T08:10:16.1952053Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/hrnet.py", line 770, in forward_features 2024-06-26T08:10:16.1952875Z yl = self.stages(x) 2024-06-26T08:10:16.1953770Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/hrnet.py", line 751, in stages 2024-06-26T08:10:16.1954745Z yl = self.stage2(xl) 2024-06-26T08:10:16.1955822Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:10:16.1956927Z return forward_call(*args, **kwargs) 2024-06-26T08:10:16.1957781Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/hrnet.py", line 506, in forward 2024-06-26T08:10:16.1958528Z x = module(x) 2024-06-26T08:10:16.1959329Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:10:16.1960158Z return forward_call(*args, **kwargs) 2024-06-26T08:10:16.1960995Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/hrnet.py", line 484, in forward 2024-06-26T08:10:16.1962002Z y = y + f(x[j]) 2024-06-26T08:10:16.1962198Z 2024-06-26T08:10:20.2289782Z pass 2024-06-26T08:10:20.2471262Z TIMING: entire_frame_compile:123.29639 code_gen:64.35734 inductor_compile:120.32434 backend_compile:111.81482 2024-06-26T08:10:20.2472591Z STATS: call_* op count: 1394 | FakeTensor.__torch_dispatch__:52796 | FakeTensorMode.__torch_dispatch__:272271 | ProxyTorchDispatchMode.__torch_dispatch__:71274 2024-06-26T08:10:20.2473740Z Dynamo produced 2 graphs covering 1394 ops with 5 graph breaks (4 unique) 2024-06-26T08:10:33.7213511Z 2024-06-26T08:10:34.8917296Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:10:34.8917904Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:10:34.8918377Z cuda train inception_v3 2024-06-26T08:11:07.7722334Z pass 2024-06-26T08:11:07.7931383Z TIMING: entire_frame_compile:24.71167 inductor_compile:5.65035 backend_compile:16.71044 2024-06-26T08:11:07.7932618Z STATS: call_* op count: 506 | FakeTensor.__torch_dispatch__:2985 | FakeTensorMode.__torch_dispatch__:50578 | ProxyTorchDispatchMode.__torch_dispatch__:20409 2024-06-26T08:11:07.7933772Z Dynamo produced 2 graphs covering 506 ops with 6 graph breaks (5 unique) 2024-06-26T08:11:12.2024359Z 2024-06-26T08:11:13.6203945Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:11:13.6204433Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:12:27.9443965Z cuda train jx_nest_base 2024-06-26T08:12:27.9445308Z W0626 08:12:27.943000 140266555093632 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T08:13:45.8998972Z pass 2024-06-26T08:13:45.9457404Z TIMING: entire_frame_compile:113.68414 code_gen:47.97132 inductor_compile:83.01145 backend_compile:96.69828 2024-06-26T08:13:45.9460559Z STATS: call_* op count: 1700 | FakeTensor.__torch_dispatch__:29051 | FakeTensorMode.__torch_dispatch__:170503 | ProxyTorchDispatchMode.__torch_dispatch__:42081 2024-06-26T08:13:45.9462850Z Dynamo produced 3 graphs covering 1700 ops with 7 graph breaks (5 unique) 2024-06-26T08:13:55.4586530Z 2024-06-26T08:13:56.0836627Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:13:56.0837360Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:13:56.0838002Z cuda train lcnet_050 2024-06-26T08:14:23.4535122Z pass 2024-06-26T08:14:23.4559276Z TIMING: entire_frame_compile:15.99007 code_gen:9.91091 inductor_compile:17.62677 backend_compile:13.59402 2024-06-26T08:14:23.4560937Z STATS: call_* op count: 168 | FakeTensor.__torch_dispatch__:6241 | FakeTensorMode.__torch_dispatch__:27828 | ProxyTorchDispatchMode.__torch_dispatch__:6908 2024-06-26T08:14:23.4562199Z Dynamo produced 2 graphs covering 168 ops with 6 graph breaks (5 unique) 2024-06-26T08:14:28.2880550Z 2024-06-26T08:14:29.1682928Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:14:29.1683408Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:14:29.1683848Z cuda train levit_128 2024-06-26T08:15:37.7772847Z skipping cudagraphs due to deterministic index put. Found from : 2024-06-26T08:15:37.7774058Z File "/var/lib/jenkins/workspace/benchmarks/dynamo/timm_models.py", line 371, in torch_dynamo_resume_in_forward_and_backward_pass_at_369 2024-06-26T08:15:37.7778151Z pred = mod(*cloned_inputs) 2024-06-26T08:15:37.7779530Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:15:37.7780801Z return forward_call(*args, **kwargs) 2024-06-26T08:15:37.7781690Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/levit.py", line 652, in forward 2024-06-26T08:15:37.7782640Z x = self.forward_features(x) 2024-06-26T08:15:37.7783861Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/levit.py", line 643, in forward_features 2024-06-26T08:15:37.7784904Z x = self.stages(x) 2024-06-26T08:15:37.7786266Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:15:37.7787425Z return forward_call(*args, **kwargs) 2024-06-26T08:15:37.7788484Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/levit.py", line 518, in forward 2024-06-26T08:15:37.7789314Z x = self.blocks(x) 2024-06-26T08:15:37.7790139Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:15:37.7790978Z return forward_call(*args, **kwargs) 2024-06-26T08:15:37.7791824Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/levit.py", line 456, in forward 2024-06-26T08:15:37.7792600Z x = x + self.drop_path1(self.attn(x)) 2024-06-26T08:15:37.7793506Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1668, in _call_impl 2024-06-26T08:15:37.7794336Z return forward_call(*args, **kwargs) 2024-06-26T08:15:37.7795173Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/levit.py", line 236, in forward 2024-06-26T08:15:37.7796059Z attn = q @ k * self.scale + self.get_attention_biases(x.device) 2024-06-26T08:15:37.7797120Z File "/opt/conda/envs/py_3.10/lib/python3.10/site-packages/timm/models/levit.py", line 211, in get_attention_biases 2024-06-26T08:15:37.7798053Z return self.attention_biases[:, self.attention_bias_idxs] 2024-06-26T08:15:37.7798448Z 2024-06-26T08:15:37.9322300Z W0626 08:15:37.931000 139793979753088 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T08:16:31.5766748Z pass 2024-06-26T08:16:31.5802620Z TIMING: entire_frame_compile:92.61247 code_gen:41.42341 inductor_compile:72.17463 backend_compile:79.82037 2024-06-26T08:16:31.5803948Z STATS: call_* op count: 1225 | FakeTensor.__torch_dispatch__:24276 | FakeTensorMode.__torch_dispatch__:135341 | ProxyTorchDispatchMode.__torch_dispatch__:32073 2024-06-26T08:16:31.5805693Z Dynamo produced 3 graphs covering 1225 ops with 7 graph breaks (5 unique) 2024-06-26T08:16:40.6973353Z 2024-06-26T08:16:41.8424227Z loading model: 0it [00:00, ?it/s] 2024-06-26T08:16:41.8425542Z loading model: 0it [00:01, ?it/s] 2024-06-26T08:16:41.8426483Z cuda train mixer_b16_224 2024-06-26T08:17:09.9071884Z W0626 08:17:09.906000 140521452999296 torch/_logging/_internal.py:1040] [6/0] Profiler function will be ignored 2024-06-26T08:17:47.6585476Z pass 2024-06-26T08:17:47.7001959Z TIMING: entire_frame_compile:49.4134 code_gen:19.89247 inductor_compile:35.45779 backend_compile:42.56522 2024-06-26T08:17:47.7003268Z STATS: call_* op count: 720 | FakeTensor.__torch_dispatch__:14404 | FakeTensorMode.__torch_dispatch__:74409 | ProxyTorchDispatchMode.__torch_dispatch__:17844 2024-06-26T08:17:47.7004396Z Dynamo produced 3 graphs covering 720 ops with 7 graph breaks (5 unique) 2024-06-26T08:17:51.0210815Z accuracy pass_rate=93.33% 2024-06-26T08:17:51.0215960Z calls_captured gmean=0.00x mean=833.533x 2024-06-26T08:17:51.0220785Z unique_graphs gmean=0.00x mean=2.533x 2024-06-26T08:17:51.0225082Z graph_breaks gmean=0.00x mean=6.267x 2024-06-26T08:17:51.0230201Z unique_graph_breaks gmean=0.00x mean=4.767x 2024-06-26T08:17:51.0234437Z autograd_captures gmean=0.00x mean=0.000x 2024-06-26T08:17:51.0238985Z autograd_compiles gmean=0.00x mean=0.000x 2024-06-26T08:17:51.0243504Z cudagraph_skips gmean=0.00x mean=0.133x 2024-06-26T08:17:52.5911448Z + python benchmarks/dynamo/check_accuracy.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy/cu124/inductor_timm_training.csv 2024-06-26T08:17:52.8460059Z adv_inception_v3 PASS 2024-06-26T08:17:52.8463788Z beit_base_patch16_224 XFAIL 2024-06-26T08:17:52.8469537Z botnet26t_256 PASS 2024-06-26T08:17:52.8474423Z cait_m36_384 XFAIL 2024-06-26T08:17:52.8479357Z coat_lite_mini PASS 2024-06-26T08:17:52.8484356Z convit_base PASS 2024-06-26T08:17:52.8489435Z convmixer_768_32 PASS 2024-06-26T08:17:52.8494216Z convnext_base PASS 2024-06-26T08:17:52.8499662Z crossvit_9_240 PASS 2024-06-26T08:17:52.8504684Z cspdarknet53 PASS 2024-06-26T08:17:52.8510026Z deit_base_distilled_patch16_224 PASS 2024-06-26T08:17:52.8515038Z dla102 PASS 2024-06-26T08:17:52.8519860Z dm_nfnet_f0 PASS 2024-06-26T08:17:52.8524656Z dpn107 PASS 2024-06-26T08:17:52.8529165Z eca_botnext26ts_256 PASS 2024-06-26T08:17:52.8533861Z eca_halonext26ts PASS 2024-06-26T08:17:52.8538573Z ese_vovnet19b_dw PASS 2024-06-26T08:17:52.8543122Z fbnetc_100 PASS 2024-06-26T08:17:52.8548085Z fbnetv3_b PASS 2024-06-26T08:17:52.8552533Z gernet_l PASS 2024-06-26T08:17:52.8557085Z ghostnet_100 PASS 2024-06-26T08:17:52.8561739Z gluon_inception_v3 PASS 2024-06-26T08:17:52.8566356Z gmixer_24_224 PASS 2024-06-26T08:17:52.8570884Z gmlp_s16_224 PASS 2024-06-26T08:17:52.8575460Z hrnet_w18 PASS 2024-06-26T08:17:52.8580317Z inception_v3 PASS 2024-06-26T08:17:52.8584787Z jx_nest_base PASS 2024-06-26T08:17:52.8589717Z lcnet_050 PASS 2024-06-26T08:17:52.8594173Z levit_128 PASS 2024-06-26T08:17:52.8598760Z mixer_b16_224 PASS 2024-06-26T08:17:52.8987494Z + python benchmarks/dynamo/check_graph_breaks.py --actual /var/lib/jenkins/workspace/test/test-reports/training_timm_models.csv --expected benchmarks/dynamo/ci_expected_accuracy/cu124/inductor_timm_training.csv 2024-06-26T08:17:53.1437448Z adv_inception_v3 PASS 2024-06-26T08:17:53.1441335Z beit_base_patch16_224 PASS 2024-06-26T08:17:53.1445838Z botnet26t_256 PASS 2024-06-26T08:17:53.1450259Z cait_m36_384 PASS 2024-06-26T08:17:53.1454723Z coat_lite_mini PASS 2024-06-26T08:17:53.1459383Z convit_base PASS 2024-06-26T08:17:53.1463819Z convmixer_768_32 PASS 2024-06-26T08:17:53.1468549Z convnext_base PASS 2024-06-26T08:17:53.1472807Z crossvit_9_240 PASS 2024-06-26T08:17:53.1477352Z cspdarknet53 PASS 2024-06-26T08:17:53.1481698Z deit_base_distilled_patch16_224 PASS 2024-06-26T08:17:53.1486105Z dla102 PASS 2024-06-26T08:17:53.1490432Z dm_nfnet_f0 PASS 2024-06-26T08:17:53.1494813Z dpn107 PASS 2024-06-26T08:17:53.1499516Z eca_botnext26ts_256 PASS 2024-06-26T08:17:53.1503748Z eca_halonext26ts PASS 2024-06-26T08:17:53.1508358Z ese_vovnet19b_dw PASS 2024-06-26T08:17:53.1512659Z fbnetc_100 PASS 2024-06-26T08:17:53.1517141Z fbnetv3_b PASS 2024-06-26T08:17:53.1521431Z gernet_l PASS 2024-06-26T08:17:53.1525811Z ghostnet_100 PASS 2024-06-26T08:17:53.1530229Z gluon_inception_v3 PASS 2024-06-26T08:17:53.1534463Z gmixer_24_224 PASS 2024-06-26T08:17:53.1539194Z gmlp_s16_224 PASS 2024-06-26T08:17:53.1543331Z hrnet_w18 PASS 2024-06-26T08:17:53.1547991Z inception_v3 PASS 2024-06-26T08:17:53.1552163Z jx_nest_base PASS 2024-06-26T08:17:53.1556510Z lcnet_050 PASS 2024-06-26T08:17:53.1560926Z levit_128 PASS 2024-06-26T08:17:53.1565278Z mixer_b16_224 PASS 2024-06-26T08:17:53.1942099Z + cleanup_workspace 2024-06-26T08:17:53.1943407Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-06-26T08:17:53.1944856Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-06-26T08:17:53.1946140Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-06-26T08:17:53.1946780Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-06-26T08:17:53.1947583Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-06-26T08:17:53.1948456Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-06-26T08:17:53.1949140Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-06-26T08:17:53.9808718Z ##[group]Run cat test/**/*_toprint.log || true 2024-06-26T08:17:53.9809246Z cat test/**/*_toprint.log || true 2024-06-26T08:17:53.9817195Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:17:53.9817714Z env: 2024-06-26T08:17:53.9817983Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:17:53.9818439Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:17:53.9819169Z DOCKER_CONTAINER_ID: 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:53.9819807Z ##[endgroup] 2024-06-26T08:17:53.9882199Z cat: test/**/*_toprint.log: No such file or directory 2024-06-26T08:17:53.9911016Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-06-26T08:17:53.9911488Z kill "$MONITOR_SCRIPT_PID" 2024-06-26T08:17:53.9919168Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:17:53.9919676Z env: 2024-06-26T08:17:53.9919949Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:17:53.9920458Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:17:53.9921196Z DOCKER_CONTAINER_ID: 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:53.9921858Z MONITOR_SCRIPT_PID: 15770 2024-06-26T08:17:53.9922205Z ##[endgroup] 2024-06-26T08:17:54.0108081Z Prepare all required actions 2024-06-26T08:17:54.0108543Z Getting action download info 2024-06-26T08:17:54.1800600Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-06-26T08:17:54.3414821Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-06-26T08:17:54.3415289Z with: 2024-06-26T08:17:54.3415777Z file-suffix: test-inductor_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688921705 2024-06-26T08:17:54.3416701Z s3-bucket: gha-artifacts 2024-06-26T08:17:54.3417034Z env: 2024-06-26T08:17:54.3417304Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:17:54.3417749Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:17:54.3418480Z DOCKER_CONTAINER_ID: 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:54.3419107Z ##[endgroup] 2024-06-26T08:17:54.3443372Z ##[group]Run # Remove any previous test jsons if they exist 2024-06-26T08:17:54.3444025Z # Remove any previous test jsons if they exist 2024-06-26T08:17:54.3444548Z rm -f test-jsons-*.zip 2024-06-26T08:17:54.3445066Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-06-26T08:17:54.3452950Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:17:54.3453475Z env: 2024-06-26T08:17:54.3453780Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:17:54.3454270Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:17:54.3455004Z DOCKER_CONTAINER_ID: 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:54.3455942Z FILE_SUFFIX: test-inductor_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688921705 2024-06-26T08:17:54.3456748Z ##[endgroup] 2024-06-26T08:17:54.3631720Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-06-26T08:17:54.3661139Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-06-26T08:17:54.3661862Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-06-26T08:17:54.3667872Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-06-26T08:17:54.3693916Z ##[group]Run # Remove any previous test reports if they exist 2024-06-26T08:17:54.3694571Z # Remove any previous test reports if they exist 2024-06-26T08:17:54.3695103Z rm -f test-reports-*.zip 2024-06-26T08:17:54.3695703Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-06-26T08:17:54.3703710Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:17:54.3704205Z env: 2024-06-26T08:17:54.3704487Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:17:54.3704948Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:17:54.3705816Z DOCKER_CONTAINER_ID: 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:54.3706676Z FILE_SUFFIX: test-inductor_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688921705 2024-06-26T08:17:54.3707283Z ##[endgroup] 2024-06-26T08:17:54.3863793Z adding: test/test-reports/inference_timm_models.csv (deflated 69%) 2024-06-26T08:17:54.3864626Z adding: test/test-reports/training_timm_models.csv (deflated 65%) 2024-06-26T08:17:54.3868487Z adding: test/test-reports/training_timm_model_graph_breaks.csv (deflated 97%) 2024-06-26T08:17:54.3869436Z adding: test/test-reports/training_timm_model_graph_break_deduped.csv (deflated 73%) 2024-06-26T08:17:54.3895408Z ##[group]Run # Remove any previous usage logs if they exist 2024-06-26T08:17:54.3896284Z # Remove any previous usage logs if they exist 2024-06-26T08:17:54.3896829Z rm -f logs-*.zip 2024-06-26T08:17:54.3897494Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-06-26T08:17:54.3898281Z # so check to see if the file exists first 2024-06-26T08:17:54.3898782Z if [ -f 'usage_log.txt' ]; then 2024-06-26T08:17:54.3899309Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-06-26T08:17:54.3899789Z fi 2024-06-26T08:17:54.3900303Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-06-26T08:17:54.3900937Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-06-26T08:17:54.3901431Z fi 2024-06-26T08:17:54.3908879Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:17:54.3909369Z env: 2024-06-26T08:17:54.3909647Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:17:54.3910095Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:17:54.3910834Z DOCKER_CONTAINER_ID: 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:54.3911733Z FILE_SUFFIX: test-inductor_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688921705 2024-06-26T08:17:54.3912339Z ##[endgroup] 2024-06-26T08:17:54.3974245Z adding: usage_log.txt (deflated 92%) 2024-06-26T08:17:54.4043471Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-06-26T08:17:54.4044201Z # Remove any previous debugging artifacts if they exist 2024-06-26T08:17:54.4044762Z rm -f debug-*.zip 2024-06-26T08:17:54.4045149Z if [ -d 'test/debug' ]; then 2024-06-26T08:17:54.4045649Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-06-26T08:17:54.4046121Z fi 2024-06-26T08:17:54.4052746Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:17:54.4053232Z env: 2024-06-26T08:17:54.4053506Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:17:54.4053953Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:17:54.4054684Z DOCKER_CONTAINER_ID: 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:54.4055529Z FILE_SUFFIX: test-inductor_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688921705 2024-06-26T08:17:54.4056391Z ##[endgroup] 2024-06-26T08:17:54.4133328Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T08:17:54.4133788Z with: 2024-06-26T08:17:54.4134076Z s3-bucket: gha-artifacts 2024-06-26T08:17:54.4134522Z s3-prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:17:54.4135035Z retention-days: 14 2024-06-26T08:17:54.4135508Z if-no-files-found: warn 2024-06-26T08:17:54.4135864Z path: test-jsons-*.zip 2024-06-26T08:17:54.4136350Z name: artifact 2024-06-26T08:17:54.4136654Z region: us-east-1 2024-06-26T08:17:54.4136952Z env: 2024-06-26T08:17:54.4137228Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:17:54.4137674Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:17:54.4138403Z DOCKER_CONTAINER_ID: 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:54.4139044Z ##[endgroup] 2024-06-26T08:17:54.7197203Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T08:17:54.7197975Z With the provided path, there will be 1 file uploaded 2024-06-26T08:17:54.7198579Z Uploading to s3 prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:17:54.7224680Z Starting upload of test-jsons-test-inductor_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688921705.zip 2024-06-26T08:17:54.9213956Z Finished upload of test-jsons-test-inductor_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688921705.zip 2024-06-26T08:17:54.9366255Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T08:17:54.9366702Z with: 2024-06-26T08:17:54.9366976Z s3-bucket: gha-artifacts 2024-06-26T08:17:54.9367400Z s3-prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:17:54.9367864Z retention-days: 14 2024-06-26T08:17:54.9368194Z if-no-files-found: error 2024-06-26T08:17:54.9368547Z path: test-reports-*.zip 2024-06-26T08:17:54.9368892Z name: artifact 2024-06-26T08:17:54.9369189Z region: us-east-1 2024-06-26T08:17:54.9369480Z env: 2024-06-26T08:17:54.9369748Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:17:54.9370193Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:17:54.9370923Z DOCKER_CONTAINER_ID: 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:54.9371553Z ##[endgroup] 2024-06-26T08:17:55.2511434Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T08:17:55.2512222Z With the provided path, there will be 1 file uploaded 2024-06-26T08:17:55.2512858Z Uploading to s3 prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:17:55.2539165Z Starting upload of test-reports-test-inductor_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688921705.zip 2024-06-26T08:17:55.3718658Z Finished upload of test-reports-test-inductor_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688921705.zip 2024-06-26T08:17:55.3855365Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T08:17:55.3855814Z with: 2024-06-26T08:17:55.3856287Z s3-bucket: gha-artifacts 2024-06-26T08:17:55.3856725Z s3-prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:17:55.3857184Z retention-days: 14 2024-06-26T08:17:55.3857510Z if-no-files-found: ignore 2024-06-26T08:17:55.3857865Z path: logs-*.zip 2024-06-26T08:17:55.3858170Z name: artifact 2024-06-26T08:17:55.3858461Z region: us-east-1 2024-06-26T08:17:55.3858756Z env: 2024-06-26T08:17:55.3859025Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:17:55.3859488Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:17:55.3860218Z DOCKER_CONTAINER_ID: 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:55.3860847Z ##[endgroup] 2024-06-26T08:17:55.6930617Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T08:17:55.6931405Z With the provided path, there will be 1 file uploaded 2024-06-26T08:17:55.6932120Z Uploading to s3 prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:17:55.6958905Z Starting upload of logs-test-inductor_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688921705.zip 2024-06-26T08:17:55.7881494Z Finished upload of logs-test-inductor_timm-1-2-linux.g5.4xlarge.nvidia.gpu_26688921705.zip 2024-06-26T08:17:55.8016028Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T08:17:55.8016709Z with: 2024-06-26T08:17:55.8016990Z s3-bucket: gha-artifacts 2024-06-26T08:17:55.8017415Z s3-prefix: pytorch/pytorch/9673646808/1/artifact 2024-06-26T08:17:55.8017875Z retention-days: 14 2024-06-26T08:17:55.8018198Z if-no-files-found: ignore 2024-06-26T08:17:55.8018708Z path: debug-*.zip 2024-06-26T08:17:55.8019015Z name: artifact 2024-06-26T08:17:55.8019307Z region: us-east-1 2024-06-26T08:17:55.8019599Z env: 2024-06-26T08:17:55.8019864Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:17:55.8020307Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:17:55.8021027Z DOCKER_CONTAINER_ID: 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:55.8021653Z ##[endgroup] 2024-06-26T08:17:56.1110065Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-06-26T08:17:56.1242278Z ##[group]Run # shellcheck disable=SC2156 2024-06-26T08:17:56.1242755Z # shellcheck disable=SC2156 2024-06-26T08:17:56.1243544Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-06-26T08:17:56.1251437Z shell: /usr/bin/bash -e {0} 2024-06-26T08:17:56.1251773Z env: 2024-06-26T08:17:56.1252043Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:17:56.1252506Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:17:56.1253228Z DOCKER_CONTAINER_ID: 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:56.1253861Z ##[endgroup] 2024-06-26T08:17:56.2866601Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-06-26T08:17:56.2867161Z with: 2024-06-26T08:17:56.2867416Z env: 2024-06-26T08:17:56.2867686Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:17:56.2868130Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:17:56.2868860Z DOCKER_CONTAINER_ID: 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:56.2869491Z ##[endgroup] 2024-06-26T08:17:56.2886492Z ##[group]Run set -eou pipefail 2024-06-26T08:17:56.2886905Z set -eou pipefail 2024-06-26T08:17:56.2887254Z  2024-06-26T08:17:56.2887761Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-06-26T08:17:56.2888392Z for _ in $(seq 1440); do 2024-06-26T08:17:56.2888876Z  # Break if no ssh session exists anymore 2024-06-26T08:17:56.2889375Z  if [ "$(who)" = "" ]; then 2024-06-26T08:17:56.2889782Z  break 2024-06-26T08:17:56.2890113Z  fi 2024-06-26T08:17:56.2890406Z  echo "." 2024-06-26T08:17:56.2890733Z  sleep 5 2024-06-26T08:17:56.2891094Z done 2024-06-26T08:17:56.2898613Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:17:56.2899099Z env: 2024-06-26T08:17:56.2899368Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:17:56.2899812Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:17:56.2900530Z DOCKER_CONTAINER_ID: 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:56.2901155Z ##[endgroup] 2024-06-26T08:17:56.2920478Z Holding runner for 2 hours until all ssh sessions have logged out 2024-06-26T08:17:56.2955419Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T08:17:56.2956188Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T08:17:56.2956777Z # shellcheck disable=SC2046 2024-06-26T08:17:56.2957243Z docker stop $(docker ps -q) || true 2024-06-26T08:17:56.2957711Z # Prune all of the docker images 2024-06-26T08:17:56.2958158Z docker system prune -af 2024-06-26T08:17:56.2964844Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:17:56.2965346Z env: 2024-06-26T08:17:56.2965612Z GIT_DEFAULT_BRANCH: main 2024-06-26T08:17:56.2966065Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-06-26T08:17:56.2966796Z DOCKER_CONTAINER_ID: 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:56.2967433Z ##[endgroup] 2024-06-26T08:17:56.6152742Z 2f0c6f998c37 2024-06-26T08:17:57.3642738Z Deleted Containers: 2024-06-26T08:17:57.3643317Z 2f0c6f998c37cc3f9e20e553e7774d3b0ce54ba4712ba4a1994871cb8341ad3b 2024-06-26T08:17:57.3644759Z 2024-06-26T08:18:02.4720148Z Deleted Images: 2024-06-26T08:18:02.4721921Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T08:18:02.4724480Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.4-cudnn9-py3-gcc9-inductor-benchmarks@sha256:037160f9b5ec877b659f1e94829492249e0bd2fc1a009ee3abbf0528226f6e1a 2024-06-26T08:18:02.4726561Z deleted: sha256:e7d48c2cbd3d9b87b03afe1b0df8f064acd2699045900da46dadf7d133480a59 2024-06-26T08:18:02.4727418Z deleted: sha256:56cb1338e9c083bd52998e89707e7796d2076b75d38a4def8bdc390b09511272 2024-06-26T08:18:02.4728371Z deleted: sha256:e4f4dc56f19dff895aba0d0764be8a9b2d5eadb0350e39ef2401ad14a1725555 2024-06-26T08:18:02.4729418Z deleted: sha256:ccd0cc8e6b80dc180d9ae9bf7129188a0183f3021d4f7205fb22c70b4865a51f 2024-06-26T08:18:02.4730538Z deleted: sha256:07ee1ed44447072463e81661242679474300fb0cab8e2efc170d318f5ca17e0b 2024-06-26T08:18:02.4731673Z deleted: sha256:7d480e3665e9347b658cdef638ff40fee05921f054b515f1bdac9fc0d5fdb8c1 2024-06-26T08:18:02.4732686Z deleted: sha256:fe676c4616c3712c172faf5e49864862b4e629766d8ab9f38af262a4468f5650 2024-06-26T08:18:02.4733777Z deleted: sha256:3dbebbbadfeceea4c3ccb7c77ef7ecbbcdfd212e4beee6d7d489e65f49d89a90 2024-06-26T08:18:02.4734638Z deleted: sha256:22bd2973961153bc8ab449fca8d86c80936170384f00ecfeeb5efbdebdb6e84e 2024-06-26T08:18:02.4735476Z deleted: sha256:10477ae85e23c18fd7dcd95c40657ad49696aa57c4c20941fa2c7ab052728e09 2024-06-26T08:18:02.4736594Z deleted: sha256:8410267dca5208c1582c4a52de7fb2801e7887a2779c181b7a23ce57ebfbb008 2024-06-26T08:18:02.4737412Z deleted: sha256:bf30542cfbf1c12c49030191b999d756519bf481b5477d9fa44c32419a9c30be 2024-06-26T08:18:02.4738203Z deleted: sha256:635e54f5585d54558c1955c10669c50393d2368f03a190f8811a4016020edb73 2024-06-26T08:18:02.4739011Z deleted: sha256:0dd1fd07a4dead7fda569708ee10f10de8fc31c76ca30041a404cea37e1c19ff 2024-06-26T08:18:02.4739856Z deleted: sha256:4d06049f595a3759da1cab03bd7c99eb1b1d97cea3ad2363d1fdcf43059970c2 2024-06-26T08:18:02.4740684Z deleted: sha256:3d3bc0f0a839991ea0e57ccfcbfd29c167d266005477821f9aca68f5a87feeb5 2024-06-26T08:18:02.4741519Z deleted: sha256:4ab7133a18afeadd5c16b3e4979be85d1fb0499128dd32d1b8c98b193c1711f5 2024-06-26T08:18:02.4742386Z deleted: sha256:fc964b13d0fd03e090216eb77a4754b629e9004e11fba0d3e1202abd27988fa1 2024-06-26T08:18:02.4743222Z deleted: sha256:a777e01489a9ff8b6e20ce95f1d6c8d15a1bded5ac905b15b648c9c92b04b101 2024-06-26T08:18:02.4744050Z deleted: sha256:2eebf26a98dffe366a7887e5899e97a80b53b838516c319048f4beb0ad8e4659 2024-06-26T08:18:02.4744876Z deleted: sha256:38bbdf32635223ffc465f6959fbadb47f90be1abd9926a2a60448c521d3fa074 2024-06-26T08:18:02.4745804Z deleted: sha256:b151839b5fa84f51c1b1fb71afd0639f67708d671f4e74850796f279467b0ac2 2024-06-26T08:18:02.4746622Z deleted: sha256:116df6e53a016f2c085d6caaf47932f4a633692267ceadb6ae7c9faf674a295f 2024-06-26T08:18:02.4747439Z deleted: sha256:e2211d0ac2a12b1346a0a3975332c72a959052abd5d616a77cb83dc7281c60e4 2024-06-26T08:18:02.4748263Z deleted: sha256:28e58c5f46e119f7b6f4fd1593400d4374c49be979440a4eafce95cec06387c8 2024-06-26T08:18:02.4749084Z deleted: sha256:01894278d553aade6feee2f7e4e8918dfbe7c6ba0157c79f35285100a4c54cfc 2024-06-26T08:18:02.4749904Z deleted: sha256:e1bf74447204782e0e427c1219add26732c05f04094f25b0af6801dbf3708d34 2024-06-26T08:18:02.4750719Z deleted: sha256:354656042a44c89c55ec82b1cd1a1495c45bab6f2a3cd49c4610b3e131be14ac 2024-06-26T08:18:02.4751584Z deleted: sha256:66a7c705ab3667d09765f0960a1dc0ae4bf89e1533be4f879225bd67c6bd5b6c 2024-06-26T08:18:02.4752626Z deleted: sha256:e63b3bfdaf329aa1767471b68871d5cfcea58f939c3ca58b873ae17280620dcb 2024-06-26T08:18:02.4753473Z deleted: sha256:65ed448ec65a94b57a576fefadfe7ae8e2255417ae04fed109c6ab154c64d0d2 2024-06-26T08:18:02.4754303Z deleted: sha256:17e3a3018dc9ae93cb79e1809453b7e45067eafa698db11fe6eff1885c46dff2 2024-06-26T08:18:02.4755128Z deleted: sha256:914a2cbbfb84a001645b97dcdaeb33fdfcb508adf5361862898214b596582384 2024-06-26T08:18:02.4756109Z deleted: sha256:5f3b10b9fc66885bc6548dcb2cb08a9a37c54bcf0db47c9821e160f707d566f7 2024-06-26T08:18:02.4756939Z deleted: sha256:ab9e8d80d594f5e8ac7fa84004b2b6727cb32d2aa902780ed699040364a2f7d3 2024-06-26T08:18:02.4757769Z deleted: sha256:16bd34f9f8150ce41e2edc26216d4d14c2225b1cc601e645c6c0386cbae43261 2024-06-26T08:18:02.4758588Z deleted: sha256:9227c94c6eaad947b21138ba760e718a302a81b71e264d909da46c24773303ed 2024-06-26T08:18:02.4759389Z deleted: sha256:04235d257525c35422388d13951e4cd36dd37a308721bbd75049f9242e139af0 2024-06-26T08:18:02.4760195Z deleted: sha256:bbf5e8b82ccde9f7f66c9038f3bb7f6717186e2a712e0fd78d318f9b67d1f372 2024-06-26T08:18:02.4761025Z deleted: sha256:a002748e7ef21d251e450ab4715c95cb34f9358ce9bcdd31c5fe7831fe74e98e 2024-06-26T08:18:02.4761850Z deleted: sha256:8acad1619d75bf2f4264dafe4df2e7103c33f31ca6162470083760c26a18609f 2024-06-26T08:18:02.4762665Z deleted: sha256:80338d70f5f600d143f45d02636da59c04612df3e9f1ce47aecb8e809d4f232c 2024-06-26T08:18:02.4763836Z deleted: sha256:38dd48bef6af3d6595132e5d52ef581cbf0693d81b1de1b0f2e11612993d3931 2024-06-26T08:18:02.4764679Z deleted: sha256:b879830930db52d73ad3ac6a096919dd54d86eeb8ee00fbaa3c5d377d876b6d7 2024-06-26T08:18:02.4765611Z deleted: sha256:806976237f8c5a8f583ebbac403d293ae72d19b628c4d73296ddff22ce7fcc21 2024-06-26T08:18:02.4766429Z deleted: sha256:12f4e35bb106aa9d1242cd608d97754669252c7d3633081be5cad2c015786ed0 2024-06-26T08:18:02.4767259Z deleted: sha256:efce19acd8c8040fcb7191130d7bbc2efa23cfc4fefda16d7b3140edd60ed51a 2024-06-26T08:18:02.4768099Z deleted: sha256:503474c932ead012cf40c51fc5744c6965cebda92fcd2ae01ca2080cc7cc8a5f 2024-06-26T08:18:02.4768913Z deleted: sha256:57e4d7c423aa2bf3667a72ff53b2062278268e7cf946839d7433aa0e986b2da9 2024-06-26T08:18:02.4769721Z deleted: sha256:59afee679a7263f5b925a6761a91fe4662d1833935efa83ae29c9f6b4c0dff5d 2024-06-26T08:18:02.4770520Z deleted: sha256:9cc102e9cf6f43b61504c2a669813c840a83667a9c73df7982504f50d5bee6a6 2024-06-26T08:18:02.4771338Z deleted: sha256:cb13d295b36de44b3ef5bf5c043afe1289e5a6ef4ff275649cf7a2526259246b 2024-06-26T08:18:02.4772182Z deleted: sha256:05c68da10d4f675a9ebb009a262bc994756446a8c7fcbf6bf5ccb1c46ece9f2d 2024-06-26T08:18:02.4773005Z deleted: sha256:aebc797800d33984f39d68d5198d6a36dee176874411e27e4c2f1afd98219fae 2024-06-26T08:18:02.4773824Z deleted: sha256:7dc0bc48c310f2dac988fb7100325f00a2169da8202bbe960b3bac06993afb44 2024-06-26T08:18:02.4774631Z deleted: sha256:63e5b2172d1a99f70b1e305a540c25a8374ad06620f39be9beef073f2a8abf39 2024-06-26T08:18:02.4775442Z deleted: sha256:3249b2c20d60a5ad96998d9f8ac9610c2a2336c9cb7d3e23aed549da8a1cefea 2024-06-26T08:18:02.4776464Z deleted: sha256:1e9d132301840d90f74e943daba8e4f210ea43a17f0361309add3cbfc4b8c3de 2024-06-26T08:18:02.4777285Z deleted: sha256:6fe299c368f1e977100d5aa0d740c430cd7f2cd052dbcce145b75073c4bbe92c 2024-06-26T08:18:02.4778100Z deleted: sha256:936e3fad232efd49441a1c15be7b4d7d33cf3fa0fea3401e364c15bb88d7a7df 2024-06-26T08:18:02.4778943Z deleted: sha256:ed89ccf22c7e15739fb9a8bba8734987bfaf1f4ed2c10d8d7cedf59344f17065 2024-06-26T08:18:02.4779777Z deleted: sha256:3edf405b311fc456bc694b304965ac6e02420a185dea5b1e271e9371685db53f 2024-06-26T08:18:02.4780605Z deleted: sha256:791d5b6d87aeebbf087b42221bed92d66f3743fcf4dac393c6faaf28a91d1178 2024-06-26T08:18:02.4781424Z deleted: sha256:8259441b85d03ea8b557e9a0e21e27d4c6ffaad0b3e0a831ac373e5571fc0b9a 2024-06-26T08:18:02.4782243Z deleted: sha256:f2b1062b70de50e2fa03bfc5369ea678fa065f9e0f79fccf296c5720d68fa40c 2024-06-26T08:18:02.4783067Z deleted: sha256:c87ec1cad0bc363e628805cf92ce50f45917aa771536cce573e6ea726904c4f5 2024-06-26T08:18:02.4783888Z deleted: sha256:f980b7a6f28ce9f397abb6f39ec3d02d54168d92fda720ae419c385610a4cbc0 2024-06-26T08:18:02.4784704Z deleted: sha256:2d81ead3471707daaf9265962f0dae7053a8cffc2baa7cc2b7e03818855f1b58 2024-06-26T08:18:02.4785647Z deleted: sha256:433544e4353c9e0269dc71fa16c00ddcecdcd5676ed0df618c7b51ac78e23053 2024-06-26T08:18:02.4786484Z deleted: sha256:fc24af9fd5121adfbc7b75df0abb1ef8c2d3868f33b7b6b1e8bf4e214fac4bd6 2024-06-26T08:18:02.4787424Z deleted: sha256:ce1492ad94b71204f8b3fb8903d0ad48a96652635e7fc9347e430639ca15a737 2024-06-26T08:18:02.4788229Z deleted: sha256:96f2ae9435e6e12f68a88dc404c8079b2796639589970c34894a86ccff3f3732 2024-06-26T08:18:02.4789043Z deleted: sha256:5310d2bc0952e7c6d8cbade7ecafd88ce3a04d9da303fa6419583e13a02b58a9 2024-06-26T08:18:02.4789854Z deleted: sha256:099076d608a55a027d60f5b572a3991a8d690a01500acf2c696933a7e6d38650 2024-06-26T08:18:02.4790657Z deleted: sha256:862b05d0f8f7e3dff2d04bd49784a9901f4e22c5dd6544ae72eb3be05e23fffe 2024-06-26T08:18:02.4791473Z deleted: sha256:f7a888e5f7904827f6d71cebcba5169dbc7a78b20f228b9b9b625f38e0c52f24 2024-06-26T08:18:02.4792292Z deleted: sha256:ab9f75a4a7a636aae480f630eaddc05133dda64f1c42445b617c97d3d209991c 2024-06-26T08:18:02.4793168Z deleted: sha256:56dda0c9cee64db5c10d5dd5f08c6f6707263f5ee98e8a075d635e5be996e855 2024-06-26T08:18:02.4794020Z deleted: sha256:5faf9c0a9efe4675ecd21a4ec417d51077d5e75da9e673161a94e7d6cd43f92c 2024-06-26T08:18:02.4794520Z 2024-06-26T08:18:02.4794685Z Total reclaimed space: 36.6GB 2024-06-26T08:18:02.4869460Z Post job cleanup. 2024-06-26T08:18:02.4917314Z Post job cleanup. 2024-06-26T08:18:02.5734783Z [command]/usr/bin/git version 2024-06-26T08:18:02.5777697Z git version 2.40.1 2024-06-26T08:18:02.5809873Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/1267f80c-f0e0-44ff-889c-8ba5191b3090/.gitconfig' 2024-06-26T08:18:02.5821454Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/1267f80c-f0e0-44ff-889c-8ba5191b3090' before making global git config changes 2024-06-26T08:18:02.5822680Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T08:18:02.5826701Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T08:18:02.5853638Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T08:18:02.5876075Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T08:18:02.6084002Z Entering 'android/libs/fbjni' 2024-06-26T08:18:02.6120273Z Entering 'third_party/FP16' 2024-06-26T08:18:02.6156548Z Entering 'third_party/FXdiv' 2024-06-26T08:18:02.6190473Z Entering 'third_party/NNPACK' 2024-06-26T08:18:02.6226837Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T08:18:02.6260832Z Entering 'third_party/XNNPACK' 2024-06-26T08:18:02.6307654Z Entering 'third_party/benchmark' 2024-06-26T08:18:02.6343846Z Entering 'third_party/cpp-httplib' 2024-06-26T08:18:02.6379968Z Entering 'third_party/cpuinfo' 2024-06-26T08:18:02.6416236Z Entering 'third_party/cudnn_frontend' 2024-06-26T08:18:02.6451238Z Entering 'third_party/cutlass' 2024-06-26T08:18:02.6491235Z Entering 'third_party/eigen' 2024-06-26T08:18:02.6529367Z Entering 'third_party/fbgemm' 2024-06-26T08:18:02.6564944Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T08:18:02.6599717Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T08:18:02.6635453Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T08:18:02.6673277Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T08:18:02.6707813Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T08:18:02.6742626Z Entering 'third_party/flatbuffers' 2024-06-26T08:18:02.6780114Z Entering 'third_party/fmt' 2024-06-26T08:18:02.6813810Z Entering 'third_party/foxi' 2024-06-26T08:18:02.6849924Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T08:18:02.6884817Z Entering 'third_party/gloo' 2024-06-26T08:18:02.6920330Z Entering 'third_party/googletest' 2024-06-26T08:18:02.6956467Z Entering 'third_party/ideep' 2024-06-26T08:18:02.6989528Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T08:18:02.7029038Z Entering 'third_party/ittapi' 2024-06-26T08:18:02.7062580Z Entering 'third_party/kineto' 2024-06-26T08:18:02.7094669Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T08:18:02.7128958Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T08:18:02.7164467Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T08:18:02.7198589Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T08:18:02.7233328Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T08:18:02.7266129Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T08:18:02.7302635Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T08:18:02.7338278Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T08:18:02.7373618Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T08:18:02.7410086Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T08:18:02.7448194Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T08:18:02.7481010Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T08:18:02.7516649Z Entering 'third_party/mimalloc' 2024-06-26T08:18:02.7553321Z Entering 'third_party/nccl/nccl' 2024-06-26T08:18:02.7590089Z Entering 'third_party/nlohmann' 2024-06-26T08:18:02.7623172Z Entering 'third_party/onnx' 2024-06-26T08:18:02.7669347Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T08:18:02.7705504Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T08:18:02.7743726Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T08:18:02.7778098Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T08:18:02.7810969Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T08:18:02.7845374Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T08:18:02.7880396Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T08:18:02.7914507Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T08:18:02.7949115Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T08:18:02.7981934Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T08:18:02.8015516Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T08:18:02.8051907Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T08:18:02.8086665Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T08:18:02.8136929Z Entering 'third_party/pocketfft' 2024-06-26T08:18:02.8171451Z Entering 'third_party/protobuf' 2024-06-26T08:18:02.8209308Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T08:18:02.8244959Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T08:18:02.8280684Z Entering 'third_party/psimd' 2024-06-26T08:18:02.8316047Z Entering 'third_party/pthreadpool' 2024-06-26T08:18:02.8351322Z Entering 'third_party/pybind11' 2024-06-26T08:18:02.8383138Z Entering 'third_party/python-peachpy' 2024-06-26T08:18:02.8418907Z Entering 'third_party/sleef' 2024-06-26T08:18:02.8453774Z Entering 'third_party/tensorpipe' 2024-06-26T08:18:02.8489091Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T08:18:02.8522283Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T08:18:02.8556745Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T08:18:02.8590336Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T08:18:02.8623584Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T08:18:02.8674355Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T08:18:02.8694254Z http.https://github.com/.extraheader 2024-06-26T08:18:02.8702958Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-06-26T08:18:02.8731332Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T08:18:02.8940949Z Entering 'android/libs/fbjni' 2024-06-26T08:18:02.8962624Z http.https://github.com/.extraheader 2024-06-26T08:18:02.8991634Z Entering 'third_party/FP16' 2024-06-26T08:18:02.9012381Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9037307Z Entering 'third_party/FXdiv' 2024-06-26T08:18:02.9058752Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9082031Z Entering 'third_party/NNPACK' 2024-06-26T08:18:02.9104265Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9128884Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T08:18:02.9150764Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9174542Z Entering 'third_party/XNNPACK' 2024-06-26T08:18:02.9196428Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9231408Z Entering 'third_party/benchmark' 2024-06-26T08:18:02.9253345Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9278537Z Entering 'third_party/cpp-httplib' 2024-06-26T08:18:02.9300467Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9325077Z Entering 'third_party/cpuinfo' 2024-06-26T08:18:02.9347509Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9371094Z Entering 'third_party/cudnn_frontend' 2024-06-26T08:18:02.9393818Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9420003Z Entering 'third_party/cutlass' 2024-06-26T08:18:02.9441988Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9471582Z Entering 'third_party/eigen' 2024-06-26T08:18:02.9493949Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9517908Z Entering 'third_party/fbgemm' 2024-06-26T08:18:02.9539979Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9563450Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T08:18:02.9583601Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9607931Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T08:18:02.9629886Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9652967Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T08:18:02.9674296Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9701190Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T08:18:02.9721889Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9745634Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T08:18:02.9765477Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9789089Z Entering 'third_party/flatbuffers' 2024-06-26T08:18:02.9810475Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9835541Z Entering 'third_party/fmt' 2024-06-26T08:18:02.9857802Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9880406Z Entering 'third_party/foxi' 2024-06-26T08:18:02.9902659Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9925575Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T08:18:02.9947867Z http.https://github.com/.extraheader 2024-06-26T08:18:02.9971285Z Entering 'third_party/gloo' 2024-06-26T08:18:02.9994645Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0018129Z Entering 'third_party/googletest' 2024-06-26T08:18:03.0040299Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0062970Z Entering 'third_party/ideep' 2024-06-26T08:18:03.0084554Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0108567Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T08:18:03.0130117Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0158313Z Entering 'third_party/ittapi' 2024-06-26T08:18:03.0180836Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0203449Z Entering 'third_party/kineto' 2024-06-26T08:18:03.0224951Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0247279Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T08:18:03.0269022Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0291144Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T08:18:03.0313068Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0337207Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T08:18:03.0358990Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0381494Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T08:18:03.0401737Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0425022Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T08:18:03.0446078Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0467375Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T08:18:03.0487912Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0513361Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T08:18:03.0533654Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0556786Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T08:18:03.0577648Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0601796Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T08:18:03.0622991Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0647866Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T08:18:03.0669077Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0694109Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T08:18:03.0715725Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0739683Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T08:18:03.0760725Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0784973Z Entering 'third_party/mimalloc' 2024-06-26T08:18:03.0806546Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0830373Z Entering 'third_party/nccl/nccl' 2024-06-26T08:18:03.0852012Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0876267Z Entering 'third_party/nlohmann' 2024-06-26T08:18:03.0897370Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0921340Z Entering 'third_party/onnx' 2024-06-26T08:18:03.0941575Z http.https://github.com/.extraheader 2024-06-26T08:18:03.0979096Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T08:18:03.1000725Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1023832Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T08:18:03.1044903Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1072048Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T08:18:03.1093836Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1118830Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T08:18:03.1140979Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1163198Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T08:18:03.1183851Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1206396Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T08:18:03.1228305Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1252075Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T08:18:03.1272643Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1298968Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T08:18:03.1319966Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1343260Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T08:18:03.1365454Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1388558Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T08:18:03.1408061Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1432278Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T08:18:03.1453937Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1478621Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T08:18:03.1500444Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1526320Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T08:18:03.1547731Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1586775Z Entering 'third_party/pocketfft' 2024-06-26T08:18:03.1608931Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1631961Z Entering 'third_party/protobuf' 2024-06-26T08:18:03.1653826Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1679978Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T08:18:03.1701727Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1726609Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T08:18:03.1748847Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1773921Z Entering 'third_party/psimd' 2024-06-26T08:18:03.1796402Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1819072Z Entering 'third_party/pthreadpool' 2024-06-26T08:18:03.1840059Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1864255Z Entering 'third_party/pybind11' 2024-06-26T08:18:03.1885544Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1906279Z Entering 'third_party/python-peachpy' 2024-06-26T08:18:03.1928019Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1952027Z Entering 'third_party/sleef' 2024-06-26T08:18:03.1972289Z http.https://github.com/.extraheader 2024-06-26T08:18:03.1994571Z Entering 'third_party/tensorpipe' 2024-06-26T08:18:03.2015922Z http.https://github.com/.extraheader 2024-06-26T08:18:03.2039405Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T08:18:03.2059867Z http.https://github.com/.extraheader 2024-06-26T08:18:03.2083834Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T08:18:03.2103879Z http.https://github.com/.extraheader 2024-06-26T08:18:03.2127320Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T08:18:03.2148464Z http.https://github.com/.extraheader 2024-06-26T08:18:03.2169994Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T08:18:03.2191437Z http.https://github.com/.extraheader 2024-06-26T08:18:03.2214718Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T08:18:03.2236537Z http.https://github.com/.extraheader 2024-06-26T08:18:03.2342399Z A job completed hook has been configured by the self-hosted runner administrator 2024-06-26T08:18:03.2359701Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T08:18:03.2366439Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T08:18:03.2366956Z ##[endgroup] 2024-06-26T08:18:04.1284657Z Cleaning up orphan processes