2024-08-07T17:56:17.7252241Z Current runner version: '2.318.0' 2024-08-07T17:56:17.7259555Z Runner name: 'i-031f7aad237b89ef3' 2024-08-07T17:56:17.7260736Z Runner group name: 'Default' 2024-08-07T17:56:17.7261887Z Machine name: 'ip-10-0-38-158' 2024-08-07T17:56:17.7267097Z ##[group]GITHUB_TOKEN Permissions 2024-08-07T17:56:17.7269719Z Actions: read 2024-08-07T17:56:17.7270495Z Attestations: read 2024-08-07T17:56:17.7271174Z Checks: read 2024-08-07T17:56:17.7271867Z Contents: read 2024-08-07T17:56:17.7272661Z Deployments: read 2024-08-07T17:56:17.7273337Z Discussions: read 2024-08-07T17:56:17.7274051Z Issues: read 2024-08-07T17:56:17.7274753Z Metadata: read 2024-08-07T17:56:17.7275414Z Packages: read 2024-08-07T17:56:17.7276120Z Pages: read 2024-08-07T17:56:17.7276822Z PullRequests: read 2024-08-07T17:56:17.7277544Z RepositoryProjects: read 2024-08-07T17:56:17.7278316Z SecurityEvents: read 2024-08-07T17:56:17.7279082Z Statuses: read 2024-08-07T17:56:17.7279755Z ##[endgroup] 2024-08-07T17:56:17.7282920Z Secret source: Actions 2024-08-07T17:56:17.7283986Z Prepare workflow directory 2024-08-07T17:56:18.1957016Z Prepare all required actions 2024-08-07T17:56:18.1996791Z Getting action download info 2024-08-07T17:56:18.4058205Z Download action repository 'pytorch/test-infra@main' (SHA:a1f5a89251fc4258ab59806094fe3108f7d6741a) 2024-08-07T17:56:19.7852333Z Download action repository 'pytorch/pytorch@main' (SHA:a62710c82039b798befd86e938d2137af3978c93) 2024-08-07T17:56:31.3900417Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-07T17:56:31.5854184Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-07T17:56:31.8609803Z Getting action download info 2024-08-07T17:56:31.9547108Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-07T17:56:32.2361520Z Getting action download info 2024-08-07T17:56:32.3347629Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-07T17:56:32.5075982Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/pull/131248/merge (f779f6b7738020e244184bded4026b37de3f9f24) 2024-08-07T17:56:32.5077768Z ##[group] Inputs 2024-08-07T17:56:32.5078147Z build-environment: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-07T17:56:32.5079915Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-07T17:56:32.5082001Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:56:32.5082764Z sync-tag: 2024-08-07T17:56:32.5083449Z timeout-minutes: 240 2024-08-07T17:56:32.5083717Z use-gha: 2024-08-07T17:56:32.5083954Z dashboard-tag: 2024-08-07T17:56:32.5084212Z s3-bucket: gha-artifacts 2024-08-07T17:56:32.5084492Z aws-role-to-assume: 2024-08-07T17:56:32.5084764Z ##[endgroup] 2024-08-07T17:56:32.5085270Z Complete job name: linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 1, 5, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-07T17:56:32.5573356Z A job started hook has been configured by the self-hosted runner administrator 2024-08-07T17:56:32.5674137Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-07T17:56:32.5685197Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:56:32.5685813Z ##[endgroup] 2024-08-07T17:56:34.3403103Z Runner Type: amz2023.linux.g5.4xlarge.nvidia.gpu 2024-08-07T17:56:34.3403632Z Instance Type: g5.4xlarge 2024-08-07T17:56:34.3404629Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-07T17:56:34.3405166Z AMI ID: ami-06c68f701d8090592 2024-08-07T17:56:40.1610851Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-07T17:56:40.1611463Z with: 2024-08-07T17:56:40.1612178Z github-secret: *** 2024-08-07T17:56:40.1612901Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-07T17:56:40.1613783Z activate-with-label: false 2024-08-07T17:56:40.1614183Z label: with-ssh 2024-08-07T17:56:40.1614488Z remove-existing-keys: true 2024-08-07T17:56:40.1614958Z fail-silently: true 2024-08-07T17:56:40.1615366Z env: 2024-08-07T17:56:40.1615646Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:56:40.1616070Z ##[endgroup] 2024-08-07T17:56:40.2530039Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-07T17:56:40.5786405Z Grabbing public ssh keys from https://github.com/zdevito.keys 2024-08-07T17:56:40.6570957Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-07T17:56:40.6584408Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-07T17:56:40.6613326Z Login using: ssh ec2-user@ec2-44-201-230-93.compute-1.amazonaws.com 2024-08-07T17:56:40.6614016Z All testing is done inside the container, to start an interactive session run: 2024-08-07T17:56:40.6614722Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-07T17:56:40.6747862Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-07T17:56:40.6748296Z with: 2024-08-07T17:56:40.6748529Z submodules: recursive 2024-08-07T17:56:40.6748798Z fetch-depth: 0 2024-08-07T17:56:40.6749038Z env: 2024-08-07T17:56:40.6749264Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:56:40.6749526Z ##[endgroup] 2024-08-07T17:56:40.6828803Z ##[group]Run retry () { 2024-08-07T17:56:40.6829126Z retry () { 2024-08-07T17:56:40.6829472Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-07T17:56:40.6829851Z } 2024-08-07T17:56:40.6830110Z echo "${GITHUB_WORKSPACE}" 2024-08-07T17:56:40.6830449Z if [ -z "${NO_SUDO}" ]; then 2024-08-07T17:56:40.6830796Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-07T17:56:40.6831128Z else 2024-08-07T17:56:40.6831379Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-07T17:56:40.6831685Z fi 2024-08-07T17:56:40.6831975Z mkdir "${GITHUB_WORKSPACE}" 2024-08-07T17:56:40.6842266Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:56:40.6842640Z env: 2024-08-07T17:56:40.6842864Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:56:40.6843135Z NO_SUDO: 2024-08-07T17:56:40.6843359Z ##[endgroup] 2024-08-07T17:56:40.6873386Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:56:45.2520211Z ##[group]Run malfet/checkout@silent-checkout 2024-08-07T17:56:45.2520554Z with: 2024-08-07T17:56:45.2520808Z ref: 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:56:45.2521127Z fetch-depth: 0 2024-08-07T17:56:45.2521376Z submodules: recursive 2024-08-07T17:56:45.2521642Z quiet-checkout: true 2024-08-07T17:56:45.2521911Z repository: pytorch/pytorch 2024-08-07T17:56:45.2522284Z token: *** 2024-08-07T17:56:45.2522515Z ssh-strict: true 2024-08-07T17:56:45.2522766Z persist-credentials: true 2024-08-07T17:56:45.2523042Z clean: true 2024-08-07T17:56:45.2523303Z sparse-checkout-cone-mode: true 2024-08-07T17:56:45.2523593Z lfs: false 2024-08-07T17:56:45.2523836Z set-safe-directory: true 2024-08-07T17:56:45.2524098Z env: 2024-08-07T17:56:45.2524321Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:56:45.2524587Z ##[endgroup] 2024-08-07T17:56:45.3515927Z Syncing repository: pytorch/pytorch 2024-08-07T17:56:45.3517066Z ##[group]Getting Git version info 2024-08-07T17:56:45.3517517Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-07T17:56:45.3518538Z [command]/usr/bin/git version 2024-08-07T17:56:45.3520731Z git version 2.40.1 2024-08-07T17:56:45.3540506Z ##[endgroup] 2024-08-07T17:56:45.3555160Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/f492d203-ff64-447d-b1d2-30d36f6a1277' before making global git config changes 2024-08-07T17:56:45.3556097Z Adding repository directory to the temporary git global config as a safe directory 2024-08-07T17:56:45.3559921Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:56:45.3611302Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-07T17:56:45.3614973Z ##[group]Initializing the repository 2024-08-07T17:56:45.3617740Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:56:45.3663379Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-07T17:56:45.3664038Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-07T17:56:45.3664593Z hint: of your new repositories, which will suppress this warning, call: 2024-08-07T17:56:45.3665020Z hint: 2024-08-07T17:56:45.3665335Z hint: git config --global init.defaultBranch 2024-08-07T17:56:45.3665699Z hint: 2024-08-07T17:56:45.3666038Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-07T17:56:45.3666591Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-07T17:56:45.3668257Z hint: 2024-08-07T17:56:45.3668512Z hint: git branch -m 2024-08-07T17:56:45.3669009Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-07T17:56:45.3675644Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-07T17:56:45.3719448Z ##[endgroup] 2024-08-07T17:56:45.3719890Z ##[group]Disabling automatic garbage collection 2024-08-07T17:56:45.3722059Z [command]/usr/bin/git config --local gc.auto 0 2024-08-07T17:56:45.3764576Z ##[endgroup] 2024-08-07T17:56:45.3764994Z ##[group]Setting up auth 2024-08-07T17:56:45.3770257Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-07T17:56:45.3813641Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-07T17:56:45.4166740Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-07T17:56:45.4208434Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-07T17:56:45.4567704Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-07T17:56:45.4631162Z ##[endgroup] 2024-08-07T17:56:45.4631631Z ##[group]Fetching the repository 2024-08-07T17:56:45.4636938Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-07T17:56:47.8530722Z remote: Enumerating objects: 1008194 2024-08-07T17:56:47.8531179Z remote: Enumerating objects: 1011089, done. 2024-08-07T17:56:47.8534861Z remote: Counting objects: 0% (1/2895) 2024-08-07T17:56:47.8536462Z remote: Counting objects: 1% (29/2895) 2024-08-07T17:56:47.8537021Z remote: Counting objects: 2% (58/2895) 2024-08-07T17:56:47.8538696Z remote: Counting objects: 3% (87/2895) 2024-08-07T17:56:47.8540087Z remote: Counting objects: 4% (116/2895) 2024-08-07T17:56:47.8542385Z remote: Counting objects: 5% (145/2895) 2024-08-07T17:56:47.8542757Z remote: Counting objects: 6% (174/2895) 2024-08-07T17:56:47.8543135Z remote: Counting objects: 7% (203/2895) 2024-08-07T17:56:47.8543802Z remote: Counting objects: 8% (232/2895) 2024-08-07T17:56:47.8544164Z remote: Counting objects: 9% (261/2895) 2024-08-07T17:56:47.8544522Z remote: Counting objects: 10% (290/2895) 2024-08-07T17:56:47.8544905Z remote: Counting objects: 11% (319/2895) 2024-08-07T17:56:47.8545270Z remote: Counting objects: 12% (348/2895) 2024-08-07T17:56:47.8545624Z remote: Counting objects: 13% (377/2895) 2024-08-07T17:56:47.8545990Z remote: Counting objects: 14% (406/2895) 2024-08-07T17:56:47.8546354Z remote: Counting objects: 15% (435/2895) 2024-08-07T17:56:47.8546725Z remote: Counting objects: 16% (464/2895) 2024-08-07T17:56:47.8547089Z remote: Counting objects: 17% (493/2895) 2024-08-07T17:56:47.8547449Z remote: Counting objects: 18% (522/2895) 2024-08-07T17:56:47.8547812Z remote: Counting objects: 19% (551/2895) 2024-08-07T17:56:47.8548174Z remote: Counting objects: 20% (579/2895) 2024-08-07T17:56:47.8548551Z remote: Counting objects: 21% (608/2895) 2024-08-07T17:56:47.8548906Z remote: Counting objects: 22% (637/2895) 2024-08-07T17:56:47.8549280Z remote: Counting objects: 23% (666/2895) 2024-08-07T17:56:47.8549643Z remote: Counting objects: 24% (695/2895) 2024-08-07T17:56:47.8549997Z remote: Counting objects: 25% (724/2895) 2024-08-07T17:56:47.8550359Z remote: Counting objects: 26% (753/2895) 2024-08-07T17:56:47.8550802Z remote: Counting objects: 27% (782/2895) 2024-08-07T17:56:47.8551331Z remote: Counting objects: 28% (811/2895) 2024-08-07T17:56:47.8551717Z remote: Counting objects: 29% (840/2895) 2024-08-07T17:56:47.8552078Z remote: Counting objects: 30% (869/2895) 2024-08-07T17:56:47.8552442Z remote: Counting objects: 31% (898/2895) 2024-08-07T17:56:47.8552794Z remote: Counting objects: 32% (927/2895) 2024-08-07T17:56:47.8553270Z remote: Counting objects: 33% (956/2895) 2024-08-07T17:56:47.8553658Z remote: Counting objects: 34% (985/2895) 2024-08-07T17:56:47.8554018Z remote: Counting objects: 35% (1014/2895) 2024-08-07T17:56:47.8554389Z remote: Counting objects: 36% (1043/2895) 2024-08-07T17:56:47.8554753Z remote: Counting objects: 37% (1072/2895) 2024-08-07T17:56:47.8555122Z remote: Counting objects: 38% (1101/2895) 2024-08-07T17:56:47.8555488Z remote: Counting objects: 39% (1130/2895) 2024-08-07T17:56:47.8555857Z remote: Counting objects: 40% (1158/2895) 2024-08-07T17:56:47.8556218Z remote: Counting objects: 41% (1187/2895) 2024-08-07T17:56:47.8556590Z remote: Counting objects: 42% (1216/2895) 2024-08-07T17:56:47.8556951Z remote: Counting objects: 43% (1245/2895) 2024-08-07T17:56:47.8557315Z remote: Counting objects: 44% (1274/2895) 2024-08-07T17:56:47.8557688Z remote: Counting objects: 45% (1303/2895) 2024-08-07T17:56:47.8558050Z remote: Counting objects: 46% (1332/2895) 2024-08-07T17:56:47.8558416Z remote: Counting objects: 47% (1361/2895) 2024-08-07T17:56:47.8558789Z remote: Counting objects: 48% (1390/2895) 2024-08-07T17:56:47.8559161Z remote: Counting objects: 49% (1419/2895) 2024-08-07T17:56:47.8559524Z remote: Counting objects: 50% (1448/2895) 2024-08-07T17:56:47.8559879Z remote: Counting objects: 51% (1477/2895) 2024-08-07T17:56:47.8560250Z remote: Counting objects: 52% (1506/2895) 2024-08-07T17:56:47.8560619Z remote: Counting objects: 53% (1535/2895) 2024-08-07T17:56:47.8561018Z remote: Counting objects: 54% (1564/2895) 2024-08-07T17:56:47.8561391Z remote: Counting objects: 55% (1593/2895) 2024-08-07T17:56:47.8561761Z remote: Counting objects: 56% (1622/2895) 2024-08-07T17:56:47.8562118Z remote: Counting objects: 57% (1651/2895) 2024-08-07T17:56:47.8562468Z remote: Counting objects: 58% (1680/2895) 2024-08-07T17:56:47.8562965Z remote: Counting objects: 59% (1709/2895) 2024-08-07T17:56:47.8563332Z remote: Counting objects: 60% (1737/2895) 2024-08-07T17:56:47.8563698Z remote: Counting objects: 61% (1766/2895) 2024-08-07T17:56:47.8564058Z remote: Counting objects: 62% (1795/2895) 2024-08-07T17:56:47.8564419Z remote: Counting objects: 63% (1824/2895) 2024-08-07T17:56:47.8564777Z remote: Counting objects: 64% (1853/2895) 2024-08-07T17:56:47.8565139Z remote: Counting objects: 65% (1882/2895) 2024-08-07T17:56:47.8565518Z remote: Counting objects: 66% (1911/2895) 2024-08-07T17:56:47.8565868Z remote: Counting objects: 67% (1940/2895) 2024-08-07T17:56:47.8566229Z remote: Counting objects: 68% (1969/2895) 2024-08-07T17:56:47.8566593Z remote: Counting objects: 69% (1998/2895) 2024-08-07T17:56:47.8566947Z remote: Counting objects: 70% (2027/2895) 2024-08-07T17:56:47.8567308Z remote: Counting objects: 71% (2056/2895) 2024-08-07T17:56:47.8567686Z remote: Counting objects: 72% (2085/2895) 2024-08-07T17:56:47.8568033Z remote: Counting objects: 73% (2114/2895) 2024-08-07T17:56:47.8568386Z remote: Counting objects: 74% (2143/2895) 2024-08-07T17:56:47.8568740Z remote: Counting objects: 75% (2172/2895) 2024-08-07T17:56:47.8569093Z remote: Counting objects: 76% (2201/2895) 2024-08-07T17:56:47.8569441Z remote: Counting objects: 77% (2230/2895) 2024-08-07T17:56:47.8569794Z remote: Counting objects: 78% (2259/2895) 2024-08-07T17:56:47.8570242Z remote: Counting objects: 79% (2288/2895) 2024-08-07T17:56:47.8570602Z remote: Counting objects: 80% (2316/2895) 2024-08-07T17:56:47.8570964Z remote: Counting objects: 81% (2345/2895) 2024-08-07T17:56:47.8571327Z remote: Counting objects: 82% (2374/2895) 2024-08-07T17:56:47.8571696Z remote: Counting objects: 83% (2403/2895) 2024-08-07T17:56:47.8572060Z remote: Counting objects: 84% (2432/2895) 2024-08-07T17:56:47.8572432Z remote: Counting objects: 85% (2461/2895) 2024-08-07T17:56:47.8572789Z remote: Counting objects: 86% (2490/2895) 2024-08-07T17:56:47.8573153Z remote: Counting objects: 87% (2519/2895) 2024-08-07T17:56:47.8573515Z remote: Counting objects: 88% (2548/2895) 2024-08-07T17:56:47.8573870Z remote: Counting objects: 89% (2577/2895) 2024-08-07T17:56:47.8574230Z remote: Counting objects: 90% (2606/2895) 2024-08-07T17:56:47.8574591Z remote: Counting objects: 91% (2635/2895) 2024-08-07T17:56:47.8574952Z remote: Counting objects: 92% (2664/2895) 2024-08-07T17:56:47.8575318Z remote: Counting objects: 93% (2693/2895) 2024-08-07T17:56:47.8575696Z remote: Counting objects: 94% (2722/2895) 2024-08-07T17:56:47.8576051Z remote: Counting objects: 95% (2751/2895) 2024-08-07T17:56:47.8576416Z remote: Counting objects: 96% (2780/2895) 2024-08-07T17:56:47.8576827Z remote: Counting objects: 97% (2809/2895) 2024-08-07T17:56:47.8577189Z remote: Counting objects: 98% (2838/2895) 2024-08-07T17:56:47.8577542Z remote: Counting objects: 99% (2867/2895) 2024-08-07T17:56:47.8577905Z remote: Counting objects: 100% (2895/2895) 2024-08-07T17:56:47.8578303Z remote: Counting objects: 100% (2895/2895), done. 2024-08-07T17:56:47.8937355Z remote: Compressing objects: 0% (1/1506) 2024-08-07T17:56:47.9352499Z remote: Compressing objects: 1% (16/1506) 2024-08-07T17:56:47.9607799Z remote: Compressing objects: 2% (31/1506) 2024-08-07T17:56:48.1041398Z remote: Compressing objects: 3% (46/1506) 2024-08-07T17:56:48.2777050Z remote: Compressing objects: 4% (61/1506) 2024-08-07T17:56:48.3650563Z remote: Compressing objects: 5% (76/1506) 2024-08-07T17:56:48.4371617Z remote: Compressing objects: 6% (91/1506) 2024-08-07T17:56:48.4859482Z remote: Compressing objects: 7% (106/1506) 2024-08-07T17:56:48.5270458Z remote: Compressing objects: 8% (121/1506) 2024-08-07T17:56:48.5561081Z remote: Compressing objects: 9% (136/1506) 2024-08-07T17:56:48.5812286Z remote: Compressing objects: 10% (151/1506) 2024-08-07T17:56:48.6038911Z remote: Compressing objects: 11% (166/1506) 2024-08-07T17:56:48.6231924Z remote: Compressing objects: 12% (181/1506) 2024-08-07T17:56:48.6416595Z remote: Compressing objects: 13% (196/1506) 2024-08-07T17:56:48.6572264Z remote: Compressing objects: 14% (211/1506) 2024-08-07T17:56:48.6686273Z remote: Compressing objects: 15% (226/1506) 2024-08-07T17:56:48.6780441Z remote: Compressing objects: 16% (241/1506) 2024-08-07T17:56:48.6854132Z remote: Compressing objects: 17% (257/1506) 2024-08-07T17:56:48.6905712Z remote: Compressing objects: 18% (272/1506) 2024-08-07T17:56:48.6935650Z remote: Compressing objects: 19% (287/1506) 2024-08-07T17:56:48.6962243Z remote: Compressing objects: 20% (302/1506) 2024-08-07T17:56:48.6964175Z remote: Compressing objects: 21% (317/1506) 2024-08-07T17:56:48.6978530Z remote: Compressing objects: 22% (332/1506) 2024-08-07T17:56:48.6999342Z remote: Compressing objects: 23% (347/1506) 2024-08-07T17:56:48.7016909Z remote: Compressing objects: 24% (362/1506) 2024-08-07T17:56:48.7031573Z remote: Compressing objects: 25% (377/1506) 2024-08-07T17:56:48.7053340Z remote: Compressing objects: 26% (392/1506) 2024-08-07T17:56:48.7062479Z remote: Compressing objects: 27% (407/1506) 2024-08-07T17:56:48.7086289Z remote: Compressing objects: 28% (422/1506) 2024-08-07T17:56:48.7105962Z remote: Compressing objects: 29% (437/1506) 2024-08-07T17:56:48.7123257Z remote: Compressing objects: 30% (452/1506) 2024-08-07T17:56:48.7144864Z remote: Compressing objects: 31% (467/1506) 2024-08-07T17:56:48.7160704Z remote: Compressing objects: 32% (482/1506) 2024-08-07T17:56:48.7179174Z remote: Compressing objects: 33% (497/1506) 2024-08-07T17:56:48.7193544Z remote: Compressing objects: 34% (513/1506) 2024-08-07T17:56:48.7202310Z remote: Compressing objects: 35% (528/1506) 2024-08-07T17:56:48.7213090Z remote: Compressing objects: 36% (543/1506) 2024-08-07T17:56:48.7222798Z remote: Compressing objects: 37% (558/1506) 2024-08-07T17:56:48.7233298Z remote: Compressing objects: 38% (573/1506) 2024-08-07T17:56:48.7245662Z remote: Compressing objects: 39% (588/1506) 2024-08-07T17:56:48.7258631Z remote: Compressing objects: 40% (603/1506) 2024-08-07T17:56:48.7267918Z remote: Compressing objects: 41% (618/1506) 2024-08-07T17:56:48.7276403Z remote: Compressing objects: 42% (633/1506) 2024-08-07T17:56:48.7282907Z remote: Compressing objects: 43% (648/1506) 2024-08-07T17:56:48.7290218Z remote: Compressing objects: 44% (663/1506) 2024-08-07T17:56:48.7296881Z remote: Compressing objects: 45% (678/1506) 2024-08-07T17:56:48.7303630Z remote: Compressing objects: 46% (693/1506) 2024-08-07T17:56:48.7310142Z remote: Compressing objects: 47% (708/1506) 2024-08-07T17:56:48.7316343Z remote: Compressing objects: 48% (723/1506) 2024-08-07T17:56:48.7321353Z remote: Compressing objects: 49% (738/1506) 2024-08-07T17:56:48.7327975Z remote: Compressing objects: 50% (753/1506) 2024-08-07T17:56:48.7334210Z remote: Compressing objects: 51% (769/1506) 2024-08-07T17:56:48.7336974Z remote: Compressing objects: 52% (784/1506) 2024-08-07T17:56:48.7340161Z remote: Compressing objects: 53% (799/1506) 2024-08-07T17:56:48.7343312Z remote: Compressing objects: 54% (814/1506) 2024-08-07T17:56:48.7345102Z remote: Compressing objects: 55% (829/1506) 2024-08-07T17:56:48.7347061Z remote: Compressing objects: 56% (844/1506) 2024-08-07T17:56:48.7348747Z remote: Compressing objects: 57% (859/1506) 2024-08-07T17:56:48.7349365Z remote: Compressing objects: 58% (874/1506) 2024-08-07T17:56:48.7349755Z remote: Compressing objects: 59% (889/1506) 2024-08-07T17:56:48.7351246Z remote: Compressing objects: 60% (904/1506) 2024-08-07T17:56:48.7351642Z remote: Compressing objects: 61% (919/1506) 2024-08-07T17:56:48.7352019Z remote: Compressing objects: 62% (934/1506) 2024-08-07T17:56:48.7352421Z remote: Compressing objects: 63% (949/1506) 2024-08-07T17:56:48.7354192Z remote: Compressing objects: 64% (964/1506) 2024-08-07T17:56:48.7372996Z remote: Compressing objects: 65% (979/1506) 2024-08-07T17:56:48.7384006Z remote: Compressing objects: 66% (994/1506) 2024-08-07T17:56:48.7393276Z remote: Compressing objects: 67% (1010/1506) 2024-08-07T17:56:48.7403500Z remote: Compressing objects: 68% (1025/1506) 2024-08-07T17:56:48.7411570Z remote: Compressing objects: 69% (1040/1506) 2024-08-07T17:56:48.7418351Z remote: Compressing objects: 70% (1055/1506) 2024-08-07T17:56:48.7424129Z remote: Compressing objects: 71% (1070/1506) 2024-08-07T17:56:48.7430305Z remote: Compressing objects: 72% (1085/1506) 2024-08-07T17:56:48.7435555Z remote: Compressing objects: 73% (1100/1506) 2024-08-07T17:56:48.7440592Z remote: Compressing objects: 74% (1115/1506) 2024-08-07T17:56:48.7445333Z remote: Compressing objects: 75% (1130/1506) 2024-08-07T17:56:48.7449830Z remote: Compressing objects: 76% (1145/1506) 2024-08-07T17:56:48.7454001Z remote: Compressing objects: 77% (1160/1506) 2024-08-07T17:56:48.7458907Z remote: Compressing objects: 78% (1175/1506) 2024-08-07T17:56:48.7463414Z remote: Compressing objects: 79% (1190/1506) 2024-08-07T17:56:48.7467523Z remote: Compressing objects: 80% (1205/1506) 2024-08-07T17:56:48.7476769Z remote: Compressing objects: 81% (1220/1506) 2024-08-07T17:56:48.7477171Z remote: Compressing objects: 82% (1235/1506) 2024-08-07T17:56:48.7479402Z remote: Compressing objects: 83% (1250/1506) 2024-08-07T17:56:48.7483407Z remote: Compressing objects: 84% (1266/1506) 2024-08-07T17:56:48.7488048Z remote: Compressing objects: 85% (1281/1506) 2024-08-07T17:56:48.7492898Z remote: Compressing objects: 86% (1296/1506) 2024-08-07T17:56:48.7493875Z remote: Compressing objects: 87% (1311/1506) 2024-08-07T17:56:48.7496128Z remote: Compressing objects: 88% (1326/1506) 2024-08-07T17:56:48.7499139Z remote: Compressing objects: 89% (1341/1506) 2024-08-07T17:56:48.7502546Z remote: Compressing objects: 90% (1356/1506) 2024-08-07T17:56:48.7505359Z remote: Compressing objects: 91% (1371/1506) 2024-08-07T17:56:48.7508058Z remote: Compressing objects: 92% (1386/1506) 2024-08-07T17:56:48.7510266Z remote: Compressing objects: 93% (1401/1506) 2024-08-07T17:56:48.7512694Z remote: Compressing objects: 94% (1416/1506) 2024-08-07T17:56:48.7514539Z remote: Compressing objects: 95% (1431/1506) 2024-08-07T17:56:48.7516284Z remote: Compressing objects: 96% (1446/1506) 2024-08-07T17:56:48.7518645Z remote: Compressing objects: 97% (1461/1506) 2024-08-07T17:56:48.7520558Z remote: Compressing objects: 98% (1476/1506) 2024-08-07T17:56:48.7522232Z remote: Compressing objects: 99% (1491/1506) 2024-08-07T17:56:48.7522629Z remote: Compressing objects: 100% (1506/1506) 2024-08-07T17:56:48.7523052Z remote: Compressing objects: 100% (1506/1506), done. 2024-08-07T17:57:11.1115469Z remote: Total 1011089 (delta 1848), reused 2247 (delta 1382), pack-reused 1008194 2024-08-07T17:57:32.5750904Z [command]/usr/bin/git rev-parse --verify --quiet 016588f53c6904b840aa56aa86f95460b4d9c996^{object} 2024-08-07T17:57:32.5788791Z 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:57:32.5794457Z ##[endgroup] 2024-08-07T17:57:32.5794883Z ##[group]Determining the checkout info 2024-08-07T17:57:32.5795359Z ##[endgroup] 2024-08-07T17:57:32.5796036Z ##[group]Checking out the ref 2024-08-07T17:57:32.5798754Z [command]/usr/bin/git checkout --quiet --force 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:57:34.3017969Z ##[endgroup] 2024-08-07T17:57:34.3018416Z ##[group]Setting up auth for fetching submodules 2024-08-07T17:57:34.3022659Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-07T17:57:34.3081785Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-07T17:57:34.3123274Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-07T17:57:34.3164112Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-07T17:57:34.3202759Z ##[endgroup] 2024-08-07T17:57:34.3203158Z ##[group]Fetching submodules 2024-08-07T17:57:34.3206560Z [command]/usr/bin/git submodule sync --recursive 2024-08-07T17:57:34.3587541Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-07T17:57:34.3955207Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-07T17:57:34.3956990Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-07T17:57:34.3960525Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-07T17:57:34.3964009Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-07T17:57:34.3967233Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-07T17:57:34.3970220Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-07T17:57:34.3973685Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-07T17:57:34.3977170Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-07T17:57:34.3980873Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-07T17:57:34.3984502Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-07T17:57:34.3988174Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-07T17:57:34.3991950Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-07T17:57:34.3995776Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-07T17:57:34.3999724Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-07T17:57:34.4004443Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-07T17:57:34.4009761Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-08-07T17:57:34.4017741Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:57:34.4022159Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-07T17:57:34.4026563Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-07T17:57:34.4031015Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-07T17:57:34.4035579Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-07T17:57:34.4040237Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-07T17:57:34.4044749Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-07T17:57:34.4049385Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-07T17:57:34.4054221Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-07T17:57:34.4058885Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-07T17:57:34.4064197Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-07T17:57:34.4068977Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-07T17:57:34.4073972Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-07T17:57:34.4079057Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-07T17:57:34.4084419Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-07T17:57:34.4089445Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-07T17:57:34.4094730Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-07T17:57:34.4102115Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-07T17:57:34.4109017Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-07T17:57:34.4143633Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-07T17:57:34.7975481Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-07T17:57:34.9795912Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-07T17:57:35.2181802Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-07T17:57:35.5223972Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-07T17:57:37.4727080Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-07T17:57:49.1087845Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-07T17:57:49.5127128Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-07T17:57:49.9447123Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-07T17:57:50.5559065Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-07T17:57:51.7712245Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-07T17:57:53.7397888Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-07T17:57:59.5548227Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-07T17:58:00.7885784Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-07T17:58:02.5621851Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-07T17:58:03.8441051Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-08-07T17:58:04.0412196Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-07T17:58:04.4657237Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-07T17:58:04.8503351Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-07T17:58:06.0922760Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-07T17:58:06.4258024Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-07T17:58:06.7211880Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-07T17:58:08.3541205Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-07T17:58:09.1173066Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-07T17:58:09.8380313Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-07T17:58:16.0719581Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-07T17:58:18.0572646Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-07T17:58:21.9826813Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-07T17:58:22.2001773Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-07T17:58:30.7443465Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-07T17:58:30.9471316Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-07T17:58:31.1382400Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-07T17:58:32.0160078Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-07T17:58:32.3214505Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-07T17:58:32.9577597Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-07T17:58:33.4128478Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-07T17:58:33.4277720Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-07T17:58:33.4395560Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-07T17:58:33.4705637Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-07T17:58:33.5143858Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-07T17:58:34.6899404Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-07T17:58:34.7187616Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-07T17:58:34.7708954Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-07T17:58:34.8838695Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-07T17:58:34.9233827Z Submodule path 'third_party/cudnn_frontend': checked out '98ca4e1941fe3263f128f74f10063a3ea35c7019' 2024-08-07T17:58:35.5314406Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-07T17:58:35.8164259Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-07T17:58:35.9099206Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-07T17:58:35.9118354Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:58:35.9121968Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:58:35.9125265Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:58:35.9128826Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:58:35.9132509Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:58:35.9168275Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-07T17:58:36.8922739Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-07T17:58:37.5198778Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-07T17:58:39.4994518Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-07T17:58:40.5989986Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-07T17:58:40.9818982Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-07T17:58:41.0926209Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-07T17:58:41.5998679Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-07T17:58:41.6701195Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-07T17:58:41.6850122Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-07T17:58:41.8377218Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-07T17:58:41.8836445Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-07T17:58:41.8958576Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-08-07T17:58:41.9419514Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-07T17:58:41.9743457Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-07T17:58:42.0282582Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-07T17:58:42.0441731Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-07T17:58:42.0461061Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-07T17:58:42.0491609Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-07T17:58:55.4239814Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-07T17:58:55.4461676Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-07T17:58:55.5452368Z Submodule path 'third_party/kineto': checked out 'da2f2682cabaf95d601fa2a9b7e0979f84fe7667' 2024-08-07T17:58:55.5473817Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:58:55.5477005Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:58:55.5480548Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:58:55.5512320Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-07T17:58:56.0981846Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-07T17:58:57.3086703Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-07T17:58:58.4785319Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-07T17:58:58.4805557Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:58:58.4808903Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:58:58.4812539Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:58:58.4816058Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:58:58.4819922Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:58:58.4824483Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:58:58.4827847Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:58:58.4831772Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:58:58.4864271Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-07T17:58:59.3385474Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-07T17:58:59.7055897Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-07T17:59:00.9171543Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-07T17:59:01.2267952Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-07T17:59:01.8214077Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-07T17:59:02.8568391Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-07T17:59:10.2221502Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-07T17:59:10.6538671Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-07T17:59:10.6772682Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-07T17:59:10.7201496Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-07T17:59:10.7367794Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-07T17:59:10.7386568Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:59:10.7418948Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-07T17:59:11.0331289Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-07T17:59:11.0561855Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-07T17:59:11.1038195Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-07T17:59:11.2277212Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-07T17:59:11.2482770Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-07T17:59:11.2933889Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-07T17:59:11.3595457Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-07T17:59:11.4058260Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-07T17:59:11.4352592Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-07T17:59:11.5610238Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-07T17:59:12.0546564Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-07T17:59:12.0584998Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-07T17:59:12.0588921Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-07T17:59:12.0623398Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-07T17:59:12.5001585Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-07T17:59:13.3781949Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-07T17:59:13.4196897Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-07T17:59:13.5103972Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-07T17:59:13.5127706Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:59:13.5131089Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:59:13.5134502Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:59:13.5138034Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:59:13.5141926Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:59:13.5145545Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:59:13.5149261Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:59:13.5152815Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:59:13.5187296Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-07T17:59:13.9259683Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-07T17:59:14.9438655Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-07T17:59:15.2434253Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-07T17:59:21.3269812Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-07T17:59:21.6178720Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-07T17:59:21.8222450Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-07T17:59:22.1120948Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-07T17:59:28.2871983Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-07T17:59:28.3351953Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-07T17:59:28.3538768Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-07T17:59:28.4780964Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-07T17:59:28.4942515Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-07T17:59:28.5135243Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-07T17:59:28.5340341Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-07T17:59:28.5359414Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:59:28.5362871Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:59:28.5392731Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-07T17:59:30.3779292Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-07T17:59:31.7013142Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-07T17:59:31.7551233Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-07T17:59:32.4211068Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-07T17:59:32.4359484Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-07T17:59:32.7543775Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-07T17:59:32.7566188Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:59:32.7569513Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-07T17:59:32.7606171Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-07T17:59:33.2285741Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-07T17:59:34.2976399Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-07T17:59:34.3795334Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-07T17:59:34.3916165Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-07T17:59:34.4074545Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-07T17:59:34.4526623Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-07T17:59:34.4876835Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-07T17:59:34.5363785Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-07T17:59:34.5697833Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-07T17:59:34.5718488Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:59:34.5721802Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:59:34.5725146Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:59:34.5728687Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:59:34.5759445Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-07T17:59:35.6031282Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-07T17:59:35.8795475Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-07T17:59:37.0654875Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-07T17:59:37.9746648Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-07T17:59:37.9936361Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-07T17:59:38.0713551Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-07T17:59:38.1060441Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-07T17:59:38.1080709Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:59:38.1112130Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-07T17:59:38.3593108Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-07T17:59:38.3640649Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-07T17:59:38.4012409Z Entering 'android/libs/fbjni' 2024-08-07T17:59:38.4065036Z Entering 'third_party/FP16' 2024-08-07T17:59:38.4117767Z Entering 'third_party/FXdiv' 2024-08-07T17:59:38.4170472Z Entering 'third_party/NNPACK' 2024-08-07T17:59:38.4224160Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:59:38.4276340Z Entering 'third_party/XNNPACK' 2024-08-07T17:59:38.4345120Z Entering 'third_party/benchmark' 2024-08-07T17:59:38.4396149Z Entering 'third_party/cpp-httplib' 2024-08-07T17:59:38.4447363Z Entering 'third_party/cpuinfo' 2024-08-07T17:59:38.4498542Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:59:38.4551378Z Entering 'third_party/cutlass' 2024-08-07T17:59:38.4612941Z Entering 'third_party/eigen' 2024-08-07T17:59:38.4667419Z Entering 'third_party/fbgemm' 2024-08-07T17:59:38.4721037Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:59:38.4772347Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:59:38.4824981Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:59:38.4882268Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:59:38.4932549Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:59:38.4985305Z Entering 'third_party/flatbuffers' 2024-08-07T17:59:38.5039599Z Entering 'third_party/fmt' 2024-08-07T17:59:38.5091031Z Entering 'third_party/foxi' 2024-08-07T17:59:38.5142592Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:59:38.5193786Z Entering 'third_party/gloo' 2024-08-07T17:59:38.5246998Z Entering 'third_party/googletest' 2024-08-07T17:59:38.5297842Z Entering 'third_party/ideep' 2024-08-07T17:59:38.5347553Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:59:38.5407484Z Entering 'third_party/ittapi' 2024-08-07T17:59:38.5459937Z Entering 'third_party/kineto' 2024-08-07T17:59:38.5511297Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:59:38.5562706Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:59:38.5616048Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:59:38.5667369Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:59:38.5720662Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:59:38.5769733Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:59:38.5825275Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:59:38.5875754Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:59:38.5927244Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:59:38.5978475Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:59:38.6032728Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:59:38.6083826Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:59:38.6137908Z Entering 'third_party/mimalloc' 2024-08-07T17:59:38.6190167Z Entering 'third_party/nccl/nccl' 2024-08-07T17:59:38.6247722Z Entering 'third_party/nlohmann' 2024-08-07T17:59:38.6301422Z Entering 'third_party/onnx' 2024-08-07T17:59:38.6367267Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:59:38.6418959Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:59:38.6476646Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:59:38.6529394Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:59:38.6579184Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:59:38.6630256Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:59:38.6684535Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:59:38.6736318Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:59:38.6786034Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:59:38.6837598Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:59:38.6886673Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:59:38.6938530Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:59:38.6990926Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:59:38.7066381Z Entering 'third_party/pocketfft' 2024-08-07T17:59:38.7118255Z Entering 'third_party/protobuf' 2024-08-07T17:59:38.7172479Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:59:38.7226676Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:59:38.7280815Z Entering 'third_party/psimd' 2024-08-07T17:59:38.7332637Z Entering 'third_party/pthreadpool' 2024-08-07T17:59:38.7384084Z Entering 'third_party/pybind11' 2024-08-07T17:59:38.7435855Z Entering 'third_party/python-peachpy' 2024-08-07T17:59:38.7486924Z Entering 'third_party/sleef' 2024-08-07T17:59:38.7542864Z Entering 'third_party/tensorpipe' 2024-08-07T17:59:38.7593929Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:59:38.7644068Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:59:38.7694236Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:59:38.7748215Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:59:38.7796908Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:59:38.7872151Z ##[endgroup] 2024-08-07T17:59:38.7874608Z ##[group]Persisting credentials for submodules 2024-08-07T17:59:38.7878209Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-07T17:59:38.8244529Z Entering 'android/libs/fbjni' 2024-08-07T17:59:38.8312805Z Entering 'third_party/FP16' 2024-08-07T17:59:38.8379364Z Entering 'third_party/FXdiv' 2024-08-07T17:59:38.8452227Z Entering 'third_party/NNPACK' 2024-08-07T17:59:38.8518879Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:59:38.8586609Z Entering 'third_party/XNNPACK' 2024-08-07T17:59:38.8669663Z Entering 'third_party/benchmark' 2024-08-07T17:59:38.8741984Z Entering 'third_party/cpp-httplib' 2024-08-07T17:59:38.8809408Z Entering 'third_party/cpuinfo' 2024-08-07T17:59:38.8877568Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:59:38.8945511Z Entering 'third_party/cutlass' 2024-08-07T17:59:38.9021736Z Entering 'third_party/eigen' 2024-08-07T17:59:38.9090186Z Entering 'third_party/fbgemm' 2024-08-07T17:59:38.9161680Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:59:38.9230729Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:59:38.9297329Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:59:38.9371278Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:59:38.9438257Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:59:38.9508620Z Entering 'third_party/flatbuffers' 2024-08-07T17:59:38.9581077Z Entering 'third_party/fmt' 2024-08-07T17:59:38.9648232Z Entering 'third_party/foxi' 2024-08-07T17:59:38.9715013Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:59:38.9782323Z Entering 'third_party/gloo' 2024-08-07T17:59:38.9850937Z Entering 'third_party/googletest' 2024-08-07T17:59:38.9923264Z Entering 'third_party/ideep' 2024-08-07T17:59:38.9987567Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:59:39.0064167Z Entering 'third_party/ittapi' 2024-08-07T17:59:39.0130937Z Entering 'third_party/kineto' 2024-08-07T17:59:39.0196211Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:59:39.0261646Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:59:39.0335898Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:59:39.0403128Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:59:39.0474631Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:59:39.0540849Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:59:39.0611846Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:59:39.0679008Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:59:39.0752172Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:59:39.0821163Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:59:39.0891836Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:59:39.0959017Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:59:39.1030395Z Entering 'third_party/mimalloc' 2024-08-07T17:59:39.1098358Z Entering 'third_party/nccl/nccl' 2024-08-07T17:59:39.1165873Z Entering 'third_party/nlohmann' 2024-08-07T17:59:39.1235234Z Entering 'third_party/onnx' 2024-08-07T17:59:39.1316260Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:59:39.1383855Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:59:39.1457585Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:59:39.1526711Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:59:39.1594866Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:59:39.1662492Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:59:39.1729705Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:59:39.1798293Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:59:39.1865666Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:59:39.1932086Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:59:39.1996722Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:59:39.2066228Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:59:39.2137147Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:59:39.2229681Z Entering 'third_party/pocketfft' 2024-08-07T17:59:39.2295406Z Entering 'third_party/protobuf' 2024-08-07T17:59:39.2366680Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:59:39.2434384Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:59:39.2505865Z Entering 'third_party/psimd' 2024-08-07T17:59:39.2572001Z Entering 'third_party/pthreadpool' 2024-08-07T17:59:39.2641641Z Entering 'third_party/pybind11' 2024-08-07T17:59:39.2710630Z Entering 'third_party/python-peachpy' 2024-08-07T17:59:39.2778891Z Entering 'third_party/sleef' 2024-08-07T17:59:39.2846508Z Entering 'third_party/tensorpipe' 2024-08-07T17:59:39.2913934Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:59:39.2980590Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:59:39.3047076Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:59:39.3114833Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:59:39.3178481Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:59:39.3272092Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-07T17:59:39.3639578Z Entering 'android/libs/fbjni' 2024-08-07T17:59:39.3702369Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-07T17:59:39.3723584Z Entering 'third_party/FP16' 2024-08-07T17:59:39.3785395Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-07T17:59:39.3806705Z Entering 'third_party/FXdiv' 2024-08-07T17:59:39.3867431Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-07T17:59:39.3889347Z Entering 'third_party/NNPACK' 2024-08-07T17:59:39.3951541Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-07T17:59:39.3973141Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:59:39.4035180Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-07T17:59:39.4056613Z Entering 'third_party/XNNPACK' 2024-08-07T17:59:39.4120182Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-07T17:59:39.4158421Z Entering 'third_party/benchmark' 2024-08-07T17:59:39.4221811Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:59:39.4244595Z Entering 'third_party/cpp-httplib' 2024-08-07T17:59:39.4306863Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-07T17:59:39.4328117Z Entering 'third_party/cpuinfo' 2024-08-07T17:59:39.4389399Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-07T17:59:39.4411944Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:59:39.4473774Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-07T17:59:39.4494588Z Entering 'third_party/cutlass' 2024-08-07T17:59:39.4556581Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-07T17:59:39.4585735Z Entering 'third_party/eigen' 2024-08-07T17:59:39.4647498Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-07T17:59:39.4671149Z Entering 'third_party/fbgemm' 2024-08-07T17:59:39.4733607Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-07T17:59:39.4753931Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:59:39.4815321Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-07T17:59:39.4836227Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:59:39.4898179Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-07T17:59:39.4925065Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:59:39.4985388Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-07T17:59:39.5013346Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:59:39.5074228Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:59:39.5095125Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:59:39.5157342Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-07T17:59:39.5181037Z Entering 'third_party/flatbuffers' 2024-08-07T17:59:39.5244250Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-07T17:59:39.5267061Z Entering 'third_party/fmt' 2024-08-07T17:59:39.5328635Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-07T17:59:39.5350254Z Entering 'third_party/foxi' 2024-08-07T17:59:39.5411344Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-08-07T17:59:39.5432361Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:59:39.5493589Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-07T17:59:39.5514712Z Entering 'third_party/gloo' 2024-08-07T17:59:39.5574662Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-07T17:59:39.5595917Z Entering 'third_party/googletest' 2024-08-07T17:59:39.5657075Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:59:39.5678706Z Entering 'third_party/ideep' 2024-08-07T17:59:39.5744315Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-07T17:59:39.5763343Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:59:39.5827626Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-07T17:59:39.5856637Z Entering 'third_party/ittapi' 2024-08-07T17:59:39.5917920Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-07T17:59:39.5940665Z Entering 'third_party/kineto' 2024-08-07T17:59:39.6003349Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-07T17:59:39.6023040Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:59:39.6090289Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-07T17:59:39.6112040Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:59:39.6173971Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-07T17:59:39.6195827Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:59:39.6257211Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-07T17:59:39.6278110Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:59:39.6340692Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-07T17:59:39.6361396Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:59:39.6423525Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-07T17:59:39.6442583Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:59:39.6505967Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-07T17:59:39.6530162Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:59:39.6592007Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-07T17:59:39.6614820Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:59:39.6677813Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:59:39.6698890Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:59:39.6761038Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-07T17:59:39.6783036Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:59:39.6844931Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-07T17:59:39.6869342Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:59:39.6932138Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-07T17:59:39.6953012Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:59:39.7014140Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-07T17:59:39.7037986Z Entering 'third_party/mimalloc' 2024-08-07T17:59:39.7100380Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-07T17:59:39.7120826Z Entering 'third_party/nccl/nccl' 2024-08-07T17:59:39.7182706Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-07T17:59:39.7206116Z Entering 'third_party/nlohmann' 2024-08-07T17:59:39.7267665Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-07T17:59:39.7290159Z Entering 'third_party/onnx' 2024-08-07T17:59:39.7352950Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-07T17:59:39.7387424Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:59:39.7450533Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:59:39.7472721Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:59:39.7535659Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-07T17:59:39.7562941Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:59:39.7626765Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-07T17:59:39.7647806Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:59:39.7709651Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:59:39.7729593Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:59:39.7791332Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:59:39.7811443Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:59:39.7876980Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-07T17:59:39.7901730Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:59:39.7963336Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-07T17:59:39.7987673Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:59:39.8048845Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-07T17:59:39.8069485Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:59:39.8130875Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-07T17:59:39.8152119Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:59:39.8215157Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-07T17:59:39.8233868Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:59:39.8296137Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-07T17:59:39.8319496Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:59:39.8381628Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-07T17:59:39.8408632Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:59:39.8468573Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-07T17:59:39.8511275Z Entering 'third_party/pocketfft' 2024-08-07T17:59:39.8572135Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-07T17:59:39.8592381Z Entering 'third_party/protobuf' 2024-08-07T17:59:39.8653490Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-07T17:59:39.8675643Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:59:39.8738788Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:59:39.8760465Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:59:39.8822602Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:59:39.8847687Z Entering 'third_party/psimd' 2024-08-07T17:59:39.8910185Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-07T17:59:39.8931070Z Entering 'third_party/pthreadpool' 2024-08-07T17:59:39.8992837Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-07T17:59:39.9016055Z Entering 'third_party/pybind11' 2024-08-07T17:59:39.9077134Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-07T17:59:39.9098593Z Entering 'third_party/python-peachpy' 2024-08-07T17:59:39.9160345Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-07T17:59:39.9181789Z Entering 'third_party/sleef' 2024-08-07T17:59:39.9243724Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-07T17:59:39.9265232Z Entering 'third_party/tensorpipe' 2024-08-07T17:59:39.9329309Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-07T17:59:39.9349553Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:59:39.9410592Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:59:39.9431625Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:59:39.9492575Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-07T17:59:39.9513808Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:59:39.9574498Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-07T17:59:39.9595577Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:59:39.9657291Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-07T17:59:39.9676706Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:59:39.9742308Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-07T17:59:40.0558642Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-07T17:59:40.0932273Z Entering 'android/libs/fbjni' 2024-08-07T17:59:40.0984982Z Entering 'third_party/FP16' 2024-08-07T17:59:40.1038038Z Entering 'third_party/FXdiv' 2024-08-07T17:59:40.1089886Z Entering 'third_party/NNPACK' 2024-08-07T17:59:40.1148165Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:59:40.1200895Z Entering 'third_party/XNNPACK' 2024-08-07T17:59:40.1274001Z Entering 'third_party/benchmark' 2024-08-07T17:59:40.1327768Z Entering 'third_party/cpp-httplib' 2024-08-07T17:59:40.1381674Z Entering 'third_party/cpuinfo' 2024-08-07T17:59:40.1434859Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:59:40.1487292Z Entering 'third_party/cutlass' 2024-08-07T17:59:40.1549617Z Entering 'third_party/eigen' 2024-08-07T17:59:40.1608072Z Entering 'third_party/fbgemm' 2024-08-07T17:59:40.1659334Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:59:40.1710804Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:59:40.1761204Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:59:40.1817964Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:59:40.1871970Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:59:40.1925249Z Entering 'third_party/flatbuffers' 2024-08-07T17:59:40.1980351Z Entering 'third_party/fmt' 2024-08-07T17:59:40.2032729Z Entering 'third_party/foxi' 2024-08-07T17:59:40.2084504Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:59:40.2137518Z Entering 'third_party/gloo' 2024-08-07T17:59:40.2189928Z Entering 'third_party/googletest' 2024-08-07T17:59:40.2242731Z Entering 'third_party/ideep' 2024-08-07T17:59:40.2293024Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:59:40.2353025Z Entering 'third_party/ittapi' 2024-08-07T17:59:40.2405826Z Entering 'third_party/kineto' 2024-08-07T17:59:40.2456268Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:59:40.2506882Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:59:40.2559271Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:59:40.2616864Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:59:40.2670120Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:59:40.2720954Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:59:40.2776516Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:59:40.2828779Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:59:40.2884733Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:59:40.2937566Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:59:40.2992318Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:59:40.3042250Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:59:40.3095359Z Entering 'third_party/mimalloc' 2024-08-07T17:59:40.3147493Z Entering 'third_party/nccl/nccl' 2024-08-07T17:59:40.3200321Z Entering 'third_party/nlohmann' 2024-08-07T17:59:40.3254280Z Entering 'third_party/onnx' 2024-08-07T17:59:40.3320963Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:59:40.3377704Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:59:40.3435670Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:59:40.3488838Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:59:40.3542124Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:59:40.3591980Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:59:40.3642668Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:59:40.3693571Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:59:40.3743944Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:59:40.3793946Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:59:40.3843246Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:59:40.3899310Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:59:40.3956425Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:59:40.4028831Z Entering 'third_party/pocketfft' 2024-08-07T17:59:40.4080753Z Entering 'third_party/protobuf' 2024-08-07T17:59:40.4143445Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:59:40.4194489Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:59:40.4250027Z Entering 'third_party/psimd' 2024-08-07T17:59:40.4302502Z Entering 'third_party/pthreadpool' 2024-08-07T17:59:40.4354390Z Entering 'third_party/pybind11' 2024-08-07T17:59:40.4407063Z Entering 'third_party/python-peachpy' 2024-08-07T17:59:40.4460651Z Entering 'third_party/sleef' 2024-08-07T17:59:40.4513422Z Entering 'third_party/tensorpipe' 2024-08-07T17:59:40.4564850Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:59:40.4620184Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:59:40.4669747Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:59:40.4724792Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:59:40.4773537Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:59:40.4850044Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-07T17:59:40.5222316Z Entering 'android/libs/fbjni' 2024-08-07T17:59:40.5274327Z Entering 'third_party/FP16' 2024-08-07T17:59:40.5327412Z Entering 'third_party/FXdiv' 2024-08-07T17:59:40.5379444Z Entering 'third_party/NNPACK' 2024-08-07T17:59:40.5433316Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:59:40.5486470Z Entering 'third_party/XNNPACK' 2024-08-07T17:59:40.5557199Z Entering 'third_party/benchmark' 2024-08-07T17:59:40.5609932Z Entering 'third_party/cpp-httplib' 2024-08-07T17:59:40.5662939Z Entering 'third_party/cpuinfo' 2024-08-07T17:59:40.5716306Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:59:40.5769016Z Entering 'third_party/cutlass' 2024-08-07T17:59:40.5829652Z Entering 'third_party/eigen' 2024-08-07T17:59:40.5883704Z Entering 'third_party/fbgemm' 2024-08-07T17:59:40.5935728Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:59:40.5986832Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:59:40.6038280Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:59:40.6094947Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:59:40.6148767Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:59:40.6201998Z Entering 'third_party/flatbuffers' 2024-08-07T17:59:40.6258832Z Entering 'third_party/fmt' 2024-08-07T17:59:40.6311467Z Entering 'third_party/foxi' 2024-08-07T17:59:40.6363639Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:59:40.6416376Z Entering 'third_party/gloo' 2024-08-07T17:59:40.6468718Z Entering 'third_party/googletest' 2024-08-07T17:59:40.6521311Z Entering 'third_party/ideep' 2024-08-07T17:59:40.6571229Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:59:40.6629447Z Entering 'third_party/ittapi' 2024-08-07T17:59:40.6682167Z Entering 'third_party/kineto' 2024-08-07T17:59:40.6734245Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:59:40.6783948Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:59:40.6838145Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:59:40.6889011Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:59:40.6943349Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:59:40.6991111Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:59:40.7046199Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:59:40.7096963Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:59:40.7149976Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:59:40.7202702Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:59:40.7258063Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:59:40.7315697Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:59:40.7369387Z Entering 'third_party/mimalloc' 2024-08-07T17:59:40.7422972Z Entering 'third_party/nccl/nccl' 2024-08-07T17:59:40.7475192Z Entering 'third_party/nlohmann' 2024-08-07T17:59:40.7528640Z Entering 'third_party/onnx' 2024-08-07T17:59:40.7594548Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:59:40.7646628Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:59:40.7703378Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:59:40.7755431Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:59:40.7805699Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:59:40.7855806Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:59:40.7905941Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:59:40.7956853Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:59:40.8007656Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:59:40.8057664Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:59:40.8108347Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:59:40.8160667Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:59:40.8213858Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:59:40.8285718Z Entering 'third_party/pocketfft' 2024-08-07T17:59:40.8337162Z Entering 'third_party/protobuf' 2024-08-07T17:59:40.8390065Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:59:40.8441762Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:59:40.8501265Z Entering 'third_party/psimd' 2024-08-07T17:59:40.8555049Z Entering 'third_party/pthreadpool' 2024-08-07T17:59:40.8608030Z Entering 'third_party/pybind11' 2024-08-07T17:59:40.8662579Z Entering 'third_party/python-peachpy' 2024-08-07T17:59:40.8717773Z Entering 'third_party/sleef' 2024-08-07T17:59:40.8771537Z Entering 'third_party/tensorpipe' 2024-08-07T17:59:40.8824283Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:59:40.8876590Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:59:40.8927137Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:59:40.8977686Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:59:40.9027779Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:59:40.9101484Z ##[endgroup] 2024-08-07T17:59:40.9152382Z [command]/usr/bin/git log -1 --format='%H' 2024-08-07T17:59:40.9190337Z '016588f53c6904b840aa56aa86f95460b4d9c996' 2024-08-07T17:59:40.9349861Z Prepare all required actions 2024-08-07T17:59:40.9350365Z Getting action download info 2024-08-07T17:59:41.0727419Z ##[group]Run ./.github/actions/setup-linux 2024-08-07T17:59:41.0727740Z env: 2024-08-07T17:59:41.0727958Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:59:41.0728225Z ##[endgroup] 2024-08-07T17:59:41.0775364Z ##[group]Run set -euo pipefail 2024-08-07T17:59:41.0775700Z set -euo pipefail 2024-08-07T17:59:41.0775993Z function get_ec2_metadata() { 2024-08-07T17:59:41.0776369Z  # Pulled from instance metadata endpoint for EC2 2024-08-07T17:59:41.0776975Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-07T17:59:41.0777508Z  category=$1 2024-08-07T17:59:41.0777871Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-07T17:59:41.0778312Z  runner_name_str=i-031f7aad237b89ef3 2024-08-07T17:59:41.0778673Z  if [[ -f /.inarc ]]; then 2024-08-07T17:59:41.0779017Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-07T17:59:41.0779389Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-07T17:59:41.0779836Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-07T17:59:41.0780360Z  else 2024-08-07T17:59:41.0780704Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-07T17:59:41.0781108Z  fi 2024-08-07T17:59:41.0781336Z } 2024-08-07T17:59:41.0781605Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-07T17:59:41.0782025Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-07T17:59:41.0782487Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-07T17:59:41.0782887Z echo "system info $(uname -a)" 2024-08-07T17:59:41.0792365Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:59:41.0792734Z env: 2024-08-07T17:59:41.0792958Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:59:41.0793236Z ##[endgroup] 2024-08-07T17:59:41.0889961Z ami-id: ami-06c68f701d8090592 2024-08-07T17:59:41.0952408Z instance-id: i-031f7aad237b89ef3 2024-08-07T17:59:41.1012228Z instance-type: g5.4xlarge 2024-08-07T17:59:41.1026586Z system info Linux ip-10-0-38-158.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-07T17:59:41.1057481Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-07T17:59:41.1058107Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-07T17:59:41.1066681Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:59:41.1067050Z env: 2024-08-07T17:59:41.1067297Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:59:41.1067556Z ##[endgroup] 2024-08-07T17:59:41.1149990Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-07T17:59:41.1150459Z if systemctl is-active --quiet docker; then 2024-08-07T17:59:41.1150841Z  echo "Docker daemon is running..."; 2024-08-07T17:59:41.1151169Z else 2024-08-07T17:59:41.1151523Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-07T17:59:41.1151916Z fi 2024-08-07T17:59:41.1160725Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:59:41.1161098Z env: 2024-08-07T17:59:41.1161320Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:59:41.1161593Z ##[endgroup] 2024-08-07T17:59:41.1249316Z Docker daemon is running... 2024-08-07T17:59:41.1293572Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-07T17:59:41.1293972Z with: 2024-08-07T17:59:41.1294191Z shell: bash 2024-08-07T17:59:41.1294607Z timeout_minutes: 5 2024-08-07T17:59:41.1294857Z max_attempts: 3 2024-08-07T17:59:41.1295107Z retry_wait_seconds: 30 2024-08-07T17:59:41.1296290Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-08-07T17:59:41.1297305Z polling_interval_seconds: 1 2024-08-07T17:59:41.1297595Z warning_on_retry: true 2024-08-07T17:59:41.1297869Z continue_on_error: false 2024-08-07T17:59:41.1298132Z env: 2024-08-07T17:59:41.1298349Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:59:41.1298630Z AWS_RETRY_MODE: standard 2024-08-07T17:59:41.1298900Z AWS_MAX_ATTEMPTS: 5 2024-08-07T17:59:41.1299163Z AWS_DEFAULT_REGION: us-east-1 2024-08-07T17:59:41.1299447Z ##[endgroup] 2024-08-07T17:59:42.2914155Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-07T17:59:42.2914857Z Configure a credential helper to remove this warning. See 2024-08-07T17:59:42.2915711Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-07T17:59:42.2916138Z 2024-08-07T17:59:42.2916278Z Login Succeeded 2024-08-07T17:59:43.1867663Z Command completed after 1 attempt(s). 2024-08-07T17:59:43.1943994Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-07T17:59:43.1944612Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-07T17:59:43.1945110Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-07T17:59:43.1954131Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:59:43.1954570Z env: 2024-08-07T17:59:43.1954930Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:59:43.1955321Z ##[endgroup] 2024-08-07T17:59:43.2058943Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T17:59:43.2059609Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T17:59:43.2060279Z # shellcheck disable=SC2046 2024-08-07T17:59:43.2060763Z docker stop $(docker ps -q) || true 2024-08-07T17:59:43.2061227Z # Prune all of the docker images 2024-08-07T17:59:43.2061644Z docker system prune -af 2024-08-07T17:59:43.2069802Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:59:43.2070221Z env: 2024-08-07T17:59:43.2070602Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:59:43.2070981Z ##[endgroup] 2024-08-07T17:59:43.2385106Z "docker stop" requires at least 1 argument. 2024-08-07T17:59:43.2386156Z See 'docker stop --help'. 2024-08-07T17:59:43.2386491Z 2024-08-07T17:59:43.2386750Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-07T17:59:43.2387207Z 2024-08-07T17:59:43.2387551Z Stop one or more running containers 2024-08-07T17:59:43.2562907Z Total reclaimed space: 0B 2024-08-07T17:59:43.2626499Z ##[group]Run set +e 2024-08-07T17:59:43.2626814Z set +e 2024-08-07T17:59:43.2627067Z set -x 2024-08-07T17:59:43.2627291Z  2024-08-07T17:59:43.2627550Z PT_DOMAIN=download.pytorch.org 2024-08-07T17:59:43.2628116Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-07T17:59:43.2628839Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-07T17:59:43.2629335Z # one is returned at random 2024-08-07T17:59:43.2629727Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-07T17:59:43.2630096Z  2024-08-07T17:59:43.2630334Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-07T17:59:43.2630764Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-07T17:59:43.2631271Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-07T17:59:43.2631647Z  2024-08-07T17:59:43.2631898Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-07T17:59:43.2632477Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-07T17:59:43.2632830Z  exit 1 2024-08-07T17:59:43.2633077Z  fi 2024-08-07T17:59:43.2633467Z fi 2024-08-07T17:59:43.2633686Z  2024-08-07T17:59:43.2633956Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-07T17:59:43.2634322Z  # Clean up any old records first 2024-08-07T17:59:43.2634677Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-07T17:59:43.2635000Z fi 2024-08-07T17:59:43.2635218Z  2024-08-07T17:59:43.2635526Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-07T17:59:43.2635910Z cat /etc/hosts 2024-08-07T17:59:43.2644631Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:59:43.2644993Z env: 2024-08-07T17:59:43.2645217Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:59:43.2645487Z ##[endgroup] 2024-08-07T17:59:43.2675500Z + PT_DOMAIN=download.pytorch.org 2024-08-07T17:59:43.2680702Z ++ dig -4 +short download.pytorch.org 2024-08-07T17:59:43.2681227Z ++ tail -n1 2024-08-07T17:59:43.2930003Z + RESOLVED_IP=18.160.10.36 2024-08-07T17:59:43.2930303Z + '[' -z 18.160.10.36 ']' 2024-08-07T17:59:43.2930597Z + grep -r download.pytorch.org /etc/hosts 2024-08-07T17:59:43.2944147Z 18.160.10.36 download.pytorch.org 2024-08-07T17:59:43.2946108Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-08-07T17:59:43.4302963Z + echo '18.160.10.36 download.pytorch.org' 2024-08-07T17:59:43.4303435Z + sudo tee -a /etc/hosts 2024-08-07T17:59:43.4625376Z 18.160.10.36 download.pytorch.org 2024-08-07T17:59:43.4646688Z + cat /etc/hosts 2024-08-07T17:59:43.4658992Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-07T17:59:43.4665290Z ::1 localhost6 localhost6.localdomain6 2024-08-07T17:59:43.4665644Z 18.160.10.36 download.pytorch.org 2024-08-07T17:59:43.4781602Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-07T17:59:43.4782116Z with: 2024-08-07T17:59:43.4782816Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:43.4783584Z docker-build-dir: .ci/docker 2024-08-07T17:59:43.4783868Z working-directory: . 2024-08-07T17:59:43.4784212Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:59:43.4784591Z force-push: false 2024-08-07T17:59:43.4784818Z env: 2024-08-07T17:59:43.4785031Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:59:43.4785289Z ##[endgroup] 2024-08-07T17:59:43.4809001Z ##[group]Run set -ex 2024-08-07T17:59:43.4809302Z set -ex 2024-08-07T17:59:43.4809541Z  2024-08-07T17:59:43.4809932Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-07T17:59:43.4810593Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-07T17:59:43.4811131Z # job could then download the pre-built image as usual 2024-08-07T17:59:43.4811632Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-07T17:59:43.4812086Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-07T17:59:43.4812521Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:59:43.4812909Z  2024-08-07T17:59:43.4813260Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-07T17:59:43.4813680Z  exit 0 2024-08-07T17:59:43.4813908Z else 2024-08-07T17:59:43.4814179Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-07T17:59:43.4814496Z fi 2024-08-07T17:59:43.4814701Z  2024-08-07T17:59:43.4815030Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-07T17:59:43.4815591Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-07T17:59:43.4816251Z  # use it as it is, but first let's extract the tag 2024-08-07T17:59:43.4816715Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-07T17:59:43.4817198Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:59:43.4817660Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:59:43.4818052Z else 2024-08-07T17:59:43.4818358Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-07T17:59:43.4818788Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:59:43.4819382Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:59:43.4819897Z fi 2024-08-07T17:59:43.4828481Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:59:43.4828841Z env: 2024-08-07T17:59:43.4829065Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:59:43.4829338Z REPO_NAME: pytorch 2024-08-07T17:59:43.4830063Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:43.4830827Z DOCKER_BUILD_DIR: .ci/docker 2024-08-07T17:59:43.4831196Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:59:43.4831568Z ##[endgroup] 2024-08-07T17:59:43.4859628Z + [[ ! -d .ci/docker ]] 2024-08-07T17:59:43.4859931Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-07T17:59:43.4860396Z + echo skip=false 2024-08-07T17:59:43.4861356Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-07T17:59:43.4867723Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:43.4868483Z ++ awk -F '[:,]' '{print $2}' 2024-08-07T17:59:43.4892026Z + DOCKER_TAG=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:43.4892458Z + echo docker-tag=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:43.4893292Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:43.4926173Z ##[group]Run set +e 2024-08-07T17:59:43.4926476Z set +e 2024-08-07T17:59:43.4926715Z set -x 2024-08-07T17:59:43.4926948Z  2024-08-07T17:59:43.4927171Z login() { 2024-08-07T17:59:43.4927631Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-07T17:59:43.4928154Z } 2024-08-07T17:59:43.4928380Z  2024-08-07T17:59:43.4928613Z retry () { 2024-08-07T17:59:43.4928888Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-07T17:59:43.4929209Z } 2024-08-07T17:59:43.4929426Z  2024-08-07T17:59:43.4929661Z retry login "${DOCKER_REGISTRY}" 2024-08-07T17:59:43.4929964Z  2024-08-07T17:59:43.4930286Z # Check if image already exists, if it does then skip building it 2024-08-07T17:59:43.4930754Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-07T17:59:43.4931108Z  exit 0 2024-08-07T17:59:43.4931334Z fi 2024-08-07T17:59:43.4931539Z  2024-08-07T17:59:43.4931880Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-07T17:59:43.4932425Z # be empty. The default action would be to continue rebuild the image 2024-08-07T17:59:43.4932911Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-07T17:59:43.4933350Z  # if we're on the base branch then use the parent commit 2024-08-07T17:59:43.4933739Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-07T17:59:43.4934207Z else 2024-08-07T17:59:43.4934542Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-07T17:59:43.4935011Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-07T17:59:43.4935374Z fi 2024-08-07T17:59:43.4935590Z  2024-08-07T17:59:43.4935841Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-07T17:59:43.4936202Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-07T17:59:43.4936520Z  2024-08-07T17:59:43.4936961Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-07T17:59:43.4937478Z  exit 0 2024-08-07T17:59:43.4937698Z fi 2024-08-07T17:59:43.4937914Z  2024-08-07T17:59:43.4938223Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-07T17:59:43.4938861Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-07T17:59:43.4939411Z  exit 1 2024-08-07T17:59:43.4939640Z fi 2024-08-07T17:59:43.4939846Z  2024-08-07T17:59:43.4940339Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-07T17:59:43.4940961Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-07T17:59:43.4941522Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-07T17:59:43.4942212Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-07T17:59:43.4942938Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-07T17:59:43.4943386Z fi 2024-08-07T17:59:43.4943606Z  2024-08-07T17:59:43.4944035Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-07T17:59:43.4951985Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:59:43.4952361Z env: 2024-08-07T17:59:43.4952591Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:59:43.4952868Z DOCKER_BUILD_DIR: .ci/docker 2024-08-07T17:59:43.4953210Z BASE_REVISION: 6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T17:59:43.4954017Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:43.4954825Z DOCKER_TAG: 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:43.4955264Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:59:43.4955643Z ##[endgroup] 2024-08-07T17:59:43.4982200Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:59:43.4982632Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:59:43.4985293Z + aws ecr get-login-password --region us-east-1 2024-08-07T17:59:43.4986915Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:59:44.0420930Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-07T17:59:44.0421509Z Configure a credential helper to remove this warning. See 2024-08-07T17:59:44.0422087Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-07T17:59:44.0422468Z 2024-08-07T17:59:44.0422613Z Login Succeeded 2024-08-07T17:59:44.0444862Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:44.2752950Z { 2024-08-07T17:59:44.2753303Z "schemaVersion": 2, 2024-08-07T17:59:44.2753797Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-07T17:59:44.2754224Z "config": { 2024-08-07T17:59:44.2754572Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-07T17:59:44.2754959Z "size": 48439, 2024-08-07T17:59:44.2755360Z "digest": "sha256:6ec36276acd88c9be8b44d856744037d399b35f4bb1703e637c27ae2b254c901" 2024-08-07T17:59:44.2756074Z }, 2024-08-07T17:59:44.2756284Z "layers": [ 2024-08-07T17:59:44.2756502Z { 2024-08-07T17:59:44.2756834Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2757237Z "size": 28580681, 2024-08-07T17:59:44.2757647Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-08-07T17:59:44.2758102Z }, 2024-08-07T17:59:44.2758314Z { 2024-08-07T17:59:44.2758668Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2759075Z "size": 7943451, 2024-08-07T17:59:44.2759465Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-08-07T17:59:44.2759908Z }, 2024-08-07T17:59:44.2760115Z { 2024-08-07T17:59:44.2760447Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2760851Z "size": 55728572, 2024-08-07T17:59:44.2761258Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-08-07T17:59:44.2761713Z }, 2024-08-07T17:59:44.2761915Z { 2024-08-07T17:59:44.2762236Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2762642Z "size": 186, 2024-08-07T17:59:44.2763050Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-08-07T17:59:44.2763498Z }, 2024-08-07T17:59:44.2763701Z { 2024-08-07T17:59:44.2764028Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2764428Z "size": 6886, 2024-08-07T17:59:44.2764831Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-08-07T17:59:44.2765280Z }, 2024-08-07T17:59:44.2765474Z { 2024-08-07T17:59:44.2765803Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2766464Z "size": 1329001756, 2024-08-07T17:59:44.2766914Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-08-07T17:59:44.2767388Z }, 2024-08-07T17:59:44.2767585Z { 2024-08-07T17:59:44.2767906Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2768306Z "size": 62501, 2024-08-07T17:59:44.2768704Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-08-07T17:59:44.2769138Z }, 2024-08-07T17:59:44.2769335Z { 2024-08-07T17:59:44.2769655Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2770048Z "size": 1684, 2024-08-07T17:59:44.2770435Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-08-07T17:59:44.2770893Z }, 2024-08-07T17:59:44.2771115Z { 2024-08-07T17:59:44.2771449Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2771861Z "size": 1523, 2024-08-07T17:59:44.2772278Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-08-07T17:59:44.2772772Z }, 2024-08-07T17:59:44.2772999Z { 2024-08-07T17:59:44.2773330Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2773738Z "size": 2528295403, 2024-08-07T17:59:44.2774155Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-08-07T17:59:44.2774600Z }, 2024-08-07T17:59:44.2774799Z { 2024-08-07T17:59:44.2775124Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2775523Z "size": 86016, 2024-08-07T17:59:44.2775935Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-08-07T17:59:44.2776395Z }, 2024-08-07T17:59:44.2776589Z { 2024-08-07T17:59:44.2776915Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2777321Z "size": 1823, 2024-08-07T17:59:44.2777718Z "digest": "sha256:66b43372aa397c4303ca4e0e1122516909bca0c87b9b4bfb3972b8fd0c1d4390" 2024-08-07T17:59:44.2778164Z }, 2024-08-07T17:59:44.2778462Z { 2024-08-07T17:59:44.2778782Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2779190Z "size": 246768020, 2024-08-07T17:59:44.2779694Z "digest": "sha256:b6662193c745ec6b991e800e920c233379c7c0e74f2f64d9b82dd5dc4a27eb14" 2024-08-07T17:59:44.2780395Z }, 2024-08-07T17:59:44.2780656Z { 2024-08-07T17:59:44.2781086Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2781627Z "size": 545, 2024-08-07T17:59:44.2782176Z "digest": "sha256:5be2b638d110dd5ed631ce7ddf7eefa26b3abd49cf3ab845be5ecb3daec46b67" 2024-08-07T17:59:44.2782791Z }, 2024-08-07T17:59:44.2783047Z { 2024-08-07T17:59:44.2783494Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2785520Z "size": 1283, 2024-08-07T17:59:44.2786140Z "digest": "sha256:71ca63790839b9bfa870ee6927d5d7b60aaa1fc65b38d3e8fc42ace8911859ef" 2024-08-07T17:59:44.2786622Z }, 2024-08-07T17:59:44.2786819Z { 2024-08-07T17:59:44.2787156Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2787577Z "size": 484, 2024-08-07T17:59:44.2787977Z "digest": "sha256:8a74804dc4fa9ad5369e1ae6677a4e17bcc2c53d209a67738dbc795420066650" 2024-08-07T17:59:44.2788431Z }, 2024-08-07T17:59:44.2788635Z { 2024-08-07T17:59:44.2788959Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2789370Z "size": 91712377, 2024-08-07T17:59:44.2789792Z "digest": "sha256:3bacb5389b745ab1f7590db3db714e689a99ee0d7c709f907ccd6906f39905c5" 2024-08-07T17:59:44.2790243Z }, 2024-08-07T17:59:44.2790441Z { 2024-08-07T17:59:44.2790768Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2791173Z "size": 3231, 2024-08-07T17:59:44.2791570Z "digest": "sha256:a8911a72541a4ab35894015b7fb1174ea61c59fedc863dfa563324af5d6ae752" 2024-08-07T17:59:44.2792144Z }, 2024-08-07T17:59:44.2792340Z { 2024-08-07T17:59:44.2792675Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2793086Z "size": 1909, 2024-08-07T17:59:44.2793475Z "digest": "sha256:55d020986bb7c1702235b111c4b83d990fa63ce6045c5ac358a026832bbe8550" 2024-08-07T17:59:44.2793918Z }, 2024-08-07T17:59:44.2794114Z { 2024-08-07T17:59:44.2794435Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2794841Z "size": 700, 2024-08-07T17:59:44.2795241Z "digest": "sha256:679e209a81f89d0be588ce19c3f5191f73883a86e44ab7b3653a3be3f267b69e" 2024-08-07T17:59:44.2795682Z }, 2024-08-07T17:59:44.2795878Z { 2024-08-07T17:59:44.2796205Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2796610Z "size": 2785856582, 2024-08-07T17:59:44.2797025Z "digest": "sha256:d4fb7093f54f7f71e63223ef934b9ab258d53922a199ac4736897cdb90df0683" 2024-08-07T17:59:44.2797472Z }, 2024-08-07T17:59:44.2797670Z { 2024-08-07T17:59:44.2797997Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2798403Z "size": 381, 2024-08-07T17:59:44.2798808Z "digest": "sha256:0d8ab4023e81a9284aef759a1b3c759a907d0cbd39361f3ef0ce4f8c3994f882" 2024-08-07T17:59:44.2799263Z }, 2024-08-07T17:59:44.2799456Z { 2024-08-07T17:59:44.2800127Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2800572Z "size": 12876, 2024-08-07T17:59:44.2800975Z "digest": "sha256:bf191f5f5a0a370ba7136fa618cd8cb1eb76e5f82b8c5773a965cdd105515924" 2024-08-07T17:59:44.2801424Z }, 2024-08-07T17:59:44.2801614Z { 2024-08-07T17:59:44.2801928Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2802324Z "size": 803, 2024-08-07T17:59:44.2802723Z "digest": "sha256:14653e4e245feef24e0aabd8a4cd81c24298f800facc0299f797b161da696a1d" 2024-08-07T17:59:44.2803164Z }, 2024-08-07T17:59:44.2803358Z { 2024-08-07T17:59:44.2803687Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2804079Z "size": 106, 2024-08-07T17:59:44.2804692Z "digest": "sha256:8bdbb000c39dd99342429f8a1183bdb36f312b532ea7e47eb7719fea84c669f6" 2024-08-07T17:59:44.2805144Z }, 2024-08-07T17:59:44.2805337Z { 2024-08-07T17:59:44.2805673Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2806085Z "size": 504, 2024-08-07T17:59:44.2806479Z "digest": "sha256:277383b63c0797c1bd9e23c6f38d6ba85e6e321e2dc6b21fcd832f1935f5af87" 2024-08-07T17:59:44.2806926Z }, 2024-08-07T17:59:44.2807127Z { 2024-08-07T17:59:44.2807449Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2807858Z "size": 121477300, 2024-08-07T17:59:44.2808276Z "digest": "sha256:890313244493db7d65ed3f1cf91a94e6e50bbdb4df87b5bb829a1a3236ffaeb3" 2024-08-07T17:59:44.2808721Z }, 2024-08-07T17:59:44.2808921Z { 2024-08-07T17:59:44.2809261Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2809665Z "size": 109, 2024-08-07T17:59:44.2810083Z "digest": "sha256:f1e3cc0f57ee16caa6ffefa72c065dfe99a5d19a3a352342dfa26b63661589a2" 2024-08-07T17:59:44.2810556Z }, 2024-08-07T17:59:44.2810749Z { 2024-08-07T17:59:44.2811081Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2811482Z "size": 491, 2024-08-07T17:59:44.2811891Z "digest": "sha256:c3cbae3fe054ce8c713ed90c42a306ecc164d8256fd73a14ff7b0e088e150b3f" 2024-08-07T17:59:44.2812350Z }, 2024-08-07T17:59:44.2812543Z { 2024-08-07T17:59:44.2812871Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2813276Z "size": 296, 2024-08-07T17:59:44.2813688Z "digest": "sha256:ccc148c4e7590ced33e52f40edecd2d5ec73cb4a42c87dacaf5c5a7a3912c17b" 2024-08-07T17:59:44.2814151Z }, 2024-08-07T17:59:44.2814348Z { 2024-08-07T17:59:44.2814670Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2815201Z "size": 103, 2024-08-07T17:59:44.2815607Z "digest": "sha256:7912f8c8e80ddc0dfc068c1282e6bd0ffd098b02458818c2c7b52a89c41d8335" 2024-08-07T17:59:44.2816058Z }, 2024-08-07T17:59:44.2816257Z { 2024-08-07T17:59:44.2816585Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2816986Z "size": 1473, 2024-08-07T17:59:44.2817383Z "digest": "sha256:d166ebb28213d6d30940b4fb9739863e9200174e7b550a1591e9028b1a039f83" 2024-08-07T17:59:44.2817823Z }, 2024-08-07T17:59:44.2818015Z { 2024-08-07T17:59:44.2818346Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2818754Z "size": 424146463, 2024-08-07T17:59:44.2819162Z "digest": "sha256:63bf315f789a755602aeb163e43e8173bc191c3dabc75e39ab31d0762bacc84f" 2024-08-07T17:59:44.2819613Z }, 2024-08-07T17:59:44.2819813Z { 2024-08-07T17:59:44.2820234Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2820644Z "size": 159, 2024-08-07T17:59:44.2821046Z "digest": "sha256:bdb818f7b2c8404f3e19777a27592349798986185a1f5b539309bbe8ea96e513" 2024-08-07T17:59:44.2821484Z }, 2024-08-07T17:59:44.2821681Z { 2024-08-07T17:59:44.2822015Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2822419Z "size": 566, 2024-08-07T17:59:44.2822826Z "digest": "sha256:89d8aea05b3a5e45fc1c48daf5ac32901006f7804ce5f2104112c2a2136acf28" 2024-08-07T17:59:44.2823280Z }, 2024-08-07T17:59:44.2823472Z { 2024-08-07T17:59:44.2823802Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2824209Z "size": 35874371, 2024-08-07T17:59:44.2824613Z "digest": "sha256:f1122e19f79064bde97285bf17ca6d8abb889972e5d95a463ffd2382145c1f22" 2024-08-07T17:59:44.2825061Z }, 2024-08-07T17:59:44.2825254Z { 2024-08-07T17:59:44.2825586Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2825993Z "size": 104, 2024-08-07T17:59:44.2826381Z "digest": "sha256:13d6ce3185e9912952041a572e2efa85b4544ec540f6050f750093d180a069f6" 2024-08-07T17:59:44.2826824Z }, 2024-08-07T17:59:44.2827121Z { 2024-08-07T17:59:44.2827443Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2827847Z "size": 425, 2024-08-07T17:59:44.2828250Z "digest": "sha256:feb3f80c392d4aef71730a9673030e955ce0e8a5c41f350eb7a00592d6b0dbb3" 2024-08-07T17:59:44.2828700Z }, 2024-08-07T17:59:44.2828896Z { 2024-08-07T17:59:44.2829220Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2829621Z "size": 20262075, 2024-08-07T17:59:44.2830034Z "digest": "sha256:4fe4cdcdfbd890964b8270a9140a5bf255709a21af4401b0428d91a735e8ac12" 2024-08-07T17:59:44.2830482Z }, 2024-08-07T17:59:44.2830674Z { 2024-08-07T17:59:44.2831001Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2831404Z "size": 440, 2024-08-07T17:59:44.2831813Z "digest": "sha256:be10b99d8ac8cfa04842a726627b1bdc764d3b6f1c591dca7933b86c93208c66" 2024-08-07T17:59:44.2832266Z }, 2024-08-07T17:59:44.2832463Z { 2024-08-07T17:59:44.2832784Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2833197Z "size": 700, 2024-08-07T17:59:44.2833594Z "digest": "sha256:679e209a81f89d0be588ce19c3f5191f73883a86e44ab7b3653a3be3f267b69e" 2024-08-07T17:59:44.2834035Z }, 2024-08-07T17:59:44.2834232Z { 2024-08-07T17:59:44.2834558Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2834957Z "size": 143, 2024-08-07T17:59:44.2835358Z "digest": "sha256:5980a36dfe02695abaecfad21a248c6c1902b07b2c9b69c61c39e342994e2f91" 2024-08-07T17:59:44.2835810Z }, 2024-08-07T17:59:44.2836002Z { 2024-08-07T17:59:44.2836326Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2836730Z "size": 135, 2024-08-07T17:59:44.2837131Z "digest": "sha256:94a4e0b3f19a399451a5f3cc7ddbde73ea16a7f180f7f047bf3ad868072c173f" 2024-08-07T17:59:44.2837683Z }, 2024-08-07T17:59:44.2837876Z { 2024-08-07T17:59:44.2838206Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2838617Z "size": 32, 2024-08-07T17:59:44.2839016Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:59:44.2849557Z }, 2024-08-07T17:59:44.2849762Z { 2024-08-07T17:59:44.2850098Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2850507Z "size": 189, 2024-08-07T17:59:44.2850898Z "digest": "sha256:2012c603f15449503b4671093a9ba6aff4fc99cf4923a92bb446fde7e52d59ee" 2024-08-07T17:59:44.2851339Z }, 2024-08-07T17:59:44.2851530Z { 2024-08-07T17:59:44.2851850Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2852249Z "size": 563, 2024-08-07T17:59:44.2852647Z "digest": "sha256:060890aa9610c5ec0050f85cafaa1f010ff178e2c8b0600aa3c43ad37ed48976" 2024-08-07T17:59:44.2853086Z }, 2024-08-07T17:59:44.2853276Z { 2024-08-07T17:59:44.2853603Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2854006Z "size": 43163116, 2024-08-07T17:59:44.2854419Z "digest": "sha256:c1a64eb8ee12a08340fb5c5a87dc012ff3074a8b683cc399feaa431de7402abd" 2024-08-07T17:59:44.2854870Z }, 2024-08-07T17:59:44.2855057Z { 2024-08-07T17:59:44.2855376Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2855774Z "size": 106, 2024-08-07T17:59:44.2856173Z "digest": "sha256:ed7686d06f1d744c9ec6dd0d75ae1581baefd7809deef8aefa11d54945c7888f" 2024-08-07T17:59:44.2856627Z }, 2024-08-07T17:59:44.2856817Z { 2024-08-07T17:59:44.2857131Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2857529Z "size": 1212, 2024-08-07T17:59:44.2857926Z "digest": "sha256:5c40be0141236773ddf2a3127f247bcc22540d4bebf4f3cc1df53f16f629ee35" 2024-08-07T17:59:44.2858365Z }, 2024-08-07T17:59:44.2858556Z { 2024-08-07T17:59:44.2858879Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2859273Z "size": 700, 2024-08-07T17:59:44.2859808Z "digest": "sha256:679e209a81f89d0be588ce19c3f5191f73883a86e44ab7b3653a3be3f267b69e" 2024-08-07T17:59:44.2860327Z }, 2024-08-07T17:59:44.2860506Z { 2024-08-07T17:59:44.2860819Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2861209Z "size": 138, 2024-08-07T17:59:44.2861591Z "digest": "sha256:95c1963010edc97c994c12e530dc7e5a5717123dfc4378fe8ecca9dbf79de394" 2024-08-07T17:59:44.2862025Z }, 2024-08-07T17:59:44.2862206Z { 2024-08-07T17:59:44.2862512Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2862900Z "size": 120, 2024-08-07T17:59:44.2863271Z "digest": "sha256:5805001913689846871dcb66b59a8d496e3c78fbf4b46c0c55cb11629af04779" 2024-08-07T17:59:44.2863693Z }, 2024-08-07T17:59:44.2863872Z { 2024-08-07T17:59:44.2864179Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2864575Z "size": 1916657670, 2024-08-07T17:59:44.2864988Z "digest": "sha256:b826637ebc384c2f2efbdc841bf6b8f0ac9b6a85060cab5d171d8ed8d49dd3de" 2024-08-07T17:59:44.2865434Z }, 2024-08-07T17:59:44.2865617Z { 2024-08-07T17:59:44.2865928Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2866315Z "size": 173, 2024-08-07T17:59:44.2866686Z "digest": "sha256:859f9c7a63754c26422062903f2a9991578a62fe2f1a81c9ad0f0e9517ab7387" 2024-08-07T17:59:44.2867107Z }, 2024-08-07T17:59:44.2867283Z { 2024-08-07T17:59:44.2867596Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2867983Z "size": 908, 2024-08-07T17:59:44.2868358Z "digest": "sha256:b89ac1530c4a96d2c4c0626a5202eb9e9a05e0d08517e1a5bf165257505309e8" 2024-08-07T17:59:44.2868789Z }, 2024-08-07T17:59:44.2868970Z { 2024-08-07T17:59:44.2869275Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2869767Z "size": 700, 2024-08-07T17:59:44.2870156Z "digest": "sha256:679e209a81f89d0be588ce19c3f5191f73883a86e44ab7b3653a3be3f267b69e" 2024-08-07T17:59:44.2870594Z }, 2024-08-07T17:59:44.2870770Z { 2024-08-07T17:59:44.2871082Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2871465Z "size": 134, 2024-08-07T17:59:44.2871854Z "digest": "sha256:4f10deed2e003fe5f78af780a6b4a71d0107d3ee59e41de3f35c031ca08e9d4d" 2024-08-07T17:59:44.2872294Z }, 2024-08-07T17:59:44.2872471Z { 2024-08-07T17:59:44.2872781Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2873165Z "size": 32, 2024-08-07T17:59:44.2873546Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:59:44.2873987Z }, 2024-08-07T17:59:44.2874166Z { 2024-08-07T17:59:44.2874472Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2874860Z "size": 156, 2024-08-07T17:59:44.2875241Z "digest": "sha256:336420751f1de11d750328660fdb6ebb9051881d009d399e893c10d61ba69b0c" 2024-08-07T17:59:44.2875658Z }, 2024-08-07T17:59:44.2875841Z { 2024-08-07T17:59:44.2876147Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2876533Z "size": 1841, 2024-08-07T17:59:44.2876912Z "digest": "sha256:f7f49611427c9bdc74d97703f780519d1d7d2b95a5377f6f625c8884cbc21d4e" 2024-08-07T17:59:44.2877335Z }, 2024-08-07T17:59:44.2877513Z { 2024-08-07T17:59:44.2877823Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2878208Z "size": 7529783, 2024-08-07T17:59:44.2878592Z "digest": "sha256:628b460c253a663b1f76b99fd2f00d63872fce39b0830a3b45bdeec4f5244660" 2024-08-07T17:59:44.2879021Z }, 2024-08-07T17:59:44.2879198Z { 2024-08-07T17:59:44.2879506Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2879894Z "size": 164, 2024-08-07T17:59:44.2880272Z "digest": "sha256:98e88ff103238559de2c0c76e43d76a01b94584edee356532b7723d1fd39dd85" 2024-08-07T17:59:44.2880697Z }, 2024-08-07T17:59:44.2880878Z { 2024-08-07T17:59:44.2881285Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2881671Z "size": 7944, 2024-08-07T17:59:44.2882054Z "digest": "sha256:6abf825f7962d4bc769dde6a63a4132694ecb9ba0f17006085d8c339aeedf887" 2024-08-07T17:59:44.2882482Z }, 2024-08-07T17:59:44.2882686Z { 2024-08-07T17:59:44.2883021Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2883406Z "size": 8063, 2024-08-07T17:59:44.2883790Z "digest": "sha256:844414c41546bd3c4dd14a45bbd58cca4a2aa0e8f37a781f8c386736ae4d4081" 2024-08-07T17:59:44.2884224Z }, 2024-08-07T17:59:44.2884399Z { 2024-08-07T17:59:44.2884707Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2885095Z "size": 300, 2024-08-07T17:59:44.2885471Z "digest": "sha256:b92a0d83e22950e600ffd0f6391f5c20b499107ba973cab4d7a54a5c65a922b1" 2024-08-07T17:59:44.2885904Z }, 2024-08-07T17:59:44.2886083Z { 2024-08-07T17:59:44.2886389Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2886781Z "size": 7629841, 2024-08-07T17:59:44.2887159Z "digest": "sha256:56e4340bc9e3886f7c099a66772a040a2d34cf0782746af58b0317af979cdfa3" 2024-08-07T17:59:44.2887578Z }, 2024-08-07T17:59:44.2887756Z { 2024-08-07T17:59:44.2888065Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2888449Z "size": 108, 2024-08-07T17:59:44.2888818Z "digest": "sha256:26f48d882588278c8763af295a6bc7147c492d82c6e4395970856a29fb8d77f0" 2024-08-07T17:59:44.2889236Z }, 2024-08-07T17:59:44.2889413Z { 2024-08-07T17:59:44.2889720Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2890104Z "size": 54145778, 2024-08-07T17:59:44.2890502Z "digest": "sha256:b6fe2821ba25ab984577df156aed9b873699ef0f46b6230d8e9a54f9ee22be1e" 2024-08-07T17:59:44.2890938Z }, 2024-08-07T17:59:44.2891230Z { 2024-08-07T17:59:44.2891549Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2891943Z "size": 473, 2024-08-07T17:59:44.2892329Z "digest": "sha256:fae8722cca7f32933f7a25f1491c31ea9a6df4fc1f9fb2360bd29c79b04f1c56" 2024-08-07T17:59:44.2892795Z }, 2024-08-07T17:59:44.2892999Z { 2024-08-07T17:59:44.2893304Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2893695Z "size": 1374858912, 2024-08-07T17:59:44.2894096Z "digest": "sha256:3c7c25c582fced622823798bd877a7fb903ebd4bfecd93c32e43dbd536bb8202" 2024-08-07T17:59:44.2894529Z }, 2024-08-07T17:59:44.2894708Z { 2024-08-07T17:59:44.2895014Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2895398Z "size": 106, 2024-08-07T17:59:44.2895782Z "digest": "sha256:75a49c2f3f0a99be9760740cce745e1ffd508a15bf5ef08077b2032b4d4d97ce" 2024-08-07T17:59:44.2896215Z }, 2024-08-07T17:59:44.2896390Z { 2024-08-07T17:59:44.2896702Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2897087Z "size": 558, 2024-08-07T17:59:44.2897472Z "digest": "sha256:b32c97699ecde27b65bfbbd8ba207755eb28584f2fc64501f4a320045ae969c8" 2024-08-07T17:59:44.2897904Z }, 2024-08-07T17:59:44.2898080Z { 2024-08-07T17:59:44.2898387Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2898773Z "size": 46248557, 2024-08-07T17:59:44.2899154Z "digest": "sha256:b926a85168171349a0ff57c87aa52b9174d3704512eb2e687184f5552883312a" 2024-08-07T17:59:44.2899568Z }, 2024-08-07T17:59:44.2899746Z { 2024-08-07T17:59:44.2900446Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2900847Z "size": 111, 2024-08-07T17:59:44.2901227Z "digest": "sha256:1c5d35b9a7607fd72af03dc281fe78215973e63c51c1b823a704727c8a0944eb" 2024-08-07T17:59:44.2901657Z }, 2024-08-07T17:59:44.2901833Z { 2024-08-07T17:59:44.2902157Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2902544Z "size": 32, 2024-08-07T17:59:44.2902928Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:59:44.2903531Z }, 2024-08-07T17:59:44.2903718Z { 2024-08-07T17:59:44.2904036Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2904433Z "size": 32, 2024-08-07T17:59:44.2904825Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:59:44.2905273Z }, 2024-08-07T17:59:44.2905463Z { 2024-08-07T17:59:44.2905779Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2906175Z "size": 32, 2024-08-07T17:59:44.2906569Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:59:44.2907010Z }, 2024-08-07T17:59:44.2907199Z { 2024-08-07T17:59:44.2907524Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:59:44.2907921Z "size": 32, 2024-08-07T17:59:44.2908315Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:59:44.2908769Z } 2024-08-07T17:59:44.2908955Z ] 2024-08-07T17:59:44.2909144Z } 2024-08-07T17:59:44.2909347Z + exit 0 2024-08-07T17:59:44.2969981Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-07T17:59:44.2970349Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-07T17:59:44.2970744Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-07T17:59:44.2979367Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:59:44.2979735Z env: 2024-08-07T17:59:44.2979964Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:59:44.2980828Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:44.2981580Z ##[endgroup] 2024-08-07T17:59:44.3012321Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:44.3062789Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-07T17:59:44.3063249Z with: 2024-08-07T17:59:44.3063930Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:44.3064776Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:59:44.3065153Z env: 2024-08-07T17:59:44.3065384Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:59:44.3065662Z ##[endgroup] 2024-08-07T17:59:44.3088468Z ##[group]Run set -x 2024-08-07T17:59:44.3088759Z set -x 2024-08-07T17:59:44.3088987Z set +e 2024-08-07T17:59:44.3089205Z  2024-08-07T17:59:44.3089430Z login() { 2024-08-07T17:59:44.3089891Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-07T17:59:44.3090372Z } 2024-08-07T17:59:44.3090590Z  2024-08-07T17:59:44.3090828Z retry () { 2024-08-07T17:59:44.3091104Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-07T17:59:44.3091415Z } 2024-08-07T17:59:44.3091621Z  2024-08-07T17:59:44.3091865Z retry login "${DOCKER_REGISTRY}" 2024-08-07T17:59:44.3092167Z  2024-08-07T17:59:44.3092374Z set -e 2024-08-07T17:59:44.3092710Z # ignore output since only exit code is used for conditional 2024-08-07T17:59:44.3093181Z # only pull docker image if it's not available locally 2024-08-07T17:59:44.3093688Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-07T17:59:44.3094158Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-07T17:59:44.3094474Z fi 2024-08-07T17:59:44.3103482Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:59:44.3103848Z env: 2024-08-07T17:59:44.3104067Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:59:44.3104783Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:44.3105794Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:59:44.3106173Z ##[endgroup] 2024-08-07T17:59:44.3134790Z + set +e 2024-08-07T17:59:44.3135266Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:59:44.3135709Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:59:44.3138414Z + aws ecr get-login-password --region us-east-1 2024-08-07T17:59:44.3140319Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:59:44.8488564Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-07T17:59:44.8489142Z Configure a credential helper to remove this warning. See 2024-08-07T17:59:44.8489689Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-07T17:59:44.8490182Z 2024-08-07T17:59:44.8490339Z Login Succeeded 2024-08-07T17:59:44.8512036Z + set -e 2024-08-07T17:59:44.8512790Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:44.8672284Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:44.8673495Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:59:45.1291139Z 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9 2024-08-07T17:59:45.1292096Z 7a2c55901189: Pulling fs layer 2024-08-07T17:59:45.1292512Z 224fe954d725: Pulling fs layer 2024-08-07T17:59:45.1292920Z 75722010b82e: Pulling fs layer 2024-08-07T17:59:45.1293343Z d527cbbb87e3: Pulling fs layer 2024-08-07T17:59:45.1294140Z b57676e46aee: Pulling fs layer 2024-08-07T17:59:45.1294619Z a8c1e85b5e14: Pulling fs layer 2024-08-07T17:59:45.1295049Z a41a8d1c11c8: Pulling fs layer 2024-08-07T17:59:45.1295456Z 0c1227890755: Pulling fs layer 2024-08-07T17:59:45.1295881Z d8d1234baab3: Pulling fs layer 2024-08-07T17:59:45.1296303Z 7ed32bc8e469: Pulling fs layer 2024-08-07T17:59:45.1296694Z ec1e7978c1fe: Pulling fs layer 2024-08-07T17:59:45.1297092Z 66b43372aa39: Pulling fs layer 2024-08-07T17:59:45.1297509Z b6662193c745: Pulling fs layer 2024-08-07T17:59:45.1298039Z b57676e46aee: Waiting 2024-08-07T17:59:45.1298421Z 5be2b638d110: Pulling fs layer 2024-08-07T17:59:45.1298844Z 71ca63790839: Pulling fs layer 2024-08-07T17:59:45.1299241Z a8c1e85b5e14: Waiting 2024-08-07T17:59:45.1299598Z a41a8d1c11c8: Waiting 2024-08-07T17:59:45.1300346Z 8a74804dc4fa: Pulling fs layer 2024-08-07T17:59:45.1300749Z 3bacb5389b74: Pulling fs layer 2024-08-07T17:59:45.1301212Z a8911a72541a: Pulling fs layer 2024-08-07T17:59:45.1301661Z 55d020986bb7: Pulling fs layer 2024-08-07T17:59:45.1302059Z 0c1227890755: Waiting 2024-08-07T17:59:45.1302440Z 679e209a81f8: Pulling fs layer 2024-08-07T17:59:45.1303413Z d4fb7093f54f: Pulling fs layer 2024-08-07T17:59:45.1303822Z 0d8ab4023e81: Pulling fs layer 2024-08-07T17:59:45.1304215Z bf191f5f5a0a: Pulling fs layer 2024-08-07T17:59:45.1304593Z ec1e7978c1fe: Waiting 2024-08-07T17:59:45.1304927Z d8d1234baab3: Waiting 2024-08-07T17:59:45.1305281Z 14653e4e245f: Pulling fs layer 2024-08-07T17:59:45.1305661Z 7ed32bc8e469: Waiting 2024-08-07T17:59:45.1306016Z 8bdbb000c39d: Pulling fs layer 2024-08-07T17:59:45.1306385Z 66b43372aa39: Waiting 2024-08-07T17:59:45.1306641Z 277383b63c07: Pulling fs layer 2024-08-07T17:59:45.1306911Z 890313244493: Pulling fs layer 2024-08-07T17:59:45.1307193Z f1e3cc0f57ee: Pulling fs layer 2024-08-07T17:59:45.1307481Z c3cbae3fe054: Pulling fs layer 2024-08-07T17:59:45.1307747Z b6662193c745: Waiting 2024-08-07T17:59:45.1308002Z ccc148c4e759: Pulling fs layer 2024-08-07T17:59:45.1308288Z 7912f8c8e80d: Pulling fs layer 2024-08-07T17:59:45.1308568Z d166ebb28213: Pulling fs layer 2024-08-07T17:59:45.1309054Z 63bf315f789a: Pulling fs layer 2024-08-07T17:59:45.1309310Z 5be2b638d110: Waiting 2024-08-07T17:59:45.1309545Z 71ca63790839: Waiting 2024-08-07T17:59:45.1309779Z 3bacb5389b74: Waiting 2024-08-07T17:59:45.1310022Z bdb818f7b2c8: Pulling fs layer 2024-08-07T17:59:45.1310298Z 89d8aea05b3a: Pulling fs layer 2024-08-07T17:59:45.1310941Z f1122e19f790: Pulling fs layer 2024-08-07T17:59:45.1311312Z 13d6ce3185e9: Pulling fs layer 2024-08-07T17:59:45.1311840Z a8911a72541a: Waiting 2024-08-07T17:59:45.1312088Z feb3f80c392d: Pulling fs layer 2024-08-07T17:59:45.1312345Z 55d020986bb7: Waiting 2024-08-07T17:59:45.1312578Z 679e209a81f8: Waiting 2024-08-07T17:59:45.1312827Z 4fe4cdcdfbd8: Pulling fs layer 2024-08-07T17:59:45.1313087Z d527cbbb87e3: Waiting 2024-08-07T17:59:45.1313335Z be10b99d8ac8: Pulling fs layer 2024-08-07T17:59:45.1313600Z 0d8ab4023e81: Waiting 2024-08-07T17:59:45.1313841Z 5980a36dfe02: Pulling fs layer 2024-08-07T17:59:45.1314122Z 94a4e0b3f19a: Pulling fs layer 2024-08-07T17:59:45.1314394Z ccc148c4e759: Waiting 2024-08-07T17:59:45.1314638Z 4f4fb700ef54: Pulling fs layer 2024-08-07T17:59:45.1314914Z 2012c603f154: Pulling fs layer 2024-08-07T17:59:45.1315172Z 89d8aea05b3a: Waiting 2024-08-07T17:59:45.1315422Z 060890aa9610: Pulling fs layer 2024-08-07T17:59:45.1315697Z c1a64eb8ee12: Pulling fs layer 2024-08-07T17:59:45.1315957Z d4fb7093f54f: Waiting 2024-08-07T17:59:45.1316195Z 14653e4e245f: Waiting 2024-08-07T17:59:45.1316430Z 8bdbb000c39d: Waiting 2024-08-07T17:59:45.1316662Z bdb818f7b2c8: Waiting 2024-08-07T17:59:45.1316911Z ed7686d06f1d: Pulling fs layer 2024-08-07T17:59:45.1317188Z 5c40be014123: Pulling fs layer 2024-08-07T17:59:45.1317561Z 890313244493: Waiting 2024-08-07T17:59:45.1317893Z 95c1963010ed: Pulling fs layer 2024-08-07T17:59:45.1318259Z 580500191368: Pulling fs layer 2024-08-07T17:59:45.1318516Z bf191f5f5a0a: Waiting 2024-08-07T17:59:45.1318763Z b826637ebc38: Pulling fs layer 2024-08-07T17:59:45.1319597Z 859f9c7a6375: Pulling fs layer 2024-08-07T17:59:45.1319882Z be10b99d8ac8: Waiting 2024-08-07T17:59:45.1320141Z b89ac1530c4a: Pulling fs layer 2024-08-07T17:59:45.1320416Z feb3f80c392d: Waiting 2024-08-07T17:59:45.1320750Z 4fe4cdcdfbd8: Waiting 2024-08-07T17:59:45.1321096Z 4f10deed2e00: Pulling fs layer 2024-08-07T17:59:45.1321525Z 336420751f1d: Pulling fs layer 2024-08-07T17:59:45.1321817Z f7f49611427c: Pulling fs layer 2024-08-07T17:59:45.1322086Z f1e3cc0f57ee: Waiting 2024-08-07T17:59:45.1322361Z 628b460c253a: Pulling fs layer 2024-08-07T17:59:45.1322630Z f1122e19f790: Waiting 2024-08-07T17:59:45.1322874Z 94a4e0b3f19a: Waiting 2024-08-07T17:59:45.1323122Z 98e88ff10323: Pulling fs layer 2024-08-07T17:59:45.1323395Z b89ac1530c4a: Waiting 2024-08-07T17:59:45.1323648Z 6abf825f7962: Pulling fs layer 2024-08-07T17:59:45.1323917Z c3cbae3fe054: Waiting 2024-08-07T17:59:45.1324168Z 844414c41546: Pulling fs layer 2024-08-07T17:59:45.1324438Z b92a0d83e229: Pulling fs layer 2024-08-07T17:59:45.1324718Z 56e4340bc9e3: Pulling fs layer 2024-08-07T17:59:45.1325100Z 26f48d882588: Pulling fs layer 2024-08-07T17:59:45.1325361Z 7912f8c8e80d: Waiting 2024-08-07T17:59:45.1325696Z b6fe2821ba25: Pulling fs layer 2024-08-07T17:59:45.1325980Z fae8722cca7f: Pulling fs layer 2024-08-07T17:59:45.1326255Z 3c7c25c582fc: Pulling fs layer 2024-08-07T17:59:45.1326533Z 75a49c2f3f0a: Pulling fs layer 2024-08-07T17:59:45.1326895Z b32c97699ecd: Pulling fs layer 2024-08-07T17:59:45.1327163Z b926a8516817: Pulling fs layer 2024-08-07T17:59:45.1327437Z 1c5d35b9a760: Pulling fs layer 2024-08-07T17:59:45.1327701Z 277383b63c07: Waiting 2024-08-07T17:59:45.1328078Z 2012c603f154: Waiting 2024-08-07T17:59:45.1328309Z 060890aa9610: Waiting 2024-08-07T17:59:45.1328615Z 3c7c25c582fc: Waiting 2024-08-07T17:59:45.1328844Z 628b460c253a: Waiting 2024-08-07T17:59:45.1329176Z 75a49c2f3f0a: Waiting 2024-08-07T17:59:45.1329406Z b826637ebc38: Waiting 2024-08-07T17:59:45.1329641Z 5980a36dfe02: Waiting 2024-08-07T17:59:45.1329874Z b926a8516817: Waiting 2024-08-07T17:59:45.1330106Z 580500191368: Waiting 2024-08-07T17:59:45.1330449Z c1a64eb8ee12: Waiting 2024-08-07T17:59:45.1330747Z 859f9c7a6375: Waiting 2024-08-07T17:59:45.1331066Z 6abf825f7962: Waiting 2024-08-07T17:59:45.1331379Z 56e4340bc9e3: Waiting 2024-08-07T17:59:45.1331622Z 844414c41546: Waiting 2024-08-07T17:59:45.1331855Z b92a0d83e229: Waiting 2024-08-07T17:59:45.1332115Z d166ebb28213: Waiting 2024-08-07T17:59:45.1332426Z b6fe2821ba25: Waiting 2024-08-07T17:59:45.1332687Z f7f49611427c: Waiting 2024-08-07T17:59:45.1332962Z fae8722cca7f: Waiting 2024-08-07T17:59:45.1333303Z 5c40be014123: Waiting 2024-08-07T17:59:45.1333594Z 95c1963010ed: Waiting 2024-08-07T17:59:45.1333928Z 98e88ff10323: Waiting 2024-08-07T17:59:45.1334263Z b32c97699ecd: Waiting 2024-08-07T17:59:45.1334586Z 4f10deed2e00: Waiting 2024-08-07T17:59:45.1334915Z 1c5d35b9a760: Waiting 2024-08-07T17:59:45.1335244Z 4f4fb700ef54: Waiting 2024-08-07T17:59:45.2686254Z 224fe954d725: Verifying Checksum 2024-08-07T17:59:45.2686600Z 224fe954d725: Download complete 2024-08-07T17:59:45.3438597Z d527cbbb87e3: Verifying Checksum 2024-08-07T17:59:45.3439142Z d527cbbb87e3: Download complete 2024-08-07T17:59:45.4214648Z b57676e46aee: Download complete 2024-08-07T17:59:45.4654247Z 7a2c55901189: Verifying Checksum 2024-08-07T17:59:45.4654575Z 7a2c55901189: Download complete 2024-08-07T17:59:45.5441293Z a41a8d1c11c8: Verifying Checksum 2024-08-07T17:59:45.5441637Z a41a8d1c11c8: Download complete 2024-08-07T17:59:45.6117607Z 0c1227890755: Verifying Checksum 2024-08-07T17:59:45.6118116Z 0c1227890755: Download complete 2024-08-07T17:59:45.6860179Z d8d1234baab3: Verifying Checksum 2024-08-07T17:59:45.6860750Z d8d1234baab3: Download complete 2024-08-07T17:59:45.7512822Z 75722010b82e: Verifying Checksum 2024-08-07T17:59:45.7513240Z 75722010b82e: Download complete 2024-08-07T17:59:45.8290187Z ec1e7978c1fe: Verifying Checksum 2024-08-07T17:59:45.8290531Z ec1e7978c1fe: Download complete 2024-08-07T17:59:45.9344152Z 66b43372aa39: Verifying Checksum 2024-08-07T17:59:45.9347286Z 66b43372aa39: Download complete 2024-08-07T17:59:46.6376207Z 7a2c55901189: Pull complete 2024-08-07T17:59:46.9540503Z 224fe954d725: Pull complete 2024-08-07T17:59:47.8874111Z 75722010b82e: Pull complete 2024-08-07T17:59:47.9038032Z d527cbbb87e3: Pull complete 2024-08-07T17:59:47.9219513Z b57676e46aee: Pull complete 2024-08-07T17:59:48.4506522Z b6662193c745: Verifying Checksum 2024-08-07T17:59:48.4506868Z b6662193c745: Download complete 2024-08-07T17:59:48.5369513Z 5be2b638d110: Verifying Checksum 2024-08-07T17:59:48.5369832Z 5be2b638d110: Download complete 2024-08-07T17:59:48.6146571Z 71ca63790839: Verifying Checksum 2024-08-07T17:59:48.6146886Z 71ca63790839: Download complete 2024-08-07T17:59:48.6857818Z 8a74804dc4fa: Verifying Checksum 2024-08-07T17:59:48.6858194Z 8a74804dc4fa: Download complete 2024-08-07T17:59:49.6505694Z 3bacb5389b74: Verifying Checksum 2024-08-07T17:59:49.6506040Z 3bacb5389b74: Download complete 2024-08-07T17:59:49.7224045Z a8911a72541a: Verifying Checksum 2024-08-07T17:59:49.7224495Z a8911a72541a: Download complete 2024-08-07T17:59:49.7846901Z 55d020986bb7: Verifying Checksum 2024-08-07T17:59:49.7847363Z 55d020986bb7: Download complete 2024-08-07T17:59:49.8509792Z 679e209a81f8: Verifying Checksum 2024-08-07T17:59:49.8510246Z 679e209a81f8: Download complete 2024-08-07T17:59:58.7582739Z a8c1e85b5e14: Verifying Checksum 2024-08-07T17:59:58.7583233Z a8c1e85b5e14: Download complete 2024-08-07T17:59:58.8319179Z 0d8ab4023e81: Verifying Checksum 2024-08-07T17:59:58.8319637Z 0d8ab4023e81: Download complete 2024-08-07T17:59:58.9075149Z bf191f5f5a0a: Download complete 2024-08-07T17:59:59.0288249Z 14653e4e245f: Verifying Checksum 2024-08-07T17:59:59.0288705Z 14653e4e245f: Download complete 2024-08-07T17:59:59.1383427Z 8bdbb000c39d: Download complete 2024-08-07T17:59:59.2125262Z 277383b63c07: Verifying Checksum 2024-08-07T17:59:59.2125692Z 277383b63c07: Download complete 2024-08-07T18:00:00.5051594Z 890313244493: Verifying Checksum 2024-08-07T18:00:00.5052046Z 890313244493: Download complete 2024-08-07T18:00:00.5678291Z f1e3cc0f57ee: Verifying Checksum 2024-08-07T18:00:00.5679646Z f1e3cc0f57ee: Download complete 2024-08-07T18:00:00.6487785Z c3cbae3fe054: Verifying Checksum 2024-08-07T18:00:00.6488254Z c3cbae3fe054: Download complete 2024-08-07T18:00:00.7204516Z ccc148c4e759: Verifying Checksum 2024-08-07T18:00:00.7205207Z ccc148c4e759: Download complete 2024-08-07T18:00:00.8121106Z 7912f8c8e80d: Download complete 2024-08-07T18:00:00.8779013Z d166ebb28213: Download complete 2024-08-07T18:00:05.1843186Z 63bf315f789a: Verifying Checksum 2024-08-07T18:00:05.1843542Z 63bf315f789a: Download complete 2024-08-07T18:00:05.2767833Z bdb818f7b2c8: Verifying Checksum 2024-08-07T18:00:05.2768214Z bdb818f7b2c8: Download complete 2024-08-07T18:00:05.3682274Z 89d8aea05b3a: Download complete 2024-08-07T18:00:05.7817827Z f1122e19f790: Verifying Checksum 2024-08-07T18:00:05.7818203Z f1122e19f790: Download complete 2024-08-07T18:00:05.8748501Z 13d6ce3185e9: Verifying Checksum 2024-08-07T18:00:05.8749076Z 13d6ce3185e9: Download complete 2024-08-07T18:00:06.0047016Z feb3f80c392d: Download complete 2024-08-07T18:00:06.2717506Z 4fe4cdcdfbd8: Verifying Checksum 2024-08-07T18:00:06.2717989Z 4fe4cdcdfbd8: Download complete 2024-08-07T18:00:06.3376543Z be10b99d8ac8: Verifying Checksum 2024-08-07T18:00:06.3376868Z be10b99d8ac8: Download complete 2024-08-07T18:00:06.4260183Z 5980a36dfe02: Download complete 2024-08-07T18:00:06.5243149Z 94a4e0b3f19a: Verifying Checksum 2024-08-07T18:00:06.5326968Z 4f4fb700ef54: Verifying Checksum 2024-08-07T18:00:06.5329765Z 4f4fb700ef54: Download complete 2024-08-07T18:00:06.6205116Z 2012c603f154: Verifying Checksum 2024-08-07T18:00:06.6205615Z 2012c603f154: Download complete 2024-08-07T18:00:06.6870943Z 060890aa9610: Verifying Checksum 2024-08-07T18:00:07.2036093Z c1a64eb8ee12: Verifying Checksum 2024-08-07T18:00:07.2036427Z c1a64eb8ee12: Download complete 2024-08-07T18:00:07.2835469Z ed7686d06f1d: Verifying Checksum 2024-08-07T18:00:07.2835910Z ed7686d06f1d: Download complete 2024-08-07T18:00:07.3669347Z 5c40be014123: Verifying Checksum 2024-08-07T18:00:07.3669701Z 5c40be014123: Download complete 2024-08-07T18:00:07.4454650Z 95c1963010ed: Verifying Checksum 2024-08-07T18:00:07.5097490Z 580500191368: Verifying Checksum 2024-08-07T18:00:07.5097926Z 580500191368: Download complete 2024-08-07T18:00:11.0095929Z 7ed32bc8e469: Verifying Checksum 2024-08-07T18:00:11.0096303Z 7ed32bc8e469: Download complete 2024-08-07T18:00:11.0873315Z 859f9c7a6375: Verifying Checksum 2024-08-07T18:00:11.0873649Z 859f9c7a6375: Download complete 2024-08-07T18:00:11.1778090Z b89ac1530c4a: Verifying Checksum 2024-08-07T18:00:11.1778541Z b89ac1530c4a: Download complete 2024-08-07T18:00:11.2530135Z 4f10deed2e00: Verifying Checksum 2024-08-07T18:00:11.2530472Z 4f10deed2e00: Download complete 2024-08-07T18:00:11.3563820Z 336420751f1d: Verifying Checksum 2024-08-07T18:00:11.3564135Z 336420751f1d: Download complete 2024-08-07T18:00:11.4363525Z f7f49611427c: Verifying Checksum 2024-08-07T18:00:11.4363966Z f7f49611427c: Download complete 2024-08-07T18:00:11.5674158Z 628b460c253a: Verifying Checksum 2024-08-07T18:00:11.5674529Z 628b460c253a: Download complete 2024-08-07T18:00:11.6530689Z 98e88ff10323: Verifying Checksum 2024-08-07T18:00:11.6531392Z 98e88ff10323: Download complete 2024-08-07T18:00:11.7221084Z 6abf825f7962: Download complete 2024-08-07T18:00:11.8161893Z 844414c41546: Verifying Checksum 2024-08-07T18:00:11.8162236Z 844414c41546: Download complete 2024-08-07T18:00:11.8898480Z b92a0d83e229: Verifying Checksum 2024-08-07T18:00:11.8898836Z b92a0d83e229: Download complete 2024-08-07T18:00:12.0001506Z a8c1e85b5e14: Pull complete 2024-08-07T18:00:12.0264014Z 56e4340bc9e3: Verifying Checksum 2024-08-07T18:00:12.0264544Z 56e4340bc9e3: Download complete 2024-08-07T18:00:12.1057322Z 26f48d882588: Verifying Checksum 2024-08-07T18:00:12.1057780Z 26f48d882588: Download complete 2024-08-07T18:00:12.1318932Z a41a8d1c11c8: Pull complete 2024-08-07T18:00:12.3117193Z 0c1227890755: Pull complete 2024-08-07T18:00:12.4623385Z d8d1234baab3: Pull complete 2024-08-07T18:00:12.7067613Z b6fe2821ba25: Verifying Checksum 2024-08-07T18:00:12.7068237Z b6fe2821ba25: Download complete 2024-08-07T18:00:12.8063695Z fae8722cca7f: Download complete 2024-08-07T18:00:17.7574206Z d4fb7093f54f: Verifying Checksum 2024-08-07T18:00:17.7574730Z d4fb7093f54f: Download complete 2024-08-07T18:00:17.8344215Z 75a49c2f3f0a: Download complete 2024-08-07T18:00:17.9049945Z b32c97699ecd: Download complete 2024-08-07T18:00:18.4287871Z b926a8516817: Verifying Checksum 2024-08-07T18:00:18.4288372Z b926a8516817: Download complete 2024-08-07T18:00:18.5204649Z 1c5d35b9a760: Verifying Checksum 2024-08-07T18:00:18.5205205Z 1c5d35b9a760: Download complete 2024-08-07T18:00:26.6179536Z 3c7c25c582fc: Verifying Checksum 2024-08-07T18:00:26.6180025Z 3c7c25c582fc: Download complete 2024-08-07T18:00:26.7304233Z b826637ebc38: Verifying Checksum 2024-08-07T18:00:26.7304632Z b826637ebc38: Download complete 2024-08-07T18:00:42.9142133Z 7ed32bc8e469: Pull complete 2024-08-07T18:00:43.1611563Z ec1e7978c1fe: Pull complete 2024-08-07T18:00:43.3772326Z 66b43372aa39: Pull complete 2024-08-07T18:00:52.0358133Z b6662193c745: Pull complete 2024-08-07T18:00:52.2536558Z 5be2b638d110: Pull complete 2024-08-07T18:00:52.4419415Z 71ca63790839: Pull complete 2024-08-07T18:00:52.5480157Z 8a74804dc4fa: Pull complete 2024-08-07T18:00:55.1923877Z 3bacb5389b74: Pull complete 2024-08-07T18:00:55.4177847Z a8911a72541a: Pull complete 2024-08-07T18:00:55.6389170Z 55d020986bb7: Pull complete 2024-08-07T18:00:55.8622192Z 679e209a81f8: Pull complete 2024-08-07T18:01:50.5600737Z d4fb7093f54f: Pull complete 2024-08-07T18:01:50.6796957Z 0d8ab4023e81: Pull complete 2024-08-07T18:01:50.7605993Z bf191f5f5a0a: Pull complete 2024-08-07T18:01:50.9408899Z 14653e4e245f: Pull complete 2024-08-07T18:01:51.1658852Z 8bdbb000c39d: Pull complete 2024-08-07T18:01:51.3908946Z 277383b63c07: Pull complete 2024-08-07T18:01:54.8342773Z 890313244493: Pull complete 2024-08-07T18:01:55.0644471Z f1e3cc0f57ee: Pull complete 2024-08-07T18:01:55.3112654Z c3cbae3fe054: Pull complete 2024-08-07T18:01:55.5306767Z ccc148c4e759: Pull complete 2024-08-07T18:01:55.7660046Z 7912f8c8e80d: Pull complete 2024-08-07T18:01:55.9974229Z d166ebb28213: Pull complete 2024-08-07T18:02:05.5366802Z 63bf315f789a: Pull complete 2024-08-07T18:02:05.7736261Z bdb818f7b2c8: Pull complete 2024-08-07T18:02:05.9973557Z 89d8aea05b3a: Pull complete 2024-08-07T18:02:06.9910539Z f1122e19f790: Pull complete 2024-08-07T18:02:07.1761525Z 13d6ce3185e9: Pull complete 2024-08-07T18:02:07.3351151Z feb3f80c392d: Pull complete 2024-08-07T18:02:07.7177019Z 4fe4cdcdfbd8: Pull complete 2024-08-07T18:02:07.8388715Z be10b99d8ac8: Pull complete 2024-08-07T18:02:08.2091787Z 5980a36dfe02: Pull complete 2024-08-07T18:02:08.4367412Z 94a4e0b3f19a: Pull complete 2024-08-07T18:02:08.6611224Z 4f4fb700ef54: Pull complete 2024-08-07T18:02:08.8880748Z 2012c603f154: Pull complete 2024-08-07T18:02:09.1316729Z 060890aa9610: Pull complete 2024-08-07T18:02:11.8469206Z c1a64eb8ee12: Pull complete 2024-08-07T18:02:12.0606422Z ed7686d06f1d: Pull complete 2024-08-07T18:02:12.2206113Z 5c40be014123: Pull complete 2024-08-07T18:02:12.5706595Z 95c1963010ed: Pull complete 2024-08-07T18:02:12.7824278Z 580500191368: Pull complete 2024-08-07T18:02:48.0999506Z b826637ebc38: Pull complete 2024-08-07T18:02:48.1153751Z 859f9c7a6375: Pull complete 2024-08-07T18:02:48.1334383Z b89ac1530c4a: Pull complete 2024-08-07T18:02:48.1675597Z 4f10deed2e00: Pull complete 2024-08-07T18:02:48.2036173Z 336420751f1d: Pull complete 2024-08-07T18:02:48.2202557Z f7f49611427c: Pull complete 2024-08-07T18:02:48.3886529Z 628b460c253a: Pull complete 2024-08-07T18:02:48.4309209Z 98e88ff10323: Pull complete 2024-08-07T18:02:48.4739195Z 6abf825f7962: Pull complete 2024-08-07T18:02:48.5039457Z 844414c41546: Pull complete 2024-08-07T18:02:48.5211667Z b92a0d83e229: Pull complete 2024-08-07T18:02:49.6275218Z 56e4340bc9e3: Pull complete 2024-08-07T18:02:49.6431041Z 26f48d882588: Pull complete 2024-08-07T18:02:51.7283745Z b6fe2821ba25: Pull complete 2024-08-07T18:02:51.9474324Z fae8722cca7f: Pull complete 2024-08-07T18:03:05.3686536Z 3c7c25c582fc: Pull complete 2024-08-07T18:03:05.5998152Z 75a49c2f3f0a: Pull complete 2024-08-07T18:03:05.8310730Z b32c97699ecd: Pull complete 2024-08-07T18:03:06.7238911Z b926a8516817: Pull complete 2024-08-07T18:03:06.9589406Z 1c5d35b9a760: Pull complete 2024-08-07T18:03:07.9539345Z Digest: sha256:00f47b036f588ca5ef8866f8635fabba5a95cdf9ff1adae7d2a674ef1d4076e9 2024-08-07T18:03:07.9985762Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T18:03:08.0268532Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T18:03:08.0330829Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-07T18:03:08.0331484Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-07T18:03:08.0342125Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:03:08.0342509Z env: 2024-08-07T18:03:08.0342730Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:03:08.0343001Z ##[endgroup] 2024-08-07T18:03:08.0673694Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-08-07T18:03:08.0674097Z with: 2024-08-07T18:03:08.0674330Z driver-version: 550.54.15 2024-08-07T18:03:08.0674598Z env: 2024-08-07T18:03:08.0674813Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:03:08.0675086Z ##[endgroup] 2024-08-07T18:03:08.0798740Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-07T18:03:08.0799152Z with: 2024-08-07T18:03:08.0799376Z timeout_minutes: 10 2024-08-07T18:03:08.0799627Z max_attempts: 3 2024-08-07T18:03:08.0822942Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2024-08-07T18:03:08.0846006Z retry_wait_seconds: 10 2024-08-07T18:03:08.0846289Z polling_interval_seconds: 1 2024-08-07T18:03:08.0846574Z warning_on_retry: true 2024-08-07T18:03:08.0846844Z continue_on_error: false 2024-08-07T18:03:08.0847112Z env: 2024-08-07T18:03:08.0847329Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:03:08.0847602Z DRIVER_VERSION: 550.54.15 2024-08-07T18:03:08.0847860Z ##[endgroup] 2024-08-07T18:03:08.1659099Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-08-07T18:03:08.1659754Z + pre_install_nvidia_driver_amzn2 2024-08-07T18:03:08.1662761Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-08-07T18:03:08.5343552Z No match for argument: nvidia-driver-latest-dkms 2024-08-07T18:03:08.5344112Z No packages marked for removal. 2024-08-07T18:03:08.5407071Z Dependencies resolved. 2024-08-07T18:03:08.5417216Z Nothing to do. 2024-08-07T18:03:08.5417840Z Complete! 2024-08-07T18:03:08.6322966Z + install_nvidia_driver_common 2024-08-07T18:03:08.6325515Z + echo 'Before installing NVIDIA driver' 2024-08-07T18:03:08.6325838Z + lspci 2024-08-07T18:03:08.6326069Z Before installing NVIDIA driver 2024-08-07T18:03:08.6439447Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-07T18:03:08.6439959Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-07T18:03:08.6440516Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-07T18:03:08.6441046Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-07T18:03:08.6441542Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-07T18:03:08.6442078Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-07T18:03:08.6442574Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-07T18:03:08.6443068Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-07T18:03:08.6443494Z + lsmod 2024-08-07T18:03:08.6490403Z Module Size Used by 2024-08-07T18:03:08.6491056Z veth 36864 0 2024-08-07T18:03:08.6491601Z nvidia_modeset 1351680 0 2024-08-07T18:03:08.6492186Z video 65536 1 nvidia_modeset 2024-08-07T18:03:08.6492811Z wmi 36864 1 video 2024-08-07T18:03:08.6493389Z nvidia_uvm 4706304 0 2024-08-07T18:03:08.6494006Z nvidia 54071296 7 nvidia_uvm,nvidia_modeset 2024-08-07T18:03:08.6494681Z drm 602112 1 nvidia 2024-08-07T18:03:08.6495316Z drm_panel_orientation_quirks 28672 1 drm 2024-08-07T18:03:08.6495713Z backlight 24576 3 video,drm,nvidia_modeset 2024-08-07T18:03:08.6496081Z i2c_core 106496 2 nvidia,drm 2024-08-07T18:03:08.6496393Z xt_conntrack 16384 1 2024-08-07T18:03:08.6496662Z nft_chain_nat 16384 3 2024-08-07T18:03:08.6496948Z xt_MASQUERADE 20480 1 2024-08-07T18:03:08.6497273Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-07T18:03:08.6497624Z nf_conntrack_netlink 57344 0 2024-08-07T18:03:08.6498046Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-07T18:03:08.6498501Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-07T18:03:08.6498827Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-07T18:03:08.6499146Z xfrm_user 57344 1 2024-08-07T18:03:08.6499430Z xfrm_algo 16384 1 xfrm_user 2024-08-07T18:03:08.6500202Z xt_addrtype 16384 2 2024-08-07T18:03:08.6500493Z nft_compat 20480 4 2024-08-07T18:03:08.6500924Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-07T18:03:08.6501361Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-07T18:03:08.6501749Z br_netfilter 36864 0 2024-08-07T18:03:08.6502045Z bridge 307200 1 br_netfilter 2024-08-07T18:03:08.6502356Z stp 16384 1 bridge 2024-08-07T18:03:08.6502660Z llc 16384 2 bridge,stp 2024-08-07T18:03:08.6502961Z overlay 167936 0 2024-08-07T18:03:08.6503228Z tls 114688 0 2024-08-07T18:03:08.6503499Z nls_ascii 16384 1 2024-08-07T18:03:08.6503766Z nls_cp437 20480 1 2024-08-07T18:03:08.6504032Z vfat 24576 1 2024-08-07T18:03:08.6504297Z fat 86016 1 vfat 2024-08-07T18:03:08.6504574Z sunrpc 692224 1 2024-08-07T18:03:08.6504936Z ena 167936 0 2024-08-07T18:03:08.6505209Z ghash_clmulni_intel 16384 0 2024-08-07T18:03:08.6505499Z ptp 36864 1 ena 2024-08-07T18:03:08.6505794Z pps_core 24576 1 ptp 2024-08-07T18:03:08.6506081Z aesni_intel 393216 0 2024-08-07T18:03:08.6506342Z i8042 45056 0 2024-08-07T18:03:08.6506610Z serio 28672 3 i8042 2024-08-07T18:03:08.6507107Z crypto_simd 16384 1 aesni_intel 2024-08-07T18:03:08.6507475Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-07T18:03:08.6507832Z button 24576 0 2024-08-07T18:03:08.6508105Z sch_fq_codel 20480 17 2024-08-07T18:03:08.6508379Z dm_mod 188416 0 2024-08-07T18:03:08.6508649Z fuse 163840 1 2024-08-07T18:03:08.6508919Z configfs 57344 1 2024-08-07T18:03:08.6509189Z dax 45056 1 dm_mod 2024-08-07T18:03:08.6509479Z loop 36864 0 2024-08-07T18:03:08.6509758Z dmi_sysfs 20480 0 2024-08-07T18:03:08.6510023Z crc32_pclmul 16384 0 2024-08-07T18:03:08.6510295Z crc32c_intel 24576 0 2024-08-07T18:03:08.6510569Z efivarfs 24576 1 2024-08-07T18:03:08.6510829Z + modinfo nvidia 2024-08-07T18:03:08.6511242Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-07T18:03:08.6511705Z alias: char-major-195-* 2024-08-07T18:03:08.6511985Z version: 550.54.15 2024-08-07T18:03:08.6512379Z supported: external 2024-08-07T18:03:08.6512644Z license: NVIDIA 2024-08-07T18:03:08.6512924Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-07T18:03:08.6513283Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-07T18:03:08.6513618Z srcversion: 833721318DA517F0C2FEC97 2024-08-07T18:03:08.6513947Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-07T18:03:08.6514303Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-07T18:03:08.6514657Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-07T18:03:08.6514982Z depends: i2c-core,drm 2024-08-07T18:03:08.6515263Z retpoline: Y 2024-08-07T18:03:08.6515573Z name: nvidia 2024-08-07T18:03:08.6515960Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-07T18:03:08.6516448Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-07T18:03:08.6516921Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-07T18:03:08.6517356Z parm: NVreg_ResmanDebugLevel:int 2024-08-07T18:03:08.6517688Z parm: NVreg_RmLogonRC:int 2024-08-07T18:03:08.6518010Z parm: NVreg_ModifyDeviceFiles:int 2024-08-07T18:03:08.6518339Z parm: NVreg_DeviceFileUID:int 2024-08-07T18:03:08.6518663Z parm: NVreg_DeviceFileGID:int 2024-08-07T18:03:08.6518989Z parm: NVreg_DeviceFileMode:int 2024-08-07T18:03:08.6519366Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-07T18:03:08.6519776Z parm: NVreg_UsePageAttributeTable:int 2024-08-07T18:03:08.6520125Z parm: NVreg_EnablePCIeGen3:int 2024-08-07T18:03:08.6520438Z parm: NVreg_EnableMSI:int 2024-08-07T18:03:08.6520746Z parm: NVreg_TCEBypassMode:int 2024-08-07T18:03:08.6521084Z parm: NVreg_EnableStreamMemOPs:int 2024-08-07T18:03:08.6521460Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-07T18:03:08.6521881Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-07T18:03:08.6522284Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-07T18:03:08.6522709Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-07T18:03:08.6523133Z parm: NVreg_DynamicPowerManagement:int 2024-08-07T18:03:08.6523570Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-07T18:03:08.6523991Z parm: NVreg_EnableGpuFirmware:int 2024-08-07T18:03:08.6524348Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-07T18:03:08.6524735Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-07T18:03:08.6525210Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-07T18:03:08.6525572Z parm: NVreg_MemoryPoolSize:int 2024-08-07T18:03:08.6525915Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-07T18:03:08.6526264Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-07T18:03:08.6526737Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-07T18:03:08.6527095Z parm: NVreg_NvLinkDisable:int 2024-08-07T18:03:08.6527495Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-07T18:03:08.6527922Z parm: NVreg_RegisterPCIDriver:int 2024-08-07T18:03:08.6528299Z parm: NVreg_EnableResizableBar:int 2024-08-07T18:03:08.6528679Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-07T18:03:08.6529076Z parm: NVreg_EnableNonblockingOpen:int 2024-08-07T18:03:08.6529465Z parm: NVreg_RegistryDwords:charp 2024-08-07T18:03:08.6529850Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-07T18:03:08.6530232Z parm: NVreg_RmMsg:charp 2024-08-07T18:03:08.6530562Z parm: NVreg_GpuBlacklist:charp 2024-08-07T18:03:08.6530929Z parm: NVreg_TemporaryFilePath:charp 2024-08-07T18:03:08.6531304Z parm: NVreg_ExcludedGpus:charp 2024-08-07T18:03:08.6531669Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-07T18:03:08.6532046Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-07T18:03:08.6532426Z parm: NVreg_ImexChannelCount:int 2024-08-07T18:03:08.6532881Z parm: rm_firmware_active:charp 2024-08-07T18:03:08.6533221Z + HAS_NVIDIA_DRIVER=0 2024-08-07T18:03:08.6533498Z ++ command -v nvidia-smi 2024-08-07T18:03:08.6533776Z + '[' -x /usr/bin/nvidia-smi ']' 2024-08-07T18:03:08.6534044Z + set +e 2024-08-07T18:03:08.6534374Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-08-07T18:03:08.6828098Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-08-07T18:03:08.6828409Z + NVIDIA_SMI_STATUS=0 2024-08-07T18:03:08.6828658Z + '[' 0 -ne 0 ']' 2024-08-07T18:03:08.6828895Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-08-07T18:03:08.6829162Z + HAS_NVIDIA_DRIVER=1 2024-08-07T18:03:08.6829597Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-08-07T18:03:08.6830193Z + set -e 2024-08-07T18:03:08.6830412Z + '[' 1 -eq 0 ']' 2024-08-07T18:03:08.6830821Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-08-07T18:03:08.6831520Z + post_install_nvidia_driver_common 2024-08-07T18:03:08.6835687Z + sudo modprobe nvidia 2024-08-07T18:03:08.8001895Z + echo 'After installing NVIDIA driver' 2024-08-07T18:03:08.8002212Z + lspci 2024-08-07T18:03:08.8002451Z After installing NVIDIA driver 2024-08-07T18:03:08.8106735Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] 2024-08-07T18:03:08.8107256Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-07T18:03:08.8107809Z 00:01.3 Non-VGA unclassified device: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08) 2024-08-07T18:03:08.8108344Z 00:03.0 VGA compatible controller: Amazon.com, Inc. Device 1111 2024-08-07T18:03:08.8108840Z 00:04.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe EBS Controller 2024-08-07T18:03:08.8109373Z 00:05.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-07T18:03:08.8109866Z 00:1e.0 3D controller: NVIDIA Corporation GA102GL [A10G] (rev a1) 2024-08-07T18:03:08.8110387Z 00:1f.0 Non-Volatile memory controller: Amazon.com, Inc. NVMe SSD Controller 2024-08-07T18:03:08.8110790Z + lsmod 2024-08-07T18:03:08.8141395Z Module Size Used by 2024-08-07T18:03:08.8141728Z veth 36864 0 2024-08-07T18:03:08.8142002Z nvidia_modeset 1351680 0 2024-08-07T18:03:08.8142290Z video 65536 1 nvidia_modeset 2024-08-07T18:03:08.8142602Z wmi 36864 1 video 2024-08-07T18:03:08.8142886Z nvidia_uvm 4706304 0 2024-08-07T18:03:08.8143192Z nvidia 54071296 7 nvidia_uvm,nvidia_modeset 2024-08-07T18:03:08.8143529Z drm 602112 1 nvidia 2024-08-07T18:03:08.8143844Z drm_panel_orientation_quirks 28672 1 drm 2024-08-07T18:03:08.8144210Z backlight 24576 3 video,drm,nvidia_modeset 2024-08-07T18:03:08.8156211Z i2c_core 106496 2 nvidia,drm 2024-08-07T18:03:08.8156690Z xt_conntrack 16384 1 2024-08-07T18:03:08.8157269Z nft_chain_nat 16384 3 2024-08-07T18:03:08.8157555Z xt_MASQUERADE 20480 1 2024-08-07T18:03:08.8157873Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-07T18:03:08.8158228Z nf_conntrack_netlink 57344 0 2024-08-07T18:03:08.8158648Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-07T18:03:08.8159094Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-07T18:03:08.8159427Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-07T18:03:08.8159739Z xfrm_user 57344 1 2024-08-07T18:03:08.8160020Z xfrm_algo 16384 1 xfrm_user 2024-08-07T18:03:08.8160331Z xt_addrtype 16384 2 2024-08-07T18:03:08.8160614Z nft_compat 20480 4 2024-08-07T18:03:08.8160937Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-07T18:03:08.8161384Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-07T18:03:08.8161788Z br_netfilter 36864 0 2024-08-07T18:03:08.8162076Z bridge 307200 1 br_netfilter 2024-08-07T18:03:08.8162598Z stp 16384 1 bridge 2024-08-07T18:03:08.8162910Z llc 16384 2 bridge,stp 2024-08-07T18:03:08.8163206Z overlay 167936 0 2024-08-07T18:03:08.8163477Z tls 114688 0 2024-08-07T18:03:08.8163750Z nls_ascii 16384 1 2024-08-07T18:03:08.8164016Z nls_cp437 20480 1 2024-08-07T18:03:08.8164285Z vfat 24576 1 2024-08-07T18:03:08.8164547Z fat 86016 1 vfat 2024-08-07T18:03:08.8164834Z sunrpc 692224 1 2024-08-07T18:03:08.8165107Z ena 167936 0 2024-08-07T18:03:08.8165378Z ghash_clmulni_intel 16384 0 2024-08-07T18:03:08.8165661Z ptp 36864 1 ena 2024-08-07T18:03:08.8166016Z pps_core 24576 1 ptp 2024-08-07T18:03:08.8166355Z aesni_intel 393216 0 2024-08-07T18:03:08.8166635Z i8042 45056 0 2024-08-07T18:03:08.8166946Z serio 28672 3 i8042 2024-08-07T18:03:08.8167265Z crypto_simd 16384 1 aesni_intel 2024-08-07T18:03:08.8167626Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-07T18:03:08.8167975Z button 24576 0 2024-08-07T18:03:08.8168241Z sch_fq_codel 20480 17 2024-08-07T18:03:08.8168505Z dm_mod 188416 0 2024-08-07T18:03:08.8168762Z fuse 163840 1 2024-08-07T18:03:08.8169022Z configfs 57344 1 2024-08-07T18:03:08.8169282Z dax 45056 1 dm_mod 2024-08-07T18:03:08.8169563Z loop 36864 0 2024-08-07T18:03:08.8169826Z dmi_sysfs 20480 0 2024-08-07T18:03:08.8170088Z crc32_pclmul 16384 0 2024-08-07T18:03:08.8170354Z crc32c_intel 24576 0 2024-08-07T18:03:08.8170618Z efivarfs 24576 1 2024-08-07T18:03:08.8170873Z + modinfo nvidia 2024-08-07T18:03:08.8171261Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-07T18:03:08.8171717Z alias: char-major-195-* 2024-08-07T18:03:08.8171998Z version: 550.54.15 2024-08-07T18:03:08.8172257Z supported: external 2024-08-07T18:03:08.8172516Z license: NVIDIA 2024-08-07T18:03:08.8172788Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-07T18:03:08.8173134Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-07T18:03:08.8173455Z srcversion: 833721318DA517F0C2FEC97 2024-08-07T18:03:08.8173772Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-07T18:03:08.8174108Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-07T18:03:08.8174453Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-07T18:03:08.8174770Z depends: i2c-core,drm 2024-08-07T18:03:08.8175041Z retpoline: Y 2024-08-07T18:03:08.8175266Z name: nvidia 2024-08-07T18:03:08.8175626Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-07T18:03:08.8176223Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-07T18:03:08.8176679Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-07T18:03:08.8177111Z parm: NVreg_ResmanDebugLevel:int 2024-08-07T18:03:08.8177436Z parm: NVreg_RmLogonRC:int 2024-08-07T18:03:08.8177745Z parm: NVreg_ModifyDeviceFiles:int 2024-08-07T18:03:08.8178077Z parm: NVreg_DeviceFileUID:int 2024-08-07T18:03:08.8178393Z parm: NVreg_DeviceFileGID:int 2024-08-07T18:03:08.8178703Z parm: NVreg_DeviceFileMode:int 2024-08-07T18:03:08.8179079Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-07T18:03:08.8179482Z parm: NVreg_UsePageAttributeTable:int 2024-08-07T18:03:08.8179987Z parm: NVreg_EnablePCIeGen3:int 2024-08-07T18:03:08.8180310Z parm: NVreg_EnableMSI:int 2024-08-07T18:03:08.8180618Z parm: NVreg_TCEBypassMode:int 2024-08-07T18:03:08.8180952Z parm: NVreg_EnableStreamMemOPs:int 2024-08-07T18:03:08.8181336Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-07T18:03:08.8181842Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-07T18:03:08.8182239Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-07T18:03:08.8182672Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-07T18:03:08.8183100Z parm: NVreg_DynamicPowerManagement:int 2024-08-07T18:03:08.8183558Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-07T18:03:08.8184063Z parm: NVreg_EnableGpuFirmware:int 2024-08-07T18:03:08.8184422Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-07T18:03:08.8184801Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-07T18:03:08.8185190Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-07T18:03:08.8185542Z parm: NVreg_MemoryPoolSize:int 2024-08-07T18:03:08.8185873Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-07T18:03:08.8186234Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-07T18:03:08.8186566Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-07T18:03:08.8186896Z parm: NVreg_NvLinkDisable:int 2024-08-07T18:03:08.8187256Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-07T18:03:08.8187627Z parm: NVreg_RegisterPCIDriver:int 2024-08-07T18:03:08.8187969Z parm: NVreg_EnableResizableBar:int 2024-08-07T18:03:08.8188319Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-07T18:03:08.8188672Z parm: NVreg_EnableNonblockingOpen:int 2024-08-07T18:03:08.8189026Z parm: NVreg_RegistryDwords:charp 2024-08-07T18:03:08.8189385Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-07T18:03:08.8189723Z parm: NVreg_RmMsg:charp 2024-08-07T18:03:08.8190026Z parm: NVreg_GpuBlacklist:charp 2024-08-07T18:03:08.8190357Z parm: NVreg_TemporaryFilePath:charp 2024-08-07T18:03:08.8190698Z parm: NVreg_ExcludedGpus:charp 2024-08-07T18:03:08.8191040Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-07T18:03:08.8191375Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-07T18:03:08.8191720Z parm: NVreg_ImexChannelCount:int 2024-08-07T18:03:08.8192104Z parm: rm_firmware_active:charp 2024-08-07T18:03:08.8192438Z + set +e 2024-08-07T18:03:08.8192646Z + nvidia-smi 2024-08-07T18:03:08.8374156Z Wed Aug 7 18:03:08 2024 2024-08-07T18:03:08.8374539Z +-----------------------------------------------------------------------------------------+ 2024-08-07T18:03:08.8375048Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-07T18:03:08.8375545Z |-----------------------------------------+------------------------+----------------------+ 2024-08-07T18:03:08.8376050Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-07T18:03:08.8376586Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-07T18:03:08.8377194Z | | | MIG M. | 2024-08-07T18:03:08.8377550Z |=========================================+========================+======================| 2024-08-07T18:03:08.8645744Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2024-08-07T18:03:08.8646334Z | 0% 31C P8 16W / 300W | 0MiB / 23028MiB | 0% Default | 2024-08-07T18:03:08.8646736Z | | | N/A | 2024-08-07T18:03:08.8647143Z +-----------------------------------------+------------------------+----------------------+ 2024-08-07T18:03:08.8650282Z 2024-08-07T18:03:08.8650705Z +-----------------------------------------------------------------------------------------+ 2024-08-07T18:03:08.8651156Z | Processes: | 2024-08-07T18:03:08.8651818Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-07T18:03:08.8652262Z | ID ID Usage | 2024-08-07T18:03:08.8652618Z |=========================================================================================| 2024-08-07T18:03:08.8655518Z | No running processes found | 2024-08-07T18:03:08.8656012Z +-----------------------------------------------------------------------------------------+ 2024-08-07T18:03:09.1552330Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-07T18:03:09.1738339Z NVIDIA A10G 2024-08-07T18:03:09.1801852Z + NVIDIA_SMI_STATUS=0 2024-08-07T18:03:09.1802119Z + '[' 0 -eq 0 ']' 2024-08-07T18:03:09.1802374Z + echo 'INFO: Ignoring allowed status 0' 2024-08-07T18:03:09.1802667Z + set -e 2024-08-07T18:03:09.1802902Z INFO: Ignoring allowed status 0 2024-08-07T18:03:09.1812283Z == Installing nvidia container toolkit for amzn2023 == 2024-08-07T18:03:09.1815875Z + sudo yum install -y yum-utils 2024-08-07T18:03:09.6057842Z Last metadata expiration check: 0:20:25 ago on Wed Aug 7 17:42:44 2024. 2024-08-07T18:03:09.6279113Z Package dnf-utils-4.3.0-13.amzn2023.0.4.noarch is already installed. 2024-08-07T18:03:09.6581763Z Dependencies resolved. 2024-08-07T18:03:09.6706043Z Nothing to do. 2024-08-07T18:03:09.6706288Z Complete! 2024-08-07T18:03:09.7675086Z + [[ amzn2023 == \a\m\z\n\2\0\2\3 ]] 2024-08-07T18:03:09.7675717Z + YUM_REPO_URL=https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-07T18:03:09.7676625Z + sudo yum-config-manager --add-repo https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-07T18:03:10.0626735Z Adding repo from: https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-07T18:03:10.1355443Z + sudo yum install -y nvidia-docker2 2024-08-07T18:03:10.6233685Z nvidia-container-toolkit 11 kB/s | 833 B 00:00 2024-08-07T18:03:10.6456307Z Package nvidia-docker2-2.14.0-1.noarch is already installed. 2024-08-07T18:03:10.6753381Z Dependencies resolved. 2024-08-07T18:03:10.6879712Z Nothing to do. 2024-08-07T18:03:10.6880179Z Complete! 2024-08-07T18:03:10.7839451Z + sudo systemctl restart docker 2024-08-07T18:03:52.4075335Z nvidia-persistenced failed to initialize. Check syslog for more details. 2024-08-07T18:03:52.4303695Z Wed Aug 7 18:03:52 2024 2024-08-07T18:03:52.4304306Z +-----------------------------------------------------------------------------------------+ 2024-08-07T18:03:52.4305068Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-07T18:03:52.4305810Z |-----------------------------------------+------------------------+----------------------+ 2024-08-07T18:03:52.4306568Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-07T18:03:52.4307825Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-07T18:03:52.4308470Z | | | MIG M. | 2024-08-07T18:03:52.4308979Z |=========================================+========================+======================| 2024-08-07T18:03:52.4519615Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2024-08-07T18:03:52.4520562Z | 0% 31C P8 16W / 300W | 0MiB / 23028MiB | 0% Default | 2024-08-07T18:03:52.4521330Z | | | N/A | 2024-08-07T18:03:52.4522133Z +-----------------------------------------+------------------------+----------------------+ 2024-08-07T18:03:52.4524107Z 2024-08-07T18:03:52.4524954Z +-----------------------------------------------------------------------------------------+ 2024-08-07T18:03:52.4526425Z | Processes: | 2024-08-07T18:03:52.4527361Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-07T18:03:52.4528221Z | ID ID Usage | 2024-08-07T18:03:52.4528778Z |=========================================================================================| 2024-08-07T18:03:52.4530845Z | No running processes found | 2024-08-07T18:03:52.4531326Z +-----------------------------------------------------------------------------------------+ 2024-08-07T18:03:53.1875051Z Command completed after 1 attempt(s). 2024-08-07T18:03:53.1955002Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-07T18:03:53.1955743Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-07T18:03:53.1956356Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-07T18:03:53.1956925Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-07T18:03:53.1973037Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:03:53.1973510Z env: 2024-08-07T18:03:53.1973786Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:03:53.1974216Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:03:53.1974669Z ##[endgroup] 2024-08-07T18:03:53.4464275Z Defaulting to user installation because normal site-packages is not writeable 2024-08-07T18:03:53.4659946Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.9/site-packages (5.9.1) 2024-08-07T18:03:53.4665267Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.9/site-packages (11.525.84) 2024-08-07T18:03:53.5988946Z Prepare all required actions 2024-08-07T18:03:53.5989481Z Getting action download info 2024-08-07T18:03:53.7014153Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-07T18:03:53.9079104Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-07T18:03:54.0947790Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-07T18:03:54.0948160Z with: 2024-08-07T18:03:54.0948423Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-07T18:03:54.0948765Z s3-bucket: gha-artifacts 2024-08-07T18:03:54.0949045Z env: 2024-08-07T18:03:54.0949273Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:03:54.0949616Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:03:54.0949973Z ##[endgroup] 2024-08-07T18:03:54.0985228Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-07T18:03:54.0985577Z with: 2024-08-07T18:03:54.0985883Z name: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-07T18:03:54.0986417Z s3-bucket: gha-artifacts 2024-08-07T18:03:54.0986701Z region: us-east-1 2024-08-07T18:03:54.0986946Z env: 2024-08-07T18:03:54.0987167Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:03:54.0987507Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:03:54.0987861Z ##[endgroup] 2024-08-07T18:03:54.5889329Z (node:72170) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-07T18:03:54.5889840Z 2024-08-07T18:03:54.5890033Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-07T18:03:54.5890551Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-07T18:03:54.5891106Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-07T18:03:54.6659903Z Found 1 objects with prefix pytorch/pytorch/10288745067/linux-focal-cuda12.1-py3.10-gcc9-sm86/ 2024-08-07T18:03:54.6661335Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-07T18:04:20.5630976Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-07T18:04:20.5637655Z Artifact download has finished successfully 2024-08-07T18:04:20.5997123Z ##[group]Run unzip -o artifacts.zip 2024-08-07T18:04:20.5997497Z unzip -o artifacts.zip 2024-08-07T18:04:20.6006652Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:04:20.6007032Z env: 2024-08-07T18:04:20.6007276Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:20.6007633Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:20.6007991Z ##[endgroup] 2024-08-07T18:04:20.6068566Z Archive: artifacts.zip 2024-08-07T18:04:20.6070124Z creating: dist/ 2024-08-07T18:04:22.7329483Z inflating: dist/torch-2.5.0a0+git016588f-cp310-cp310-linux_x86_64.whl 2024-08-07T18:04:22.7330129Z creating: build/custom_test_artifacts/ 2024-08-07T18:04:22.7330596Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-07T18:04:22.7331214Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-07T18:04:22.7331793Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-07T18:04:22.7338758Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-07T18:04:22.7339601Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-07T18:04:22.7340548Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-07T18:04:22.7341317Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-07T18:04:22.7341991Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-07T18:04:22.7343530Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-07T18:04:22.7345649Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-07T18:04:22.7346504Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-07T18:04:22.7347185Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-07T18:04:22.7349355Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-07T18:04:22.7350713Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-07T18:04:22.7353027Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-07T18:04:22.7353974Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-07T18:04:22.7355472Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-07T18:04:22.7356971Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-07T18:04:22.7358026Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-07T18:04:22.7358772Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-07T18:04:22.7398805Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-07T18:04:22.7439700Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-07T18:04:22.7440790Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-07T18:04:22.7486665Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-07T18:04:22.7487828Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-07T18:04:22.7490260Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-07T18:04:22.7492886Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-07T18:04:22.7495374Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-07T18:04:22.7497863Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-07T18:04:22.7499074Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-07T18:04:22.7500395Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-07T18:04:22.7501518Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-07T18:04:22.7502431Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-07T18:04:22.7503290Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-07T18:04:22.7504133Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-07T18:04:22.7504972Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-07T18:04:22.7505793Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-07T18:04:22.7506621Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-07T18:04:22.7562142Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-07T18:04:22.7622895Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-07T18:04:22.7623908Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-07T18:04:22.7624743Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-07T18:04:22.7625472Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-08-07T18:04:22.7626237Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-07T18:04:22.7627045Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-07T18:04:22.7627937Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-07T18:04:22.7628922Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-07T18:04:22.7630067Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-07T18:04:22.7631008Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-07T18:04:22.7631901Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-07T18:04:22.7632790Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-07T18:04:22.7633730Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-07T18:04:22.7634761Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-07T18:04:22.7635518Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-07T18:04:22.7654813Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-07T18:04:22.7797951Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-07T18:04:22.7800210Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-07T18:04:22.7802262Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-07T18:04:22.7804459Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-07T18:04:22.7806476Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-07T18:04:22.7807963Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-07T18:04:22.7808874Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-07T18:04:22.7809667Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-07T18:04:22.7810463Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-07T18:04:22.7811241Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-07T18:04:22.7812008Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-07T18:04:22.7827318Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-07T18:04:22.7910578Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-07T18:04:22.7911598Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-07T18:04:22.7912621Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-07T18:04:22.7913596Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-07T18:04:22.7914369Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-07T18:04:22.7915420Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-07T18:04:22.7916195Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-08-07T18:04:22.7919205Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-07T18:04:22.7919984Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-07T18:04:22.7920647Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-07T18:04:22.8040952Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-07T18:04:22.8104413Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-07T18:04:22.8105417Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-07T18:04:22.8106352Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-07T18:04:22.8107797Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-07T18:04:22.8112694Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-07T18:04:22.8113335Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-07T18:04:22.8113950Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-07T18:04:22.8114621Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-07T18:04:22.8115272Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-07T18:04:22.8117311Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-07T18:04:22.8118787Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-07T18:04:22.8119461Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-07T18:04:22.8120137Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-07T18:04:22.8122639Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-07T18:04:22.8124033Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-07T18:04:22.8126200Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-07T18:04:22.8126979Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-07T18:04:22.8128922Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-07T18:04:22.8130073Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-07T18:04:22.8130796Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-07T18:04:22.8131478Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-07T18:04:22.8171932Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-07T18:04:22.8211662Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-07T18:04:22.8212639Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-07T18:04:22.8258880Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-07T18:04:22.8259880Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-07T18:04:22.8261340Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-07T18:04:22.8262607Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-07T18:04:22.8263800Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-07T18:04:22.8264911Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-07T18:04:22.8266008Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-07T18:04:22.8267188Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-07T18:04:22.8268325Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-07T18:04:22.8269560Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-07T18:04:22.8270585Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-07T18:04:22.8271515Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-07T18:04:22.8272420Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-07T18:04:22.8273419Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-07T18:04:22.8274255Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-07T18:04:22.8334049Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-07T18:04:22.8394374Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-07T18:04:22.8395705Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-07T18:04:22.8396482Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-07T18:04:22.8397245Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-08-07T18:04:22.8397948Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-07T18:04:22.8398780Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-07T18:04:22.8399647Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-07T18:04:22.8400921Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-07T18:04:22.8401913Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-07T18:04:22.8402753Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-07T18:04:22.8403697Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-07T18:04:22.8404619Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-07T18:04:22.8405533Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-07T18:04:22.8406411Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-07T18:04:22.8407157Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-07T18:04:22.8429441Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-07T18:04:22.8494962Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-07T18:04:22.8497016Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-07T18:04:22.8498488Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-07T18:04:22.8499215Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-07T18:04:22.8500106Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-07T18:04:22.8500717Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-07T18:04:22.8501351Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-08-07T18:04:22.8502115Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-07T18:04:22.8502771Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-07T18:04:22.8503658Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-07T18:04:22.8553559Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-07T18:04:22.8554081Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-07T18:04:22.8554584Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-07T18:04:22.8555194Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-07T18:04:22.8562287Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-07T18:04:22.8562968Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-07T18:04:22.8563641Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-07T18:04:22.8564355Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-07T18:04:22.8565059Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-07T18:04:22.8567157Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-07T18:04:22.8568946Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-07T18:04:22.8569667Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-07T18:04:22.8570376Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-07T18:04:22.8572476Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-07T18:04:22.8573956Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-07T18:04:22.8576121Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-07T18:04:22.8576956Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-07T18:04:22.8578875Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-07T18:04:22.8580095Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-07T18:04:22.8580865Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-07T18:04:22.8581597Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-07T18:04:22.8622110Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-07T18:04:22.8662306Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-07T18:04:22.8663557Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-07T18:04:22.8709611Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-07T18:04:22.8710631Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-07T18:04:22.8711661Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-07T18:04:22.8712710Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-07T18:04:22.8713733Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-07T18:04:22.8714730Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-07T18:04:22.8715912Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-07T18:04:22.8716910Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-07T18:04:22.8717892Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-07T18:04:22.8718874Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-07T18:04:22.8719768Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-07T18:04:22.8720653Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-07T18:04:22.8721555Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-07T18:04:22.8722434Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-07T18:04:22.8723343Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-07T18:04:22.8784116Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-07T18:04:22.8844790Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-07T18:04:22.8845661Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-07T18:04:22.8846386Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-07T18:04:22.8847026Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-08-07T18:04:22.8847824Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-07T18:04:22.8848552Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-07T18:04:22.8849370Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-07T18:04:22.8850257Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-07T18:04:22.8851105Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-07T18:04:22.8851892Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-07T18:04:22.8852709Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-07T18:04:22.8853923Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-07T18:04:22.8854765Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-07T18:04:22.8855595Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-07T18:04:22.8856408Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-07T18:04:22.8860883Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-07T18:04:22.8986775Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-07T18:04:22.8987598Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-07T18:04:22.8988419Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-07T18:04:22.8989350Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-07T18:04:22.8990406Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-07T18:04:22.8991241Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-07T18:04:22.8992101Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-07T18:04:22.8992976Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-07T18:04:22.8993847Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-07T18:04:22.8994712Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-07T18:04:22.8995576Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-07T18:04:22.9016056Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-07T18:04:22.9071160Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-07T18:04:22.9072056Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-07T18:04:22.9072852Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-07T18:04:22.9073580Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-07T18:04:22.9074514Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-07T18:04:22.9076214Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-07T18:04:22.9076886Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-08-07T18:04:22.9079547Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-07T18:04:22.9080469Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-07T18:04:22.9081413Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-07T18:04:22.9185618Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-07T18:04:22.9228986Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-07T18:04:22.9229497Z creating: build/lib/ 2024-08-07T18:04:22.9318546Z inflating: build/lib/libprotobuf-lite.a 2024-08-07T18:04:22.9779738Z inflating: build/lib/libprotobuf.a 2024-08-07T18:04:22.9789060Z inflating: build/lib/libpthreadpool.a 2024-08-07T18:04:22.9797543Z inflating: build/lib/libcpuinfo.a 2024-08-07T18:04:22.9806065Z inflating: build/lib/libcpuinfo_internals.a 2024-08-07T18:04:22.9806756Z inflating: build/lib/libclog.a 2024-08-07T18:04:22.9824882Z inflating: build/lib/libnnpack.a 2024-08-07T18:04:22.9827559Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-07T18:04:22.9892773Z inflating: build/lib/libgtest.a 2024-08-07T18:04:22.9968378Z inflating: build/lib/libbenchmark.a 2024-08-07T18:04:23.0031931Z inflating: build/lib/libasmjit.a 2024-08-07T18:04:23.0039595Z inflating: build/lib/libittnotify.a 2024-08-07T18:04:23.0067974Z inflating: build/lib/libtensorpipe_uv.a 2024-08-07T18:04:23.0194765Z inflating: build/lib/libgloo.a 2024-08-07T18:04:23.0215956Z inflating: build/lib/libfmt.a 2024-08-07T18:04:23.0312504Z inflating: build/lib/libc10.so 2024-08-07T18:04:23.0314431Z inflating: build/lib/libcaffe2_nvrtc.so 2024-08-07T18:04:23.0315051Z inflating: build/lib/libfoxi_loader.a 2024-08-07T18:04:23.0316899Z inflating: build/lib/libtorch_global_deps.so 2024-08-07T18:04:23.0336371Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-07T18:04:23.0354569Z inflating: build/lib/libgmock.a 2024-08-07T18:04:23.0355202Z inflating: build/lib/libgtest_main.a 2024-08-07T18:04:23.0356212Z inflating: build/lib/libbenchmark_main.a 2024-08-07T18:04:23.0860185Z inflating: build/lib/libprotoc.a 2024-08-07T18:04:23.1253570Z inflating: build/lib/libgloo_cuda.a 2024-08-07T18:04:24.1459170Z inflating: build/lib/libdnnl.a 2024-08-07T18:04:24.2033215Z inflating: build/lib/libtensorpipe.a 2024-08-07T18:04:24.2091365Z inflating: build/lib/libc10_cuda.so 2024-08-07T18:04:24.2091979Z inflating: build/lib/libgmock_main.a 2024-08-07T18:04:24.3352832Z inflating: build/lib/libfbgemm.a 2024-08-07T18:04:24.3859238Z inflating: build/lib/libkineto.a 2024-08-07T18:04:24.4115979Z inflating: build/lib/libtensorpipe_cuda.a 2024-08-07T18:04:24.4301904Z inflating: build/lib/libXNNPACK.a 2024-08-07T18:04:24.4344395Z inflating: build/lib/libonnx_proto.a 2024-08-07T18:04:24.5038558Z inflating: build/lib/libonnx.a 2024-08-07T18:04:26.9375949Z inflating: build/lib/libtorch_cpu.so 2024-08-07T18:04:26.9380701Z inflating: build/lib/libunbox_lib.a 2024-08-07T18:04:26.9384997Z inflating: build/lib/libshm.so 2024-08-07T18:04:29.0116409Z inflating: build/lib/libtorch_cuda.so 2024-08-07T18:04:29.0117301Z inflating: build/lib/libtorch.so 2024-08-07T18:04:29.0120339Z inflating: build/lib/libc10d_cuda_test.so 2024-08-07T18:04:29.8336861Z inflating: build/lib/libtorch_cuda_linalg.so 2024-08-07T18:04:30.0339704Z inflating: build/lib/libtorch_python.so 2024-08-07T18:04:30.0411662Z inflating: build/lib/libtorchbind_test.so 2024-08-07T18:04:30.0433039Z inflating: build/lib/libjitbackend_test.so 2024-08-07T18:04:30.0459025Z inflating: build/lib/libaoti_custom_ops.so 2024-08-07T18:04:30.0484648Z inflating: build/lib/libbackend_with_compiler.so 2024-08-07T18:04:30.0519722Z inflating: build/lib/libnnapi_backend.so 2024-08-07T18:04:30.0520078Z creating: build/bin/ 2024-08-07T18:04:30.0572057Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-07T18:04:30.0624466Z inflating: build/bin/c10_DeviceGuard_test 2024-08-07T18:04:30.0675065Z inflating: build/bin/c10_Device_test 2024-08-07T18:04:30.0734935Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-07T18:04:30.0789253Z inflating: build/bin/c10_Scalar_test 2024-08-07T18:04:30.0838588Z inflating: build/bin/c10_StreamGuard_test 2024-08-07T18:04:30.0889829Z inflating: build/bin/c10_SymInt_test 2024-08-07T18:04:30.0944973Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-07T18:04:30.1001192Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-07T18:04:30.1057799Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-07T18:04:30.1128979Z inflating: build/bin/c10_cow_test 2024-08-07T18:04:30.1182782Z inflating: build/bin/c10_Bitset_test 2024-08-07T18:04:30.1232298Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-07T18:04:30.1282049Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-07T18:04:30.1333408Z inflating: build/bin/c10_Half_test 2024-08-07T18:04:30.1390551Z inflating: build/bin/c10_LeftRight_test 2024-08-07T18:04:30.1446118Z inflating: build/bin/c10_Metaprogramming_test 2024-08-07T18:04:30.1496159Z inflating: build/bin/c10_Synchronized_test 2024-08-07T18:04:30.1553696Z inflating: build/bin/c10_ThreadLocal_test 2024-08-07T18:04:30.1605719Z inflating: build/bin/c10_TypeIndex_test 2024-08-07T18:04:30.1658074Z inflating: build/bin/c10_TypeList_test 2024-08-07T18:04:30.1708050Z inflating: build/bin/c10_TypeTraits_test 2024-08-07T18:04:30.1759807Z inflating: build/bin/c10_accumulate_test 2024-08-07T18:04:30.1817154Z inflating: build/bin/c10_bfloat16_test 2024-08-07T18:04:30.1867999Z inflating: build/bin/c10_bit_cast_test 2024-08-07T18:04:30.1925858Z inflating: build/bin/c10_complex_math_test 2024-08-07T18:04:30.1982698Z inflating: build/bin/c10_complex_test 2024-08-07T18:04:30.2034932Z inflating: build/bin/c10_exception_test 2024-08-07T18:04:30.2085590Z inflating: build/bin/c10_flags_test 2024-08-07T18:04:30.2136334Z inflating: build/bin/c10_generic_math_test 2024-08-07T18:04:30.2301917Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-07T18:04:30.2352965Z inflating: build/bin/c10_irange_test 2024-08-07T18:04:30.2407022Z inflating: build/bin/c10_lazy_test 2024-08-07T18:04:30.2464745Z inflating: build/bin/c10_logging_test 2024-08-07T18:04:30.2540824Z inflating: build/bin/c10_optional_test 2024-08-07T18:04:30.2603656Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-07T18:04:30.2658223Z inflating: build/bin/c10_registry_test 2024-08-07T18:04:30.2811652Z inflating: build/bin/c10_small_vector_test 2024-08-07T18:04:30.2863641Z inflating: build/bin/c10_ssize_test 2024-08-07T18:04:30.2916147Z inflating: build/bin/c10_string_util_test 2024-08-07T18:04:30.2975928Z inflating: build/bin/c10_string_view_test 2024-08-07T18:04:30.3026337Z inflating: build/bin/c10_tempfile_test 2024-08-07T18:04:30.3083744Z inflating: build/bin/c10_typeid_test 2024-08-07T18:04:30.3132788Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-07T18:04:30.3580682Z inflating: build/bin/protoc-3.13.0.0 2024-08-07T18:04:30.4034208Z inflating: build/bin/protoc 2024-08-07T18:04:30.4088395Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-08-07T18:04:30.4141925Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-08-07T18:04:30.4194422Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-08-07T18:04:30.4246898Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-08-07T18:04:30.4300760Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-08-07T18:04:30.4353672Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-08-07T18:04:30.4403474Z inflating: build/bin/c10_cuda_CUDATest 2024-08-07T18:04:30.4456897Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-08-07T18:04:30.4790342Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-07T18:04:30.5136263Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-07T18:04:30.5497953Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-07T18:04:30.5550525Z inflating: build/bin/BackoffTest 2024-08-07T18:04:30.5604165Z inflating: build/bin/FileStoreTest 2024-08-07T18:04:30.5661391Z inflating: build/bin/TCPStoreTest 2024-08-07T18:04:30.5716918Z inflating: build/bin/HashStoreTest 2024-08-07T18:04:30.5730625Z inflating: build/bin/ProcessGroupMPITest 2024-08-07T18:04:30.5786112Z inflating: build/bin/test_edge_op_registration 2024-08-07T18:04:30.5790350Z inflating: build/bin/torch_shm_manager 2024-08-07T18:04:30.5793628Z inflating: build/bin/example_allreduce 2024-08-07T18:04:30.5849065Z inflating: build/bin/test_dist_autograd 2024-08-07T18:04:30.5918705Z inflating: build/bin/test_cpp_rpc 2024-08-07T18:04:30.5921221Z inflating: build/bin/parallel_benchmark 2024-08-07T18:04:30.5989475Z inflating: build/bin/test_mobile_nnc 2024-08-07T18:04:30.5998579Z inflating: build/bin/aot_model_compiler_test 2024-08-07T18:04:30.6346234Z inflating: build/bin/test_lazy 2024-08-07T18:04:30.7535813Z inflating: build/bin/test_api 2024-08-07T18:04:30.7610299Z inflating: build/bin/Dict_test 2024-08-07T18:04:30.7663694Z inflating: build/bin/Dimname_test 2024-08-07T18:04:30.7728700Z inflating: build/bin/MaybeOwned_test 2024-08-07T18:04:30.7786620Z inflating: build/bin/NamedTensor_test 2024-08-07T18:04:30.7846213Z inflating: build/bin/apply_utils_test 2024-08-07T18:04:30.7906066Z inflating: build/bin/atest 2024-08-07T18:04:30.7969583Z inflating: build/bin/basic 2024-08-07T18:04:30.8025470Z inflating: build/bin/broadcast_test 2024-08-07T18:04:30.8076440Z inflating: build/bin/cpu_allocator_test 2024-08-07T18:04:30.8135139Z inflating: build/bin/cpu_generator_test 2024-08-07T18:04:30.8189047Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-07T18:04:30.8282845Z inflating: build/bin/cpu_rng_test 2024-08-07T18:04:30.8335058Z inflating: build/bin/dispatch_key_set_test 2024-08-07T18:04:30.8385362Z inflating: build/bin/dlconvertor_test 2024-08-07T18:04:30.8444819Z inflating: build/bin/extension_backend_test 2024-08-07T18:04:30.8500149Z inflating: build/bin/half_test 2024-08-07T18:04:30.8596889Z inflating: build/bin/ivalue_test 2024-08-07T18:04:30.8647054Z inflating: build/bin/lazy_tensor_test 2024-08-07T18:04:30.8703103Z inflating: build/bin/math_kernel_test 2024-08-07T18:04:30.8757383Z inflating: build/bin/memory_format_test 2024-08-07T18:04:30.8811209Z inflating: build/bin/memory_overlapping_test 2024-08-07T18:04:30.8864809Z inflating: build/bin/mobile_memory_cleanup 2024-08-07T18:04:30.8921520Z inflating: build/bin/native_test 2024-08-07T18:04:30.8972315Z inflating: build/bin/operator_name_test 2024-08-07T18:04:30.9025069Z inflating: build/bin/operators_test 2024-08-07T18:04:30.9077304Z inflating: build/bin/packedtensoraccessor_test 2024-08-07T18:04:30.9145368Z inflating: build/bin/pow_test 2024-08-07T18:04:30.9203775Z inflating: build/bin/quantized_test 2024-08-07T18:04:30.9253912Z inflating: build/bin/reduce_ops_test 2024-08-07T18:04:30.9305850Z inflating: build/bin/reportMemoryUsage_test 2024-08-07T18:04:30.9364263Z inflating: build/bin/scalar_tensor_test 2024-08-07T18:04:30.9423574Z inflating: build/bin/scalar_test 2024-08-07T18:04:30.9475757Z inflating: build/bin/StorageUtils_test 2024-08-07T18:04:30.9529157Z inflating: build/bin/stride_properties_test 2024-08-07T18:04:30.9608866Z inflating: build/bin/tensor_iterator_test 2024-08-07T18:04:30.9663428Z inflating: build/bin/test_parallel 2024-08-07T18:04:30.9666595Z inflating: build/bin/thread_init_test 2024-08-07T18:04:30.9722989Z inflating: build/bin/type_ptr_test 2024-08-07T18:04:30.9783622Z inflating: build/bin/type_test 2024-08-07T18:04:30.9836474Z inflating: build/bin/undefined_tensor_test 2024-08-07T18:04:30.9838073Z inflating: build/bin/verify_api_visibility 2024-08-07T18:04:30.9908066Z inflating: build/bin/legacy_vmap_test 2024-08-07T18:04:30.9960209Z inflating: build/bin/weakref_test 2024-08-07T18:04:31.0012001Z inflating: build/bin/wrapdim_test 2024-08-07T18:04:31.0065029Z inflating: build/bin/xla_tensor_test 2024-08-07T18:04:31.0125206Z inflating: build/bin/IListRef_test 2024-08-07T18:04:31.0232212Z inflating: build/bin/List_test 2024-08-07T18:04:31.0299305Z inflating: build/bin/KernelFunction_test 2024-08-07T18:04:31.0422551Z inflating: build/bin/kernel_function_legacy_test 2024-08-07T18:04:31.0522471Z inflating: build/bin/kernel_function_test 2024-08-07T18:04:31.0651964Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-07T18:04:31.0756370Z inflating: build/bin/kernel_lambda_test 2024-08-07T18:04:31.0818054Z inflating: build/bin/kernel_stackbased_test 2024-08-07T18:04:31.0915062Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-07T18:04:31.0967463Z inflating: build/bin/CppSignature_test 2024-08-07T18:04:31.1023360Z inflating: build/bin/backend_fallback_test 2024-08-07T18:04:31.1072494Z inflating: build/bin/op_allowlist_test 2024-08-07T18:04:31.1376996Z inflating: build/bin/op_registration_test 2024-08-07T18:04:31.1441510Z inflating: build/bin/inline_container_test 2024-08-07T18:04:31.1494645Z inflating: build/bin/cuda_allocator_test 2024-08-07T18:04:31.1550372Z inflating: build/bin/cuda_apply_test 2024-08-07T18:04:31.1605576Z inflating: build/bin/cuda_caching_host_allocator_test 2024-08-07T18:04:31.1666604Z inflating: build/bin/cuda_atomic_ops_test 2024-08-07T18:04:31.1737153Z inflating: build/bin/cuda_complex_math_test 2024-08-07T18:04:31.1796880Z inflating: build/bin/cuda_complex_test 2024-08-07T18:04:31.1846864Z inflating: build/bin/cuda_device_test 2024-08-07T18:04:31.1904898Z inflating: build/bin/cuda_cub_test 2024-08-07T18:04:31.1956051Z inflating: build/bin/cuda_dlconvertor_test 2024-08-07T18:04:31.2021736Z inflating: build/bin/cuda_distributions_test 2024-08-07T18:04:31.2078972Z inflating: build/bin/cuda_generator_test 2024-08-07T18:04:31.2129030Z inflating: build/bin/cuda_half_test 2024-08-07T18:04:31.2180987Z inflating: build/bin/cuda_integer_divider_test 2024-08-07T18:04:31.2230669Z inflating: build/bin/cuda_optional_test 2024-08-07T18:04:31.2283536Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-08-07T18:04:31.2336041Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-08-07T18:04:31.2388808Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-08-07T18:04:31.2450522Z inflating: build/bin/cuda_stream_test 2024-08-07T18:04:31.2500721Z inflating: build/bin/cuda_cudnn_test 2024-08-07T18:04:31.2554674Z inflating: build/bin/cuda_vectorized_test 2024-08-07T18:04:31.2569248Z inflating: build/bin/tutorial_tensorexpr 2024-08-07T18:04:31.2635574Z inflating: build/bin/ProcessGroupGlooTest 2024-08-07T18:04:31.2695310Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-08-07T18:04:31.2759661Z inflating: build/bin/ProcessGroupNCCLTest 2024-08-07T18:04:31.2822476Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-08-07T18:04:31.3653779Z inflating: build/bin/test_tensorexpr 2024-08-07T18:04:31.4238904Z inflating: build/bin/test_jit 2024-08-07T18:04:31.4239337Z creating: .additional_ci_files/ 2024-08-07T18:04:31.4297719Z inflating: .additional_ci_files/test-times.json 2024-08-07T18:04:31.4530002Z inflating: .additional_ci_files/test-class-times.json 2024-08-07T18:04:31.4566571Z ##[group]Run rm artifacts.zip 2024-08-07T18:04:31.4566897Z rm artifacts.zip 2024-08-07T18:04:31.4575745Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:04:31.4576122Z env: 2024-08-07T18:04:31.4576340Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:31.4576662Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:31.4577009Z ##[endgroup] 2024-08-07T18:04:31.7155916Z ##[group]Run df -H 2024-08-07T18:04:31.7156195Z df -H 2024-08-07T18:04:31.7164713Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:04:31.7165080Z env: 2024-08-07T18:04:31.7165300Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:31.7165626Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:31.7165977Z ##[endgroup] 2024-08-07T18:04:31.7215335Z Filesystem Size Used Avail Use% Mounted on 2024-08-07T18:04:31.7215724Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-07T18:04:31.7216047Z tmpfs 34G 25k 34G 1% /dev/shm 2024-08-07T18:04:31.7216375Z tmpfs 14G 553k 14G 1% /run 2024-08-07T18:04:31.7216894Z /dev/nvme0n1p1 161G 38G 124G 24% / 2024-08-07T18:04:31.7217227Z tmpfs 34G 8.2k 34G 1% /tmp 2024-08-07T18:04:31.7217566Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-07T18:04:31.7217913Z tmpfs 6.7G 0 6.7G 0% /run/user/0 2024-08-07T18:04:31.7252209Z Prepare all required actions 2024-08-07T18:04:31.7252577Z Getting action download info 2024-08-07T18:04:31.8582572Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-07T18:04:31.8594277Z with: 2024-08-07T18:04:31.8594586Z env: 2024-08-07T18:04:31.8594898Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:31.8595312Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:31.8595691Z ##[endgroup] 2024-08-07T18:04:31.8632723Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-07T18:04:31.8633067Z with: 2024-08-07T18:04:31.8633292Z name: td_results 2024-08-07T18:04:31.8633544Z s3-bucket: gha-artifacts 2024-08-07T18:04:31.8633825Z region: us-east-1 2024-08-07T18:04:31.8634067Z env: 2024-08-07T18:04:31.8634495Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:31.8634838Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:31.8635200Z ##[endgroup] 2024-08-07T18:04:32.3529648Z (node:72196) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-07T18:04:32.3530611Z 2024-08-07T18:04:32.3531094Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-07T18:04:32.3532146Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-07T18:04:32.3532899Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-07T18:04:32.4401349Z Found 1 objects with prefix pytorch/pytorch/10288745067/td_results/ 2024-08-07T18:04:32.4402562Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-07T18:04:32.4912592Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-07T18:04:32.4918839Z Artifact download has finished successfully 2024-08-07T18:04:32.5248665Z ##[group]Run mkdir -p .additional_ci_files 2024-08-07T18:04:32.5249041Z mkdir -p .additional_ci_files 2024-08-07T18:04:32.5249459Z mv td_results.json .additional_ci_files/td_results.json 2024-08-07T18:04:32.5260734Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:04:32.5261107Z env: 2024-08-07T18:04:32.5261331Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:32.5261672Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:32.5262031Z ##[endgroup] 2024-08-07T18:04:32.5354675Z ##[group]Run .github/scripts/parse_ref.py 2024-08-07T18:04:32.5355052Z .github/scripts/parse_ref.py 2024-08-07T18:04:32.5362871Z shell: /usr/bin/bash -e {0} 2024-08-07T18:04:32.5363136Z env: 2024-08-07T18:04:32.5363357Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:32.5363692Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:32.5364042Z ##[endgroup] 2024-08-07T18:04:32.5638800Z Prepare all required actions 2024-08-07T18:04:32.5675103Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-07T18:04:32.5675437Z with: 2024-08-07T18:04:32.5675818Z github-token: *** 2024-08-07T18:04:32.5676052Z env: 2024-08-07T18:04:32.5676272Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:32.5676598Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:32.5676944Z ##[endgroup] 2024-08-07T18:04:32.5700937Z ##[group]Run set -eux 2024-08-07T18:04:32.5701201Z set -eux 2024-08-07T18:04:32.5701631Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-07T18:04:32.5710204Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:04:32.5710584Z env: 2024-08-07T18:04:32.5710814Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:32.5711145Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:32.5711649Z GITHUB_TOKEN: *** 2024-08-07T18:04:32.5711889Z ##[endgroup] 2024-08-07T18:04:32.5739123Z + python3 .github/scripts/get_workflow_job_id.py 10288745067 i-031f7aad237b89ef3 2024-08-07T18:04:35.5641811Z setting job-id=28476695132 2024-08-07T18:04:35.5642402Z setting job-name=linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 1, 5, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-07T18:04:35.5812496Z Prepare all required actions 2024-08-07T18:04:35.5813015Z Getting action download info 2024-08-07T18:04:35.6972501Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-07T18:04:35.6972845Z with: 2024-08-07T18:04:35.6973227Z github-token: *** 2024-08-07T18:04:35.6974800Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]} 2024-08-07T18:04:35.6976896Z job-name: linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 1, 5, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-07T18:04:35.6977432Z env: 2024-08-07T18:04:35.6977696Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:35.6978040Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:35.6978396Z ##[endgroup] 2024-08-07T18:04:35.7031971Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-07T18:04:35.7032372Z with: 2024-08-07T18:04:35.7032594Z shell: bash 2024-08-07T18:04:35.7032823Z timeout_minutes: 10 2024-08-07T18:04:35.7033076Z max_attempts: 5 2024-08-07T18:04:35.7033322Z retry_wait_seconds: 30 2024-08-07T18:04:35.7034055Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-07T18:04:35.7034831Z polling_interval_seconds: 1 2024-08-07T18:04:35.7035119Z warning_on_retry: true 2024-08-07T18:04:35.7035383Z continue_on_error: false 2024-08-07T18:04:35.7035643Z env: 2024-08-07T18:04:35.7035863Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:35.7036193Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:35.7036831Z GITHUB_TOKEN: *** 2024-08-07T18:04:35.7037072Z ##[endgroup] 2024-08-07T18:04:35.7794608Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-07T18:04:36.0161030Z Defaulting to user installation because normal site-packages is not writeable 2024-08-07T18:04:36.0341551Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.9/site-packages (2.27.1) 2024-08-07T18:04:36.0344553Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.9/site-packages (6.0.1) 2024-08-07T18:04:36.0465454Z Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2.0.12) 2024-08-07T18:04:36.0469521Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2024.7.4) 2024-08-07T18:04:36.0478368Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-08-07T18:04:36.0483409Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-08-07T18:04:36.7584162Z Command completed after 1 attempt(s). 2024-08-07T18:04:36.7683969Z ##[group]Run set -x 2024-08-07T18:04:36.7684249Z set -x 2024-08-07T18:04:36.7684485Z  2024-08-07T18:04:36.7684870Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-07T18:04:36.7685338Z # in runner workspace 2024-08-07T18:04:36.7685718Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-07T18:04:36.7695091Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:04:36.7695478Z env: 2024-08-07T18:04:36.7695698Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:36.7696033Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:36.7696389Z ##[endgroup] 2024-08-07T18:04:36.7725874Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-07T18:04:36.7969274Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-07T18:04:36.7969676Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-07T18:04:36.7970025Z echo "Job name: ${JOB_NAME}" 2024-08-07T18:04:36.7970320Z  2024-08-07T18:04:36.7970701Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-07T18:04:36.7971168Z # in runner workspace 2024-08-07T18:04:36.7971598Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-07T18:04:36.7972061Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-07T18:04:36.7972596Z  --job-name "${JOB_NAME}" \ 2024-08-07T18:04:36.7974279Z  --test-matrix "{"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]}" \ 2024-08-07T18:04:36.7975955Z  --selected-test-configs "" \ 2024-08-07T18:04:36.7976290Z  --pr-number "${PR_NUMBER}" \ 2024-08-07T18:04:36.7976605Z  --tag "${TAG}" \ 2024-08-07T18:04:36.7976902Z  --event-name "${EVENT_NAME}" \ 2024-08-07T18:04:36.7977223Z  --schedule "${SCHEDULE}" \ 2024-08-07T18:04:36.7977545Z  --branch "${HEAD_BRANCH}" 2024-08-07T18:04:36.7985937Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:04:36.7986297Z env: 2024-08-07T18:04:36.7986520Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:36.7986860Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:36.7987373Z GITHUB_TOKEN: *** 2024-08-07T18:04:36.7987859Z JOB_NAME: linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 1, 5, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-07T18:04:36.7988396Z PR_NUMBER: 131248 2024-08-07T18:04:36.7988628Z TAG: 2024-08-07T18:04:36.7988844Z EVENT_NAME: pull_request 2024-08-07T18:04:36.7989109Z SCHEDULE: 2024-08-07T18:04:36.7989326Z HEAD_BRANCH: 2024-08-07T18:04:36.7989563Z ##[endgroup] 2024-08-07T18:04:36.8015852Z Workflow: pull 2024-08-07T18:04:36.8016791Z Job name: linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 1, 5, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-07T18:04:37.0319580Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-07T18:04:37.1981671Z ##[group]Run echo "Filtered matrix:" 2024-08-07T18:04:37.1982017Z echo "Filtered matrix:" 2024-08-07T18:04:37.1983653Z echo "{"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "amz2023.linux.g5.4xlarge.nvidia.gpu"}]}" 2024-08-07T18:04:37.1985264Z  2024-08-07T18:04:37.1985485Z echo 2024-08-07T18:04:37.1985762Z echo "Is the current job unstable? False" 2024-08-07T18:04:37.1986100Z  2024-08-07T18:04:37.1986318Z echo 2024-08-07T18:04:37.1986581Z echo "Is keep-going label set? False" 2024-08-07T18:04:37.1986916Z  2024-08-07T18:04:37.1987133Z echo 2024-08-07T18:04:37.1987374Z echo "Renabled issues? " 2024-08-07T18:04:37.1995956Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:04:37.1996330Z env: 2024-08-07T18:04:37.1996552Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:37.1996891Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:37.1997243Z ##[endgroup] 2024-08-07T18:04:37.2025484Z Filtered matrix: 2024-08-07T18:04:37.2027038Z {include: [{config: default, shard: 1, num_shards: 5, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: default, shard: 2, num_shards: 5, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: default, shard: 3, num_shards: 5, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: default, shard: 4, num_shards: 5, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}, {config: default, shard: 5, num_shards: 5, runner: amz2023.linux.g5.4xlarge.nvidia.gpu}]} 2024-08-07T18:04:37.2028786Z 2024-08-07T18:04:37.2028920Z Is the current job unstable? False 2024-08-07T18:04:37.2029121Z 2024-08-07T18:04:37.2029235Z Is keep-going label set? False 2024-08-07T18:04:37.2029429Z 2024-08-07T18:04:37.2029535Z Renabled issues? 2024-08-07T18:04:37.2069888Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-07T18:04:37.2070385Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-07T18:04:37.2078366Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:04:37.2078741Z env: 2024-08-07T18:04:37.2078963Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:37.2079301Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:37.2079660Z JOB_TIMEOUT: 240 2024-08-07T18:04:37.2079898Z ##[endgroup] 2024-08-07T18:04:37.2152183Z ##[group]Run set -x 2024-08-07T18:04:37.2152495Z set -x 2024-08-07T18:04:37.2152753Z  2024-08-07T18:04:37.2153037Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-07T18:04:37.2153426Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-07T18:04:37.2153807Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-07T18:04:37.2154160Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-07T18:04:37.2154449Z else 2024-08-07T18:04:37.2154705Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-07T18:04:37.2155007Z fi 2024-08-07T18:04:37.2155215Z  2024-08-07T18:04:37.2155550Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-07T18:04:37.2156060Z # TODO: Stop building test binaries as part of the build phase 2024-08-07T18:04:37.2156500Z # Used for GPU_FLAG since that doesn't play nice 2024-08-07T18:04:37.2156900Z # shellcheck disable=SC2086,SC2090 2024-08-07T18:04:37.2157222Z container_name=$(docker run \ 2024-08-07T18:04:37.2157529Z  ${GPU_FLAG:-} \ 2024-08-07T18:04:37.2157809Z  -e BUILD_ENVIRONMENT \ 2024-08-07T18:04:37.2158099Z  -e PR_NUMBER \ 2024-08-07T18:04:37.2158371Z  -e GITHUB_ACTIONS \ 2024-08-07T18:04:37.2158656Z  -e GITHUB_REPOSITORY \ 2024-08-07T18:04:37.2158942Z  -e GITHUB_WORKFLOW \ 2024-08-07T18:04:37.2159225Z  -e GITHUB_JOB \ 2024-08-07T18:04:37.2159490Z  -e GITHUB_RUN_ID \ 2024-08-07T18:04:37.2159758Z  -e GITHUB_RUN_NUMBER \ 2024-08-07T18:04:37.2160048Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-07T18:04:37.2160339Z  -e JOB_ID \ 2024-08-07T18:04:37.2160583Z  -e JOB_NAME \ 2024-08-07T18:04:37.2160840Z  -e BASE_SHA \ 2024-08-07T18:04:37.2161096Z  -e BRANCH \ 2024-08-07T18:04:37.2161339Z  -e SHA1 \ 2024-08-07T18:04:37.2161595Z  -e AWS_DEFAULT_REGION \ 2024-08-07T18:04:37.2161885Z  -e IN_WHEEL_TEST \ 2024-08-07T18:04:37.2162149Z  -e SHARD_NUMBER \ 2024-08-07T18:04:37.2162419Z  -e TEST_CONFIG \ 2024-08-07T18:04:37.2162697Z  -e NUM_TEST_SHARDS \ 2024-08-07T18:04:37.2163024Z  -e REENABLED_ISSUES \ 2024-08-07T18:04:37.2163323Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-07T18:04:37.2163628Z  -e VERBOSE_TEST_LOGS \ 2024-08-07T18:04:37.2163908Z  -e TEST_SHOWLOCALS \ 2024-08-07T18:04:37.2164192Z  -e NO_TEST_TIMEOUT \ 2024-08-07T18:04:37.2164462Z  -e NO_TD \ 2024-08-07T18:04:37.2164709Z  -e TD_DISTRIBUTED \ 2024-08-07T18:04:37.2164992Z  -e PR_LABELS \ 2024-08-07T18:04:37.2165281Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-07T18:04:37.2165598Z  -e SCCACHE_BUCKET \ 2024-08-07T18:04:37.2165891Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-07T18:04:37.2166186Z  -e XLA_CUDA \ 2024-08-07T18:04:37.2166470Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-07T18:04:37.2166823Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-07T18:04:37.2167182Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-07T18:04:37.2167715Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-07T18:04:37.2168059Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-07T18:04:37.2168370Z  -e DASHBOARD_TAG \ 2024-08-07T18:04:37.2168701Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-07T18:04:37.2169094Z  --security-opt seccomp=unconfined \ 2024-08-07T18:04:37.2169433Z  --cap-add=SYS_PTRACE \ 2024-08-07T18:04:37.2169715Z  --ipc=host \ 2024-08-07T18:04:37.2169985Z  --shm-size="${SHM_SIZE}" \ 2024-08-07T18:04:37.2170275Z  --tty \ 2024-08-07T18:04:37.2170507Z  --detach \ 2024-08-07T18:04:37.2170773Z  --name="${container_name}" \ 2024-08-07T18:04:37.2171199Z  --user jenkins \ 2024-08-07T18:04:37.2171544Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-07T18:04:37.2171940Z  -w /var/lib/jenkins/workspace \ 2024-08-07T18:04:37.2172262Z  "${DOCKER_IMAGE}" 2024-08-07T18:04:37.2172514Z ) 2024-08-07T18:04:37.2172807Z # Propagate download.pytorch.org IP to container 2024-08-07T18:04:37.2173479Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-07T18:04:37.2174134Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-07T18:04:37.2174750Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-07T18:04:37.2182946Z shell: /usr/bin/bash -e {0} 2024-08-07T18:04:37.2183220Z env: 2024-08-07T18:04:37.2183442Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:04:37.2183774Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:04:37.2184217Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-07T18:04:37.2184582Z PR_NUMBER: 131248 2024-08-07T18:04:37.2184841Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-07T18:04:37.2185152Z GITHUB_WORKFLOW: pull 2024-08-07T18:04:37.2185397Z GITHUB_JOB: test 2024-08-07T18:04:37.2185639Z GITHUB_RUN_ID: 10288745067 2024-08-07T18:04:37.2185914Z GITHUB_RUN_NUMBER: 234358 2024-08-07T18:04:37.2186176Z GITHUB_RUN_ATTEMPT: 1 2024-08-07T18:04:37.2186424Z JOB_ID: 28476695132 2024-08-07T18:04:37.2186912Z JOB_NAME: linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 1, 5, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-07T18:04:37.2187442Z BRANCH: pull/131248 2024-08-07T18:04:37.2187722Z SHA1: 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T18:04:37.2188081Z BASE_SHA: 6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T18:04:37.2188404Z TEST_CONFIG: default 2024-08-07T18:04:37.2188652Z SHARD_NUMBER: 1 2024-08-07T18:04:37.2188889Z NUM_TEST_SHARDS: 5 2024-08-07T18:04:37.2189133Z REENABLED_ISSUES: 2024-08-07T18:04:37.2189390Z CONTINUE_THROUGH_ERROR: False 2024-08-07T18:04:37.2189674Z VERBOSE_TEST_LOGS: False 2024-08-07T18:04:37.2189939Z TEST_SHOWLOCALS: False 2024-08-07T18:04:37.2190204Z NO_TEST_TIMEOUT: False 2024-08-07T18:04:37.2190454Z NO_TD: False 2024-08-07T18:04:37.2190682Z TD_DISTRIBUTED: False 2024-08-07T18:04:37.2190994Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-07T18:04:37.2191356Z SCCACHE_S3_KEY_PREFIX: pull 2024-08-07T18:04:37.2191619Z SHM_SIZE: 2g 2024-08-07T18:04:37.2192298Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T18:04:37.2193078Z XLA_CUDA: 2024-08-07T18:04:37.2193424Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-07T18:04:37.2193864Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-07T18:04:37.2194182Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-07T18:04:37.2194479Z DASHBOARD_TAG: 2024-08-07T18:04:37.2194720Z HUGGING_FACE_HUB_TOKEN: 2024-08-07T18:04:37.2194981Z ##[endgroup] 2024-08-07T18:04:37.2221413Z + [[ default == \m\u\l\t\i\g\p\u ]] 2024-08-07T18:04:37.2221989Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *onnx* ]] 2024-08-07T18:04:37.2222367Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-07T18:04:37.2230160Z +++ nproc --ignore=2 2024-08-07T18:04:37.2246516Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10288745067 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T18:04:48.5777871Z + container_name=17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:04:48.5785025Z + grep download.pytorch.org /etc/hosts 2024-08-07T18:04:48.5787021Z + docker exec -i 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-07T18:04:48.7367881Z + echo DOCKER_CONTAINER_ID=17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:04:48.7371451Z ++ echo dist/torch-2.5.0a0+git016588f-cp310-cp310-linux_x86_64.whl 2024-08-07T18:04:48.7374429Z + docker exec -t 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a sh -c 'pip install dist/torch-2.5.0a0+git016588f-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-07T18:04:49.1549122Z Processing ./dist/torch-2.5.0a0+git016588f-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+git016588f) 2024-08-07T18:04:49.4769410Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (3.13.1) 2024-08-07T18:04:49.4773158Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (4.12.2) 2024-08-07T18:04:49.4776247Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (2.8.8) 2024-08-07T18:04:49.4778849Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (3.1.4) 2024-08-07T18:04:49.4782336Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (2024.6.1) 2024-08-07T18:04:49.4792408Z Requirement already satisfied: sympy>=1.13.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (1.13.1) 2024-08-07T18:04:49.4822812Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (3.3.0) 2024-08-07T18:04:49.4882532Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (1.21.2) 2024-08-07T18:04:49.4915708Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy>=1.13.0->torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (1.3.0) 2024-08-07T18:04:49.5819127Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (2.1.5) 2024-08-07T18:04:50.3932054Z Installing collected packages: torch 2024-08-07T18:05:00.8580288Z Successfully installed torch-2.5.0a0+git016588f 2024-08-07T18:05:00.9359664Z ++ dirname .ci/pytorch/test.sh 2024-08-07T18:05:00.9369411Z + source .ci/pytorch/common.sh 2024-08-07T18:05:00.9373486Z +++ dirname .ci/pytorch/common.sh 2024-08-07T18:05:00.9383082Z ++ source .ci/pytorch/common_utils.sh 2024-08-07T18:05:00.9385292Z +++ declare -f -t trap_add 2024-08-07T18:05:00.9391320Z ++ set -ex 2024-08-07T18:05:00.9393684Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-07T18:05:00.9394642Z ++ BUILD_TEST_LIBTORCH=0 2024-08-07T18:05:00.9395024Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *rocm* ]] 2024-08-07T18:05:00.9396133Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-07T18:05:00.9416804Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-07T18:05:00.9417187Z + trap_add cleanup_workspace EXIT 2024-08-07T18:05:00.9417495Z + trap_add_cmd=cleanup_workspace 2024-08-07T18:05:00.9417773Z + shift 2024-08-07T18:05:00.9417998Z + for trap_add_name in "$@" 2024-08-07T18:05:00.9426109Z +++ trap -p EXIT 2024-08-07T18:05:00.9429028Z ++ eval 'extract_trap_cmd ' 2024-08-07T18:05:00.9429427Z +++ extract_trap_cmd 2024-08-07T18:05:00.9429803Z +++ printf '%s\n' '' 2024-08-07T18:05:00.9430188Z ++ printf '%s\n' cleanup_workspace 2024-08-07T18:05:00.9433087Z + trap -- ' 2024-08-07T18:05:00.9433326Z cleanup_workspace' EXIT 2024-08-07T18:05:00.9433643Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-07T18:05:01.6247043Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-07T18:05:01.6270448Z + echo 'Environment variables:' 2024-08-07T18:05:01.6270771Z Environment variables: 2024-08-07T18:05:01.6271018Z + env 2024-08-07T18:05:01.6282300Z INSTALLED_DB=yes 2024-08-07T18:05:01.6282669Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-07T18:05:01.6283131Z NVIDIA_VISIBLE_DEVICES=all 2024-08-07T18:05:01.6283461Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-07T18:05:01.6284028Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T18:05:01.6284577Z CONTINUE_THROUGH_ERROR=False 2024-08-07T18:05:01.6284911Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-07T18:05:01.6285341Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-07T18:05:01.6285926Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-07T18:05:01.6286470Z HOSTNAME=17250618960b 2024-08-07T18:05:01.6287298Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_656853b4-1594-4d68-a695-030c54287386 2024-08-07T18:05:01.6288064Z GITHUB_ACTION=__self 2024-08-07T18:05:01.6300934Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-07T18:05:01.6303717Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-07T18:05:01.6306466Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-07T18:05:01.6306831Z NV_NVTX_VERSION=12.1.105-1 2024-08-07T18:05:01.6307105Z GITHUB_RUN_NUMBER=234358 2024-08-07T18:05:01.6307360Z TEST_CONFIG=default 2024-08-07T18:05:01.6307616Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-07T18:05:01.6307939Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-07T18:05:01.6308554Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-07T18:05:01.6308871Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-07T18:05:01.6309180Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-07T18:05:01.6309467Z GITHUB_TRIGGERING_ACTOR=zdevito 2024-08-07T18:05:01.6309807Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-07T18:05:01.6310152Z GITHUB_REF_TYPE=branch 2024-08-07T18:05:01.6310430Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-07T18:05:01.6310706Z NCCL_VERSION=2.17.1-1 2024-08-07T18:05:01.6310983Z BASE_SHA=6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T18:05:01.6311310Z XLA_CUDA= 2024-08-07T18:05:01.6311530Z HUGGING_FACE_HUB_TOKEN= 2024-08-07T18:05:01.6311948Z *** 2024-08-07T18:05:01.6312183Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-07T18:05:01.6312491Z GITHUB_REPOSITORY_ID=65600975 2024-08-07T18:05:01.6312926Z GITHUB_ACTIONS=true 2024-08-07T18:05:01.6313201Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:05:01.6313538Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-07T18:05:01.6313894Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-07T18:05:01.6314260Z SHA1=016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T18:05:01.6314606Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-07T18:05:01.6314957Z GITHUB_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T18:05:01.6315514Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/131248/merge 2024-08-07T18:05:01.6315994Z UCC_HOME=/usr 2024-08-07T18:05:01.6316246Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-07T18:05:01.6316545Z VERBOSE_TEST_LOGS=False 2024-08-07T18:05:01.6316820Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-07T18:05:01.6317142Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-07T18:05:01.6317487Z GITHUB_REF=refs/pull/131248/merge 2024-08-07T18:05:01.6317778Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-07T18:05:01.6318058Z SHARD_NUMBER=1 2024-08-07T18:05:01.6318291Z GITHUB_REF_PROTECTED=false 2024-08-07T18:05:01.6318557Z HOME=/var/lib/jenkins 2024-08-07T18:05:01.6318835Z GITHUB_API_URL=https://api.github.com 2024-08-07T18:05:01.6319177Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-07T18:05:01.6319524Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-07T18:05:01.6319997Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-07T18:05:01.6320287Z CUDA_VERSION=12.1.1 2024-08-07T18:05:01.6320601Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-07T18:05:01.6320926Z NUM_TEST_SHARDS=5 2024-08-07T18:05:01.6321162Z UCX_HOME=/usr 2024-08-07T18:05:01.6321512Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-07T18:05:01.6322228Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_656853b4-1594-4d68-a695-030c54287386 2024-08-07T18:05:01.6323063Z JOB_NAME=linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 1, 5, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-07T18:05:01.6323884Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_656853b4-1594-4d68-a695-030c54287386 2024-08-07T18:05:01.6324644Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-07T18:05:01.6325139Z GITHUB_EVENT_NAME=pull_request 2024-08-07T18:05:01.6325424Z DASHBOARD_TAG= 2024-08-07T18:05:01.6325667Z GITHUB_RUN_ID=10288745067 2024-08-07T18:05:01.6326055Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-07T18:05:01.6326423Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-07T18:05:01.6327078Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_656853b4-1594-4d68-a695-030c54287386 2024-08-07T18:05:01.6327694Z GITHUB_ACTOR=zdevito 2024-08-07T18:05:01.6327959Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-07T18:05:01.6328249Z PR_NUMBER=131248 2024-08-07T18:05:01.6328482Z GITHUB_RUN_ATTEMPT=1 2024-08-07T18:05:01.6328748Z ANACONDA_PYTHON_VERSION=3.10 2024-08-07T18:05:01.6329086Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-07T18:05:01.6329417Z TERM=xterm 2024-08-07T18:05:01.6329658Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-07T18:05:01.6329966Z INSTALLED_VISION=yes 2024-08-07T18:05:01.6330359Z BRANCH=pull/131248 2024-08-07T18:05:01.6330643Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-07T18:05:01.6330948Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-07T18:05:01.6331262Z CUDA_PATH=/usr/local/cuda 2024-08-07T18:05:01.6331770Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-07T18:05:01.6332319Z GITHUB_SERVER_URL=https://github.com 2024-08-07T18:05:01.6332659Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-07T18:05:01.6332989Z REENABLED_ISSUES= 2024-08-07T18:05:01.6333214Z SHLVL=1 2024-08-07T18:05:01.6333411Z MAX_JOBS=14 2024-08-07T18:05:01.6333637Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-07T18:05:01.6333895Z NVARCH=x86_64 2024-08-07T18:05:01.6334108Z GITHUB_ACTOR_ID=370202 2024-08-07T18:05:01.6334520Z GITHUB_WORKFLOW_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T18:05:01.6334896Z GITHUB_REF_NAME=131248/merge 2024-08-07T18:05:01.6335198Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-07T18:05:01.6335646Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-07T18:05:01.6336054Z GITHUB_JOB=test 2024-08-07T18:05:01.6336315Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-07T18:05:01.6336720Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-07T18:05:01.6337085Z NO_TEST_TIMEOUT=False 2024-08-07T18:05:01.6337339Z TD_DISTRIBUTED=False 2024-08-07T18:05:01.6337606Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-07T18:05:01.6337922Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-07T18:05:01.6338219Z NV_NVPROF_VERSION=12.1.105-1 2024-08-07T18:05:01.6338492Z GITHUB_RETENTION_DAYS=90 2024-08-07T18:05:01.6338750Z OPENSSL_DIR=/opt/openssl 2024-08-07T18:05:01.6339016Z GITHUB_ACTION_REPOSITORY= 2024-08-07T18:05:01.6339871Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T18:05:01.6340626Z GITHUB_BASE_REF=gh/zdevito/267/base 2024-08-07T18:05:01.6340953Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-07T18:05:01.6341232Z CI=true 2024-08-07T18:05:01.6341458Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-07T18:05:01.6341772Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-07T18:05:01.6342057Z JOB_ID=28476695132 2024-08-07T18:05:01.6342296Z INSTALLED_PROTOBUF=yes 2024-08-07T18:05:01.6342569Z GITHUB_HEAD_REF=gh/zdevito/267/head 2024-08-07T18:05:01.6342866Z GITHUB_ACTION_REF= 2024-08-07T18:05:01.6343166Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-07T18:05:01.6343520Z TEST_SHOWLOCALS=False 2024-08-07T18:05:01.6343779Z GITHUB_WORKFLOW=pull 2024-08-07T18:05:01.6344042Z DEBIAN_FRONTEND=noninteractive 2024-08-07T18:05:01.6344632Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_656853b4-1594-4d68-a695-030c54287386 2024-08-07T18:05:01.6345231Z NO_TD=False 2024-08-07T18:05:01.6345468Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-07T18:05:01.6345752Z _=/usr/bin/env 2024-08-07T18:05:01.6346075Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-07T18:05:01.6459323Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-08-07T18:05:01.6460089Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-07T18:05:01.6460704Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-08-07T18:05:01.6461256Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-08-07T18:05:01.6461762Z + BUILD_DIR=build 2024-08-07T18:05:01.6462010Z + BUILD_RENAMED_DIR=build_renamed 2024-08-07T18:05:01.6462292Z + BUILD_BIN_DIR=build/bin 2024-08-07T18:05:01.6462545Z + SHARD_NUMBER=1 2024-08-07T18:05:01.6462842Z + NUM_TEST_SHARDS=5 2024-08-07T18:05:01.6463078Z + export VALGRIND=ON 2024-08-07T18:05:01.6463322Z + VALGRIND=ON 2024-08-07T18:05:01.6463640Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *clang9* ]] 2024-08-07T18:05:01.6463989Z + [[ 0 == \1 ]] 2024-08-07T18:05:01.6464208Z + [[ False == \1 ]] 2024-08-07T18:05:01.6464500Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *bazel* ]] 2024-08-07T18:05:01.6469430Z ++ realpath build/custom_test_artifacts 2024-08-07T18:05:01.6481285Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-07T18:05:01.6481770Z + [[ -n '' ]] 2024-08-07T18:05:01.6482016Z + echo 'Environment variables' 2024-08-07T18:05:01.6482311Z Environment variables 2024-08-07T18:05:01.6482560Z + env 2024-08-07T18:05:01.6491670Z INSTALLED_DB=yes 2024-08-07T18:05:01.6492028Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-07T18:05:01.6492403Z NVIDIA_VISIBLE_DEVICES=all 2024-08-07T18:05:01.6492734Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-07T18:05:01.6493278Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T18:05:01.6493851Z CONTINUE_THROUGH_ERROR=False 2024-08-07T18:05:01.6494459Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-07T18:05:01.6494951Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-07T18:05:01.6495490Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-07T18:05:01.6495868Z HOSTNAME=17250618960b 2024-08-07T18:05:01.6496415Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_656853b4-1594-4d68-a695-030c54287386 2024-08-07T18:05:01.6496996Z GITHUB_ACTION=__self 2024-08-07T18:05:01.6497340Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-07T18:05:01.6500267Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-07T18:05:01.6503190Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-07T18:05:01.6503559Z NV_NVTX_VERSION=12.1.105-1 2024-08-07T18:05:01.6503888Z GITHUB_RUN_NUMBER=234358 2024-08-07T18:05:01.6504142Z TEST_CONFIG=default 2024-08-07T18:05:01.6504404Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-07T18:05:01.6504783Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-07T18:05:01.6505158Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-07T18:05:01.6505506Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-07T18:05:01.6505816Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-07T18:05:01.6506160Z GITHUB_TRIGGERING_ACTOR=zdevito 2024-08-07T18:05:01.6506616Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-07T18:05:01.6507031Z GITHUB_REF_TYPE=branch 2024-08-07T18:05:01.6507294Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-07T18:05:01.6507565Z NCCL_VERSION=2.17.1-1 2024-08-07T18:05:01.6507853Z BASE_SHA=6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T18:05:01.6508186Z XLA_CUDA= 2024-08-07T18:05:01.6508401Z HUGGING_FACE_HUB_TOKEN= 2024-08-07T18:05:01.6508727Z *** 2024-08-07T18:05:01.6508950Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-07T18:05:01.6509242Z GITHUB_REPOSITORY_ID=65600975 2024-08-07T18:05:01.6509516Z GITHUB_ACTIONS=true 2024-08-07T18:05:01.6509762Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:05:01.6510087Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-07T18:05:01.6510482Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-07T18:05:01.6510839Z SHA1=016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T18:05:01.6511174Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-07T18:05:01.6511525Z GITHUB_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T18:05:01.6512030Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/131248/merge 2024-08-07T18:05:01.6512497Z UCC_HOME=/usr 2024-08-07T18:05:01.6512735Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-07T18:05:01.6513028Z VERBOSE_TEST_LOGS=False 2024-08-07T18:05:01.6513494Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-07T18:05:01.6513810Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-07T18:05:01.6514178Z GITHUB_REF=refs/pull/131248/merge 2024-08-07T18:05:01.6514580Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-07T18:05:01.6514890Z SHARD_NUMBER=1 2024-08-07T18:05:01.6515126Z GITHUB_REF_PROTECTED=false 2024-08-07T18:05:01.6515399Z HOME=/var/lib/jenkins 2024-08-07T18:05:01.6515690Z GITHUB_API_URL=https://api.github.com 2024-08-07T18:05:01.6516019Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-07T18:05:01.6516367Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-07T18:05:01.6516720Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-07T18:05:01.6516980Z CUDA_VERSION=12.1.1 2024-08-07T18:05:01.6517257Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-07T18:05:01.6517723Z NUM_TEST_SHARDS=5 2024-08-07T18:05:01.6517961Z UCX_HOME=/usr 2024-08-07T18:05:01.6518309Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-07T18:05:01.6519037Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_656853b4-1594-4d68-a695-030c54287386 2024-08-07T18:05:01.6519870Z JOB_NAME=linux-focal-cuda12.1-py3.10-gcc9-sm86 / test (default, 1, 5, amz2023.linux.g5.4xlarge.nvidia.gpu) 2024-08-07T18:05:01.6520684Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_656853b4-1594-4d68-a695-030c54287386 2024-08-07T18:05:01.6521427Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-07T18:05:01.6522008Z GITHUB_EVENT_NAME=pull_request 2024-08-07T18:05:01.6522340Z DASHBOARD_TAG= 2024-08-07T18:05:01.6522587Z GITHUB_RUN_ID=10288745067 2024-08-07T18:05:01.6522903Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-07T18:05:01.6523290Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-07T18:05:01.6523944Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_656853b4-1594-4d68-a695-030c54287386 2024-08-07T18:05:01.6524640Z GITHUB_ACTOR=zdevito 2024-08-07T18:05:01.6524908Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-07T18:05:01.6525202Z PR_NUMBER=131248 2024-08-07T18:05:01.6525440Z GITHUB_RUN_ATTEMPT=1 2024-08-07T18:05:01.6525684Z VALGRIND=ON 2024-08-07T18:05:01.6525919Z ANACONDA_PYTHON_VERSION=3.10 2024-08-07T18:05:01.6526300Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-07T18:05:01.6526642Z TERM=xterm 2024-08-07T18:05:01.6526877Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-07T18:05:01.6527175Z INSTALLED_VISION=yes 2024-08-07T18:05:01.6527483Z BRANCH=pull/131248 2024-08-07T18:05:01.6527742Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-07T18:05:01.6528041Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-07T18:05:01.6528364Z CUDA_PATH=/usr/local/cuda 2024-08-07T18:05:01.6528878Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-07T18:05:01.6529426Z GITHUB_SERVER_URL=https://github.com 2024-08-07T18:05:01.6529788Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-07T18:05:01.6530136Z REENABLED_ISSUES= 2024-08-07T18:05:01.6530361Z SHLVL=1 2024-08-07T18:05:01.6530571Z MAX_JOBS=14 2024-08-07T18:05:01.6530797Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-07T18:05:01.6531069Z NVARCH=x86_64 2024-08-07T18:05:01.6531305Z GITHUB_ACTOR_ID=370202 2024-08-07T18:05:01.6531631Z GITHUB_WORKFLOW_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T18:05:01.6532011Z GITHUB_REF_NAME=131248/merge 2024-08-07T18:05:01.6532314Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-07T18:05:01.6532752Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-07T18:05:01.6533170Z GITHUB_JOB=test 2024-08-07T18:05:01.6533432Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-07T18:05:01.6533827Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-07T18:05:01.6534205Z NO_TEST_TIMEOUT=False 2024-08-07T18:05:01.6534467Z TD_DISTRIBUTED=False 2024-08-07T18:05:01.6534741Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-07T18:05:01.6535070Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-07T18:05:01.6535492Z NV_NVPROF_VERSION=12.1.105-1 2024-08-07T18:05:01.6535769Z GITHUB_RETENTION_DAYS=90 2024-08-07T18:05:01.6536042Z OPENSSL_DIR=/opt/openssl 2024-08-07T18:05:01.6536320Z GITHUB_ACTION_REPOSITORY= 2024-08-07T18:05:01.6537041Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T18:05:01.6537804Z GITHUB_BASE_REF=gh/zdevito/267/base 2024-08-07T18:05:01.6538131Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-07T18:05:01.6538424Z CI=true 2024-08-07T18:05:01.6538712Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-07T18:05:01.6539024Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-07T18:05:01.6539301Z JOB_ID=28476695132 2024-08-07T18:05:01.6539647Z INSTALLED_PROTOBUF=yes 2024-08-07T18:05:01.6540061Z GITHUB_HEAD_REF=gh/zdevito/267/head 2024-08-07T18:05:01.6540352Z GITHUB_ACTION_REF= 2024-08-07T18:05:01.6540650Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-07T18:05:01.6541009Z TEST_SHOWLOCALS=False 2024-08-07T18:05:01.6541261Z GITHUB_WORKFLOW=pull 2024-08-07T18:05:01.6541528Z DEBIAN_FRONTEND=noninteractive 2024-08-07T18:05:01.6542116Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_656853b4-1594-4d68-a695-030c54287386 2024-08-07T18:05:01.6542698Z NO_TD=False 2024-08-07T18:05:01.6542942Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-07T18:05:01.6543213Z _=/usr/bin/env 2024-08-07T18:05:01.6543457Z + echo 'Testing pytorch' 2024-08-07T18:05:01.6543718Z Testing pytorch 2024-08-07T18:05:01.6543959Z + export LANG=C.UTF-8 2024-08-07T18:05:01.6544207Z + LANG=C.UTF-8 2024-08-07T18:05:01.6544437Z + PR_NUMBER=131248 2024-08-07T18:05:01.6544674Z + [[ default == \d\e\f\a\u\l\t ]] 2024-08-07T18:05:01.6544975Z + export CUDA_VISIBLE_DEVICES=0 2024-08-07T18:05:01.6545264Z + CUDA_VISIBLE_DEVICES=0 2024-08-07T18:05:01.6545531Z + export HIP_VISIBLE_DEVICES=0 2024-08-07T18:05:01.6545817Z + HIP_VISIBLE_DEVICES=0 2024-08-07T18:05:01.6546098Z + [[ default == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-07T18:05:01.6546401Z + [[ default == \s\l\o\w ]] 2024-08-07T18:05:01.6546775Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *slow-gradcheck* ]] 2024-08-07T18:05:01.6547222Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *cuda* ]] 2024-08-07T18:05:01.6547681Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-07T18:05:01.6548031Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-07T18:05:01.6548346Z + [[ default == *crossref* ]] 2024-08-07T18:05:01.6548675Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-07T18:05:01.6549081Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *xpu* ]] 2024-08-07T18:05:01.6551322Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-07T18:05:01.6551711Z + pip_install --user ninja==1.10.2 2024-08-07T18:05:01.6552079Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-07T18:05:03.1045802Z Collecting ninja==1.10.2 2024-08-07T18:05:03.1206920Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-07T18:05:03.1776519Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-07T18:05:04.0754064Z Installing collected packages: ninja 2024-08-07T18:05:04.0836635Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-07T18:05:04.0837528Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-07T18:05:04.0880223Z Successfully installed ninja-1.10.2 2024-08-07T18:05:04.1627942Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T18:05:04.1629409Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T18:05:04.1630690Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-07T18:05:04.1631092Z + install_tlparse 2024-08-07T18:05:04.1631389Z + pip_install --user tlparse==0.3.7 2024-08-07T18:05:04.1631760Z + pip install --progress-bar off --user tlparse==0.3.7 2024-08-07T18:05:04.5609376Z Collecting tlparse==0.3.7 2024-08-07T18:05:04.5764676Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-08-07T18:05:04.5868705Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-07T18:05:05.4603485Z Installing collected packages: tlparse 2024-08-07T18:05:05.5042830Z Successfully installed tlparse-0.3.7 2024-08-07T18:05:05.5786976Z ++ python -m site --user-base 2024-08-07T18:05:05.5969093Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T18:05:05.5970419Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *asan* ]] 2024-08-07T18:05:05.5970855Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-debug* ]] 2024-08-07T18:05:05.5971292Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 != *-bazel-* ]] 2024-08-07T18:05:05.5971867Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass' 2024-08-07T18:05:05.5972553Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9-sm86. Expect the assertion to pass 2024-08-07T18:05:05.5975888Z + cd test 2024-08-07T18:05:05.5976473Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-07T18:05:07.2652532Z + [[ default == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-07T18:05:07.2652924Z + [[ default == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-07T18:05:07.2656405Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-07T18:05:07.2656726Z + [[ default == *dynamo_eager* ]] 2024-08-07T18:05:07.2657113Z + [[ default == *aot_eager* ]] 2024-08-07T18:05:07.2657420Z + [[ default == *aot_inductor* ]] 2024-08-07T18:05:07.2657744Z + [[ default == *inductor* ]] 2024-08-07T18:05:07.2658077Z + [[ default == *dynamic* ]] 2024-08-07T18:05:07.2658353Z + [[ default == *cpu* ]] 2024-08-07T18:05:07.2658701Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-07T18:05:07.2688758Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-07T18:05:07.2689304Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *-bazel-* ]] 2024-08-07T18:05:07.2691750Z + cd test 2024-08-07T18:05:07.2692152Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-07T18:05:08.7618219Z PyTorch built with: 2024-08-07T18:05:08.7618519Z - GCC 9.4 2024-08-07T18:05:08.7618738Z - C++ Version: 201703 2024-08-07T18:05:08.7619300Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-07T18:05:08.7620080Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-07T18:05:08.7620519Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-07T18:05:08.7620867Z - LAPACK is enabled (usually provided by MKL) 2024-08-07T18:05:08.7621202Z - NNPACK is enabled 2024-08-07T18:05:08.7621472Z - CPU capability usage: AVX2 2024-08-07T18:05:08.7621760Z - CUDA Runtime 12.1 2024-08-07T18:05:08.7622105Z - NVCC architecture flags: -gencode;arch=compute_86,code=sm_86 2024-08-07T18:05:08.7622516Z - CuDNN 90.1 (built against CUDA 12.4) 2024-08-07T18:05:08.7622827Z - Magma 2.6.1 2024-08-07T18:05:08.7627197Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-07T18:05:08.7633287Z 2024-08-07T18:05:09.0859895Z + cd test 2024-08-07T18:05:09.0860837Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-07T18:05:10.4333876Z ATen/Parallel: 2024-08-07T18:05:10.4334207Z at::get_num_threads() : 8 2024-08-07T18:05:10.4334529Z at::get_num_interop_threads() : 16 2024-08-07T18:05:10.4334846Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-07T18:05:10.4335149Z omp_get_max_threads() : 8 2024-08-07T18:05:10.4335675Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-07T18:05:10.4336232Z mkl_get_max_threads() : 8 2024-08-07T18:05:10.4336606Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-07T18:05:10.4337033Z std::thread::hardware_concurrency() : 16 2024-08-07T18:05:10.4337349Z Environment variables: 2024-08-07T18:05:10.4337614Z OMP_NUM_THREADS : [not set] 2024-08-07T18:05:10.4337883Z MKL_NUM_THREADS : [not set] 2024-08-07T18:05:10.4338165Z ATen parallel backend: OpenMP 2024-08-07T18:05:10.4338348Z 2024-08-07T18:05:10.7133806Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *aarch64* ]] 2024-08-07T18:05:10.7134219Z + [[ default == *backward* ]] 2024-08-07T18:05:10.7134495Z + [[ default == *xla* ]] 2024-08-07T18:05:10.7134776Z + [[ default == *executorch* ]] 2024-08-07T18:05:10.7135053Z + [[ default == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-07T18:05:10.7135417Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *libtorch* ]] 2024-08-07T18:05:10.7135804Z + [[ default == distributed ]] 2024-08-07T18:05:10.7136094Z + [[ default == *inductor_distributed* ]] 2024-08-07T18:05:10.7136501Z + [[ default == *inductor-halide* ]] 2024-08-07T18:05:10.7136884Z + [[ default == *inductor-micro-benchmark* ]] 2024-08-07T18:05:10.7137215Z + [[ default == *huggingface* ]] 2024-08-07T18:05:10.7137503Z + [[ default == *timm* ]] 2024-08-07T18:05:10.7137773Z + [[ default == *torchbench* ]] 2024-08-07T18:05:10.7138108Z + [[ default == *inductor_cpp_wrapper_abi_compatible* ]] 2024-08-07T18:05:10.7138464Z + [[ default == *inductor* ]] 2024-08-07T18:05:10.7138748Z + [[ default == *dynamo* ]] 2024-08-07T18:05:10.7139070Z + [[ linux-focal-cuda12.1-py3.10-gcc9-sm86 == *rocm* ]] 2024-08-07T18:05:10.7139413Z + [[ 1 == 1 ]] 2024-08-07T18:05:10.7139646Z + [[ 5 -gt 1 ]] 2024-08-07T18:05:10.7139980Z + test_without_numpy 2024-08-07T18:05:10.7140248Z ++ dirname .ci/pytorch/test.sh 2024-08-07T18:05:10.7152554Z + pushd .ci/pytorch 2024-08-07T18:05:10.7152821Z ~/workspace/.ci/pytorch ~/workspace 2024-08-07T18:05:10.7153581Z + python -c 'import sys;sys.path.insert(0, '\''fake_numpy'\'');from unittest import TestCase;import torch;x=torch.randn(3,3);TestCase().assertRaises(RuntimeError, lambda: x.numpy())' 2024-08-07T18:05:11.6193776Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_subclasses/functional_tensor.py:271: UserWarning: Failed to initialize NumPy: Sorry PyTorch, but our NumPy is in the other folder (Triggered internally at /var/lib/jenkins/workspace/torch/csrc/utils/tensor_numpy.cpp:84.) 2024-08-07T18:05:11.6195112Z cpu = _conversion_method_template(device=torch.device("cpu")) 2024-08-07T18:05:12.2776737Z + python -c 'import sys;sys.path.insert(0, '\''fake_numpy'\'');import torch;print(torch.tensor([torch.tensor(0.), torch.tensor(1.)]))' 2024-08-07T18:05:13.1567467Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_subclasses/functional_tensor.py:271: UserWarning: Failed to initialize NumPy: Sorry PyTorch, but our NumPy is in the other folder (Triggered internally at /var/lib/jenkins/workspace/torch/csrc/utils/tensor_numpy.cpp:84.) 2024-08-07T18:05:13.1568757Z cpu = _conversion_method_template(device=torch.device("cpu")) 2024-08-07T18:05:13.5436009Z tensor([0., 1.]) 2024-08-07T18:05:13.8106103Z + [[ default == *dynamo* ]] 2024-08-07T18:05:13.8106400Z + popd 2024-08-07T18:05:13.8106611Z ~/workspace 2024-08-07T18:05:13.8106835Z + install_torchvision 2024-08-07T18:05:13.8107080Z + local orig_preload 2024-08-07T18:05:13.8107325Z + local commit 2024-08-07T18:05:13.8109861Z ++ get_pinned_commit vision 2024-08-07T18:05:13.8110469Z ++ cat .github/ci_commit_pins/vision.txt 2024-08-07T18:05:13.8131364Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-08-07T18:05:13.8131719Z + orig_preload= 2024-08-07T18:05:13.8131970Z + '[' -n '' ']' 2024-08-07T18:05:13.8132577Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-07T18:05:13.8133491Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-07T18:05:14.1630621Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-07T18:05:14.1635707Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-fa0ep9ho 2024-08-07T18:05:14.1664105Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-fa0ep9ho 2024-08-07T18:05:15.6448838Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-08-07T18:05:15.6477179Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-08-07T18:05:16.9554293Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-08-07T18:05:17.2435290Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-08-07T18:05:19.7328530Z Preparing metadata (setup.py) ... [?25l- \ done 2024-08-07T18:05:19.7374078Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.21.2) 2024-08-07T18:05:19.7376648Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+git016588f) 2024-08-07T18:05:19.7381985Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-08-07T18:05:19.7610620Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-08-07T18:05:19.7615412Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-08-07T18:05:19.7617801Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-08-07T18:05:19.7621106Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-08-07T18:05:19.7623835Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.6.1) 2024-08-07T18:05:19.7633941Z Requirement already satisfied: sympy>=1.13.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.13.1) 2024-08-07T18:05:19.7664928Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy>=1.13.0->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-08-07T18:05:19.8550516Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-08-07T18:05:19.8765844Z Building wheels for collected packages: torchvision 2024-08-07T18:06:38.0744577Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - done 2024-08-07T18:06:38.0776892Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2116115 sha256=aa6013fd5758061157e0f4563f6ea05f018094dd5ca9461775f4b1e984412111 2024-08-07T18:06:38.0780730Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-08-07T18:06:38.0815279Z Successfully built torchvision 2024-08-07T18:06:38.7833755Z Installing collected packages: torchvision 2024-08-07T18:06:39.2034507Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-08-07T18:06:39.3409004Z + '[' -n '' ']' 2024-08-07T18:06:39.3409391Z + test_python_shard 1 2024-08-07T18:06:39.3409687Z + [[ -z 5 ]] 2024-08-07T18:06:39.3410160Z + python test/run_test.py --exclude-jit-executor --exclude-distributed-tests --shard 1 5 --verbose 2024-08-07T18:06:39.4378693Z /var/lib/jenkins/workspace/test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-07T18:06:39.4379624Z import pkg_resources 2024-08-07T18:06:42.9799407Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-07T18:06:43.0436920Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-07T18:06:43.0938958Z Ignoring disabled issues: [''] 2024-08-07T18:06:43.1050141Z Found test times from artifacts 2024-08-07T18:06:43.1466776Z Found test times from artifacts 2024-08-07T18:06:43.1479455Z Running 25% of tests based on TD 2024-08-07T18:06:43.1768197Z Running parallel tests on 3 processes 2024-08-07T18:06:43.1769629Z Name: tests to run (est. time: 65.23min) 2024-08-07T18:06:43.1770227Z Serial tests (6): 2024-08-07T18:06:43.1770674Z test_torch 1/1 2024-08-07T18:06:43.1771133Z test_fake_tensor 1/1 2024-08-07T18:06:43.1771688Z test_cpp_api_parity 1/1 2024-08-07T18:06:43.1772018Z test_utils 1/1 2024-08-07T18:06:43.1772267Z inductor/test_max_autotune 1/1 2024-08-07T18:06:43.1772570Z test_show_pickle 1/1 2024-08-07T18:06:43.1772830Z Parallel tests (22): 2024-08-07T18:06:43.1773102Z functorch/test_ops 2/6 2024-08-07T18:06:43.1773372Z test_decomp 1/14 2024-08-07T18:06:43.1773616Z test_decomp 5/14 2024-08-07T18:06:43.1773862Z test_decomp 6/14 2024-08-07T18:06:43.1774140Z inductor/test_torchinductor_opinfo 2/13 2024-08-07T18:06:43.1774493Z inductor/test_torchinductor_opinfo 3/13 2024-08-07T18:06:43.1774849Z inductor/test_torchinductor_opinfo 4/13 2024-08-07T18:06:43.1775184Z inductor/test_aot_inductor 3/16 2024-08-07T18:06:43.1775489Z inductor/test_aot_inductor 5/16 2024-08-07T18:06:43.1775798Z inductor/test_aot_inductor 6/16 2024-08-07T18:06:43.1776105Z inductor/test_torchinductor 2/5 2024-08-07T18:06:43.1776409Z inductor/test_torchinductor 3/5 2024-08-07T18:06:43.1776722Z inductor/test_torchinductor 4/5 2024-08-07T18:06:43.1777039Z inductor/test_cpu_cpp_wrapper 1/1 2024-08-07T18:06:43.1777385Z inductor/test_torchinductor_dynamic_shapes 5/5 2024-08-07T18:06:43.1777746Z inductor/test_cuda_cpp_wrapper 2/4 2024-08-07T18:06:43.1778076Z inductor/test_cuda_cpp_wrapper 4/4 2024-08-07T18:06:43.1778400Z inductor/test_compiled_optimizers 4/4 2024-08-07T18:06:43.1778719Z export/test_tools 1/1 2024-08-07T18:06:43.1779016Z dynamo/test_inline_inbuilt_nn_modules 1/1 2024-08-07T18:06:43.1779803Z inductor/test_debug_trace 1/1 2024-08-07T18:06:43.1780126Z inductor/test_mkldnn_pattern_matcher 1/1 2024-08-07T18:06:43.1780482Z Name: excluded (est. time: 20.31min) 2024-08-07T18:06:43.1780773Z Serial tests (33): 2024-08-07T18:06:43.1781042Z inductor/test_flex_attention 1/2 2024-08-07T18:06:43.1781342Z test_fx 1/1 2024-08-07T18:06:43.1781580Z test_reductions 1/1 2024-08-07T18:06:43.1781853Z test_multiprocessing 1/1 2024-08-07T18:06:43.1782147Z test_tensor_creation_ops 1/1 2024-08-07T18:06:43.1782453Z distributions/test_distributions 1/1 2024-08-07T18:06:43.1782791Z inductor/test_cutlass_backend 1/1 2024-08-07T18:06:43.1783127Z inductor/test_distributed_patterns 1/1 2024-08-07T18:06:43.1783617Z test_multiprocessing_spawn 1/1 2024-08-07T18:06:43.1783930Z test_spectral_ops 1/1 2024-08-07T18:06:43.1784233Z functorch/test_memory_efficient_fusion 1/1 2024-08-07T18:06:43.1784607Z test_cpp_extensions_open_device_registration 1/1 2024-08-07T18:06:43.1784956Z nn/test_pooling 1/1 2024-08-07T18:06:43.1785216Z test_sort_and_select 1/1 2024-08-07T18:06:43.1785488Z test_mobile_optimizer 1/1 2024-08-07T18:06:43.1785765Z test_overrides 1/1 2024-08-07T18:06:43.1786012Z test_cuda_trace 1/1 2024-08-07T18:06:43.1786262Z test_python_dispatch 1/1 2024-08-07T18:06:43.1786546Z test_namedtuple_return_api 1/1 2024-08-07T18:06:43.1786838Z test_autocast 1/1 2024-08-07T18:06:43.1787076Z test_native_mha 1/1 2024-08-07T18:06:43.1787344Z test_cpp_extensions_aot_ninja 1/1 2024-08-07T18:06:43.1787645Z test_autograd_fallback 1/1 2024-08-07T18:06:43.1787941Z test_cpp_extensions_aot_no_ninja 1/1 2024-08-07T18:06:43.1788257Z test_cuda_nvml_based_avail 1/1 2024-08-07T18:06:43.1788547Z test_jit_disabled 1/1 2024-08-07T18:06:43.1788813Z test_cuda_primary_ctx 1/1 2024-08-07T18:06:43.1789104Z test_cpp_extensions_mtia_backend 1/1 2024-08-07T18:06:43.1789431Z test_cpp_extensions_stream_and_event 1/1 2024-08-07T18:06:43.1789765Z test_ci_sanity_check_fail 1/1 2024-08-07T18:06:43.1790063Z test_autoload_enable 1/1 2024-08-07T18:06:43.1790320Z doctests 1/1 2024-08-07T18:06:43.1790562Z test_autoload_disable 1/1 2024-08-07T18:06:43.1790842Z Parallel tests (4): 2024-08-07T18:06:43.1791101Z inductor/test_unbacked_symints 1/1 2024-08-07T18:06:43.1791412Z inductor/test_xpu_basic 1/1 2024-08-07T18:06:43.1791686Z xpu/test_gemm 1/1 2024-08-07T18:06:43.1791928Z optim/test_optim 1/1 2024-08-07T18:06:43.1792299Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-07T18:06:43.1838137Z Running test_torch 1/1 ... [2024-08-07 18:06:43.183369] 2024-08-07T18:06:43.1841611Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_torch.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:06:43.183766] 2024-08-07T18:06:57.6041995Z 2024-08-07T18:06:57.6042851Z PRINTING LOG FILE of test_torch 1/1 (test/test-reports/test_torch_1.1_561344a28600abd3_.log) 2024-08-07T18:06:57.6043572Z Test results will be stored in test-reports/python-pytest/test_torch/test_torch-d27faa9db46fde99.xml 2024-08-07T18:06:57.6044586Z ============================= test session starts ============================== 2024-08-07T18:06:57.6045233Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.5.0 -- /opt/conda/envs/py_3.10/bin/python 2024-08-07T18:06:57.6045722Z cachedir: .pytest_cache 2024-08-07T18:06:57.6046310Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-08-07T18:06:57.6047117Z rootdir: /var/lib/jenkins/workspace 2024-08-07T18:06:57.6047490Z configfile: pytest.ini 2024-08-07T18:06:57.6048099Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-08-07T18:06:57.6048717Z collecting ... collected 1043 items 2024-08-07T18:06:57.6049600Z stepcurrent: Cannot find last run test, not skipping 2024-08-07T18:06:57.6332167Z Running 1043 items in this shard: test/test_torch.py::TestBasicVitalSigns::test_basic_vitals, test/test_torch.py::TestBasicVitalSigns::test_basic_vitals_read_write, test/test_torch.py::TestBasicVitalSigns::test_dataloader_vitals, test/test_torch.py::TestTorch::test_RNGState, test/test_torch.py::TestTorch::test_RNGStateAliasing, test/test_torch.py::TestTorch::test_RNG_after_pickle, test/test_torch.py::TestTorch::test_Size, test/test_torch.py::TestTorch::test_Size_iter, test/test_torch.py::TestTorch::test_Size_scalar, test/test_torch.py::TestTorch::test_add_meta_scalar, test/test_torch.py::TestTorch::test_allow_tensor_metadata_change, test/test_torch.py::TestTorch::test_apply, test/test_torch.py::TestTorch::test_as_subclass, test/test_torch.py::TestTorch::test_assert_async, test/test_torch.py::TestTorch::test_backward_hooks_traverse, test/test_torch.py::TestTorch::test_batch_norm_cpu_inference, test/test_torch.py::TestTorch::test_bf16_supported_on_cpu, test/test_torch.py::TestTorch::test_bmm_multithreaded, test/test_torch.py::TestTorch::test_boxMullerState, test/test_torch.py::TestTorch::test_c10_feature_enabled, test/test_torch.py::TestTorch::test_cat_neg_dim, test/test_torch.py::TestTorch::test_check, test/test_torch.py::TestTorch::test_chunk_neg_dim, test/test_torch.py::TestTorch::test_conj_neg_tolist, test/test_torch.py::TestTorch::test_contains, test/test_torch.py::TestTorch::test_copy_broadcast, test/test_torch.py::TestTorch::test_copy_dtypes, test/test_torch.py::TestTorch::test_copy_float16, test/test_torch.py::TestTorch::test_copy_many_to_one, test/test_torch.py::TestTorch::test_copy_transpose, test/test_torch.py::TestTorch::test_cuda_not_built, test/test_torch.py::TestTorch::test_cummax_neg_dim, test/test_torch.py::TestTorch::test_cummin_neg_dim, test/test_torch.py::TestTorch::test_cumprod_neg_dim, test/test_torch.py::TestTorch::test_cumsum_neg_dim, test/test_torch.py::TestTorch::test_cxx_flags, test/test_torch.py::TestTorch::test_data_ptr_of_empty_tensor_with_storage, test/test_torch.py::TestTorch::test_data_ptr_of_empty_view_with_storage, test/test_torch.py::TestTorch::test_deepcopy_gradient, test/test_torch.py::TestTorch::test_deepcopy_parameter, test/test_torch.py::TestTorch::test_deterministic_fill_uninitialized_memory, test/test_torch.py::TestTorch::test_deterministic_flag, test/test_torch.py::TestTorch::test_device, test/test_torch.py::TestTorch::test_dim_order, test/test_torch.py::TestTorch::test_dir, test/test_torch.py::TestTorch::test_doc, test/test_torch.py::TestTorch::test_doc_template, test/test_torch.py::TestTorch::test_dot_data_use, test/test_torch.py::TestTorch::test_dtype_is_signed, test/test_torch.py::TestTorch::test_element_size, test/test_torch.py::TestTorch::test_empty_meta, test/test_torch.py::TestTorch::test_empty_storage_view, test/test_torch.py::TestTorch::test_equal, test/test_torch.py::TestTorch::test_error_msg_type_translation, test/test_torch.py::TestTorch::test_fill_diagonal, test/test_torch.py::TestTorch::test_format_scalar_meta, test/test_torch.py::TestTorch::test_from_buffer, test/test_torch.py::TestTorch::test_from_file, test/test_torch.py::TestTorch::test_gather_neg_dim, test/test_torch.py::TestTorch::test_generator_cpu, test/test_torch.py::TestTorch::test_get_cpu_capability, test/test_torch.py::TestTorch::test_has_internal_overlap, test/test_torch.py::TestTorch::test_has_storage, test/test_torch.py::TestTorch::test_index_add, test/test_torch.py::TestTorch::test_index_add_all_dtypes, test/test_torch.py::TestTorch::test_index_add_cornercase, test/test_torch.py::TestTorch::test_index_add_correctness, test/test_torch.py::TestTorch::test_index_add_neg_dim, test/test_torch.py::TestTorch::test_index_copy_neg_dim, test/test_torch.py::TestTorch::test_index_fill_neg_dim, test/test_torch.py::TestTorch::test_index_select_neg_dim, test/test_torch.py::TestTorch::test_invalid_arg_error_handling, test/test_torch.py::TestTorch::test_invalid_generator_raises, test/test_torch.py::TestTorch::test_is_nonzero, test/test_torch.py::TestTorch::test_is_same_size, test/test_torch.py::TestTorch::test_iter, test/test_torch.py::TestTorch::test_kthvalue_neg_dim, test/test_torch.py::TestTorch::test_linspace_logspace, test/test_torch.py::TestTorch::test_logcumsumexp_neg_dim, test/test_torch.py::TestTorch::test_manual_seed, test/test_torch.py::TestTorch::test_map, test/test_torch.py::TestTorch::test_map2, test/test_torch.py::TestTorch::test_max_neg_dim, test/test_torch.py::TestTorch::test_mean_neg_dim, test/test_torch.py::TestTorch::test_median_neg_dim, test/test_torch.py::TestTorch::test_memory_format, test/test_torch.py::TestTorch::test_memory_format_contiguous_returns_same_tensor_if_already_satisfies, test/test_torch.py::TestTorch::test_memory_format_empty, test/test_torch.py::TestTorch::test_min_neg_dim, test/test_torch.py::TestTorch::test_mode_neg_dim, test/test_torch.py::TestTorch::test_multinomial_invalid_probs, test/test_torch.py::TestTorch::test_nanmedian_neg_dim, test/test_torch.py::TestTorch::test_narrow_neg_dim, test/test_torch.py::TestTorch::test_nbytes, test/test_torch.py::TestTorch::test_ndim, test/test_torch.py::TestTorch::test_new, test/test_torch.py::TestTorch::test_newaxis_numpy_comparison, test/test_torch.py::TestTorch::test_newindex, test/test_torch.py::TestTorch::test_no_cuda_monkeypatch, test/test_torch.py::TestTorch::test_norm_neg_dim, test/test_torch.py::TestTorch::test_normal_shape, test/test_torch.py::TestTorch::test_numel, test/test_torch.py::TestTorch::test_parallel_info, test/test_torch.py::TestTorch::test_parsing_double, test/test_torch.py::TestTorch::test_parsing_int64, test/test_torch.py::TestTorch::test_parsing_intlist, test/test_torch.py::TestTorch::test_permute, test/test_torch.py::TestTorch::test_pickle, test/test_torch.py::TestTorch::test_pickle_dtype, test/test_torch.py::TestTorch::test_pickle_function, test/test_torch.py::TestTorch::test_pickle_generator, test/test_torch.py::TestTorch::test_pickle_parameter, test/test_torch.py::TestTorch::test_pickle_parameter_no_requires_grad, test/test_torch.py::TestTorch::test_pickle_size, test/test_torch.py::TestTorch::test_pin_memory, test/test_torch.py::TestTorch::test_print, test/test_torch.py::TestTorch::test_prod_neg_dim, test/test_torch.py::TestTorch::test_pyobj_preserved, test/test_torch.py::TestTorch::test_qengine, test/test_torch.py::TestTorch::test_renorm_neg_dim, test/test_torch.py::TestTorch::test_resizable, test/test_torch.py::TestTorch::test_reversed, test/test_torch.py::TestTorch::test_scatter_neg_dim, test/test_torch.py::TestTorch::test_select_neg_dim, test/test_torch.py::TestTorch::test_set_flush_denormal, test/test_torch.py::TestTorch::test_setting_real_imag_to_a_number, test/test_torch.py::TestTorch::test_show_config, test/test_torch.py::TestTorch::test_size_neg_dim, test/test_torch.py::TestTorch::test_size_stride, test/test_torch.py::TestTorch::test_sizeof, test/test_torch.py::TestTorch::test_slice, test/test_torch.py::TestTorch::test_slow_test, test/test_torch.py::TestTorch::test_sobolengine_bounds, test/test_torch.py::TestTorch::test_sobolengine_bounds_scrambled, test/test_torch.py::TestTorch::test_sobolengine_continuing, test/test_torch.py::TestTorch::test_sobolengine_continuing_scrambled, test/test_torch.py::TestTorch::test_sobolengine_default_dtype, test/test_torch.py::TestTorch::test_sobolengine_distribution, test/test_torch.py::TestTorch::test_sobolengine_distribution_scrambled, test/test_torch.py::TestTorch::test_sobolengine_draw, test/test_torch.py::TestTorch::test_sobolengine_draw_base2, test/test_torch.py::TestTorch::test_sobolengine_draw_base2_scrambled, test/test_torch.py::TestTorch::test_sobolengine_draw_scrambled, test/test_torch.py::TestTorch::test_sobolengine_fast_forward, test/test_torch.py::TestTorch::test_sobolengine_fast_forward_scrambled, test/test_torch.py::TestTorch::test_sobolengine_first_point, test/test_torch.py::TestTorch::test_sobolengine_high_dim, test/test_torch.py::TestTorch::test_sobolengine_raise, test/test_torch.py::TestTorch::test_sobolengine_reset, test/test_torch.py::TestTorch::test_sobolengine_reset_scrambled, test/test_torch.py::TestTorch::test_sort_neg_dim, test/test_torch.py::TestTorch::test_split_neg_dim, test/test_torch.py::TestTorch::test_split_with_sizes_copy_out, test/test_torch.py::TestTorch::test_squeeze_neg_dim, test/test_torch.py::TestTorch::test_std_neg_dim, test/test_torch.py::TestTorch::test_storage_base_init, test/test_torch.py::TestTorch::test_storage_base_new, test/test_torch.py::TestTorch::test_storage_byteswap, test/test_torch.py::TestTorch::test_storage_casts, test/test_torch.py::TestTorch::test_storage_cycle_via_dict, test/test_torch.py::TestTorch::test_storage_cycle_via_slots, test/test_torch.py::TestTorch::test_storage_dead_weak_ref, test/test_torch.py::TestTorch::test_storage_dealloc, test/test_torch.py::TestTorch::test_storage_dealloc_resurrected, test/test_torch.py::TestTorch::test_storage_dealloc_subclass_resurrected, test/test_torch.py::TestTorch::test_storage_dealloc_subclass_zombie, test/test_torch.py::TestTorch::test_storage_dict_dealloc, test/test_torch.py::TestTorch::test_storage_error, test/test_torch.py::TestTorch::test_storage_error_no_attribute, test/test_torch.py::TestTorch::test_storage_finalizer_dealloc, test/test_torch.py::TestTorch::test_storage_fix_weakref_no_leak, test/test_torch.py::TestTorch::test_storage_from_tensor_dealloc, test/test_torch.py::TestTorch::test_storage_from_tensor_dealloc_resurrected, test/test_torch.py::TestTorch::test_storage_from_tensor_dealloc_zombie, test/test_torch.py::TestTorch::test_storage_preserve_nonhermetic_in_hermetic_context, test/test_torch.py::TestTorch::test_storage_resurrected_weak_ref, test/test_torch.py::TestTorch::test_storage_slot_dealloc, test/test_torch.py::TestTorch::test_storage_weakref_dealloc, test/test_torch.py::TestTorch::test_structseq_repr, test/test_torch.py::TestTorch::test_subclass_preserved, test/test_torch.py::TestTorch::test_subclass_tensors, test/test_torch.py::TestTorch::test_sum_neg_dim, test/test_torch.py::TestTorch::test_swap_basic, test/test_torch.py::TestTorch::test_swap_fail_slots, test/test_torch.py::TestTorch::test_t_not_2d_error, test/test_torch.py::TestTorch::test_tensor_base_init, test/test_torch.py::TestTorch::test_tensor_base_new, test/test_torch.py::TestTorch::test_tensor_ctor_scalar, test/test_torch.py::TestTorch::test_tensor_cycle_via_dict, test/test_torch.py::TestTorch::test_tensor_cycle_via_slots, test/test_torch.py::TestTorch::test_tensor_dead_weak_ref, test/test_torch.py::TestTorch::test_tensor_dict_dealloc, test/test_torch.py::TestTorch::test_tensor_finalizer_dealloc, test/test_torch.py::TestTorch::test_tensor_fix_weakref_no_leak, test/test_torch.py::TestTorch::test_tensor_resurrected_weak_ref, test/test_torch.py::TestTorch::test_tensor_set, test/test_torch.py::TestTorch::test_tensor_set_errors, test/test_torch.py::TestTorch::test_tensor_slot_dealloc, test/test_torch.py::TestTorch::test_tensor_weakref_dealloc, test/test_torch.py::TestTorch::test_tensor_where_scalar, test/test_torch.py::TestTorch::test_tensoriterator_output_setup, test/test_torch.py::TestTorch::test_terminate_handler_on_crash, test/test_torch.py::TestTorch::test_to, test/test_torch.py::TestTorch::test_to_with_tensor, test/test_torch.py::TestTorch::test_topk_neg_dim, test/test_torch.py::TestTorch::test_torch_from_file, test/test_torch.py::TestTorch::test_transpose_neg_dim, test/test_torch.py::TestTorch::test_type, test/test_torch.py::TestTorch::test_type_alias, test/test_torch.py::TestTorch::test_type_conversion_via_dtype_name, test/test_torch.py::TestTorch::test_typed_storage_deprecation_warning, test/test_torch.py::TestTorch::test_typed_storage_internal_no_warning, test/test_torch.py::TestTorch::test_unbind_neg_dim, test/test_torch.py::TestTorch::test_unflatten, test/test_torch.py::TestTorch::test_unfold_neg_dim, test/test_torch.py::TestTorch::test_unsqueeze_neg_dim, test/test_torch.py::TestTorch::test_upsample_nearest1d_meta, test/test_torch.py::TestTorch::test_upsample_nearest2d_meta, test/test_torch.py::TestTorch::test_var_neg_dim, test/test_torch.py::TestTorch::test_warn_types, test/test_torch.py::TestTorch::test_wildcard_import, test/test_torch.py::TestVitalSignsCudaCUDA::test_cuda_vitals_gpu_only_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_assertRaisesRegex_ignore_msg_non_native_device_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_edge_cases_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_edge_cases_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_edge_cases_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_p_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_p_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_p_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bfloat16_neg_abs_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bool_tensor_value_change_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_add_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_addcdiv_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_addcmul_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_atan2_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_copy_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_dist_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_div_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_eq_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_fmod_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_ge_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_gt_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_le_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_lerp_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_lt_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_map2_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_map_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_masked_fill_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_masked_scatter_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_masked_select_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_max_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_min_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_mul_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_ne_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_pow_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_remainder_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_sub_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_kstest_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_no_inf_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_no_inf_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_cuda_backward_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_euclidean_large_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_grad_p_lt_1_no_nan_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_large_batch_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_large_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_non_contiguous_batch_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_non_contiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_norm_batch_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_norm_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_same_inputs_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_check_tensor_all_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_check_tensor_internal_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_clone_all_dtypes_and_devices_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_clone_not_memory_dense_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_clone_zero_stride_dim_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_complex_half_experimental_warning_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_constants_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_conv_transposed_backward_agnostic_to_memory_format_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_conv_transposed_large_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_complex32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_all_dtypes_and_devices_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_math_view_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_mem_overlap_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_transpose_math_view_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_transpose_math_view_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_transpose_math_view_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_corrcoef_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_corrcoef_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_corrcoef_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cov_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cpp_warnings_have_python_context_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cublas_config_nondeterministic_alert_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cummax_cummin_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cummax_discontiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cummin_discontiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cumprod_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cumsum_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deepcopy_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deepcopy_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deepcopy_scalar_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deepcopy_scalar_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_interpolate_bilinear_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_replication_pad2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_device_guard_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_dim_function_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_discontiguous_out_cumsum_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_dist_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_dtypetensor_warnings_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_errors_index_copy_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_expected_failure_xla_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_kstest_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_kstest_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_kstest_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_kstest_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_no_zero_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_no_zero_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gather_backward_deterministic_path_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gather_backward_one_dim_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_kstest_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scale_will_not_overflow_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaler_deprecated_warning_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaler_pass_itself_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_accumulation_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach0_fused0_AdamW_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach0_fused0_Adam_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach0_fused0_SGD_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach2_fused_True_AdamW_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach2_fused_True_Adam_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach2_fused_True_SGD_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach_True_fused1_AdamW_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach_True_fused1_Adam_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach_True_fused1_SGD_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_clipping_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_clipping_separate_unscale_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_multiple_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_penalty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_state_dict_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_unscale_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_unscale_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_unscale_sparse_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_update_scale_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_all_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_all_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_all_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_extreme_cases_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_extreme_cases_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_extreme_cases_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_spacing_list_length_error_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_spacing_list_length_error_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_spacing_list_length_error_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_type_promotion_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_hook_remove_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_add_deterministic_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_add_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_deterministic_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_put_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_put_non_accumulate_deterministic_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_invalid_shapes_grid_sampler_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_is_set_to_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_is_signed_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_complex32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_large_cumprod_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_large_cumsum_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_log_normal_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_log_normal_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_log_normal_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_log_normal_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_logcumsumexp_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lognormal_kstest_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_bool_tensor_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_bfloat16_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_bfloat16_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_bool_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_bool_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_complex128_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_complex128_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_complex64_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_complex64_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float16_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float16_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float32_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float32_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float64_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float64_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int16_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int16_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int32_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int32_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int64_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int64_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int8_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int8_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_uint8_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_uint8_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_bool_tensor_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_inplace_noncontiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_large_tensor_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_discontiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_clone_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_consistency_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_cpu_and_cuda_ops_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_empty_like_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_factory_like_functions_preserve_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_operators_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_preserved_after_permute_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_propagation_rules_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_to_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_type_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_type_shortcuts_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_module_share_memory_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cpu_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cpu_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cpu_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_deterministic_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_deterministic_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_deterministic_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_device_constrain_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_empty_w_replacement_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_empty_wo_replacement_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_gpu_device_constrain_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_rng_state_advance_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_narrow_copy_non_contiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_narrow_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_AdaptiveAvgPool2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_AdaptiveAvgPool3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_AdaptiveMaxPool2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_AvgPool3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_CTCLoss_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_EmbeddingBag_max_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_FractionalMaxPool2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_FractionalMaxPool3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxPool3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool1d_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool1d_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool1d_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool2d_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool2d_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool2d_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool3d_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool3d_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool3d_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_NLLLoss_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReflectionPad1d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReflectionPad2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReflectionPad3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReplicationPad1d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReplicationPad2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReplicationPad3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_bincount_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_grid_sample_2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_grid_sample_3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_histc_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_interpolate_bicubic_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_interpolate_bilinear_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_interpolate_linear_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_interpolate_trilinear_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_kthvalue_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_median_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_put_accumulate_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_put_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_resize_quantized_cuda_qint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_resize_quantized_cuda_qint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_resize_quantized_cuda_quint2x4, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_resize_quantized_cuda_quint4x2, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_resize_quantized_cuda_quint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_normal_kstest_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_normal_kstest_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_normal_kstest_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_normal_kstest_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nullary_op_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_pairwise_distance_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_params_invalidated_with_grads_invalidated_between_unscale_and_step_AdamW_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_params_invalidated_with_grads_invalidated_between_unscale_and_step_Adam_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_params_invalidated_with_grads_invalidated_between_unscale_and_step_SGD_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_pdist_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_pdist_norm_large_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_pickle_gradscaler_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_pin_memory_from_constructor_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_reduced_type_float_copy_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_reduced_type_float_copy_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_repeat_interleave_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scalar_check_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_add_bool_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_add_non_unique_index_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_add_one_dim_deterministic_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_add_to_large_input_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_bool_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_multiply_unsupported_dtypes_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_multiply_unsupported_dtypes_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_to_large_input_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_zero_size_index_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_serialization_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_default_tensor_type_warnings_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_shift_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_skip_xla_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_all_devices_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_qint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_qint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_quint4x2, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_quint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_strides_propagation_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_sync_warning_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_set_errors_multigpu_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_shape_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_type_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_ternary_op_mem_overlap_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_unfold_all_devices_and_dtypes_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_unfold_scalars_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_uniform_kstest_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_uniform_kstest_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_uniform_kstest_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_uniform_kstest_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_untyped_storage_meta_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_warn_always_caught_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_where_scalar_handcrafted_values_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_advancedindex_mixed_cpu_devices_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_advancedindex_mixed_devices_error_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_clamp_cuda_float32, test/test_torch.py::TestDevicePrecisionCUDA::test_clamp_cuda_float64, test/test_torch.py::TestDevicePrecisionCUDA::test_clamp_cuda_int64, test/test_torch.py::TestDevicePrecisionCUDA::test_copy_broadcast_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_copy_noncontig_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_cuda_device_idx_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_device_serialization_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_float16, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_float32, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_float64, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_int16, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_int32, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_int64, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_int8, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_uint8, test/test_torch.py::TestDevicePrecisionCUDA::test_index_add_bfloat16_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_multidevice_serialization_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_type_conversions_same_device_cuda 2024-08-07T18:06:57.6607417Z 2024-08-07T18:06:57.6607947Z test_torch.py::TestBasicVitalSigns::test_basic_vitals PASSED [0.0026s] [ 0%] 2024-08-07T18:06:57.6608841Z test_torch.py::TestBasicVitalSigns::test_basic_vitals_read_write PASSED [0.0011s] [ 0%] 2024-08-07T18:06:57.6609675Z test_torch.py::TestBasicVitalSigns::test_dataloader_vitals PASSED [0.0018s] [ 0%] 2024-08-07T18:06:57.6610441Z test_torch.py::TestTorch::test_RNGState PASSED [0.0018s] [ 0%] 2024-08-07T18:06:57.6611192Z test_torch.py::TestTorch::test_RNGStateAliasing PASSED [0.0017s] [ 0%] 2024-08-07T18:06:57.6611936Z test_torch.py::TestTorch::test_RNG_after_pickle PASSED [0.0035s] [ 0%] 2024-08-07T18:06:57.6612661Z test_torch.py::TestTorch::test_Size PASSED [0.0014s] [ 0%] 2024-08-07T18:06:57.6613365Z test_torch.py::TestTorch::test_Size_iter PASSED [0.0012s] [ 0%] 2024-08-07T18:06:57.6614081Z test_torch.py::TestTorch::test_Size_scalar PASSED [0.0011s] [ 0%] 2024-08-07T18:06:57.6614805Z test_torch.py::TestTorch::test_add_meta_scalar PASSED [0.0014s] [ 0%] 2024-08-07T18:06:57.6615550Z test_torch.py::TestTorch::test_allow_tensor_metadata_change PASSED [0.0010s] [ 1%] 2024-08-07T18:06:57.6616322Z test_torch.py::TestTorch::test_apply PASSED [0.0012s] [ 1%] 2024-08-07T18:06:57.6617069Z test_torch.py::TestTorch::test_as_subclass PASSED [0.0028s] [ 1%] 2024-08-07T18:06:57.6617784Z test_torch.py::TestTorch::test_assert_async PASSED [0.0019s] [ 1%] 2024-08-07T18:06:57.6618516Z test_torch.py::TestTorch::test_backward_hooks_traverse PASSED [0.1147s] [ 1%] 2024-08-07T18:06:57.6619258Z test_torch.py::TestTorch::test_batch_norm_cpu_inference PASSED [0.0049s] [ 1%] 2024-08-07T18:06:57.6620186Z test_torch.py::TestTorch::test_bf16_supported_on_cpu SKIPPED [0.0002s] (Test specific for CPU) [ 1%] 2024-08-07T18:06:57.6620990Z test_torch.py::TestTorch::test_bmm_multithreaded PASSED [0.5705s] [ 1%] 2024-08-07T18:06:57.6621738Z test_torch.py::TestTorch::test_boxMullerState PASSED [0.0018s] [ 1%] 2024-08-07T18:06:57.6622501Z test_torch.py::TestTorch::test_c10_feature_enabled ('RERUN', {'yellow': True}) [0.0011s] [ 1%] 2024-08-07T18:06:57.6623435Z test_torch.py::TestTorch::test_c10_feature_enabled ('RERUN', {'yellow': True}) [0.0010s] [ 1%] 2024-08-07T18:06:57.6624181Z test_torch.py::TestTorch::test_c10_feature_enabled FAILED [0.0010s] [ 1%] 2024-08-07T18:06:57.6624558Z 2024-08-07T18:06:57.6624689Z ==================================== RERUNS ==================================== 2024-08-07T18:06:57.6625231Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:06:57.6625665Z Traceback (most recent call last): 2024-08-07T18:06:57.6626167Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:06:57.6626670Z with mocksetting(True): 2024-08-07T18:06:57.6627215Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:06:57.6627678Z return next(self.gen) 2024-08-07T18:06:57.6628107Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:06:57.6628568Z del os.environ[feature] 2024-08-07T18:06:57.6628970Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:06:57.6629407Z raise KeyError(key) from None 2024-08-07T18:06:57.6630340Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:06:57.6631479Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:06:57.6631921Z Traceback (most recent call last): 2024-08-07T18:06:57.6632417Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:06:57.6632910Z with mocksetting(True): 2024-08-07T18:06:57.6633335Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:06:57.6633790Z return next(self.gen) 2024-08-07T18:06:57.6634207Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:06:57.6634664Z del os.environ[feature] 2024-08-07T18:06:57.6635063Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:06:57.6635489Z raise KeyError(key) from None 2024-08-07T18:06:57.6636416Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:06:57.6637379Z =================================== FAILURES =================================== 2024-08-07T18:06:57.6637933Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:06:57.6638369Z Traceback (most recent call last): 2024-08-07T18:06:57.6638860Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:06:57.6639364Z with mocksetting(True): 2024-08-07T18:06:57.6639782Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:06:57.6640229Z return next(self.gen) 2024-08-07T18:06:57.6640656Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:06:57.6641109Z del os.environ[feature] 2024-08-07T18:06:57.6641506Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:06:57.6641942Z raise KeyError(key) from None 2024-08-07T18:06:57.6642869Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:06:57.6644138Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_torch/test_torch-d27faa9db46fde99.xml - 2024-08-07T18:06:57.6645031Z =========================== short test summary info ============================ 2024-08-07T18:06:57.6648711Z FAILED [0.0010s] test_torch.py::TestTorch::test_c10_feature_enabled - KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:06:57.6650040Z !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!! 2024-08-07T18:06:57.6650748Z =============== 1 failed, 18 passed, 1 skipped, 2 rerun in 0.82s =============== 2024-08-07T18:06:57.6651384Z [TORCH_VITAL] Dataloader.enabled True 2024-08-07T18:06:57.6651779Z [TORCH_VITAL] Dataloader.basic_unit_test TEST_VALUE_STRING 2024-08-07T18:06:57.6652194Z [TORCH_VITAL] CUDA.used true 2024-08-07T18:06:57.6652480Z Got exit code 1 2024-08-07T18:06:57.6652721Z Retrying single test... 2024-08-07T18:06:57.6653201Z Test results will be stored in test-reports/python-pytest/test_torch/test_torch-34ab51e8fb61672a.xml 2024-08-07T18:06:57.6653858Z ============================= test session starts ============================== 2024-08-07T18:06:57.6654407Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.5.0 -- /opt/conda/envs/py_3.10/bin/python 2024-08-07T18:06:57.6654882Z cachedir: .pytest_cache 2024-08-07T18:06:57.6655453Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-08-07T18:06:57.6656073Z rootdir: /var/lib/jenkins/workspace 2024-08-07T18:06:57.6656363Z configfile: pytest.ini 2024-08-07T18:06:57.6656847Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-08-07T18:06:57.6657541Z collecting ... collected 1043 items / 1042 deselected / 1 selected 2024-08-07T18:06:57.6658211Z stepcurrent: skipping 19 already run items. Running only test/test_torch.py::TestTorch::test_c10_feature_enabled 2024-08-07T18:06:57.6658762Z Running 1 items in this shard 2024-08-07T18:06:57.6658952Z 2024-08-07T18:06:57.6659339Z test_torch.py::TestTorch::test_c10_feature_enabled ('RERUN', {'yellow': True}) [0.0026s] [100%] 2024-08-07T18:06:57.6660267Z test_torch.py::TestTorch::test_c10_feature_enabled ('RERUN', {'yellow': True}) [0.0011s] [100%] 2024-08-07T18:06:57.6661010Z test_torch.py::TestTorch::test_c10_feature_enabled FAILED [0.0012s] [100%] 2024-08-07T18:06:57.6661388Z 2024-08-07T18:06:57.6661518Z ==================================== RERUNS ==================================== 2024-08-07T18:06:57.6662053Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:06:57.6662482Z Traceback (most recent call last): 2024-08-07T18:06:57.6662977Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:06:57.6663475Z with mocksetting(True): 2024-08-07T18:06:57.6663903Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:06:57.6664341Z return next(self.gen) 2024-08-07T18:06:57.6664754Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:06:57.6665206Z del os.environ[feature] 2024-08-07T18:06:57.6665589Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:06:57.6666009Z raise KeyError(key) from None 2024-08-07T18:06:57.6666939Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:06:57.6668061Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:06:57.6668597Z Traceback (most recent call last): 2024-08-07T18:06:57.6669086Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:06:57.6669576Z with mocksetting(True): 2024-08-07T18:06:57.6670003Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:06:57.6670457Z return next(self.gen) 2024-08-07T18:06:57.6670873Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:06:57.6671331Z del os.environ[feature] 2024-08-07T18:06:57.6671727Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:06:57.6672155Z raise KeyError(key) from None 2024-08-07T18:06:57.6673161Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:06:57.6674123Z =================================== FAILURES =================================== 2024-08-07T18:06:57.6674665Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:06:57.6675091Z Traceback (most recent call last): 2024-08-07T18:06:57.6675582Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:06:57.6676075Z with mocksetting(True): 2024-08-07T18:06:57.6676491Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:06:57.6676985Z return next(self.gen) 2024-08-07T18:06:57.6677406Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:06:57.6677863Z del os.environ[feature] 2024-08-07T18:06:57.6678257Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:06:57.6678685Z raise KeyError(key) from None 2024-08-07T18:06:57.6679608Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:06:57.6680868Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_torch/test_torch-34ab51e8fb61672a.xml - 2024-08-07T18:06:57.6681660Z =========================== short test summary info ============================ 2024-08-07T18:06:57.6683182Z FAILED [0.0012s] test_torch.py::TestTorch::test_c10_feature_enabled - KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:06:57.6684491Z !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!! 2024-08-07T18:06:57.6685174Z ================= 1 failed, 1042 deselected, 2 rerun in 0.08s ================== 2024-08-07T18:06:57.6685651Z Got exit code 1 2024-08-07T18:06:57.6685897Z Retrying single test... 2024-08-07T18:06:57.6686372Z Test results will be stored in test-reports/python-pytest/test_torch/test_torch-41dc643beba15b53.xml 2024-08-07T18:06:57.6687019Z ============================= test session starts ============================== 2024-08-07T18:06:57.6687565Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.5.0 -- /opt/conda/envs/py_3.10/bin/python 2024-08-07T18:06:57.6688041Z cachedir: .pytest_cache 2024-08-07T18:06:57.6688618Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-08-07T18:06:57.6689236Z rootdir: /var/lib/jenkins/workspace 2024-08-07T18:06:57.6689534Z configfile: pytest.ini 2024-08-07T18:06:57.6690016Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-08-07T18:06:57.6690832Z collecting ... collected 1043 items / 1042 deselected / 1 selected 2024-08-07T18:06:57.6691487Z stepcurrent: skipping 19 already run items. Running only test/test_torch.py::TestTorch::test_c10_feature_enabled 2024-08-07T18:06:57.6692043Z Running 1 items in this shard 2024-08-07T18:06:57.6692223Z 2024-08-07T18:06:57.6692615Z test_torch.py::TestTorch::test_c10_feature_enabled ('RERUN', {'yellow': True}) [0.0027s] [100%] 2024-08-07T18:06:57.6693376Z test_torch.py::TestTorch::test_c10_feature_enabled ('RERUN', {'yellow': True}) [0.0011s] [100%] 2024-08-07T18:06:57.6694119Z test_torch.py::TestTorch::test_c10_feature_enabled FAILED [0.0012s] [100%] 2024-08-07T18:06:57.6694494Z 2024-08-07T18:06:57.6694710Z ==================================== RERUNS ==================================== 2024-08-07T18:06:57.6695241Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:06:57.6695678Z Traceback (most recent call last): 2024-08-07T18:06:57.6696166Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:06:57.6696710Z with mocksetting(True): 2024-08-07T18:06:57.6697132Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:06:57.6697577Z return next(self.gen) 2024-08-07T18:06:57.6697999Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:06:57.6698450Z del os.environ[feature] 2024-08-07T18:06:57.6698847Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:06:57.6699280Z raise KeyError(key) from None 2024-08-07T18:06:57.6700641Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:06:57.6701793Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:06:57.6702230Z Traceback (most recent call last): 2024-08-07T18:06:57.6702725Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:06:57.6703216Z with mocksetting(True): 2024-08-07T18:06:57.6703640Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:06:57.6704083Z return next(self.gen) 2024-08-07T18:06:57.6704492Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:06:57.6704944Z del os.environ[feature] 2024-08-07T18:06:57.6705344Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:06:57.6705762Z raise KeyError(key) from None 2024-08-07T18:06:57.6706728Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:06:57.6707685Z =================================== FAILURES =================================== 2024-08-07T18:06:57.6708226Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:06:57.6717040Z Traceback (most recent call last): 2024-08-07T18:06:57.6717564Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:06:57.6718073Z with mocksetting(True): 2024-08-07T18:06:57.6718502Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:06:57.6718951Z return next(self.gen) 2024-08-07T18:06:57.6719381Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:06:57.6719832Z del os.environ[feature] 2024-08-07T18:06:57.6720226Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:06:57.6720830Z raise KeyError(key) from None 2024-08-07T18:06:57.6721751Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:06:57.6723017Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_torch/test_torch-41dc643beba15b53.xml - 2024-08-07T18:06:57.6723848Z =========================== short test summary info ============================ 2024-08-07T18:06:57.6725492Z FAILED [0.0012s] test_torch.py::TestTorch::test_c10_feature_enabled - KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:06:57.6726831Z !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!! 2024-08-07T18:06:57.6727511Z ================= 1 failed, 1042 deselected, 2 rerun in 0.08s ================== 2024-08-07T18:06:57.6727994Z Got exit code 1 2024-08-07T18:06:57.6728250Z Stopping at first consistent failure 2024-08-07T18:06:57.6728786Z The following tests failed consistently: ['test/test_torch.py::TestTorch::test_c10_feature_enabled'] 2024-08-07T18:06:57.6729206Z 2024-08-07T18:06:57.6729511Z FINISHED PRINTING LOG FILE of test_torch 1/1 (test/test-reports/test_torch_1.1_561344a28600abd3_.log) 2024-08-07T18:06:57.6729904Z 2024-08-07T18:06:57.6730010Z test_torch 1/1 failed! 2024-08-07T18:06:57.6730284Z Emiting td_test_failure_stats_v2 2024-08-07T18:06:57.8379234Z Traceback (most recent call last): 2024-08-07T18:06:57.8379797Z File "/var/lib/jenkins/workspace/test/run_test.py", line 1879, in 2024-08-07T18:06:57.8380249Z main() 2024-08-07T18:06:57.8380592Z File "/var/lib/jenkins/workspace/test/run_test.py", line 1834, in main 2024-08-07T18:06:57.8381004Z run_tests( 2024-08-07T18:06:57.8381377Z File "/var/lib/jenkins/workspace/test/run_test.py", line 1686, in run_tests 2024-08-07T18:06:57.8381878Z raise RuntimeError(failure.message + keep_going_message) 2024-08-07T18:06:57.8382262Z RuntimeError: test_torch 1/1 failed! 2024-08-07T18:06:57.8382463Z 2024-08-07T18:06:57.8382738Z Tip: You can keep running tests even on failure by passing --keep-going to run_test.py. 2024-08-07T18:06:57.8383309Z If running on CI, add the 'keep-going' label to your PR and rerun your jobs. 2024-08-07T18:06:58.3491446Z 2024-08-07T18:06:58.3491876Z real 0m19.008s 2024-08-07T18:06:58.3492164Z user 0m25.021s 2024-08-07T18:06:58.3492435Z sys 0m13.534s 2024-08-07T18:06:58.3492682Z + cleanup_workspace 2024-08-07T18:06:58.3493177Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-07T18:06:58.3493944Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-07T18:06:58.3494548Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-07T18:06:58.3495002Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-07T18:06:58.3495531Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-07T18:06:58.3496102Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-07T18:06:58.3496557Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-07T18:06:59.0774487Z ##[error]Process completed with exit code 1. 2024-08-07T18:06:59.0836318Z Prepare all required actions 2024-08-07T18:06:59.0836674Z Getting action download info 2024-08-07T18:06:59.2921709Z ##[group]Run ./.github/actions/pytest-cache-upload 2024-08-07T18:06:59.2922055Z with: 2024-08-07T18:06:59.2922280Z cache_dir: .pytest_cache 2024-08-07T18:06:59.2922533Z shard: 1 2024-08-07T18:06:59.2922969Z sha: 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T18:06:59.2923296Z test_config: default 2024-08-07T18:06:59.2923632Z job_identifier: pull_linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-07T18:06:59.2924010Z env: 2024-08-07T18:06:59.2924235Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:06:59.2924566Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:06:59.2925097Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:06:59.2925567Z ##[endgroup] 2024-08-07T18:06:59.2967943Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-07T18:06:59.2968339Z with: 2024-08-07T18:06:59.2968590Z shell: bash 2024-08-07T18:06:59.2968819Z timeout_minutes: 5 2024-08-07T18:06:59.2969073Z max_attempts: 5 2024-08-07T18:06:59.2969324Z retry_wait_seconds: 30 2024-08-07T18:06:59.2969653Z command: set -eu python3 -m pip install boto3==1.19.12 2024-08-07T18:06:59.2970032Z polling_interval_seconds: 1 2024-08-07T18:06:59.2970328Z warning_on_retry: true 2024-08-07T18:06:59.2970590Z continue_on_error: false 2024-08-07T18:06:59.2970847Z env: 2024-08-07T18:06:59.2971066Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:06:59.2971388Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:06:59.2971911Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:06:59.2972376Z ##[endgroup] 2024-08-07T18:06:59.6127507Z Defaulting to user installation because normal site-packages is not writeable 2024-08-07T18:07:00.7457092Z Collecting boto3==1.19.12 2024-08-07T18:07:00.7851247Z Downloading boto3-1.19.12-py3-none-any.whl (131 kB) 2024-08-07T18:07:02.0134155Z Collecting botocore<1.23.0,>=1.22.12 2024-08-07T18:07:02.0210571Z Downloading botocore-1.22.12-py3-none-any.whl (8.1 MB) 2024-08-07T18:07:02.2570423Z Collecting s3transfer<0.6.0,>=0.5.0 2024-08-07T18:07:02.2623455Z Downloading s3transfer-0.5.2-py3-none-any.whl (79 kB) 2024-08-07T18:07:02.2666258Z Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/lib/python3.9/site-packages (from boto3==1.19.12) (0.10.0) 2024-08-07T18:07:02.2710043Z Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/lib/python3.9/site-packages (from botocore<1.23.0,>=1.22.12->boto3==1.19.12) (2.8.1) 2024-08-07T18:07:02.2713939Z Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python3.9/site-packages (from botocore<1.23.0,>=1.22.12->boto3==1.19.12) (1.25.10) 2024-08-07T18:07:02.4853779Z Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.23.0,>=1.22.12->boto3==1.19.12) (1.15.0) 2024-08-07T18:07:02.5947409Z Installing collected packages: botocore, s3transfer, boto3 2024-08-07T18:07:03.1735140Z Successfully installed boto3-1.19.12 botocore-1.22.12 s3transfer-0.5.2 2024-08-07T18:07:03.3522352Z Command completed after 1 attempt(s). 2024-08-07T18:07:03.3588421Z ##[group]Run python3 .github/scripts/pytest_cache.py \ 2024-08-07T18:07:03.3588854Z python3 .github/scripts/pytest_cache.py \ 2024-08-07T18:07:03.3589214Z  --upload \ 2024-08-07T18:07:03.3589517Z  --cache_dir $GITHUB_WORKSPACE/$CACHE_DIR \ 2024-08-07T18:07:03.3589882Z  --pr_identifier $GITHUB_REF \ 2024-08-07T18:07:03.3590216Z  --job_identifier $JOB_IDENTIFIER \ 2024-08-07T18:07:03.3590532Z  --sha $SHA \ 2024-08-07T18:07:03.3590809Z  --test_config $TEST_CONFIG \ 2024-08-07T18:07:03.3591113Z  --shard $SHARD \ 2024-08-07T18:07:03.3591387Z  --repo $REPO \ 2024-08-07T18:07:03.3591666Z  --temp_dir $RUNNER_TEMP \ 2024-08-07T18:07:03.3603761Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:07:03.3604145Z env: 2024-08-07T18:07:03.3604375Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:03.3604708Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:03.3605235Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:03.3605871Z CACHE_DIR: .pytest_cache 2024-08-07T18:07:03.3606228Z JOB_IDENTIFIER: pull_linux-focal-cuda12.1-py3.10-gcc9-sm86 2024-08-07T18:07:03.3606638Z SHA: 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T18:07:03.3606961Z TEST_CONFIG: default 2024-08-07T18:07:03.3607206Z SHARD: 1 2024-08-07T18:07:03.3607438Z REPO: pytorch/pytorch 2024-08-07T18:07:03.3607700Z ##[endgroup] 2024-08-07T18:07:03.6795666Z PR identifier for `refs/pull/131248/merge` is `cbc34607c96079dfcdbe69d358ad18c0` 2024-08-07T18:07:03.6798611Z Uploading cache with args Namespace(upload=True, download=False, cache_dir='/home/ec2-user/actions-runner/_work/pytorch/pytorch/.pytest_cache', pr_identifier='refs/pull/131248/merge', job_identifier='pull_linux-focal-cuda12.1-py3.10-gcc9-sm86', sha='016588f53c6904b840aa56aa86f95460b4d9c996', test_config='default', shard='1', repo='pytorch/pytorch', temp_dir='/home/ec2-user/actions-runner/_work/_temp', bucket=None) 2024-08-07T18:07:03.6803524Z Zipping /home/ec2-user/actions-runner/_work/pytorch/pytorch/.pytest_cache 2024-08-07T18:07:03.6805284Z to /home/ec2-user/actions-runner/_work/_temp/zip-upload/pytest_cache/pytorch/pytorch/cbc34607c96079dfcdbe69d358ad18c0/pull_linux-focal-cuda12_1-py3_10-gcc9-sm86/016588f53c6904b840aa56aa86f95460b4d9c996/default/1 2024-08-07T18:07:03.6807898Z Uploading /home/ec2-user/actions-runner/_work/_temp/zip-upload/pytest_cache/pytorch/pytorch/cbc34607c96079dfcdbe69d358ad18c0/pull_linux-focal-cuda12_1-py3_10-gcc9-sm86/016588f53c6904b840aa56aa86f95460b4d9c996/default/1.zip 2024-08-07T18:07:03.6810350Z to s3://gha-artifacts/pytest_cache/pytorch/pytorch/cbc34607c96079dfcdbe69d358ad18c0/pull_linux-focal-cuda12_1-py3_10-gcc9-sm86/016588f53c6904b840aa56aa86f95460b4d9c996/default/1.zip 2024-08-07T18:07:03.7198036Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-07T18:07:03.7198413Z cat test/**/*_toprint.log || true 2024-08-07T18:07:03.7207691Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:07:03.7208087Z env: 2024-08-07T18:07:03.7208329Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:03.7208654Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:03.7209174Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:03.7209628Z ##[endgroup] 2024-08-07T18:07:03.7294354Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-07T18:07:03.7330299Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-07T18:07:03.7330654Z kill "$MONITOR_SCRIPT_PID" 2024-08-07T18:07:03.7338574Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:07:03.7338936Z env: 2024-08-07T18:07:03.7339152Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:03.7339733Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:03.7340263Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:03.7340739Z MONITOR_SCRIPT_PID: 72156 2024-08-07T18:07:03.7341012Z ##[endgroup] 2024-08-07T18:07:03.7475680Z Prepare all required actions 2024-08-07T18:07:03.7476064Z Getting action download info 2024-08-07T18:07:03.8660577Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-07T18:07:04.0734376Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-07T18:07:04.0734726Z with: 2024-08-07T18:07:04.0735101Z file-suffix: test-default-1-5-amz2023.linux.g5.4xlarge.nvidia.gpu_28476695132 2024-08-07T18:07:04.0735567Z s3-bucket: gha-artifacts 2024-08-07T18:07:04.0735830Z env: 2024-08-07T18:07:04.0736039Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:04.0736371Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:04.0736896Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:04.0737354Z ##[endgroup] 2024-08-07T18:07:04.0766832Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-07T18:07:04.0767299Z # Remove any previous test jsons if they exist 2024-08-07T18:07:04.0767847Z rm -f test-jsons-*.zip 2024-08-07T18:07:04.0768232Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-07T18:07:04.0776699Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:07:04.0777070Z env: 2024-08-07T18:07:04.0777303Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:04.0777639Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:04.0778171Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:04.0778856Z FILE_SUFFIX: test-default-1-5-amz2023.linux.g5.4xlarge.nvidia.gpu_28476695132 2024-08-07T18:07:04.0779313Z ##[endgroup] 2024-08-07T18:07:04.1056188Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-07T18:07:04.1085411Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-07T18:07:04.1085923Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-07T18:07:04.1092288Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-07T18:07:04.1095444Z adding: test/test-reports/td_exclusions-142838d25421081b7d48.json (deflated 81%) 2024-08-07T18:07:04.1101128Z adding: test/.pytorch-slow-tests.json (deflated 81%) 2024-08-07T18:07:04.1114338Z adding: test/.pytorch-disabled-tests.json (deflated 89%) 2024-08-07T18:07:04.1158955Z ##[group]Run # Remove any previous test reports if they exist 2024-08-07T18:07:04.1159409Z # Remove any previous test reports if they exist 2024-08-07T18:07:04.1159784Z rm -f test-reports-*.zip 2024-08-07T18:07:04.1160204Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-07T18:07:04.1168810Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:07:04.1169171Z env: 2024-08-07T18:07:04.1169398Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:04.1169737Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:04.1170268Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:04.1170906Z FILE_SUFFIX: test-default-1-5-amz2023.linux.g5.4xlarge.nvidia.gpu_28476695132 2024-08-07T18:07:04.1171365Z ##[endgroup] 2024-08-07T18:07:04.1432781Z adding: test/test-reports/python-pytest/test_torch/test_torch-d27faa9db46fde99.xml (deflated 90%) 2024-08-07T18:07:04.1433608Z adding: test/test-reports/python-pytest/test_torch/test_torch-34ab51e8fb61672a.xml (deflated 82%) 2024-08-07T18:07:04.1434354Z adding: test/test-reports/python-pytest/test_torch/test_torch-41dc643beba15b53.xml (deflated 82%) 2024-08-07T18:07:04.1471131Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-07T18:07:04.1471631Z # Remove any previous usage logs if they exist 2024-08-07T18:07:04.1471992Z rm -f logs-*.zip 2024-08-07T18:07:04.1472442Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-07T18:07:04.1473181Z # so check to see if the file exists first 2024-08-07T18:07:04.1473557Z if [ -f 'usage_log.txt' ]; then 2024-08-07T18:07:04.1473925Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-07T18:07:04.1474266Z fi 2024-08-07T18:07:04.1474536Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-07T18:07:04.1474916Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-07T18:07:04.1475256Z fi 2024-08-07T18:07:04.1483507Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:07:04.1483870Z env: 2024-08-07T18:07:04.1484097Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:04.1484430Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:04.1484948Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:04.1485573Z FILE_SUFFIX: test-default-1-5-amz2023.linux.g5.4xlarge.nvidia.gpu_28476695132 2024-08-07T18:07:04.1486026Z ##[endgroup] 2024-08-07T18:07:04.1537302Z adding: usage_log.txt (deflated 91%) 2024-08-07T18:07:04.1830456Z adding: test/test-reports/test_torch_1.1_561344a28600abd3_.log (deflated 91%) 2024-08-07T18:07:04.1868044Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-07T18:07:04.1868561Z # Remove any previous debugging artifacts if they exist 2024-08-07T18:07:04.1868956Z rm -f debug-*.zip 2024-08-07T18:07:04.1869247Z if [ -d 'test/debug' ]; then 2024-08-07T18:07:04.1869605Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-07T18:07:04.1869947Z fi 2024-08-07T18:07:04.1878614Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:07:04.1878983Z env: 2024-08-07T18:07:04.1879203Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:04.1879543Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:04.1880076Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:04.1880717Z FILE_SUFFIX: test-default-1-5-amz2023.linux.g5.4xlarge.nvidia.gpu_28476695132 2024-08-07T18:07:04.1881191Z ##[endgroup] 2024-08-07T18:07:04.2004025Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T18:07:04.2004362Z with: 2024-08-07T18:07:04.2004598Z s3-bucket: gha-artifacts 2024-08-07T18:07:04.2004913Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T18:07:04.2005252Z retention-days: 14 2024-08-07T18:07:04.2005507Z if-no-files-found: warn 2024-08-07T18:07:04.2005777Z path: test-jsons-*.zip 2024-08-07T18:07:04.2006039Z name: artifact 2024-08-07T18:07:04.2006273Z region: us-east-1 2024-08-07T18:07:04.2006494Z env: 2024-08-07T18:07:04.2006707Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:04.2007038Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:04.2007568Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:04.2008033Z ##[endgroup] 2024-08-07T18:07:04.5658227Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-07T18:07:04.5659086Z With the provided path, there will be 1 file uploaded 2024-08-07T18:07:04.5660058Z Uploading to s3 prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T18:07:04.5716289Z Starting upload of test-jsons-test-default-1-5-amz2023.linux.g5.4xlarge.nvidia.gpu_28476695132.zip 2024-08-07T18:07:04.7156703Z Finished upload of test-jsons-test-default-1-5-amz2023.linux.g5.4xlarge.nvidia.gpu_28476695132.zip 2024-08-07T18:07:04.7422211Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T18:07:04.7422547Z with: 2024-08-07T18:07:04.7422777Z s3-bucket: gha-artifacts 2024-08-07T18:07:04.7423103Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T18:07:04.7423444Z retention-days: 14 2024-08-07T18:07:04.7423695Z if-no-files-found: error 2024-08-07T18:07:04.7423975Z path: test-reports-*.zip 2024-08-07T18:07:04.7424239Z name: artifact 2024-08-07T18:07:04.7424463Z region: us-east-1 2024-08-07T18:07:04.7424692Z env: 2024-08-07T18:07:04.7424907Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:04.7425451Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:04.7425988Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:04.7426460Z ##[endgroup] 2024-08-07T18:07:05.0966346Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-07T18:07:05.0966901Z With the provided path, there will be 1 file uploaded 2024-08-07T18:07:05.0967335Z Uploading to s3 prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T18:07:05.1025009Z Starting upload of test-reports-test-default-1-5-amz2023.linux.g5.4xlarge.nvidia.gpu_28476695132.zip 2024-08-07T18:07:05.1972293Z Finished upload of test-reports-test-default-1-5-amz2023.linux.g5.4xlarge.nvidia.gpu_28476695132.zip 2024-08-07T18:07:05.2271594Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T18:07:05.2271923Z with: 2024-08-07T18:07:05.2272156Z s3-bucket: gha-artifacts 2024-08-07T18:07:05.2272483Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T18:07:05.2272838Z retention-days: 14 2024-08-07T18:07:05.2273302Z if-no-files-found: ignore 2024-08-07T18:07:05.2273575Z path: logs-*.zip 2024-08-07T18:07:05.2273802Z name: artifact 2024-08-07T18:07:05.2274034Z region: us-east-1 2024-08-07T18:07:05.2274264Z env: 2024-08-07T18:07:05.2274477Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:05.2274808Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:05.2275336Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:05.2275797Z ##[endgroup] 2024-08-07T18:07:05.5796652Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-07T18:07:05.5797309Z With the provided path, there will be 1 file uploaded 2024-08-07T18:07:05.5797786Z Uploading to s3 prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T18:07:05.5853641Z Starting upload of logs-test-default-1-5-amz2023.linux.g5.4xlarge.nvidia.gpu_28476695132.zip 2024-08-07T18:07:05.7142494Z Finished upload of logs-test-default-1-5-amz2023.linux.g5.4xlarge.nvidia.gpu_28476695132.zip 2024-08-07T18:07:05.7409935Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T18:07:05.7410277Z with: 2024-08-07T18:07:05.7410511Z s3-bucket: gha-artifacts 2024-08-07T18:07:05.7410843Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T18:07:05.7411200Z retention-days: 14 2024-08-07T18:07:05.7411460Z if-no-files-found: ignore 2024-08-07T18:07:05.7411735Z path: debug-*.zip 2024-08-07T18:07:05.7411980Z name: artifact 2024-08-07T18:07:05.7412209Z region: us-east-1 2024-08-07T18:07:05.7412441Z env: 2024-08-07T18:07:05.7412668Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:05.7412995Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:05.7413519Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:05.7413985Z ##[endgroup] 2024-08-07T18:07:06.0932120Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-07T18:07:06.1213656Z ##[group]Run # shellcheck disable=SC2156 2024-08-07T18:07:06.1214043Z # shellcheck disable=SC2156 2024-08-07T18:07:06.1214601Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-07T18:07:06.1223466Z shell: /usr/bin/bash -e {0} 2024-08-07T18:07:06.1223747Z env: 2024-08-07T18:07:06.1223966Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:06.1224287Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:06.1224812Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:06.1225278Z ##[endgroup] 2024-08-07T18:07:06.3844778Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T18:07:06.3845274Z with: 2024-08-07T18:07:06.3845759Z name: coredumps-default-1-5-amz2023.linux.g5.4xlarge.nvidia.gpu 2024-08-07T18:07:06.3846407Z retention-days: 14 2024-08-07T18:07:06.3846800Z if-no-files-found: ignore 2024-08-07T18:07:06.3847223Z path: ./**/core.[1-9]* 2024-08-07T18:07:06.3847634Z s3-bucket: gha-artifacts 2024-08-07T18:07:06.3848063Z region: us-east-1 2024-08-07T18:07:06.3848418Z env: 2024-08-07T18:07:06.3848760Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:06.3849267Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:06.3850201Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:06.3850967Z ##[endgroup] 2024-08-07T18:07:16.6984625Z No files were found with the provided path: ./**/core.[1-9]*. No artifacts will be uploaded. 2024-08-07T18:07:16.7431789Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-07T18:07:16.7432257Z with: 2024-08-07T18:07:16.7432485Z env: 2024-08-07T18:07:16.7432715Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:16.7433057Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:16.7433583Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:16.7434062Z ##[endgroup] 2024-08-07T18:07:16.7457733Z ##[group]Run set -eou pipefail 2024-08-07T18:07:16.7458056Z set -eou pipefail 2024-08-07T18:07:16.7458323Z  2024-08-07T18:07:16.7458689Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-07T18:07:16.7459138Z for _ in $(seq 1440); do 2024-08-07T18:07:16.7459624Z  # Break if no ssh session exists anymore 2024-08-07T18:07:16.7459969Z  if [ "$(who)" = "" ]; then 2024-08-07T18:07:16.7460269Z  break 2024-08-07T18:07:16.7460533Z  fi 2024-08-07T18:07:16.7460760Z  echo "." 2024-08-07T18:07:16.7461011Z  sleep 5 2024-08-07T18:07:16.7461254Z done 2024-08-07T18:07:16.7469843Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:07:16.7470221Z env: 2024-08-07T18:07:16.7470446Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:16.7470778Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:16.7471304Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:16.7471784Z ##[endgroup] 2024-08-07T18:07:16.7500755Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-07T18:07:16.7561532Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T18:07:16.7562097Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T18:07:16.7562531Z # shellcheck disable=SC2046 2024-08-07T18:07:16.7562876Z docker stop $(docker ps -q) || true 2024-08-07T18:07:16.7563274Z # Prune all of the docker images 2024-08-07T18:07:16.7563593Z docker system prune -af 2024-08-07T18:07:16.7571444Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:07:16.7571815Z env: 2024-08-07T18:07:16.7572038Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:16.7572418Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:16.7572959Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:16.7573430Z ##[endgroup] 2024-08-07T18:07:17.4740197Z 17250618960b 2024-08-07T18:07:18.2202503Z Deleted Containers: 2024-08-07T18:07:18.2202875Z 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:18.2203197Z 2024-08-07T18:07:26.2885708Z Deleted Images: 2024-08-07T18:07:26.2886633Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T18:07:26.2887940Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9@sha256:00f47b036f588ca5ef8866f8635fabba5a95cdf9ff1adae7d2a674ef1d4076e9 2024-08-07T18:07:26.2888942Z deleted: sha256:6ec36276acd88c9be8b44d856744037d399b35f4bb1703e637c27ae2b254c901 2024-08-07T18:07:26.2889568Z deleted: sha256:6fbc5fe2ebb0dc33846ab2ade7c5296a0a521e16f71c3b15b8a0c40a8fce5ed3 2024-08-07T18:07:26.2890196Z deleted: sha256:31fb5fadd0be2cc6e0d198fbc00e4f3c25925bc9c9ce79be23f02aeb56f6a55a 2024-08-07T18:07:26.2890821Z deleted: sha256:d1d89c6e648d792c08fdaae4fdad1273f93571a1b8d03c76f38f9e2be7cfe7f1 2024-08-07T18:07:26.2891438Z deleted: sha256:0f1adf9d1a1d4eeb62caa063c1090ea2f50246ed031d8d6627df2f5fe5963067 2024-08-07T18:07:26.2892046Z deleted: sha256:771e69c2306df070c5a944ea3043f81ce5890ce90de66fb6f16edfaf09912b35 2024-08-07T18:07:26.2892644Z deleted: sha256:288aa01dcd110ad9e2e3f48756d987529733a41225b6e8d4898f7386666beeae 2024-08-07T18:07:26.2893662Z deleted: sha256:d9d7c6c9bef79d8c5ce30d89b81820f3664c04fe17ae396b2a121e96fcfdeccf 2024-08-07T18:07:26.2894291Z deleted: sha256:1efa95271d86fd98beb626e63f4c67732daca36bf6e25d864735e43ef4c708f4 2024-08-07T18:07:26.2894898Z deleted: sha256:b1699f7cf1967593f94664569bb49ba431deaac0d1ceaf0d0584046a78ca3be0 2024-08-07T18:07:26.2895507Z deleted: sha256:b70eb1cb717d1c2ce90d5fa7f48e1af32d45f3735b33e1f944762686d8459aa6 2024-08-07T18:07:26.2896116Z deleted: sha256:6460a71f50d3c4bf7773fa072665e881e2a7e0e4e0bd45302e6a73c35ec03898 2024-08-07T18:07:26.2896902Z deleted: sha256:1c1787b4980844f6bfea1fe2386125ac286e9dfd80e253164a7f82b90f9c37bc 2024-08-07T18:07:26.2897504Z deleted: sha256:758fb95d0e1c4b4c78ee59fc4d66c06acfbc3a3ffa870e94200529367b8998e8 2024-08-07T18:07:26.2898127Z deleted: sha256:99beb19b8911fc7c49d248960a97d8ba4bbcbe8afc9fe3142e5af37f08c1c821 2024-08-07T18:07:26.2898743Z deleted: sha256:55596f5748c7f4d8e4d9fea0d1a6cda4627ccc442e652c8578169efe5992a382 2024-08-07T18:07:26.2899360Z deleted: sha256:c279b10b40b22a2fba8aeaad85e7705ffb1afa28e000323b35cc947649b7637d 2024-08-07T18:07:26.2900334Z deleted: sha256:8c882a49501c1d973eacd2ed8da81bce818403153d3fb4cdc84baf14307f9517 2024-08-07T18:07:26.2900949Z deleted: sha256:9fdbb157fcea7486d127e106ff114401c3102b9e9ed879a59247a296b3a908af 2024-08-07T18:07:26.2901546Z deleted: sha256:25b821c2e5d4106b0214b3aa4c88265bda5f90f045f012af912f0a3d2979f919 2024-08-07T18:07:26.2902157Z deleted: sha256:fada342ffd1729d0c38b7afe6767d6840580548d8dac7c62ddf61e24803ad66e 2024-08-07T18:07:26.2902773Z deleted: sha256:14d55c394ec592bb1417a21f696089e0549403bff381cda63d3e92ec80bca298 2024-08-07T18:07:26.2903374Z deleted: sha256:1032dedb70a12fc3c1cae4b10cdd7a07a9d20a64736533d640dd133d46f1ebfe 2024-08-07T18:07:26.2903985Z deleted: sha256:c0f5baad08f078d4210c254240524d9063c5e797564ada8a4bf39270e0fc1300 2024-08-07T18:07:26.2904588Z deleted: sha256:1a8bc02cd2c8f897e1dfd14920703ea75d172660c85026068b39136a1a25db51 2024-08-07T18:07:26.2905188Z deleted: sha256:b2221232f25b8d3a916bf8f74248542af7830e6198814ffbdbab43484bcb700b 2024-08-07T18:07:26.2905793Z deleted: sha256:14dc7064caf62d3d51b65e5175f9ae0e31af4fbed83413736f80d881f0fd742e 2024-08-07T18:07:26.2906586Z deleted: sha256:162ef080ee355a240e7e8fd6761113ad61586e1c883207ba4290c90abf208b54 2024-08-07T18:07:26.2907253Z deleted: sha256:56e3e897838040eb3fb86255ef1be2e3be6705d2e4a1ead67bbd400350ff6d13 2024-08-07T18:07:26.2907853Z deleted: sha256:a4f1392c3713fac843566ae891b9e41c28824e377af84190d33a4132cd4b268d 2024-08-07T18:07:26.2908459Z deleted: sha256:d1a8ab1bb8f3238d77db318f5161058057f84f7357afd2932e1d7edded9a2efb 2024-08-07T18:07:26.2909074Z deleted: sha256:7cab46d23f9fd91e62280b25d274d6c8a03be6d66030adf598944913d280c54f 2024-08-07T18:07:26.2909673Z deleted: sha256:764da7f97ae4318f5157e363c03c86c016b8d1b6d2c4758d203fa8f194753f57 2024-08-07T18:07:26.2910278Z deleted: sha256:373ab2f02049fb6ccab3f886c53934457751a14e5cb68b5e144b04bb7afeed87 2024-08-07T18:07:26.2910891Z deleted: sha256:e46d90cf4cc84661ac93ae9bcfbc675c0fab0582328e4595d075e4f259c4389a 2024-08-07T18:07:26.2911496Z deleted: sha256:1579b987297253e32b13fe0d160a95c567910203783be2efcc3a76650255e658 2024-08-07T18:07:26.2912093Z deleted: sha256:f8542342382d7443bc8be95e90ac743f70abdafcd9987e3785f58ad50509a145 2024-08-07T18:07:26.2912696Z deleted: sha256:57432242bf8090a0c1238a5f004be4da246f4488a2193be182e5cd82237509e4 2024-08-07T18:07:26.2913296Z deleted: sha256:e3c369d15f59fb5f34178d8dcb6bb4a6ff1d527e9baf3182444c24c55600bdf4 2024-08-07T18:07:26.2913905Z deleted: sha256:4215189c6756d59c8a217cc09447c312318d07c0b0d5c9bfb7e4bfb942f05cae 2024-08-07T18:07:26.2914497Z deleted: sha256:b787e6329788262404e4b8293a110727833b63e2e287ccd569a21cb3fb450388 2024-08-07T18:07:26.2915108Z deleted: sha256:69bb5dca4b9fded8d0c731b73bcbed0c3e9ce170c89e79796956a444b0d58c4c 2024-08-07T18:07:26.2915725Z deleted: sha256:8c945f56e57fb4fbfa3f2c74d6109a47db8df7644c4988d0c5791e4214bf30c9 2024-08-07T18:07:26.2916340Z deleted: sha256:fdd079e0e07e11d86ffc744f1036793f3db2aea378660f7489fbf50002439620 2024-08-07T18:07:26.2917139Z deleted: sha256:6a743a461982e1a1b73f29cd187e354f2119f3bd985e1da6ca0b802134cb91ba 2024-08-07T18:07:26.2917743Z deleted: sha256:7b3cbce1a91c69d6a499ef524311c14537930fc04f0ef4b1d73030216ac4a568 2024-08-07T18:07:26.2918339Z deleted: sha256:d7120733e426141c8a9e8f2c3596b12055cd5b1956d141ad640365cb11628a00 2024-08-07T18:07:26.2918932Z deleted: sha256:841ae666d028560ea37e6314ae5f80d72fb071fe7592b577dbbd8156c08cdda2 2024-08-07T18:07:26.2919538Z deleted: sha256:9d85a8c4ea0c0bc631a4d3e5472e8ed8e2312a4dc5a38f6e9890edeb37183186 2024-08-07T18:07:26.2920276Z deleted: sha256:606f6418680b5c53158271dfaca16c668f3b9dd25bb2b738f5621b6a56d08cdb 2024-08-07T18:07:26.2920896Z deleted: sha256:5ca4df537c3efc44dfde3bebb377ed490bc4d1cd5d6e5fa2fd9549dcb456c471 2024-08-07T18:07:26.2921510Z deleted: sha256:c6f3f6b7969e62b95fe926a1963e6628cb2f1b5388f0b12115564855e59423f1 2024-08-07T18:07:26.2922120Z deleted: sha256:ce178391f5f25e4ebe6ba8e84c84831fb31b8a9d4e82dbc270a73ddeccbf1c2c 2024-08-07T18:07:26.2922743Z deleted: sha256:d154e0a00edb13898d564f38c090096dfeb7a90beb0bd8addd57eca33f52151e 2024-08-07T18:07:26.2923354Z deleted: sha256:7226a8914ccef500b1d0327370cf9e3a66fbcb0b59a5f3b018981372152fc645 2024-08-07T18:07:26.2923951Z deleted: sha256:ce5fab2081603cd22d9f253d98828ddd60c0f8c44c5413d0615f4264f1cd6a7b 2024-08-07T18:07:26.2924566Z deleted: sha256:b3bc21b2cb6a9ca2b1dbbc412812581089c8ace8cc6b8d2b2767f0b3cbe8b99c 2024-08-07T18:07:26.2925180Z deleted: sha256:53e8f21eb1afc2c8550bef59d67810a5f79671d1d3d3924577f810321e66886b 2024-08-07T18:07:26.2925781Z deleted: sha256:f9770918a5cc46e8f6aa13da6903754fa89a6bcc9f6e11df1d61f9340e452cc8 2024-08-07T18:07:26.2926403Z deleted: sha256:bce2de65cb97e6a64f59fe7fc78644a2f3d62cf7769073cb59dcbb52009fc5d0 2024-08-07T18:07:26.2927017Z deleted: sha256:568d161c6888a8b0b09d7feb561f790a31d47c7ff7ca1c89c0ff4025f5b02e3a 2024-08-07T18:07:26.2927615Z deleted: sha256:2da22d4dfb814e7a3f4444f60464109149d35bc3e078e197f600bd9f6cbb9f6b 2024-08-07T18:07:26.2928217Z deleted: sha256:4af2daf4a1277af00005823106d88d704d5d0499eef553f01875c6f94380b2e3 2024-08-07T18:07:26.2928829Z deleted: sha256:e52d382a6cbfc1bb5122186d4b1401de79de002a9cc323ba5047dbf266f4d3a3 2024-08-07T18:07:26.2929429Z deleted: sha256:d10fe93dc5314d752e8882c5877457a6ae733a93a1e246cd6f79301de9325e9c 2024-08-07T18:07:26.2930034Z deleted: sha256:58d5ecb60ded3f99102b25d909a729386d92a44bf68b77ed3b49bf27d978e26a 2024-08-07T18:07:26.2930727Z deleted: sha256:dd522c9c6fdd98e9e4584578a66d9122b36ee8f856bba2140fcaaf908c7a68e8 2024-08-07T18:07:26.2931329Z deleted: sha256:3584fb50d1cbc80b57118953f1dc36ecb092b5986d56211f711b9161f403d66c 2024-08-07T18:07:26.2931942Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-08-07T18:07:26.2932549Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-08-07T18:07:26.2933148Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-08-07T18:07:26.2933745Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-08-07T18:07:26.2934360Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-08-07T18:07:26.2934966Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-08-07T18:07:26.2935578Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-08-07T18:07:26.2936203Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-08-07T18:07:26.2936825Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-08-07T18:07:26.2937446Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-08-07T18:07:26.2938052Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-08-07T18:07:26.2938404Z 2024-08-07T18:07:26.2938533Z Total reclaimed space: 25.15GB 2024-08-07T18:07:26.2971098Z ##[group]Run set +e 2024-08-07T18:07:26.2971409Z set +e 2024-08-07T18:07:26.2971649Z set -x 2024-08-07T18:07:26.2971870Z  2024-08-07T18:07:26.2972095Z nvidia-smi 2024-08-07T18:07:26.2972540Z # NB: Surprisingly, nvidia-smi command returns successfully with return code 0 even in 2024-08-07T18:07:26.2973192Z # the case where the driver has already crashed as it still can get the driver version 2024-08-07T18:07:26.2973823Z # and some basic information like the bus ID. However, the rest of the information 2024-08-07T18:07:26.2974323Z # would be missing (ERR!), for example: 2024-08-07T18:07:26.2974739Z # 2024-08-07T18:07:26.2975042Z # +-----------------------------------------------------------------------------+ 2024-08-07T18:07:26.2975544Z # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | 2024-08-07T18:07:26.2976056Z # |-------------------------------+----------------------+----------------------+ 2024-08-07T18:07:26.2976551Z # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-07T18:07:26.2977089Z # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | 2024-08-07T18:07:26.2977539Z # | | | MIG M. | 2024-08-07T18:07:26.2977892Z # |===============================+======================+======================| 2024-08-07T18:07:26.2978279Z # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | 2024-08-07T18:07:26.2978729Z # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | 2024-08-07T18:07:26.2979148Z # | | | ERR! | 2024-08-07T18:07:26.2979776Z # +-------------------------------+----------------------+----------------------+ 2024-08-07T18:07:26.2980136Z # 2024-08-07T18:07:26.2980437Z # +-----------------------------------------------------------------------------+ 2024-08-07T18:07:26.2980877Z # | Processes: | 2024-08-07T18:07:26.2981328Z # | GPU GI CI PID Type Process name GPU Memory | 2024-08-07T18:07:26.2981760Z # | ID ID Usage | 2024-08-07T18:07:26.2982130Z # |=============================================================================| 2024-08-07T18:07:26.2982533Z # +-----------------------------------------------------------------------------+ 2024-08-07T18:07:26.2982899Z # 2024-08-07T18:07:26.2983281Z # This should be reported as a failure instead as it will guarantee to fail when 2024-08-07T18:07:26.2983766Z # Docker tries to run with --gpus all 2024-08-07T18:07:26.2984071Z # 2024-08-07T18:07:26.2984421Z # So, the correct check here is to query one of the missing piece of info like 2024-08-07T18:07:26.2984908Z # GPU name, so that the command can fail accordingly 2024-08-07T18:07:26.2985364Z nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-07T18:07:26.2985771Z NVIDIA_SMI_STATUS=$? 2024-08-07T18:07:26.2986034Z  2024-08-07T18:07:26.2986450Z # These are acceptable return code from nvidia-smi as copied from setup-nvidia GitHub action 2024-08-07T18:07:26.2987057Z if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then 2024-08-07T18:07:26.2987612Z  echo "NVIDIA driver installation has failed, shutting down the runner..." 2024-08-07T18:07:26.2988087Z  .github/scripts/stop_runner_service.sh 2024-08-07T18:07:26.2988401Z fi 2024-08-07T18:07:26.2988614Z  2024-08-07T18:07:26.2989138Z # For runner with multiple GPUs, we also want to confirm that the number of GPUs are the 2024-08-07T18:07:26.2989725Z # power of 2, i.e. 1, 2, 4, or 8. This is to avoid flaky test issue when one GPU fails 2024-08-07T18:07:26.2990230Z # https://github.com/pytorch/test-infra/issues/4000 2024-08-07T18:07:26.2990663Z GPU_COUNT=$(nvidia-smi --list-gpus | wc -l) 2024-08-07T18:07:26.2991010Z NVIDIA_SMI_STATUS=$? 2024-08-07T18:07:26.2991272Z  2024-08-07T18:07:26.2991687Z # These are acceptable return code from nvidia-smi as copied from setup-nvidia GitHub action 2024-08-07T18:07:26.2992289Z if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then 2024-08-07T18:07:26.2992931Z  echo "NVIDIA driver installation has failed, shutting down the runner..." 2024-08-07T18:07:26.2993418Z  .github/scripts/stop_runner_service.sh 2024-08-07T18:07:26.2993747Z fi 2024-08-07T18:07:26.2993966Z  2024-08-07T18:07:26.2994226Z # Check the GPU count to be a power of 2 2024-08-07T18:07:26.2994752Z if [ "$GPU_COUNT" -le 8 ] && [ "$GPU_COUNT" -ne 1 ] && [ "$GPU_COUNT" -ne 2 ] && [ "$GPU_COUNT" -ne 4 ] && [ "$GPU_COUNT" -ne 8 ]; then 2024-08-07T18:07:26.2995491Z  echo "NVIDIA driver detects $GPU_COUNT GPUs. The runner has a broken GPU, shutting it down..." 2024-08-07T18:07:26.2996031Z  .github/scripts/stop_runner_service.sh 2024-08-07T18:07:26.2996353Z fi 2024-08-07T18:07:26.3008167Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:07:26.3008530Z env: 2024-08-07T18:07:26.3008744Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:07:26.3009071Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:07:26.3009585Z DOCKER_CONTAINER_ID: 17250618960b8a0724ee153a8867e11606fe97d7cecd0b8d3888bb09af1ce52a 2024-08-07T18:07:26.3010138Z RUNNER_WORKSPACE: /home/ec2-user/actions-runner/_work/pytorch 2024-08-07T18:07:26.3010511Z ##[endgroup] 2024-08-07T18:07:26.3040264Z + nvidia-smi 2024-08-07T18:07:26.3274425Z Wed Aug 7 18:07:26 2024 2024-08-07T18:07:26.3274962Z +-----------------------------------------------------------------------------------------+ 2024-08-07T18:07:26.3275620Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-07T18:07:26.3276161Z |-----------------------------------------+------------------------+----------------------+ 2024-08-07T18:07:26.3276662Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-07T18:07:26.3277201Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-07T18:07:26.3277677Z | | | MIG M. | 2024-08-07T18:07:26.3278049Z |=========================================+========================+======================| 2024-08-07T18:07:26.3530733Z | 0 NVIDIA A10G On | 00000000:00:1E.0 Off | 0 | 2024-08-07T18:07:26.3531349Z | 0% 32C P8 16W / 300W | 0MiB / 23028MiB | 0% Default | 2024-08-07T18:07:26.3531743Z | | | N/A | 2024-08-07T18:07:26.3532146Z +-----------------------------------------+------------------------+----------------------+ 2024-08-07T18:07:26.3534702Z 2024-08-07T18:07:26.3535305Z +-----------------------------------------------------------------------------------------+ 2024-08-07T18:07:26.3535749Z | Processes: | 2024-08-07T18:07:26.3536222Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-07T18:07:26.3536991Z | ID ID Usage | 2024-08-07T18:07:26.3537356Z |=========================================================================================| 2024-08-07T18:07:26.3539208Z | No running processes found | 2024-08-07T18:07:26.3539820Z +-----------------------------------------------------------------------------------------+ 2024-08-07T18:07:26.6483611Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-07T18:07:26.6674629Z NVIDIA A10G 2024-08-07T18:07:26.6738380Z + NVIDIA_SMI_STATUS=0 2024-08-07T18:07:26.6738644Z + '[' 0 -ne 0 ']' 2024-08-07T18:07:26.6745238Z ++ nvidia-smi --list-gpus 2024-08-07T18:07:26.6746866Z ++ wc -l 2024-08-07T18:07:26.7002189Z + GPU_COUNT=1 2024-08-07T18:07:26.7003027Z + NVIDIA_SMI_STATUS=0 2024-08-07T18:07:26.7003333Z + '[' 0 -ne 0 ']' 2024-08-07T18:07:26.7003557Z + '[' 1 -le 8 ']' 2024-08-07T18:07:26.7003779Z + '[' 1 -ne 1 ']' 2024-08-07T18:07:26.7061495Z Post job cleanup. 2024-08-07T18:07:26.7107580Z Post job cleanup. 2024-08-07T18:07:26.7996027Z [command]/usr/bin/git version 2024-08-07T18:07:26.8050771Z git version 2.40.1 2024-08-07T18:07:26.8087913Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/28070ddc-04d2-499c-906d-c177142bda76' before making global git config changes 2024-08-07T18:07:26.8088789Z Adding repository directory to the temporary git global config as a safe directory 2024-08-07T18:07:26.8093246Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T18:07:26.8142292Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-07T18:07:26.8186839Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-07T18:07:26.8562897Z Entering 'android/libs/fbjni' 2024-08-07T18:07:26.8633110Z Entering 'third_party/FP16' 2024-08-07T18:07:26.8700932Z Entering 'third_party/FXdiv' 2024-08-07T18:07:26.8769168Z Entering 'third_party/NNPACK' 2024-08-07T18:07:26.8841945Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T18:07:26.8913722Z Entering 'third_party/XNNPACK' 2024-08-07T18:07:26.9002056Z Entering 'third_party/benchmark' 2024-08-07T18:07:26.9070354Z Entering 'third_party/cpp-httplib' 2024-08-07T18:07:26.9137325Z Entering 'third_party/cpuinfo' 2024-08-07T18:07:26.9205452Z Entering 'third_party/cudnn_frontend' 2024-08-07T18:07:26.9273097Z Entering 'third_party/cutlass' 2024-08-07T18:07:26.9349843Z Entering 'third_party/eigen' 2024-08-07T18:07:26.9421009Z Entering 'third_party/fbgemm' 2024-08-07T18:07:26.9489459Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T18:07:26.9555009Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T18:07:26.9622495Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T18:07:26.9695877Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T18:07:26.9760992Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T18:07:26.9829307Z Entering 'third_party/flatbuffers' 2024-08-07T18:07:26.9900817Z Entering 'third_party/fmt' 2024-08-07T18:07:26.9968881Z Entering 'third_party/foxi' 2024-08-07T18:07:27.0052571Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T18:07:27.0109558Z Entering 'third_party/gloo' 2024-08-07T18:07:27.0177461Z Entering 'third_party/googletest' 2024-08-07T18:07:27.0247507Z Entering 'third_party/ideep' 2024-08-07T18:07:27.0313989Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T18:07:27.0388015Z Entering 'third_party/ittapi' 2024-08-07T18:07:27.0455181Z Entering 'third_party/kineto' 2024-08-07T18:07:27.0517799Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T18:07:27.0579302Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T18:07:27.0645850Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T18:07:27.0715944Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T18:07:27.0785768Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T18:07:27.0853131Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T18:07:27.0925839Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T18:07:27.0992925Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T18:07:27.1065835Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T18:07:27.1188424Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T18:07:27.1212950Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T18:07:27.1276773Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T18:07:27.1344736Z Entering 'third_party/mimalloc' 2024-08-07T18:07:27.1411233Z Entering 'third_party/nccl/nccl' 2024-08-07T18:07:27.1475443Z Entering 'third_party/nlohmann' 2024-08-07T18:07:27.1547621Z Entering 'third_party/onnx' 2024-08-07T18:07:27.1632251Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T18:07:27.1701184Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T18:07:27.1773759Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T18:07:27.1842248Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T18:07:27.1908728Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T18:07:27.1974548Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T18:07:27.2040520Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T18:07:27.2113115Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T18:07:27.2178669Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T18:07:27.2245005Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T18:07:27.2308895Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T18:07:27.2376880Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T18:07:27.2448106Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T18:07:27.2537649Z Entering 'third_party/pocketfft' 2024-08-07T18:07:27.2605862Z Entering 'third_party/protobuf' 2024-08-07T18:07:27.2674733Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T18:07:27.2741905Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T18:07:27.2811598Z Entering 'third_party/psimd' 2024-08-07T18:07:27.2879739Z Entering 'third_party/pthreadpool' 2024-08-07T18:07:27.2949751Z Entering 'third_party/pybind11' 2024-08-07T18:07:27.3018083Z Entering 'third_party/python-peachpy' 2024-08-07T18:07:27.3085959Z Entering 'third_party/sleef' 2024-08-07T18:07:27.3159519Z Entering 'third_party/tensorpipe' 2024-08-07T18:07:27.3228739Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T18:07:27.3294209Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T18:07:27.3360180Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T18:07:27.3426640Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T18:07:27.3490196Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T18:07:27.3586656Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-07T18:07:27.3627369Z http.https://github.com/.extraheader 2024-08-07T18:07:27.3637921Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-07T18:07:27.3686543Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-07T18:07:27.4055284Z Entering 'android/libs/fbjni' 2024-08-07T18:07:27.4102974Z http.https://github.com/.extraheader 2024-08-07T18:07:27.4148316Z Entering 'third_party/FP16' 2024-08-07T18:07:27.4194053Z http.https://github.com/.extraheader 2024-08-07T18:07:27.4237827Z Entering 'third_party/FXdiv' 2024-08-07T18:07:27.4281298Z http.https://github.com/.extraheader 2024-08-07T18:07:27.4325548Z Entering 'third_party/NNPACK' 2024-08-07T18:07:27.4369231Z http.https://github.com/.extraheader 2024-08-07T18:07:27.4413722Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T18:07:27.4457117Z http.https://github.com/.extraheader 2024-08-07T18:07:27.4500876Z Entering 'third_party/XNNPACK' 2024-08-07T18:07:27.4548417Z http.https://github.com/.extraheader 2024-08-07T18:07:27.4609859Z Entering 'third_party/benchmark' 2024-08-07T18:07:27.4654982Z http.https://github.com/.extraheader 2024-08-07T18:07:27.4703424Z Entering 'third_party/cpp-httplib' 2024-08-07T18:07:27.4747624Z http.https://github.com/.extraheader 2024-08-07T18:07:27.4791159Z Entering 'third_party/cpuinfo' 2024-08-07T18:07:27.4836413Z http.https://github.com/.extraheader 2024-08-07T18:07:27.4881247Z Entering 'third_party/cudnn_frontend' 2024-08-07T18:07:27.4926710Z http.https://github.com/.extraheader 2024-08-07T18:07:27.4970736Z Entering 'third_party/cutlass' 2024-08-07T18:07:27.5016232Z http.https://github.com/.extraheader 2024-08-07T18:07:27.5068225Z Entering 'third_party/eigen' 2024-08-07T18:07:27.5112493Z http.https://github.com/.extraheader 2024-08-07T18:07:27.5158160Z Entering 'third_party/fbgemm' 2024-08-07T18:07:27.5203479Z http.https://github.com/.extraheader 2024-08-07T18:07:27.5247370Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T18:07:27.5289763Z http.https://github.com/.extraheader 2024-08-07T18:07:27.5332767Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T18:07:27.5376093Z http.https://github.com/.extraheader 2024-08-07T18:07:27.5422381Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T18:07:27.5464120Z http.https://github.com/.extraheader 2024-08-07T18:07:27.5514297Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T18:07:27.5557638Z http.https://github.com/.extraheader 2024-08-07T18:07:27.5601036Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T18:07:27.5642747Z http.https://github.com/.extraheader 2024-08-07T18:07:27.5688537Z Entering 'third_party/flatbuffers' 2024-08-07T18:07:27.5733798Z http.https://github.com/.extraheader 2024-08-07T18:07:27.5780033Z Entering 'third_party/fmt' 2024-08-07T18:07:27.5824996Z http.https://github.com/.extraheader 2024-08-07T18:07:27.5868983Z Entering 'third_party/foxi' 2024-08-07T18:07:27.5913599Z http.https://github.com/.extraheader 2024-08-07T18:07:27.5957634Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T18:07:27.6004110Z http.https://github.com/.extraheader 2024-08-07T18:07:27.6048575Z Entering 'third_party/gloo' 2024-08-07T18:07:27.6092662Z http.https://github.com/.extraheader 2024-08-07T18:07:27.6137067Z Entering 'third_party/googletest' 2024-08-07T18:07:27.6180264Z http.https://github.com/.extraheader 2024-08-07T18:07:27.6224248Z Entering 'third_party/ideep' 2024-08-07T18:07:27.6267560Z http.https://github.com/.extraheader 2024-08-07T18:07:27.6309275Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T18:07:27.6350880Z http.https://github.com/.extraheader 2024-08-07T18:07:27.6403387Z Entering 'third_party/ittapi' 2024-08-07T18:07:27.6449132Z http.https://github.com/.extraheader 2024-08-07T18:07:27.6493180Z Entering 'third_party/kineto' 2024-08-07T18:07:27.6538594Z http.https://github.com/.extraheader 2024-08-07T18:07:27.6582432Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T18:07:27.6624620Z http.https://github.com/.extraheader 2024-08-07T18:07:27.6666752Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T18:07:27.6710681Z http.https://github.com/.extraheader 2024-08-07T18:07:27.6755621Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T18:07:27.6798593Z http.https://github.com/.extraheader 2024-08-07T18:07:27.6845059Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T18:07:27.6888467Z http.https://github.com/.extraheader 2024-08-07T18:07:27.6932922Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T18:07:27.6975851Z http.https://github.com/.extraheader 2024-08-07T18:07:27.7018451Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T18:07:27.7061705Z http.https://github.com/.extraheader 2024-08-07T18:07:27.7109768Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T18:07:27.7152929Z http.https://github.com/.extraheader 2024-08-07T18:07:27.7196891Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T18:07:27.7240162Z http.https://github.com/.extraheader 2024-08-07T18:07:27.7284368Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T18:07:27.7328168Z http.https://github.com/.extraheader 2024-08-07T18:07:27.7373243Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T18:07:27.7422384Z http.https://github.com/.extraheader 2024-08-07T18:07:27.7469681Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T18:07:27.7513307Z http.https://github.com/.extraheader 2024-08-07T18:07:27.7556801Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T18:07:27.7605869Z http.https://github.com/.extraheader 2024-08-07T18:07:27.7652120Z Entering 'third_party/mimalloc' 2024-08-07T18:07:27.7696632Z http.https://github.com/.extraheader 2024-08-07T18:07:27.7743189Z Entering 'third_party/nccl/nccl' 2024-08-07T18:07:27.7786532Z http.https://github.com/.extraheader 2024-08-07T18:07:27.7834040Z Entering 'third_party/nlohmann' 2024-08-07T18:07:27.7877848Z http.https://github.com/.extraheader 2024-08-07T18:07:27.7924806Z Entering 'third_party/onnx' 2024-08-07T18:07:27.7969275Z http.https://github.com/.extraheader 2024-08-07T18:07:27.8029773Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T18:07:27.8072792Z http.https://github.com/.extraheader 2024-08-07T18:07:27.8116525Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T18:07:27.8160581Z http.https://github.com/.extraheader 2024-08-07T18:07:27.8208859Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T18:07:27.8253467Z http.https://github.com/.extraheader 2024-08-07T18:07:27.8297873Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T18:07:27.8340395Z http.https://github.com/.extraheader 2024-08-07T18:07:27.8382534Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T18:07:27.8426047Z http.https://github.com/.extraheader 2024-08-07T18:07:27.8468281Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T18:07:27.8511159Z http.https://github.com/.extraheader 2024-08-07T18:07:27.8552780Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T18:07:27.8595313Z http.https://github.com/.extraheader 2024-08-07T18:07:27.8639506Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T18:07:27.8682139Z http.https://github.com/.extraheader 2024-08-07T18:07:27.8724936Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T18:07:27.8767254Z http.https://github.com/.extraheader 2024-08-07T18:07:27.8809688Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T18:07:27.8851921Z http.https://github.com/.extraheader 2024-08-07T18:07:27.8892794Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T18:07:27.8936480Z http.https://github.com/.extraheader 2024-08-07T18:07:27.8981960Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T18:07:27.9026490Z http.https://github.com/.extraheader 2024-08-07T18:07:27.9073401Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T18:07:27.9116787Z http.https://github.com/.extraheader 2024-08-07T18:07:27.9181665Z Entering 'third_party/pocketfft' 2024-08-07T18:07:27.9226860Z http.https://github.com/.extraheader 2024-08-07T18:07:27.9271242Z Entering 'third_party/protobuf' 2024-08-07T18:07:27.9316772Z http.https://github.com/.extraheader 2024-08-07T18:07:27.9362481Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T18:07:27.9406393Z http.https://github.com/.extraheader 2024-08-07T18:07:27.9450490Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T18:07:27.9495549Z http.https://github.com/.extraheader 2024-08-07T18:07:27.9545386Z Entering 'third_party/psimd' 2024-08-07T18:07:27.9589935Z http.https://github.com/.extraheader 2024-08-07T18:07:27.9634674Z Entering 'third_party/pthreadpool' 2024-08-07T18:07:27.9678305Z http.https://github.com/.extraheader 2024-08-07T18:07:27.9722236Z Entering 'third_party/pybind11' 2024-08-07T18:07:27.9765865Z http.https://github.com/.extraheader 2024-08-07T18:07:27.9809703Z Entering 'third_party/python-peachpy' 2024-08-07T18:07:27.9853874Z http.https://github.com/.extraheader 2024-08-07T18:07:27.9897240Z Entering 'third_party/sleef' 2024-08-07T18:07:27.9940940Z http.https://github.com/.extraheader 2024-08-07T18:07:27.9984141Z Entering 'third_party/tensorpipe' 2024-08-07T18:07:28.0029050Z http.https://github.com/.extraheader 2024-08-07T18:07:28.0072199Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T18:07:28.0115750Z http.https://github.com/.extraheader 2024-08-07T18:07:28.0160537Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T18:07:28.0204815Z http.https://github.com/.extraheader 2024-08-07T18:07:28.0247251Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T18:07:28.0296610Z http.https://github.com/.extraheader 2024-08-07T18:07:28.0340569Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T18:07:28.0383128Z http.https://github.com/.extraheader 2024-08-07T18:07:28.0424686Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T18:07:28.0468128Z http.https://github.com/.extraheader 2024-08-07T18:07:28.0625850Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-07T18:07:28.0650897Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-07T18:07:28.0658407Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:07:28.0658783Z ##[endgroup] 2024-08-07T18:07:34.9437062Z Cleaning up orphan processes