2024-08-07T17:48:11.1398790Z Current runner version: '2.318.0' 2024-08-07T17:48:11.1408478Z Runner name: 'i-0d5f0f54c385d63e2' 2024-08-07T17:48:11.1409700Z Runner group name: 'Default' 2024-08-07T17:48:11.1411034Z Machine name: 'ip-10-0-41-236' 2024-08-07T17:48:11.1417454Z ##[group]GITHUB_TOKEN Permissions 2024-08-07T17:48:11.1420849Z Actions: read 2024-08-07T17:48:11.1421780Z Attestations: read 2024-08-07T17:48:11.1422645Z Checks: read 2024-08-07T17:48:11.1423583Z Contents: read 2024-08-07T17:48:11.1424444Z Deployments: read 2024-08-07T17:48:11.1425309Z Discussions: read 2024-08-07T17:48:11.1426282Z Issues: read 2024-08-07T17:48:11.1427130Z Metadata: read 2024-08-07T17:48:11.1427975Z Packages: read 2024-08-07T17:48:11.1428866Z Pages: read 2024-08-07T17:48:11.1429739Z PullRequests: read 2024-08-07T17:48:11.1430643Z RepositoryProjects: read 2024-08-07T17:48:11.1431640Z SecurityEvents: read 2024-08-07T17:48:11.1432538Z Statuses: read 2024-08-07T17:48:11.1433374Z ##[endgroup] 2024-08-07T17:48:11.1437881Z Secret source: Actions 2024-08-07T17:48:11.1439296Z Prepare workflow directory 2024-08-07T17:48:11.2140270Z Prepare all required actions 2024-08-07T17:48:11.2192439Z Getting action download info 2024-08-07T17:48:11.4129038Z Download action repository 'pytorch/test-infra@main' (SHA:a1f5a89251fc4258ab59806094fe3108f7d6741a) 2024-08-07T17:48:12.9806218Z Download action repository 'pytorch/pytorch@main' (SHA:a62710c82039b798befd86e938d2137af3978c93) 2024-08-07T17:48:25.5919574Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-07T17:48:25.7819226Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-07T17:48:26.0903646Z Getting action download info 2024-08-07T17:48:26.2000279Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-07T17:48:26.5149686Z Getting action download info 2024-08-07T17:48:26.6389080Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-07T17:48:26.8222616Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/pull/131248/merge (f779f6b7738020e244184bded4026b37de3f9f24) 2024-08-07T17:48:26.8225423Z ##[group] Inputs 2024-08-07T17:48:26.8225895Z build-environment: linux-focal-cuda12.1-py3.10-gcc9 2024-08-07T17:48:26.8228243Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}]} 2024-08-07T17:48:26.8231007Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:48:26.8231996Z sync-tag: 2024-08-07T17:48:26.8233087Z timeout-minutes: 360 2024-08-07T17:48:26.8233465Z use-gha: 2024-08-07T17:48:26.8233770Z dashboard-tag: 2024-08-07T17:48:26.8234075Z s3-bucket: gha-artifacts 2024-08-07T17:48:26.8234437Z aws-role-to-assume: 2024-08-07T17:48:26.8234822Z ##[endgroup] 2024-08-07T17:48:26.8235493Z Complete job name: linux-focal-cuda12.1-py3.10-gcc9 / test (default, 1, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-07T17:48:26.8866381Z A job started hook has been configured by the self-hosted runner administrator 2024-08-07T17:48:26.9020688Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-07T17:48:26.9031939Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:48:26.9032986Z ##[endgroup] 2024-08-07T17:48:28.2089157Z Runner Type: amz2023.linux.4xlarge.nvidia.gpu 2024-08-07T17:48:28.2089789Z Instance Type: g3.4xlarge 2024-08-07T17:48:28.2090704Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-07T17:48:28.2091184Z AMI ID: ami-06c68f701d8090592 2024-08-07T17:48:35.4167367Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-07T17:48:35.4167978Z with: 2024-08-07T17:48:35.4168787Z github-secret: *** 2024-08-07T17:48:35.4169692Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-07T17:48:35.4170635Z activate-with-label: false 2024-08-07T17:48:35.4171006Z label: with-ssh 2024-08-07T17:48:35.4171613Z remove-existing-keys: true 2024-08-07T17:48:35.4171982Z fail-silently: true 2024-08-07T17:48:35.4172290Z env: 2024-08-07T17:48:35.4172575Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:48:35.4172917Z ##[endgroup] 2024-08-07T17:48:35.5310724Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-07T17:48:35.8607549Z Grabbing public ssh keys from https://github.com/zdevito.keys 2024-08-07T17:48:35.9336307Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-07T17:48:35.9358414Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-07T17:48:35.9401505Z Login using: ssh ec2-user@ec2-3-86-110-44.compute-1.amazonaws.com 2024-08-07T17:48:35.9402538Z All testing is done inside the container, to start an interactive session run: 2024-08-07T17:48:35.9403392Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-07T17:48:35.9540459Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-07T17:48:35.9541063Z with: 2024-08-07T17:48:35.9541356Z submodules: recursive 2024-08-07T17:48:35.9541724Z fetch-depth: 0 2024-08-07T17:48:35.9542037Z env: 2024-08-07T17:48:35.9542313Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:48:35.9542664Z ##[endgroup] 2024-08-07T17:48:35.9649768Z ##[group]Run retry () { 2024-08-07T17:48:35.9650166Z retry () { 2024-08-07T17:48:35.9650625Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-07T17:48:35.9651130Z } 2024-08-07T17:48:35.9651448Z echo "${GITHUB_WORKSPACE}" 2024-08-07T17:48:35.9651844Z if [ -z "${NO_SUDO}" ]; then 2024-08-07T17:48:35.9652292Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-07T17:48:35.9652725Z else 2024-08-07T17:48:35.9653052Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-07T17:48:35.9653463Z fi 2024-08-07T17:48:35.9653826Z mkdir "${GITHUB_WORKSPACE}" 2024-08-07T17:48:35.9664301Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:48:35.9664801Z env: 2024-08-07T17:48:35.9665079Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:48:35.9665429Z NO_SUDO: 2024-08-07T17:48:35.9665720Z ##[endgroup] 2024-08-07T17:48:35.9696353Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:48:36.1207582Z ##[group]Run malfet/checkout@silent-checkout 2024-08-07T17:48:36.1208041Z with: 2024-08-07T17:48:36.1208357Z ref: 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:48:36.1208788Z fetch-depth: 0 2024-08-07T17:48:36.1209113Z submodules: recursive 2024-08-07T17:48:36.1209446Z quiet-checkout: true 2024-08-07T17:48:36.1209806Z repository: pytorch/pytorch 2024-08-07T17:48:36.1210315Z token: *** 2024-08-07T17:48:36.1210628Z ssh-strict: true 2024-08-07T17:48:36.1210943Z persist-credentials: true 2024-08-07T17:48:36.1211305Z clean: true 2024-08-07T17:48:36.1211631Z sparse-checkout-cone-mode: true 2024-08-07T17:48:36.1212024Z lfs: false 2024-08-07T17:48:36.1212335Z set-safe-directory: true 2024-08-07T17:48:36.1212669Z env: 2024-08-07T17:48:36.1212964Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:48:36.1213300Z ##[endgroup] 2024-08-07T17:48:36.2531022Z Syncing repository: pytorch/pytorch 2024-08-07T17:48:36.2532820Z ##[group]Getting Git version info 2024-08-07T17:48:36.2533460Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-07T17:48:36.2534622Z [command]/usr/bin/git version 2024-08-07T17:48:36.2535051Z git version 2.40.1 2024-08-07T17:48:36.2564236Z ##[endgroup] 2024-08-07T17:48:36.2585556Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/ebcdf845-3cc3-447d-ae6d-4fde3e8ff676' before making global git config changes 2024-08-07T17:48:36.2586804Z Adding repository directory to the temporary git global config as a safe directory 2024-08-07T17:48:36.2592532Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:48:36.2639526Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-07T17:48:36.2645551Z ##[group]Initializing the repository 2024-08-07T17:48:36.2649454Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:48:36.2683912Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-07T17:48:36.2684652Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-07T17:48:36.2685386Z hint: of your new repositories, which will suppress this warning, call: 2024-08-07T17:48:36.2685911Z hint: 2024-08-07T17:48:36.2686460Z hint: git config --global init.defaultBranch 2024-08-07T17:48:36.2686940Z hint: 2024-08-07T17:48:36.2687376Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-07T17:48:36.2688073Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-07T17:48:36.2688878Z hint: 2024-08-07T17:48:36.2689228Z hint: git branch -m 2024-08-07T17:48:36.2689881Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-07T17:48:36.2699633Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-07T17:48:36.2735995Z ##[endgroup] 2024-08-07T17:48:36.2736553Z ##[group]Disabling automatic garbage collection 2024-08-07T17:48:36.2740379Z [command]/usr/bin/git config --local gc.auto 0 2024-08-07T17:48:36.2776630Z ##[endgroup] 2024-08-07T17:48:36.2777167Z ##[group]Setting up auth 2024-08-07T17:48:36.2785898Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-07T17:48:36.2822437Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-07T17:48:36.3168080Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-07T17:48:36.3203420Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-07T17:48:36.3530595Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-07T17:48:36.3583873Z ##[endgroup] 2024-08-07T17:48:36.3584478Z ##[group]Fetching the repository 2024-08-07T17:48:36.3592678Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-07T17:48:38.7145405Z remote: Enumerating objects: 1008246 2024-08-07T17:48:38.7146037Z remote: Enumerating objects: 1011035, done. 2024-08-07T17:48:38.7147956Z remote: Counting objects: 0% (1/2789) 2024-08-07T17:48:38.7149576Z remote: Counting objects: 1% (28/2789) 2024-08-07T17:48:38.7150088Z remote: Counting objects: 2% (56/2789) 2024-08-07T17:48:38.7150583Z remote: Counting objects: 3% (84/2789) 2024-08-07T17:48:38.7151016Z remote: Counting objects: 4% (112/2789) 2024-08-07T17:48:38.7152945Z remote: Counting objects: 5% (140/2789) 2024-08-07T17:48:38.7156442Z remote: Counting objects: 6% (168/2789) 2024-08-07T17:48:38.7157218Z remote: Counting objects: 7% (196/2789) 2024-08-07T17:48:38.7158056Z remote: Counting objects: 8% (224/2789) 2024-08-07T17:48:38.7158689Z remote: Counting objects: 9% (252/2789) 2024-08-07T17:48:38.7159232Z remote: Counting objects: 10% (279/2789) 2024-08-07T17:48:38.7159702Z remote: Counting objects: 11% (307/2789) 2024-08-07T17:48:38.7160171Z remote: Counting objects: 12% (335/2789) 2024-08-07T17:48:38.7160654Z remote: Counting objects: 13% (363/2789) 2024-08-07T17:48:38.7161111Z remote: Counting objects: 14% (391/2789) 2024-08-07T17:48:38.7161605Z remote: Counting objects: 15% (419/2789) 2024-08-07T17:48:38.7163242Z remote: Counting objects: 16% (447/2789) 2024-08-07T17:48:38.7163693Z remote: Counting objects: 17% (475/2789) 2024-08-07T17:48:38.7164461Z remote: Counting objects: 18% (503/2789) 2024-08-07T17:48:38.7164934Z remote: Counting objects: 19% (530/2789) 2024-08-07T17:48:38.7165612Z remote: Counting objects: 20% (558/2789) 2024-08-07T17:48:38.7166097Z remote: Counting objects: 21% (586/2789) 2024-08-07T17:48:38.7166564Z remote: Counting objects: 22% (614/2789) 2024-08-07T17:48:38.7167013Z remote: Counting objects: 23% (642/2789) 2024-08-07T17:48:38.7167476Z remote: Counting objects: 24% (670/2789) 2024-08-07T17:48:38.7167939Z remote: Counting objects: 25% (698/2789) 2024-08-07T17:48:38.7168395Z remote: Counting objects: 26% (726/2789) 2024-08-07T17:48:38.7168966Z remote: Counting objects: 27% (754/2789) 2024-08-07T17:48:38.7169687Z remote: Counting objects: 28% (781/2789) 2024-08-07T17:48:38.7170196Z remote: Counting objects: 29% (809/2789) 2024-08-07T17:48:38.7170652Z remote: Counting objects: 30% (837/2789) 2024-08-07T17:48:38.7171132Z remote: Counting objects: 31% (865/2789) 2024-08-07T17:48:38.7171587Z remote: Counting objects: 32% (893/2789) 2024-08-07T17:48:38.7172036Z remote: Counting objects: 33% (921/2789) 2024-08-07T17:48:38.7172519Z remote: Counting objects: 34% (949/2789) 2024-08-07T17:48:38.7172983Z remote: Counting objects: 35% (977/2789) 2024-08-07T17:48:38.7173432Z remote: Counting objects: 36% (1005/2789) 2024-08-07T17:48:38.7173902Z remote: Counting objects: 37% (1032/2789) 2024-08-07T17:48:38.7174371Z remote: Counting objects: 38% (1060/2789) 2024-08-07T17:48:38.7174816Z remote: Counting objects: 39% (1088/2789) 2024-08-07T17:48:38.7175290Z remote: Counting objects: 40% (1116/2789) 2024-08-07T17:48:38.7175766Z remote: Counting objects: 41% (1144/2789) 2024-08-07T17:48:38.7176213Z remote: Counting objects: 42% (1172/2789) 2024-08-07T17:48:38.7176681Z remote: Counting objects: 43% (1200/2789) 2024-08-07T17:48:38.7177145Z remote: Counting objects: 44% (1228/2789) 2024-08-07T17:48:38.7177592Z remote: Counting objects: 45% (1256/2789) 2024-08-07T17:48:38.7178098Z remote: Counting objects: 46% (1283/2789) 2024-08-07T17:48:38.7178561Z remote: Counting objects: 47% (1311/2789) 2024-08-07T17:48:38.7179012Z remote: Counting objects: 48% (1339/2789) 2024-08-07T17:48:38.7179491Z remote: Counting objects: 49% (1367/2789) 2024-08-07T17:48:38.7179952Z remote: Counting objects: 50% (1395/2789) 2024-08-07T17:48:38.7180647Z remote: Counting objects: 51% (1423/2789) 2024-08-07T17:48:38.7181144Z remote: Counting objects: 52% (1451/2789) 2024-08-07T17:48:38.7181641Z remote: Counting objects: 53% (1479/2789) 2024-08-07T17:48:38.7182102Z remote: Counting objects: 54% (1507/2789) 2024-08-07T17:48:38.7182597Z remote: Counting objects: 55% (1534/2789) 2024-08-07T17:48:38.7183064Z remote: Counting objects: 56% (1562/2789) 2024-08-07T17:48:38.7183532Z remote: Counting objects: 57% (1590/2789) 2024-08-07T17:48:38.7183984Z remote: Counting objects: 58% (1618/2789) 2024-08-07T17:48:38.7184453Z remote: Counting objects: 59% (1646/2789) 2024-08-07T17:48:38.7185063Z remote: Counting objects: 60% (1674/2789) 2024-08-07T17:48:38.7185516Z remote: Counting objects: 61% (1702/2789) 2024-08-07T17:48:38.7185983Z remote: Counting objects: 62% (1730/2789) 2024-08-07T17:48:38.7186497Z remote: Counting objects: 63% (1758/2789) 2024-08-07T17:48:38.7187207Z remote: Counting objects: 64% (1785/2789) 2024-08-07T17:48:38.7187681Z remote: Counting objects: 65% (1813/2789) 2024-08-07T17:48:38.7188162Z remote: Counting objects: 66% (1841/2789) 2024-08-07T17:48:38.7188634Z remote: Counting objects: 67% (1869/2789) 2024-08-07T17:48:38.7189086Z remote: Counting objects: 68% (1897/2789) 2024-08-07T17:48:38.7189553Z remote: Counting objects: 69% (1925/2789) 2024-08-07T17:48:38.7190028Z remote: Counting objects: 70% (1953/2789) 2024-08-07T17:48:38.7190477Z remote: Counting objects: 71% (1981/2789) 2024-08-07T17:48:38.7190942Z remote: Counting objects: 72% (2009/2789) 2024-08-07T17:48:38.7191419Z remote: Counting objects: 73% (2036/2789) 2024-08-07T17:48:38.7191871Z remote: Counting objects: 74% (2064/2789) 2024-08-07T17:48:38.7192350Z remote: Counting objects: 75% (2092/2789) 2024-08-07T17:48:38.7192824Z remote: Counting objects: 76% (2120/2789) 2024-08-07T17:48:38.7193267Z remote: Counting objects: 77% (2148/2789) 2024-08-07T17:48:38.7193757Z remote: Counting objects: 78% (2176/2789) 2024-08-07T17:48:38.7194338Z remote: Counting objects: 79% (2204/2789) 2024-08-07T17:48:38.7194808Z remote: Counting objects: 80% (2232/2789) 2024-08-07T17:48:38.7195273Z remote: Counting objects: 81% (2260/2789) 2024-08-07T17:48:38.7195745Z remote: Counting objects: 82% (2287/2789) 2024-08-07T17:48:38.7196191Z remote: Counting objects: 83% (2315/2789) 2024-08-07T17:48:38.7196658Z remote: Counting objects: 84% (2343/2789) 2024-08-07T17:48:38.7197475Z remote: Counting objects: 85% (2371/2789) 2024-08-07T17:48:38.7197928Z remote: Counting objects: 86% (2399/2789) 2024-08-07T17:48:38.7198396Z remote: Counting objects: 87% (2427/2789) 2024-08-07T17:48:38.7198865Z remote: Counting objects: 88% (2455/2789) 2024-08-07T17:48:38.7199314Z remote: Counting objects: 89% (2483/2789) 2024-08-07T17:48:38.7199780Z remote: Counting objects: 90% (2511/2789) 2024-08-07T17:48:38.7200251Z remote: Counting objects: 91% (2538/2789) 2024-08-07T17:48:38.7200712Z remote: Counting objects: 92% (2566/2789) 2024-08-07T17:48:38.7201183Z remote: Counting objects: 93% (2594/2789) 2024-08-07T17:48:38.7201656Z remote: Counting objects: 94% (2622/2789) 2024-08-07T17:48:38.7202103Z remote: Counting objects: 95% (2650/2789) 2024-08-07T17:48:38.7202572Z remote: Counting objects: 96% (2678/2789) 2024-08-07T17:48:38.7203047Z remote: Counting objects: 97% (2706/2789) 2024-08-07T17:48:38.7203510Z remote: Counting objects: 98% (2734/2789) 2024-08-07T17:48:38.7203977Z remote: Counting objects: 99% (2762/2789) 2024-08-07T17:48:38.7204446Z remote: Counting objects: 100% (2789/2789) 2024-08-07T17:48:38.7204951Z remote: Counting objects: 100% (2789/2789), done. 2024-08-07T17:48:38.7572868Z remote: Compressing objects: 0% (1/1486) 2024-08-07T17:48:38.7765842Z remote: Compressing objects: 1% (15/1486) 2024-08-07T17:48:38.9002196Z remote: Compressing objects: 2% (30/1486) 2024-08-07T17:48:39.0923423Z remote: Compressing objects: 3% (45/1486) 2024-08-07T17:48:39.2083718Z remote: Compressing objects: 4% (60/1486) 2024-08-07T17:48:39.2830110Z remote: Compressing objects: 5% (75/1486) 2024-08-07T17:48:39.3480876Z remote: Compressing objects: 6% (90/1486) 2024-08-07T17:48:39.3946096Z remote: Compressing objects: 7% (105/1486) 2024-08-07T17:48:39.4420181Z remote: Compressing objects: 8% (119/1486) 2024-08-07T17:48:39.4719037Z remote: Compressing objects: 9% (134/1486) 2024-08-07T17:48:39.4995416Z remote: Compressing objects: 10% (149/1486) 2024-08-07T17:48:39.5226302Z remote: Compressing objects: 11% (164/1486) 2024-08-07T17:48:39.5442535Z remote: Compressing objects: 12% (179/1486) 2024-08-07T17:48:39.5633662Z remote: Compressing objects: 13% (194/1486) 2024-08-07T17:48:39.5779759Z remote: Compressing objects: 14% (209/1486) 2024-08-07T17:48:39.5906454Z remote: Compressing objects: 15% (223/1486) 2024-08-07T17:48:39.6003798Z remote: Compressing objects: 16% (238/1486) 2024-08-07T17:48:39.6076224Z remote: Compressing objects: 17% (253/1486) 2024-08-07T17:48:39.6129711Z remote: Compressing objects: 18% (268/1486) 2024-08-07T17:48:39.6171928Z remote: Compressing objects: 19% (283/1486) 2024-08-07T17:48:39.6196795Z remote: Compressing objects: 20% (298/1486) 2024-08-07T17:48:39.6209868Z remote: Compressing objects: 21% (313/1486) 2024-08-07T17:48:39.6216020Z remote: Compressing objects: 22% (327/1486) 2024-08-07T17:48:39.6247226Z remote: Compressing objects: 23% (342/1486) 2024-08-07T17:48:39.6270930Z remote: Compressing objects: 24% (357/1486) 2024-08-07T17:48:39.6293850Z remote: Compressing objects: 25% (372/1486) 2024-08-07T17:48:39.6302859Z remote: Compressing objects: 26% (387/1486) 2024-08-07T17:48:39.6326053Z remote: Compressing objects: 27% (402/1486) 2024-08-07T17:48:39.6341199Z remote: Compressing objects: 28% (417/1486) 2024-08-07T17:48:39.6354578Z remote: Compressing objects: 29% (431/1486) 2024-08-07T17:48:39.6379070Z remote: Compressing objects: 30% (446/1486) 2024-08-07T17:48:39.6405858Z remote: Compressing objects: 31% (461/1486) 2024-08-07T17:48:39.6420219Z remote: Compressing objects: 32% (476/1486) 2024-08-07T17:48:39.6432089Z remote: Compressing objects: 33% (491/1486) 2024-08-07T17:48:39.6444785Z remote: Compressing objects: 34% (506/1486) 2024-08-07T17:48:39.6454636Z remote: Compressing objects: 35% (521/1486) 2024-08-07T17:48:39.6465650Z remote: Compressing objects: 36% (535/1486) 2024-08-07T17:48:39.6473189Z remote: Compressing objects: 37% (550/1486) 2024-08-07T17:48:39.6485401Z remote: Compressing objects: 38% (565/1486) 2024-08-07T17:48:39.6496603Z remote: Compressing objects: 39% (580/1486) 2024-08-07T17:48:39.6504461Z remote: Compressing objects: 40% (595/1486) 2024-08-07T17:48:39.6514790Z remote: Compressing objects: 41% (610/1486) 2024-08-07T17:48:39.6522010Z remote: Compressing objects: 42% (625/1486) 2024-08-07T17:48:39.6532627Z remote: Compressing objects: 43% (639/1486) 2024-08-07T17:48:39.6538091Z remote: Compressing objects: 44% (654/1486) 2024-08-07T17:48:39.6545476Z remote: Compressing objects: 45% (669/1486) 2024-08-07T17:48:39.6550761Z remote: Compressing objects: 46% (684/1486) 2024-08-07T17:48:39.6557512Z remote: Compressing objects: 47% (699/1486) 2024-08-07T17:48:39.6564253Z remote: Compressing objects: 48% (714/1486) 2024-08-07T17:48:39.6568314Z remote: Compressing objects: 49% (729/1486) 2024-08-07T17:48:39.6573358Z remote: Compressing objects: 50% (743/1486) 2024-08-07T17:48:39.6577805Z remote: Compressing objects: 51% (758/1486) 2024-08-07T17:48:39.6582328Z remote: Compressing objects: 52% (773/1486) 2024-08-07T17:48:39.6585984Z remote: Compressing objects: 53% (788/1486) 2024-08-07T17:48:39.6588770Z remote: Compressing objects: 54% (803/1486) 2024-08-07T17:48:39.6591478Z remote: Compressing objects: 55% (818/1486) 2024-08-07T17:48:39.6592895Z remote: Compressing objects: 56% (833/1486) 2024-08-07T17:48:39.6594110Z remote: Compressing objects: 57% (848/1486) 2024-08-07T17:48:39.6596032Z remote: Compressing objects: 58% (862/1486) 2024-08-07T17:48:39.6597484Z remote: Compressing objects: 59% (877/1486) 2024-08-07T17:48:39.6598072Z remote: Compressing objects: 60% (892/1486) 2024-08-07T17:48:39.6598566Z remote: Compressing objects: 61% (907/1486) 2024-08-07T17:48:39.6599301Z remote: Compressing objects: 62% (922/1486) 2024-08-07T17:48:39.6600601Z remote: Compressing objects: 63% (937/1486) 2024-08-07T17:48:39.6601106Z remote: Compressing objects: 64% (952/1486) 2024-08-07T17:48:39.6601618Z remote: Compressing objects: 65% (966/1486) 2024-08-07T17:48:39.6618890Z remote: Compressing objects: 66% (981/1486) 2024-08-07T17:48:39.6630077Z remote: Compressing objects: 67% (996/1486) 2024-08-07T17:48:39.6640231Z remote: Compressing objects: 68% (1011/1486) 2024-08-07T17:48:39.6648818Z remote: Compressing objects: 69% (1026/1486) 2024-08-07T17:48:39.6657436Z remote: Compressing objects: 70% (1041/1486) 2024-08-07T17:48:39.6663702Z remote: Compressing objects: 71% (1056/1486) 2024-08-07T17:48:39.6670060Z remote: Compressing objects: 72% (1070/1486) 2024-08-07T17:48:39.6676336Z remote: Compressing objects: 73% (1085/1486) 2024-08-07T17:48:39.6679930Z remote: Compressing objects: 74% (1100/1486) 2024-08-07T17:48:39.6683551Z remote: Compressing objects: 75% (1115/1486) 2024-08-07T17:48:39.6687370Z remote: Compressing objects: 76% (1130/1486) 2024-08-07T17:48:39.6692062Z remote: Compressing objects: 77% (1145/1486) 2024-08-07T17:48:39.6694989Z remote: Compressing objects: 78% (1160/1486) 2024-08-07T17:48:39.6700392Z remote: Compressing objects: 79% (1174/1486) 2024-08-07T17:48:39.6703812Z remote: Compressing objects: 80% (1189/1486) 2024-08-07T17:48:39.6707388Z remote: Compressing objects: 81% (1204/1486) 2024-08-07T17:48:39.6710473Z remote: Compressing objects: 82% (1219/1486) 2024-08-07T17:48:39.6713183Z remote: Compressing objects: 83% (1234/1486) 2024-08-07T17:48:39.6716559Z remote: Compressing objects: 84% (1249/1486) 2024-08-07T17:48:39.6719064Z remote: Compressing objects: 85% (1264/1486) 2024-08-07T17:48:39.6721213Z remote: Compressing objects: 86% (1278/1486) 2024-08-07T17:48:39.6723460Z remote: Compressing objects: 87% (1293/1486) 2024-08-07T17:48:39.6725277Z remote: Compressing objects: 88% (1308/1486) 2024-08-07T17:48:39.6727745Z remote: Compressing objects: 89% (1323/1486) 2024-08-07T17:48:39.6730550Z remote: Compressing objects: 90% (1338/1486) 2024-08-07T17:48:39.6734091Z remote: Compressing objects: 91% (1353/1486) 2024-08-07T17:48:39.6734859Z remote: Compressing objects: 92% (1368/1486) 2024-08-07T17:48:39.6738113Z remote: Compressing objects: 93% (1382/1486) 2024-08-07T17:48:39.6738912Z remote: Compressing objects: 94% (1397/1486) 2024-08-07T17:48:39.6741955Z remote: Compressing objects: 95% (1412/1486) 2024-08-07T17:48:39.6742902Z remote: Compressing objects: 96% (1427/1486) 2024-08-07T17:48:39.6745863Z remote: Compressing objects: 97% (1442/1486) 2024-08-07T17:48:39.6746847Z remote: Compressing objects: 98% (1457/1486) 2024-08-07T17:48:39.6747868Z remote: Compressing objects: 99% (1472/1486) 2024-08-07T17:48:39.6748660Z remote: Compressing objects: 100% (1486/1486) 2024-08-07T17:48:39.6749207Z remote: Compressing objects: 100% (1486/1486), done. 2024-08-07T17:49:05.9401266Z remote: Total 1011035 (delta 1772), reused 2144 (delta 1296), pack-reused 1008246 2024-08-07T17:49:31.8899780Z [command]/usr/bin/git rev-parse --verify --quiet 016588f53c6904b840aa56aa86f95460b4d9c996^{object} 2024-08-07T17:49:31.8930727Z 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:49:31.8936054Z ##[endgroup] 2024-08-07T17:49:31.8936577Z ##[group]Determining the checkout info 2024-08-07T17:49:31.8938582Z ##[endgroup] 2024-08-07T17:49:31.8939110Z ##[group]Checking out the ref 2024-08-07T17:49:31.8942965Z [command]/usr/bin/git checkout --quiet --force 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:49:33.7737380Z ##[endgroup] 2024-08-07T17:49:33.7737981Z ##[group]Setting up auth for fetching submodules 2024-08-07T17:49:33.7743645Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-07T17:49:33.7805303Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-07T17:49:33.7838459Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-07T17:49:33.7874000Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-07T17:49:33.7905257Z ##[endgroup] 2024-08-07T17:49:33.7905776Z ##[group]Fetching submodules 2024-08-07T17:49:33.7910615Z [command]/usr/bin/git submodule sync --recursive 2024-08-07T17:49:33.8254229Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-07T17:49:33.8587288Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-07T17:49:33.8590405Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-07T17:49:33.8593791Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-07T17:49:33.8598146Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-07T17:49:33.8602607Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-07T17:49:33.8606083Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-07T17:49:33.8610049Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-07T17:49:33.8614251Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-07T17:49:33.8618673Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-07T17:49:33.8623200Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-07T17:49:33.8627705Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-07T17:49:33.8632248Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-07T17:49:33.8637190Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-07T17:49:33.8642101Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-07T17:49:33.8647025Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-07T17:49:33.8653048Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-08-07T17:49:33.8660531Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:49:33.8665810Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-07T17:49:33.8671322Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-07T17:49:33.8676974Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-07T17:49:33.8682665Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-07T17:49:33.8688529Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-07T17:49:33.8694504Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-07T17:49:33.8701109Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-07T17:49:33.8707265Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-07T17:49:33.8713372Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-07T17:49:33.8720060Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-07T17:49:33.8726355Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-07T17:49:33.8732985Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-07T17:49:33.8739710Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-07T17:49:33.8746881Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-07T17:49:33.8753472Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-07T17:49:33.8760594Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-07T17:49:33.8769473Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-07T17:49:33.8776913Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-07T17:49:33.8808289Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-07T17:49:34.2097790Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-07T17:49:34.4022746Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-07T17:49:34.5968927Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-07T17:49:34.8587109Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-07T17:49:36.9269531Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-07T17:49:50.5124504Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-07T17:49:50.9751230Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-07T17:49:51.4798875Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-07T17:49:52.1563074Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-07T17:49:53.5894748Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-07T17:49:55.8145565Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-07T17:50:05.8062166Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-07T17:50:07.3213466Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-07T17:50:09.2604012Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-07T17:50:10.6437299Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-08-07T17:50:10.8398216Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-07T17:50:11.2909281Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-07T17:50:11.6265733Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-07T17:50:12.8779440Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-07T17:50:13.2945058Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-07T17:50:13.5637166Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-07T17:50:15.4219028Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-07T17:50:16.2430102Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-07T17:50:17.0617632Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-07T17:50:24.3840920Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-07T17:50:26.6526285Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-07T17:50:31.8038516Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-07T17:50:32.0607901Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-07T17:50:42.1494663Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-07T17:50:42.3343244Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-07T17:50:42.5663568Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-07T17:50:43.5711627Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-07T17:50:43.8880106Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-07T17:50:44.5290957Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-07T17:50:44.9731162Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-07T17:50:44.9873209Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-07T17:50:44.9980333Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-07T17:50:45.0294593Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-07T17:50:45.0768144Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-07T17:50:46.3601856Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-07T17:50:46.3900894Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-07T17:50:46.4457072Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-07T17:50:46.5698184Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-07T17:50:46.6120627Z Submodule path 'third_party/cudnn_frontend': checked out '98ca4e1941fe3263f128f74f10063a3ea35c7019' 2024-08-07T17:50:47.2926703Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-07T17:50:47.6243179Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-07T17:50:47.7278438Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-07T17:50:47.7301747Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:50:47.7305867Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:50:47.7310226Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:50:47.7314406Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:50:47.7318445Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:50:47.7349166Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-07T17:50:48.9609721Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-07T17:50:49.6255158Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-07T17:50:51.9163161Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-07T17:50:53.1513133Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-07T17:50:53.5207511Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-07T17:50:53.6413405Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-07T17:50:54.2371356Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-07T17:50:54.3413794Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-07T17:50:54.3613959Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-07T17:50:54.5208626Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-07T17:50:54.5702592Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-07T17:50:54.5813260Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-08-07T17:50:54.6287937Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-07T17:50:54.6619841Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-07T17:50:54.7172234Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-07T17:50:54.7323780Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-07T17:50:54.7344127Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-07T17:50:54.7371653Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-07T17:51:10.0136900Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-07T17:51:10.0354453Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-07T17:51:10.1460843Z Submodule path 'third_party/kineto': checked out 'da2f2682cabaf95d601fa2a9b7e0979f84fe7667' 2024-08-07T17:51:10.1481750Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:51:10.1485262Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:51:10.1488950Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:51:10.1520543Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-07T17:51:10.7855669Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-07T17:51:12.1256469Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-07T17:51:13.5046734Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-07T17:51:13.5066613Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:51:13.5070205Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:51:13.5074236Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:51:13.5078135Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:51:13.5082137Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:51:13.5086720Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:51:13.5090767Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:51:13.5095012Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:51:13.5126950Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-07T17:51:14.5249747Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-07T17:51:14.9238013Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-07T17:51:16.3596333Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-07T17:51:16.7107502Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-07T17:51:17.3095573Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-07T17:51:18.5439773Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-07T17:51:25.8877330Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-07T17:51:26.3698461Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-07T17:51:26.3938783Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-07T17:51:26.4396156Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-07T17:51:26.4559753Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-07T17:51:26.4578969Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:51:26.4609280Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-07T17:51:26.8063056Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-07T17:51:26.8289483Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-07T17:51:26.8804067Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-07T17:51:27.0116096Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-07T17:51:27.0320419Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-07T17:51:27.0799372Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-07T17:51:27.1505817Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-07T17:51:27.1985850Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-07T17:51:27.2290893Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-07T17:51:27.3641019Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-07T17:51:27.8757401Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-07T17:51:27.8799442Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-07T17:51:27.8802875Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-07T17:51:27.8833677Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-07T17:51:28.3801145Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-07T17:51:29.4341997Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-07T17:51:29.4789305Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-07T17:51:29.5749366Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-07T17:51:29.5771456Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:51:29.5775098Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:51:29.5778800Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:51:29.5782710Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:51:29.5786870Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:51:29.5790807Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:51:29.5794960Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:51:29.5799872Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:51:29.5831297Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-07T17:51:30.0279783Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-07T17:51:31.2317611Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-07T17:51:31.5658177Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-07T17:51:38.9088542Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-07T17:51:39.1871531Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-07T17:51:39.3973346Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-07T17:51:39.7286254Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-07T17:51:46.9034661Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-07T17:51:46.9546177Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-07T17:51:46.9734126Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-07T17:51:47.1054555Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-07T17:51:47.1214399Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-07T17:51:47.1400259Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-07T17:51:47.1595102Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-07T17:51:47.1614130Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:51:47.1617709Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:51:47.1646289Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-07T17:51:49.4004226Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-07T17:51:50.9401484Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-07T17:51:50.9987705Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-07T17:51:51.6817076Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-07T17:51:51.6957196Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-07T17:51:52.0435250Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-07T17:51:52.0461182Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:51:52.0464774Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-07T17:51:52.0495186Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-07T17:51:52.5651208Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-07T17:51:53.8182855Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-07T17:51:53.9097665Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-07T17:51:53.9205098Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-07T17:51:53.9357010Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-07T17:51:53.9834193Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-07T17:51:54.0205874Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-07T17:51:54.0743370Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-07T17:51:54.1093611Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-07T17:51:54.1114512Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:51:54.1118123Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:51:54.1121840Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:51:54.1125773Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:51:54.1154389Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-07T17:51:55.3593816Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-07T17:51:55.6041401Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-07T17:51:56.9563063Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-07T17:51:58.0207304Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-07T17:51:58.0402132Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-07T17:51:58.1272020Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-07T17:51:58.1625593Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-07T17:51:58.1645859Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:51:58.1675602Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-07T17:51:58.4152755Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-07T17:51:58.4192027Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-07T17:51:58.4540150Z Entering 'android/libs/fbjni' 2024-08-07T17:51:58.4588752Z Entering 'third_party/FP16' 2024-08-07T17:51:58.4637382Z Entering 'third_party/FXdiv' 2024-08-07T17:51:58.4685294Z Entering 'third_party/NNPACK' 2024-08-07T17:51:58.4733544Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:51:58.4781095Z Entering 'third_party/XNNPACK' 2024-08-07T17:51:58.4847498Z Entering 'third_party/benchmark' 2024-08-07T17:51:58.4894865Z Entering 'third_party/cpp-httplib' 2024-08-07T17:51:58.4942584Z Entering 'third_party/cpuinfo' 2024-08-07T17:51:58.4990435Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:51:58.5038326Z Entering 'third_party/cutlass' 2024-08-07T17:51:58.5093402Z Entering 'third_party/eigen' 2024-08-07T17:51:58.5144347Z Entering 'third_party/fbgemm' 2024-08-07T17:51:58.5192094Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:51:58.5239611Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:51:58.5287577Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:51:58.5342778Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:51:58.5390609Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:51:58.5438203Z Entering 'third_party/flatbuffers' 2024-08-07T17:51:58.5488457Z Entering 'third_party/fmt' 2024-08-07T17:51:58.5537433Z Entering 'third_party/foxi' 2024-08-07T17:51:58.5584410Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:51:58.5632217Z Entering 'third_party/gloo' 2024-08-07T17:51:58.5679895Z Entering 'third_party/googletest' 2024-08-07T17:51:58.5728247Z Entering 'third_party/ideep' 2024-08-07T17:51:58.5775029Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:51:58.5829443Z Entering 'third_party/ittapi' 2024-08-07T17:51:58.5876480Z Entering 'third_party/kineto' 2024-08-07T17:51:58.5924000Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:51:58.5970719Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:51:58.6020196Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:51:58.6067287Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:51:58.6114782Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:51:58.6161185Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:51:58.6211009Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:51:58.6257599Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:51:58.6305825Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:51:58.6354177Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:51:58.6404169Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:51:58.6451114Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:51:58.6499826Z Entering 'third_party/mimalloc' 2024-08-07T17:51:58.6547099Z Entering 'third_party/nccl/nccl' 2024-08-07T17:51:58.6594696Z Entering 'third_party/nlohmann' 2024-08-07T17:51:58.6645340Z Entering 'third_party/onnx' 2024-08-07T17:51:58.6709945Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:51:58.6756399Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:51:58.6806276Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:51:58.6854654Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:51:58.6901301Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:51:58.6946706Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:51:58.6993683Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:51:58.7042933Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:51:58.7088038Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:51:58.7134984Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:51:58.7180619Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:51:58.7231043Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:51:58.7279000Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:51:58.7349185Z Entering 'third_party/pocketfft' 2024-08-07T17:51:58.7395747Z Entering 'third_party/protobuf' 2024-08-07T17:51:58.7446013Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:51:58.7491702Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:51:58.7540606Z Entering 'third_party/psimd' 2024-08-07T17:51:58.7587371Z Entering 'third_party/pthreadpool' 2024-08-07T17:51:58.7635239Z Entering 'third_party/pybind11' 2024-08-07T17:51:58.7682358Z Entering 'third_party/python-peachpy' 2024-08-07T17:51:58.7729372Z Entering 'third_party/sleef' 2024-08-07T17:51:58.7776268Z Entering 'third_party/tensorpipe' 2024-08-07T17:51:58.7824124Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:51:58.7870108Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:51:58.7918376Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:51:58.7964624Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:51:58.8011229Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:51:58.8072773Z ##[endgroup] 2024-08-07T17:51:58.8076204Z ##[group]Persisting credentials for submodules 2024-08-07T17:51:58.8080614Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-07T17:51:58.8419273Z Entering 'android/libs/fbjni' 2024-08-07T17:51:58.8480673Z Entering 'third_party/FP16' 2024-08-07T17:51:58.8543849Z Entering 'third_party/FXdiv' 2024-08-07T17:51:58.8606155Z Entering 'third_party/NNPACK' 2024-08-07T17:51:58.8668080Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:51:58.8730703Z Entering 'third_party/XNNPACK' 2024-08-07T17:51:58.8811315Z Entering 'third_party/benchmark' 2024-08-07T17:51:58.8872958Z Entering 'third_party/cpp-httplib' 2024-08-07T17:51:58.8935293Z Entering 'third_party/cpuinfo' 2024-08-07T17:51:58.8997324Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:51:58.9058576Z Entering 'third_party/cutlass' 2024-08-07T17:51:58.9129637Z Entering 'third_party/eigen' 2024-08-07T17:51:58.9192655Z Entering 'third_party/fbgemm' 2024-08-07T17:51:58.9255209Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:51:58.9317093Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:51:58.9379108Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:51:58.9448706Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:51:58.9509716Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:51:58.9571851Z Entering 'third_party/flatbuffers' 2024-08-07T17:51:58.9638082Z Entering 'third_party/fmt' 2024-08-07T17:51:58.9699977Z Entering 'third_party/foxi' 2024-08-07T17:51:58.9761244Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:51:58.9823770Z Entering 'third_party/gloo' 2024-08-07T17:51:58.9885801Z Entering 'third_party/googletest' 2024-08-07T17:51:58.9948314Z Entering 'third_party/ideep' 2024-08-07T17:51:59.0010135Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:51:59.0078792Z Entering 'third_party/ittapi' 2024-08-07T17:51:59.0140286Z Entering 'third_party/kineto' 2024-08-07T17:51:59.0201487Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:51:59.0261664Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:51:59.0325582Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:51:59.0388804Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:51:59.0451340Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:51:59.0511662Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:51:59.0574596Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:51:59.0636168Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:51:59.0699718Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:51:59.0761610Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:51:59.0827251Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:51:59.0888213Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:51:59.0952001Z Entering 'third_party/mimalloc' 2024-08-07T17:51:59.1013967Z Entering 'third_party/nccl/nccl' 2024-08-07T17:51:59.1075191Z Entering 'third_party/nlohmann' 2024-08-07T17:51:59.1137984Z Entering 'third_party/onnx' 2024-08-07T17:51:59.1217893Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:51:59.1279787Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:51:59.1344571Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:51:59.1407842Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:51:59.1468160Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:51:59.1531462Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:51:59.1591074Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:51:59.1653954Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:51:59.1714752Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:51:59.1775554Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:51:59.1837293Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:51:59.1902500Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:51:59.1964832Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:51:59.2049774Z Entering 'third_party/pocketfft' 2024-08-07T17:51:59.2112765Z Entering 'third_party/protobuf' 2024-08-07T17:51:59.2177117Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:51:59.2239397Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:51:59.2303458Z Entering 'third_party/psimd' 2024-08-07T17:51:59.2363857Z Entering 'third_party/pthreadpool' 2024-08-07T17:51:59.2425615Z Entering 'third_party/pybind11' 2024-08-07T17:51:59.2487009Z Entering 'third_party/python-peachpy' 2024-08-07T17:51:59.2550347Z Entering 'third_party/sleef' 2024-08-07T17:51:59.2612529Z Entering 'third_party/tensorpipe' 2024-08-07T17:51:59.2674049Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:51:59.2735795Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:51:59.2796899Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:51:59.2857853Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:51:59.2918546Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:51:59.2999029Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-07T17:51:59.3344726Z Entering 'android/libs/fbjni' 2024-08-07T17:51:59.3404535Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-07T17:51:59.3423777Z Entering 'third_party/FP16' 2024-08-07T17:51:59.3482577Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-07T17:51:59.3501900Z Entering 'third_party/FXdiv' 2024-08-07T17:51:59.3559675Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-07T17:51:59.3578126Z Entering 'third_party/NNPACK' 2024-08-07T17:51:59.3637129Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-07T17:51:59.3656045Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:51:59.3714955Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-07T17:51:59.3734127Z Entering 'third_party/XNNPACK' 2024-08-07T17:51:59.3791609Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-07T17:51:59.3830026Z Entering 'third_party/benchmark' 2024-08-07T17:51:59.3887880Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:51:59.3906815Z Entering 'third_party/cpp-httplib' 2024-08-07T17:51:59.3963725Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-07T17:51:59.3981976Z Entering 'third_party/cpuinfo' 2024-08-07T17:51:59.4039356Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-07T17:51:59.4058819Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:51:59.4118441Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-07T17:51:59.4136463Z Entering 'third_party/cutlass' 2024-08-07T17:51:59.4193512Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-07T17:51:59.4221536Z Entering 'third_party/eigen' 2024-08-07T17:51:59.4279000Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-07T17:51:59.4299957Z Entering 'third_party/fbgemm' 2024-08-07T17:51:59.4358351Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-07T17:51:59.4376575Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:51:59.4435267Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-07T17:51:59.4453225Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:51:59.4513983Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-07T17:51:59.4532048Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:51:59.4590300Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-07T17:51:59.4616676Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:51:59.4673968Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:51:59.4691983Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:51:59.4750595Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-07T17:51:59.4769760Z Entering 'third_party/flatbuffers' 2024-08-07T17:51:59.4828589Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-07T17:51:59.4850084Z Entering 'third_party/fmt' 2024-08-07T17:51:59.4907713Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-07T17:51:59.4926202Z Entering 'third_party/foxi' 2024-08-07T17:51:59.4984376Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-08-07T17:51:59.5004208Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:51:59.5061450Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-07T17:51:59.5079644Z Entering 'third_party/gloo' 2024-08-07T17:51:59.5138986Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-07T17:51:59.5157556Z Entering 'third_party/googletest' 2024-08-07T17:51:59.5217120Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:51:59.5235249Z Entering 'third_party/ideep' 2024-08-07T17:51:59.5293142Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-07T17:51:59.5310922Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:51:59.5368341Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-07T17:51:59.5395141Z Entering 'third_party/ittapi' 2024-08-07T17:51:59.5454268Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-07T17:51:59.5472368Z Entering 'third_party/kineto' 2024-08-07T17:51:59.5532592Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-07T17:51:59.5550489Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:51:59.5610761Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-07T17:51:59.5628023Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:51:59.5686627Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-07T17:51:59.5708629Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:51:59.5766640Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-07T17:51:59.5785203Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:51:59.5846224Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-07T17:51:59.5864764Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:51:59.5925393Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-07T17:51:59.5942045Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:51:59.6003074Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-07T17:51:59.6024124Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:51:59.6082360Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-07T17:51:59.6100987Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:51:59.6158919Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:51:59.6177562Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:51:59.6237312Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-07T17:51:59.6257502Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:51:59.6317552Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-07T17:51:59.6337374Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:51:59.6395346Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-07T17:51:59.6414031Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:51:59.6471810Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-07T17:51:59.6491816Z Entering 'third_party/mimalloc' 2024-08-07T17:51:59.6552061Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-07T17:51:59.6571309Z Entering 'third_party/nccl/nccl' 2024-08-07T17:51:59.6629556Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-07T17:51:59.6647921Z Entering 'third_party/nlohmann' 2024-08-07T17:51:59.6706278Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-07T17:51:59.6725595Z Entering 'third_party/onnx' 2024-08-07T17:51:59.6783442Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-07T17:51:59.6819454Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:51:59.6876774Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:51:59.6894750Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:51:59.6953644Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-07T17:51:59.6974556Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:51:59.7034391Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-07T17:51:59.7053794Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:51:59.7112038Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:51:59.7129769Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:51:59.7186808Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:51:59.7205426Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:51:59.7263475Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-07T17:51:59.7281006Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:51:59.7339243Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-07T17:51:59.7358807Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:51:59.7417291Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-07T17:51:59.7434998Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:51:59.7492873Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-07T17:51:59.7511838Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:51:59.7569448Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-07T17:51:59.7586493Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:51:59.7644587Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-07T17:51:59.7664806Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:51:59.7724496Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-07T17:51:59.7744677Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:51:59.7802019Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-07T17:51:59.7842943Z Entering 'third_party/pocketfft' 2024-08-07T17:51:59.7902021Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-07T17:51:59.7920587Z Entering 'third_party/protobuf' 2024-08-07T17:51:59.7977947Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-07T17:51:59.8000883Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:51:59.8059655Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:51:59.8077463Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:51:59.8136304Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:51:59.8156496Z Entering 'third_party/psimd' 2024-08-07T17:51:59.8214560Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-07T17:51:59.8232556Z Entering 'third_party/pthreadpool' 2024-08-07T17:51:59.8290100Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-07T17:51:59.8309336Z Entering 'third_party/pybind11' 2024-08-07T17:51:59.8366786Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-07T17:51:59.8385269Z Entering 'third_party/python-peachpy' 2024-08-07T17:51:59.8443055Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-07T17:51:59.8461358Z Entering 'third_party/sleef' 2024-08-07T17:51:59.8518961Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-07T17:51:59.8537293Z Entering 'third_party/tensorpipe' 2024-08-07T17:51:59.8596319Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-07T17:51:59.8614538Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:51:59.8672630Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:51:59.8690570Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:51:59.8749536Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-07T17:51:59.8767465Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:51:59.8825782Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-07T17:51:59.8843990Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:51:59.8902813Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-07T17:51:59.8919608Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:51:59.8977604Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-07T17:52:00.0112950Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-07T17:52:00.0460816Z Entering 'android/libs/fbjni' 2024-08-07T17:52:00.0508326Z Entering 'third_party/FP16' 2024-08-07T17:52:00.0556463Z Entering 'third_party/FXdiv' 2024-08-07T17:52:00.0604506Z Entering 'third_party/NNPACK' 2024-08-07T17:52:00.0651825Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:52:00.0699645Z Entering 'third_party/XNNPACK' 2024-08-07T17:52:00.0765712Z Entering 'third_party/benchmark' 2024-08-07T17:52:00.0813906Z Entering 'third_party/cpp-httplib' 2024-08-07T17:52:00.0861472Z Entering 'third_party/cpuinfo' 2024-08-07T17:52:00.0909685Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:52:00.0957655Z Entering 'third_party/cutlass' 2024-08-07T17:52:00.1014915Z Entering 'third_party/eigen' 2024-08-07T17:52:00.1065258Z Entering 'third_party/fbgemm' 2024-08-07T17:52:00.1114791Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:52:00.1161037Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:52:00.1208169Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:52:00.1261899Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:52:00.1310235Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:52:00.1358189Z Entering 'third_party/flatbuffers' 2024-08-07T17:52:00.1409218Z Entering 'third_party/fmt' 2024-08-07T17:52:00.1456736Z Entering 'third_party/foxi' 2024-08-07T17:52:00.1504727Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:52:00.1552194Z Entering 'third_party/gloo' 2024-08-07T17:52:00.1600873Z Entering 'third_party/googletest' 2024-08-07T17:52:00.1648468Z Entering 'third_party/ideep' 2024-08-07T17:52:00.1695096Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:52:00.1751178Z Entering 'third_party/ittapi' 2024-08-07T17:52:00.1798640Z Entering 'third_party/kineto' 2024-08-07T17:52:00.1845964Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:52:00.1892591Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:52:00.1941879Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:52:00.1989079Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:52:00.2037549Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:52:00.2084623Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:52:00.2135878Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:52:00.2182724Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:52:00.2232921Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:52:00.2281304Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:52:00.2330967Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:52:00.2377881Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:52:00.2427935Z Entering 'third_party/mimalloc' 2024-08-07T17:52:00.2475019Z Entering 'third_party/nccl/nccl' 2024-08-07T17:52:00.2526339Z Entering 'third_party/nlohmann' 2024-08-07T17:52:00.2573902Z Entering 'third_party/onnx' 2024-08-07T17:52:00.2640002Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:52:00.2687331Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:52:00.2738591Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:52:00.2788303Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:52:00.2836512Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:52:00.2883144Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:52:00.2930548Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:52:00.2979351Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:52:00.3026454Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:52:00.3072800Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:52:00.3122881Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:52:00.3172636Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:52:00.3221726Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:52:00.3292466Z Entering 'third_party/pocketfft' 2024-08-07T17:52:00.3340476Z Entering 'third_party/protobuf' 2024-08-07T17:52:00.3392547Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:52:00.3442211Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:52:00.3491968Z Entering 'third_party/psimd' 2024-08-07T17:52:00.3540748Z Entering 'third_party/pthreadpool' 2024-08-07T17:52:00.3588705Z Entering 'third_party/pybind11' 2024-08-07T17:52:00.3636806Z Entering 'third_party/python-peachpy' 2024-08-07T17:52:00.3685247Z Entering 'third_party/sleef' 2024-08-07T17:52:00.3735436Z Entering 'third_party/tensorpipe' 2024-08-07T17:52:00.3783717Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:52:00.3831124Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:52:00.3878168Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:52:00.3927030Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:52:00.3972982Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:52:00.4038700Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-07T17:52:00.4377149Z Entering 'android/libs/fbjni' 2024-08-07T17:52:00.4424044Z Entering 'third_party/FP16' 2024-08-07T17:52:00.4471758Z Entering 'third_party/FXdiv' 2024-08-07T17:52:00.4519801Z Entering 'third_party/NNPACK' 2024-08-07T17:52:00.4567416Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:52:00.4617376Z Entering 'third_party/XNNPACK' 2024-08-07T17:52:00.4682416Z Entering 'third_party/benchmark' 2024-08-07T17:52:00.4730650Z Entering 'third_party/cpp-httplib' 2024-08-07T17:52:00.4777793Z Entering 'third_party/cpuinfo' 2024-08-07T17:52:00.4826037Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:52:00.4875071Z Entering 'third_party/cutlass' 2024-08-07T17:52:00.4931900Z Entering 'third_party/eigen' 2024-08-07T17:52:00.4981922Z Entering 'third_party/fbgemm' 2024-08-07T17:52:00.5029921Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:52:00.5076044Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:52:00.5124270Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:52:00.5177033Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:52:00.5224371Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:52:00.5272698Z Entering 'third_party/flatbuffers' 2024-08-07T17:52:00.5323226Z Entering 'third_party/fmt' 2024-08-07T17:52:00.5370423Z Entering 'third_party/foxi' 2024-08-07T17:52:00.5417507Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:52:00.5464693Z Entering 'third_party/gloo' 2024-08-07T17:52:00.5513500Z Entering 'third_party/googletest' 2024-08-07T17:52:00.5561148Z Entering 'third_party/ideep' 2024-08-07T17:52:00.5610694Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:52:00.5665499Z Entering 'third_party/ittapi' 2024-08-07T17:52:00.5712854Z Entering 'third_party/kineto' 2024-08-07T17:52:00.5759977Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:52:00.5807314Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:52:00.5855597Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:52:00.5902560Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:52:00.5950368Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:52:00.5997867Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:52:00.6048243Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:52:00.6095675Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:52:00.6143949Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:52:00.6192218Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:52:00.6242626Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:52:00.6290110Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:52:00.6339629Z Entering 'third_party/mimalloc' 2024-08-07T17:52:00.6386875Z Entering 'third_party/nccl/nccl' 2024-08-07T17:52:00.6436079Z Entering 'third_party/nlohmann' 2024-08-07T17:52:00.6485023Z Entering 'third_party/onnx' 2024-08-07T17:52:00.6551493Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:52:00.6599363Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:52:00.6649189Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:52:00.6700020Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:52:00.6746561Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:52:00.6794165Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:52:00.6841095Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:52:00.6889170Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:52:00.6937978Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:52:00.6984687Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:52:00.7031780Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:52:00.7080356Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:52:00.7130110Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:52:00.7200133Z Entering 'third_party/pocketfft' 2024-08-07T17:52:00.7247347Z Entering 'third_party/protobuf' 2024-08-07T17:52:00.7299330Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:52:00.7346749Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:52:00.7396659Z Entering 'third_party/psimd' 2024-08-07T17:52:00.7444006Z Entering 'third_party/pthreadpool' 2024-08-07T17:52:00.7491795Z Entering 'third_party/pybind11' 2024-08-07T17:52:00.7540089Z Entering 'third_party/python-peachpy' 2024-08-07T17:52:00.7587957Z Entering 'third_party/sleef' 2024-08-07T17:52:00.7637969Z Entering 'third_party/tensorpipe' 2024-08-07T17:52:00.7684974Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:52:00.7731936Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:52:00.7778320Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:52:00.7826415Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:52:00.7872277Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:52:00.7935093Z ##[endgroup] 2024-08-07T17:52:00.7980523Z [command]/usr/bin/git log -1 --format='%H' 2024-08-07T17:52:00.8020932Z '016588f53c6904b840aa56aa86f95460b4d9c996' 2024-08-07T17:52:00.8224538Z Prepare all required actions 2024-08-07T17:52:00.8225226Z Getting action download info 2024-08-07T17:52:00.9583964Z ##[group]Run ./.github/actions/setup-linux 2024-08-07T17:52:00.9584395Z env: 2024-08-07T17:52:00.9584674Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:52:00.9585024Z ##[endgroup] 2024-08-07T17:52:00.9642858Z ##[group]Run set -euo pipefail 2024-08-07T17:52:00.9643285Z set -euo pipefail 2024-08-07T17:52:00.9643653Z function get_ec2_metadata() { 2024-08-07T17:52:00.9644135Z  # Pulled from instance metadata endpoint for EC2 2024-08-07T17:52:00.9644911Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-07T17:52:00.9645592Z  category=$1 2024-08-07T17:52:00.9646060Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-07T17:52:00.9646634Z  runner_name_str=i-0d5f0f54c385d63e2 2024-08-07T17:52:00.9647118Z  if [[ -f /.inarc ]]; then 2024-08-07T17:52:00.9647570Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-07T17:52:00.9648055Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-07T17:52:00.9648646Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-07T17:52:00.9649182Z  else 2024-08-07T17:52:00.9649604Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-07T17:52:00.9650128Z  fi 2024-08-07T17:52:00.9650416Z } 2024-08-07T17:52:00.9650810Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-07T17:52:00.9651363Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-07T17:52:00.9651960Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-07T17:52:00.9652469Z echo "system info $(uname -a)" 2024-08-07T17:52:00.9661688Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:52:00.9662168Z env: 2024-08-07T17:52:00.9662454Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:52:00.9662806Z ##[endgroup] 2024-08-07T17:52:00.9775869Z ami-id: ami-06c68f701d8090592 2024-08-07T17:52:00.9838645Z instance-id: i-0d5f0f54c385d63e2 2024-08-07T17:52:00.9895779Z instance-type: g3.4xlarge 2024-08-07T17:52:00.9908842Z system info Linux ip-10-0-41-236.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-07T17:52:00.9944425Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-07T17:52:00.9945232Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-07T17:52:00.9952833Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:52:00.9953311Z env: 2024-08-07T17:52:00.9953602Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:52:00.9953954Z ##[endgroup] 2024-08-07T17:52:01.0040114Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-07T17:52:01.0040691Z if systemctl is-active --quiet docker; then 2024-08-07T17:52:01.0041185Z  echo "Docker daemon is running..."; 2024-08-07T17:52:01.0041581Z else 2024-08-07T17:52:01.0042030Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-07T17:52:01.0042558Z fi 2024-08-07T17:52:01.0048633Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:52:01.0049109Z env: 2024-08-07T17:52:01.0049395Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:52:01.0049716Z ##[endgroup] 2024-08-07T17:52:01.0137330Z Docker daemon is running... 2024-08-07T17:52:01.0219828Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-07T17:52:01.0220334Z with: 2024-08-07T17:52:01.0220618Z shell: bash 2024-08-07T17:52:01.0220914Z timeout_minutes: 5 2024-08-07T17:52:01.0221411Z max_attempts: 3 2024-08-07T17:52:01.0221719Z retry_wait_seconds: 30 2024-08-07T17:52:01.0223157Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-08-07T17:52:01.0224490Z polling_interval_seconds: 1 2024-08-07T17:52:01.0224839Z warning_on_retry: true 2024-08-07T17:52:01.0225179Z continue_on_error: false 2024-08-07T17:52:01.0225508Z env: 2024-08-07T17:52:01.0225767Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:52:01.0226115Z AWS_RETRY_MODE: standard 2024-08-07T17:52:01.0226459Z AWS_MAX_ATTEMPTS: 5 2024-08-07T17:52:01.0226774Z AWS_DEFAULT_REGION: us-east-1 2024-08-07T17:52:01.0227124Z ##[endgroup] 2024-08-07T17:52:02.4834313Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-07T17:52:02.4835094Z Configure a credential helper to remove this warning. See 2024-08-07T17:52:02.4835896Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-07T17:52:02.4836368Z 2024-08-07T17:52:02.4836530Z Login Succeeded 2024-08-07T17:52:03.0969551Z Command completed after 1 attempt(s). 2024-08-07T17:52:03.1041686Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-07T17:52:03.1042342Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-07T17:52:03.1042902Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-07T17:52:03.1051274Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:52:03.1051760Z env: 2024-08-07T17:52:03.1052029Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:52:03.1052375Z ##[endgroup] 2024-08-07T17:52:03.1152617Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T17:52:03.1153324Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T17:52:03.1153862Z # shellcheck disable=SC2046 2024-08-07T17:52:03.1154291Z docker stop $(docker ps -q) || true 2024-08-07T17:52:03.1154744Z # Prune all of the docker images 2024-08-07T17:52:03.1155151Z docker system prune -af 2024-08-07T17:52:03.1161467Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:52:03.1161943Z env: 2024-08-07T17:52:03.1162211Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:52:03.1162556Z ##[endgroup] 2024-08-07T17:52:03.1495391Z "docker stop" requires at least 1 argument. 2024-08-07T17:52:03.1495861Z See 'docker stop --help'. 2024-08-07T17:52:03.1496086Z 2024-08-07T17:52:03.1496299Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-07T17:52:03.1496652Z 2024-08-07T17:52:03.1496801Z Stop one or more running containers 2024-08-07T17:52:03.1705831Z Total reclaimed space: 0B 2024-08-07T17:52:03.1748615Z ##[group]Run set +e 2024-08-07T17:52:03.1749004Z set +e 2024-08-07T17:52:03.1749319Z set -x 2024-08-07T17:52:03.1749616Z  2024-08-07T17:52:03.1749945Z PT_DOMAIN=download.pytorch.org 2024-08-07T17:52:03.1750683Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-07T17:52:03.1751644Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-07T17:52:03.1752297Z # one is returned at random 2024-08-07T17:52:03.1752800Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-07T17:52:03.1753264Z  2024-08-07T17:52:03.1753572Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-07T17:52:03.1754108Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-07T17:52:03.1754779Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-07T17:52:03.1755282Z  2024-08-07T17:52:03.1755585Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-07T17:52:03.1756079Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-07T17:52:03.1756766Z  exit 1 2024-08-07T17:52:03.1757067Z  fi 2024-08-07T17:52:03.1757514Z fi 2024-08-07T17:52:03.1757824Z  2024-08-07T17:52:03.1758161Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-07T17:52:03.1758632Z  # Clean up any old records first 2024-08-07T17:52:03.1759097Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-07T17:52:03.1759502Z fi 2024-08-07T17:52:03.1759791Z  2024-08-07T17:52:03.1760194Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-07T17:52:03.1760685Z cat /etc/hosts 2024-08-07T17:52:03.1767465Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:52:03.1767948Z env: 2024-08-07T17:52:03.1768220Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:52:03.1768571Z ##[endgroup] 2024-08-07T17:52:03.1795597Z + PT_DOMAIN=download.pytorch.org 2024-08-07T17:52:03.1802464Z ++ dig -4 +short download.pytorch.org 2024-08-07T17:52:03.1803255Z ++ tail -n1 2024-08-07T17:52:03.2197975Z + RESOLVED_IP=18.160.10.28 2024-08-07T17:52:03.2198375Z + '[' -z 18.160.10.28 ']' 2024-08-07T17:52:03.2198764Z + grep -r download.pytorch.org /etc/hosts 2024-08-07T17:52:03.2215211Z + echo '18.160.10.28 download.pytorch.org' 2024-08-07T17:52:03.2215715Z + sudo tee -a /etc/hosts 2024-08-07T17:52:03.4699188Z 18.160.10.28 download.pytorch.org 2024-08-07T17:52:03.4719800Z + cat /etc/hosts 2024-08-07T17:52:03.4729852Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-07T17:52:03.4736761Z ::1 localhost6 localhost6.localdomain6 2024-08-07T17:52:03.4737246Z 18.160.10.28 download.pytorch.org 2024-08-07T17:52:03.4933386Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-07T17:52:03.4934019Z with: 2024-08-07T17:52:03.4934925Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:03.4935961Z docker-build-dir: .ci/docker 2024-08-07T17:52:03.4936324Z working-directory: . 2024-08-07T17:52:03.4936777Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:52:03.4937282Z force-push: false 2024-08-07T17:52:03.4937579Z env: 2024-08-07T17:52:03.4937867Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:52:03.4938196Z ##[endgroup] 2024-08-07T17:52:03.4967406Z ##[group]Run set -ex 2024-08-07T17:52:03.4967805Z set -ex 2024-08-07T17:52:03.4968104Z  2024-08-07T17:52:03.4968601Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-07T17:52:03.4969458Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-07T17:52:03.4970153Z # job could then download the pre-built image as usual 2024-08-07T17:52:03.4970790Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-07T17:52:03.4971406Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-07T17:52:03.4971967Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:52:03.4972459Z  2024-08-07T17:52:03.4972909Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-07T17:52:03.4973459Z  exit 0 2024-08-07T17:52:03.4973742Z else 2024-08-07T17:52:03.4974091Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-07T17:52:03.4974507Z fi 2024-08-07T17:52:03.4974772Z  2024-08-07T17:52:03.4975203Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-07T17:52:03.4975947Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-07T17:52:03.4976588Z  # use it as it is, but first let's extract the tag 2024-08-07T17:52:03.4977188Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-07T17:52:03.4978034Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:52:03.4978630Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:52:03.4979141Z else 2024-08-07T17:52:03.4979544Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-07T17:52:03.4980102Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:52:03.4980893Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:52:03.4981580Z fi 2024-08-07T17:52:03.4989940Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:52:03.4990411Z env: 2024-08-07T17:52:03.4990701Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:52:03.4991035Z REPO_NAME: pytorch 2024-08-07T17:52:03.4991966Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:03.4992971Z DOCKER_BUILD_DIR: .ci/docker 2024-08-07T17:52:03.4993449Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:52:03.4993926Z ##[endgroup] 2024-08-07T17:52:03.5023355Z + [[ ! -d .ci/docker ]] 2024-08-07T17:52:03.5023725Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-07T17:52:03.5024102Z + echo skip=false 2024-08-07T17:52:03.5026917Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-07T17:52:03.5032634Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:03.5033601Z ++ awk -F '[:,]' '{print $2}' 2024-08-07T17:52:03.5058487Z + DOCKER_TAG=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:03.5059625Z + echo docker-tag=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:03.5061675Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:03.5100843Z ##[group]Run set +e 2024-08-07T17:52:03.5101245Z set +e 2024-08-07T17:52:03.5101530Z set -x 2024-08-07T17:52:03.5101829Z  2024-08-07T17:52:03.5102116Z login() { 2024-08-07T17:52:03.5102688Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-07T17:52:03.5103326Z } 2024-08-07T17:52:03.5103607Z  2024-08-07T17:52:03.5103877Z retry () { 2024-08-07T17:52:03.5104234Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-07T17:52:03.5104640Z } 2024-08-07T17:52:03.5104897Z  2024-08-07T17:52:03.5105208Z retry login "${DOCKER_REGISTRY}" 2024-08-07T17:52:03.5105594Z  2024-08-07T17:52:03.5106013Z # Check if image already exists, if it does then skip building it 2024-08-07T17:52:03.5106627Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-07T17:52:03.5107089Z  exit 0 2024-08-07T17:52:03.5107366Z fi 2024-08-07T17:52:03.5107650Z  2024-08-07T17:52:03.5108077Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-07T17:52:03.5108781Z # be empty. The default action would be to continue rebuild the image 2024-08-07T17:52:03.5109420Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-07T17:52:03.5109991Z  # if we're on the base branch then use the parent commit 2024-08-07T17:52:03.5110478Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-07T17:52:03.5110884Z else 2024-08-07T17:52:03.5111302Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-07T17:52:03.5112063Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-07T17:52:03.5112529Z fi 2024-08-07T17:52:03.5112815Z  2024-08-07T17:52:03.5113114Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-07T17:52:03.5113572Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-07T17:52:03.5113993Z  2024-08-07T17:52:03.5114556Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-07T17:52:03.5115231Z  exit 0 2024-08-07T17:52:03.5115533Z fi 2024-08-07T17:52:03.5115797Z  2024-08-07T17:52:03.5116185Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-07T17:52:03.5117016Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-07T17:52:03.5117710Z  exit 1 2024-08-07T17:52:03.5118012Z fi 2024-08-07T17:52:03.5118293Z  2024-08-07T17:52:03.5118741Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-07T17:52:03.5119535Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-07T17:52:03.5120255Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-07T17:52:03.5121077Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-07T17:52:03.5121986Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-07T17:52:03.5122551Z fi 2024-08-07T17:52:03.5122828Z  2024-08-07T17:52:03.5123290Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-07T17:52:03.5129946Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:52:03.5130426Z env: 2024-08-07T17:52:03.5130693Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:52:03.5131042Z DOCKER_BUILD_DIR: .ci/docker 2024-08-07T17:52:03.5131488Z BASE_REVISION: 6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T17:52:03.5132500Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:03.5133527Z DOCKER_TAG: 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:03.5134094Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:52:03.5134560Z ##[endgroup] 2024-08-07T17:52:03.5161185Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:52:03.5161744Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:52:03.5164245Z + aws ecr get-login-password --region us-east-1 2024-08-07T17:52:03.5165660Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:52:04.2021386Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-07T17:52:04.2022117Z Configure a credential helper to remove this warning. See 2024-08-07T17:52:04.2022874Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-07T17:52:04.2023349Z 2024-08-07T17:52:04.2023502Z Login Succeeded 2024-08-07T17:52:04.2035518Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:04.4667688Z { 2024-08-07T17:52:04.4668277Z "schemaVersion": 2, 2024-08-07T17:52:04.4668828Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-07T17:52:04.4669389Z "config": { 2024-08-07T17:52:04.4669873Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-07T17:52:04.4670438Z "size": 48439, 2024-08-07T17:52:04.4671009Z "digest": "sha256:6ec36276acd88c9be8b44d856744037d399b35f4bb1703e637c27ae2b254c901" 2024-08-07T17:52:04.4671654Z }, 2024-08-07T17:52:04.4671932Z "layers": [ 2024-08-07T17:52:04.4672213Z { 2024-08-07T17:52:04.4672975Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4673555Z "size": 28580681, 2024-08-07T17:52:04.4674121Z "digest": "sha256:7a2c559011895d255fce249c00396abff5ae7e0c0a92931d0ed493e71de78e3a" 2024-08-07T17:52:04.4674769Z }, 2024-08-07T17:52:04.4675034Z { 2024-08-07T17:52:04.4675461Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4676032Z "size": 7943451, 2024-08-07T17:52:04.4676585Z "digest": "sha256:224fe954d7252f10539d243d6c9688806f7d13ad775ed02e7f7c79077844510d" 2024-08-07T17:52:04.4677198Z }, 2024-08-07T17:52:04.4677464Z { 2024-08-07T17:52:04.4677916Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4678470Z "size": 55728572, 2024-08-07T17:52:04.4679036Z "digest": "sha256:75722010b82e31715876aeeed0b2cee414296f0124fdfa061ab845ba2a158450" 2024-08-07T17:52:04.4679668Z }, 2024-08-07T17:52:04.4679917Z { 2024-08-07T17:52:04.4680367Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4681068Z "size": 186, 2024-08-07T17:52:04.4681589Z "digest": "sha256:d527cbbb87e3016fd72a18a9b468c945ad0ca27c5770b39debd6ed704db3a195" 2024-08-07T17:52:04.4682181Z }, 2024-08-07T17:52:04.4682434Z { 2024-08-07T17:52:04.4682836Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4683358Z "size": 6886, 2024-08-07T17:52:04.4683871Z "digest": "sha256:b57676e46aee1a8c82e528d78e5a13e31142524eea31c8b213d69ddcb6f1fe80" 2024-08-07T17:52:04.4684430Z }, 2024-08-07T17:52:04.4684689Z { 2024-08-07T17:52:04.4685086Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4685781Z "size": 1329001756, 2024-08-07T17:52:04.4686358Z "digest": "sha256:a8c1e85b5e14cec7af70bf304cb4d4cee6a1d25eb8215b2cf4fdc33e5af5e108" 2024-08-07T17:52:04.4686951Z }, 2024-08-07T17:52:04.4687189Z { 2024-08-07T17:52:04.4687605Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4688165Z "size": 62501, 2024-08-07T17:52:04.4688733Z "digest": "sha256:a41a8d1c11c8d80fe4e82b0d05478f8d51176ff20b8350905fc1b25c93a51198" 2024-08-07T17:52:04.4689371Z }, 2024-08-07T17:52:04.4689614Z { 2024-08-07T17:52:04.4690054Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4690619Z "size": 1684, 2024-08-07T17:52:04.4691155Z "digest": "sha256:0c12278907551c2962927d27c115f6f7bf0df894318b8aea6ece3ef01ccd0a8a" 2024-08-07T17:52:04.4691781Z }, 2024-08-07T17:52:04.4692042Z { 2024-08-07T17:52:04.4692469Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4693034Z "size": 1523, 2024-08-07T17:52:04.4693615Z "digest": "sha256:d8d1234baab3ec9ccb8bb710fc6b8ff6c10896ba2e8d27a347583eca770f9ff1" 2024-08-07T17:52:04.4694255Z }, 2024-08-07T17:52:04.4694617Z { 2024-08-07T17:52:04.4695064Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4695650Z "size": 2528295403, 2024-08-07T17:52:04.4696222Z "digest": "sha256:7ed32bc8e4696fcdb2feef850781160597b2275ad756819c4add88236b0577d5" 2024-08-07T17:52:04.4696862Z }, 2024-08-07T17:52:04.4697608Z { 2024-08-07T17:52:04.4698028Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4698559Z "size": 86016, 2024-08-07T17:52:04.4699096Z "digest": "sha256:ec1e7978c1fe161ced1d98092a51e7c5953ca5fda5577f54df9dbda4afff1b2b" 2024-08-07T17:52:04.4699670Z }, 2024-08-07T17:52:04.4699928Z { 2024-08-07T17:52:04.4700360Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4700911Z "size": 1823, 2024-08-07T17:52:04.4701474Z "digest": "sha256:66b43372aa397c4303ca4e0e1122516909bca0c87b9b4bfb3972b8fd0c1d4390" 2024-08-07T17:52:04.4702119Z }, 2024-08-07T17:52:04.4702366Z { 2024-08-07T17:52:04.4702809Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4703390Z "size": 246768020, 2024-08-07T17:52:04.4704107Z "digest": "sha256:b6662193c745ec6b991e800e920c233379c7c0e74f2f64d9b82dd5dc4a27eb14" 2024-08-07T17:52:04.4704744Z }, 2024-08-07T17:52:04.4705001Z { 2024-08-07T17:52:04.4705447Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4706009Z "size": 545, 2024-08-07T17:52:04.4706576Z "digest": "sha256:5be2b638d110dd5ed631ce7ddf7eefa26b3abd49cf3ab845be5ecb3daec46b67" 2024-08-07T17:52:04.4707240Z }, 2024-08-07T17:52:04.4707502Z { 2024-08-07T17:52:04.4707927Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4708490Z "size": 1283, 2024-08-07T17:52:04.4709056Z "digest": "sha256:71ca63790839b9bfa870ee6927d5d7b60aaa1fc65b38d3e8fc42ace8911859ef" 2024-08-07T17:52:04.4709691Z }, 2024-08-07T17:52:04.4709952Z { 2024-08-07T17:52:04.4710395Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4710954Z "size": 484, 2024-08-07T17:52:04.4711511Z "digest": "sha256:8a74804dc4fa9ad5369e1ae6677a4e17bcc2c53d209a67738dbc795420066650" 2024-08-07T17:52:04.4712171Z }, 2024-08-07T17:52:04.4712422Z { 2024-08-07T17:52:04.4712866Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4713432Z "size": 91712377, 2024-08-07T17:52:04.4713996Z "digest": "sha256:3bacb5389b745ab1f7590db3db714e689a99ee0d7c709f907ccd6906f39905c5" 2024-08-07T17:52:04.4714644Z }, 2024-08-07T17:52:04.4714906Z { 2024-08-07T17:52:04.4715338Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4715902Z "size": 3231, 2024-08-07T17:52:04.4716453Z "digest": "sha256:a8911a72541a4ab35894015b7fb1174ea61c59fedc863dfa563324af5d6ae752" 2024-08-07T17:52:04.4717217Z }, 2024-08-07T17:52:04.4717506Z { 2024-08-07T17:52:04.4717968Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4718525Z "size": 1909, 2024-08-07T17:52:04.4719078Z "digest": "sha256:55d020986bb7c1702235b111c4b83d990fa63ce6045c5ac358a026832bbe8550" 2024-08-07T17:52:04.4719726Z }, 2024-08-07T17:52:04.4719979Z { 2024-08-07T17:52:04.4720422Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4721000Z "size": 700, 2024-08-07T17:52:04.4721544Z "digest": "sha256:679e209a81f89d0be588ce19c3f5191f73883a86e44ab7b3653a3be3f267b69e" 2024-08-07T17:52:04.4722182Z }, 2024-08-07T17:52:04.4722450Z { 2024-08-07T17:52:04.4722884Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4723455Z "size": 2785856582, 2024-08-07T17:52:04.4723891Z + exit 0 2024-08-07T17:52:04.4724407Z "digest": "sha256:d4fb7093f54f7f71e63223ef934b9ab258d53922a199ac4736897cdb90df0683" 2024-08-07T17:52:04.4725048Z }, 2024-08-07T17:52:04.4725299Z { 2024-08-07T17:52:04.4725754Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4726326Z "size": 381, 2024-08-07T17:52:04.4726867Z "digest": "sha256:0d8ab4023e81a9284aef759a1b3c759a907d0cbd39361f3ef0ce4f8c3994f882" 2024-08-07T17:52:04.4727529Z }, 2024-08-07T17:52:04.4727791Z { 2024-08-07T17:52:04.4728222Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4728795Z "size": 12876, 2024-08-07T17:52:04.4729355Z "digest": "sha256:bf191f5f5a0a370ba7136fa618cd8cb1eb76e5f82b8c5773a965cdd105515924" 2024-08-07T17:52:04.4729982Z }, 2024-08-07T17:52:04.4730249Z { 2024-08-07T17:52:04.4730690Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4731239Z "size": 803, 2024-08-07T17:52:04.4731802Z "digest": "sha256:14653e4e245feef24e0aabd8a4cd81c24298f800facc0299f797b161da696a1d" 2024-08-07T17:52:04.4732467Z }, 2024-08-07T17:52:04.4732734Z { 2024-08-07T17:52:04.4733181Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4733722Z "size": 106, 2024-08-07T17:52:04.4734273Z "digest": "sha256:8bdbb000c39dd99342429f8a1183bdb36f312b532ea7e47eb7719fea84c669f6" 2024-08-07T17:52:04.4734911Z }, 2024-08-07T17:52:04.4735260Z { 2024-08-07T17:52:04.4735711Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4736258Z "size": 504, 2024-08-07T17:52:04.4736813Z "digest": "sha256:277383b63c0797c1bd9e23c6f38d6ba85e6e321e2dc6b21fcd832f1935f5af87" 2024-08-07T17:52:04.4737451Z }, 2024-08-07T17:52:04.4737697Z { 2024-08-07T17:52:04.4738139Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4738708Z "size": 121477300, 2024-08-07T17:52:04.4739265Z "digest": "sha256:890313244493db7d65ed3f1cf91a94e6e50bbdb4df87b5bb829a1a3236ffaeb3" 2024-08-07T17:52:04.4739924Z }, 2024-08-07T17:52:04.4740189Z { 2024-08-07T17:52:04.4740621Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4741188Z "size": 109, 2024-08-07T17:52:04.4741767Z "digest": "sha256:f1e3cc0f57ee16caa6ffefa72c065dfe99a5d19a3a352342dfa26b63661589a2" 2024-08-07T17:52:04.4742409Z }, 2024-08-07T17:52:04.4742674Z { 2024-08-07T17:52:04.4743127Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4743683Z "size": 491, 2024-08-07T17:52:04.4744255Z "digest": "sha256:c3cbae3fe054ce8c713ed90c42a306ecc164d8256fd73a14ff7b0e088e150b3f" 2024-08-07T17:52:04.4744907Z }, 2024-08-07T17:52:04.4745155Z { 2024-08-07T17:52:04.4745599Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4746170Z "size": 296, 2024-08-07T17:52:04.4746730Z "digest": "sha256:ccc148c4e7590ced33e52f40edecd2d5ec73cb4a42c87dacaf5c5a7a3912c17b" 2024-08-07T17:52:04.4747401Z }, 2024-08-07T17:52:04.4747667Z { 2024-08-07T17:52:04.4748191Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4748778Z "size": 103, 2024-08-07T17:52:04.4749343Z "digest": "sha256:7912f8c8e80ddc0dfc068c1282e6bd0ffd098b02458818c2c7b52a89c41d8335" 2024-08-07T17:52:04.4749971Z }, 2024-08-07T17:52:04.4750234Z { 2024-08-07T17:52:04.4750679Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4751284Z "size": 1473, 2024-08-07T17:52:04.4751802Z "digest": "sha256:d166ebb28213d6d30940b4fb9739863e9200174e7b550a1591e9028b1a039f83" 2024-08-07T17:52:04.4752382Z }, 2024-08-07T17:52:04.4752625Z { 2024-08-07T17:52:04.4753042Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4753607Z "size": 424146463, 2024-08-07T17:52:04.4754161Z "digest": "sha256:63bf315f789a755602aeb163e43e8173bc191c3dabc75e39ab31d0762bacc84f" 2024-08-07T17:52:04.4754795Z }, 2024-08-07T17:52:04.4755040Z { 2024-08-07T17:52:04.4755493Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4756072Z "size": 159, 2024-08-07T17:52:04.4756603Z "digest": "sha256:bdb818f7b2c8404f3e19777a27592349798986185a1f5b539309bbe8ea96e513" 2024-08-07T17:52:04.4757234Z }, 2024-08-07T17:52:04.4757496Z { 2024-08-07T17:52:04.4757921Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4758498Z "size": 566, 2024-08-07T17:52:04.4759051Z "digest": "sha256:89d8aea05b3a5e45fc1c48daf5ac32901006f7804ce5f2104112c2a2136acf28" 2024-08-07T17:52:04.4759673Z }, 2024-08-07T17:52:04.4759936Z { 2024-08-07T17:52:04.4760375Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4760923Z "size": 35874371, 2024-08-07T17:52:04.4761496Z "digest": "sha256:f1122e19f79064bde97285bf17ca6d8abb889972e5d95a463ffd2382145c1f22" 2024-08-07T17:52:04.4762133Z }, 2024-08-07T17:52:04.4762380Z { 2024-08-07T17:52:04.4762817Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4763383Z "size": 104, 2024-08-07T17:52:04.4763922Z "digest": "sha256:13d6ce3185e9912952041a572e2efa85b4544ec540f6050f750093d180a069f6" 2024-08-07T17:52:04.4764568Z }, 2024-08-07T17:52:04.4764831Z { 2024-08-07T17:52:04.4765261Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4765831Z "size": 425, 2024-08-07T17:52:04.4766548Z "digest": "sha256:feb3f80c392d4aef71730a9673030e955ce0e8a5c41f350eb7a00592d6b0dbb3" 2024-08-07T17:52:04.4767174Z }, 2024-08-07T17:52:04.4767440Z { 2024-08-07T17:52:04.4767887Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4768437Z "size": 20262075, 2024-08-07T17:52:04.4769013Z "digest": "sha256:4fe4cdcdfbd890964b8270a9140a5bf255709a21af4401b0428d91a735e8ac12" 2024-08-07T17:52:04.4769651Z }, 2024-08-07T17:52:04.4769902Z { 2024-08-07T17:52:04.4770341Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4770913Z "size": 440, 2024-08-07T17:52:04.4771469Z "digest": "sha256:be10b99d8ac8cfa04842a726627b1bdc764d3b6f1c591dca7933b86c93208c66" 2024-08-07T17:52:04.4772112Z }, 2024-08-07T17:52:04.4772377Z { 2024-08-07T17:52:04.4772805Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4773376Z "size": 700, 2024-08-07T17:52:04.4773926Z "digest": "sha256:679e209a81f89d0be588ce19c3f5191f73883a86e44ab7b3653a3be3f267b69e" 2024-08-07T17:52:04.4774561Z }, 2024-08-07T17:52:04.4774823Z { 2024-08-07T17:52:04.4775249Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4775811Z "size": 143, 2024-08-07T17:52:04.4776370Z "digest": "sha256:5980a36dfe02695abaecfad21a248c6c1902b07b2c9b69c61c39e342994e2f91" 2024-08-07T17:52:04.4776993Z }, 2024-08-07T17:52:04.4777253Z { 2024-08-07T17:52:04.4777690Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4778241Z "size": 135, 2024-08-07T17:52:04.4778888Z "digest": "sha256:94a4e0b3f19a399451a5f3cc7ddbde73ea16a7f180f7f047bf3ad868072c173f" 2024-08-07T17:52:04.4779550Z }, 2024-08-07T17:52:04.4779794Z { 2024-08-07T17:52:04.4780239Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4780803Z "size": 32, 2024-08-07T17:52:04.4781343Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:52:04.4781994Z }, 2024-08-07T17:52:04.4782254Z { 2024-08-07T17:52:04.4782676Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4783239Z "size": 189, 2024-08-07T17:52:04.4783785Z "digest": "sha256:2012c603f15449503b4671093a9ba6aff4fc99cf4923a92bb446fde7e52d59ee" 2024-08-07T17:52:04.4784400Z }, 2024-08-07T17:52:04.4784658Z { 2024-08-07T17:52:04.4785103Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4785648Z "size": 563, 2024-08-07T17:52:04.4786206Z "digest": "sha256:060890aa9610c5ec0050f85cafaa1f010ff178e2c8b0600aa3c43ad37ed48976" 2024-08-07T17:52:04.4786845Z }, 2024-08-07T17:52:04.4787097Z { 2024-08-07T17:52:04.4787541Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4788102Z "size": 43163116, 2024-08-07T17:52:04.4788663Z "digest": "sha256:c1a64eb8ee12a08340fb5c5a87dc012ff3074a8b683cc399feaa431de7402abd" 2024-08-07T17:52:04.4789316Z }, 2024-08-07T17:52:04.4789580Z { 2024-08-07T17:52:04.4790015Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4790584Z "size": 106, 2024-08-07T17:52:04.4791155Z "digest": "sha256:ed7686d06f1d744c9ec6dd0d75ae1581baefd7809deef8aefa11d54945c7888f" 2024-08-07T17:52:04.4791793Z }, 2024-08-07T17:52:04.4792057Z { 2024-08-07T17:52:04.4792483Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4793054Z "size": 1212, 2024-08-07T17:52:04.4793616Z "digest": "sha256:5c40be0141236773ddf2a3127f247bcc22540d4bebf4f3cc1df53f16f629ee35" 2024-08-07T17:52:04.4794245Z }, 2024-08-07T17:52:04.4794514Z { 2024-08-07T17:52:04.4794961Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4795512Z "size": 700, 2024-08-07T17:52:04.4796069Z "digest": "sha256:679e209a81f89d0be588ce19c3f5191f73883a86e44ab7b3653a3be3f267b69e" 2024-08-07T17:52:04.4796709Z }, 2024-08-07T17:52:04.4797430Z { 2024-08-07T17:52:04.4797906Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4798474Z "size": 138, 2024-08-07T17:52:04.4799013Z "digest": "sha256:95c1963010edc97c994c12e530dc7e5a5717123dfc4378fe8ecca9dbf79de394" 2024-08-07T17:52:04.4799654Z }, 2024-08-07T17:52:04.4799918Z { 2024-08-07T17:52:04.4800345Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4800909Z "size": 120, 2024-08-07T17:52:04.4801455Z "digest": "sha256:5805001913689846871dcb66b59a8d496e3c78fbf4b46c0c55cb11629af04779" 2024-08-07T17:52:04.4802072Z }, 2024-08-07T17:52:04.4802334Z { 2024-08-07T17:52:04.4802787Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4803340Z "size": 1916657670, 2024-08-07T17:52:04.4803933Z "digest": "sha256:b826637ebc384c2f2efbdc841bf6b8f0ac9b6a85060cab5d171d8ed8d49dd3de" 2024-08-07T17:52:04.4804582Z }, 2024-08-07T17:52:04.4804826Z { 2024-08-07T17:52:04.4805275Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4805845Z "size": 173, 2024-08-07T17:52:04.4806374Z "digest": "sha256:859f9c7a63754c26422062903f2a9991578a62fe2f1a81c9ad0f0e9517ab7387" 2024-08-07T17:52:04.4807007Z }, 2024-08-07T17:52:04.4807271Z { 2024-08-07T17:52:04.4807698Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4808263Z "size": 908, 2024-08-07T17:52:04.4808820Z "digest": "sha256:b89ac1530c4a96d2c4c0626a5202eb9e9a05e0d08517e1a5bf165257505309e8" 2024-08-07T17:52:04.4809443Z }, 2024-08-07T17:52:04.4809711Z { 2024-08-07T17:52:04.4810142Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4810850Z "size": 700, 2024-08-07T17:52:04.4811431Z "digest": "sha256:679e209a81f89d0be588ce19c3f5191f73883a86e44ab7b3653a3be3f267b69e" 2024-08-07T17:52:04.4812058Z }, 2024-08-07T17:52:04.4812321Z { 2024-08-07T17:52:04.4812765Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4813329Z "size": 134, 2024-08-07T17:52:04.4813906Z "digest": "sha256:4f10deed2e003fe5f78af780a6b4a71d0107d3ee59e41de3f35c031ca08e9d4d" 2024-08-07T17:52:04.4814552Z }, 2024-08-07T17:52:04.4814800Z { 2024-08-07T17:52:04.4815245Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4815814Z "size": 32, 2024-08-07T17:52:04.4816358Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:52:04.4817007Z }, 2024-08-07T17:52:04.4817276Z { 2024-08-07T17:52:04.4817703Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4818269Z "size": 156, 2024-08-07T17:52:04.4818822Z "digest": "sha256:336420751f1de11d750328660fdb6ebb9051881d009d399e893c10d61ba69b0c" 2024-08-07T17:52:04.4819440Z }, 2024-08-07T17:52:04.4819705Z { 2024-08-07T17:52:04.4820156Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4820723Z "size": 1841, 2024-08-07T17:52:04.4821276Z "digest": "sha256:f7f49611427c9bdc74d97703f780519d1d7d2b95a5377f6f625c8884cbc21d4e" 2024-08-07T17:52:04.4821915Z }, 2024-08-07T17:52:04.4822163Z { 2024-08-07T17:52:04.4822606Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4823170Z "size": 7529783, 2024-08-07T17:52:04.4823721Z "digest": "sha256:628b460c253a663b1f76b99fd2f00d63872fce39b0830a3b45bdeec4f5244660" 2024-08-07T17:52:04.4824353Z }, 2024-08-07T17:52:04.4824614Z { 2024-08-07T17:52:04.4825039Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4825605Z "size": 164, 2024-08-07T17:52:04.4826165Z "digest": "sha256:98e88ff103238559de2c0c76e43d76a01b94584edee356532b7723d1fd39dd85" 2024-08-07T17:52:04.4826783Z }, 2024-08-07T17:52:04.4827043Z { 2024-08-07T17:52:04.4827482Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4828028Z "size": 7944, 2024-08-07T17:52:04.4828719Z "digest": "sha256:6abf825f7962d4bc769dde6a63a4132694ecb9ba0f17006085d8c339aeedf887" 2024-08-07T17:52:04.4829357Z }, 2024-08-07T17:52:04.4829602Z { 2024-08-07T17:52:04.4830044Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4830593Z "size": 8063, 2024-08-07T17:52:04.4831156Z "digest": "sha256:844414c41546bd3c4dd14a45bbd58cca4a2aa0e8f37a781f8c386736ae4d4081" 2024-08-07T17:52:04.4831798Z }, 2024-08-07T17:52:04.4832043Z { 2024-08-07T17:52:04.4832485Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4833047Z "size": 300, 2024-08-07T17:52:04.4833591Z "digest": "sha256:b92a0d83e22950e600ffd0f6391f5c20b499107ba973cab4d7a54a5c65a922b1" 2024-08-07T17:52:04.4834233Z }, 2024-08-07T17:52:04.4834494Z { 2024-08-07T17:52:04.4834916Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4835481Z "size": 7629841, 2024-08-07T17:52:04.4836027Z "digest": "sha256:56e4340bc9e3886f7c099a66772a040a2d34cf0782746af58b0317af979cdfa3" 2024-08-07T17:52:04.4836651Z }, 2024-08-07T17:52:04.4836921Z { 2024-08-07T17:52:04.4837367Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4837912Z "size": 108, 2024-08-07T17:52:04.4838457Z "digest": "sha256:26f48d882588278c8763af295a6bc7147c492d82c6e4395970856a29fb8d77f0" 2024-08-07T17:52:04.4839083Z }, 2024-08-07T17:52:04.4839331Z { 2024-08-07T17:52:04.4839771Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4840335Z "size": 54145778, 2024-08-07T17:52:04.4840898Z "digest": "sha256:b6fe2821ba25ab984577df156aed9b873699ef0f46b6230d8e9a54f9ee22be1e" 2024-08-07T17:52:04.4841642Z }, 2024-08-07T17:52:04.4841925Z { 2024-08-07T17:52:04.4842356Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4842922Z "size": 473, 2024-08-07T17:52:04.4843489Z "digest": "sha256:fae8722cca7f32933f7a25f1491c31ea9a6df4fc1f9fb2360bd29c79b04f1c56" 2024-08-07T17:52:04.4844137Z }, 2024-08-07T17:52:04.4844407Z { 2024-08-07T17:52:04.4844855Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4845407Z "size": 1374858912, 2024-08-07T17:52:04.4845995Z "digest": "sha256:3c7c25c582fced622823798bd877a7fb903ebd4bfecd93c32e43dbd536bb8202" 2024-08-07T17:52:04.4846646Z }, 2024-08-07T17:52:04.4846898Z { 2024-08-07T17:52:04.4847342Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4847890Z "size": 106, 2024-08-07T17:52:04.4848446Z "digest": "sha256:75a49c2f3f0a99be9760740cce745e1ffd508a15bf5ef08077b2032b4d4d97ce" 2024-08-07T17:52:04.4849087Z }, 2024-08-07T17:52:04.4849338Z { 2024-08-07T17:52:04.4849780Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4850353Z "size": 558, 2024-08-07T17:52:04.4850888Z "digest": "sha256:b32c97699ecde27b65bfbbd8ba207755eb28584f2fc64501f4a320045ae969c8" 2024-08-07T17:52:04.4851580Z }, 2024-08-07T17:52:04.4851838Z { 2024-08-07T17:52:04.4852243Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4852808Z "size": 46248557, 2024-08-07T17:52:04.4853366Z "digest": "sha256:b926a85168171349a0ff57c87aa52b9174d3704512eb2e687184f5552883312a" 2024-08-07T17:52:04.4853981Z }, 2024-08-07T17:52:04.4854246Z { 2024-08-07T17:52:04.4854688Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4855242Z "size": 111, 2024-08-07T17:52:04.4855790Z "digest": "sha256:1c5d35b9a7607fd72af03dc281fe78215973e63c51c1b823a704727c8a0944eb" 2024-08-07T17:52:04.4856438Z }, 2024-08-07T17:52:04.4856683Z { 2024-08-07T17:52:04.4857129Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4857702Z "size": 32, 2024-08-07T17:52:04.4858243Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:52:04.4859169Z }, 2024-08-07T17:52:04.4859769Z { 2024-08-07T17:52:04.4860374Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4861208Z "size": 32, 2024-08-07T17:52:04.4862008Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:52:04.4862930Z }, 2024-08-07T17:52:04.4863355Z { 2024-08-07T17:52:04.4864135Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4865094Z "size": 32, 2024-08-07T17:52:04.4866095Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:52:04.4867181Z }, 2024-08-07T17:52:04.4867620Z { 2024-08-07T17:52:04.4868437Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:52:04.4869275Z "size": 32, 2024-08-07T17:52:04.4869826Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:52:04.4870476Z } 2024-08-07T17:52:04.4870747Z ] 2024-08-07T17:52:04.4870994Z } 2024-08-07T17:52:04.4950769Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-07T17:52:04.4951327Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-07T17:52:04.4951837Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-07T17:52:04.4958807Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:52:04.4959269Z env: 2024-08-07T17:52:04.4959556Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:52:04.4960494Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:04.4961448Z ##[endgroup] 2024-08-07T17:52:04.4992516Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9-02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:04.5052978Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-07T17:52:04.5053551Z with: 2024-08-07T17:52:04.5054429Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:04.5055529Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:52:04.5056022Z env: 2024-08-07T17:52:04.5056292Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:52:04.5056649Z ##[endgroup] 2024-08-07T17:52:04.5080723Z ##[group]Run set -x 2024-08-07T17:52:04.5081084Z set -x 2024-08-07T17:52:04.5081371Z set +e 2024-08-07T17:52:04.5081668Z  2024-08-07T17:52:04.5081951Z login() { 2024-08-07T17:52:04.5082557Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-07T17:52:04.5083263Z } 2024-08-07T17:52:04.5083552Z  2024-08-07T17:52:04.5083883Z retry () { 2024-08-07T17:52:04.5084232Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-07T17:52:04.5084657Z } 2024-08-07T17:52:04.5084935Z  2024-08-07T17:52:04.5085229Z retry login "${DOCKER_REGISTRY}" 2024-08-07T17:52:04.5085649Z  2024-08-07T17:52:04.5085927Z set -e 2024-08-07T17:52:04.5086351Z # ignore output since only exit code is used for conditional 2024-08-07T17:52:04.5087001Z # only pull docker image if it's not available locally 2024-08-07T17:52:04.5087737Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-07T17:52:04.5088390Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-07T17:52:04.5088813Z fi 2024-08-07T17:52:04.5094857Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:52:04.5095336Z env: 2024-08-07T17:52:04.5095616Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:52:04.5096639Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:04.5098418Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:52:04.5099130Z ##[endgroup] 2024-08-07T17:52:04.5124665Z + set +e 2024-08-07T17:52:04.5125112Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:52:04.5125656Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:52:04.5128499Z + aws ecr get-login-password --region us-east-1 2024-08-07T17:52:04.5130209Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:52:05.1843001Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-07T17:52:05.1844174Z Configure a credential helper to remove this warning. See 2024-08-07T17:52:05.1845257Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-07T17:52:05.1845781Z 2024-08-07T17:52:05.1845993Z Login Succeeded 2024-08-07T17:52:05.1855984Z + set -e 2024-08-07T17:52:05.1857291Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:05.2019361Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:05.2021141Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:52:05.4396492Z 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9: Pulling from pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9 2024-08-07T17:52:05.4398971Z 7a2c55901189: Pulling fs layer 2024-08-07T17:52:05.4400041Z 224fe954d725: Pulling fs layer 2024-08-07T17:52:05.4400995Z 75722010b82e: Pulling fs layer 2024-08-07T17:52:05.4401909Z d527cbbb87e3: Pulling fs layer 2024-08-07T17:52:05.4403371Z b57676e46aee: Pulling fs layer 2024-08-07T17:52:05.4404336Z a8c1e85b5e14: Pulling fs layer 2024-08-07T17:52:05.4406249Z a41a8d1c11c8: Pulling fs layer 2024-08-07T17:52:05.4407129Z 0c1227890755: Pulling fs layer 2024-08-07T17:52:05.4444091Z d8d1234baab3: Pulling fs layer 2024-08-07T17:52:05.4444513Z 7ed32bc8e469: Pulling fs layer 2024-08-07T17:52:05.4444934Z ec1e7978c1fe: Pulling fs layer 2024-08-07T17:52:05.4445326Z 66b43372aa39: Pulling fs layer 2024-08-07T17:52:05.4445750Z b6662193c745: Pulling fs layer 2024-08-07T17:52:05.4446098Z 5be2b638d110: Pulling fs layer 2024-08-07T17:52:05.4446461Z 71ca63790839: Pulling fs layer 2024-08-07T17:52:05.4446845Z 8a74804dc4fa: Pulling fs layer 2024-08-07T17:52:05.4447241Z 3bacb5389b74: Pulling fs layer 2024-08-07T17:52:05.4447582Z a8911a72541a: Pulling fs layer 2024-08-07T17:52:05.4447980Z 55d020986bb7: Pulling fs layer 2024-08-07T17:52:05.4448286Z 679e209a81f8: Pulling fs layer 2024-08-07T17:52:05.4448648Z d4fb7093f54f: Pulling fs layer 2024-08-07T17:52:05.4449010Z 0d8ab4023e81: Pulling fs layer 2024-08-07T17:52:05.4449368Z a8c1e85b5e14: Waiting 2024-08-07T17:52:05.4449704Z bf191f5f5a0a: Pulling fs layer 2024-08-07T17:52:05.4450053Z 14653e4e245f: Pulling fs layer 2024-08-07T17:52:05.4450423Z 8bdbb000c39d: Pulling fs layer 2024-08-07T17:52:05.4450778Z a41a8d1c11c8: Waiting 2024-08-07T17:52:05.4451078Z b6662193c745: Waiting 2024-08-07T17:52:05.4451438Z 277383b63c07: Pulling fs layer 2024-08-07T17:52:05.4451864Z 890313244493: Pulling fs layer 2024-08-07T17:52:05.4452212Z f1e3cc0f57ee: Pulling fs layer 2024-08-07T17:52:05.4452546Z d527cbbb87e3: Waiting 2024-08-07T17:52:05.4452875Z c3cbae3fe054: Pulling fs layer 2024-08-07T17:52:05.4453238Z 5be2b638d110: Waiting 2024-08-07T17:52:05.4453511Z 0c1227890755: Waiting 2024-08-07T17:52:05.4453831Z ccc148c4e759: Pulling fs layer 2024-08-07T17:52:05.4454208Z 7912f8c8e80d: Pulling fs layer 2024-08-07T17:52:05.4454508Z 71ca63790839: Waiting 2024-08-07T17:52:05.4454828Z d166ebb28213: Pulling fs layer 2024-08-07T17:52:05.4455183Z b57676e46aee: Waiting 2024-08-07T17:52:05.4455483Z ec1e7978c1fe: Waiting 2024-08-07T17:52:05.4455809Z 63bf315f789a: Pulling fs layer 2024-08-07T17:52:05.4456155Z bdb818f7b2c8: Pulling fs layer 2024-08-07T17:52:05.4456514Z 89d8aea05b3a: Pulling fs layer 2024-08-07T17:52:05.4457122Z 7ed32bc8e469: Waiting 2024-08-07T17:52:05.4457433Z f1122e19f790: Pulling fs layer 2024-08-07T17:52:05.4457790Z 13d6ce3185e9: Pulling fs layer 2024-08-07T17:52:05.4458172Z feb3f80c392d: Pulling fs layer 2024-08-07T17:52:05.4458487Z 277383b63c07: Waiting 2024-08-07T17:52:05.4458812Z 4fe4cdcdfbd8: Pulling fs layer 2024-08-07T17:52:05.4459216Z be10b99d8ac8: Pulling fs layer 2024-08-07T17:52:05.4459521Z 5980a36dfe02: Pulling fs layer 2024-08-07T17:52:05.4459870Z f1e3cc0f57ee: Waiting 2024-08-07T17:52:05.4460187Z c3cbae3fe054: Waiting 2024-08-07T17:52:05.4460521Z 94a4e0b3f19a: Pulling fs layer 2024-08-07T17:52:05.4460866Z 4f4fb700ef54: Pulling fs layer 2024-08-07T17:52:05.4461199Z ccc148c4e759: Waiting 2024-08-07T17:52:05.4461514Z 2012c603f154: Pulling fs layer 2024-08-07T17:52:05.4461898Z 060890aa9610: Pulling fs layer 2024-08-07T17:52:05.4462224Z d8d1234baab3: Waiting 2024-08-07T17:52:05.4462531Z c1a64eb8ee12: Pulling fs layer 2024-08-07T17:52:05.4462898Z ed7686d06f1d: Pulling fs layer 2024-08-07T17:52:05.4463297Z 5c40be014123: Pulling fs layer 2024-08-07T17:52:05.4463600Z 95c1963010ed: Pulling fs layer 2024-08-07T17:52:05.4463951Z 580500191368: Pulling fs layer 2024-08-07T17:52:05.4464279Z 63bf315f789a: Waiting 2024-08-07T17:52:05.4464608Z b826637ebc38: Pulling fs layer 2024-08-07T17:52:05.4464962Z 859f9c7a6375: Pulling fs layer 2024-08-07T17:52:05.4465292Z d166ebb28213: Waiting 2024-08-07T17:52:05.4465652Z b89ac1530c4a: Pulling fs layer 2024-08-07T17:52:05.4465986Z 4f10deed2e00: Pulling fs layer 2024-08-07T17:52:05.4466317Z 336420751f1d: Pulling fs layer 2024-08-07T17:52:05.4466675Z f7f49611427c: Pulling fs layer 2024-08-07T17:52:05.4467057Z 3bacb5389b74: Waiting 2024-08-07T17:52:05.4467330Z 628b460c253a: Pulling fs layer 2024-08-07T17:52:05.4467803Z 14653e4e245f: Waiting 2024-08-07T17:52:05.4468152Z 98e88ff10323: Pulling fs layer 2024-08-07T17:52:05.4468485Z 5980a36dfe02: Waiting 2024-08-07T17:52:05.4468813Z 6abf825f7962: Pulling fs layer 2024-08-07T17:52:05.4469157Z 844414c41546: Pulling fs layer 2024-08-07T17:52:05.4469523Z b92a0d83e229: Pulling fs layer 2024-08-07T17:52:05.4469850Z 94a4e0b3f19a: Waiting 2024-08-07T17:52:05.4470173Z 56e4340bc9e3: Pulling fs layer 2024-08-07T17:52:05.4470512Z 26f48d882588: Pulling fs layer 2024-08-07T17:52:05.4470859Z 8bdbb000c39d: Waiting 2024-08-07T17:52:05.4471185Z b6fe2821ba25: Pulling fs layer 2024-08-07T17:52:05.4471529Z fae8722cca7f: Pulling fs layer 2024-08-07T17:52:05.4471903Z 13d6ce3185e9: Waiting 2024-08-07T17:52:05.4472158Z 3c7c25c582fc: Pulling fs layer 2024-08-07T17:52:05.4472505Z 55d020986bb7: Waiting 2024-08-07T17:52:05.4472824Z 75a49c2f3f0a: Pulling fs layer 2024-08-07T17:52:05.4473172Z 890313244493: Waiting 2024-08-07T17:52:05.4473476Z b32c97699ecd: Pulling fs layer 2024-08-07T17:52:05.4473832Z 89d8aea05b3a: Waiting 2024-08-07T17:52:05.4474146Z b926a8516817: Pulling fs layer 2024-08-07T17:52:05.4474500Z feb3f80c392d: Waiting 2024-08-07T17:52:05.4474795Z a8911a72541a: Waiting 2024-08-07T17:52:05.4475106Z d4fb7093f54f: Waiting 2024-08-07T17:52:05.4475432Z f7f49611427c: Waiting 2024-08-07T17:52:05.4475739Z 1c5d35b9a760: Pulling fs layer 2024-08-07T17:52:05.4476087Z 8a74804dc4fa: Waiting 2024-08-07T17:52:05.4476395Z 66b43372aa39: Waiting 2024-08-07T17:52:05.4476722Z 7912f8c8e80d: Waiting 2024-08-07T17:52:05.4476993Z b32c97699ecd: Waiting 2024-08-07T17:52:05.4477302Z 859f9c7a6375: Waiting 2024-08-07T17:52:05.4477593Z b89ac1530c4a: Waiting 2024-08-07T17:52:05.4477923Z b926a8516817: Waiting 2024-08-07T17:52:05.4478190Z 679e209a81f8: Waiting 2024-08-07T17:52:05.4478480Z 1c5d35b9a760: Waiting 2024-08-07T17:52:05.4478789Z c1a64eb8ee12: Waiting 2024-08-07T17:52:05.4479126Z 2012c603f154: Waiting 2024-08-07T17:52:05.4479390Z 4f10deed2e00: Waiting 2024-08-07T17:52:05.4479696Z ed7686d06f1d: Waiting 2024-08-07T17:52:05.4480016Z 4fe4cdcdfbd8: Waiting 2024-08-07T17:52:05.4480324Z 5c40be014123: Waiting 2024-08-07T17:52:05.4480626Z bdb818f7b2c8: Waiting 2024-08-07T17:52:05.4480922Z b6fe2821ba25: Waiting 2024-08-07T17:52:05.4481236Z 4f4fb700ef54: Waiting 2024-08-07T17:52:05.4481657Z fae8722cca7f: Waiting 2024-08-07T17:52:05.4481953Z 6abf825f7962: Waiting 2024-08-07T17:52:05.4482264Z 98e88ff10323: Waiting 2024-08-07T17:52:05.4482567Z 3c7c25c582fc: Waiting 2024-08-07T17:52:05.4482858Z 336420751f1d: Waiting 2024-08-07T17:52:05.4483164Z be10b99d8ac8: Waiting 2024-08-07T17:52:05.4483469Z 844414c41546: Waiting 2024-08-07T17:52:05.4483751Z 580500191368: Waiting 2024-08-07T17:52:05.4484057Z 56e4340bc9e3: Waiting 2024-08-07T17:52:05.4484365Z b92a0d83e229: Waiting 2024-08-07T17:52:05.4484655Z 95c1963010ed: Waiting 2024-08-07T17:52:05.4484963Z 26f48d882588: Waiting 2024-08-07T17:52:05.4485269Z 75a49c2f3f0a: Waiting 2024-08-07T17:52:05.4485560Z bf191f5f5a0a: Waiting 2024-08-07T17:52:05.4485871Z 628b460c253a: Waiting 2024-08-07T17:52:05.4486167Z b826637ebc38: Waiting 2024-08-07T17:52:05.5695843Z 224fe954d725: Verifying Checksum 2024-08-07T17:52:05.5696845Z 224fe954d725: Download complete 2024-08-07T17:52:05.6655771Z d527cbbb87e3: Verifying Checksum 2024-08-07T17:52:05.6656346Z d527cbbb87e3: Download complete 2024-08-07T17:52:05.7502084Z b57676e46aee: Download complete 2024-08-07T17:52:05.7731447Z 7a2c55901189: Verifying Checksum 2024-08-07T17:52:05.7731978Z 7a2c55901189: Download complete 2024-08-07T17:52:05.8580069Z a41a8d1c11c8: Download complete 2024-08-07T17:52:05.9491894Z 0c1227890755: Verifying Checksum 2024-08-07T17:52:05.9492469Z 0c1227890755: Download complete 2024-08-07T17:52:06.0289327Z d8d1234baab3: Verifying Checksum 2024-08-07T17:52:06.0959764Z 75722010b82e: Verifying Checksum 2024-08-07T17:52:06.0960380Z 75722010b82e: Download complete 2024-08-07T17:52:06.1899247Z ec1e7978c1fe: Verifying Checksum 2024-08-07T17:52:06.1900253Z ec1e7978c1fe: Download complete 2024-08-07T17:52:07.1386059Z 7a2c55901189: Pull complete 2024-08-07T17:52:07.4940834Z 224fe954d725: Pull complete 2024-08-07T17:52:08.6358056Z 75722010b82e: Pull complete 2024-08-07T17:52:08.6547732Z d527cbbb87e3: Pull complete 2024-08-07T17:52:08.6747896Z b57676e46aee: Pull complete 2024-08-07T17:52:08.7910557Z b6662193c745: Verifying Checksum 2024-08-07T17:52:08.7911172Z b6662193c745: Download complete 2024-08-07T17:52:08.8629597Z 5be2b638d110: Verifying Checksum 2024-08-07T17:52:08.8630381Z 5be2b638d110: Download complete 2024-08-07T17:52:08.9344320Z 71ca63790839: Verifying Checksum 2024-08-07T17:52:08.9344828Z 71ca63790839: Download complete 2024-08-07T17:52:09.0099454Z 8a74804dc4fa: Download complete 2024-08-07T17:52:09.9784465Z 3bacb5389b74: Verifying Checksum 2024-08-07T17:52:09.9784935Z 3bacb5389b74: Download complete 2024-08-07T17:52:10.0629144Z a8911a72541a: Verifying Checksum 2024-08-07T17:52:10.0629631Z a8911a72541a: Download complete 2024-08-07T17:52:10.1354443Z 55d020986bb7: Verifying Checksum 2024-08-07T17:52:10.1354907Z 55d020986bb7: Download complete 2024-08-07T17:52:10.2123238Z 679e209a81f8: Verifying Checksum 2024-08-07T17:52:10.2124877Z 679e209a81f8: Download complete 2024-08-07T17:52:19.1027591Z a8c1e85b5e14: Verifying Checksum 2024-08-07T17:52:19.1028065Z a8c1e85b5e14: Download complete 2024-08-07T17:52:19.1873633Z 0d8ab4023e81: Verifying Checksum 2024-08-07T17:52:19.1874278Z 0d8ab4023e81: Download complete 2024-08-07T17:52:19.2808112Z bf191f5f5a0a: Download complete 2024-08-07T17:52:19.3630047Z 14653e4e245f: Verifying Checksum 2024-08-07T17:52:19.3630478Z 14653e4e245f: Download complete 2024-08-07T17:52:19.4440005Z 8bdbb000c39d: Download complete 2024-08-07T17:52:19.5262654Z 277383b63c07: Verifying Checksum 2024-08-07T17:52:19.5263097Z 277383b63c07: Download complete 2024-08-07T17:52:20.7914515Z 890313244493: Verifying Checksum 2024-08-07T17:52:20.7915319Z 890313244493: Download complete 2024-08-07T17:52:20.8921987Z f1e3cc0f57ee: Verifying Checksum 2024-08-07T17:52:20.8922718Z f1e3cc0f57ee: Download complete 2024-08-07T17:52:20.9772863Z c3cbae3fe054: Verifying Checksum 2024-08-07T17:52:20.9773318Z c3cbae3fe054: Download complete 2024-08-07T17:52:21.0619267Z ccc148c4e759: Download complete 2024-08-07T17:52:21.1346235Z 7912f8c8e80d: Download complete 2024-08-07T17:52:21.2143751Z d166ebb28213: Download complete 2024-08-07T17:52:25.5421890Z 63bf315f789a: Verifying Checksum 2024-08-07T17:52:25.5422670Z 63bf315f789a: Download complete 2024-08-07T17:52:25.6233530Z bdb818f7b2c8: Verifying Checksum 2024-08-07T17:52:25.6234205Z bdb818f7b2c8: Download complete 2024-08-07T17:52:25.7141770Z 89d8aea05b3a: Verifying Checksum 2024-08-07T17:52:25.7142576Z 89d8aea05b3a: Download complete 2024-08-07T17:52:26.1281046Z f1122e19f790: Verifying Checksum 2024-08-07T17:52:26.1281512Z f1122e19f790: Download complete 2024-08-07T17:52:26.2047925Z 13d6ce3185e9: Verifying Checksum 2024-08-07T17:52:26.2048763Z 13d6ce3185e9: Download complete 2024-08-07T17:52:26.2895086Z feb3f80c392d: Verifying Checksum 2024-08-07T17:52:26.2895697Z feb3f80c392d: Download complete 2024-08-07T17:52:26.5659725Z 4fe4cdcdfbd8: Verifying Checksum 2024-08-07T17:52:26.5660205Z 4fe4cdcdfbd8: Download complete 2024-08-07T17:52:26.6472520Z be10b99d8ac8: Verifying Checksum 2024-08-07T17:52:26.6473006Z be10b99d8ac8: Download complete 2024-08-07T17:52:26.7203628Z 5980a36dfe02: Verifying Checksum 2024-08-07T17:52:26.7204416Z 5980a36dfe02: Download complete 2024-08-07T17:52:26.7970127Z 94a4e0b3f19a: Verifying Checksum 2024-08-07T17:52:26.7970826Z 94a4e0b3f19a: Download complete 2024-08-07T17:52:26.8082046Z 4f4fb700ef54: Verifying Checksum 2024-08-07T17:52:26.8082695Z 4f4fb700ef54: Download complete 2024-08-07T17:52:26.8894490Z 2012c603f154: Download complete 2024-08-07T17:52:26.9724951Z 060890aa9610: Verifying Checksum 2024-08-07T17:52:26.9725773Z 060890aa9610: Download complete 2024-08-07T17:52:27.5978282Z c1a64eb8ee12: Verifying Checksum 2024-08-07T17:52:27.5978766Z c1a64eb8ee12: Download complete 2024-08-07T17:52:27.6720171Z ed7686d06f1d: Verifying Checksum 2024-08-07T17:52:27.6720800Z ed7686d06f1d: Download complete 2024-08-07T17:52:27.7529206Z 5c40be014123: Verifying Checksum 2024-08-07T17:52:27.7529849Z 5c40be014123: Download complete 2024-08-07T17:52:27.8314352Z 95c1963010ed: Verifying Checksum 2024-08-07T17:52:27.8315147Z 95c1963010ed: Download complete 2024-08-07T17:52:27.9194387Z 580500191368: Verifying Checksum 2024-08-07T17:52:27.9195286Z 580500191368: Download complete 2024-08-07T17:52:31.4621488Z 7ed32bc8e469: Verifying Checksum 2024-08-07T17:52:31.4622205Z 7ed32bc8e469: Download complete 2024-08-07T17:52:31.5396928Z 859f9c7a6375: Verifying Checksum 2024-08-07T17:52:31.5398099Z 859f9c7a6375: Download complete 2024-08-07T17:52:31.6186421Z b89ac1530c4a: Verifying Checksum 2024-08-07T17:52:31.6187018Z b89ac1530c4a: Download complete 2024-08-07T17:52:31.6893694Z 4f10deed2e00: Verifying Checksum 2024-08-07T17:52:31.6894159Z 4f10deed2e00: Download complete 2024-08-07T17:52:31.7704752Z 336420751f1d: Verifying Checksum 2024-08-07T17:52:31.7705203Z 336420751f1d: Download complete 2024-08-07T17:52:32.0572249Z f7f49611427c: Download complete 2024-08-07T17:52:32.1900250Z 628b460c253a: Verifying Checksum 2024-08-07T17:52:32.1901049Z 628b460c253a: Download complete 2024-08-07T17:52:32.2895964Z 98e88ff10323: Verifying Checksum 2024-08-07T17:52:32.2896722Z 98e88ff10323: Download complete 2024-08-07T17:52:32.3836224Z 6abf825f7962: Verifying Checksum 2024-08-07T17:52:32.3837058Z 6abf825f7962: Download complete 2024-08-07T17:52:32.4570623Z 844414c41546: Verifying Checksum 2024-08-07T17:52:32.4571148Z 844414c41546: Download complete 2024-08-07T17:52:32.5324723Z b92a0d83e229: Verifying Checksum 2024-08-07T17:52:32.5325408Z b92a0d83e229: Download complete 2024-08-07T17:52:32.6721209Z 56e4340bc9e3: Verifying Checksum 2024-08-07T17:52:32.6721803Z 56e4340bc9e3: Download complete 2024-08-07T17:52:32.7488145Z 26f48d882588: Verifying Checksum 2024-08-07T17:52:32.7489591Z 26f48d882588: Download complete 2024-08-07T17:52:33.3728534Z b6fe2821ba25: Verifying Checksum 2024-08-07T17:52:33.3729056Z b6fe2821ba25: Download complete 2024-08-07T17:52:33.4512573Z fae8722cca7f: Verifying Checksum 2024-08-07T17:52:33.4514072Z fae8722cca7f: Download complete 2024-08-07T17:52:35.6601300Z a8c1e85b5e14: Pull complete 2024-08-07T17:52:36.1083216Z a41a8d1c11c8: Pull complete 2024-08-07T17:52:36.5455844Z 0c1227890755: Pull complete 2024-08-07T17:52:36.9937836Z d8d1234baab3: Pull complete 2024-08-07T17:52:38.1425872Z d4fb7093f54f: Verifying Checksum 2024-08-07T17:52:38.1427151Z d4fb7093f54f: Download complete 2024-08-07T17:52:38.2220642Z 75a49c2f3f0a: Verifying Checksum 2024-08-07T17:52:38.3004716Z 75a49c2f3f0a: Download complete 2024-08-07T17:52:38.3005366Z b32c97699ecd: Download complete 2024-08-07T17:52:38.8211138Z b926a8516817: Verifying Checksum 2024-08-07T17:52:38.8211923Z b926a8516817: Download complete 2024-08-07T17:52:38.9054692Z 1c5d35b9a760: Verifying Checksum 2024-08-07T17:52:38.9055184Z 1c5d35b9a760: Download complete 2024-08-07T17:52:47.1390080Z b826637ebc38: Verifying Checksum 2024-08-07T17:52:47.1390539Z b826637ebc38: Download complete 2024-08-07T17:52:47.2504376Z 3c7c25c582fc: Download complete 2024-08-07T17:53:13.1770217Z 7ed32bc8e469: Pull complete 2024-08-07T17:53:13.6347350Z ec1e7978c1fe: Pull complete 2024-08-07T17:53:14.0845450Z 66b43372aa39: Pull complete 2024-08-07T17:53:23.6968328Z b6662193c745: Pull complete 2024-08-07T17:53:24.1018684Z 5be2b638d110: Pull complete 2024-08-07T17:53:24.5574584Z 71ca63790839: Pull complete 2024-08-07T17:53:25.0026212Z 8a74804dc4fa: Pull complete 2024-08-07T17:53:28.2363989Z 3bacb5389b74: Pull complete 2024-08-07T17:53:28.6607355Z a8911a72541a: Pull complete 2024-08-07T17:53:29.1205985Z 55d020986bb7: Pull complete 2024-08-07T17:53:29.5654307Z 679e209a81f8: Pull complete 2024-08-07T17:54:31.9408414Z d4fb7093f54f: Pull complete 2024-08-07T17:54:32.3917317Z 0d8ab4023e81: Pull complete 2024-08-07T17:54:32.8778820Z bf191f5f5a0a: Pull complete 2024-08-07T17:54:33.3219079Z 14653e4e245f: Pull complete 2024-08-07T17:54:33.7639509Z 8bdbb000c39d: Pull complete 2024-08-07T17:54:34.2174722Z 277383b63c07: Pull complete 2024-08-07T17:54:39.1218531Z 890313244493: Pull complete 2024-08-07T17:54:39.6672676Z f1e3cc0f57ee: Pull complete 2024-08-07T17:54:40.0056548Z c3cbae3fe054: Pull complete 2024-08-07T17:54:40.3722485Z ccc148c4e759: Pull complete 2024-08-07T17:54:40.6363171Z 7912f8c8e80d: Pull complete 2024-08-07T17:54:40.8528821Z d166ebb28213: Pull complete 2024-08-07T17:54:51.8688562Z 63bf315f789a: Pull complete 2024-08-07T17:54:52.2671243Z bdb818f7b2c8: Pull complete 2024-08-07T17:54:52.6069198Z 89d8aea05b3a: Pull complete 2024-08-07T17:54:53.8876426Z f1122e19f790: Pull complete 2024-08-07T17:54:54.3455596Z 13d6ce3185e9: Pull complete 2024-08-07T17:54:54.6519302Z feb3f80c392d: Pull complete 2024-08-07T17:54:55.2647797Z 4fe4cdcdfbd8: Pull complete 2024-08-07T17:54:55.5609452Z be10b99d8ac8: Pull complete 2024-08-07T17:54:56.1034075Z 5980a36dfe02: Pull complete 2024-08-07T17:54:56.4696393Z 94a4e0b3f19a: Pull complete 2024-08-07T17:54:56.8041004Z 4f4fb700ef54: Pull complete 2024-08-07T17:54:57.2393753Z 2012c603f154: Pull complete 2024-08-07T17:54:57.6347949Z 060890aa9610: Pull complete 2024-08-07T17:55:00.8819919Z c1a64eb8ee12: Pull complete 2024-08-07T17:55:01.3329093Z ed7686d06f1d: Pull complete 2024-08-07T17:55:01.7995825Z 5c40be014123: Pull complete 2024-08-07T17:55:02.4948987Z 95c1963010ed: Pull complete 2024-08-07T17:55:02.9040934Z 580500191368: Pull complete 2024-08-07T17:55:46.0101437Z b826637ebc38: Pull complete 2024-08-07T17:55:46.4261101Z 859f9c7a6375: Pull complete 2024-08-07T17:55:46.8796341Z b89ac1530c4a: Pull complete 2024-08-07T17:55:47.7870834Z 4f10deed2e00: Pull complete 2024-08-07T17:55:48.6859010Z 336420751f1d: Pull complete 2024-08-07T17:55:49.1369355Z f7f49611427c: Pull complete 2024-08-07T17:55:49.7858840Z 628b460c253a: Pull complete 2024-08-07T17:55:50.2339579Z 98e88ff10323: Pull complete 2024-08-07T17:55:50.6833424Z 6abf825f7962: Pull complete 2024-08-07T17:55:51.1350931Z 844414c41546: Pull complete 2024-08-07T17:55:51.5849310Z b92a0d83e229: Pull complete 2024-08-07T17:55:53.3796692Z 56e4340bc9e3: Pull complete 2024-08-07T17:55:53.8376538Z 26f48d882588: Pull complete 2024-08-07T17:55:56.5515516Z b6fe2821ba25: Pull complete 2024-08-07T17:55:56.9981075Z fae8722cca7f: Pull complete 2024-08-07T17:56:12.9411250Z 3c7c25c582fc: Pull complete 2024-08-07T17:56:13.2688521Z 75a49c2f3f0a: Pull complete 2024-08-07T17:56:13.7030410Z b32c97699ecd: Pull complete 2024-08-07T17:56:14.9269229Z b926a8516817: Pull complete 2024-08-07T17:56:15.2881145Z 1c5d35b9a760: Pull complete 2024-08-07T17:56:17.1504329Z Digest: sha256:00f47b036f588ca5ef8866f8635fabba5a95cdf9ff1adae7d2a674ef1d4076e9 2024-08-07T17:56:17.2447515Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:56:17.2978442Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:56:17.3034678Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-07T17:56:17.3035552Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-07T17:56:17.3044915Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:56:17.3045421Z env: 2024-08-07T17:56:17.3045695Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:56:17.3046037Z ##[endgroup] 2024-08-07T17:56:17.3709049Z ##[group]Run pytorch/test-infra/.github/actions/setup-nvidia@main 2024-08-07T17:56:17.3709605Z with: 2024-08-07T17:56:17.3709903Z driver-version: 550.54.15 2024-08-07T17:56:17.3710244Z env: 2024-08-07T17:56:17.3710513Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:56:17.3710856Z ##[endgroup] 2024-08-07T17:56:17.3918478Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-07T17:56:17.3919014Z with: 2024-08-07T17:56:17.3919308Z timeout_minutes: 10 2024-08-07T17:56:17.3919626Z max_attempts: 3 2024-08-07T17:56:17.3949870Z command: # Is it disgusting to have a full shell script here in this github action? Sure # But is it the best way to make it so that this action relies on nothing else? Absolutely set -eou pipefail DISTRIBUTION=$(. /etc/os-release;echo $ID$VERSION_ID) DRIVER_FN="NVIDIA-Linux-x86_64-${DRIVER_VERSION}.run" install_nvidia_docker2_amzn2() { ( set -x # Needed for yum-config-manager sudo yum install -y yum-utils if [[ "${DISTRIBUTION}" == "amzn2023" ]] ; then YUM_REPO_URL="https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo" else # Amazon Linux 2 YUM_REPO_URL="https://nvidia.github.io/nvidia-docker/${DISTRIBUTION}/nvidia-docker.repo" fi sudo yum-config-manager --add-repo "${YUM_REPO_URL}" sudo yum install -y nvidia-docker2 sudo systemctl restart docker ) } install_nvidia_docker2_ubuntu20() { ( set -x # Install nvidia-driver package if not installed status="$(dpkg-query -W --showformat='${db:Status-Status}' nvidia-docker2 2>&1)" if [ ! $? = 0 ] || [ ! "$status" = installed ]; then sudo apt-get install -y nvidia-docker2 sudo systemctl restart docker fi ) } pre_install_nvidia_driver_amzn2() { ( # Purge any nvidia driver installed from RHEL repo sudo yum remove -y nvidia-driver-latest-dkms ) } install_nvidia_driver_common() { ( # Try to gather more information about the runner and its existing NVIDIA driver if any echo "Before installing NVIDIA driver" lspci lsmod modinfo nvidia || true HAS_NVIDIA_DRIVER=0 # Check if NVIDIA driver has already been installed if [ -x "$(command -v nvidia-smi)" ]; then set +e # The driver exists, check its version next. Also check only the first GPU if there are more than one of them # so that the same driver version is not print over multiple lines INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then echo "Failed to get NVIDIA driver version ($INSTALLED_DRIVER_VERSION). Continuing" elif [ "$INSTALLED_DRIVER_VERSION" != "$DRIVER_VERSION" ]; then echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has been installed, but we expect to have $DRIVER_VERSION instead. Continuing" else HAS_NVIDIA_DRIVER=1 echo "NVIDIA driver ($INSTALLED_DRIVER_VERSION) has already been installed. Skipping NVIDIA driver installation" fi set -e fi if [ "$HAS_NVIDIA_DRIVER" -eq 0 ]; then # CAUTION: this may need to be updated in future if [ "${DISTRIBUTION}" != ubuntu20.04 ]; then sudo yum groupinstall -y "Development Tools" # ensure our kernel install is the same as our underlying kernel, # groupinstall "Development Tools" has a habit of mismatching kernel headers sudo yum install -y "kernel-devel-uname-r == $(uname -r)" sudo modprobe backlight fi sudo curl -fsL -o /tmp/nvidia_driver "https://s3.amazonaws.com/ossci-linux/nvidia_driver/$DRIVER_FN" set +e sudo /bin/bash /tmp/nvidia_driver -s --no-drm NVIDIA_INSTALLATION_STATUS=$? RESET_GPU=0 if [ "$NVIDIA_INSTALLATION_STATUS" -ne 0 ]; then sudo cat /var/log/nvidia-installer.log # Fail to install NVIDIA driver, try to reset the GPU RESET_GPU=1 elif [ -x "$(command -v nvidia-smi)" ]; then # Check again if nvidia-smi works even if the driver installation completes successfully INSTALLED_DRIVER_VERSION=$(nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0) NVIDIA_SMI_STATUS=$? if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then RESET_GPU=1 fi fi if [ "$RESET_GPU" -eq 1 ]; then NVIDIA_DEVICES=$(lspci -D | grep -i NVIDIA | cut -d' ' -f1) # The GPU can get stuck in a failure state if somehow the test crashs the GPU microcode. When this # happens, we'll try to reset all NVIDIA devices https://github.com/pytorch/pytorch/issues/88388 for PCI_ID in $NVIDIA_DEVICES; do DEVICE_ENABLED=$(cat /sys/bus/pci/devices/$PCI_ID/enable) echo "Reseting $PCI_ID (enabled state: $DEVICE_ENABLED)" # This requires sudo permission of course echo "1" | sudo tee /sys/bus/pci/devices/$PCI_ID/reset sleep 1 done fi sudo rm -fv /tmp/nvidia_driver set -e fi ) } post_install_nvidia_driver_common() { ( sudo modprobe nvidia || true echo "After installing NVIDIA driver" lspci lsmod modinfo nvidia || true ( set +e nvidia-smi # NB: Annoyingly, nvidia-smi command returns successfully with return code 0 even in # the case where the driver has already crashed as it still can get the driver version # and some basic information like the bus ID. However, the rest of the information # would be missing (ERR!), for example: # # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | # | | | MIG M. | # |===============================+======================+======================| # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | # | | | ERR! | # +-------------------------------+----------------------+----------------------+ # # +-----------------------------------------------------------------------------+ # | Processes: | # | GPU GI CI PID Type Process name GPU Memory | # | ID ID Usage | # |=============================================================================| # +-----------------------------------------------------------------------------+ # # This should be reported as a failure instead as it will guarantee to fail when # Docker tries to run with --gpus all # # So, the correct check here is to query one of the missing piece of info like # GPU name, so that the command can fail accordingly nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 NVIDIA_SMI_STATUS=$? # Allowable exit statuses for nvidia-smi, see: https://github.com/NVIDIA/gpu-operator/issues/285 if [ "$NVIDIA_SMI_STATUS" -eq 0 ] || [ "$NVIDIA_SMI_STATUS" -eq 14 ]; then echo "INFO: Ignoring allowed status ${NVIDIA_SMI_STATUS}" else echo "ERROR: nvidia-smi exited with unresolved status ${NVIDIA_SMI_STATUS}" exit ${NVIDIA_SMI_STATUS} fi set -e ) ) } install_nvidia_driver_amzn2() { ( set -x pre_install_nvidia_driver_amzn2 install_nvidia_driver_common post_install_nvidia_driver_common ) } install_nvidia_driver_ubuntu20() { ( set -x install_nvidia_driver_common post_install_nvidia_driver_common ) } echo "== Installing nvidia driver ${DRIVER_FN} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_driver_amzn2 ;; ubuntu20.04) install_nvidia_driver_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac # Install container toolkit based on distribution echo "== Installing nvidia container toolkit for ${DISTRIBUTION} ==" case "${DISTRIBUTION}" in amzn*) install_nvidia_docker2_amzn2 ;; ubuntu20.04) install_nvidia_docker2_ubuntu20 ;; *) echo "ERROR: Unknown distribution ${DISTRIBUTION}" exit 1 ;; esac echo "GPU_FLAG=--gpus all -e NVIDIA_DRIVER_CAPABILITIES=all" >> "${GITHUB_ENV}" # Fix https://github.com/NVIDIA/nvidia-docker/issues/1648 on runners with # more than one GPUs. This just needs to be run once. The command fails # on subsequent runs and complains that the mode is already on, but that's # ok sudo nvidia-persistenced || true # This should show persistence mode ON nvidia-smi 2024-08-07T17:56:17.3980470Z retry_wait_seconds: 10 2024-08-07T17:56:17.3980839Z polling_interval_seconds: 1 2024-08-07T17:56:17.3981198Z warning_on_retry: true 2024-08-07T17:56:17.3981547Z continue_on_error: false 2024-08-07T17:56:17.3981886Z env: 2024-08-07T17:56:17.3982154Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:56:17.3982506Z DRIVER_VERSION: 550.54.15 2024-08-07T17:56:17.3982836Z ##[endgroup] 2024-08-07T17:56:17.4953889Z == Installing nvidia driver NVIDIA-Linux-x86_64-550.54.15.run == 2024-08-07T17:56:17.4956211Z + pre_install_nvidia_driver_amzn2 2024-08-07T17:56:17.4956671Z + sudo yum remove -y nvidia-driver-latest-dkms 2024-08-07T17:56:18.1376820Z No match for argument: nvidia-driver-latest-dkms 2024-08-07T17:56:18.1380269Z No packages marked for removal. 2024-08-07T17:56:18.1457244Z Dependencies resolved. 2024-08-07T17:56:18.1470375Z Nothing to do. 2024-08-07T17:56:18.1473012Z Complete! 2024-08-07T17:56:18.2892597Z + install_nvidia_driver_common 2024-08-07T17:56:18.2895644Z + echo 'Before installing NVIDIA driver' 2024-08-07T17:56:18.2898403Z Before installing NVIDIA driver 2024-08-07T17:56:18.2899077Z + lspci 2024-08-07T17:56:18.3791383Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-08-07T17:56:18.3792113Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-07T17:56:18.3792807Z 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 2024-08-07T17:56:18.3793483Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) 2024-08-07T17:56:18.3794034Z 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 2024-08-07T17:56:18.3794645Z 00:03.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-07T17:56:18.3795378Z 00:1e.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-08-07T17:56:18.3796489Z 00:1f.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) 2024-08-07T17:56:18.3797474Z + lsmod 2024-08-07T17:56:18.3834575Z Module Size Used by 2024-08-07T17:56:18.3835285Z xt_conntrack 16384 1 2024-08-07T17:56:18.3836043Z nft_chain_nat 16384 3 2024-08-07T17:56:18.3837351Z xt_MASQUERADE 20480 1 2024-08-07T17:56:18.3838254Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-07T17:56:18.3839107Z nf_conntrack_netlink 57344 0 2024-08-07T17:56:18.3840051Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-07T17:56:18.3840776Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-07T17:56:18.3841357Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-07T17:56:18.3841953Z xfrm_user 57344 1 2024-08-07T17:56:18.3842453Z xfrm_algo 16384 1 xfrm_user 2024-08-07T17:56:18.3842942Z xt_addrtype 16384 2 2024-08-07T17:56:18.3843478Z nft_compat 20480 4 2024-08-07T17:56:18.3844043Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-07T17:56:18.3844703Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-07T17:56:18.3845399Z br_netfilter 36864 0 2024-08-07T17:56:18.3845924Z bridge 307200 1 br_netfilter 2024-08-07T17:56:18.3846409Z stp 16384 1 bridge 2024-08-07T17:56:18.3847016Z llc 16384 2 bridge,stp 2024-08-07T17:56:18.3847588Z overlay 167936 0 2024-08-07T17:56:18.3848013Z tls 114688 0 2024-08-07T17:56:18.3848547Z ata_piix 45056 0 2024-08-07T17:56:18.3849051Z libata 401408 1 ata_piix 2024-08-07T17:56:18.3849528Z scsi_mod 290816 1 libata 2024-08-07T17:56:18.3850090Z ena 167936 0 2024-08-07T17:56:18.3850615Z scsi_common 16384 2 scsi_mod,libata 2024-08-07T17:56:18.3851140Z ghash_clmulni_intel 16384 0 2024-08-07T17:56:18.3851669Z ptp 36864 1 ena 2024-08-07T17:56:18.3852202Z pps_core 24576 1 ptp 2024-08-07T17:56:18.3852671Z nls_ascii 16384 1 2024-08-07T17:56:18.3853192Z aesni_intel 393216 0 2024-08-07T17:56:18.3853730Z nls_cp437 20480 1 2024-08-07T17:56:18.3854178Z i8042 45056 0 2024-08-07T17:56:18.3854706Z crypto_simd 16384 1 aesni_intel 2024-08-07T17:56:18.3855251Z serio 28672 3 i8042 2024-08-07T17:56:18.3855800Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-07T17:56:18.3856432Z vfat 24576 1 2024-08-07T17:56:18.3856959Z fat 86016 1 vfat 2024-08-07T17:56:18.3857505Z button 24576 0 2024-08-07T17:56:18.3857943Z sunrpc 692224 1 2024-08-07T17:56:18.3858420Z sch_fq_codel 20480 9 2024-08-07T17:56:18.3858945Z dm_mod 188416 0 2024-08-07T17:56:18.3859381Z fuse 163840 1 2024-08-07T17:56:18.3859898Z loop 36864 0 2024-08-07T17:56:18.3860707Z dax 45056 1 dm_mod 2024-08-07T17:56:18.3861171Z configfs 57344 1 2024-08-07T17:56:18.3861651Z dmi_sysfs 20480 0 2024-08-07T17:56:18.3862211Z crc32_pclmul 16384 0 2024-08-07T17:56:18.3862640Z crc32c_intel 24576 0 2024-08-07T17:56:18.3863113Z + modinfo nvidia 2024-08-07T17:56:18.3863813Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-07T17:56:18.3864484Z alias: char-major-195-* 2024-08-07T17:56:18.3864991Z version: 550.54.15 2024-08-07T17:56:18.3865527Z supported: external 2024-08-07T17:56:18.3865939Z license: NVIDIA 2024-08-07T17:56:18.3866478Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-07T17:56:18.3867131Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-07T17:56:18.3867642Z srcversion: 833721318DA517F0C2FEC97 2024-08-07T17:56:18.3868218Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-07T17:56:18.3868876Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-07T17:56:18.3869508Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-07T17:56:18.3870096Z depends: i2c-core,drm 2024-08-07T17:56:18.3870646Z retpoline: Y 2024-08-07T17:56:18.3871027Z name: nvidia 2024-08-07T17:56:18.3871650Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-07T17:56:18.3872465Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-07T17:56:18.3873243Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-07T17:56:18.3873881Z parm: NVreg_ResmanDebugLevel:int 2024-08-07T17:56:18.3874486Z parm: NVreg_RmLogonRC:int 2024-08-07T17:56:18.3875038Z parm: NVreg_ModifyDeviceFiles:int 2024-08-07T17:56:18.3875542Z parm: NVreg_DeviceFileUID:int 2024-08-07T17:56:18.3876134Z parm: NVreg_DeviceFileGID:int 2024-08-07T17:56:18.3876690Z parm: NVreg_DeviceFileMode:int 2024-08-07T17:56:18.3877262Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-07T17:56:18.3877972Z parm: NVreg_UsePageAttributeTable:int 2024-08-07T17:56:18.3878578Z parm: NVreg_EnablePCIeGen3:int 2024-08-07T17:56:18.3879062Z parm: NVreg_EnableMSI:int 2024-08-07T17:56:18.3879701Z parm: NVreg_TCEBypassMode:int 2024-08-07T17:56:18.3880326Z parm: NVreg_EnableStreamMemOPs:int 2024-08-07T17:56:18.3880888Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-07T17:56:18.3881620Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-07T17:56:18.3882255Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-07T17:56:18.3882879Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-07T17:56:18.3883621Z parm: NVreg_DynamicPowerManagement:int 2024-08-07T17:56:18.3884313Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-07T17:56:18.3884937Z parm: NVreg_EnableGpuFirmware:int 2024-08-07T17:56:18.3885601Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-07T17:56:18.3886231Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-07T17:56:18.3886819Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-07T17:56:18.3887500Z parm: NVreg_MemoryPoolSize:int 2024-08-07T17:56:18.3888113Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-07T17:56:18.3888640Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-07T17:56:18.3889308Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-07T17:56:18.3889858Z parm: NVreg_NvLinkDisable:int 2024-08-07T17:56:18.3890403Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-07T17:56:18.3891097Z parm: NVreg_RegisterPCIDriver:int 2024-08-07T17:56:18.3891672Z parm: NVreg_EnableResizableBar:int 2024-08-07T17:56:18.3892217Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-07T17:56:18.3892880Z parm: NVreg_EnableNonblockingOpen:int 2024-08-07T17:56:18.3893577Z parm: NVreg_RegistryDwords:charp 2024-08-07T17:56:18.3894162Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-07T17:56:18.3894826Z parm: NVreg_RmMsg:charp 2024-08-07T17:56:18.3895364Z parm: NVreg_GpuBlacklist:charp 2024-08-07T17:56:18.3895913Z parm: NVreg_TemporaryFilePath:charp 2024-08-07T17:56:18.3896525Z parm: NVreg_ExcludedGpus:charp 2024-08-07T17:56:18.3897513Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-07T17:56:18.3898101Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-07T17:56:18.3898749Z parm: NVreg_ImexChannelCount:int 2024-08-07T17:56:18.3899321Z parm: rm_firmware_active:charp 2024-08-07T17:56:18.3899820Z + HAS_NVIDIA_DRIVER=0 2024-08-07T17:56:18.3900312Z ++ command -v nvidia-smi 2024-08-07T17:56:18.3900820Z + '[' -x /usr/bin/nvidia-smi ']' 2024-08-07T17:56:18.3901286Z + set +e 2024-08-07T17:56:18.3901913Z ++ nvidia-smi --query-gpu=driver_version --format=csv,noheader --id=0 2024-08-07T17:56:21.1572801Z + INSTALLED_DRIVER_VERSION=550.54.15 2024-08-07T17:56:21.1573267Z + NVIDIA_SMI_STATUS=0 2024-08-07T17:56:21.1574002Z + '[' 0 -ne 0 ']' 2024-08-07T17:56:21.1574368Z + '[' 550.54.15 '!=' 550.54.15 ']' 2024-08-07T17:56:21.1574725Z + HAS_NVIDIA_DRIVER=1 2024-08-07T17:56:21.1575317Z + echo 'NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation' 2024-08-07T17:56:21.1575952Z + set -e 2024-08-07T17:56:21.1576232Z + '[' 1 -eq 0 ']' 2024-08-07T17:56:21.1576763Z NVIDIA driver (550.54.15) has already been installed. Skipping NVIDIA driver installation 2024-08-07T17:56:21.1577387Z + post_install_nvidia_driver_common 2024-08-07T17:56:21.1579609Z + sudo modprobe nvidia 2024-08-07T17:56:21.2806081Z + echo 'After installing NVIDIA driver' 2024-08-07T17:56:21.2806558Z + lspci 2024-08-07T17:56:21.2806870Z After installing NVIDIA driver 2024-08-07T17:56:21.3016783Z 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02) 2024-08-07T17:56:21.3017516Z 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] 2024-08-07T17:56:21.3018272Z 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] 2024-08-07T17:56:21.3018926Z 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01) 2024-08-07T17:56:21.3019498Z 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 2024-08-07T17:56:21.3020113Z 00:03.0 Ethernet controller: Amazon.com, Inc. Elastic Network Adapter (ENA) 2024-08-07T17:56:21.3020829Z 00:1e.0 VGA compatible controller: NVIDIA Corporation GM204GL [Tesla M60] (rev a1) 2024-08-07T17:56:21.3021559Z 00:1f.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01) 2024-08-07T17:56:21.3022101Z + lsmod 2024-08-07T17:56:21.3047778Z Module Size Used by 2024-08-07T17:56:21.3048192Z nvidia_uvm 4706304 0 2024-08-07T17:56:21.3048573Z nvidia 54071296 1 nvidia_uvm 2024-08-07T17:56:21.3048980Z drm 602112 1 nvidia 2024-08-07T17:56:21.3049387Z drm_panel_orientation_quirks 28672 1 drm 2024-08-07T17:56:21.3049833Z backlight 24576 1 drm 2024-08-07T17:56:21.3050241Z i2c_core 106496 2 nvidia,drm 2024-08-07T17:56:21.3050632Z xt_conntrack 16384 1 2024-08-07T17:56:21.3051005Z nft_chain_nat 16384 3 2024-08-07T17:56:21.3051378Z xt_MASQUERADE 20480 1 2024-08-07T17:56:21.3051775Z nf_nat 57344 2 nft_chain_nat,xt_MASQUERADE 2024-08-07T17:56:21.3052235Z nf_conntrack_netlink 57344 0 2024-08-07T17:56:21.3052778Z nf_conntrack 184320 4 xt_conntrack,nf_nat,nf_conntrack_netlink,xt_MASQUERADE 2024-08-07T17:56:21.3053350Z nf_defrag_ipv6 24576 1 nf_conntrack 2024-08-07T17:56:21.3053783Z nf_defrag_ipv4 16384 1 nf_conntrack 2024-08-07T17:56:21.3054194Z xfrm_user 57344 1 2024-08-07T17:56:21.3054549Z xfrm_algo 16384 1 xfrm_user 2024-08-07T17:56:21.3054947Z xt_addrtype 16384 2 2024-08-07T17:56:21.3055311Z nft_compat 20480 4 2024-08-07T17:56:21.3056005Z nf_tables 307200 57 nft_compat,nft_chain_nat 2024-08-07T17:56:21.3056579Z nfnetlink 20480 4 nft_compat,nf_conntrack_netlink,nf_tables 2024-08-07T17:56:21.3057096Z br_netfilter 36864 0 2024-08-07T17:56:21.3057462Z bridge 307200 1 br_netfilter 2024-08-07T17:56:21.3057870Z stp 16384 1 bridge 2024-08-07T17:56:21.3058266Z llc 16384 2 bridge,stp 2024-08-07T17:56:21.3058644Z overlay 167936 0 2024-08-07T17:56:21.3058995Z tls 114688 0 2024-08-07T17:56:21.3059352Z ata_piix 45056 0 2024-08-07T17:56:21.3059702Z libata 401408 1 ata_piix 2024-08-07T17:56:21.3060097Z scsi_mod 290816 1 libata 2024-08-07T17:56:21.3060478Z ena 167936 0 2024-08-07T17:56:21.3060848Z scsi_common 16384 2 scsi_mod,libata 2024-08-07T17:56:21.3061274Z ghash_clmulni_intel 16384 0 2024-08-07T17:56:21.3061629Z ptp 36864 1 ena 2024-08-07T17:56:21.3062021Z pps_core 24576 1 ptp 2024-08-07T17:56:21.3062575Z nls_ascii 16384 1 2024-08-07T17:56:21.3062947Z aesni_intel 393216 0 2024-08-07T17:56:21.3063308Z nls_cp437 20480 1 2024-08-07T17:56:21.3063657Z i8042 45056 0 2024-08-07T17:56:21.3064016Z crypto_simd 16384 1 aesni_intel 2024-08-07T17:56:21.3064428Z serio 28672 3 i8042 2024-08-07T17:56:21.3064885Z cryptd 28672 2 crypto_simd,ghash_clmulni_intel 2024-08-07T17:56:21.3065334Z vfat 24576 1 2024-08-07T17:56:21.3065696Z fat 86016 1 vfat 2024-08-07T17:56:21.3066069Z button 24576 0 2024-08-07T17:56:21.3066404Z sunrpc 692224 1 2024-08-07T17:56:21.3066766Z sch_fq_codel 20480 9 2024-08-07T17:56:21.3067122Z dm_mod 188416 0 2024-08-07T17:56:21.3067456Z fuse 163840 1 2024-08-07T17:56:21.3067814Z loop 36864 0 2024-08-07T17:56:21.3068184Z dax 45056 1 dm_mod 2024-08-07T17:56:21.3068558Z configfs 57344 1 2024-08-07T17:56:21.3068921Z dmi_sysfs 20480 0 2024-08-07T17:56:21.3069281Z crc32_pclmul 16384 0 2024-08-07T17:56:21.3069623Z crc32c_intel 24576 0 2024-08-07T17:56:21.3070008Z + modinfo nvidia 2024-08-07T17:56:21.3070520Z filename: /lib/modules/6.1.94-99.176.amzn2023.x86_64/kernel/drivers/video/nvidia.ko 2024-08-07T17:56:21.3071111Z alias: char-major-195-* 2024-08-07T17:56:21.3071496Z version: 550.54.15 2024-08-07T17:56:21.3071841Z supported: external 2024-08-07T17:56:21.3072165Z license: NVIDIA 2024-08-07T17:56:21.3072531Z firmware: nvidia/550.54.15/gsp_tu10x.bin 2024-08-07T17:56:21.3072992Z firmware: nvidia/550.54.15/gsp_ga10x.bin 2024-08-07T17:56:21.3073415Z srcversion: 833721318DA517F0C2FEC97 2024-08-07T17:56:21.3073849Z alias: pci:v000010DEd*sv*sd*bc06sc80i00* 2024-08-07T17:56:21.3074325Z alias: pci:v000010DEd*sv*sd*bc03sc02i00* 2024-08-07T17:56:21.3074773Z alias: pci:v000010DEd*sv*sd*bc03sc00i00* 2024-08-07T17:56:21.3075207Z depends: i2c-core,drm 2024-08-07T17:56:21.3075567Z retpoline: Y 2024-08-07T17:56:21.3075864Z name: nvidia 2024-08-07T17:56:21.3076341Z vermagic: 6.1.94-99.176.amzn2023.x86_64 SMP preempt mod_unload modversions 2024-08-07T17:56:21.3076956Z parm: NvSwitchRegDwords:NvSwitch regkey (charp) 2024-08-07T17:56:21.3077541Z parm: NvSwitchBlacklist:NvSwitchBlacklist=uuid[,uuid...] (charp) 2024-08-07T17:56:21.3078108Z parm: NVreg_ResmanDebugLevel:int 2024-08-07T17:56:21.3078534Z parm: NVreg_RmLogonRC:int 2024-08-07T17:56:21.3078933Z parm: NVreg_ModifyDeviceFiles:int 2024-08-07T17:56:21.3079369Z parm: NVreg_DeviceFileUID:int 2024-08-07T17:56:21.3079836Z parm: NVreg_DeviceFileGID:int 2024-08-07T17:56:21.3080245Z parm: NVreg_DeviceFileMode:int 2024-08-07T17:56:21.3080863Z parm: NVreg_InitializeSystemMemoryAllocations:int 2024-08-07T17:56:21.3081401Z parm: NVreg_UsePageAttributeTable:int 2024-08-07T17:56:21.3081844Z parm: NVreg_EnablePCIeGen3:int 2024-08-07T17:56:21.3082259Z parm: NVreg_EnableMSI:int 2024-08-07T17:56:21.3082661Z parm: NVreg_TCEBypassMode:int 2024-08-07T17:56:21.3083081Z parm: NVreg_EnableStreamMemOPs:int 2024-08-07T17:56:21.3083574Z parm: NVreg_RestrictProfilingToAdminUsers:int 2024-08-07T17:56:21.3084110Z parm: NVreg_PreserveVideoMemoryAllocations:int 2024-08-07T17:56:21.3084611Z parm: NVreg_EnableS0ixPowerManagement:int 2024-08-07T17:56:21.3085165Z parm: NVreg_S0ixPowerManagementVideoMemoryThreshold:int 2024-08-07T17:56:21.3085714Z parm: NVreg_DynamicPowerManagement:int 2024-08-07T17:56:21.3086260Z parm: NVreg_DynamicPowerManagementVideoMemoryThreshold:int 2024-08-07T17:56:21.3086816Z parm: NVreg_EnableGpuFirmware:int 2024-08-07T17:56:21.3087292Z parm: NVreg_EnableGpuFirmwareLogs:int 2024-08-07T17:56:21.3087869Z parm: NVreg_OpenRmEnableUnsupportedGpus:int 2024-08-07T17:56:21.3088408Z parm: NVreg_EnableUserNUMAManagement:int 2024-08-07T17:56:21.3088874Z parm: NVreg_MemoryPoolSize:int 2024-08-07T17:56:21.3089296Z parm: NVreg_KMallocHeapMaxSize:int 2024-08-07T17:56:21.3089752Z parm: NVreg_VMallocHeapMaxSize:int 2024-08-07T17:56:21.3090198Z parm: NVreg_IgnoreMMIOCheck:int 2024-08-07T17:56:21.3090612Z parm: NVreg_NvLinkDisable:int 2024-08-07T17:56:21.3091089Z parm: NVreg_EnablePCIERelaxedOrderingMode:int 2024-08-07T17:56:21.3091579Z parm: NVreg_RegisterPCIDriver:int 2024-08-07T17:56:21.3092012Z parm: NVreg_EnableResizableBar:int 2024-08-07T17:56:21.3092474Z parm: NVreg_EnableDbgBreakpoint:int 2024-08-07T17:56:21.3092950Z parm: NVreg_EnableNonblockingOpen:int 2024-08-07T17:56:21.3093402Z parm: NVreg_RegistryDwords:charp 2024-08-07T17:56:21.3093877Z parm: NVreg_RegistryDwordsPerDevice:charp 2024-08-07T17:56:21.3094337Z parm: NVreg_RmMsg:charp 2024-08-07T17:56:21.3094717Z parm: NVreg_GpuBlacklist:charp 2024-08-07T17:56:21.3095164Z parm: NVreg_TemporaryFilePath:charp 2024-08-07T17:56:21.3095617Z parm: NVreg_ExcludedGpus:charp 2024-08-07T17:56:21.3096035Z parm: NVreg_DmaRemapPeerMmio:int 2024-08-07T17:56:21.3096487Z parm: NVreg_RmNvlinkBandwidth:charp 2024-08-07T17:56:21.3096940Z parm: NVreg_ImexChannelCount:int 2024-08-07T17:56:21.3097576Z parm: rm_firmware_active:charp 2024-08-07T17:56:21.3097969Z + set +e 2024-08-07T17:56:21.3098258Z + nvidia-smi 2024-08-07T17:56:23.1868662Z Wed Aug 7 17:56:23 2024 2024-08-07T17:56:23.1869231Z +-----------------------------------------------------------------------------------------+ 2024-08-07T17:56:23.1869914Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-07T17:56:23.1870619Z |-----------------------------------------+------------------------+----------------------+ 2024-08-07T17:56:23.1871307Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-07T17:56:23.1872017Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-07T17:56:23.1872614Z | | | MIG M. | 2024-08-07T17:56:23.1873070Z |=========================================+========================+======================| 2024-08-07T17:56:23.1915601Z | 0 Tesla M60 Off | 00000000:00:1E.0 Off | 0 | 2024-08-07T17:56:23.1916211Z | N/A 33C P0 39W / 150W | 0MiB / 7680MiB | 99% Default | 2024-08-07T17:56:23.1916739Z | | | N/A | 2024-08-07T17:56:23.1917695Z +-----------------------------------------+------------------------+----------------------+ 2024-08-07T17:56:23.1918248Z 2024-08-07T17:56:23.1918788Z +-----------------------------------------------------------------------------------------+ 2024-08-07T17:56:23.1919355Z | Processes: | 2024-08-07T17:56:23.1919967Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-07T17:56:23.1920542Z | ID ID Usage | 2024-08-07T17:56:23.1921007Z |=========================================================================================| 2024-08-07T17:56:23.1921576Z | No running processes found | 2024-08-07T17:56:23.1922218Z +-----------------------------------------------------------------------------------------+ 2024-08-07T17:56:23.4664946Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-07T17:56:25.3370383Z Tesla M60 2024-08-07T17:56:25.5868841Z + NVIDIA_SMI_STATUS=0 2024-08-07T17:56:25.5869268Z + '[' 0 -eq 0 ']' 2024-08-07T17:56:25.5869615Z + echo 'INFO: Ignoring allowed status 0' 2024-08-07T17:56:25.5870013Z + set -e 2024-08-07T17:56:25.5871754Z INFO: Ignoring allowed status 0 2024-08-07T17:56:25.5875251Z == Installing nvidia container toolkit for amzn2023 == 2024-08-07T17:56:25.5879967Z + sudo yum install -y yum-utils 2024-08-07T17:56:26.1449699Z Last metadata expiration check: 0:08:35 ago on Wed Aug 7 17:47:51 2024. 2024-08-07T17:56:26.1743932Z Package dnf-utils-4.3.0-13.amzn2023.0.4.noarch is already installed. 2024-08-07T17:56:26.2178485Z Dependencies resolved. 2024-08-07T17:56:26.2362030Z Nothing to do. 2024-08-07T17:56:26.2362534Z Complete! 2024-08-07T17:56:26.2781442Z + [[ amzn2023 == \a\m\z\n\2\0\2\3 ]] 2024-08-07T17:56:26.2782834Z + YUM_REPO_URL=https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-07T17:56:26.2785018Z + sudo yum-config-manager --add-repo https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-07T17:56:26.6308262Z Adding repo from: https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo 2024-08-07T17:56:26.7103545Z + sudo yum install -y nvidia-docker2 2024-08-07T17:56:27.3473851Z nvidia-container-toolkit 8.3 kB/s | 833 B 00:00 2024-08-07T17:56:27.3778533Z Package nvidia-docker2-2.14.0-1.noarch is already installed. 2024-08-07T17:56:27.4196450Z Dependencies resolved. 2024-08-07T17:56:27.4382132Z Nothing to do. 2024-08-07T17:56:27.4382884Z Complete! 2024-08-07T17:56:27.6906942Z + sudo systemctl restart docker 2024-08-07T17:56:48.8421419Z Wed Aug 7 17:56:48 2024 2024-08-07T17:56:48.8422178Z +-----------------------------------------------------------------------------------------+ 2024-08-07T17:56:48.8422918Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-07T17:56:48.8423584Z |-----------------------------------------+------------------------+----------------------+ 2024-08-07T17:56:48.8424400Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-07T17:56:48.8425183Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-07T17:56:48.8425748Z | | | MIG M. | 2024-08-07T17:56:48.8426213Z |=========================================+========================+======================| 2024-08-07T17:56:48.8505269Z | 0 Tesla M60 On | 00000000:00:1E.0 Off | 0 | 2024-08-07T17:56:48.8505852Z | N/A 34C P0 38W / 150W | 0MiB / 7680MiB | 99% Default | 2024-08-07T17:56:48.8506381Z | | | N/A | 2024-08-07T17:56:48.8507325Z +-----------------------------------------+------------------------+----------------------+ 2024-08-07T17:56:48.8507861Z 2024-08-07T17:56:48.8508371Z +-----------------------------------------------------------------------------------------+ 2024-08-07T17:56:48.8508942Z | Processes: | 2024-08-07T17:56:48.8509552Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-07T17:56:48.8510105Z | ID ID Usage | 2024-08-07T17:56:48.8510573Z |=========================================================================================| 2024-08-07T17:56:48.8511147Z | No running processes found | 2024-08-07T17:56:48.8511952Z +-----------------------------------------------------------------------------------------+ 2024-08-07T17:56:49.5041499Z Command completed after 1 attempt(s). 2024-08-07T17:56:49.5115130Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-07T17:56:49.5115885Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-07T17:56:49.5116510Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-07T17:56:49.5117102Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:56:49.5128460Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:56:49.5128940Z env: 2024-08-07T17:56:49.5129231Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:56:49.5129639Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:56:49.5130093Z ##[endgroup] 2024-08-07T17:56:49.8946366Z Defaulting to user installation because normal site-packages is not writeable 2024-08-07T17:56:50.4152590Z Collecting psutil==5.9.1 2024-08-07T17:56:50.4663543Z Downloading psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB) 2024-08-07T17:56:50.5401465Z Collecting nvidia-ml-py==11.525.84 2024-08-07T17:56:50.5450002Z Downloading nvidia_ml_py-11.525.84-py3-none-any.whl (34 kB) 2024-08-07T17:56:50.6572284Z Installing collected packages: psutil, nvidia-ml-py 2024-08-07T17:56:50.8638216Z Successfully installed nvidia-ml-py-11.525.84 psutil-5.9.1 2024-08-07T17:56:51.0993873Z Prepare all required actions 2024-08-07T17:56:51.0995550Z Getting action download info 2024-08-07T17:56:51.2243764Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-07T17:56:51.4802928Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-07T17:56:51.6603765Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-07T17:56:51.6604252Z with: 2024-08-07T17:56:51.6604573Z name: linux-focal-cuda12.1-py3.10-gcc9 2024-08-07T17:56:51.6604976Z s3-bucket: gha-artifacts 2024-08-07T17:56:51.6605315Z env: 2024-08-07T17:56:51.6605600Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:56:51.6606006Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:56:51.6606459Z ##[endgroup] 2024-08-07T17:56:51.6647426Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-07T17:56:51.6647866Z with: 2024-08-07T17:56:51.6648233Z name: linux-focal-cuda12.1-py3.10-gcc9 2024-08-07T17:56:51.6648638Z s3-bucket: gha-artifacts 2024-08-07T17:56:51.6648986Z region: us-east-1 2024-08-07T17:56:51.6649290Z env: 2024-08-07T17:56:51.6649553Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:56:51.6649983Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:56:51.6650443Z ##[endgroup] 2024-08-07T17:56:52.2889454Z (node:53139) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-07T17:56:52.2890446Z 2024-08-07T17:56:52.2890701Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-07T17:56:52.2891368Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-07T17:56:52.2892054Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-07T17:56:52.3746980Z Found 1 objects with prefix pytorch/pytorch/10288745067/linux-focal-cuda12.1-py3.10-gcc9/ 2024-08-07T17:56:52.3747914Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-07T17:57:16.8521325Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-07T17:57:16.8532134Z Artifact download has finished successfully 2024-08-07T17:57:16.8767442Z ##[group]Run unzip -o artifacts.zip 2024-08-07T17:57:16.8767889Z unzip -o artifacts.zip 2024-08-07T17:57:16.8775461Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:57:16.8775942Z env: 2024-08-07T17:57:16.8776255Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:16.8776667Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:16.8777122Z ##[endgroup] 2024-08-07T17:57:16.8847095Z Archive: artifacts.zip 2024-08-07T17:57:16.8848605Z creating: dist/ 2024-08-07T17:57:19.2777232Z inflating: dist/torch-2.5.0a0+git016588f-cp310-cp310-linux_x86_64.whl 2024-08-07T17:57:19.2777841Z creating: build/custom_test_artifacts/ 2024-08-07T17:57:19.2778354Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-07T17:57:19.2779009Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-07T17:57:19.2779737Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-07T17:57:19.2787888Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-07T17:57:19.2788729Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-07T17:57:19.2789527Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-07T17:57:19.2790521Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-07T17:57:19.2791470Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-07T17:57:19.2792916Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-07T17:57:19.2795775Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-07T17:57:19.2796701Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-07T17:57:19.2798011Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-07T17:57:19.2799059Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-07T17:57:19.2801391Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-07T17:57:19.2803115Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-07T17:57:19.2804160Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-07T17:57:19.2806534Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-07T17:57:19.2807815Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-07T17:57:19.2808720Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-07T17:57:19.2809614Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-07T17:57:19.2856642Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-07T17:57:19.2903697Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-07T17:57:19.2905369Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-07T17:57:19.2958455Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-07T17:57:19.2959815Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-07T17:57:19.2961237Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-07T17:57:19.2962691Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-07T17:57:19.2964084Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-07T17:57:19.2965454Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-07T17:57:19.2966833Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-07T17:57:19.2968228Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-07T17:57:19.2969565Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-07T17:57:19.2970828Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-07T17:57:19.2972072Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-07T17:57:19.2973318Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-07T17:57:19.2974534Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-07T17:57:19.2975729Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-07T17:57:19.2976941Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-07T17:57:19.3047082Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-07T17:57:19.3119915Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-07T17:57:19.3120963Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-07T17:57:19.3121927Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-07T17:57:19.3122801Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/ 2024-08-07T17:57:19.3123708Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-07T17:57:19.3124614Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-07T17:57:19.3125644Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-07T17:57:19.3126787Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-07T17:57:19.3127888Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-07T17:57:19.3128938Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-07T17:57:19.3130002Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-07T17:57:19.3131071Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-07T17:57:19.3132326Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-07T17:57:19.3133412Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-07T17:57:19.3134470Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-07T17:57:19.3155701Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-07T17:57:19.3326780Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-07T17:57:19.3327687Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-07T17:57:19.3328628Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-07T17:57:19.3329716Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-07T17:57:19.3330764Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-07T17:57:19.3331701Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-07T17:57:19.3332698Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-07T17:57:19.3333718Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-07T17:57:19.3334727Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-07T17:57:19.3335734Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-07T17:57:19.3336673Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-07T17:57:19.3360603Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-07T17:57:19.3460851Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-07T17:57:19.3461948Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-07T17:57:19.3463038Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-07T17:57:19.3464219Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-07T17:57:19.3465119Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-07T17:57:19.3466044Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-07T17:57:19.3466936Z inflating: build/custom_test_artifacts/custom-op-build/detect_cuda_version.cc 2024-08-07T17:57:19.3470261Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-07T17:57:19.3470998Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-07T17:57:19.3472068Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-07T17:57:19.3616618Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-07T17:57:19.3692141Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-07T17:57:19.3692781Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-07T17:57:19.3693406Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-07T17:57:19.3694092Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-07T17:57:19.3702927Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-07T17:57:19.3703741Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-07T17:57:19.3704545Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-07T17:57:19.3705660Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-07T17:57:19.3706478Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-07T17:57:19.3707813Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-07T17:57:19.3710055Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-07T17:57:19.3710943Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-07T17:57:19.3711750Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-07T17:57:19.3713864Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-07T17:57:19.3715661Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-07T17:57:19.3717939Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-07T17:57:19.3718962Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-07T17:57:19.3721077Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-07T17:57:19.3722352Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-07T17:57:19.3723250Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-07T17:57:19.3724093Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-07T17:57:19.3770985Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-07T17:57:19.3818744Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-07T17:57:19.3819981Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-07T17:57:19.3873703Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-07T17:57:19.3875099Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-07T17:57:19.3876725Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-07T17:57:19.3878202Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-07T17:57:19.3879611Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-07T17:57:19.3880992Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-07T17:57:19.3882390Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-07T17:57:19.3883739Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-07T17:57:19.3885089Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-07T17:57:19.3886405Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-07T17:57:19.3887518Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-07T17:57:19.3888598Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-07T17:57:19.3889673Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-07T17:57:19.3890878Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-07T17:57:19.3891930Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-07T17:57:19.3962153Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-07T17:57:19.4035199Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-07T17:57:19.4036268Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-07T17:57:19.4037099Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-07T17:57:19.4037854Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/ 2024-08-07T17:57:19.4038672Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-07T17:57:19.4039461Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-07T17:57:19.4040424Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-07T17:57:19.4041453Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-07T17:57:19.4042475Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-07T17:57:19.4043402Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-07T17:57:19.4044379Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-07T17:57:19.4045379Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-07T17:57:19.4046357Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-07T17:57:19.4047386Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-07T17:57:19.4048493Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-07T17:57:19.4071975Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-07T17:57:19.4148830Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-07T17:57:19.4149934Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-07T17:57:19.4150916Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-07T17:57:19.4151889Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-07T17:57:19.4152758Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-07T17:57:19.4153680Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-07T17:57:19.4154577Z inflating: build/custom_test_artifacts/jit-hook-build/detect_cuda_version.cc 2024-08-07T17:57:19.4158043Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-07T17:57:19.4158791Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-07T17:57:19.4159994Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-07T17:57:19.4220387Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-07T17:57:19.4221024Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-07T17:57:19.4221684Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-07T17:57:19.4222452Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-07T17:57:19.4230134Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-07T17:57:19.4231060Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-07T17:57:19.4232010Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-07T17:57:19.4233065Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-07T17:57:19.4234072Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-07T17:57:19.4235226Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-07T17:57:19.4237114Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-07T17:57:19.4238043Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-07T17:57:19.4238956Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-07T17:57:19.4241194Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-07T17:57:19.4242886Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-07T17:57:19.4245151Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-07T17:57:19.4246239Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-07T17:57:19.4248088Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-07T17:57:19.4249655Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-07T17:57:19.4250747Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/ 2024-08-07T17:57:19.4251778Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/ 2024-08-07T17:57:19.4298750Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp4.ii 2024-08-07T17:57:19.4345563Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.cpp 2024-08-07T17:57:19.4347328Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.module_id 2024-08-07T17:57:19.4400858Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cpp1.ii 2024-08-07T17:57:19.4402250Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.c 2024-08-07T17:57:19.4403712Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.gpu 2024-08-07T17:57:19.4405231Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.cudafe1.stub.c 2024-08-07T17:57:19.4406729Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.ptx 2024-08-07T17:57:19.4408186Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.sm_52.cubin 2024-08-07T17:57:19.4409658Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin 2024-08-07T17:57:19.4411119Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.fatbin.c 2024-08-07T17:57:19.4412513Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/CMakeCUDACompilerId.o 2024-08-07T17:57:19.4414389Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.sm_52.cubin 2024-08-07T17:57:19.4415696Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.reg.c 2024-08-07T17:57:19.4416970Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin 2024-08-07T17:57:19.4418285Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.fatbin.c 2024-08-07T17:57:19.4419555Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/tmp/a_dlink.o 2024-08-07T17:57:19.4420817Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/CMakeCUDACompilerId.cu 2024-08-07T17:57:19.4489309Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCUDA/a.out 2024-08-07T17:57:19.4562432Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CUDA.bin 2024-08-07T17:57:19.4563582Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCUDACompiler.cmake 2024-08-07T17:57:19.4564502Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-07T17:57:19.4565307Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/ 2024-08-07T17:57:19.4566162Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-07T17:57:19.4567058Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-07T17:57:19.4568059Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-07T17:57:19.4569198Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-07T17:57:19.4570299Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-07T17:57:19.4571307Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-07T17:57:19.4572357Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-07T17:57:19.4573605Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-07T17:57:19.4574700Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-07T17:57:19.4575745Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-07T17:57:19.4576795Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-07T17:57:19.4579737Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-07T17:57:19.4730234Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-07T17:57:19.4731261Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-07T17:57:19.4732334Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-07T17:57:19.4733514Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-07T17:57:19.4734661Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-07T17:57:19.4735719Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-07T17:57:19.4736817Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-07T17:57:19.4738114Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-07T17:57:19.4739213Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-07T17:57:19.4740391Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-07T17:57:19.4741641Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-07T17:57:19.4763960Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-07T17:57:19.4830525Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-07T17:57:19.4831712Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-07T17:57:19.4832806Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-07T17:57:19.4833825Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-07T17:57:19.4834762Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-07T17:57:19.4836040Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-07T17:57:19.4836941Z inflating: build/custom_test_artifacts/custom-backend-build/detect_cuda_version.cc 2024-08-07T17:57:19.4839770Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-07T17:57:19.4840849Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-07T17:57:19.4841691Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-07T17:57:19.4966586Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-07T17:57:19.5018620Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-07T17:57:19.5019240Z creating: build/lib/ 2024-08-07T17:57:19.5030826Z inflating: build/lib/libpthreadpool.a 2024-08-07T17:57:19.5040770Z inflating: build/lib/libcpuinfo.a 2024-08-07T17:57:19.5050354Z inflating: build/lib/libcpuinfo_internals.a 2024-08-07T17:57:19.5051481Z inflating: build/lib/libclog.a 2024-08-07T17:57:19.5158585Z inflating: build/lib/libprotobuf-lite.a 2024-08-07T17:57:19.5161030Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-07T17:57:19.5707754Z inflating: build/lib/libprotobuf.a 2024-08-07T17:57:19.5785847Z inflating: build/lib/libgtest.a 2024-08-07T17:57:19.5876670Z inflating: build/lib/libbenchmark.a 2024-08-07T17:57:19.5885943Z inflating: build/lib/libittnotify.a 2024-08-07T17:57:19.5908342Z inflating: build/lib/libnnpack.a 2024-08-07T17:57:19.5941781Z inflating: build/lib/libtensorpipe_uv.a 2024-08-07T17:57:19.6019675Z inflating: build/lib/libasmjit.a 2024-08-07T17:57:19.6172653Z inflating: build/lib/libgloo.a 2024-08-07T17:57:19.6198345Z inflating: build/lib/libfmt.a 2024-08-07T17:57:19.6312971Z inflating: build/lib/libc10.so 2024-08-07T17:57:19.6315109Z inflating: build/lib/libcaffe2_nvrtc.so 2024-08-07T17:57:19.6316062Z inflating: build/lib/libfoxi_loader.a 2024-08-07T17:57:19.6318094Z inflating: build/lib/libtorch_global_deps.so 2024-08-07T17:57:19.6341557Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-07T17:57:19.6363409Z inflating: build/lib/libgmock.a 2024-08-07T17:57:19.6364404Z inflating: build/lib/libgtest_main.a 2024-08-07T17:57:19.6380638Z inflating: build/lib/libbenchmark_main.a 2024-08-07T17:57:20.8469830Z inflating: build/lib/libdnnl.a 2024-08-07T17:57:20.9074806Z inflating: build/lib/libprotoc.a 2024-08-07T17:57:20.9760821Z inflating: build/lib/libtensorpipe.a 2024-08-07T17:57:20.9830465Z inflating: build/lib/libc10_cuda.so 2024-08-07T17:57:21.1326919Z inflating: build/lib/libfbgemm.a 2024-08-07T17:57:21.1327807Z inflating: build/lib/libgmock_main.a 2024-08-07T17:57:21.1634029Z inflating: build/lib/libtensorpipe_cuda.a 2024-08-07T17:57:21.2249622Z inflating: build/lib/libkineto.a 2024-08-07T17:57:21.2710048Z inflating: build/lib/libgloo_cuda.a 2024-08-07T17:57:21.2760591Z inflating: build/lib/libonnx_proto.a 2024-08-07T17:57:21.2983279Z inflating: build/lib/libXNNPACK.a 2024-08-07T17:57:21.3813687Z inflating: build/lib/libonnx.a 2024-08-07T17:57:24.2944726Z inflating: build/lib/libtorch_cpu.so 2024-08-07T17:57:24.2949615Z inflating: build/lib/libshm.so 2024-08-07T17:57:24.2955528Z inflating: build/lib/libunbox_lib.a 2024-08-07T17:57:26.7468457Z inflating: build/lib/libtorch_cuda.so 2024-08-07T17:57:26.7469653Z inflating: build/lib/libtorch.so 2024-08-07T17:57:26.7473262Z inflating: build/lib/libc10d_cuda_test.so 2024-08-07T17:57:27.7429907Z inflating: build/lib/libtorch_cuda_linalg.so 2024-08-07T17:57:27.7454753Z inflating: build/lib/libjitbackend_test.so 2024-08-07T17:57:27.7541483Z inflating: build/lib/libtorchbind_test.so 2024-08-07T17:57:27.7572857Z inflating: build/lib/libaoti_custom_ops.so 2024-08-07T17:57:27.7604023Z inflating: build/lib/libbackend_with_compiler.so 2024-08-07T17:57:28.0004653Z inflating: build/lib/libtorch_python.so 2024-08-07T17:57:28.0046260Z inflating: build/lib/libnnapi_backend.so 2024-08-07T17:57:28.0046677Z creating: build/bin/ 2024-08-07T17:57:28.0108016Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-07T17:57:28.0170489Z inflating: build/bin/c10_DeviceGuard_test 2024-08-07T17:57:28.0231905Z inflating: build/bin/c10_Device_test 2024-08-07T17:57:28.0303894Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-07T17:57:28.0368531Z inflating: build/bin/c10_Scalar_test 2024-08-07T17:57:28.0428465Z inflating: build/bin/c10_StreamGuard_test 2024-08-07T17:57:28.0490644Z inflating: build/bin/c10_SymInt_test 2024-08-07T17:57:28.0557364Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-07T17:57:28.0625240Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-07T17:57:28.0693444Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-07T17:57:28.0779821Z inflating: build/bin/c10_cow_test 2024-08-07T17:57:28.0844380Z inflating: build/bin/c10_Bitset_test 2024-08-07T17:57:28.0904694Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-07T17:57:28.0964650Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-07T17:57:28.1026689Z inflating: build/bin/c10_Half_test 2024-08-07T17:57:28.1094915Z inflating: build/bin/c10_LeftRight_test 2024-08-07T17:57:28.1162131Z inflating: build/bin/c10_Metaprogramming_test 2024-08-07T17:57:28.1223329Z inflating: build/bin/c10_Synchronized_test 2024-08-07T17:57:28.1290677Z inflating: build/bin/c10_ThreadLocal_test 2024-08-07T17:57:28.1353494Z inflating: build/bin/c10_TypeIndex_test 2024-08-07T17:57:28.1416188Z inflating: build/bin/c10_TypeList_test 2024-08-07T17:57:28.1475650Z inflating: build/bin/c10_TypeTraits_test 2024-08-07T17:57:28.1539269Z inflating: build/bin/c10_accumulate_test 2024-08-07T17:57:28.1606940Z inflating: build/bin/c10_bfloat16_test 2024-08-07T17:57:28.1668348Z inflating: build/bin/c10_bit_cast_test 2024-08-07T17:57:28.1736879Z inflating: build/bin/c10_complex_math_test 2024-08-07T17:57:28.1804978Z inflating: build/bin/c10_complex_test 2024-08-07T17:57:28.1868787Z inflating: build/bin/c10_exception_test 2024-08-07T17:57:28.1930227Z inflating: build/bin/c10_flags_test 2024-08-07T17:57:28.1992041Z inflating: build/bin/c10_irange_test 2024-08-07T17:57:28.2053035Z inflating: build/bin/c10_generic_math_test 2024-08-07T17:57:28.2251378Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-07T17:57:28.2316539Z inflating: build/bin/c10_lazy_test 2024-08-07T17:57:28.2385914Z inflating: build/bin/c10_logging_test 2024-08-07T17:57:28.2476888Z inflating: build/bin/c10_optional_test 2024-08-07T17:57:28.2543111Z inflating: build/bin/c10_registry_test 2024-08-07T17:57:28.2619066Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-07T17:57:28.2801806Z inflating: build/bin/c10_small_vector_test 2024-08-07T17:57:28.2864362Z inflating: build/bin/c10_ssize_test 2024-08-07T17:57:28.2927690Z inflating: build/bin/c10_string_util_test 2024-08-07T17:57:28.2988772Z inflating: build/bin/c10_tempfile_test 2024-08-07T17:57:28.3060517Z inflating: build/bin/c10_string_view_test 2024-08-07T17:57:28.3128692Z inflating: build/bin/c10_typeid_test 2024-08-07T17:57:28.3187627Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-07T17:57:28.3730328Z inflating: build/bin/protoc-3.13.0.0 2024-08-07T17:57:28.4272492Z inflating: build/bin/protoc 2024-08-07T17:57:28.4336953Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_1_var_test 2024-08-07T17:57:28.4401437Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_stream 2024-08-07T17:57:28.4465452Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_catches_thread_and_block_and_device 2024-08-07T17:57:28.4528788Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_from_2_processes 2024-08-07T17:57:28.4593107Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_blocks_and_threads 2024-08-07T17:57:28.4657584Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_multiple_blocks 2024-08-07T17:57:28.4721752Z inflating: build/bin/c10_cuda_CUDAAssertionsTest_multiple_writes_from_same_block 2024-08-07T17:57:28.4781199Z inflating: build/bin/c10_cuda_CUDATest 2024-08-07T17:57:28.5178584Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-07T17:57:28.5592411Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-07T17:57:28.6025245Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-07T17:57:28.6090020Z inflating: build/bin/HashStoreTest 2024-08-07T17:57:28.6154623Z inflating: build/bin/FileStoreTest 2024-08-07T17:57:28.6218914Z inflating: build/bin/BackoffTest 2024-08-07T17:57:28.6286489Z inflating: build/bin/TCPStoreTest 2024-08-07T17:57:28.6303552Z inflating: build/bin/ProcessGroupMPITest 2024-08-07T17:57:28.6308841Z inflating: build/bin/torch_shm_manager 2024-08-07T17:57:28.6374316Z inflating: build/bin/test_edge_op_registration 2024-08-07T17:57:28.6378097Z inflating: build/bin/example_allreduce 2024-08-07T17:57:28.6441599Z inflating: build/bin/Dimname_test 2024-08-07T17:57:28.6530938Z inflating: build/bin/Dict_test 2024-08-07T17:57:28.6609573Z inflating: build/bin/MaybeOwned_test 2024-08-07T17:57:28.6679102Z inflating: build/bin/NamedTensor_test 2024-08-07T17:57:28.6751546Z inflating: build/bin/apply_utils_test 2024-08-07T17:57:28.6823233Z inflating: build/bin/atest 2024-08-07T17:57:28.6900184Z inflating: build/bin/basic 2024-08-07T17:57:28.6966737Z inflating: build/bin/broadcast_test 2024-08-07T17:57:28.7028884Z inflating: build/bin/cpu_allocator_test 2024-08-07T17:57:28.7093528Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-07T17:57:28.7164450Z inflating: build/bin/cpu_generator_test 2024-08-07T17:57:28.7225651Z inflating: build/bin/dispatch_key_set_test 2024-08-07T17:57:28.7338592Z inflating: build/bin/cpu_rng_test 2024-08-07T17:57:28.7400088Z inflating: build/bin/dlconvertor_test 2024-08-07T17:57:28.7470768Z inflating: build/bin/extension_backend_test 2024-08-07T17:57:28.7538010Z inflating: build/bin/half_test 2024-08-07T17:57:28.7654641Z inflating: build/bin/ivalue_test 2024-08-07T17:57:28.7715368Z inflating: build/bin/lazy_tensor_test 2024-08-07T17:57:28.7781262Z inflating: build/bin/math_kernel_test 2024-08-07T17:57:28.7847315Z inflating: build/bin/memory_format_test 2024-08-07T17:57:28.7912108Z inflating: build/bin/memory_overlapping_test 2024-08-07T17:57:28.7976438Z inflating: build/bin/mobile_memory_cleanup 2024-08-07T17:57:28.8038386Z inflating: build/bin/operator_name_test 2024-08-07T17:57:28.8106552Z inflating: build/bin/native_test 2024-08-07T17:57:28.8170567Z inflating: build/bin/packedtensoraccessor_test 2024-08-07T17:57:28.8232612Z inflating: build/bin/operators_test 2024-08-07T17:57:28.8314386Z inflating: build/bin/pow_test 2024-08-07T17:57:28.8383764Z inflating: build/bin/quantized_test 2024-08-07T17:57:28.8445477Z inflating: build/bin/reduce_ops_test 2024-08-07T17:57:28.8507905Z inflating: build/bin/reportMemoryUsage_test 2024-08-07T17:57:28.8576481Z inflating: build/bin/scalar_tensor_test 2024-08-07T17:57:28.8639840Z inflating: build/bin/StorageUtils_test 2024-08-07T17:57:28.8711338Z inflating: build/bin/scalar_test 2024-08-07T17:57:28.8775004Z inflating: build/bin/stride_properties_test 2024-08-07T17:57:28.8870792Z inflating: build/bin/tensor_iterator_test 2024-08-07T17:57:28.8936819Z inflating: build/bin/test_parallel 2024-08-07T17:57:28.9005407Z inflating: build/bin/type_ptr_test 2024-08-07T17:57:28.9009037Z inflating: build/bin/thread_init_test 2024-08-07T17:57:28.9081926Z inflating: build/bin/type_test 2024-08-07T17:57:28.9145664Z inflating: build/bin/undefined_tensor_test 2024-08-07T17:57:28.9147684Z inflating: build/bin/verify_api_visibility 2024-08-07T17:57:28.9232141Z inflating: build/bin/legacy_vmap_test 2024-08-07T17:57:28.9294799Z inflating: build/bin/weakref_test 2024-08-07T17:57:28.9357825Z inflating: build/bin/wrapdim_test 2024-08-07T17:57:28.9421470Z inflating: build/bin/xla_tensor_test 2024-08-07T17:57:28.9549567Z inflating: build/bin/List_test 2024-08-07T17:57:28.9622720Z inflating: build/bin/IListRef_test 2024-08-07T17:57:28.9770107Z inflating: build/bin/kernel_function_legacy_test 2024-08-07T17:57:28.9887355Z inflating: build/bin/kernel_function_test 2024-08-07T17:57:28.9968229Z inflating: build/bin/KernelFunction_test 2024-08-07T17:57:29.0123770Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-07T17:57:29.0197801Z inflating: build/bin/kernel_stackbased_test 2024-08-07T17:57:29.0324779Z inflating: build/bin/kernel_lambda_test 2024-08-07T17:57:29.0386471Z inflating: build/bin/CppSignature_test 2024-08-07T17:57:29.0503839Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-07T17:57:29.0571551Z inflating: build/bin/backend_fallback_test 2024-08-07T17:57:29.0631774Z inflating: build/bin/op_allowlist_test 2024-08-07T17:57:29.0708845Z inflating: build/bin/inline_container_test 2024-08-07T17:57:29.1073016Z inflating: build/bin/op_registration_test 2024-08-07T17:57:29.1137715Z inflating: build/bin/cuda_apply_test 2024-08-07T17:57:29.1201176Z inflating: build/bin/cuda_allocator_test 2024-08-07T17:57:29.1272578Z inflating: build/bin/cuda_atomic_ops_test 2024-08-07T17:57:29.1339370Z inflating: build/bin/cuda_caching_host_allocator_test 2024-08-07T17:57:29.1424834Z inflating: build/bin/cuda_complex_math_test 2024-08-07T17:57:29.1496556Z inflating: build/bin/cuda_complex_test 2024-08-07T17:57:29.1556989Z inflating: build/bin/cuda_device_test 2024-08-07T17:57:29.1627168Z inflating: build/bin/cuda_cub_test 2024-08-07T17:57:29.1688795Z inflating: build/bin/cuda_dlconvertor_test 2024-08-07T17:57:29.1751407Z inflating: build/bin/cuda_integer_divider_test 2024-08-07T17:57:29.1831415Z inflating: build/bin/cuda_distributions_test 2024-08-07T17:57:29.1900386Z inflating: build/bin/cuda_generator_test 2024-08-07T17:57:29.1960708Z inflating: build/bin/cuda_half_test 2024-08-07T17:57:29.2021265Z inflating: build/bin/cuda_optional_test 2024-08-07T17:57:29.2085140Z inflating: build/bin/cuda_reportMemoryUsage_test 2024-08-07T17:57:29.2146174Z inflating: build/bin/cuda_allocatorTraceTracker_test 2024-08-07T17:57:29.2220151Z inflating: build/bin/cuda_stream_test 2024-08-07T17:57:29.2282848Z inflating: build/bin/cuda_packedtensoraccessor_test 2024-08-07T17:57:29.2343305Z inflating: build/bin/cuda_cudnn_test 2024-08-07T17:57:29.2406806Z inflating: build/bin/cuda_vectorized_test 2024-08-07T17:57:29.2424306Z inflating: build/bin/tutorial_tensorexpr 2024-08-07T17:57:29.2504298Z inflating: build/bin/ProcessGroupGlooTest 2024-08-07T17:57:29.2574011Z inflating: build/bin/ProcessGroupGlooAsyncTest 2024-08-07T17:57:29.2651709Z inflating: build/bin/ProcessGroupNCCLTest 2024-08-07T17:57:29.2727303Z inflating: build/bin/ProcessGroupNCCLErrorsTest 2024-08-07T17:57:29.2793778Z inflating: build/bin/test_dist_autograd 2024-08-07T17:57:29.2878011Z inflating: build/bin/test_cpp_rpc 2024-08-07T17:57:29.2881171Z inflating: build/bin/parallel_benchmark 2024-08-07T17:57:29.2962928Z inflating: build/bin/test_mobile_nnc 2024-08-07T17:57:29.2974052Z inflating: build/bin/aot_model_compiler_test 2024-08-07T17:57:29.3391672Z inflating: build/bin/test_lazy 2024-08-07T17:57:29.4394786Z inflating: build/bin/test_tensorexpr 2024-08-07T17:57:29.5824421Z inflating: build/bin/test_api 2024-08-07T17:57:29.6526755Z inflating: build/bin/test_jit 2024-08-07T17:57:29.6527217Z creating: .additional_ci_files/ 2024-08-07T17:57:29.6596159Z inflating: .additional_ci_files/test-times.json 2024-08-07T17:57:29.6875399Z inflating: .additional_ci_files/test-class-times.json 2024-08-07T17:57:29.7132574Z ##[group]Run rm artifacts.zip 2024-08-07T17:57:29.7132989Z rm artifacts.zip 2024-08-07T17:57:29.7139877Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:57:29.7140359Z env: 2024-08-07T17:57:29.7140645Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:29.7141051Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:29.7141509Z ##[endgroup] 2024-08-07T17:57:29.8028727Z ##[group]Run df -H 2024-08-07T17:57:29.8029091Z df -H 2024-08-07T17:57:29.8035872Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:57:29.8036387Z env: 2024-08-07T17:57:29.8036685Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:29.8037102Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:29.8037573Z ##[endgroup] 2024-08-07T17:57:29.8083895Z Filesystem Size Used Avail Use% Mounted on 2024-08-07T17:57:29.8084387Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-07T17:57:29.8084798Z tmpfs 65G 0 65G 0% /dev/shm 2024-08-07T17:57:29.8085230Z tmpfs 26G 562k 26G 1% /run 2024-08-07T17:57:29.8085872Z /dev/xvda1 161G 38G 124G 24% / 2024-08-07T17:57:29.8086554Z tmpfs 65G 4.1k 65G 1% /tmp 2024-08-07T17:57:29.8086989Z /dev/xvda128 11M 1.4M 9.2M 13% /boot/efi 2024-08-07T17:57:29.8087474Z tmpfs 13G 0 13G 0% /run/user/0 2024-08-07T17:57:29.8127738Z Prepare all required actions 2024-08-07T17:57:29.8128239Z Getting action download info 2024-08-07T17:57:29.9377409Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-07T17:57:29.9377897Z with: 2024-08-07T17:57:29.9378151Z env: 2024-08-07T17:57:29.9378440Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:29.9378878Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:29.9379324Z ##[endgroup] 2024-08-07T17:57:29.9423912Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-07T17:57:29.9424343Z with: 2024-08-07T17:57:29.9424633Z name: td_results 2024-08-07T17:57:29.9424963Z s3-bucket: gha-artifacts 2024-08-07T17:57:29.9425302Z region: us-east-1 2024-08-07T17:57:29.9425612Z env: 2024-08-07T17:57:29.9425902Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:29.9426322Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:29.9426788Z ##[endgroup] 2024-08-07T17:57:30.5564616Z (node:53164) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-07T17:57:30.5565198Z 2024-08-07T17:57:30.5565464Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-07T17:57:30.5566125Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-07T17:57:30.5567217Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-07T17:57:30.6398262Z Found 1 objects with prefix pytorch/pytorch/10288745067/td_results/ 2024-08-07T17:57:30.6399063Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-07T17:57:30.7399821Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-07T17:57:30.7410838Z Artifact download has finished successfully 2024-08-07T17:57:30.7622944Z ##[group]Run mkdir -p .additional_ci_files 2024-08-07T17:57:30.7623420Z mkdir -p .additional_ci_files 2024-08-07T17:57:30.7623936Z mv td_results.json .additional_ci_files/td_results.json 2024-08-07T17:57:30.7632093Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:57:30.7632572Z env: 2024-08-07T17:57:30.7632841Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:30.7633266Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:30.7633743Z ##[endgroup] 2024-08-07T17:57:30.7755355Z ##[group]Run .github/scripts/parse_ref.py 2024-08-07T17:57:30.7755877Z .github/scripts/parse_ref.py 2024-08-07T17:57:30.7762594Z shell: /usr/bin/bash -e {0} 2024-08-07T17:57:30.7762985Z env: 2024-08-07T17:57:30.7763273Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:30.7763683Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:30.7764155Z ##[endgroup] 2024-08-07T17:57:30.8107357Z Prepare all required actions 2024-08-07T17:57:30.8154571Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-07T17:57:30.8155013Z with: 2024-08-07T17:57:30.8155497Z github-token: *** 2024-08-07T17:57:30.8155806Z env: 2024-08-07T17:57:30.8156096Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:30.8156503Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:30.8156959Z ##[endgroup] 2024-08-07T17:57:30.8182009Z ##[group]Run set -eux 2024-08-07T17:57:30.8182358Z set -eux 2024-08-07T17:57:30.8182920Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-07T17:57:30.8189422Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:57:30.8189905Z env: 2024-08-07T17:57:30.8190175Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:30.8190606Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:30.8191242Z GITHUB_TOKEN: *** 2024-08-07T17:57:30.8191541Z ##[endgroup] 2024-08-07T17:57:30.8218281Z + python3 .github/scripts/get_workflow_job_id.py 10288745067 i-0d5f0f54c385d63e2 2024-08-07T17:57:33.5217368Z setting job-id=28476181947 2024-08-07T17:57:33.5218158Z setting job-name=linux-focal-cuda12.1-py3.10-gcc9 / test (default, 1, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-07T17:57:33.5437195Z Prepare all required actions 2024-08-07T17:57:33.5437720Z Getting action download info 2024-08-07T17:57:33.6451369Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-07T17:57:33.6451823Z with: 2024-08-07T17:57:33.6452342Z github-token: *** 2024-08-07T17:57:33.6454410Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}]} 2024-08-07T17:57:33.6456789Z job-name: linux-focal-cuda12.1-py3.10-gcc9 / test (default, 1, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-07T17:57:33.6457450Z env: 2024-08-07T17:57:33.6457816Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:33.6458243Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:33.6458699Z ##[endgroup] 2024-08-07T17:57:33.6513370Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-07T17:57:33.6514219Z with: 2024-08-07T17:57:33.6514505Z shell: bash 2024-08-07T17:57:33.6514822Z timeout_minutes: 10 2024-08-07T17:57:33.6515148Z max_attempts: 5 2024-08-07T17:57:33.6515476Z retry_wait_seconds: 30 2024-08-07T17:57:33.6516562Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-07T17:57:33.6517713Z polling_interval_seconds: 1 2024-08-07T17:57:33.6518090Z warning_on_retry: true 2024-08-07T17:57:33.6518461Z continue_on_error: false 2024-08-07T17:57:33.6518799Z env: 2024-08-07T17:57:33.6519083Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:33.6519555Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:33.6520659Z GITHUB_TOKEN: *** 2024-08-07T17:57:33.6520999Z ##[endgroup] 2024-08-07T17:57:33.7434080Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-07T17:57:34.0764137Z Defaulting to user installation because normal site-packages is not writeable 2024-08-07T17:57:34.2615081Z Collecting requests==2.27.1 2024-08-07T17:57:34.3088236Z Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB) 2024-08-07T17:57:34.5441555Z Collecting pyyaml==6.0.1 2024-08-07T17:57:34.5487095Z Downloading PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (738 kB) 2024-08-07T17:57:34.6458879Z Collecting certifi>=2017.4.17 2024-08-07T17:57:34.6502561Z Downloading certifi-2024.7.4-py3-none-any.whl (162 kB) 2024-08-07T17:57:35.0163061Z Collecting charset-normalizer~=2.0.0 2024-08-07T17:57:35.0208702Z Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB) 2024-08-07T17:57:35.0262639Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-08-07T17:57:35.0283152Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-08-07T17:57:35.1358121Z Installing collected packages: charset-normalizer, certifi, requests, pyyaml 2024-08-07T17:57:35.2921112Z Successfully installed certifi-2024.7.4 charset-normalizer-2.0.12 pyyaml-6.0.1 requests-2.27.1 2024-08-07T17:57:35.7240204Z Command completed after 1 attempt(s). 2024-08-07T17:57:35.7303517Z ##[group]Run set -x 2024-08-07T17:57:35.7303873Z set -x 2024-08-07T17:57:35.7304162Z  2024-08-07T17:57:35.7304655Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-07T17:57:35.7305256Z # in runner workspace 2024-08-07T17:57:35.7305731Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-07T17:57:35.7313010Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:57:35.7313740Z env: 2024-08-07T17:57:35.7314049Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:35.7314490Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:35.7314961Z ##[endgroup] 2024-08-07T17:57:35.7344223Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-07T17:57:35.7665265Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-07T17:57:35.7665781Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-07T17:57:35.7666219Z echo "Job name: ${JOB_NAME}" 2024-08-07T17:57:35.7666581Z  2024-08-07T17:57:35.7667058Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-07T17:57:35.7667652Z # in runner workspace 2024-08-07T17:57:35.7668174Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-07T17:57:35.7668774Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-07T17:57:35.7669539Z  --job-name "${JOB_NAME}" \ 2024-08-07T17:57:35.7671767Z  --test-matrix "{"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}]}" \ 2024-08-07T17:57:35.7674180Z  --selected-test-configs "" \ 2024-08-07T17:57:35.7674619Z  --pr-number "${PR_NUMBER}" \ 2024-08-07T17:57:35.7675026Z  --tag "${TAG}" \ 2024-08-07T17:57:35.7675387Z  --event-name "${EVENT_NAME}" \ 2024-08-07T17:57:35.7675805Z  --schedule "${SCHEDULE}" \ 2024-08-07T17:57:35.7676208Z  --branch "${HEAD_BRANCH}" 2024-08-07T17:57:35.7683029Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:57:35.7683515Z env: 2024-08-07T17:57:35.7683804Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:35.7684219Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:35.7684898Z GITHUB_TOKEN: *** 2024-08-07T17:57:35.7685499Z JOB_NAME: linux-focal-cuda12.1-py3.10-gcc9 / test (default, 1, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-07T17:57:35.7686165Z PR_NUMBER: 131248 2024-08-07T17:57:35.7686450Z TAG: 2024-08-07T17:57:35.7686731Z EVENT_NAME: pull_request 2024-08-07T17:57:35.7687070Z SCHEDULE: 2024-08-07T17:57:35.7687337Z HEAD_BRANCH: 2024-08-07T17:57:35.7687632Z ##[endgroup] 2024-08-07T17:57:35.7715474Z Workflow: pull 2024-08-07T17:57:35.7716070Z Job name: linux-focal-cuda12.1-py3.10-gcc9 / test (default, 1, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-07T17:57:36.0490415Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-07T17:57:36.2590401Z ##[group]Run echo "Filtered matrix:" 2024-08-07T17:57:36.2590870Z echo "Filtered matrix:" 2024-08-07T17:57:36.2592972Z echo "{"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "amz2023.linux.4xlarge.nvidia.gpu"}]}" 2024-08-07T17:57:36.2595116Z  2024-08-07T17:57:36.2595405Z echo 2024-08-07T17:57:36.2595837Z echo "Is the current job unstable? False" 2024-08-07T17:57:36.2596284Z  2024-08-07T17:57:36.2596572Z echo 2024-08-07T17:57:36.2596902Z echo "Is keep-going label set? False" 2024-08-07T17:57:36.2597766Z  2024-08-07T17:57:36.2598254Z echo 2024-08-07T17:57:36.2598587Z echo "Renabled issues? " 2024-08-07T17:57:36.2605505Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:57:36.2605994Z env: 2024-08-07T17:57:36.2606292Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:36.2606733Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:36.2607203Z ##[endgroup] 2024-08-07T17:57:36.2635099Z Filtered matrix: 2024-08-07T17:57:36.2639050Z {include: [{config: default, shard: 1, num_shards: 5, runner: amz2023.linux.4xlarge.nvidia.gpu}, {config: default, shard: 2, num_shards: 5, runner: amz2023.linux.4xlarge.nvidia.gpu}, {config: default, shard: 3, num_shards: 5, runner: amz2023.linux.4xlarge.nvidia.gpu}, {config: default, shard: 4, num_shards: 5, runner: amz2023.linux.4xlarge.nvidia.gpu}, {config: default, shard: 5, num_shards: 5, runner: amz2023.linux.4xlarge.nvidia.gpu}]} 2024-08-07T17:57:36.2641284Z 2024-08-07T17:57:36.2641473Z Is the current job unstable? False 2024-08-07T17:57:36.2641779Z 2024-08-07T17:57:36.2641960Z Is keep-going label set? False 2024-08-07T17:57:36.2642246Z 2024-08-07T17:57:36.2642379Z Renabled issues? 2024-08-07T17:57:36.2705936Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-07T17:57:36.2706778Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-07T17:57:36.2713173Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:57:36.2713651Z env: 2024-08-07T17:57:36.2713925Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:36.2714353Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:36.2714811Z JOB_TIMEOUT: 360 2024-08-07T17:57:36.2715100Z ##[endgroup] 2024-08-07T17:57:36.2800648Z ##[group]Run set -x 2024-08-07T17:57:36.2801092Z set -x 2024-08-07T17:57:36.2801376Z  2024-08-07T17:57:36.2801711Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-07T17:57:36.2802216Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-07T17:57:36.2802714Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-07T17:57:36.2803174Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-07T17:57:36.2803573Z else 2024-08-07T17:57:36.2803888Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-07T17:57:36.2804301Z fi 2024-08-07T17:57:36.2804584Z  2024-08-07T17:57:36.2805000Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-07T17:57:36.2805665Z # TODO: Stop building test binaries as part of the build phase 2024-08-07T17:57:36.2806255Z # Used for GPU_FLAG since that doesn't play nice 2024-08-07T17:57:36.2806777Z # shellcheck disable=SC2086,SC2090 2024-08-07T17:57:36.2807202Z container_name=$(docker run \ 2024-08-07T17:57:36.2807607Z  ${GPU_FLAG:-} \ 2024-08-07T17:57:36.2807972Z  -e BUILD_ENVIRONMENT \ 2024-08-07T17:57:36.2808343Z  -e PR_NUMBER \ 2024-08-07T17:57:36.2808703Z  -e GITHUB_ACTIONS \ 2024-08-07T17:57:36.2809084Z  -e GITHUB_REPOSITORY \ 2024-08-07T17:57:36.2809456Z  -e GITHUB_WORKFLOW \ 2024-08-07T17:57:36.2809833Z  -e GITHUB_JOB \ 2024-08-07T17:57:36.2810198Z  -e GITHUB_RUN_ID \ 2024-08-07T17:57:36.2810551Z  -e GITHUB_RUN_NUMBER \ 2024-08-07T17:57:36.2810941Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-07T17:57:36.2811321Z  -e JOB_ID \ 2024-08-07T17:57:36.2811639Z  -e JOB_NAME \ 2024-08-07T17:57:36.2811982Z  -e BASE_SHA \ 2024-08-07T17:57:36.2812319Z  -e BRANCH \ 2024-08-07T17:57:36.2812633Z  -e SHA1 \ 2024-08-07T17:57:36.2812976Z  -e AWS_DEFAULT_REGION \ 2024-08-07T17:57:36.2813362Z  -e IN_WHEEL_TEST \ 2024-08-07T17:57:36.2813715Z  -e SHARD_NUMBER \ 2024-08-07T17:57:36.2814078Z  -e TEST_CONFIG \ 2024-08-07T17:57:36.2814441Z  -e NUM_TEST_SHARDS \ 2024-08-07T17:57:36.2814805Z  -e REENABLED_ISSUES \ 2024-08-07T17:57:36.2815203Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-07T17:57:36.2815612Z  -e VERBOSE_TEST_LOGS \ 2024-08-07T17:57:36.2815983Z  -e TEST_SHOWLOCALS \ 2024-08-07T17:57:36.2816368Z  -e NO_TEST_TIMEOUT \ 2024-08-07T17:57:36.2816738Z  -e NO_TD \ 2024-08-07T17:57:36.2817060Z  -e TD_DISTRIBUTED \ 2024-08-07T17:57:36.2817434Z  -e PR_LABELS \ 2024-08-07T17:57:36.2817823Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-07T17:57:36.2818240Z  -e SCCACHE_BUCKET \ 2024-08-07T17:57:36.2818635Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-07T17:57:36.2819029Z  -e XLA_CUDA \ 2024-08-07T17:57:36.2819402Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-07T17:57:36.2820198Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-07T17:57:36.2821028Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-07T17:57:36.2821518Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-07T17:57:36.2822215Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-07T17:57:36.2822644Z  -e DASHBOARD_TAG \ 2024-08-07T17:57:36.2823104Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-07T17:57:36.2823854Z  --security-opt seccomp=unconfined \ 2024-08-07T17:57:36.2824304Z  --cap-add=SYS_PTRACE \ 2024-08-07T17:57:36.2824675Z  --ipc=host \ 2024-08-07T17:57:36.2825038Z  --shm-size="${SHM_SIZE}" \ 2024-08-07T17:57:36.2825426Z  --tty \ 2024-08-07T17:57:36.2825732Z  --detach \ 2024-08-07T17:57:36.2826352Z  --name="${container_name}" \ 2024-08-07T17:57:36.2826990Z  --user jenkins \ 2024-08-07T17:57:36.2827447Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-07T17:57:36.2827961Z  -w /var/lib/jenkins/workspace \ 2024-08-07T17:57:36.2828378Z  "${DOCKER_IMAGE}" 2024-08-07T17:57:36.2828707Z ) 2024-08-07T17:57:36.2829087Z # Propagate download.pytorch.org IP to container 2024-08-07T17:57:36.2829912Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-07T17:57:36.2830780Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-07T17:57:36.2831590Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-07T17:57:36.2837867Z shell: /usr/bin/bash -e {0} 2024-08-07T17:57:36.2838208Z env: 2024-08-07T17:57:36.2838499Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:57:36.2838927Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:57:36.2839479Z BUILD_ENVIRONMENT: linux-focal-cuda12.1-py3.10-gcc9 2024-08-07T17:57:36.2839925Z PR_NUMBER: 131248 2024-08-07T17:57:36.2840265Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-07T17:57:36.2840648Z GITHUB_WORKFLOW: pull 2024-08-07T17:57:36.2840979Z GITHUB_JOB: test 2024-08-07T17:57:36.2841293Z GITHUB_RUN_ID: 10288745067 2024-08-07T17:57:36.2841631Z GITHUB_RUN_NUMBER: 234358 2024-08-07T17:57:36.2841987Z GITHUB_RUN_ATTEMPT: 1 2024-08-07T17:57:36.2842310Z JOB_ID: 28476181947 2024-08-07T17:57:36.2842886Z JOB_NAME: linux-focal-cuda12.1-py3.10-gcc9 / test (default, 1, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-07T17:57:36.2843581Z BRANCH: pull/131248 2024-08-07T17:57:36.2843941Z SHA1: 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:57:36.2844394Z BASE_SHA: 6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T17:57:36.2844827Z TEST_CONFIG: default 2024-08-07T17:57:36.2845147Z SHARD_NUMBER: 1 2024-08-07T17:57:36.2845436Z NUM_TEST_SHARDS: 5 2024-08-07T17:57:36.2845768Z REENABLED_ISSUES: 2024-08-07T17:57:36.2846100Z CONTINUE_THROUGH_ERROR: False 2024-08-07T17:57:36.2846454Z VERBOSE_TEST_LOGS: False 2024-08-07T17:57:36.2846809Z TEST_SHOWLOCALS: False 2024-08-07T17:57:36.2847149Z NO_TEST_TIMEOUT: False 2024-08-07T17:57:36.2847460Z NO_TD: False 2024-08-07T17:57:36.2847760Z TD_DISTRIBUTED: False 2024-08-07T17:57:36.2848155Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-07T17:57:36.2848607Z SCCACHE_S3_KEY_PREFIX: pull 2024-08-07T17:57:36.2848962Z SHM_SIZE: 2g 2024-08-07T17:57:36.2849847Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:57:36.2850785Z XLA_CUDA: 2024-08-07T17:57:36.2851246Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-07T17:57:36.2851817Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-07T17:57:36.2852215Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-07T17:57:36.2852615Z DASHBOARD_TAG: 2024-08-07T17:57:36.2852934Z HUGGING_FACE_HUB_TOKEN: 2024-08-07T17:57:36.2853259Z ##[endgroup] 2024-08-07T17:57:36.2878816Z + [[ default == \m\u\l\t\i\g\p\u ]] 2024-08-07T17:57:36.2879490Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *onnx* ]] 2024-08-07T17:57:36.2879959Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-07T17:57:36.2888276Z +++ nproc --ignore=2 2024-08-07T17:57:36.2917017Z ++ docker run --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=14 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10288745067 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=2g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:57:47.3085703Z + container_name=ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T17:57:47.3088471Z + grep download.pytorch.org /etc/hosts 2024-08-07T17:57:47.3089698Z + docker exec -i ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-07T17:57:47.4651028Z + echo DOCKER_CONTAINER_ID=ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T17:57:47.4655465Z ++ echo dist/torch-2.5.0a0+git016588f-cp310-cp310-linux_x86_64.whl 2024-08-07T17:57:47.4657951Z + docker exec -t ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d sh -c 'pip install dist/torch-2.5.0a0+git016588f-cp310-cp310-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-07T17:57:48.0026281Z Processing ./dist/torch-2.5.0a0+git016588f-cp310-cp310-linux_x86_64.whl (from torch==2.5.0a0+git016588f) 2024-08-07T17:57:49.1077406Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (3.13.1) 2024-08-07T17:57:49.1082608Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (4.12.2) 2024-08-07T17:57:49.1087064Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (2.8.8) 2024-08-07T17:57:49.1091995Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (3.1.4) 2024-08-07T17:57:49.1096835Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (2024.6.1) 2024-08-07T17:57:49.1111250Z Requirement already satisfied: sympy>=1.13.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (1.13.1) 2024-08-07T17:57:49.1150373Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (3.3.0) 2024-08-07T17:57:49.1231586Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (1.21.2) 2024-08-07T17:57:49.1277866Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy>=1.13.0->torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (1.3.0) 2024-08-07T17:57:49.2557149Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (2.1.5) 2024-08-07T17:57:50.3904454Z Installing collected packages: torch 2024-08-07T17:58:03.5066888Z Successfully installed torch-2.5.0a0+git016588f 2024-08-07T17:58:03.5755517Z ++ dirname .ci/pytorch/test.sh 2024-08-07T17:58:03.5762149Z + source .ci/pytorch/common.sh 2024-08-07T17:58:03.5766328Z +++ dirname .ci/pytorch/common.sh 2024-08-07T17:58:03.5774765Z ++ source .ci/pytorch/common_utils.sh 2024-08-07T17:58:03.5777388Z +++ declare -f -t trap_add 2024-08-07T17:58:03.5785187Z ++ set -ex 2024-08-07T17:58:03.5785558Z ++ [[ linux-focal-cuda12.1-py3.10-gcc9 == *rocm* ]] 2024-08-07T17:58:03.5786017Z ++ BUILD_TEST_LIBTORCH=0 2024-08-07T17:58:03.5787400Z + [[ linux-focal-cuda12.1-py3.10-gcc9 != *rocm* ]] 2024-08-07T17:58:03.5791465Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-07T17:58:03.5809263Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-07T17:58:03.5809662Z + trap_add cleanup_workspace EXIT 2024-08-07T17:58:03.5810077Z + trap_add_cmd=cleanup_workspace 2024-08-07T17:58:03.5810441Z + shift 2024-08-07T17:58:03.5810723Z + for trap_add_name in "$@" 2024-08-07T17:58:03.5818247Z +++ trap -p EXIT 2024-08-07T17:58:03.5821544Z ++ eval 'extract_trap_cmd ' 2024-08-07T17:58:03.5821916Z +++ extract_trap_cmd 2024-08-07T17:58:03.5822292Z +++ printf '%s\n' '' 2024-08-07T17:58:03.5823549Z ++ printf '%s\n' cleanup_workspace 2024-08-07T17:58:03.5825741Z + trap -- ' 2024-08-07T17:58:03.5826216Z cleanup_workspace' EXIT 2024-08-07T17:58:03.5826622Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-07T17:58:04.1724260Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-07T17:58:04.1745583Z + echo 'Environment variables:' 2024-08-07T17:58:04.1746178Z Environment variables: 2024-08-07T17:58:04.1746499Z + env 2024-08-07T17:58:04.1756031Z INSTALLED_DB=yes 2024-08-07T17:58:04.1756962Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-07T17:58:04.1757597Z NVIDIA_VISIBLE_DEVICES=all 2024-08-07T17:58:04.1758130Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-07T17:58:04.1758717Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:58:04.1759292Z CONTINUE_THROUGH_ERROR=False 2024-08-07T17:58:04.1759696Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-07T17:58:04.1760206Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-07T17:58:04.1760693Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9 2024-08-07T17:58:04.1761126Z HOSTNAME=ee14918686c2 2024-08-07T17:58:04.1761866Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_e93bea3b-7f96-4d75-86ad-c811793761b3 2024-08-07T17:58:04.1762622Z GITHUB_ACTION=__self 2024-08-07T17:58:04.1762956Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-07T17:58:04.1766857Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-07T17:58:04.1770415Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-07T17:58:04.1770879Z NV_NVTX_VERSION=12.1.105-1 2024-08-07T17:58:04.1771213Z GITHUB_RUN_NUMBER=234358 2024-08-07T17:58:04.1771554Z TEST_CONFIG=default 2024-08-07T17:58:04.1771886Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-07T17:58:04.1772286Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-07T17:58:04.1772704Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-07T17:58:04.1773105Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-07T17:58:04.1773477Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-07T17:58:04.1773846Z GITHUB_TRIGGERING_ACTOR=zdevito 2024-08-07T17:58:04.1774270Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-07T17:58:04.1774697Z GITHUB_REF_TYPE=branch 2024-08-07T17:58:04.1775319Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-07T17:58:04.1775672Z NCCL_VERSION=2.17.1-1 2024-08-07T17:58:04.1776022Z BASE_SHA=6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T17:58:04.1776438Z XLA_CUDA= 2024-08-07T17:58:04.1776720Z HUGGING_FACE_HUB_TOKEN= 2024-08-07T17:58:04.1777211Z *** 2024-08-07T17:58:04.1777500Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-07T17:58:04.1777888Z GITHUB_REPOSITORY_ID=65600975 2024-08-07T17:58:04.1778232Z GITHUB_ACTIONS=true 2024-08-07T17:58:04.1778717Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:58:04.1779168Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-07T17:58:04.1779621Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-07T17:58:04.1780060Z SHA1=016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:58:04.1780501Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-07T17:58:04.1780932Z GITHUB_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T17:58:04.1781591Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/131248/merge 2024-08-07T17:58:04.1782207Z UCC_HOME=/usr 2024-08-07T17:58:04.1782504Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-07T17:58:04.1782889Z VERBOSE_TEST_LOGS=False 2024-08-07T17:58:04.1783228Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-07T17:58:04.1783610Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-07T17:58:04.1784060Z GITHUB_REF=refs/pull/131248/merge 2024-08-07T17:58:04.1784485Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-07T17:58:04.1784832Z SHARD_NUMBER=1 2024-08-07T17:58:04.1785154Z GITHUB_REF_PROTECTED=false 2024-08-07T17:58:04.1785506Z HOME=/var/lib/jenkins 2024-08-07T17:58:04.1785853Z GITHUB_API_URL=https://api.github.com 2024-08-07T17:58:04.1786280Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-07T17:58:04.1786729Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-07T17:58:04.1787171Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-07T17:58:04.1787517Z CUDA_VERSION=12.1.1 2024-08-07T17:58:04.1787870Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-07T17:58:04.1788283Z NUM_TEST_SHARDS=5 2024-08-07T17:58:04.1788599Z UCX_HOME=/usr 2024-08-07T17:58:04.1789039Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-07T17:58:04.1789954Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_e93bea3b-7f96-4d75-86ad-c811793761b3 2024-08-07T17:58:04.1791008Z JOB_NAME=linux-focal-cuda12.1-py3.10-gcc9 / test (default, 1, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-07T17:58:04.1792088Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_e93bea3b-7f96-4d75-86ad-c811793761b3 2024-08-07T17:58:04.1793061Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-07T17:58:04.1793677Z GITHUB_EVENT_NAME=pull_request 2024-08-07T17:58:04.1794038Z DASHBOARD_TAG= 2024-08-07T17:58:04.1794323Z GITHUB_RUN_ID=10288745067 2024-08-07T17:58:04.1794717Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-07T17:58:04.1795191Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-07T17:58:04.1796019Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_e93bea3b-7f96-4d75-86ad-c811793761b3 2024-08-07T17:58:04.1796851Z GITHUB_ACTOR=zdevito 2024-08-07T17:58:04.1797682Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-07T17:58:04.1798058Z PR_NUMBER=131248 2024-08-07T17:58:04.1798355Z GITHUB_RUN_ATTEMPT=1 2024-08-07T17:58:04.1798690Z ANACONDA_PYTHON_VERSION=3.10 2024-08-07T17:58:04.1799105Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-07T17:58:04.1799561Z TERM=xterm 2024-08-07T17:58:04.1799876Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-07T17:58:04.1800258Z INSTALLED_VISION=yes 2024-08-07T17:58:04.1800582Z BRANCH=pull/131248 2024-08-07T17:58:04.1800907Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-07T17:58:04.1801278Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-07T17:58:04.1801689Z CUDA_PATH=/usr/local/cuda 2024-08-07T17:58:04.1802329Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-07T17:58:04.1803208Z GITHUB_SERVER_URL=https://github.com 2024-08-07T17:58:04.1803676Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-07T17:58:04.1804121Z REENABLED_ISSUES= 2024-08-07T17:58:04.1804403Z SHLVL=1 2024-08-07T17:58:04.1804681Z MAX_JOBS=14 2024-08-07T17:58:04.1804979Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-07T17:58:04.1805311Z NVARCH=x86_64 2024-08-07T17:58:04.1805614Z GITHUB_ACTOR_ID=370202 2024-08-07T17:58:04.1806159Z GITHUB_WORKFLOW_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T17:58:04.1806657Z GITHUB_REF_NAME=131248/merge 2024-08-07T17:58:04.1807046Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-07T17:58:04.1807610Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-07T17:58:04.1808129Z GITHUB_JOB=test 2024-08-07T17:58:04.1808478Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-07T17:58:04.1808994Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-07T17:58:04.1809464Z NO_TEST_TIMEOUT=False 2024-08-07T17:58:04.1809797Z TD_DISTRIBUTED=False 2024-08-07T17:58:04.1810153Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-07T17:58:04.1810556Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-07T17:58:04.1810951Z NV_NVPROF_VERSION=12.1.105-1 2024-08-07T17:58:04.1811292Z GITHUB_RETENTION_DAYS=90 2024-08-07T17:58:04.1811639Z OPENSSL_DIR=/opt/openssl 2024-08-07T17:58:04.1811994Z GITHUB_ACTION_REPOSITORY= 2024-08-07T17:58:04.1812915Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T17:58:04.1813901Z GITHUB_BASE_REF=gh/zdevito/267/base 2024-08-07T17:58:04.1814307Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-07T17:58:04.1814644Z CI=true 2024-08-07T17:58:04.1814940Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-07T17:58:04.1815342Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-07T17:58:04.1815679Z JOB_ID=28476181947 2024-08-07T17:58:04.1815988Z INSTALLED_PROTOBUF=yes 2024-08-07T17:58:04.1816335Z GITHUB_HEAD_REF=gh/zdevito/267/head 2024-08-07T17:58:04.1816704Z GITHUB_ACTION_REF= 2024-08-07T17:58:04.1817076Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-07T17:58:04.1817525Z TEST_SHOWLOCALS=False 2024-08-07T17:58:04.1817835Z GITHUB_WORKFLOW=pull 2024-08-07T17:58:04.1818173Z DEBIAN_FRONTEND=noninteractive 2024-08-07T17:58:04.1818943Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_e93bea3b-7f96-4d75-86ad-c811793761b3 2024-08-07T17:58:04.1819701Z NO_TD=False 2024-08-07T17:58:04.1820013Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-07T17:58:04.1820418Z _=/usr/bin/env 2024-08-07T17:58:04.1820836Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-07T17:58:04.1987394Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch 2024-08-07T17:58:04.1988577Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin 2024-08-07T17:58:04.1989725Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib 2024-08-07T17:58:04.1990480Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test 2024-08-07T17:58:04.1991102Z + BUILD_DIR=build 2024-08-07T17:58:04.1991421Z + BUILD_RENAMED_DIR=build_renamed 2024-08-07T17:58:04.1991809Z + BUILD_BIN_DIR=build/bin 2024-08-07T17:58:04.1992155Z + SHARD_NUMBER=1 2024-08-07T17:58:04.1992443Z + NUM_TEST_SHARDS=5 2024-08-07T17:58:04.1992878Z + export VALGRIND=ON 2024-08-07T17:58:04.1993507Z + VALGRIND=ON 2024-08-07T17:58:04.2009425Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *clang9* ]] 2024-08-07T17:58:04.2009908Z + [[ 0 == \1 ]] 2024-08-07T17:58:04.2010222Z + [[ False == \1 ]] 2024-08-07T17:58:04.2010574Z + [[ linux-focal-cuda12.1-py3.10-gcc9 != *bazel* ]] 2024-08-07T17:58:04.2011058Z ++ realpath build/custom_test_artifacts 2024-08-07T17:58:04.2011685Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-07T17:58:04.2012276Z + [[ -n '' ]] 2024-08-07T17:58:04.2012748Z + echo 'Environment variables' 2024-08-07T17:58:04.2013455Z Environment variables 2024-08-07T17:58:04.2014361Z + env 2024-08-07T17:58:04.2015180Z INSTALLED_DB=yes 2024-08-07T17:58:04.2015694Z NV_LIBCUBLAS_VERSION=12.1.3.1-1 2024-08-07T17:58:04.2016261Z NVIDIA_VISIBLE_DEVICES=all 2024-08-07T17:58:04.2016689Z NV_NVML_DEV_VERSION=12.1.105-1 2024-08-07T17:58:04.2017385Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:58:04.2017905Z CONTINUE_THROUGH_ERROR=False 2024-08-07T17:58:04.2018505Z NV_LIBNCCL_DEV_PACKAGE=libnccl-dev=2.17.1-1+cuda12.1 2024-08-07T17:58:04.2019201Z NV_LIBNCCL_DEV_PACKAGE_VERSION=2.17.1-1 2024-08-07T17:58:04.2019660Z BUILD_ENVIRONMENT=linux-focal-cuda12.1-py3.10-gcc9 2024-08-07T17:58:04.2020128Z HOSTNAME=ee14918686c2 2024-08-07T17:58:04.2020839Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_e93bea3b-7f96-4d75-86ad-c811793761b3 2024-08-07T17:58:04.2021605Z GITHUB_ACTION=__self 2024-08-07T17:58:04.2021944Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-07T17:58:04.2025925Z NVIDIA_REQUIRE_CUDA=cuda>=12.1 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 2024-08-07T17:58:04.2030404Z NV_LIBCUBLAS_DEV_PACKAGE=libcublas-dev-12-1=12.1.3.1-1 2024-08-07T17:58:04.2030876Z NV_NVTX_VERSION=12.1.105-1 2024-08-07T17:58:04.2031212Z GITHUB_RUN_NUMBER=234358 2024-08-07T17:58:04.2031561Z TEST_CONFIG=default 2024-08-07T17:58:04.2031896Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-07T17:58:04.2032310Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-07T17:58:04.2032736Z NV_CUDA_CUDART_DEV_VERSION=12.1.105-1 2024-08-07T17:58:04.2033147Z NV_LIBCUSPARSE_VERSION=12.1.0.106-1 2024-08-07T17:58:04.2033523Z NV_LIBNPP_VERSION=12.1.0.40-1 2024-08-07T17:58:04.2033902Z GITHUB_TRIGGERING_ACTOR=zdevito 2024-08-07T17:58:04.2034314Z CMAKE_CUDA_COMPILER_LAUNCHER=/opt/cache/bin/sccache 2024-08-07T17:58:04.2034764Z GITHUB_REF_TYPE=branch 2024-08-07T17:58:04.2035122Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-07T17:58:04.2035466Z NCCL_VERSION=2.17.1-1 2024-08-07T17:58:04.2035839Z BASE_SHA=6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T17:58:04.2036271Z XLA_CUDA= 2024-08-07T17:58:04.2036538Z HUGGING_FACE_HUB_TOKEN= 2024-08-07T17:58:04.2036973Z *** 2024-08-07T17:58:04.2037272Z CARGO_NET_GIT_FETCH_WITH_CLI=true 2024-08-07T17:58:04.2037646Z GITHUB_REPOSITORY_ID=65600975 2024-08-07T17:58:04.2038046Z GITHUB_ACTIONS=true 2024-08-07T17:58:04.2038385Z NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T17:58:04.2038819Z NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-1=12.1.105-1 2024-08-07T17:58:04.2039271Z NV_LIBNPP_PACKAGE=libnpp-12-1=12.1.0.40-1 2024-08-07T17:58:04.2039722Z SHA1=016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:58:04.2040166Z NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-dev 2024-08-07T17:58:04.2040601Z GITHUB_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T17:58:04.2041267Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/131248/merge 2024-08-07T17:58:04.2041882Z UCC_HOME=/usr 2024-08-07T17:58:04.2042183Z NV_LIBCUBLAS_DEV_VERSION=12.1.3.1-1 2024-08-07T17:58:04.2042578Z VERBOSE_TEST_LOGS=False 2024-08-07T17:58:04.2042916Z NVIDIA_PRODUCT_NAME=CUDA 2024-08-07T17:58:04.2043297Z NV_LIBCUBLAS_DEV_PACKAGE_NAME=libcublas-dev-12-1 2024-08-07T17:58:04.2043756Z GITHUB_REF=refs/pull/131248/merge 2024-08-07T17:58:04.2044142Z NV_CUDA_CUDART_VERSION=12.1.105-1 2024-08-07T17:58:04.2044501Z SHARD_NUMBER=1 2024-08-07T17:58:04.2044809Z GITHUB_REF_PROTECTED=false 2024-08-07T17:58:04.2045311Z HOME=/var/lib/jenkins 2024-08-07T17:58:04.2045663Z GITHUB_API_URL=https://api.github.com 2024-08-07T17:58:04.2046090Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-07T17:58:04.2046532Z UCX_COMMIT=7bb2722ff2187a0cad557ae4a6afa090569f83fb 2024-08-07T17:58:04.2046970Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-07T17:58:04.2047321Z CUDA_VERSION=12.1.1 2024-08-07T17:58:04.2047678Z NV_LIBCUBLAS_PACKAGE=libcublas-12-1=12.1.3.1-1 2024-08-07T17:58:04.2048083Z NUM_TEST_SHARDS=5 2024-08-07T17:58:04.2048476Z UCX_HOME=/usr 2024-08-07T17:58:04.2048909Z NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-1=12.1.1-1 2024-08-07T17:58:04.2049822Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_e93bea3b-7f96-4d75-86ad-c811793761b3 2024-08-07T17:58:04.2050879Z JOB_NAME=linux-focal-cuda12.1-py3.10-gcc9 / test (default, 1, 5, amz2023.linux.4xlarge.nvidia.gpu) 2024-08-07T17:58:04.2051906Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_e93bea3b-7f96-4d75-86ad-c811793761b3 2024-08-07T17:58:04.2052872Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-07T17:58:04.2053504Z GITHUB_EVENT_NAME=pull_request 2024-08-07T17:58:04.2053869Z DASHBOARD_TAG= 2024-08-07T17:58:04.2054143Z GITHUB_RUN_ID=10288745067 2024-08-07T17:58:04.2054517Z NV_LIBNPP_DEV_PACKAGE=libnpp-dev-12-1=12.1.0.40-1 2024-08-07T17:58:04.2054979Z NV_LIBCUBLAS_PACKAGE_NAME=libcublas-12-1 2024-08-07T17:58:04.2055829Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_e93bea3b-7f96-4d75-86ad-c811793761b3 2024-08-07T17:58:04.2056648Z GITHUB_ACTOR=zdevito 2024-08-07T17:58:04.2056970Z NV_LIBNPP_DEV_VERSION=12.1.0.40-1 2024-08-07T17:58:04.2057342Z PR_NUMBER=131248 2024-08-07T17:58:04.2057651Z GITHUB_RUN_ATTEMPT=1 2024-08-07T17:58:04.2057952Z VALGRIND=ON 2024-08-07T17:58:04.2058249Z ANACONDA_PYTHON_VERSION=3.10 2024-08-07T17:58:04.2058677Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-07T17:58:04.2059106Z TERM=xterm 2024-08-07T17:58:04.2059424Z NV_LIBCUSPARSE_DEV_VERSION=12.1.0.106-1 2024-08-07T17:58:04.2059819Z INSTALLED_VISION=yes 2024-08-07T17:58:04.2060129Z BRANCH=pull/131248 2024-08-07T17:58:04.2060451Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-07T17:58:04.2060834Z LIBRARY_PATH=/usr/local/cuda/lib64/stubs 2024-08-07T17:58:04.2061228Z CUDA_PATH=/usr/local/cuda 2024-08-07T17:58:04.2061877Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-07T17:58:04.2062590Z GITHUB_SERVER_URL=https://github.com 2024-08-07T17:58:04.2063033Z UCC_COMMIT=20eae37090a4ce1b32bcce6144ccad0b49943e0b 2024-08-07T17:58:04.2063483Z REENABLED_ISSUES= 2024-08-07T17:58:04.2063783Z SHLVL=1 2024-08-07T17:58:04.2064040Z MAX_JOBS=14 2024-08-07T17:58:04.2064333Z NV_CUDA_LIB_VERSION=12.1.1-1 2024-08-07T17:58:04.2064683Z NVARCH=x86_64 2024-08-07T17:58:04.2064964Z GITHUB_ACTOR_ID=370202 2024-08-07T17:58:04.2065387Z GITHUB_WORKFLOW_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T17:58:04.2065881Z GITHUB_REF_NAME=131248/merge 2024-08-07T17:58:04.2066247Z NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-1 2024-08-07T17:58:04.2066806Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-07T17:58:04.2067343Z GITHUB_JOB=test 2024-08-07T17:58:04.2067669Z NV_LIBNCCL_PACKAGE=libnccl2=2.17.1-1+cuda12.1 2024-08-07T17:58:04.2068184Z LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2024-08-07T17:58:04.2068665Z NO_TEST_TIMEOUT=False 2024-08-07T17:58:04.2068986Z TD_DISTRIBUTED=False 2024-08-07T17:58:04.2069335Z NV_CUDA_NSIGHT_COMPUTE_VERSION=12.1.1-1 2024-08-07T17:58:04.2069739Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-07T17:58:04.2070136Z NV_NVPROF_VERSION=12.1.105-1 2024-08-07T17:58:04.2070495Z GITHUB_RETENTION_DAYS=90 2024-08-07T17:58:04.2070822Z OPENSSL_DIR=/opt/openssl 2024-08-07T17:58:04.2071170Z GITHUB_ACTION_REPOSITORY= 2024-08-07T17:58:04.2072104Z PATH=/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T17:58:04.2073171Z GITHUB_BASE_REF=gh/zdevito/267/base 2024-08-07T17:58:04.2073577Z NV_LIBNCCL_PACKAGE_NAME=libnccl2 2024-08-07T17:58:04.2073934Z CI=true 2024-08-07T17:58:04.2074213Z NV_LIBNCCL_PACKAGE_VERSION=2.17.1-1 2024-08-07T17:58:04.2074616Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-07T17:58:04.2074977Z JOB_ID=28476181947 2024-08-07T17:58:04.2075272Z INSTALLED_PROTOBUF=yes 2024-08-07T17:58:04.2075707Z GITHUB_HEAD_REF=gh/zdevito/267/head 2024-08-07T17:58:04.2076105Z GITHUB_ACTION_REF= 2024-08-07T17:58:04.2076465Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-07T17:58:04.2076917Z TEST_SHOWLOCALS=False 2024-08-07T17:58:04.2077247Z GITHUB_WORKFLOW=pull 2024-08-07T17:58:04.2077570Z DEBIAN_FRONTEND=noninteractive 2024-08-07T17:58:04.2078341Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_e93bea3b-7f96-4d75-86ad-c811793761b3 2024-08-07T17:58:04.2079125Z NO_TD=False 2024-08-07T17:58:04.2079417Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-07T17:58:04.2079790Z _=/usr/bin/env 2024-08-07T17:58:04.2080099Z + echo 'Testing pytorch' 2024-08-07T17:58:04.2080422Z Testing pytorch 2024-08-07T17:58:04.2080751Z + export LANG=C.UTF-8 2024-08-07T17:58:04.2081072Z + LANG=C.UTF-8 2024-08-07T17:58:04.2081352Z + PR_NUMBER=131248 2024-08-07T17:58:04.2081672Z + [[ default == \d\e\f\a\u\l\t ]] 2024-08-07T17:58:04.2082033Z + export CUDA_VISIBLE_DEVICES=0 2024-08-07T17:58:04.2082399Z + CUDA_VISIBLE_DEVICES=0 2024-08-07T17:58:04.2082756Z + export HIP_VISIBLE_DEVICES=0 2024-08-07T17:58:04.2083103Z + HIP_VISIBLE_DEVICES=0 2024-08-07T17:58:04.2083453Z + [[ default == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-07T17:58:04.2083849Z + [[ default == \s\l\o\w ]] 2024-08-07T17:58:04.2084271Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *slow-gradcheck* ]] 2024-08-07T17:58:04.2084812Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *cuda* ]] 2024-08-07T17:58:04.2085293Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-07T17:58:04.2085722Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda 2024-08-07T17:58:04.2086131Z + [[ default == *crossref* ]] 2024-08-07T17:58:04.2086548Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *rocm* ]] 2024-08-07T17:58:04.2087017Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *xpu* ]] 2024-08-07T17:58:04.2087519Z + [[ linux-focal-cuda12.1-py3.10-gcc9 != *-bazel-* ]] 2024-08-07T17:58:04.2087982Z + pip_install --user ninja==1.10.2 2024-08-07T17:58:04.2088415Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-07T17:58:04.7930642Z Collecting ninja==1.10.2 2024-08-07T17:58:04.8133355Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-07T17:58:04.8308022Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-07T17:58:06.0495174Z Installing collected packages: ninja 2024-08-07T17:58:06.0605240Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-07T17:58:06.0606424Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-07T17:58:06.0659798Z Successfully installed ninja-1.10.2 2024-08-07T17:58:06.1423043Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T17:58:06.1424951Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T17:58:06.1426131Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *aarch64* ]] 2024-08-07T17:58:06.1426589Z + install_tlparse 2024-08-07T17:58:06.1426922Z + pip_install --user tlparse==0.3.7 2024-08-07T17:58:06.1427392Z + pip install --progress-bar off --user tlparse==0.3.7 2024-08-07T17:58:06.6688243Z Collecting tlparse==0.3.7 2024-08-07T17:58:06.6875602Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-08-07T17:58:06.6965911Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-07T17:58:07.9161137Z Installing collected packages: tlparse 2024-08-07T17:58:07.9648455Z Successfully installed tlparse-0.3.7 2024-08-07T17:58:08.0366909Z ++ python -m site --user-base 2024-08-07T17:58:08.0599083Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T17:58:08.0600712Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *asan* ]] 2024-08-07T17:58:08.0601548Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *-debug* ]] 2024-08-07T17:58:08.0602072Z + [[ linux-focal-cuda12.1-py3.10-gcc9 != *-bazel-* ]] 2024-08-07T17:58:08.0602769Z + echo 'We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9. Expect the assertion to pass' 2024-08-07T17:58:08.0603630Z We are not in debug mode: linux-focal-cuda12.1-py3.10-gcc9. Expect the assertion to pass 2024-08-07T17:58:08.0605540Z + cd test 2024-08-07T17:58:08.0606046Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-07T17:58:10.2110459Z + [[ default == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-07T17:58:10.2110978Z + [[ default == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-07T17:58:10.2116536Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-07T17:58:10.2117158Z + [[ default == *dynamo_eager* ]] 2024-08-07T17:58:10.2117546Z + [[ default == *aot_eager* ]] 2024-08-07T17:58:10.2117898Z + [[ default == *aot_inductor* ]] 2024-08-07T17:58:10.2118268Z + [[ default == *inductor* ]] 2024-08-07T17:58:10.2118630Z + [[ default == *dynamic* ]] 2024-08-07T17:58:10.2118973Z + [[ default == *cpu* ]] 2024-08-07T17:58:10.2121826Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-07T17:58:10.2158902Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *libtorch* ]] 2024-08-07T17:58:10.2159439Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *-bazel-* ]] 2024-08-07T17:58:10.2162685Z + cd test 2024-08-07T17:58:10.2163094Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-07T17:58:12.1077047Z PyTorch built with: 2024-08-07T17:58:12.1077449Z - GCC 9.4 2024-08-07T17:58:12.1077749Z - C++ Version: 201703 2024-08-07T17:58:12.1078445Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-07T17:58:12.1079369Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-07T17:58:12.1079947Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-07T17:58:12.1080370Z - LAPACK is enabled (usually provided by MKL) 2024-08-07T17:58:12.1080803Z - NNPACK is enabled 2024-08-07T17:58:12.1081149Z - CPU capability usage: AVX2 2024-08-07T17:58:12.1081497Z - CUDA Runtime 12.1 2024-08-07T17:58:12.1081953Z - NVCC architecture flags: -gencode;arch=compute_52,code=sm_52 2024-08-07T17:58:12.1082483Z - CuDNN 90.1 (built against CUDA 12.4) 2024-08-07T17:58:12.1082887Z - Magma 2.6.1 2024-08-07T17:58:12.1088656Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, FORCE_FALLBACK_CUDA_MPI=1, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=ON, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-07T17:58:12.1095004Z 2024-08-07T17:58:12.4142371Z + cd test 2024-08-07T17:58:12.4142905Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-07T17:58:14.2819669Z ATen/Parallel: 2024-08-07T17:58:14.2820135Z at::get_num_threads() : 8 2024-08-07T17:58:14.2820537Z at::get_num_interop_threads() : 8 2024-08-07T17:58:14.2820940Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-07T17:58:14.2821329Z omp_get_max_threads() : 8 2024-08-07T17:58:14.2822030Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-07T17:58:14.2822766Z mkl_get_max_threads() : 8 2024-08-07T17:58:14.2823261Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-07T17:58:14.2823829Z std::thread::hardware_concurrency() : 16 2024-08-07T17:58:14.2824248Z Environment variables: 2024-08-07T17:58:14.2824595Z OMP_NUM_THREADS : [not set] 2024-08-07T17:58:14.2824946Z MKL_NUM_THREADS : [not set] 2024-08-07T17:58:14.2825311Z ATen parallel backend: OpenMP 2024-08-07T17:58:14.2825556Z 2024-08-07T17:58:14.5596189Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *aarch64* ]] 2024-08-07T17:58:14.5596938Z + [[ default == *backward* ]] 2024-08-07T17:58:14.5597749Z + [[ default == *xla* ]] 2024-08-07T17:58:14.5598092Z + [[ default == *executorch* ]] 2024-08-07T17:58:14.5598475Z + [[ default == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-07T17:58:14.5598937Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *libtorch* ]] 2024-08-07T17:58:14.5599396Z + [[ default == distributed ]] 2024-08-07T17:58:14.5599782Z + [[ default == *inductor_distributed* ]] 2024-08-07T17:58:14.5600204Z + [[ default == *inductor-halide* ]] 2024-08-07T17:58:14.5600616Z + [[ default == *inductor-micro-benchmark* ]] 2024-08-07T17:58:14.5601078Z + [[ default == *huggingface* ]] 2024-08-07T17:58:14.5601444Z + [[ default == *timm* ]] 2024-08-07T17:58:14.5601782Z + [[ default == *torchbench* ]] 2024-08-07T17:58:14.5602200Z + [[ default == *inductor_cpp_wrapper_abi_compatible* ]] 2024-08-07T17:58:14.5602665Z + [[ default == *inductor* ]] 2024-08-07T17:58:14.5603006Z + [[ default == *dynamo* ]] 2024-08-07T17:58:14.5603404Z + [[ linux-focal-cuda12.1-py3.10-gcc9 == *rocm* ]] 2024-08-07T17:58:14.5604014Z + [[ 1 == 1 ]] 2024-08-07T17:58:14.5604330Z + [[ 5 -gt 1 ]] 2024-08-07T17:58:14.5604633Z + test_without_numpy 2024-08-07T17:58:14.5604959Z ++ dirname .ci/pytorch/test.sh 2024-08-07T17:58:14.5616659Z + pushd .ci/pytorch 2024-08-07T17:58:14.5617186Z ~/workspace/.ci/pytorch ~/workspace 2024-08-07T17:58:14.5618357Z + python -c 'import sys;sys.path.insert(0, '\''fake_numpy'\'');from unittest import TestCase;import torch;x=torch.randn(3,3);TestCase().assertRaises(RuntimeError, lambda: x.numpy())' 2024-08-07T17:58:15.7163404Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_subclasses/functional_tensor.py:271: UserWarning: Failed to initialize NumPy: Sorry PyTorch, but our NumPy is in the other folder (Triggered internally at /var/lib/jenkins/workspace/torch/csrc/utils/tensor_numpy.cpp:84.) 2024-08-07T17:58:15.7165135Z cpu = _conversion_method_template(device=torch.device("cpu")) 2024-08-07T17:58:16.5311524Z + python -c 'import sys;sys.path.insert(0, '\''fake_numpy'\'');import torch;print(torch.tensor([torch.tensor(0.), torch.tensor(1.)]))' 2024-08-07T17:58:17.6929411Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/_subclasses/functional_tensor.py:271: UserWarning: Failed to initialize NumPy: Sorry PyTorch, but our NumPy is in the other folder (Triggered internally at /var/lib/jenkins/workspace/torch/csrc/utils/tensor_numpy.cpp:84.) 2024-08-07T17:58:17.6931115Z cpu = _conversion_method_template(device=torch.device("cpu")) 2024-08-07T17:58:18.2346248Z tensor([0., 1.]) 2024-08-07T17:58:18.5185318Z + [[ default == *dynamo* ]] 2024-08-07T17:58:18.5186114Z + popd 2024-08-07T17:58:18.5186403Z ~/workspace 2024-08-07T17:58:18.5186686Z + install_torchvision 2024-08-07T17:58:18.5187029Z + local orig_preload 2024-08-07T17:58:18.5187336Z + local commit 2024-08-07T17:58:18.5189424Z ++ get_pinned_commit vision 2024-08-07T17:58:18.5189848Z ++ cat .github/ci_commit_pins/vision.txt 2024-08-07T17:58:18.5207010Z + commit=d23a6e1664d20707c11781299611436e1f0c104f 2024-08-07T17:58:18.5207621Z + orig_preload= 2024-08-07T17:58:18.5208204Z + '[' -n '' ']' 2024-08-07T17:58:18.5208947Z + pip_install --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-07T17:58:18.5210130Z + pip install --progress-bar off --no-use-pep517 --user git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-07T17:58:19.0028026Z Collecting git+https://github.com/pytorch/vision.git@d23a6e1664d20707c11781299611436e1f0c104f 2024-08-07T17:58:19.0035384Z Cloning https://github.com/pytorch/vision.git (to revision d23a6e1664d20707c11781299611436e1f0c104f) to /tmp/pip-req-build-03dgnab9 2024-08-07T17:58:19.0062772Z Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-03dgnab9 2024-08-07T17:58:20.5985949Z Running command git rev-parse -q --verify 'sha^d23a6e1664d20707c11781299611436e1f0c104f' 2024-08-07T17:58:20.6014029Z Running command git fetch -q https://github.com/pytorch/vision.git d23a6e1664d20707c11781299611436e1f0c104f 2024-08-07T17:58:22.2762536Z Running command git checkout -q d23a6e1664d20707c11781299611436e1f0c104f 2024-08-07T17:58:22.6314990Z Resolved https://github.com/pytorch/vision.git to commit d23a6e1664d20707c11781299611436e1f0c104f 2024-08-07T17:58:25.4987643Z Preparing metadata (setup.py) ... [?25l- \ done 2024-08-07T17:58:25.5051830Z [?25hRequirement already satisfied: numpy in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (1.21.2) 2024-08-07T17:58:25.5056360Z Requirement already satisfied: torch in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (2.5.0a0+git016588f) 2024-08-07T17:58:25.5064668Z Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torchvision==0.19.0a0+d23a6e1) (10.3.0) 2024-08-07T17:58:25.5399504Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.13.1) 2024-08-07T17:58:25.5407642Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (4.12.2) 2024-08-07T17:58:25.5412539Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2.8.8) 2024-08-07T17:58:25.5417124Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (3.1.4) 2024-08-07T17:58:25.5422490Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (2024.6.1) 2024-08-07T17:58:25.5438613Z Requirement already satisfied: sympy>=1.13.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch->torchvision==0.19.0a0+d23a6e1) (1.13.1) 2024-08-07T17:58:25.5484853Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy>=1.13.0->torch->torchvision==0.19.0a0+d23a6e1) (1.3.0) 2024-08-07T17:58:25.6761889Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch->torchvision==0.19.0a0+d23a6e1) (2.1.5) 2024-08-07T17:58:25.7069934Z Building wheels for collected packages: torchvision 2024-08-07T18:00:10.5669086Z Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | done 2024-08-07T18:00:10.5754343Z [?25h Created wheel for torchvision: filename=torchvision-0.19.0a0+d23a6e1-cp310-cp310-linux_x86_64.whl size=2115968 sha256=cb0209fb5fcc20b3f36b6eb9657fb487e8de28786f4a025dca60cc829c0c91c6 2024-08-07T18:00:10.5755889Z Stored in directory: /var/lib/jenkins/.cache/pip/wheels/0e/56/35/02931e71eb23fd2b85591c7ec05b733ca7c8b328a2fd151f96 2024-08-07T18:00:10.5804940Z Successfully built torchvision 2024-08-07T18:00:11.5766899Z Installing collected packages: torchvision 2024-08-07T18:00:12.1514188Z Successfully installed torchvision-0.19.0a0+d23a6e1 2024-08-07T18:00:12.2774392Z + '[' -n '' ']' 2024-08-07T18:00:12.2774743Z + test_python_shard 1 2024-08-07T18:00:12.2775068Z + [[ -z 5 ]] 2024-08-07T18:00:12.2775683Z + python test/run_test.py --exclude-jit-executor --exclude-distributed-tests --shard 1 5 --verbose 2024-08-07T18:00:12.4114335Z /var/lib/jenkins/workspace/test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-07T18:00:12.4115436Z import pkg_resources 2024-08-07T18:00:17.0184771Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-07T18:00:17.0779553Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-07T18:00:17.1249210Z Ignoring disabled issues: [''] 2024-08-07T18:00:17.1390418Z Found test times from artifacts 2024-08-07T18:00:17.1931024Z Found test times from artifacts 2024-08-07T18:00:17.1951153Z Running 25% of tests based on TD 2024-08-07T18:00:17.2289930Z Running parallel tests on 2 processes 2024-08-07T18:00:17.2292214Z Name: tests to run (est. time: 74.1min) 2024-08-07T18:00:17.2292671Z Serial tests (6): 2024-08-07T18:00:17.2292987Z test_torch 1/1 2024-08-07T18:00:17.2293287Z test_fake_tensor 1/1 2024-08-07T18:00:17.2293679Z test_cpp_api_parity 1/1 2024-08-07T18:00:17.2294024Z test_utils 1/1 2024-08-07T18:00:17.2294333Z inductor/test_max_autotune 1/1 2024-08-07T18:00:17.2294718Z test_show_pickle 1/1 2024-08-07T18:00:17.2295058Z Parallel tests (25): 2024-08-07T18:00:17.2295377Z functorch/test_ops 3/9 2024-08-07T18:00:17.2295724Z test_ops 3/11 2024-08-07T18:00:17.2296031Z test_ops 4/11 2024-08-07T18:00:17.2296372Z test_decomp 2/19 2024-08-07T18:00:17.2297280Z test_decomp 3/19 2024-08-07T18:00:17.2297813Z test_decomp 12/19 2024-08-07T18:00:17.2298414Z test_decomp 13/19 2024-08-07T18:00:17.2298854Z functorch/test_aotdispatch 1/1 2024-08-07T18:00:17.2299268Z inductor/test_compiled_autograd 1/2 2024-08-07T18:00:17.2299676Z inductor/test_compiled_autograd 2/2 2024-08-07T18:00:17.2300092Z inductor/test_aot_inductor 1/1 2024-08-07T18:00:17.2300499Z inductor/test_torchinductor 4/4 2024-08-07T18:00:17.2300876Z test_proxy_tensor 1/1 2024-08-07T18:00:17.2301268Z inductor/test_efficient_conv_bn_eval 1/1 2024-08-07T18:00:17.2301694Z functorch/test_vmap 1/1 2024-08-07T18:00:17.2302044Z dynamo/test_aot_autograd 1/1 2024-08-07T18:00:17.2302412Z test_package 1/1 2024-08-07T18:00:17.2302732Z test_ops_fwd_gradients 1/2 2024-08-07T18:00:17.2303157Z inductor/test_torchinductor_dynamic_shapes 4/4 2024-08-07T18:00:17.2303614Z inductor/test_b2b_gemm 1/1 2024-08-07T18:00:17.2304062Z inductor/test_torchinductor_codegen_dynamic_shapes 2/4 2024-08-07T18:00:17.2304560Z inductor/test_debug_trace 1/1 2024-08-07T18:00:17.2304969Z inductor/test_mkldnn_pattern_matcher 1/2 2024-08-07T18:00:17.2305397Z inductor/test_mkldnn_pattern_matcher 2/2 2024-08-07T18:00:17.2305879Z inductor/test_aot_inductor_package 1/1 2024-08-07T18:00:17.2306345Z Name: excluded (est. time: 22.57min) 2024-08-07T18:00:17.2306734Z Serial tests (38): 2024-08-07T18:00:17.2307045Z test_fx 1/1 2024-08-07T18:00:17.2307357Z test_reductions 1/1 2024-08-07T18:00:17.2308006Z test_tensorexpr 1/1 2024-08-07T18:00:17.2308340Z test_multiprocessing 1/1 2024-08-07T18:00:17.2308714Z test_cpp_extensions_jit 1/1 2024-08-07T18:00:17.2309094Z test_tensor_creation_ops 1/1 2024-08-07T18:00:17.2309473Z distributions/test_distributions 1/1 2024-08-07T18:00:17.2309914Z inductor/test_distributed_patterns 1/1 2024-08-07T18:00:17.2310331Z nn/test_convolution 1/1 2024-08-07T18:00:17.2310670Z test_dispatch 1/1 2024-08-07T18:00:17.2311192Z test_multiprocessing_spawn 1/1 2024-08-07T18:00:17.2311613Z test_spectral_ops 1/1 2024-08-07T18:00:17.2311963Z inductor/test_benchmark_fusion 1/1 2024-08-07T18:00:17.2312361Z nn/test_pooling 1/1 2024-08-07T18:00:17.2312738Z functorch/test_memory_efficient_fusion 1/1 2024-08-07T18:00:17.2313210Z test_cpp_extensions_open_device_registration 1/1 2024-08-07T18:00:17.2313662Z test_sort_and_select 1/1 2024-08-07T18:00:17.2313996Z test_overrides 1/1 2024-08-07T18:00:17.2314337Z test_mobile_optimizer 1/1 2024-08-07T18:00:17.2314720Z test_python_dispatch 1/1 2024-08-07T18:00:17.2315054Z test_autocast 1/1 2024-08-07T18:00:17.2315397Z test_namedtuple_return_api 1/1 2024-08-07T18:00:17.2315776Z test_cuda_trace 1/1 2024-08-07T18:00:17.2316087Z test_native_mha 1/1 2024-08-07T18:00:17.2316427Z test_autograd_fallback 1/1 2024-08-07T18:00:17.2316789Z test_jit_disabled 1/1 2024-08-07T18:00:17.2317142Z test_cpp_extensions_aot_ninja 1/1 2024-08-07T18:00:17.2317562Z test_cpp_extensions_aot_no_ninja 1/1 2024-08-07T18:00:17.2317980Z test_cuda_nvml_based_avail 1/1 2024-08-07T18:00:17.2318355Z inductor/test_flex_attention 1/1 2024-08-07T18:00:17.2318771Z test_cpp_extensions_mtia_backend 1/1 2024-08-07T18:00:17.2319181Z test_cuda_primary_ctx 1/1 2024-08-07T18:00:17.2319561Z test_cpp_extensions_stream_and_event 1/1 2024-08-07T18:00:17.2319995Z inductor/test_cutlass_backend 1/1 2024-08-07T18:00:17.2320397Z test_ci_sanity_check_fail 1/1 2024-08-07T18:00:17.2320767Z test_autoload_enable 1/1 2024-08-07T18:00:17.2321121Z doctests 1/1 2024-08-07T18:00:17.2321439Z test_autoload_disable 1/1 2024-08-07T18:00:17.2321783Z Parallel tests (38): 2024-08-07T18:00:17.2322120Z dynamo/test_export 1/1 2024-08-07T18:00:17.2322489Z dynamo/test_input_attr_tracking 1/1 2024-08-07T18:00:17.2322874Z test_maskedtensor 1/1 2024-08-07T18:00:17.2323225Z test_type_promotion 1/1 2024-08-07T18:00:17.2323579Z dynamo/test_bytecode_utils 1/1 2024-08-07T18:00:17.2323995Z torch_np/numpy_tests/fft/test_helper 1/1 2024-08-07T18:00:17.2324441Z test_functionalization_of_rng_ops 1/1 2024-08-07T18:00:17.2324861Z functorch/test_vmap_registrations 1/1 2024-08-07T18:00:17.2325281Z torch_np/test_reductions 1/1 2024-08-07T18:00:17.2325695Z inductor/test_triton_extension_backend 1/1 2024-08-07T18:00:17.2326151Z torch_np/numpy_tests/lib/test_function_base 1/1 2024-08-07T18:00:17.2326591Z export/test_hop 1/1 2024-08-07T18:00:17.2326946Z torch_np/test_ufuncs_basic 1/1 2024-08-07T18:00:17.2327316Z dynamo/test_recompiles 1/1 2024-08-07T18:00:17.2327674Z nn/test_dropout 1/1 2024-08-07T18:00:17.2328022Z dynamo/test_python_autograd 1/1 2024-08-07T18:00:17.2328391Z test_function_schema 1/1 2024-08-07T18:00:17.2328755Z functorch/test_minifier 1/1 2024-08-07T18:00:17.2329127Z dynamo/test_reorder_logs 1/1 2024-08-07T18:00:17.2329510Z torch_np/numpy_tests/core/test_dtype 1/1 2024-08-07T18:00:17.2329926Z test_prims 1/1 2024-08-07T18:00:17.2330249Z test_functional_optim 1/1 2024-08-07T18:00:17.2330645Z torch_np/numpy_tests/lib/test_index_tricks 1/1 2024-08-07T18:00:17.2331103Z inductor/test_group_batch_fusion 1/1 2024-08-07T18:00:17.2331508Z lazy/test_debug_util 1/1 2024-08-07T18:00:17.2331850Z dynamo/test_debug_utils 1/1 2024-08-07T18:00:17.2332245Z test_set_default_mobile_cpu_allocator 1/1 2024-08-07T18:00:17.2332682Z profiler/test_execution_trace 1/1 2024-08-07T18:00:17.2333063Z test_numpy_interop 1/1 2024-08-07T18:00:17.2333558Z torch_np/test_nep50_examples 1/1 2024-08-07T18:00:17.2333951Z test_content_store 1/1 2024-08-07T18:00:17.2334299Z inductor/test_triton_heuristics 1/1 2024-08-07T18:00:17.2334710Z inductor/test_autoheuristic 1/1 2024-08-07T18:00:17.2335110Z inductor/test_layout_optim 1/1 2024-08-07T18:00:17.2335495Z inductor/test_scatter_optimization 1/1 2024-08-07T18:00:17.2335924Z inductor/test_cudagraph_trees 1/1 2024-08-07T18:00:17.2336402Z xpu/test_gemm 1/1 2024-08-07T18:00:17.2336738Z optim/test_swa_utils 1/1 2024-08-07T18:00:17.2337240Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-07T18:00:17.2371853Z Running test_torch 1/1 ... [2024-08-07 18:00:17.236831] 2024-08-07T18:00:17.2376363Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_torch.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:00:17.237254] 2024-08-07T18:00:35.5742649Z 2024-08-07T18:00:35.5743934Z PRINTING LOG FILE of test_torch 1/1 (test/test-reports/test_torch_1.1_1a26bcf1801a71c9_.log) 2024-08-07T18:00:35.5745248Z Test results will be stored in test-reports/python-pytest/test_torch/test_torch-0f3c0fa08db3a8d6.xml 2024-08-07T18:00:35.5746693Z ============================= test session starts ============================== 2024-08-07T18:00:35.5747732Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.5.0 -- /opt/conda/envs/py_3.10/bin/python 2024-08-07T18:00:35.5748715Z cachedir: .pytest_cache 2024-08-07T18:00:35.5750249Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-08-07T18:00:35.5751096Z rootdir: /var/lib/jenkins/workspace 2024-08-07T18:00:35.5751634Z configfile: pytest.ini 2024-08-07T18:00:35.5752991Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-08-07T18:00:35.5753876Z collecting ... collected 1043 items 2024-08-07T18:00:35.5754371Z stepcurrent: Cannot find last run test, not skipping 2024-08-07T18:00:35.6128435Z Running 1043 items in this shard: test/test_torch.py::TestBasicVitalSigns::test_basic_vitals, test/test_torch.py::TestBasicVitalSigns::test_basic_vitals_read_write, test/test_torch.py::TestBasicVitalSigns::test_dataloader_vitals, test/test_torch.py::TestTorch::test_RNGState, test/test_torch.py::TestTorch::test_RNGStateAliasing, test/test_torch.py::TestTorch::test_RNG_after_pickle, test/test_torch.py::TestTorch::test_Size, test/test_torch.py::TestTorch::test_Size_iter, test/test_torch.py::TestTorch::test_Size_scalar, test/test_torch.py::TestTorch::test_add_meta_scalar, test/test_torch.py::TestTorch::test_allow_tensor_metadata_change, test/test_torch.py::TestTorch::test_apply, test/test_torch.py::TestTorch::test_as_subclass, test/test_torch.py::TestTorch::test_assert_async, test/test_torch.py::TestTorch::test_backward_hooks_traverse, test/test_torch.py::TestTorch::test_batch_norm_cpu_inference, test/test_torch.py::TestTorch::test_bf16_supported_on_cpu, test/test_torch.py::TestTorch::test_bmm_multithreaded, test/test_torch.py::TestTorch::test_boxMullerState, test/test_torch.py::TestTorch::test_c10_feature_enabled, test/test_torch.py::TestTorch::test_cat_neg_dim, test/test_torch.py::TestTorch::test_check, test/test_torch.py::TestTorch::test_chunk_neg_dim, test/test_torch.py::TestTorch::test_conj_neg_tolist, test/test_torch.py::TestTorch::test_contains, test/test_torch.py::TestTorch::test_copy_broadcast, test/test_torch.py::TestTorch::test_copy_dtypes, test/test_torch.py::TestTorch::test_copy_float16, test/test_torch.py::TestTorch::test_copy_many_to_one, test/test_torch.py::TestTorch::test_copy_transpose, test/test_torch.py::TestTorch::test_cuda_not_built, test/test_torch.py::TestTorch::test_cummax_neg_dim, test/test_torch.py::TestTorch::test_cummin_neg_dim, test/test_torch.py::TestTorch::test_cumprod_neg_dim, test/test_torch.py::TestTorch::test_cumsum_neg_dim, test/test_torch.py::TestTorch::test_cxx_flags, test/test_torch.py::TestTorch::test_data_ptr_of_empty_tensor_with_storage, test/test_torch.py::TestTorch::test_data_ptr_of_empty_view_with_storage, test/test_torch.py::TestTorch::test_deepcopy_gradient, test/test_torch.py::TestTorch::test_deepcopy_parameter, test/test_torch.py::TestTorch::test_deterministic_fill_uninitialized_memory, test/test_torch.py::TestTorch::test_deterministic_flag, test/test_torch.py::TestTorch::test_device, test/test_torch.py::TestTorch::test_dim_order, test/test_torch.py::TestTorch::test_dir, test/test_torch.py::TestTorch::test_doc, test/test_torch.py::TestTorch::test_doc_template, test/test_torch.py::TestTorch::test_dot_data_use, test/test_torch.py::TestTorch::test_dtype_is_signed, test/test_torch.py::TestTorch::test_element_size, test/test_torch.py::TestTorch::test_empty_meta, test/test_torch.py::TestTorch::test_empty_storage_view, test/test_torch.py::TestTorch::test_equal, test/test_torch.py::TestTorch::test_error_msg_type_translation, test/test_torch.py::TestTorch::test_fill_diagonal, test/test_torch.py::TestTorch::test_format_scalar_meta, test/test_torch.py::TestTorch::test_from_buffer, test/test_torch.py::TestTorch::test_from_file, test/test_torch.py::TestTorch::test_gather_neg_dim, test/test_torch.py::TestTorch::test_generator_cpu, test/test_torch.py::TestTorch::test_get_cpu_capability, test/test_torch.py::TestTorch::test_has_internal_overlap, test/test_torch.py::TestTorch::test_has_storage, test/test_torch.py::TestTorch::test_index_add, test/test_torch.py::TestTorch::test_index_add_all_dtypes, test/test_torch.py::TestTorch::test_index_add_cornercase, test/test_torch.py::TestTorch::test_index_add_correctness, test/test_torch.py::TestTorch::test_index_add_neg_dim, test/test_torch.py::TestTorch::test_index_copy_neg_dim, test/test_torch.py::TestTorch::test_index_fill_neg_dim, test/test_torch.py::TestTorch::test_index_select_neg_dim, test/test_torch.py::TestTorch::test_invalid_arg_error_handling, test/test_torch.py::TestTorch::test_invalid_generator_raises, test/test_torch.py::TestTorch::test_is_nonzero, test/test_torch.py::TestTorch::test_is_same_size, test/test_torch.py::TestTorch::test_iter, test/test_torch.py::TestTorch::test_kthvalue_neg_dim, test/test_torch.py::TestTorch::test_linspace_logspace, test/test_torch.py::TestTorch::test_logcumsumexp_neg_dim, test/test_torch.py::TestTorch::test_manual_seed, test/test_torch.py::TestTorch::test_map, test/test_torch.py::TestTorch::test_map2, test/test_torch.py::TestTorch::test_max_neg_dim, test/test_torch.py::TestTorch::test_mean_neg_dim, test/test_torch.py::TestTorch::test_median_neg_dim, test/test_torch.py::TestTorch::test_memory_format, test/test_torch.py::TestTorch::test_memory_format_contiguous_returns_same_tensor_if_already_satisfies, test/test_torch.py::TestTorch::test_memory_format_empty, test/test_torch.py::TestTorch::test_min_neg_dim, test/test_torch.py::TestTorch::test_mode_neg_dim, test/test_torch.py::TestTorch::test_multinomial_invalid_probs, test/test_torch.py::TestTorch::test_nanmedian_neg_dim, test/test_torch.py::TestTorch::test_narrow_neg_dim, test/test_torch.py::TestTorch::test_nbytes, test/test_torch.py::TestTorch::test_ndim, test/test_torch.py::TestTorch::test_new, test/test_torch.py::TestTorch::test_newaxis_numpy_comparison, test/test_torch.py::TestTorch::test_newindex, test/test_torch.py::TestTorch::test_no_cuda_monkeypatch, test/test_torch.py::TestTorch::test_norm_neg_dim, test/test_torch.py::TestTorch::test_normal_shape, test/test_torch.py::TestTorch::test_numel, test/test_torch.py::TestTorch::test_parallel_info, test/test_torch.py::TestTorch::test_parsing_double, test/test_torch.py::TestTorch::test_parsing_int64, test/test_torch.py::TestTorch::test_parsing_intlist, test/test_torch.py::TestTorch::test_permute, test/test_torch.py::TestTorch::test_pickle, test/test_torch.py::TestTorch::test_pickle_dtype, test/test_torch.py::TestTorch::test_pickle_function, test/test_torch.py::TestTorch::test_pickle_generator, test/test_torch.py::TestTorch::test_pickle_parameter, test/test_torch.py::TestTorch::test_pickle_parameter_no_requires_grad, test/test_torch.py::TestTorch::test_pickle_size, test/test_torch.py::TestTorch::test_pin_memory, test/test_torch.py::TestTorch::test_print, test/test_torch.py::TestTorch::test_prod_neg_dim, test/test_torch.py::TestTorch::test_pyobj_preserved, test/test_torch.py::TestTorch::test_qengine, test/test_torch.py::TestTorch::test_renorm_neg_dim, test/test_torch.py::TestTorch::test_resizable, test/test_torch.py::TestTorch::test_reversed, test/test_torch.py::TestTorch::test_scatter_neg_dim, test/test_torch.py::TestTorch::test_select_neg_dim, test/test_torch.py::TestTorch::test_set_flush_denormal, test/test_torch.py::TestTorch::test_setting_real_imag_to_a_number, test/test_torch.py::TestTorch::test_show_config, test/test_torch.py::TestTorch::test_size_neg_dim, test/test_torch.py::TestTorch::test_size_stride, test/test_torch.py::TestTorch::test_sizeof, test/test_torch.py::TestTorch::test_slice, test/test_torch.py::TestTorch::test_slow_test, test/test_torch.py::TestTorch::test_sobolengine_bounds, test/test_torch.py::TestTorch::test_sobolengine_bounds_scrambled, test/test_torch.py::TestTorch::test_sobolengine_continuing, test/test_torch.py::TestTorch::test_sobolengine_continuing_scrambled, test/test_torch.py::TestTorch::test_sobolengine_default_dtype, test/test_torch.py::TestTorch::test_sobolengine_distribution, test/test_torch.py::TestTorch::test_sobolengine_distribution_scrambled, test/test_torch.py::TestTorch::test_sobolengine_draw, test/test_torch.py::TestTorch::test_sobolengine_draw_base2, test/test_torch.py::TestTorch::test_sobolengine_draw_base2_scrambled, test/test_torch.py::TestTorch::test_sobolengine_draw_scrambled, test/test_torch.py::TestTorch::test_sobolengine_fast_forward, test/test_torch.py::TestTorch::test_sobolengine_fast_forward_scrambled, test/test_torch.py::TestTorch::test_sobolengine_first_point, test/test_torch.py::TestTorch::test_sobolengine_high_dim, test/test_torch.py::TestTorch::test_sobolengine_raise, test/test_torch.py::TestTorch::test_sobolengine_reset, test/test_torch.py::TestTorch::test_sobolengine_reset_scrambled, test/test_torch.py::TestTorch::test_sort_neg_dim, test/test_torch.py::TestTorch::test_split_neg_dim, test/test_torch.py::TestTorch::test_split_with_sizes_copy_out, test/test_torch.py::TestTorch::test_squeeze_neg_dim, test/test_torch.py::TestTorch::test_std_neg_dim, test/test_torch.py::TestTorch::test_storage_base_init, test/test_torch.py::TestTorch::test_storage_base_new, test/test_torch.py::TestTorch::test_storage_byteswap, test/test_torch.py::TestTorch::test_storage_casts, test/test_torch.py::TestTorch::test_storage_cycle_via_dict, test/test_torch.py::TestTorch::test_storage_cycle_via_slots, test/test_torch.py::TestTorch::test_storage_dead_weak_ref, test/test_torch.py::TestTorch::test_storage_dealloc, test/test_torch.py::TestTorch::test_storage_dealloc_resurrected, test/test_torch.py::TestTorch::test_storage_dealloc_subclass_resurrected, test/test_torch.py::TestTorch::test_storage_dealloc_subclass_zombie, test/test_torch.py::TestTorch::test_storage_dict_dealloc, test/test_torch.py::TestTorch::test_storage_error, test/test_torch.py::TestTorch::test_storage_error_no_attribute, test/test_torch.py::TestTorch::test_storage_finalizer_dealloc, test/test_torch.py::TestTorch::test_storage_fix_weakref_no_leak, test/test_torch.py::TestTorch::test_storage_from_tensor_dealloc, test/test_torch.py::TestTorch::test_storage_from_tensor_dealloc_resurrected, test/test_torch.py::TestTorch::test_storage_from_tensor_dealloc_zombie, test/test_torch.py::TestTorch::test_storage_preserve_nonhermetic_in_hermetic_context, test/test_torch.py::TestTorch::test_storage_resurrected_weak_ref, test/test_torch.py::TestTorch::test_storage_slot_dealloc, test/test_torch.py::TestTorch::test_storage_weakref_dealloc, test/test_torch.py::TestTorch::test_structseq_repr, test/test_torch.py::TestTorch::test_subclass_preserved, test/test_torch.py::TestTorch::test_subclass_tensors, test/test_torch.py::TestTorch::test_sum_neg_dim, test/test_torch.py::TestTorch::test_swap_basic, test/test_torch.py::TestTorch::test_swap_fail_slots, test/test_torch.py::TestTorch::test_t_not_2d_error, test/test_torch.py::TestTorch::test_tensor_base_init, test/test_torch.py::TestTorch::test_tensor_base_new, test/test_torch.py::TestTorch::test_tensor_ctor_scalar, test/test_torch.py::TestTorch::test_tensor_cycle_via_dict, test/test_torch.py::TestTorch::test_tensor_cycle_via_slots, test/test_torch.py::TestTorch::test_tensor_dead_weak_ref, test/test_torch.py::TestTorch::test_tensor_dict_dealloc, test/test_torch.py::TestTorch::test_tensor_finalizer_dealloc, test/test_torch.py::TestTorch::test_tensor_fix_weakref_no_leak, test/test_torch.py::TestTorch::test_tensor_resurrected_weak_ref, test/test_torch.py::TestTorch::test_tensor_set, test/test_torch.py::TestTorch::test_tensor_set_errors, test/test_torch.py::TestTorch::test_tensor_slot_dealloc, test/test_torch.py::TestTorch::test_tensor_weakref_dealloc, test/test_torch.py::TestTorch::test_tensor_where_scalar, test/test_torch.py::TestTorch::test_tensoriterator_output_setup, test/test_torch.py::TestTorch::test_terminate_handler_on_crash, test/test_torch.py::TestTorch::test_to, test/test_torch.py::TestTorch::test_to_with_tensor, test/test_torch.py::TestTorch::test_topk_neg_dim, test/test_torch.py::TestTorch::test_torch_from_file, test/test_torch.py::TestTorch::test_transpose_neg_dim, test/test_torch.py::TestTorch::test_type, test/test_torch.py::TestTorch::test_type_alias, test/test_torch.py::TestTorch::test_type_conversion_via_dtype_name, test/test_torch.py::TestTorch::test_typed_storage_deprecation_warning, test/test_torch.py::TestTorch::test_typed_storage_internal_no_warning, test/test_torch.py::TestTorch::test_unbind_neg_dim, test/test_torch.py::TestTorch::test_unflatten, test/test_torch.py::TestTorch::test_unfold_neg_dim, test/test_torch.py::TestTorch::test_unsqueeze_neg_dim, test/test_torch.py::TestTorch::test_upsample_nearest1d_meta, test/test_torch.py::TestTorch::test_upsample_nearest2d_meta, test/test_torch.py::TestTorch::test_var_neg_dim, test/test_torch.py::TestTorch::test_warn_types, test/test_torch.py::TestTorch::test_wildcard_import, test/test_torch.py::TestVitalSignsCudaCUDA::test_cuda_vitals_gpu_only_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcdiv_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_addcmul_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_assertRaisesRegex_ignore_msg_non_native_device_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_edge_cases_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_edge_cases_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_edge_cases_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_p_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_p_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_p_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bernoulli_self_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bfloat16_neg_abs_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bool_tensor_value_change_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_add_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_addcdiv_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_addcmul_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_atan2_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_copy_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_dist_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_div_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_eq_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_fmod_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_ge_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_gt_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_le_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_lerp_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_lt_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_map2_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_map_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_masked_fill_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_masked_scatter_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_masked_select_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_max_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_min_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_mul_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_ne_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_pow_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_remainder_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_broadcast_fn_sub_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_bytes_to_scalar_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_kstest_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_no_inf_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cauchy_no_inf_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_cuda_backward_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_euclidean_large_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_grad_p_lt_1_no_nan_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_large_batch_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_large_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_non_contiguous_batch_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_non_contiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_norm_batch_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_norm_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cdist_same_inputs_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_check_tensor_all_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_check_tensor_internal_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_clone_all_dtypes_and_devices_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_clone_not_memory_dense_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_clone_zero_stride_dim_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_complex_half_experimental_warning_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_constants_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_conv_transposed_backward_agnostic_to_memory_format_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_conv_transposed_large_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_complex32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy__cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_all_dtypes_and_devices_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_math_view_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_mem_overlap_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_transpose_math_view_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_transpose_math_view_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_copy_transpose_math_view_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_corrcoef_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_corrcoef_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_corrcoef_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cov_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cpp_warnings_have_python_context_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cublas_config_nondeterministic_alert_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cummax_cummin_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cummax_discontiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cummin_discontiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cumprod_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_cumsum_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deepcopy_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deepcopy_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deepcopy_scalar_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deepcopy_scalar_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_empty_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_interpolate_bilinear_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_replication_pad2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_deterministic_resize_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_device_guard_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_diff_noncontig_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_dim_function_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_discontiguous_out_cumsum_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_dist_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_dtypetensor_warnings_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_errors_index_copy_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_expected_failure_xla_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_kstest_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_kstest_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_kstest_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_kstest_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_no_zero_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_exponential_no_zero_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gather_backward_deterministic_path_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gather_backward_one_dim_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_geometric_kstest_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scale_will_not_overflow_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaler_deprecated_warning_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaler_pass_itself_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_accumulation_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach0_fused0_AdamW_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach0_fused0_Adam_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach0_fused0_SGD_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach2_fused_True_AdamW_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach2_fused_True_Adam_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach2_fused_True_SGD_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach_True_fused1_AdamW_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach_True_fused1_Adam_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_autocast_foreach_True_fused1_SGD_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_clipping_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_clipping_separate_unscale_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_multiple_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_penalty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_state_dict_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_unscale_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_unscale_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_unscale_sparse_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_grad_scaling_update_scale_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_all_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_all_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_all_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_extreme_cases_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_extreme_cases_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_extreme_cases_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_spacing_list_length_error_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_spacing_list_length_error_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_spacing_list_length_error_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_gradient_type_promotion_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_hook_remove_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_add_deterministic_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_add_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_deterministic_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_copy_scalars_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_fill_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_put_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_put_non_accumulate_deterministic_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amax_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_amin_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_mean_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_reduce_reduce_prod_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_index_select_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_invalid_shapes_grid_sampler_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_is_set_to_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_is_signed_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_complex32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_item_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_large_cumprod_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_large_cumsum_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_binary_op_no_materialize_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lazy_clone_view_materialize_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_log_normal_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_log_normal_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_log_normal_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_log_normal_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_logcumsumexp_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_lognormal_kstest_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_bool_tensor_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_bfloat16_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_bfloat16_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_bool_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_bool_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_complex128_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_complex128_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_complex64_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_complex64_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float16_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float16_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float32_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float32_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float64_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_float64_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int16_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int16_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int32_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int32_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int64_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int64_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int8_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_int8_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_uint8_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_cuda_uint8_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_fill_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_bool_tensor_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_inplace_noncontiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_large_tensor_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_scatter_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_masked_select_discontiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_clone_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_consistency_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_cpu_and_cuda_ops_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_empty_like_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_factory_like_functions_preserve_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_operators_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_preserved_after_permute_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_propagation_rules_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_to_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_type_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_memory_format_type_shortcuts_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_module_share_memory_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cpu_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cpu_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cpu_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_deterministic_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_deterministic_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_deterministic_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_device_constrain_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_empty_w_replacement_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_empty_wo_replacement_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_gpu_device_constrain_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_multinomial_rng_state_advance_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_narrow_copy_non_contiguous_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_narrow_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_AdaptiveAvgPool2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_AdaptiveAvgPool3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_AdaptiveMaxPool2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_AvgPool3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_CTCLoss_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_EmbeddingBag_max_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_FractionalMaxPool2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_FractionalMaxPool3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxPool3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool1d_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool1d_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool1d_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool2d_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool2d_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool2d_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool3d_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool3d_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_MaxUnpool3d_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_NLLLoss_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReflectionPad1d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReflectionPad2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReflectionPad3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReplicationPad1d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReplicationPad2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_ReplicationPad3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_bincount_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_cumsum_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_grid_sample_2d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_grid_sample_3d_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_histc_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_interpolate_bicubic_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_interpolate_bilinear_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_interpolate_linear_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_interpolate_trilinear_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_kthvalue_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_median_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_put_accumulate_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_alert_put_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_resize_quantized_cuda_qint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_resize_quantized_cuda_qint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_resize_quantized_cuda_quint2x4, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_resize_quantized_cuda_quint4x2, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nondeterministic_resize_quantized_cuda_quint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_normal_kstest_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_normal_kstest_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_normal_kstest_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_normal_kstest_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_nullary_op_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_pairwise_distance_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_parallel_cow_materialize_error_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_params_invalidated_with_grads_invalidated_between_unscale_and_step_AdamW_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_params_invalidated_with_grads_invalidated_between_unscale_and_step_Adam_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_params_invalidated_with_grads_invalidated_between_unscale_and_step_SGD_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_pdist_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_pdist_norm_large_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_pickle_gradscaler_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_pin_memory_from_constructor_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_accumulate_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_put_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_reduced_type_float_copy_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_reduced_type_float_copy_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_repeat_interleave_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scalar_check_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_add_bool_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_add_non_unique_index_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_add_one_dim_deterministic_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_add_to_large_input_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_bool_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_multiply_unsupported_dtypes_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_multiply_unsupported_dtypes_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_non_unique_index_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_operations_to_large_input_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_reduce_scalar_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_to_large_input_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_scatter_zero_size_index_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_serialization_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_default_tensor_type_warnings_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_set_storage_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_shift_mem_overlap_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_skip_xla_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_all_devices_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_errors_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_from_tensor_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_meta_ok_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_qint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_qint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_quint4x2, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_quint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_storage_setitem_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_strides_propagation_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_sync_warning_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_take_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_uint16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_uint32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_uint64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_from_storage_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_set_errors_multigpu_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_shape_empty_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_storage_type_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_tensor_type_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_ternary_op_mem_overlap_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_bool, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_complex128, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_complex64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_int16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_int32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_int64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_int8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_typed_storage_meta_cuda_uint8, test/test_torch.py::TestTorchDeviceTypeCUDA::test_unfold_all_devices_and_dtypes_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_unfold_scalars_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_uniform_kstest_cuda_bfloat16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_uniform_kstest_cuda_float16, test/test_torch.py::TestTorchDeviceTypeCUDA::test_uniform_kstest_cuda_float32, test/test_torch.py::TestTorchDeviceTypeCUDA::test_uniform_kstest_cuda_float64, test/test_torch.py::TestTorchDeviceTypeCUDA::test_untyped_storage_meta_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_warn_always_caught_cuda, test/test_torch.py::TestTorchDeviceTypeCUDA::test_where_scalar_handcrafted_values_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_advancedindex_mixed_cpu_devices_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_advancedindex_mixed_devices_error_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_clamp_cuda_float32, test/test_torch.py::TestDevicePrecisionCUDA::test_clamp_cuda_float64, test/test_torch.py::TestDevicePrecisionCUDA::test_clamp_cuda_int64, test/test_torch.py::TestDevicePrecisionCUDA::test_copy_broadcast_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_copy_noncontig_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_cuda_device_idx_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_device_serialization_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_float16, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_float32, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_float64, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_int16, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_int32, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_int64, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_int8, test/test_torch.py::TestDevicePrecisionCUDA::test_from_sequence_cuda_uint8, test/test_torch.py::TestDevicePrecisionCUDA::test_index_add_bfloat16_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_multidevice_serialization_cuda, test/test_torch.py::TestDevicePrecisionCUDA::test_type_conversions_same_device_cuda 2024-08-07T18:00:35.6499501Z 2024-08-07T18:00:35.6500264Z test_torch.py::TestBasicVitalSigns::test_basic_vitals PASSED [0.0034s] [ 0%] 2024-08-07T18:00:35.6501362Z test_torch.py::TestBasicVitalSigns::test_basic_vitals_read_write PASSED [0.0014s] [ 0%] 2024-08-07T18:00:35.6502445Z test_torch.py::TestBasicVitalSigns::test_dataloader_vitals PASSED [0.0021s] [ 0%] 2024-08-07T18:00:35.6503438Z test_torch.py::TestTorch::test_RNGState PASSED [0.0024s] [ 0%] 2024-08-07T18:00:35.6504840Z test_torch.py::TestTorch::test_RNGStateAliasing PASSED [0.0026s] [ 0%] 2024-08-07T18:00:35.6505819Z test_torch.py::TestTorch::test_RNG_after_pickle PASSED [0.0046s] [ 0%] 2024-08-07T18:00:35.6506769Z test_torch.py::TestTorch::test_Size PASSED [0.0020s] [ 0%] 2024-08-07T18:00:35.6507846Z test_torch.py::TestTorch::test_Size_iter PASSED [0.0017s] [ 0%] 2024-08-07T18:00:35.6508882Z test_torch.py::TestTorch::test_Size_scalar PASSED [0.0016s] [ 0%] 2024-08-07T18:00:35.6509835Z test_torch.py::TestTorch::test_add_meta_scalar PASSED [0.0019s] [ 0%] 2024-08-07T18:00:35.6510813Z test_torch.py::TestTorch::test_allow_tensor_metadata_change PASSED [0.0014s] [ 1%] 2024-08-07T18:00:35.6511802Z test_torch.py::TestTorch::test_apply PASSED [0.0019s] [ 1%] 2024-08-07T18:00:35.6512793Z test_torch.py::TestTorch::test_as_subclass PASSED [0.0039s] [ 1%] 2024-08-07T18:00:35.6513740Z test_torch.py::TestTorch::test_assert_async PASSED [0.0025s] [ 1%] 2024-08-07T18:00:35.6514711Z test_torch.py::TestTorch::test_backward_hooks_traverse PASSED [0.1432s] [ 1%] 2024-08-07T18:00:35.6515697Z test_torch.py::TestTorch::test_batch_norm_cpu_inference PASSED [0.0067s] [ 1%] 2024-08-07T18:00:35.6516793Z test_torch.py::TestTorch::test_bf16_supported_on_cpu SKIPPED [0.0003s] (Test specific for CPU) [ 1%] 2024-08-07T18:00:35.6517856Z test_torch.py::TestTorch::test_bmm_multithreaded PASSED [0.7641s] [ 1%] 2024-08-07T18:00:35.6518848Z test_torch.py::TestTorch::test_boxMullerState PASSED [0.0045s] [ 1%] 2024-08-07T18:00:35.6519872Z test_torch.py::TestTorch::test_c10_feature_enabled ('RERUN', {'yellow': True}) [0.0026s] [ 1%] 2024-08-07T18:00:35.6520887Z test_torch.py::TestTorch::test_c10_feature_enabled ('RERUN', {'yellow': True}) [0.0026s] [ 1%] 2024-08-07T18:00:35.6521897Z test_torch.py::TestTorch::test_c10_feature_enabled FAILED [0.0025s] [ 1%] 2024-08-07T18:00:35.6522386Z 2024-08-07T18:00:35.6522583Z ==================================== RERUNS ==================================== 2024-08-07T18:00:35.6523310Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:00:35.6523897Z Traceback (most recent call last): 2024-08-07T18:00:35.6524551Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:00:35.6525214Z with mocksetting(True): 2024-08-07T18:00:35.6525762Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:00:35.6526358Z return next(self.gen) 2024-08-07T18:00:35.6526918Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:00:35.6527503Z del os.environ[feature] 2024-08-07T18:00:35.6528017Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:00:35.6528585Z raise KeyError(key) from None 2024-08-07T18:00:35.6529812Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:00:35.6531350Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:00:35.6531932Z Traceback (most recent call last): 2024-08-07T18:00:35.6532561Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:00:35.6533212Z with mocksetting(True): 2024-08-07T18:00:35.6533763Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:00:35.6534453Z return next(self.gen) 2024-08-07T18:00:35.6535001Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:00:35.6535601Z del os.environ[feature] 2024-08-07T18:00:35.6536095Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:00:35.6536658Z raise KeyError(key) from None 2024-08-07T18:00:35.6537987Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:00:35.6539271Z =================================== FAILURES =================================== 2024-08-07T18:00:35.6539996Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:00:35.6540569Z Traceback (most recent call last): 2024-08-07T18:00:35.6541217Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:00:35.6541849Z with mocksetting(True): 2024-08-07T18:00:35.6542399Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:00:35.6542985Z return next(self.gen) 2024-08-07T18:00:35.6543514Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:00:35.6544116Z del os.environ[feature] 2024-08-07T18:00:35.6544634Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:00:35.6545179Z raise KeyError(key) from None 2024-08-07T18:00:35.6546388Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:00:35.6548053Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_torch/test_torch-0f3c0fa08db3a8d6.xml - 2024-08-07T18:00:35.6549149Z =========================== short test summary info ============================ 2024-08-07T18:00:35.6554556Z FAILED [0.0025s] test_torch.py::TestTorch::test_c10_feature_enabled - KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:00:35.6556376Z !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!! 2024-08-07T18:00:35.6557332Z =============== 1 failed, 18 passed, 1 skipped, 2 rerun in 1.11s =============== 2024-08-07T18:00:35.6558037Z [TORCH_VITAL] Dataloader.enabled True 2024-08-07T18:00:35.6558537Z [TORCH_VITAL] Dataloader.basic_unit_test TEST_VALUE_STRING 2024-08-07T18:00:35.6559058Z [TORCH_VITAL] CUDA.used true 2024-08-07T18:00:35.6559423Z Got exit code 1 2024-08-07T18:00:35.6559721Z Retrying single test... 2024-08-07T18:00:35.6560343Z Test results will be stored in test-reports/python-pytest/test_torch/test_torch-5f16e6a2d7696715.xml 2024-08-07T18:00:35.6561233Z ============================= test session starts ============================== 2024-08-07T18:00:35.6561951Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.5.0 -- /opt/conda/envs/py_3.10/bin/python 2024-08-07T18:00:35.6562589Z cachedir: .pytest_cache 2024-08-07T18:00:35.6563354Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-08-07T18:00:35.6564150Z rootdir: /var/lib/jenkins/workspace 2024-08-07T18:00:35.6564557Z configfile: pytest.ini 2024-08-07T18:00:35.6565190Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-08-07T18:00:35.6566245Z collecting ... collected 1043 items / 1042 deselected / 1 selected 2024-08-07T18:00:35.6567126Z stepcurrent: skipping 19 already run items. Running only test/test_torch.py::TestTorch::test_c10_feature_enabled 2024-08-07T18:00:35.6567859Z Running 1 items in this shard 2024-08-07T18:00:35.6568097Z 2024-08-07T18:00:35.6568630Z test_torch.py::TestTorch::test_c10_feature_enabled ('RERUN', {'yellow': True}) [0.0036s] [100%] 2024-08-07T18:00:35.6569781Z test_torch.py::TestTorch::test_c10_feature_enabled ('RERUN', {'yellow': True}) [0.0018s] [100%] 2024-08-07T18:00:35.6570800Z test_torch.py::TestTorch::test_c10_feature_enabled FAILED [0.0016s] [100%] 2024-08-07T18:00:35.6571293Z 2024-08-07T18:00:35.6571488Z ==================================== RERUNS ==================================== 2024-08-07T18:00:35.6572185Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:00:35.6572771Z Traceback (most recent call last): 2024-08-07T18:00:35.6573425Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:00:35.6574062Z with mocksetting(True): 2024-08-07T18:00:35.6574620Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:00:35.6575205Z return next(self.gen) 2024-08-07T18:00:35.6575734Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:00:35.6576345Z del os.environ[feature] 2024-08-07T18:00:35.6576858Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:00:35.6577404Z raise KeyError(key) from None 2024-08-07T18:00:35.6578630Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:00:35.6580136Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:00:35.6580718Z Traceback (most recent call last): 2024-08-07T18:00:35.6581343Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:00:35.6581997Z with mocksetting(True): 2024-08-07T18:00:35.6582551Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:00:35.6583137Z return next(self.gen) 2024-08-07T18:00:35.6583679Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:00:35.6584277Z del os.environ[feature] 2024-08-07T18:00:35.6584778Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:00:35.6585339Z raise KeyError(key) from None 2024-08-07T18:00:35.6586567Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:00:35.6587829Z =================================== FAILURES =================================== 2024-08-07T18:00:35.6588554Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:00:35.6589129Z Traceback (most recent call last): 2024-08-07T18:00:35.6589763Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:00:35.6590417Z with mocksetting(True): 2024-08-07T18:00:35.6590969Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:00:35.6591545Z return next(self.gen) 2024-08-07T18:00:35.6592097Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:00:35.6592698Z del os.environ[feature] 2024-08-07T18:00:35.6593193Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:00:35.6593875Z raise KeyError(key) from None 2024-08-07T18:00:35.6595086Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:00:35.6596741Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_torch/test_torch-5f16e6a2d7696715.xml - 2024-08-07T18:00:35.6598534Z =========================== short test summary info ============================ 2024-08-07T18:00:35.6600683Z FAILED [0.0016s] test_torch.py::TestTorch::test_c10_feature_enabled - KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:00:35.6602457Z !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!! 2024-08-07T18:00:35.6603392Z ================= 1 failed, 1042 deselected, 2 rerun in 0.12s ================== 2024-08-07T18:00:35.6604022Z Got exit code 1 2024-08-07T18:00:35.6604340Z Retrying single test... 2024-08-07T18:00:35.6604962Z Test results will be stored in test-reports/python-pytest/test_torch/test_torch-af656628d03c1b88.xml 2024-08-07T18:00:35.6605817Z ============================= test session starts ============================== 2024-08-07T18:00:35.6606546Z platform linux -- Python 3.10.14, pytest-7.3.2, pluggy-1.5.0 -- /opt/conda/envs/py_3.10/bin/python 2024-08-07T18:00:35.6607178Z cachedir: .pytest_cache 2024-08-07T18:00:35.6607941Z hypothesis profile 'pytorch_ci' -> database=None, max_examples=50, derandomize=True, suppress_health_check=[HealthCheck.too_slow] 2024-08-07T18:00:35.6608765Z rootdir: /var/lib/jenkins/workspace 2024-08-07T18:00:35.6609163Z configfile: pytest.ini 2024-08-07T18:00:35.6609783Z plugins: hypothesis-5.35.1, cpp-2.3.0, flakefinder-1.1.0, rerunfailures-14.0, xdist-3.3.1, xdoctest-1.1.0 2024-08-07T18:00:35.6610705Z collecting ... collected 1043 items / 1042 deselected / 1 selected 2024-08-07T18:00:35.6611580Z stepcurrent: skipping 19 already run items. Running only test/test_torch.py::TestTorch::test_c10_feature_enabled 2024-08-07T18:00:35.6612308Z Running 1 items in this shard 2024-08-07T18:00:35.6612544Z 2024-08-07T18:00:35.6613074Z test_torch.py::TestTorch::test_c10_feature_enabled ('RERUN', {'yellow': True}) [0.0034s] [100%] 2024-08-07T18:00:35.6614100Z test_torch.py::TestTorch::test_c10_feature_enabled ('RERUN', {'yellow': True}) [0.0014s] [100%] 2024-08-07T18:00:35.6615087Z test_torch.py::TestTorch::test_c10_feature_enabled FAILED [0.0016s] [100%] 2024-08-07T18:00:35.6615577Z 2024-08-07T18:00:35.6615750Z ==================================== RERUNS ==================================== 2024-08-07T18:00:35.6616460Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:00:35.6617054Z Traceback (most recent call last): 2024-08-07T18:00:35.6617677Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:00:35.6618326Z with mocksetting(True): 2024-08-07T18:00:35.6618881Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:00:35.6619458Z return next(self.gen) 2024-08-07T18:00:35.6620006Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:00:35.6620603Z del os.environ[feature] 2024-08-07T18:00:35.6621096Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:00:35.6621659Z raise KeyError(key) from None 2024-08-07T18:00:35.6622875Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:00:35.6624559Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:00:35.6625111Z Traceback (most recent call last): 2024-08-07T18:00:35.6625747Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:00:35.6626397Z with mocksetting(True): 2024-08-07T18:00:35.6627018Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:00:35.6627623Z return next(self.gen) 2024-08-07T18:00:35.6628169Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:00:35.6628749Z del os.environ[feature] 2024-08-07T18:00:35.6629261Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:00:35.6629826Z raise KeyError(key) from None 2024-08-07T18:00:35.6631023Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:00:35.6632307Z =================================== FAILURES =================================== 2024-08-07T18:00:35.6633036Z ______________________ TestTorch.test_c10_feature_enabled ______________________ 2024-08-07T18:00:35.6633618Z Traceback (most recent call last): 2024-08-07T18:00:35.6634254Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10698, in test_c10_feature_enabled 2024-08-07T18:00:35.6634904Z with mocksetting(True): 2024-08-07T18:00:35.6635459Z File "/opt/conda/envs/py_3.10/lib/python3.10/contextlib.py", line 135, in __enter__ 2024-08-07T18:00:35.6636035Z return next(self.gen) 2024-08-07T18:00:35.6636584Z File "/var/lib/jenkins/workspace/test/test_torch.py", line 10694, in mocksetting 2024-08-07T18:00:35.6637184Z del os.environ[feature] 2024-08-07T18:00:35.6637692Z File "/opt/conda/envs/py_3.10/lib/python3.10/os.py", line 696, in __delitem__ 2024-08-07T18:00:35.6638265Z raise KeyError(key) from None 2024-08-07T18:00:35.6639475Z KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:00:35.6641136Z - generated xml file: /var/lib/jenkins/workspace/test/test-reports/python-pytest/test_torch/test_torch-af656628d03c1b88.xml - 2024-08-07T18:00:35.6642187Z =========================== short test summary info ============================ 2024-08-07T18:00:35.6644232Z FAILED [0.0016s] test_torch.py::TestTorch::test_c10_feature_enabled - KeyError: 'use_justknobs\n\nTo execute this test, run the following from the base repo dir:\n python test/test_torch.py -k TestTorch.test_c10_feature_enabled\n\nThis message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0' 2024-08-07T18:00:35.6646008Z !!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!! 2024-08-07T18:00:35.6646901Z ================= 1 failed, 1042 deselected, 2 rerun in 0.11s ================== 2024-08-07T18:00:35.6647543Z Got exit code 1 2024-08-07T18:00:35.6647881Z Stopping at first consistent failure 2024-08-07T18:00:35.6648557Z The following tests failed consistently: ['test/test_torch.py::TestTorch::test_c10_feature_enabled'] 2024-08-07T18:00:35.6649120Z 2024-08-07T18:00:35.6649514Z FINISHED PRINTING LOG FILE of test_torch 1/1 (test/test-reports/test_torch_1.1_1a26bcf1801a71c9_.log) 2024-08-07T18:00:35.6650056Z 2024-08-07T18:00:35.6650188Z test_torch 1/1 failed! 2024-08-07T18:00:35.6650561Z Emiting td_test_failure_stats_v2 2024-08-07T18:00:35.8193955Z Traceback (most recent call last): 2024-08-07T18:00:35.8194866Z File "/var/lib/jenkins/workspace/test/run_test.py", line 1879, in 2024-08-07T18:00:35.8195776Z main() 2024-08-07T18:00:35.8196200Z File "/var/lib/jenkins/workspace/test/run_test.py", line 1834, in main 2024-08-07T18:00:35.8196732Z run_tests( 2024-08-07T18:00:35.8197711Z File "/var/lib/jenkins/workspace/test/run_test.py", line 1686, in run_tests 2024-08-07T18:00:35.8198376Z raise RuntimeError(failure.message + keep_going_message) 2024-08-07T18:00:35.8198882Z RuntimeError: test_torch 1/1 failed! 2024-08-07T18:00:35.8199332Z 2024-08-07T18:00:35.8199717Z Tip: You can keep running tests even on failure by passing --keep-going to run_test.py. 2024-08-07T18:00:35.8200469Z If running on CI, add the 'keep-going' label to your PR and rerun your jobs. 2024-08-07T18:00:36.3756707Z 2024-08-07T18:00:36.3757423Z real 0m24.098s 2024-08-07T18:00:36.3757866Z user 0m33.516s 2024-08-07T18:00:36.3758380Z sys 0m9.812s 2024-08-07T18:00:36.3758672Z + cleanup_workspace 2024-08-07T18:00:36.3759521Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-07T18:00:36.3760689Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-07T18:00:36.3761566Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-07T18:00:36.3762383Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-07T18:00:36.3763076Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-07T18:00:36.3764032Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-07T18:00:36.3764856Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-07T18:00:37.0043406Z ##[error]Process completed with exit code 1. 2024-08-07T18:00:37.0125375Z Prepare all required actions 2024-08-07T18:00:37.0125876Z Getting action download info 2024-08-07T18:00:37.1482375Z ##[group]Run ./.github/actions/pytest-cache-upload 2024-08-07T18:00:37.1482865Z with: 2024-08-07T18:00:37.1483191Z cache_dir: .pytest_cache 2024-08-07T18:00:37.1483563Z shard: 1 2024-08-07T18:00:37.1483907Z sha: 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T18:00:37.1484359Z test_config: default 2024-08-07T18:00:37.1484789Z job_identifier: pull_linux-focal-cuda12.1-py3.10-gcc9 2024-08-07T18:00:37.1485289Z env: 2024-08-07T18:00:37.1485582Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:37.1486034Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:37.1486797Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:37.1487482Z ##[endgroup] 2024-08-07T18:00:37.1541609Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-07T18:00:37.1542134Z with: 2024-08-07T18:00:37.1542495Z shell: bash 2024-08-07T18:00:37.1542789Z timeout_minutes: 5 2024-08-07T18:00:37.1543114Z max_attempts: 5 2024-08-07T18:00:37.1543433Z retry_wait_seconds: 30 2024-08-07T18:00:37.1543853Z command: set -eu python3 -m pip install boto3==1.19.12 2024-08-07T18:00:37.1544358Z polling_interval_seconds: 1 2024-08-07T18:00:37.1544732Z warning_on_retry: true 2024-08-07T18:00:37.1545055Z continue_on_error: false 2024-08-07T18:00:37.1545408Z env: 2024-08-07T18:00:37.1545686Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:37.1546076Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:37.1546764Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:37.1547386Z ##[endgroup] 2024-08-07T18:00:37.5880581Z Defaulting to user installation because normal site-packages is not writeable 2024-08-07T18:00:39.0053162Z Collecting boto3==1.19.12 2024-08-07T18:00:39.0566411Z Downloading boto3-1.19.12-py3-none-any.whl (131 kB) 2024-08-07T18:00:40.5679176Z Collecting botocore<1.23.0,>=1.22.12 2024-08-07T18:00:40.5740675Z Downloading botocore-1.22.12-py3-none-any.whl (8.1 MB) 2024-08-07T18:00:40.7122409Z Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/lib/python3.9/site-packages (from boto3==1.19.12) (0.10.0) 2024-08-07T18:00:40.7582936Z Collecting s3transfer<0.6.0,>=0.5.0 2024-08-07T18:00:40.7641108Z Downloading s3transfer-0.5.2-py3-none-any.whl (79 kB) 2024-08-07T18:00:40.7756825Z Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python3.9/site-packages (from botocore<1.23.0,>=1.22.12->boto3==1.19.12) (1.25.10) 2024-08-07T18:00:40.7763566Z Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/lib/python3.9/site-packages (from botocore<1.23.0,>=1.22.12->boto3==1.19.12) (2.8.1) 2024-08-07T18:00:41.0927870Z Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.23.0,>=1.22.12->boto3==1.19.12) (1.15.0) 2024-08-07T18:00:41.2373958Z Installing collected packages: botocore, s3transfer, boto3 2024-08-07T18:00:41.9965547Z Successfully installed boto3-1.19.12 botocore-1.22.12 s3transfer-0.5.2 2024-08-07T18:00:42.2289305Z Command completed after 1 attempt(s). 2024-08-07T18:00:42.2353576Z ##[group]Run python3 .github/scripts/pytest_cache.py \ 2024-08-07T18:00:42.2354167Z python3 .github/scripts/pytest_cache.py \ 2024-08-07T18:00:42.2354629Z  --upload \ 2024-08-07T18:00:42.2355007Z  --cache_dir $GITHUB_WORKSPACE/$CACHE_DIR \ 2024-08-07T18:00:42.2355483Z  --pr_identifier $GITHUB_REF \ 2024-08-07T18:00:42.2355936Z  --job_identifier $JOB_IDENTIFIER \ 2024-08-07T18:00:42.2356349Z  --sha $SHA \ 2024-08-07T18:00:42.2356710Z  --test_config $TEST_CONFIG \ 2024-08-07T18:00:42.2357119Z  --shard $SHARD \ 2024-08-07T18:00:42.2357468Z  --repo $REPO \ 2024-08-07T18:00:42.2357836Z  --temp_dir $RUNNER_TEMP \ 2024-08-07T18:00:42.2368458Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:00:42.2368959Z env: 2024-08-07T18:00:42.2369254Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:42.2369698Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:42.2370376Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:42.2371028Z CACHE_DIR: .pytest_cache 2024-08-07T18:00:42.2371463Z JOB_IDENTIFIER: pull_linux-focal-cuda12.1-py3.10-gcc9 2024-08-07T18:00:42.2371945Z SHA: 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T18:00:42.2372371Z TEST_CONFIG: default 2024-08-07T18:00:42.2372799Z SHARD: 1 2024-08-07T18:00:42.2373080Z REPO: pytorch/pytorch 2024-08-07T18:00:42.2373416Z ##[endgroup] 2024-08-07T18:00:42.6907238Z PR identifier for `refs/pull/131248/merge` is `cbc34607c96079dfcdbe69d358ad18c0` 2024-08-07T18:00:42.6909698Z Uploading cache with args Namespace(upload=True, download=False, cache_dir='/home/ec2-user/actions-runner/_work/pytorch/pytorch/.pytest_cache', pr_identifier='refs/pull/131248/merge', job_identifier='pull_linux-focal-cuda12.1-py3.10-gcc9', sha='016588f53c6904b840aa56aa86f95460b4d9c996', test_config='default', shard='1', repo='pytorch/pytorch', temp_dir='/home/ec2-user/actions-runner/_work/_temp', bucket=None) 2024-08-07T18:00:42.6912008Z Zipping /home/ec2-user/actions-runner/_work/pytorch/pytorch/.pytest_cache 2024-08-07T18:00:42.6913423Z to /home/ec2-user/actions-runner/_work/_temp/zip-upload/pytest_cache/pytorch/pytorch/cbc34607c96079dfcdbe69d358ad18c0/pull_linux-focal-cuda12_1-py3_10-gcc9/016588f53c6904b840aa56aa86f95460b4d9c996/default/1 2024-08-07T18:00:42.6915595Z Uploading /home/ec2-user/actions-runner/_work/_temp/zip-upload/pytest_cache/pytorch/pytorch/cbc34607c96079dfcdbe69d358ad18c0/pull_linux-focal-cuda12_1-py3_10-gcc9/016588f53c6904b840aa56aa86f95460b4d9c996/default/1.zip 2024-08-07T18:00:42.6917644Z to s3://gha-artifacts/pytest_cache/pytorch/pytorch/cbc34607c96079dfcdbe69d358ad18c0/pull_linux-focal-cuda12_1-py3_10-gcc9/016588f53c6904b840aa56aa86f95460b4d9c996/default/1.zip 2024-08-07T18:00:42.7367235Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-07T18:00:42.7367749Z cat test/**/*_toprint.log || true 2024-08-07T18:00:42.7374720Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:00:42.7375438Z env: 2024-08-07T18:00:42.7375732Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:42.7376144Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:42.7376841Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:42.7377461Z ##[endgroup] 2024-08-07T18:00:42.7460608Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-07T18:00:42.7505153Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-07T18:00:42.7505645Z kill "$MONITOR_SCRIPT_PID" 2024-08-07T18:00:42.7512239Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:00:42.7512703Z env: 2024-08-07T18:00:42.7512994Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:42.7513425Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:42.7514100Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:42.7514738Z MONITOR_SCRIPT_PID: 53124 2024-08-07T18:00:42.7515120Z ##[endgroup] 2024-08-07T18:00:42.7699068Z Prepare all required actions 2024-08-07T18:00:42.7699824Z Getting action download info 2024-08-07T18:00:42.9051400Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-07T18:00:43.1439144Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-07T18:00:43.1439597Z with: 2024-08-07T18:00:43.1440083Z file-suffix: test-default-1-5-amz2023.linux.4xlarge.nvidia.gpu_28476181947 2024-08-07T18:00:43.1440681Z s3-bucket: gha-artifacts 2024-08-07T18:00:43.1441011Z env: 2024-08-07T18:00:43.1441300Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:43.1441727Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:43.1442408Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:43.1443030Z ##[endgroup] 2024-08-07T18:00:43.1479758Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-07T18:00:43.1480386Z # Remove any previous test jsons if they exist 2024-08-07T18:00:43.1480895Z rm -f test-jsons-*.zip 2024-08-07T18:00:43.1481378Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-07T18:00:43.1488362Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:00:43.1488838Z env: 2024-08-07T18:00:43.1489124Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:43.1489535Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:43.1490223Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:43.1491029Z FILE_SUFFIX: test-default-1-5-amz2023.linux.4xlarge.nvidia.gpu_28476181947 2024-08-07T18:00:43.1491580Z ##[endgroup] 2024-08-07T18:00:43.1820916Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-07T18:00:43.1855558Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-07T18:00:43.1856263Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-07T18:00:43.1864366Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-07T18:00:43.1868356Z adding: test/test-reports/td_exclusions-7a505e99506d44491da8.json (deflated 81%) 2024-08-07T18:00:43.1874460Z adding: test/.pytorch-slow-tests.json (deflated 81%) 2024-08-07T18:00:43.1890075Z adding: test/.pytorch-disabled-tests.json (deflated 89%) 2024-08-07T18:00:43.1931361Z ##[group]Run # Remove any previous test reports if they exist 2024-08-07T18:00:43.1931944Z # Remove any previous test reports if they exist 2024-08-07T18:00:43.1932437Z rm -f test-reports-*.zip 2024-08-07T18:00:43.1932985Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-07T18:00:43.1939892Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:00:43.1940377Z env: 2024-08-07T18:00:43.1940669Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:43.1941081Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:43.1941786Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:43.1942849Z FILE_SUFFIX: test-default-1-5-amz2023.linux.4xlarge.nvidia.gpu_28476181947 2024-08-07T18:00:43.1943405Z ##[endgroup] 2024-08-07T18:00:43.2181169Z adding: test/test-reports/python-pytest/test_torch/test_torch-0f3c0fa08db3a8d6.xml (deflated 90%) 2024-08-07T18:00:43.2182181Z adding: test/test-reports/python-pytest/test_torch/test_torch-5f16e6a2d7696715.xml (deflated 82%) 2024-08-07T18:00:43.2183127Z adding: test/test-reports/python-pytest/test_torch/test_torch-af656628d03c1b88.xml (deflated 82%) 2024-08-07T18:00:43.2223120Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-07T18:00:43.2223970Z # Remove any previous usage logs if they exist 2024-08-07T18:00:43.2224453Z rm -f logs-*.zip 2024-08-07T18:00:43.2225047Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-07T18:00:43.2225993Z # so check to see if the file exists first 2024-08-07T18:00:43.2226502Z if [ -f 'usage_log.txt' ]; then 2024-08-07T18:00:43.2226971Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-07T18:00:43.2227406Z fi 2024-08-07T18:00:43.2227746Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-07T18:00:43.2228249Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-07T18:00:43.2228678Z fi 2024-08-07T18:00:43.2235325Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:00:43.2235790Z env: 2024-08-07T18:00:43.2236080Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:43.2236503Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:43.2237195Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:43.2237991Z FILE_SUFFIX: test-default-1-5-amz2023.linux.4xlarge.nvidia.gpu_28476181947 2024-08-07T18:00:43.2238558Z ##[endgroup] 2024-08-07T18:00:43.2289833Z adding: usage_log.txt (deflated 92%) 2024-08-07T18:00:43.2554668Z adding: test/test-reports/test_torch_1.1_1a26bcf1801a71c9_.log (deflated 91%) 2024-08-07T18:00:43.2617884Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-07T18:00:43.2618554Z # Remove any previous debugging artifacts if they exist 2024-08-07T18:00:43.2619065Z rm -f debug-*.zip 2024-08-07T18:00:43.2619418Z if [ -d 'test/debug' ]; then 2024-08-07T18:00:43.2619883Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-07T18:00:43.2620325Z fi 2024-08-07T18:00:43.2627050Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:00:43.2627536Z env: 2024-08-07T18:00:43.2627831Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:43.2628246Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:43.2628938Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:43.2629764Z FILE_SUFFIX: test-default-1-5-amz2023.linux.4xlarge.nvidia.gpu_28476181947 2024-08-07T18:00:43.2630329Z ##[endgroup] 2024-08-07T18:00:43.2733244Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T18:00:43.2733674Z with: 2024-08-07T18:00:43.2733950Z s3-bucket: gha-artifacts 2024-08-07T18:00:43.2734362Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T18:00:43.2734810Z retention-days: 14 2024-08-07T18:00:43.2735127Z if-no-files-found: warn 2024-08-07T18:00:43.2735481Z path: test-jsons-*.zip 2024-08-07T18:00:43.2735816Z name: artifact 2024-08-07T18:00:43.2736102Z region: us-east-1 2024-08-07T18:00:43.2736413Z env: 2024-08-07T18:00:43.2736695Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:43.2737095Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:43.2737784Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:43.2738394Z ##[endgroup] 2024-08-07T18:00:43.7433679Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-07T18:00:43.7434955Z With the provided path, there will be 1 file uploaded 2024-08-07T18:00:43.7435538Z Uploading to s3 prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T18:00:43.8959445Z Starting upload of test-jsons-test-default-1-5-amz2023.linux.4xlarge.nvidia.gpu_28476181947.zip 2024-08-07T18:00:44.0607957Z Finished upload of test-jsons-test-default-1-5-amz2023.linux.4xlarge.nvidia.gpu_28476181947.zip 2024-08-07T18:00:44.0794787Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T18:00:44.0795225Z with: 2024-08-07T18:00:44.0795502Z s3-bucket: gha-artifacts 2024-08-07T18:00:44.0795930Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T18:00:44.0796379Z retention-days: 14 2024-08-07T18:00:44.0796700Z if-no-files-found: error 2024-08-07T18:00:44.0797570Z path: test-reports-*.zip 2024-08-07T18:00:44.0797928Z name: artifact 2024-08-07T18:00:44.0798214Z region: us-east-1 2024-08-07T18:00:44.0798518Z env: 2024-08-07T18:00:44.0798800Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:44.0799451Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:44.0800172Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:44.0800793Z ##[endgroup] 2024-08-07T18:00:44.5341839Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-07T18:00:44.5342646Z With the provided path, there will be 1 file uploaded 2024-08-07T18:00:44.5343225Z Uploading to s3 prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T18:00:44.5396202Z Starting upload of test-reports-test-default-1-5-amz2023.linux.4xlarge.nvidia.gpu_28476181947.zip 2024-08-07T18:00:44.6448858Z Finished upload of test-reports-test-default-1-5-amz2023.linux.4xlarge.nvidia.gpu_28476181947.zip 2024-08-07T18:00:44.6640967Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T18:00:44.6641412Z with: 2024-08-07T18:00:44.6641692Z s3-bucket: gha-artifacts 2024-08-07T18:00:44.6642105Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T18:00:44.6642553Z retention-days: 14 2024-08-07T18:00:44.6642911Z if-no-files-found: ignore 2024-08-07T18:00:44.6643267Z path: logs-*.zip 2024-08-07T18:00:44.6643576Z name: artifact 2024-08-07T18:00:44.6643861Z region: us-east-1 2024-08-07T18:00:44.6644162Z env: 2024-08-07T18:00:44.6644446Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:44.6644852Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:44.6645547Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:44.6646166Z ##[endgroup] 2024-08-07T18:00:45.1187555Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-07T18:00:45.1188123Z With the provided path, there will be 1 file uploaded 2024-08-07T18:00:45.1188698Z Uploading to s3 prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T18:00:45.1241885Z Starting upload of logs-test-default-1-5-amz2023.linux.4xlarge.nvidia.gpu_28476181947.zip 2024-08-07T18:00:45.2674041Z Finished upload of logs-test-default-1-5-amz2023.linux.4xlarge.nvidia.gpu_28476181947.zip 2024-08-07T18:00:45.2868573Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T18:00:45.2869015Z with: 2024-08-07T18:00:45.2869295Z s3-bucket: gha-artifacts 2024-08-07T18:00:45.2869709Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T18:00:45.2870155Z retention-days: 14 2024-08-07T18:00:45.2870477Z if-no-files-found: ignore 2024-08-07T18:00:45.2870838Z path: debug-*.zip 2024-08-07T18:00:45.2871147Z name: artifact 2024-08-07T18:00:45.2871431Z region: us-east-1 2024-08-07T18:00:45.2871739Z env: 2024-08-07T18:00:45.2872005Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:45.2872429Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:45.2873123Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:45.2873738Z ##[endgroup] 2024-08-07T18:00:45.7346887Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-07T18:00:45.7542722Z ##[group]Run # shellcheck disable=SC2156 2024-08-07T18:00:45.7543428Z # shellcheck disable=SC2156 2024-08-07T18:00:45.7544160Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-07T18:00:45.7553411Z shell: /usr/bin/bash -e {0} 2024-08-07T18:00:45.7553777Z env: 2024-08-07T18:00:45.7554064Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:45.7554477Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:45.7555183Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:45.7555812Z ##[endgroup] 2024-08-07T18:00:46.0366021Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T18:00:46.0366489Z with: 2024-08-07T18:00:46.0366916Z name: coredumps-default-1-5-amz2023.linux.4xlarge.nvidia.gpu 2024-08-07T18:00:46.0367438Z retention-days: 14 2024-08-07T18:00:46.0367766Z if-no-files-found: ignore 2024-08-07T18:00:46.0368127Z path: ./**/core.[1-9]* 2024-08-07T18:00:46.0368468Z s3-bucket: gha-artifacts 2024-08-07T18:00:46.0368823Z region: us-east-1 2024-08-07T18:00:46.0369132Z env: 2024-08-07T18:00:46.0369411Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:46.0369814Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:46.0370507Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:46.0371122Z ##[endgroup] 2024-08-07T18:00:57.8885812Z No files were found with the provided path: ./**/core.[1-9]*. No artifacts will be uploaded. 2024-08-07T18:00:57.9209939Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-07T18:00:57.9210575Z with: 2024-08-07T18:00:57.9210837Z env: 2024-08-07T18:00:57.9211119Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:57.9211550Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:57.9212232Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:57.9212849Z ##[endgroup] 2024-08-07T18:00:57.9238992Z ##[group]Run set -eou pipefail 2024-08-07T18:00:57.9239428Z set -eou pipefail 2024-08-07T18:00:57.9239763Z  2024-08-07T18:00:57.9240228Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-07T18:00:57.9240800Z for _ in $(seq 1440); do 2024-08-07T18:00:57.9241209Z  # Break if no ssh session exists anymore 2024-08-07T18:00:57.9241655Z  if [ "$(who)" = "" ]; then 2024-08-07T18:00:57.9242036Z  break 2024-08-07T18:00:57.9242383Z  fi 2024-08-07T18:00:57.9242674Z  echo "." 2024-08-07T18:00:57.9243009Z  sleep 5 2024-08-07T18:00:57.9243319Z done 2024-08-07T18:00:57.9250494Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:00:57.9250980Z env: 2024-08-07T18:00:57.9251269Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:57.9251686Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:57.9252386Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:57.9253023Z ##[endgroup] 2024-08-07T18:00:57.9281692Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-07T18:00:57.9374244Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T18:00:57.9374938Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T18:00:57.9375453Z # shellcheck disable=SC2046 2024-08-07T18:00:57.9375895Z docker stop $(docker ps -q) || true 2024-08-07T18:00:57.9376339Z # Prune all of the docker images 2024-08-07T18:00:57.9376738Z docker system prune -af 2024-08-07T18:00:57.9382967Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:00:57.9383438Z env: 2024-08-07T18:00:57.9383709Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:00:57.9384135Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:00:57.9384825Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:57.9385662Z ##[endgroup] 2024-08-07T18:00:58.6141441Z ee14918686c2 2024-08-07T18:00:59.1190476Z Deleted Containers: 2024-08-07T18:00:59.1191303Z ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:00:59.1192008Z 2024-08-07T18:01:04.3066463Z Deleted Images: 2024-08-07T18:01:04.3067476Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T18:01:04.3069207Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-focal-cuda12.1-cudnn9-py3-gcc9@sha256:00f47b036f588ca5ef8866f8635fabba5a95cdf9ff1adae7d2a674ef1d4076e9 2024-08-07T18:01:04.3070494Z deleted: sha256:6ec36276acd88c9be8b44d856744037d399b35f4bb1703e637c27ae2b254c901 2024-08-07T18:01:04.3071305Z deleted: sha256:6fbc5fe2ebb0dc33846ab2ade7c5296a0a521e16f71c3b15b8a0c40a8fce5ed3 2024-08-07T18:01:04.3072108Z deleted: sha256:31fb5fadd0be2cc6e0d198fbc00e4f3c25925bc9c9ce79be23f02aeb56f6a55a 2024-08-07T18:01:04.3072939Z deleted: sha256:d1d89c6e648d792c08fdaae4fdad1273f93571a1b8d03c76f38f9e2be7cfe7f1 2024-08-07T18:01:04.3073747Z deleted: sha256:0f1adf9d1a1d4eeb62caa063c1090ea2f50246ed031d8d6627df2f5fe5963067 2024-08-07T18:01:04.3074528Z deleted: sha256:771e69c2306df070c5a944ea3043f81ce5890ce90de66fb6f16edfaf09912b35 2024-08-07T18:01:04.3075309Z deleted: sha256:288aa01dcd110ad9e2e3f48756d987529733a41225b6e8d4898f7386666beeae 2024-08-07T18:01:04.3076405Z deleted: sha256:d9d7c6c9bef79d8c5ce30d89b81820f3664c04fe17ae396b2a121e96fcfdeccf 2024-08-07T18:01:04.3077228Z deleted: sha256:1efa95271d86fd98beb626e63f4c67732daca36bf6e25d864735e43ef4c708f4 2024-08-07T18:01:04.3078020Z deleted: sha256:b1699f7cf1967593f94664569bb49ba431deaac0d1ceaf0d0584046a78ca3be0 2024-08-07T18:01:04.3078814Z deleted: sha256:b70eb1cb717d1c2ce90d5fa7f48e1af32d45f3735b33e1f944762686d8459aa6 2024-08-07T18:01:04.3079581Z deleted: sha256:6460a71f50d3c4bf7773fa072665e881e2a7e0e4e0bd45302e6a73c35ec03898 2024-08-07T18:01:04.3080366Z deleted: sha256:1c1787b4980844f6bfea1fe2386125ac286e9dfd80e253164a7f82b90f9c37bc 2024-08-07T18:01:04.3081155Z deleted: sha256:758fb95d0e1c4b4c78ee59fc4d66c06acfbc3a3ffa870e94200529367b8998e8 2024-08-07T18:01:04.3081930Z deleted: sha256:99beb19b8911fc7c49d248960a97d8ba4bbcbe8afc9fe3142e5af37f08c1c821 2024-08-07T18:01:04.3082887Z deleted: sha256:55596f5748c7f4d8e4d9fea0d1a6cda4627ccc442e652c8578169efe5992a382 2024-08-07T18:01:04.3083906Z deleted: sha256:c279b10b40b22a2fba8aeaad85e7705ffb1afa28e000323b35cc947649b7637d 2024-08-07T18:01:04.3084706Z deleted: sha256:8c882a49501c1d973eacd2ed8da81bce818403153d3fb4cdc84baf14307f9517 2024-08-07T18:01:04.3085472Z deleted: sha256:9fdbb157fcea7486d127e106ff114401c3102b9e9ed879a59247a296b3a908af 2024-08-07T18:01:04.3086245Z deleted: sha256:25b821c2e5d4106b0214b3aa4c88265bda5f90f045f012af912f0a3d2979f919 2024-08-07T18:01:04.3087028Z deleted: sha256:fada342ffd1729d0c38b7afe6767d6840580548d8dac7c62ddf61e24803ad66e 2024-08-07T18:01:04.3087814Z deleted: sha256:14d55c394ec592bb1417a21f696089e0549403bff381cda63d3e92ec80bca298 2024-08-07T18:01:04.3088604Z deleted: sha256:1032dedb70a12fc3c1cae4b10cdd7a07a9d20a64736533d640dd133d46f1ebfe 2024-08-07T18:01:04.3089377Z deleted: sha256:c0f5baad08f078d4210c254240524d9063c5e797564ada8a4bf39270e0fc1300 2024-08-07T18:01:04.3090125Z deleted: sha256:1a8bc02cd2c8f897e1dfd14920703ea75d172660c85026068b39136a1a25db51 2024-08-07T18:01:04.3090891Z deleted: sha256:b2221232f25b8d3a916bf8f74248542af7830e6198814ffbdbab43484bcb700b 2024-08-07T18:01:04.3091674Z deleted: sha256:14dc7064caf62d3d51b65e5175f9ae0e31af4fbed83413736f80d881f0fd742e 2024-08-07T18:01:04.3092438Z deleted: sha256:162ef080ee355a240e7e8fd6761113ad61586e1c883207ba4290c90abf208b54 2024-08-07T18:01:04.3093217Z deleted: sha256:56e3e897838040eb3fb86255ef1be2e3be6705d2e4a1ead67bbd400350ff6d13 2024-08-07T18:01:04.3093995Z deleted: sha256:a4f1392c3713fac843566ae891b9e41c28824e377af84190d33a4132cd4b268d 2024-08-07T18:01:04.3094769Z deleted: sha256:d1a8ab1bb8f3238d77db318f5161058057f84f7357afd2932e1d7edded9a2efb 2024-08-07T18:01:04.3095780Z deleted: sha256:7cab46d23f9fd91e62280b25d274d6c8a03be6d66030adf598944913d280c54f 2024-08-07T18:01:04.3096564Z deleted: sha256:764da7f97ae4318f5157e363c03c86c016b8d1b6d2c4758d203fa8f194753f57 2024-08-07T18:01:04.3097913Z deleted: sha256:373ab2f02049fb6ccab3f886c53934457751a14e5cb68b5e144b04bb7afeed87 2024-08-07T18:01:04.3098690Z deleted: sha256:e46d90cf4cc84661ac93ae9bcfbc675c0fab0582328e4595d075e4f259c4389a 2024-08-07T18:01:04.3099476Z deleted: sha256:1579b987297253e32b13fe0d160a95c567910203783be2efcc3a76650255e658 2024-08-07T18:01:04.3100249Z deleted: sha256:f8542342382d7443bc8be95e90ac743f70abdafcd9987e3785f58ad50509a145 2024-08-07T18:01:04.3101007Z deleted: sha256:57432242bf8090a0c1238a5f004be4da246f4488a2193be182e5cd82237509e4 2024-08-07T18:01:04.3101786Z deleted: sha256:e3c369d15f59fb5f34178d8dcb6bb4a6ff1d527e9baf3182444c24c55600bdf4 2024-08-07T18:01:04.3102568Z deleted: sha256:4215189c6756d59c8a217cc09447c312318d07c0b0d5c9bfb7e4bfb942f05cae 2024-08-07T18:01:04.3103370Z deleted: sha256:b787e6329788262404e4b8293a110727833b63e2e287ccd569a21cb3fb450388 2024-08-07T18:01:04.3104180Z deleted: sha256:69bb5dca4b9fded8d0c731b73bcbed0c3e9ce170c89e79796956a444b0d58c4c 2024-08-07T18:01:04.3104956Z deleted: sha256:8c945f56e57fb4fbfa3f2c74d6109a47db8df7644c4988d0c5791e4214bf30c9 2024-08-07T18:01:04.3105745Z deleted: sha256:fdd079e0e07e11d86ffc744f1036793f3db2aea378660f7489fbf50002439620 2024-08-07T18:01:04.3106700Z deleted: sha256:6a743a461982e1a1b73f29cd187e354f2119f3bd985e1da6ca0b802134cb91ba 2024-08-07T18:01:04.3107518Z deleted: sha256:7b3cbce1a91c69d6a499ef524311c14537930fc04f0ef4b1d73030216ac4a568 2024-08-07T18:01:04.3108331Z deleted: sha256:d7120733e426141c8a9e8f2c3596b12055cd5b1956d141ad640365cb11628a00 2024-08-07T18:01:04.3109085Z deleted: sha256:841ae666d028560ea37e6314ae5f80d72fb071fe7592b577dbbd8156c08cdda2 2024-08-07T18:01:04.3109867Z deleted: sha256:9d85a8c4ea0c0bc631a4d3e5472e8ed8e2312a4dc5a38f6e9890edeb37183186 2024-08-07T18:01:04.3110662Z deleted: sha256:606f6418680b5c53158271dfaca16c668f3b9dd25bb2b738f5621b6a56d08cdb 2024-08-07T18:01:04.3111502Z deleted: sha256:5ca4df537c3efc44dfde3bebb377ed490bc4d1cd5d6e5fa2fd9549dcb456c471 2024-08-07T18:01:04.3112285Z deleted: sha256:c6f3f6b7969e62b95fe926a1963e6628cb2f1b5388f0b12115564855e59423f1 2024-08-07T18:01:04.3113069Z deleted: sha256:ce178391f5f25e4ebe6ba8e84c84831fb31b8a9d4e82dbc270a73ddeccbf1c2c 2024-08-07T18:01:04.3113872Z deleted: sha256:d154e0a00edb13898d564f38c090096dfeb7a90beb0bd8addd57eca33f52151e 2024-08-07T18:01:04.3114644Z deleted: sha256:7226a8914ccef500b1d0327370cf9e3a66fbcb0b59a5f3b018981372152fc645 2024-08-07T18:01:04.3115425Z deleted: sha256:ce5fab2081603cd22d9f253d98828ddd60c0f8c44c5413d0615f4264f1cd6a7b 2024-08-07T18:01:04.3116214Z deleted: sha256:b3bc21b2cb6a9ca2b1dbbc412812581089c8ace8cc6b8d2b2767f0b3cbe8b99c 2024-08-07T18:01:04.3116984Z deleted: sha256:53e8f21eb1afc2c8550bef59d67810a5f79671d1d3d3924577f810321e66886b 2024-08-07T18:01:04.3117771Z deleted: sha256:f9770918a5cc46e8f6aa13da6903754fa89a6bcc9f6e11df1d61f9340e452cc8 2024-08-07T18:01:04.3118574Z deleted: sha256:bce2de65cb97e6a64f59fe7fc78644a2f3d62cf7769073cb59dcbb52009fc5d0 2024-08-07T18:01:04.3119346Z deleted: sha256:568d161c6888a8b0b09d7feb561f790a31d47c7ff7ca1c89c0ff4025f5b02e3a 2024-08-07T18:01:04.3120124Z deleted: sha256:2da22d4dfb814e7a3f4444f60464109149d35bc3e078e197f600bd9f6cbb9f6b 2024-08-07T18:01:04.3120899Z deleted: sha256:4af2daf4a1277af00005823106d88d704d5d0499eef553f01875c6f94380b2e3 2024-08-07T18:01:04.3121664Z deleted: sha256:e52d382a6cbfc1bb5122186d4b1401de79de002a9cc323ba5047dbf266f4d3a3 2024-08-07T18:01:04.3122446Z deleted: sha256:d10fe93dc5314d752e8882c5877457a6ae733a93a1e246cd6f79301de9325e9c 2024-08-07T18:01:04.3123223Z deleted: sha256:58d5ecb60ded3f99102b25d909a729386d92a44bf68b77ed3b49bf27d978e26a 2024-08-07T18:01:04.3124004Z deleted: sha256:dd522c9c6fdd98e9e4584578a66d9122b36ee8f856bba2140fcaaf908c7a68e8 2024-08-07T18:01:04.3124762Z deleted: sha256:3584fb50d1cbc80b57118953f1dc36ecb092b5986d56211f711b9161f403d66c 2024-08-07T18:01:04.3125695Z deleted: sha256:35a033279d5c9bd6b862bfa9331d8fcbf98bbaf778a778fabc882384db9204f8 2024-08-07T18:01:04.3126476Z deleted: sha256:ab9b00c496073d62e69e032178858d3d6d4c4bab9e87065d3785c6da57351d00 2024-08-07T18:01:04.3127226Z deleted: sha256:7ed947299e109c2459de6e240d86f049c30f93f2380659ce441d8737b8cd065f 2024-08-07T18:01:04.3128004Z deleted: sha256:410d5a7f7a9a1cb4551c106b9cc728a9dcff598f9be231a351ce6a0a33f81e64 2024-08-07T18:01:04.3128794Z deleted: sha256:5f0021bb56efa14bb93978c01513e2a1187ba30e69bdb0546d0ef39b30873f88 2024-08-07T18:01:04.3129560Z deleted: sha256:c1601aa97eb84151c14da3aeea351201bd99144d36d66b397f6555d80245d86d 2024-08-07T18:01:04.3130354Z deleted: sha256:72af05a89be22accdc1ca5d66dcbbb33993a9ef5997f849df1d4ba4c48049f25 2024-08-07T18:01:04.3131152Z deleted: sha256:38b90c9663dcaa2bc57a4dd3008298e7ea93e9535fa42312b4dc4246e7491af9 2024-08-07T18:01:04.3131934Z deleted: sha256:bda61b6cefb3ec8eeb74fef1ca1c7f9a5845fe5e8f07b8123323e897425d5c29 2024-08-07T18:01:04.3132743Z deleted: sha256:5a18e1aa877074529a84cbddf19f8d5403787823378ceae6b72fb62f78d43037 2024-08-07T18:01:04.3133522Z deleted: sha256:6c3e7df31590f02f10cb71fc4eb27653e9b428df2e6e5421a455b062bd2e39f9 2024-08-07T18:01:04.3133983Z 2024-08-07T18:01:04.3134153Z Total reclaimed space: 25.14GB 2024-08-07T18:01:04.3175610Z ##[group]Run set +e 2024-08-07T18:01:04.3176061Z set +e 2024-08-07T18:01:04.3176371Z set -x 2024-08-07T18:01:04.3176648Z  2024-08-07T18:01:04.3176933Z nvidia-smi 2024-08-07T18:01:04.3177502Z # NB: Surprisingly, nvidia-smi command returns successfully with return code 0 even in 2024-08-07T18:01:04.3178335Z # the case where the driver has already crashed as it still can get the driver version 2024-08-07T18:01:04.3179145Z # and some basic information like the bus ID. However, the rest of the information 2024-08-07T18:01:04.3179782Z # would be missing (ERR!), for example: 2024-08-07T18:01:04.3180182Z # 2024-08-07T18:01:04.3180609Z # +-----------------------------------------------------------------------------+ 2024-08-07T18:01:04.3181272Z # | NVIDIA-SMI 525.89.02 Driver Version: 525.89.02 CUDA Version: 12.0 | 2024-08-07T18:01:04.3181972Z # |-------------------------------+----------------------+----------------------+ 2024-08-07T18:01:04.3182640Z # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-07T18:01:04.3183341Z # | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | 2024-08-07T18:01:04.3184187Z # | | | MIG M. | 2024-08-07T18:01:04.3184800Z # |===============================+======================+======================| 2024-08-07T18:01:04.3185314Z # | 0 ERR! Off | 00000000:00:1E.0 Off | ERR! | 2024-08-07T18:01:04.3185923Z # |ERR! ERR! ERR! ERR! / ERR! | 4184MiB / 23028MiB | ERR! Default | 2024-08-07T18:01:04.3186487Z # | | | ERR! | 2024-08-07T18:01:04.3187038Z # +-------------------------------+----------------------+----------------------+ 2024-08-07T18:01:04.3187507Z # 2024-08-07T18:01:04.3187901Z # +-----------------------------------------------------------------------------+ 2024-08-07T18:01:04.3188487Z # | Processes: | 2024-08-07T18:01:04.3189074Z # | GPU GI CI PID Type Process name GPU Memory | 2024-08-07T18:01:04.3189646Z # | ID ID Usage | 2024-08-07T18:01:04.3190137Z # |=============================================================================| 2024-08-07T18:01:04.3190679Z # +-----------------------------------------------------------------------------+ 2024-08-07T18:01:04.3191313Z # 2024-08-07T18:01:04.3191807Z # This should be reported as a failure instead as it will guarantee to fail when 2024-08-07T18:01:04.3192439Z # Docker tries to run with --gpus all 2024-08-07T18:01:04.3192839Z # 2024-08-07T18:01:04.3193305Z # So, the correct check here is to query one of the missing piece of info like 2024-08-07T18:01:04.3193954Z # GPU name, so that the command can fail accordingly 2024-08-07T18:01:04.3194551Z nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-07T18:01:04.3195090Z NVIDIA_SMI_STATUS=$? 2024-08-07T18:01:04.3195442Z  2024-08-07T18:01:04.3195977Z # These are acceptable return code from nvidia-smi as copied from setup-nvidia GitHub action 2024-08-07T18:01:04.3196775Z if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then 2024-08-07T18:01:04.3198163Z  echo "NVIDIA driver installation has failed, shutting down the runner..." 2024-08-07T18:01:04.3198793Z  .github/scripts/stop_runner_service.sh 2024-08-07T18:01:04.3199205Z fi 2024-08-07T18:01:04.3199492Z  2024-08-07T18:01:04.3200195Z # For runner with multiple GPUs, we also want to confirm that the number of GPUs are the 2024-08-07T18:01:04.3200975Z # power of 2, i.e. 1, 2, 4, or 8. This is to avoid flaky test issue when one GPU fails 2024-08-07T18:01:04.3201640Z # https://github.com/pytorch/test-infra/issues/4000 2024-08-07T18:01:04.3202184Z GPU_COUNT=$(nvidia-smi --list-gpus | wc -l) 2024-08-07T18:01:04.3202621Z NVIDIA_SMI_STATUS=$? 2024-08-07T18:01:04.3202972Z  2024-08-07T18:01:04.3203513Z # These are acceptable return code from nvidia-smi as copied from setup-nvidia GitHub action 2024-08-07T18:01:04.3204282Z if [ "$NVIDIA_SMI_STATUS" -ne 0 ] && [ "$NVIDIA_SMI_STATUS" -ne 14 ]; then 2024-08-07T18:01:04.3205009Z  echo "NVIDIA driver installation has failed, shutting down the runner..." 2024-08-07T18:01:04.3205634Z  .github/scripts/stop_runner_service.sh 2024-08-07T18:01:04.3206060Z fi 2024-08-07T18:01:04.3206330Z  2024-08-07T18:01:04.3206658Z # Check the GPU count to be a power of 2 2024-08-07T18:01:04.3207351Z if [ "$GPU_COUNT" -le 8 ] && [ "$GPU_COUNT" -ne 1 ] && [ "$GPU_COUNT" -ne 2 ] && [ "$GPU_COUNT" -ne 4 ] && [ "$GPU_COUNT" -ne 8 ]; then 2024-08-07T18:01:04.3208244Z  echo "NVIDIA driver detects $GPU_COUNT GPUs. The runner has a broken GPU, shutting it down..." 2024-08-07T18:01:04.3208946Z  .github/scripts/stop_runner_service.sh 2024-08-07T18:01:04.3209370Z fi 2024-08-07T18:01:04.3218640Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:01:04.3219122Z env: 2024-08-07T18:01:04.3219412Z GIT_DEFAULT_BRANCH: main 2024-08-07T18:01:04.3219838Z GPU_FLAG: --gpus all -e NVIDIA_DRIVER_CAPABILITIES=all 2024-08-07T18:01:04.3220531Z DOCKER_CONTAINER_ID: ee14918686c22313c4c7ccd9c11663ebfed8e0dff09ba4acc18b623a3089b84d 2024-08-07T18:01:04.3221278Z RUNNER_WORKSPACE: /home/ec2-user/actions-runner/_work/pytorch 2024-08-07T18:01:04.3221767Z ##[endgroup] 2024-08-07T18:01:04.3251413Z + nvidia-smi 2024-08-07T18:01:04.3470865Z Wed Aug 7 18:01:04 2024 2024-08-07T18:01:04.3471353Z +-----------------------------------------------------------------------------------------+ 2024-08-07T18:01:04.3471997Z | NVIDIA-SMI 550.54.15 Driver Version: 550.54.15 CUDA Version: 12.4 | 2024-08-07T18:01:04.3472659Z |-----------------------------------------+------------------------+----------------------+ 2024-08-07T18:01:04.3473329Z | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | 2024-08-07T18:01:04.3474032Z | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | 2024-08-07T18:01:04.3474932Z | | | MIG M. | 2024-08-07T18:01:04.3475396Z |=========================================+========================+======================| 2024-08-07T18:01:04.3560822Z | 0 Tesla M60 On | 00000000:00:1E.0 Off | 0 | 2024-08-07T18:01:04.3561466Z | N/A 32C P8 16W / 150W | 0MiB / 7680MiB | 0% Default | 2024-08-07T18:01:04.3561977Z | | | N/A | 2024-08-07T18:01:04.3562526Z +-----------------------------------------+------------------------+----------------------+ 2024-08-07T18:01:04.3563045Z 2024-08-07T18:01:04.3563565Z +-----------------------------------------------------------------------------------------+ 2024-08-07T18:01:04.3564148Z | Processes: | 2024-08-07T18:01:04.3565101Z | GPU GI CI PID Type Process name GPU Memory | 2024-08-07T18:01:04.3565954Z | ID ID Usage | 2024-08-07T18:01:04.3566477Z |=========================================================================================| 2024-08-07T18:01:04.3567042Z | No running processes found | 2024-08-07T18:01:04.3567677Z +-----------------------------------------------------------------------------------------+ 2024-08-07T18:01:04.4175508Z + nvidia-smi --query-gpu=gpu_name --format=csv,noheader --id=0 2024-08-07T18:01:04.4367996Z Tesla M60 2024-08-07T18:01:04.4454719Z + NVIDIA_SMI_STATUS=0 2024-08-07T18:01:04.4461087Z + '[' 0 -ne 0 ']' 2024-08-07T18:01:04.4461422Z ++ nvidia-smi --list-gpus 2024-08-07T18:01:04.4462285Z ++ wc -l 2024-08-07T18:01:04.4726417Z + GPU_COUNT=1 2024-08-07T18:01:04.4726726Z + NVIDIA_SMI_STATUS=0 2024-08-07T18:01:04.4727067Z + '[' 0 -ne 0 ']' 2024-08-07T18:01:04.4727347Z + '[' 1 -le 8 ']' 2024-08-07T18:01:04.4727644Z + '[' 1 -ne 1 ']' 2024-08-07T18:01:04.4797353Z Post job cleanup. 2024-08-07T18:01:04.4857443Z Post job cleanup. 2024-08-07T18:01:04.6042706Z [command]/usr/bin/git version 2024-08-07T18:01:04.6102261Z git version 2.40.1 2024-08-07T18:01:04.6163120Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/02b470c5-c51c-4a8b-8da4-1ecab28fd71a' before making global git config changes 2024-08-07T18:01:04.6164449Z Adding repository directory to the temporary git global config as a safe directory 2024-08-07T18:01:04.6170851Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T18:01:04.6218407Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-07T18:01:04.6261113Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-07T18:01:04.6617228Z Entering 'android/libs/fbjni' 2024-08-07T18:01:04.6679144Z Entering 'third_party/FP16' 2024-08-07T18:01:04.6743589Z Entering 'third_party/FXdiv' 2024-08-07T18:01:04.6806203Z Entering 'third_party/NNPACK' 2024-08-07T18:01:04.6868568Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T18:01:04.6933415Z Entering 'third_party/XNNPACK' 2024-08-07T18:01:04.7015676Z Entering 'third_party/benchmark' 2024-08-07T18:01:04.7080327Z Entering 'third_party/cpp-httplib' 2024-08-07T18:01:04.7142888Z Entering 'third_party/cpuinfo' 2024-08-07T18:01:04.7210594Z Entering 'third_party/cudnn_frontend' 2024-08-07T18:01:04.7272804Z Entering 'third_party/cutlass' 2024-08-07T18:01:04.7345735Z Entering 'third_party/eigen' 2024-08-07T18:01:04.7412999Z Entering 'third_party/fbgemm' 2024-08-07T18:01:04.7476115Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T18:01:04.7538611Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T18:01:04.7604405Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T18:01:04.7677859Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T18:01:04.7740680Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T18:01:04.7805251Z Entering 'third_party/flatbuffers' 2024-08-07T18:01:04.7871635Z Entering 'third_party/fmt' 2024-08-07T18:01:04.7937968Z Entering 'third_party/foxi' 2024-08-07T18:01:04.8002156Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T18:01:04.8065307Z Entering 'third_party/gloo' 2024-08-07T18:01:04.8130890Z Entering 'third_party/googletest' 2024-08-07T18:01:04.8193880Z Entering 'third_party/ideep' 2024-08-07T18:01:04.8259642Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T18:01:04.8336723Z Entering 'third_party/ittapi' 2024-08-07T18:01:04.8402704Z Entering 'third_party/kineto' 2024-08-07T18:01:04.8467269Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T18:01:04.8530872Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T18:01:04.8595332Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T18:01:04.8660706Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T18:01:04.8723970Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T18:01:04.8785169Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T18:01:04.8849650Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T18:01:04.8915746Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T18:01:04.8982425Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T18:01:04.9048298Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T18:01:04.9115144Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T18:01:04.9177944Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T18:01:04.9244259Z Entering 'third_party/mimalloc' 2024-08-07T18:01:04.9308515Z Entering 'third_party/nccl/nccl' 2024-08-07T18:01:04.9372048Z Entering 'third_party/nlohmann' 2024-08-07T18:01:04.9437112Z Entering 'third_party/onnx' 2024-08-07T18:01:04.9521227Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T18:01:04.9587484Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T18:01:04.9658052Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T18:01:04.9725817Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T18:01:04.9790116Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T18:01:04.9856917Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T18:01:04.9920309Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T18:01:04.9985307Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T18:01:05.0049022Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T18:01:05.0111456Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T18:01:05.0172682Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T18:01:05.0242152Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T18:01:05.0313481Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T18:01:05.0403102Z Entering 'third_party/pocketfft' 2024-08-07T18:01:05.0468282Z Entering 'third_party/protobuf' 2024-08-07T18:01:05.0536825Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T18:01:05.0600780Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T18:01:05.0665965Z Entering 'third_party/psimd' 2024-08-07T18:01:05.0730562Z Entering 'third_party/pthreadpool' 2024-08-07T18:01:05.0793142Z Entering 'third_party/pybind11' 2024-08-07T18:01:05.0858662Z Entering 'third_party/python-peachpy' 2024-08-07T18:01:05.0922847Z Entering 'third_party/sleef' 2024-08-07T18:01:05.0990178Z Entering 'third_party/tensorpipe' 2024-08-07T18:01:05.1055641Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T18:01:05.1118970Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T18:01:05.1182307Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T18:01:05.1247540Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T18:01:05.1311133Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T18:01:05.1399003Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-07T18:01:05.1429651Z http.https://github.com/.extraheader 2024-08-07T18:01:05.1441403Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-07T18:01:05.1483284Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-07T18:01:05.1835512Z Entering 'android/libs/fbjni' 2024-08-07T18:01:05.1878032Z http.https://github.com/.extraheader 2024-08-07T18:01:05.1917542Z Entering 'third_party/FP16' 2024-08-07T18:01:05.1961038Z http.https://github.com/.extraheader 2024-08-07T18:01:05.2002202Z Entering 'third_party/FXdiv' 2024-08-07T18:01:05.2045997Z http.https://github.com/.extraheader 2024-08-07T18:01:05.2085782Z Entering 'third_party/NNPACK' 2024-08-07T18:01:05.2129923Z http.https://github.com/.extraheader 2024-08-07T18:01:05.2169172Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T18:01:05.2214074Z http.https://github.com/.extraheader 2024-08-07T18:01:05.2252908Z Entering 'third_party/XNNPACK' 2024-08-07T18:01:05.2296823Z http.https://github.com/.extraheader 2024-08-07T18:01:05.2356008Z Entering 'third_party/benchmark' 2024-08-07T18:01:05.2400770Z http.https://github.com/.extraheader 2024-08-07T18:01:05.2440516Z Entering 'third_party/cpp-httplib' 2024-08-07T18:01:05.2483925Z http.https://github.com/.extraheader 2024-08-07T18:01:05.2523125Z Entering 'third_party/cpuinfo' 2024-08-07T18:01:05.2568085Z http.https://github.com/.extraheader 2024-08-07T18:01:05.2609336Z Entering 'third_party/cudnn_frontend' 2024-08-07T18:01:05.2652018Z http.https://github.com/.extraheader 2024-08-07T18:01:05.2690778Z Entering 'third_party/cutlass' 2024-08-07T18:01:05.2735001Z http.https://github.com/.extraheader 2024-08-07T18:01:05.2783413Z Entering 'third_party/eigen' 2024-08-07T18:01:05.2827974Z http.https://github.com/.extraheader 2024-08-07T18:01:05.2870078Z Entering 'third_party/fbgemm' 2024-08-07T18:01:05.2915414Z http.https://github.com/.extraheader 2024-08-07T18:01:05.2954829Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T18:01:05.2999985Z http.https://github.com/.extraheader 2024-08-07T18:01:05.3038933Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T18:01:05.3083550Z http.https://github.com/.extraheader 2024-08-07T18:01:05.3125110Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T18:01:05.3169201Z http.https://github.com/.extraheader 2024-08-07T18:01:05.3220279Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T18:01:05.3263131Z http.https://github.com/.extraheader 2024-08-07T18:01:05.3304023Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T18:01:05.3346609Z http.https://github.com/.extraheader 2024-08-07T18:01:05.3387005Z Entering 'third_party/flatbuffers' 2024-08-07T18:01:05.3430930Z http.https://github.com/.extraheader 2024-08-07T18:01:05.3473729Z Entering 'third_party/fmt' 2024-08-07T18:01:05.3517325Z http.https://github.com/.extraheader 2024-08-07T18:01:05.3556143Z Entering 'third_party/foxi' 2024-08-07T18:01:05.3600330Z http.https://github.com/.extraheader 2024-08-07T18:01:05.3638853Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T18:01:05.3682034Z http.https://github.com/.extraheader 2024-08-07T18:01:05.3722508Z Entering 'third_party/gloo' 2024-08-07T18:01:05.3765705Z http.https://github.com/.extraheader 2024-08-07T18:01:05.3806602Z Entering 'third_party/googletest' 2024-08-07T18:01:05.3849158Z http.https://github.com/.extraheader 2024-08-07T18:01:05.3887469Z Entering 'third_party/ideep' 2024-08-07T18:01:05.3930063Z http.https://github.com/.extraheader 2024-08-07T18:01:05.3969429Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T18:01:05.4013640Z http.https://github.com/.extraheader 2024-08-07T18:01:05.4061890Z Entering 'third_party/ittapi' 2024-08-07T18:01:05.4106512Z http.https://github.com/.extraheader 2024-08-07T18:01:05.4145223Z Entering 'third_party/kineto' 2024-08-07T18:01:05.4189672Z http.https://github.com/.extraheader 2024-08-07T18:01:05.4229078Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T18:01:05.4273863Z http.https://github.com/.extraheader 2024-08-07T18:01:05.4315541Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T18:01:05.4359353Z http.https://github.com/.extraheader 2024-08-07T18:01:05.4402073Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T18:01:05.4445111Z http.https://github.com/.extraheader 2024-08-07T18:01:05.4484882Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T18:01:05.4528487Z http.https://github.com/.extraheader 2024-08-07T18:01:05.4567456Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T18:01:05.4612484Z http.https://github.com/.extraheader 2024-08-07T18:01:05.4650302Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T18:01:05.4693500Z http.https://github.com/.extraheader 2024-08-07T18:01:05.4735494Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T18:01:05.4777903Z http.https://github.com/.extraheader 2024-08-07T18:01:05.4817596Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T18:01:05.4859819Z http.https://github.com/.extraheader 2024-08-07T18:01:05.4900403Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T18:01:05.4945702Z http.https://github.com/.extraheader 2024-08-07T18:01:05.4989156Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T18:01:05.5034527Z http.https://github.com/.extraheader 2024-08-07T18:01:05.5077804Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T18:01:05.5120843Z http.https://github.com/.extraheader 2024-08-07T18:01:05.5160328Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T18:01:05.5206185Z http.https://github.com/.extraheader 2024-08-07T18:01:05.5246304Z Entering 'third_party/mimalloc' 2024-08-07T18:01:05.5288845Z http.https://github.com/.extraheader 2024-08-07T18:01:05.5328583Z Entering 'third_party/nccl/nccl' 2024-08-07T18:01:05.5370709Z http.https://github.com/.extraheader 2024-08-07T18:01:05.5409996Z Entering 'third_party/nlohmann' 2024-08-07T18:01:05.5452109Z http.https://github.com/.extraheader 2024-08-07T18:01:05.5492144Z Entering 'third_party/onnx' 2024-08-07T18:01:05.5535501Z http.https://github.com/.extraheader 2024-08-07T18:01:05.5593030Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T18:01:05.5638153Z http.https://github.com/.extraheader 2024-08-07T18:01:05.5677983Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T18:01:05.5722350Z http.https://github.com/.extraheader 2024-08-07T18:01:05.5765433Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T18:01:05.5812118Z http.https://github.com/.extraheader 2024-08-07T18:01:05.5854921Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T18:01:05.5898588Z http.https://github.com/.extraheader 2024-08-07T18:01:05.5938432Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T18:01:05.5981154Z http.https://github.com/.extraheader 2024-08-07T18:01:05.6020435Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T18:01:05.6063043Z http.https://github.com/.extraheader 2024-08-07T18:01:05.6101067Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T18:01:05.6144000Z http.https://github.com/.extraheader 2024-08-07T18:01:05.6183491Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T18:01:05.6226284Z http.https://github.com/.extraheader 2024-08-07T18:01:05.6266355Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T18:01:05.6313193Z http.https://github.com/.extraheader 2024-08-07T18:01:05.6352279Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T18:01:05.6393936Z http.https://github.com/.extraheader 2024-08-07T18:01:05.6432893Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T18:01:05.6476101Z http.https://github.com/.extraheader 2024-08-07T18:01:05.6517978Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T18:01:05.6560807Z http.https://github.com/.extraheader 2024-08-07T18:01:05.6602764Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T18:01:05.6645976Z http.https://github.com/.extraheader 2024-08-07T18:01:05.6711040Z Entering 'third_party/pocketfft' 2024-08-07T18:01:05.6754065Z http.https://github.com/.extraheader 2024-08-07T18:01:05.6791615Z Entering 'third_party/protobuf' 2024-08-07T18:01:05.6834534Z http.https://github.com/.extraheader 2024-08-07T18:01:05.6876626Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T18:01:05.6919647Z http.https://github.com/.extraheader 2024-08-07T18:01:05.6961027Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T18:01:05.7004871Z http.https://github.com/.extraheader 2024-08-07T18:01:05.7047334Z Entering 'third_party/psimd' 2024-08-07T18:01:05.7091269Z http.https://github.com/.extraheader 2024-08-07T18:01:05.7130997Z Entering 'third_party/pthreadpool' 2024-08-07T18:01:05.7174075Z http.https://github.com/.extraheader 2024-08-07T18:01:05.7212455Z Entering 'third_party/pybind11' 2024-08-07T18:01:05.7255959Z http.https://github.com/.extraheader 2024-08-07T18:01:05.7294782Z Entering 'third_party/python-peachpy' 2024-08-07T18:01:05.7338893Z http.https://github.com/.extraheader 2024-08-07T18:01:05.7376800Z Entering 'third_party/sleef' 2024-08-07T18:01:05.7420401Z http.https://github.com/.extraheader 2024-08-07T18:01:05.7458253Z Entering 'third_party/tensorpipe' 2024-08-07T18:01:05.7502033Z http.https://github.com/.extraheader 2024-08-07T18:01:05.7542190Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T18:01:05.7585267Z http.https://github.com/.extraheader 2024-08-07T18:01:05.7627222Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T18:01:05.7670553Z http.https://github.com/.extraheader 2024-08-07T18:01:05.7710832Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T18:01:05.7754864Z http.https://github.com/.extraheader 2024-08-07T18:01:05.7796234Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T18:01:05.7841209Z http.https://github.com/.extraheader 2024-08-07T18:01:05.7880354Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T18:01:05.7923697Z http.https://github.com/.extraheader 2024-08-07T18:01:05.8068874Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-07T18:01:05.8099728Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-07T18:01:05.8105758Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T18:01:05.8106260Z ##[endgroup] 2024-08-07T18:01:14.3972508Z Cleaning up orphan processes