2024-08-07T17:31:31.3959636Z Current runner version: '2.318.0' 2024-08-07T17:31:31.3967869Z Runner name: 'i-036a6435dcb00f35e' 2024-08-07T17:31:31.3968826Z Runner group name: 'Default' 2024-08-07T17:31:31.3969926Z Machine name: 'ip-10-0-41-45' 2024-08-07T17:31:31.3974587Z ##[group]GITHUB_TOKEN Permissions 2024-08-07T17:31:31.3977506Z Actions: read 2024-08-07T17:31:31.3978226Z Attestations: read 2024-08-07T17:31:31.3978851Z Checks: read 2024-08-07T17:31:31.3979416Z Contents: read 2024-08-07T17:31:31.3980079Z Deployments: read 2024-08-07T17:31:31.3980713Z Discussions: read 2024-08-07T17:31:31.3981295Z Issues: read 2024-08-07T17:31:31.3981939Z Metadata: read 2024-08-07T17:31:31.3982574Z Packages: read 2024-08-07T17:31:31.3983198Z Pages: read 2024-08-07T17:31:31.3983820Z PullRequests: read 2024-08-07T17:31:31.3984435Z RepositoryProjects: read 2024-08-07T17:31:31.3985123Z SecurityEvents: read 2024-08-07T17:31:31.3985777Z Statuses: read 2024-08-07T17:31:31.3986347Z ##[endgroup] 2024-08-07T17:31:31.3989933Z Secret source: Actions 2024-08-07T17:31:31.3990870Z Prepare workflow directory 2024-08-07T17:31:31.7098174Z Prepare all required actions 2024-08-07T17:31:31.7140715Z Getting action download info 2024-08-07T17:31:31.8913089Z Download action repository 'pytorch/test-infra@main' (SHA:a1f5a89251fc4258ab59806094fe3108f7d6741a) 2024-08-07T17:31:33.2210913Z Download action repository 'pytorch/pytorch@main' (SHA:32a284c27549449c19e3b1455f531692439d6316) 2024-08-07T17:31:43.3983940Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-07T17:31:43.6066780Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-07T17:31:43.8648789Z Getting action download info 2024-08-07T17:31:43.9870276Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-07T17:31:44.3032238Z Getting action download info 2024-08-07T17:31:44.3885411Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-07T17:31:44.5745137Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/pull/131248/merge (f779f6b7738020e244184bded4026b37de3f9f24) 2024-08-07T17:31:44.5747299Z ##[group] Inputs 2024-08-07T17:31:44.5747671Z build-environment: linux-jammy-py3.8-gcc11 2024-08-07T17:31:44.5750559Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]} 2024-08-07T17:31:44.5753888Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:31:44.5754644Z sync-tag: 2024-08-07T17:31:44.5755506Z timeout-minutes: 240 2024-08-07T17:31:44.5755788Z use-gha: 2024-08-07T17:31:44.5756031Z dashboard-tag: 2024-08-07T17:31:44.5756287Z s3-bucket: gha-artifacts 2024-08-07T17:31:44.5756574Z aws-role-to-assume: 2024-08-07T17:31:44.5756848Z ##[endgroup] 2024-08-07T17:31:44.5757284Z Complete job name: linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-07T17:31:44.6222543Z A job started hook has been configured by the self-hosted runner administrator 2024-08-07T17:31:44.6336810Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-07T17:31:44.6347037Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:31:44.6347790Z ##[endgroup] 2024-08-07T17:31:46.1976271Z Runner Type: amz2023.linux.2xlarge 2024-08-07T17:31:46.1976791Z Instance Type: c5.2xlarge 2024-08-07T17:31:46.1977170Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-07T17:31:46.1977596Z AMI ID: ami-06c68f701d8090592 2024-08-07T17:31:51.7586296Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-07T17:31:51.7586749Z with: 2024-08-07T17:31:51.7587406Z github-secret: *** 2024-08-07T17:31:51.7588157Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-07T17:31:51.7588944Z activate-with-label: false 2024-08-07T17:31:51.7589240Z label: with-ssh 2024-08-07T17:31:51.7589505Z remove-existing-keys: true 2024-08-07T17:31:51.7589787Z fail-silently: true 2024-08-07T17:31:51.7590062Z env: 2024-08-07T17:31:51.7590317Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:31:51.7590587Z ##[endgroup] 2024-08-07T17:31:51.8413870Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-07T17:31:52.1944964Z Grabbing public ssh keys from https://github.com/zdevito.keys 2024-08-07T17:31:52.2673310Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-07T17:31:52.2690842Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-07T17:31:52.2811189Z Login using: ssh ec2-user@ec2-44-203-143-200.compute-1.amazonaws.com 2024-08-07T17:31:52.2811793Z All testing is done inside the container, to start an interactive session run: 2024-08-07T17:31:52.2812406Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-07T17:31:52.2922495Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-07T17:31:52.2922953Z with: 2024-08-07T17:31:52.2923185Z submodules: recursive 2024-08-07T17:31:52.2923471Z fetch-depth: 0 2024-08-07T17:31:52.2923712Z env: 2024-08-07T17:31:52.2923924Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:31:52.2924197Z ##[endgroup] 2024-08-07T17:31:52.3002113Z ##[group]Run retry () { 2024-08-07T17:31:52.3002443Z retry () { 2024-08-07T17:31:52.3002791Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-07T17:31:52.3003197Z } 2024-08-07T17:31:52.3003472Z echo "${GITHUB_WORKSPACE}" 2024-08-07T17:31:52.3003828Z if [ -z "${NO_SUDO}" ]; then 2024-08-07T17:31:52.3004186Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-07T17:31:52.3004582Z else 2024-08-07T17:31:52.3004841Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-07T17:31:52.3005172Z fi 2024-08-07T17:31:52.3005466Z mkdir "${GITHUB_WORKSPACE}" 2024-08-07T17:31:52.3013044Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:31:52.3013455Z env: 2024-08-07T17:31:52.3013687Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:31:52.3013962Z NO_SUDO: 2024-08-07T17:31:52.3014188Z ##[endgroup] 2024-08-07T17:31:52.3039772Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:31:54.4319462Z ##[group]Run malfet/checkout@silent-checkout 2024-08-07T17:31:54.4319820Z with: 2024-08-07T17:31:54.4320063Z ref: 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:31:54.4320404Z fetch-depth: 0 2024-08-07T17:31:54.4320662Z submodules: recursive 2024-08-07T17:31:54.4320917Z quiet-checkout: true 2024-08-07T17:31:54.4321191Z repository: pytorch/pytorch 2024-08-07T17:31:54.4321613Z token: *** 2024-08-07T17:31:54.4321847Z ssh-strict: true 2024-08-07T17:31:54.4322092Z persist-credentials: true 2024-08-07T17:31:54.4322377Z clean: true 2024-08-07T17:31:54.4322632Z sparse-checkout-cone-mode: true 2024-08-07T17:31:54.4322935Z lfs: false 2024-08-07T17:31:54.4323173Z set-safe-directory: true 2024-08-07T17:31:54.4323432Z env: 2024-08-07T17:31:54.4323657Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:31:54.4324132Z ##[endgroup] 2024-08-07T17:31:54.5253268Z Syncing repository: pytorch/pytorch 2024-08-07T17:31:54.5254624Z ##[group]Getting Git version info 2024-08-07T17:31:54.5255118Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-07T17:31:54.5255798Z [command]/usr/bin/git version 2024-08-07T17:31:54.5256098Z git version 2.40.1 2024-08-07T17:31:54.5268925Z ##[endgroup] 2024-08-07T17:31:54.5282163Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/fd684c6b-b756-4203-81c9-8594c1f0e6f5' before making global git config changes 2024-08-07T17:31:54.5283154Z Adding repository directory to the temporary git global config as a safe directory 2024-08-07T17:31:54.5287119Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:31:54.5327821Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-07T17:31:54.5331868Z ##[group]Initializing the repository 2024-08-07T17:31:54.5334730Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:31:54.5443155Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-07T17:31:54.5443854Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-07T17:31:54.5444435Z hint: of your new repositories, which will suppress this warning, call: 2024-08-07T17:31:54.5444873Z hint: 2024-08-07T17:31:54.5445191Z hint: git config --global init.defaultBranch 2024-08-07T17:31:54.5445687Z hint: 2024-08-07T17:31:54.5446336Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-07T17:31:54.5446932Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-07T17:31:54.5447612Z hint: 2024-08-07T17:31:54.5447857Z hint: git branch -m 2024-08-07T17:31:54.5448404Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-07T17:31:54.5452917Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-07T17:31:54.5489004Z ##[endgroup] 2024-08-07T17:31:54.5489453Z ##[group]Disabling automatic garbage collection 2024-08-07T17:31:54.5492025Z [command]/usr/bin/git config --local gc.auto 0 2024-08-07T17:31:54.5521362Z ##[endgroup] 2024-08-07T17:31:54.5521760Z ##[group]Setting up auth 2024-08-07T17:31:54.5527306Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-07T17:31:54.5558096Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-07T17:31:54.5809647Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-07T17:31:54.5839321Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-07T17:31:54.6096841Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-07T17:31:54.6141812Z ##[endgroup] 2024-08-07T17:31:54.6142272Z ##[group]Fetching the repository 2024-08-07T17:31:54.6147804Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-07T17:31:57.0169750Z remote: Enumerating objects: 1008194 2024-08-07T17:31:57.0170224Z remote: Enumerating objects: 1011031, done. 2024-08-07T17:31:57.0171440Z remote: Counting objects: 0% (1/2837) 2024-08-07T17:31:57.0172283Z remote: Counting objects: 1% (29/2837) 2024-08-07T17:31:57.0173532Z remote: Counting objects: 2% (57/2837) 2024-08-07T17:31:57.0175040Z remote: Counting objects: 3% (86/2837) 2024-08-07T17:31:57.0176989Z remote: Counting objects: 4% (114/2837) 2024-08-07T17:31:57.0178102Z remote: Counting objects: 5% (142/2837) 2024-08-07T17:31:57.0179553Z remote: Counting objects: 6% (171/2837) 2024-08-07T17:31:57.0180273Z remote: Counting objects: 7% (199/2837) 2024-08-07T17:31:57.0180986Z remote: Counting objects: 8% (227/2837) 2024-08-07T17:31:57.0181493Z remote: Counting objects: 9% (256/2837) 2024-08-07T17:31:57.0182162Z remote: Counting objects: 10% (284/2837) 2024-08-07T17:31:57.0182862Z remote: Counting objects: 11% (313/2837) 2024-08-07T17:31:57.0183589Z remote: Counting objects: 12% (341/2837) 2024-08-07T17:31:57.0184091Z remote: Counting objects: 13% (369/2837) 2024-08-07T17:31:57.0184575Z remote: Counting objects: 14% (398/2837) 2024-08-07T17:31:57.0185295Z remote: Counting objects: 15% (426/2837) 2024-08-07T17:31:57.0185756Z remote: Counting objects: 16% (454/2837) 2024-08-07T17:31:57.0186137Z remote: Counting objects: 17% (483/2837) 2024-08-07T17:31:57.0186798Z remote: Counting objects: 18% (511/2837) 2024-08-07T17:31:57.0187439Z remote: Counting objects: 19% (540/2837) 2024-08-07T17:31:57.0187823Z remote: Counting objects: 20% (568/2837) 2024-08-07T17:31:57.0188521Z remote: Counting objects: 21% (596/2837) 2024-08-07T17:31:57.0189237Z remote: Counting objects: 22% (625/2837) 2024-08-07T17:31:57.0189837Z remote: Counting objects: 23% (653/2837) 2024-08-07T17:31:57.0190209Z remote: Counting objects: 24% (681/2837) 2024-08-07T17:31:57.0190872Z remote: Counting objects: 25% (710/2837) 2024-08-07T17:31:57.0191555Z remote: Counting objects: 26% (738/2837) 2024-08-07T17:31:57.0192004Z remote: Counting objects: 27% (766/2837) 2024-08-07T17:31:57.0193069Z remote: Counting objects: 28% (795/2837) 2024-08-07T17:31:57.0193645Z remote: Counting objects: 29% (823/2837) 2024-08-07T17:31:57.0194247Z remote: Counting objects: 30% (852/2837) 2024-08-07T17:31:57.0194953Z remote: Counting objects: 31% (880/2837) 2024-08-07T17:31:57.0195344Z remote: Counting objects: 32% (908/2837) 2024-08-07T17:31:57.0195882Z remote: Counting objects: 33% (937/2837) 2024-08-07T17:31:57.0196593Z remote: Counting objects: 34% (965/2837) 2024-08-07T17:31:57.0197032Z remote: Counting objects: 35% (993/2837) 2024-08-07T17:31:57.0197612Z remote: Counting objects: 36% (1022/2837) 2024-08-07T17:31:57.0198322Z remote: Counting objects: 37% (1050/2837) 2024-08-07T17:31:57.0198962Z remote: Counting objects: 38% (1079/2837) 2024-08-07T17:31:57.0199540Z remote: Counting objects: 39% (1107/2837) 2024-08-07T17:31:57.0200211Z remote: Counting objects: 40% (1135/2837) 2024-08-07T17:31:57.0200596Z remote: Counting objects: 41% (1164/2837) 2024-08-07T17:31:57.0200986Z remote: Counting objects: 42% (1192/2837) 2024-08-07T17:31:57.0201644Z remote: Counting objects: 43% (1220/2837) 2024-08-07T17:31:57.0202196Z remote: Counting objects: 44% (1249/2837) 2024-08-07T17:31:57.0202577Z remote: Counting objects: 45% (1277/2837) 2024-08-07T17:31:57.0203051Z remote: Counting objects: 46% (1306/2837) 2024-08-07T17:31:57.0203706Z remote: Counting objects: 47% (1334/2837) 2024-08-07T17:31:57.0204205Z remote: Counting objects: 48% (1362/2837) 2024-08-07T17:31:57.0204786Z remote: Counting objects: 49% (1391/2837) 2024-08-07T17:31:57.0205219Z remote: Counting objects: 50% (1419/2837) 2024-08-07T17:31:57.0205603Z remote: Counting objects: 51% (1447/2837) 2024-08-07T17:31:57.0206087Z remote: Counting objects: 52% (1476/2837) 2024-08-07T17:31:57.0206517Z remote: Counting objects: 53% (1504/2837) 2024-08-07T17:31:57.0206910Z remote: Counting objects: 54% (1532/2837) 2024-08-07T17:31:57.0207438Z remote: Counting objects: 55% (1561/2837) 2024-08-07T17:31:57.0208127Z remote: Counting objects: 56% (1589/2837) 2024-08-07T17:31:57.0208519Z remote: Counting objects: 57% (1618/2837) 2024-08-07T17:31:57.0209039Z remote: Counting objects: 58% (1646/2837) 2024-08-07T17:31:57.0209588Z remote: Counting objects: 59% (1674/2837) 2024-08-07T17:31:57.0209976Z remote: Counting objects: 60% (1703/2837) 2024-08-07T17:31:57.0210387Z remote: Counting objects: 61% (1731/2837) 2024-08-07T17:31:57.0210891Z remote: Counting objects: 62% (1759/2837) 2024-08-07T17:31:57.0211360Z remote: Counting objects: 63% (1788/2837) 2024-08-07T17:31:57.0211752Z remote: Counting objects: 64% (1816/2837) 2024-08-07T17:31:57.0212125Z remote: Counting objects: 65% (1845/2837) 2024-08-07T17:31:57.0212603Z remote: Counting objects: 66% (1873/2837) 2024-08-07T17:31:57.0213079Z remote: Counting objects: 67% (1901/2837) 2024-08-07T17:31:57.0213457Z remote: Counting objects: 68% (1930/2837) 2024-08-07T17:31:57.0213850Z remote: Counting objects: 69% (1958/2837) 2024-08-07T17:31:57.0214232Z remote: Counting objects: 70% (1986/2837) 2024-08-07T17:31:57.0214605Z remote: Counting objects: 71% (2015/2837) 2024-08-07T17:31:57.0214987Z remote: Counting objects: 72% (2043/2837) 2024-08-07T17:31:57.0215373Z remote: Counting objects: 73% (2072/2837) 2024-08-07T17:31:57.0215775Z remote: Counting objects: 74% (2100/2837) 2024-08-07T17:31:57.0216144Z remote: Counting objects: 75% (2128/2837) 2024-08-07T17:31:57.0216526Z remote: Counting objects: 76% (2157/2837) 2024-08-07T17:31:57.0216915Z remote: Counting objects: 77% (2185/2837) 2024-08-07T17:31:57.0217290Z remote: Counting objects: 78% (2213/2837) 2024-08-07T17:31:57.0217779Z remote: Counting objects: 79% (2242/2837) 2024-08-07T17:31:57.0218172Z remote: Counting objects: 80% (2270/2837) 2024-08-07T17:31:57.0218547Z remote: Counting objects: 81% (2298/2837) 2024-08-07T17:31:57.0218939Z remote: Counting objects: 82% (2327/2837) 2024-08-07T17:31:57.0219328Z remote: Counting objects: 83% (2355/2837) 2024-08-07T17:31:57.0219696Z remote: Counting objects: 84% (2384/2837) 2024-08-07T17:31:57.0220081Z remote: Counting objects: 85% (2412/2837) 2024-08-07T17:31:57.0220464Z remote: Counting objects: 86% (2440/2837) 2024-08-07T17:31:57.0220835Z remote: Counting objects: 87% (2469/2837) 2024-08-07T17:31:57.0221220Z remote: Counting objects: 88% (2497/2837) 2024-08-07T17:31:57.0221602Z remote: Counting objects: 89% (2525/2837) 2024-08-07T17:31:57.0221970Z remote: Counting objects: 90% (2554/2837) 2024-08-07T17:31:57.0222351Z remote: Counting objects: 91% (2582/2837) 2024-08-07T17:31:57.0222735Z remote: Counting objects: 92% (2611/2837) 2024-08-07T17:31:57.0223105Z remote: Counting objects: 93% (2639/2837) 2024-08-07T17:31:57.0223488Z remote: Counting objects: 94% (2667/2837) 2024-08-07T17:31:57.0223875Z remote: Counting objects: 95% (2696/2837) 2024-08-07T17:31:57.0224245Z remote: Counting objects: 96% (2724/2837) 2024-08-07T17:31:57.0224628Z remote: Counting objects: 97% (2752/2837) 2024-08-07T17:31:57.0225011Z remote: Counting objects: 98% (2781/2837) 2024-08-07T17:31:57.0225380Z remote: Counting objects: 99% (2809/2837) 2024-08-07T17:31:57.0225762Z remote: Counting objects: 100% (2837/2837) 2024-08-07T17:31:57.0226179Z remote: Counting objects: 100% (2837/2837), done. 2024-08-07T17:31:57.0822904Z remote: Compressing objects: 0% (1/1487) 2024-08-07T17:31:57.1108587Z remote: Compressing objects: 1% (15/1487) 2024-08-07T17:31:57.1695408Z remote: Compressing objects: 2% (30/1487) 2024-08-07T17:31:57.3477908Z remote: Compressing objects: 3% (45/1487) 2024-08-07T17:31:57.4611172Z remote: Compressing objects: 4% (60/1487) 2024-08-07T17:31:57.5599815Z remote: Compressing objects: 5% (75/1487) 2024-08-07T17:31:57.6311682Z remote: Compressing objects: 6% (90/1487) 2024-08-07T17:31:57.6692853Z remote: Compressing objects: 7% (105/1487) 2024-08-07T17:31:57.7122401Z remote: Compressing objects: 8% (119/1487) 2024-08-07T17:31:57.7342558Z remote: Compressing objects: 9% (134/1487) 2024-08-07T17:31:57.7587635Z remote: Compressing objects: 10% (149/1487) 2024-08-07T17:31:57.7823843Z remote: Compressing objects: 11% (164/1487) 2024-08-07T17:31:57.8039238Z remote: Compressing objects: 12% (179/1487) 2024-08-07T17:31:57.8244128Z remote: Compressing objects: 13% (194/1487) 2024-08-07T17:31:57.8397515Z remote: Compressing objects: 14% (209/1487) 2024-08-07T17:31:57.8505815Z remote: Compressing objects: 15% (224/1487) 2024-08-07T17:31:57.8605053Z remote: Compressing objects: 16% (238/1487) 2024-08-07T17:31:57.8682649Z remote: Compressing objects: 17% (253/1487) 2024-08-07T17:31:57.8737262Z remote: Compressing objects: 18% (268/1487) 2024-08-07T17:31:57.8771678Z remote: Compressing objects: 19% (283/1487) 2024-08-07T17:31:57.8798856Z remote: Compressing objects: 20% (298/1487) 2024-08-07T17:31:57.8804716Z remote: Compressing objects: 21% (313/1487) 2024-08-07T17:31:57.8824402Z remote: Compressing objects: 22% (328/1487) 2024-08-07T17:31:57.8841278Z remote: Compressing objects: 23% (343/1487) 2024-08-07T17:31:57.8868078Z remote: Compressing objects: 24% (357/1487) 2024-08-07T17:31:57.8886471Z remote: Compressing objects: 25% (372/1487) 2024-08-07T17:31:57.8908647Z remote: Compressing objects: 26% (387/1487) 2024-08-07T17:31:57.8938157Z remote: Compressing objects: 27% (402/1487) 2024-08-07T17:31:57.8953434Z remote: Compressing objects: 28% (417/1487) 2024-08-07T17:31:57.8971493Z remote: Compressing objects: 29% (432/1487) 2024-08-07T17:31:57.8994400Z remote: Compressing objects: 30% (447/1487) 2024-08-07T17:31:57.9011894Z remote: Compressing objects: 31% (461/1487) 2024-08-07T17:31:57.9042091Z remote: Compressing objects: 32% (476/1487) 2024-08-07T17:31:57.9049427Z remote: Compressing objects: 33% (491/1487) 2024-08-07T17:31:57.9057809Z remote: Compressing objects: 34% (506/1487) 2024-08-07T17:31:57.9070435Z remote: Compressing objects: 35% (521/1487) 2024-08-07T17:31:57.9080051Z remote: Compressing objects: 36% (536/1487) 2024-08-07T17:31:57.9093280Z remote: Compressing objects: 37% (551/1487) 2024-08-07T17:31:57.9107290Z remote: Compressing objects: 38% (566/1487) 2024-08-07T17:31:57.9116602Z remote: Compressing objects: 39% (580/1487) 2024-08-07T17:31:57.9129427Z remote: Compressing objects: 40% (595/1487) 2024-08-07T17:31:57.9136997Z remote: Compressing objects: 41% (610/1487) 2024-08-07T17:31:57.9144387Z remote: Compressing objects: 42% (625/1487) 2024-08-07T17:31:57.9158753Z remote: Compressing objects: 43% (640/1487) 2024-08-07T17:31:57.9163204Z remote: Compressing objects: 44% (655/1487) 2024-08-07T17:31:57.9171151Z remote: Compressing objects: 45% (670/1487) 2024-08-07T17:31:57.9179390Z remote: Compressing objects: 46% (685/1487) 2024-08-07T17:31:57.9186685Z remote: Compressing objects: 47% (699/1487) 2024-08-07T17:31:57.9192177Z remote: Compressing objects: 48% (714/1487) 2024-08-07T17:31:57.9198702Z remote: Compressing objects: 49% (729/1487) 2024-08-07T17:31:57.9202872Z remote: Compressing objects: 50% (744/1487) 2024-08-07T17:31:57.9209389Z remote: Compressing objects: 51% (759/1487) 2024-08-07T17:31:57.9214860Z remote: Compressing objects: 52% (774/1487) 2024-08-07T17:31:57.9218709Z remote: Compressing objects: 53% (789/1487) 2024-08-07T17:31:57.9222283Z remote: Compressing objects: 54% (803/1487) 2024-08-07T17:31:57.9224693Z remote: Compressing objects: 55% (818/1487) 2024-08-07T17:31:57.9226920Z remote: Compressing objects: 56% (833/1487) 2024-08-07T17:31:57.9228322Z remote: Compressing objects: 57% (848/1487) 2024-08-07T17:31:57.9230595Z remote: Compressing objects: 58% (863/1487) 2024-08-07T17:31:57.9235348Z remote: Compressing objects: 59% (878/1487) 2024-08-07T17:31:57.9235768Z remote: Compressing objects: 60% (893/1487) 2024-08-07T17:31:57.9236163Z remote: Compressing objects: 61% (908/1487) 2024-08-07T17:31:57.9236575Z remote: Compressing objects: 62% (922/1487) 2024-08-07T17:31:57.9236988Z remote: Compressing objects: 63% (937/1487) 2024-08-07T17:31:57.9238685Z remote: Compressing objects: 64% (952/1487) 2024-08-07T17:31:57.9258065Z remote: Compressing objects: 65% (967/1487) 2024-08-07T17:31:57.9271724Z remote: Compressing objects: 66% (982/1487) 2024-08-07T17:31:57.9280333Z remote: Compressing objects: 67% (997/1487) 2024-08-07T17:31:57.9292691Z remote: Compressing objects: 68% (1012/1487) 2024-08-07T17:31:57.9297173Z remote: Compressing objects: 69% (1027/1487) 2024-08-07T17:31:57.9303777Z remote: Compressing objects: 70% (1041/1487) 2024-08-07T17:31:57.9309990Z remote: Compressing objects: 71% (1056/1487) 2024-08-07T17:31:57.9315947Z remote: Compressing objects: 72% (1071/1487) 2024-08-07T17:31:57.9320998Z remote: Compressing objects: 73% (1086/1487) 2024-08-07T17:31:57.9327118Z remote: Compressing objects: 74% (1101/1487) 2024-08-07T17:31:57.9332398Z remote: Compressing objects: 75% (1116/1487) 2024-08-07T17:31:57.9338209Z remote: Compressing objects: 76% (1131/1487) 2024-08-07T17:31:57.9342219Z remote: Compressing objects: 77% (1145/1487) 2024-08-07T17:31:57.9347550Z remote: Compressing objects: 78% (1160/1487) 2024-08-07T17:31:57.9352879Z remote: Compressing objects: 79% (1175/1487) 2024-08-07T17:31:57.9356844Z remote: Compressing objects: 80% (1190/1487) 2024-08-07T17:31:57.9361834Z remote: Compressing objects: 81% (1205/1487) 2024-08-07T17:31:57.9366611Z remote: Compressing objects: 82% (1220/1487) 2024-08-07T17:31:57.9370320Z remote: Compressing objects: 83% (1235/1487) 2024-08-07T17:31:57.9373919Z remote: Compressing objects: 84% (1250/1487) 2024-08-07T17:31:57.9378493Z remote: Compressing objects: 85% (1264/1487) 2024-08-07T17:31:57.9381673Z remote: Compressing objects: 86% (1279/1487) 2024-08-07T17:31:57.9384140Z remote: Compressing objects: 87% (1294/1487) 2024-08-07T17:31:57.9385984Z remote: Compressing objects: 88% (1309/1487) 2024-08-07T17:31:57.9388960Z remote: Compressing objects: 89% (1324/1487) 2024-08-07T17:31:57.9392782Z remote: Compressing objects: 90% (1339/1487) 2024-08-07T17:31:57.9396110Z remote: Compressing objects: 91% (1354/1487) 2024-08-07T17:31:57.9398462Z remote: Compressing objects: 92% (1369/1487) 2024-08-07T17:31:57.9400521Z remote: Compressing objects: 93% (1383/1487) 2024-08-07T17:31:57.9402826Z remote: Compressing objects: 94% (1398/1487) 2024-08-07T17:31:57.9406085Z remote: Compressing objects: 95% (1413/1487) 2024-08-07T17:31:57.9407839Z remote: Compressing objects: 96% (1428/1487) 2024-08-07T17:31:57.9410397Z remote: Compressing objects: 97% (1443/1487) 2024-08-07T17:31:57.9412848Z remote: Compressing objects: 98% (1458/1487) 2024-08-07T17:31:57.9414559Z remote: Compressing objects: 99% (1473/1487) 2024-08-07T17:31:57.9415380Z remote: Compressing objects: 100% (1487/1487) 2024-08-07T17:31:57.9415850Z remote: Compressing objects: 100% (1487/1487), done. 2024-08-07T17:32:18.8270708Z remote: Total 1011031 (delta 1809), reused 2212 (delta 1343), pack-reused 1008194 2024-08-07T17:32:45.8118606Z [command]/usr/bin/git rev-parse --verify --quiet 016588f53c6904b840aa56aa86f95460b4d9c996^{object} 2024-08-07T17:32:45.8149252Z 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:32:45.8153538Z ##[endgroup] 2024-08-07T17:32:45.8153970Z ##[group]Determining the checkout info 2024-08-07T17:32:45.8155198Z ##[endgroup] 2024-08-07T17:32:45.8155603Z ##[group]Checking out the ref 2024-08-07T17:32:45.8158592Z [command]/usr/bin/git checkout --quiet --force 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:32:47.1858990Z ##[endgroup] 2024-08-07T17:32:47.1859467Z ##[group]Setting up auth for fetching submodules 2024-08-07T17:32:47.1864127Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-07T17:32:47.1920967Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-07T17:32:47.1956972Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-07T17:32:47.1991238Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-07T17:32:47.2021331Z ##[endgroup] 2024-08-07T17:32:47.2021752Z ##[group]Fetching submodules 2024-08-07T17:32:47.2025913Z [command]/usr/bin/git submodule sync --recursive 2024-08-07T17:32:47.2337028Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-07T17:32:47.2644124Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-07T17:32:47.2645216Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-07T17:32:47.2646519Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-07T17:32:47.2649486Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-07T17:32:47.2652151Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-07T17:32:47.2654538Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-07T17:32:47.2657486Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-07T17:32:47.2660565Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-07T17:32:47.2664752Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-07T17:32:47.2667128Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-07T17:32:47.2670894Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-07T17:32:47.2674393Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-07T17:32:47.2678025Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-07T17:32:47.2681800Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-07T17:32:47.2685492Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-07T17:32:47.2689335Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-08-07T17:32:47.2694951Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:32:47.2698829Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-07T17:32:47.2702915Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-07T17:32:47.2707161Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-07T17:32:47.2711461Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-07T17:32:47.2716065Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-07T17:32:47.2720666Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-07T17:32:47.2725176Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-07T17:32:47.2729827Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-07T17:32:47.2734462Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-07T17:32:47.2739500Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-07T17:32:47.2744311Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-07T17:32:47.2749383Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-07T17:32:47.2754624Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-07T17:32:47.2760115Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-07T17:32:47.2765831Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-07T17:32:47.2771360Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-07T17:32:47.2778443Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-07T17:32:47.2784151Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-07T17:32:47.2817715Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-07T17:32:47.5740180Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-07T17:32:47.7496738Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-07T17:32:47.9645838Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-07T17:32:48.2372319Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-07T17:32:50.4912685Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-07T17:33:04.6130547Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-07T17:33:05.0392407Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-07T17:33:05.5285279Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-07T17:33:06.1215899Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-07T17:33:07.2828805Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-07T17:33:09.1982637Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-07T17:33:14.4388959Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-07T17:33:15.8300304Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-07T17:33:17.6982635Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-07T17:33:19.1590999Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-08-07T17:33:19.3362600Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-07T17:33:19.7983057Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-07T17:33:20.1170602Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-07T17:33:21.3539523Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-07T17:33:21.7037298Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-07T17:33:21.9637415Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-07T17:33:23.4604704Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-07T17:33:24.2528155Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-07T17:33:24.9054483Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-07T17:33:31.1653646Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-07T17:33:33.3505444Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-07T17:33:39.2404189Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-07T17:33:39.4645025Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-07T17:33:48.6816712Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-07T17:33:48.8362336Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-07T17:33:49.0444989Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-07T17:33:49.9300180Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-07T17:33:50.2082739Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-07T17:33:50.9082147Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-07T17:33:51.3040573Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-07T17:33:51.3152029Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-07T17:33:51.3234673Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-07T17:33:51.3462639Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-07T17:33:51.3803898Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-07T17:33:52.2901406Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-07T17:33:52.3124225Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-07T17:33:52.3555038Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-07T17:33:52.4476164Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-07T17:33:52.4785945Z Submodule path 'third_party/cudnn_frontend': checked out '98ca4e1941fe3263f128f74f10063a3ea35c7019' 2024-08-07T17:33:52.9555286Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-07T17:33:53.1971945Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-07T17:33:53.2744494Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-07T17:33:53.2762067Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:33:53.2764793Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:33:53.2767167Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:33:53.2770158Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:33:53.2772763Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:33:53.2798419Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-07T17:33:54.3586215Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-07T17:33:54.9737151Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-07T17:33:56.8528721Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-07T17:33:57.9865791Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-07T17:33:58.3151822Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-07T17:33:58.4068156Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-07T17:33:58.7898446Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-07T17:33:58.8496917Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-07T17:33:58.8613184Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-07T17:33:58.9719599Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-07T17:33:59.0108521Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-07T17:33:59.0198580Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-08-07T17:33:59.0582598Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-07T17:33:59.0823906Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-07T17:33:59.1252928Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-07T17:33:59.1368887Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-07T17:33:59.1384045Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-07T17:33:59.1405597Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-07T17:34:13.1816536Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-07T17:34:13.1990037Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-07T17:34:13.2827014Z Submodule path 'third_party/kineto': checked out 'da2f2682cabaf95d601fa2a9b7e0979f84fe7667' 2024-08-07T17:34:13.2842956Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:34:13.2845706Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:34:13.2847556Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:34:13.2872039Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-07T17:34:13.8402593Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-07T17:34:15.1433691Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-07T17:34:16.3769917Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-07T17:34:16.3785936Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:34:16.3788016Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:34:16.3790418Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:34:16.3792966Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:34:16.3795646Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:34:16.3798642Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:34:16.3801135Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:34:16.3803969Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:34:16.3830251Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-07T17:34:17.1958107Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-07T17:34:17.6724650Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-07T17:34:18.9763302Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-07T17:34:19.2590581Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-07T17:34:19.8217342Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-07T17:34:21.0604787Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-07T17:34:28.8731527Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-07T17:34:29.3063327Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-07T17:34:29.3241739Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-07T17:34:29.3596493Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-07T17:34:29.3723938Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-07T17:34:29.3738723Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:34:29.3763944Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-07T17:34:29.8230589Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-07T17:34:29.8405473Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-07T17:34:29.8790652Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-07T17:34:29.9742859Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-07T17:34:29.9897628Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-07T17:34:30.0277840Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-07T17:34:30.0850838Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-07T17:34:30.1197046Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-07T17:34:30.1422700Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-07T17:34:30.2409591Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-07T17:34:30.5739344Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-07T17:34:30.5775332Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-07T17:34:30.5777125Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-07T17:34:30.5802819Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-07T17:34:31.0429574Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-07T17:34:31.9751156Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-07T17:34:32.0066861Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-07T17:34:32.0702727Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-07T17:34:32.0719741Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:34:32.0721709Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:34:32.0723890Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:34:32.0726352Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:34:32.0729009Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:34:32.0731539Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:34:32.0734158Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:34:32.0736759Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:34:32.0760777Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-07T17:34:32.4890491Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-07T17:34:33.6322774Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-07T17:34:33.9591130Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-07T17:34:40.2501195Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-07T17:34:40.5415611Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-07T17:34:40.7407542Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-07T17:34:41.0347145Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-07T17:34:48.3710502Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-07T17:34:48.4102113Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-07T17:34:48.4256673Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-07T17:34:48.5204385Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-07T17:34:48.5323264Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-07T17:34:48.5458966Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-07T17:34:48.5601341Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-07T17:34:48.5614696Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:34:48.5617128Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:34:48.5641082Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-07T17:34:50.7318685Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-07T17:34:52.1315548Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-07T17:34:52.1756041Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-07T17:34:52.6184674Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-07T17:34:52.6295400Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-07T17:34:52.8834721Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-07T17:34:52.8857262Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:34:52.8858884Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-07T17:34:52.8883921Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-07T17:34:53.3264903Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-07T17:34:54.4973579Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-07T17:34:54.5664259Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-07T17:34:54.5753403Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-07T17:34:54.5869770Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-07T17:34:54.6219370Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-07T17:34:54.6491797Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-07T17:34:54.6893235Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-07T17:34:54.7142334Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-07T17:34:54.7157253Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:34:54.7159338Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:34:54.7161736Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:34:54.7164917Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:34:54.7188435Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-07T17:34:55.9377849Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-07T17:34:56.1747356Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-07T17:34:57.3647975Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-07T17:34:58.2793509Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-07T17:34:58.2945106Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-07T17:34:58.3613608Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-07T17:34:58.3893251Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-07T17:34:58.3908818Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:34:58.3934592Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-07T17:34:58.7605107Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-07T17:34:58.7638653Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-07T17:34:58.7920643Z Entering 'android/libs/fbjni' 2024-08-07T17:34:58.7959938Z Entering 'third_party/FP16' 2024-08-07T17:34:58.7998585Z Entering 'third_party/FXdiv' 2024-08-07T17:34:58.8036255Z Entering 'third_party/NNPACK' 2024-08-07T17:34:58.8075348Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:34:58.8113430Z Entering 'third_party/XNNPACK' 2024-08-07T17:34:58.8169958Z Entering 'third_party/benchmark' 2024-08-07T17:34:58.8209888Z Entering 'third_party/cpp-httplib' 2024-08-07T17:34:58.8248510Z Entering 'third_party/cpuinfo' 2024-08-07T17:34:58.8288251Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:34:58.8326708Z Entering 'third_party/cutlass' 2024-08-07T17:34:58.8373354Z Entering 'third_party/eigen' 2024-08-07T17:34:58.8414315Z Entering 'third_party/fbgemm' 2024-08-07T17:34:58.8453411Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:34:58.8491290Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:34:58.8528391Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:34:58.8574125Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:34:58.8611148Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:34:58.8650268Z Entering 'third_party/flatbuffers' 2024-08-07T17:34:58.8694577Z Entering 'third_party/fmt' 2024-08-07T17:34:58.8733190Z Entering 'third_party/foxi' 2024-08-07T17:34:58.8771791Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:34:58.8811158Z Entering 'third_party/gloo' 2024-08-07T17:34:58.8850179Z Entering 'third_party/googletest' 2024-08-07T17:34:58.8891876Z Entering 'third_party/ideep' 2024-08-07T17:34:58.8930093Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:34:58.8975454Z Entering 'third_party/ittapi' 2024-08-07T17:34:58.9014104Z Entering 'third_party/kineto' 2024-08-07T17:34:58.9052016Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:34:58.9091738Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:34:58.9130778Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:34:58.9169534Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:34:58.9208926Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:34:58.9246361Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:34:58.9286830Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:34:58.9324955Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:34:58.9364047Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:34:58.9403767Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:34:58.9443609Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:34:58.9483025Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:34:58.9523188Z Entering 'third_party/mimalloc' 2024-08-07T17:34:58.9562710Z Entering 'third_party/nccl/nccl' 2024-08-07T17:34:58.9601091Z Entering 'third_party/nlohmann' 2024-08-07T17:34:58.9640811Z Entering 'third_party/onnx' 2024-08-07T17:34:58.9695981Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:34:58.9733816Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:34:58.9773918Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:34:58.9814821Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:34:58.9851895Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:34:58.9889879Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:34:58.9926326Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:34:58.9964525Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:34:59.0002331Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:34:59.0039054Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:34:59.0076280Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:34:59.0114925Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:34:59.0155604Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:34:59.0213490Z Entering 'third_party/pocketfft' 2024-08-07T17:34:59.0252649Z Entering 'third_party/protobuf' 2024-08-07T17:34:59.0296070Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:34:59.0333545Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:34:59.0374411Z Entering 'third_party/psimd' 2024-08-07T17:34:59.0413028Z Entering 'third_party/pthreadpool' 2024-08-07T17:34:59.0450903Z Entering 'third_party/pybind11' 2024-08-07T17:34:59.0489675Z Entering 'third_party/python-peachpy' 2024-08-07T17:34:59.0528036Z Entering 'third_party/sleef' 2024-08-07T17:34:59.0566897Z Entering 'third_party/tensorpipe' 2024-08-07T17:34:59.0605441Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:34:59.0643178Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:34:59.0680555Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:34:59.0718149Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:34:59.0754356Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:34:59.0808404Z ##[endgroup] 2024-08-07T17:34:59.0810203Z ##[group]Persisting credentials for submodules 2024-08-07T17:34:59.0815886Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-07T17:34:59.1098320Z Entering 'android/libs/fbjni' 2024-08-07T17:34:59.1149334Z Entering 'third_party/FP16' 2024-08-07T17:34:59.1199471Z Entering 'third_party/FXdiv' 2024-08-07T17:34:59.1249832Z Entering 'third_party/NNPACK' 2024-08-07T17:34:59.1300509Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:34:59.1350780Z Entering 'third_party/XNNPACK' 2024-08-07T17:34:59.1418541Z Entering 'third_party/benchmark' 2024-08-07T17:34:59.1469437Z Entering 'third_party/cpp-httplib' 2024-08-07T17:34:59.1519829Z Entering 'third_party/cpuinfo' 2024-08-07T17:34:59.1570203Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:34:59.1620342Z Entering 'third_party/cutlass' 2024-08-07T17:34:59.1679747Z Entering 'third_party/eigen' 2024-08-07T17:34:59.1731916Z Entering 'third_party/fbgemm' 2024-08-07T17:34:59.1782366Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:34:59.1831017Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:34:59.1878652Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:34:59.1934303Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:34:59.1983321Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:34:59.2033255Z Entering 'third_party/flatbuffers' 2024-08-07T17:34:59.2088991Z Entering 'third_party/fmt' 2024-08-07T17:34:59.2138432Z Entering 'third_party/foxi' 2024-08-07T17:34:59.2188667Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:34:59.2239422Z Entering 'third_party/gloo' 2024-08-07T17:34:59.2289793Z Entering 'third_party/googletest' 2024-08-07T17:34:59.2340236Z Entering 'third_party/ideep' 2024-08-07T17:34:59.2391356Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:34:59.2447880Z Entering 'third_party/ittapi' 2024-08-07T17:34:59.2500424Z Entering 'third_party/kineto' 2024-08-07T17:34:59.2550049Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:34:59.2600056Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:34:59.2650084Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:34:59.2699354Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:34:59.2747884Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:34:59.2801473Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:34:59.2852079Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:34:59.2902139Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:34:59.2951830Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:34:59.3002878Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:34:59.3054127Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:34:59.3103877Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:34:59.3154413Z Entering 'third_party/mimalloc' 2024-08-07T17:34:59.3205850Z Entering 'third_party/nccl/nccl' 2024-08-07T17:34:59.3256265Z Entering 'third_party/nlohmann' 2024-08-07T17:34:59.3308225Z Entering 'third_party/onnx' 2024-08-07T17:34:59.3375640Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:34:59.3426567Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:34:59.3478903Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:34:59.3530635Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:34:59.3580670Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:34:59.3629265Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:34:59.3678812Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:34:59.3729721Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:34:59.3777783Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:34:59.3826851Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:34:59.3874770Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:34:59.3927179Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:34:59.3978937Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:34:59.4048593Z Entering 'third_party/pocketfft' 2024-08-07T17:34:59.4100851Z Entering 'third_party/protobuf' 2024-08-07T17:34:59.4153518Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:34:59.4204027Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:34:59.4258339Z Entering 'third_party/psimd' 2024-08-07T17:34:59.4309003Z Entering 'third_party/pthreadpool' 2024-08-07T17:34:59.4359478Z Entering 'third_party/pybind11' 2024-08-07T17:34:59.4412950Z Entering 'third_party/python-peachpy' 2024-08-07T17:34:59.4463123Z Entering 'third_party/sleef' 2024-08-07T17:34:59.4514285Z Entering 'third_party/tensorpipe' 2024-08-07T17:34:59.4564329Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:34:59.4613475Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:34:59.4662346Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:34:59.4713266Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:34:59.4765121Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:34:59.4834375Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-07T17:34:59.5117897Z Entering 'android/libs/fbjni' 2024-08-07T17:34:59.5165447Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-07T17:34:59.5181258Z Entering 'third_party/FP16' 2024-08-07T17:34:59.5227484Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-07T17:34:59.5241403Z Entering 'third_party/FXdiv' 2024-08-07T17:34:59.5287947Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-07T17:34:59.5301828Z Entering 'third_party/NNPACK' 2024-08-07T17:34:59.5348096Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-07T17:34:59.5362770Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:34:59.5408852Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-07T17:34:59.5423062Z Entering 'third_party/XNNPACK' 2024-08-07T17:34:59.5470432Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-07T17:34:59.5502938Z Entering 'third_party/benchmark' 2024-08-07T17:34:59.5549559Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:34:59.5565121Z Entering 'third_party/cpp-httplib' 2024-08-07T17:34:59.5611275Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-07T17:34:59.5625958Z Entering 'third_party/cpuinfo' 2024-08-07T17:34:59.5672393Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-07T17:34:59.5688291Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:34:59.5734431Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-07T17:34:59.5748923Z Entering 'third_party/cutlass' 2024-08-07T17:34:59.5797186Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-07T17:34:59.5819657Z Entering 'third_party/eigen' 2024-08-07T17:34:59.5865677Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-07T17:34:59.5882735Z Entering 'third_party/fbgemm' 2024-08-07T17:34:59.5931106Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-07T17:34:59.5945786Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:34:59.5994880Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-07T17:34:59.6009770Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:34:59.6055844Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-07T17:34:59.6072111Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:34:59.6117992Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-07T17:34:59.6139349Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:34:59.6187725Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:34:59.6200298Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:34:59.6246255Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-07T17:34:59.6262121Z Entering 'third_party/flatbuffers' 2024-08-07T17:34:59.6309135Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-07T17:34:59.6328221Z Entering 'third_party/fmt' 2024-08-07T17:34:59.6375399Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-07T17:34:59.6391201Z Entering 'third_party/foxi' 2024-08-07T17:34:59.6438374Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-08-07T17:34:59.6453686Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:34:59.6500867Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-07T17:34:59.6516238Z Entering 'third_party/gloo' 2024-08-07T17:34:59.6561214Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-07T17:34:59.6576483Z Entering 'third_party/googletest' 2024-08-07T17:34:59.6621608Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:34:59.6636796Z Entering 'third_party/ideep' 2024-08-07T17:34:59.6683113Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-07T17:34:59.6696785Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:34:59.6742451Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-07T17:34:59.6765240Z Entering 'third_party/ittapi' 2024-08-07T17:34:59.6811975Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-07T17:34:59.6827049Z Entering 'third_party/kineto' 2024-08-07T17:34:59.6873798Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-07T17:34:59.6889096Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:34:59.6936454Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-07T17:34:59.6950512Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:34:59.6998046Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-07T17:34:59.7014171Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:34:59.7061114Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-07T17:34:59.7075248Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:34:59.7121407Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-07T17:34:59.7135114Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:34:59.7182077Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-07T17:34:59.7195970Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:34:59.7242735Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-07T17:34:59.7259308Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:34:59.7306397Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-07T17:34:59.7321290Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:34:59.7369210Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:34:59.7384068Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:34:59.7430517Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-07T17:34:59.7446208Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:34:59.7492936Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-07T17:34:59.7509453Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:34:59.7555563Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-07T17:34:59.7570173Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:34:59.7615948Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-07T17:34:59.7632612Z Entering 'third_party/mimalloc' 2024-08-07T17:34:59.7681583Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-07T17:34:59.7697769Z Entering 'third_party/nccl/nccl' 2024-08-07T17:34:59.7744672Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-07T17:34:59.7760808Z Entering 'third_party/nlohmann' 2024-08-07T17:34:59.7807534Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-07T17:34:59.7823873Z Entering 'third_party/onnx' 2024-08-07T17:34:59.7870524Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-07T17:34:59.7902063Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:34:59.7948855Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:34:59.7963706Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:34:59.8010722Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-07T17:34:59.8028495Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:34:59.8079933Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-07T17:34:59.8096911Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:34:59.8142345Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:34:59.8157125Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:34:59.8204118Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:34:59.8218415Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:34:59.8264375Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-07T17:34:59.8278889Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:34:59.8325026Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-07T17:34:59.8341520Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:34:59.8387576Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-07T17:34:59.8401813Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:34:59.8447050Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-07T17:34:59.8461191Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:34:59.8508366Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-07T17:34:59.8521982Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:34:59.8568099Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-07T17:34:59.8584677Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:34:59.8630782Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-07T17:34:59.8647149Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:34:59.8694050Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-07T17:34:59.8728821Z Entering 'third_party/pocketfft' 2024-08-07T17:34:59.8775821Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-07T17:34:59.8791623Z Entering 'third_party/protobuf' 2024-08-07T17:34:59.8839457Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-07T17:34:59.8858361Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:34:59.8905233Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:34:59.8919705Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:34:59.8966758Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:34:59.8983578Z Entering 'third_party/psimd' 2024-08-07T17:34:59.9029755Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-07T17:34:59.9045104Z Entering 'third_party/pthreadpool' 2024-08-07T17:34:59.9091441Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-07T17:34:59.9106609Z Entering 'third_party/pybind11' 2024-08-07T17:34:59.9152739Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-07T17:34:59.9168796Z Entering 'third_party/python-peachpy' 2024-08-07T17:34:59.9214742Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-07T17:34:59.9230093Z Entering 'third_party/sleef' 2024-08-07T17:34:59.9276667Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-07T17:34:59.9291956Z Entering 'third_party/tensorpipe' 2024-08-07T17:34:59.9337969Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-07T17:34:59.9353040Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:34:59.9400369Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:34:59.9415046Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:34:59.9462687Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-07T17:34:59.9478722Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:34:59.9524044Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-07T17:34:59.9538767Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:34:59.9585583Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-07T17:34:59.9599461Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:34:59.9646146Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-07T17:35:00.0392504Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-07T17:35:00.0677970Z Entering 'android/libs/fbjni' 2024-08-07T17:35:00.0717454Z Entering 'third_party/FP16' 2024-08-07T17:35:00.0756153Z Entering 'third_party/FXdiv' 2024-08-07T17:35:00.0795501Z Entering 'third_party/NNPACK' 2024-08-07T17:35:00.0833903Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:35:00.0873601Z Entering 'third_party/XNNPACK' 2024-08-07T17:35:00.0928839Z Entering 'third_party/benchmark' 2024-08-07T17:35:00.0968139Z Entering 'third_party/cpp-httplib' 2024-08-07T17:35:00.1006168Z Entering 'third_party/cpuinfo' 2024-08-07T17:35:00.1044453Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:35:00.1084185Z Entering 'third_party/cutlass' 2024-08-07T17:35:00.1130164Z Entering 'third_party/eigen' 2024-08-07T17:35:00.1170430Z Entering 'third_party/fbgemm' 2024-08-07T17:35:00.1208816Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:35:00.1246443Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:35:00.1284462Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:35:00.1328450Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:35:00.1367173Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:35:00.1406677Z Entering 'third_party/flatbuffers' 2024-08-07T17:35:00.1447660Z Entering 'third_party/fmt' 2024-08-07T17:35:00.1486773Z Entering 'third_party/foxi' 2024-08-07T17:35:00.1524678Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:35:00.1563189Z Entering 'third_party/gloo' 2024-08-07T17:35:00.1602449Z Entering 'third_party/googletest' 2024-08-07T17:35:00.1640436Z Entering 'third_party/ideep' 2024-08-07T17:35:00.1678371Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:35:00.1726376Z Entering 'third_party/ittapi' 2024-08-07T17:35:00.1766969Z Entering 'third_party/kineto' 2024-08-07T17:35:00.1807052Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:35:00.1845116Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:35:00.1884551Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:35:00.1921944Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:35:00.1960707Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:35:00.1997887Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:35:00.2037563Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:35:00.2075667Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:35:00.2113669Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:35:00.2152568Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:35:00.2192435Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:35:00.2230794Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:35:00.2270351Z Entering 'third_party/mimalloc' 2024-08-07T17:35:00.2309925Z Entering 'third_party/nccl/nccl' 2024-08-07T17:35:00.2349466Z Entering 'third_party/nlohmann' 2024-08-07T17:35:00.2392568Z Entering 'third_party/onnx' 2024-08-07T17:35:00.2447369Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:35:00.2486173Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:35:00.2526962Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:35:00.2568534Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:35:00.2605745Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:35:00.2642870Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:35:00.2680753Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:35:00.2720354Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:35:00.2757566Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:35:00.2795760Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:35:00.2832800Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:35:00.2873389Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:35:00.2913324Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:35:00.2971560Z Entering 'third_party/pocketfft' 2024-08-07T17:35:00.3012221Z Entering 'third_party/protobuf' 2024-08-07T17:35:00.3054214Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:35:00.3092111Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:35:00.3130963Z Entering 'third_party/psimd' 2024-08-07T17:35:00.3170176Z Entering 'third_party/pthreadpool' 2024-08-07T17:35:00.3209833Z Entering 'third_party/pybind11' 2024-08-07T17:35:00.3248110Z Entering 'third_party/python-peachpy' 2024-08-07T17:35:00.3287809Z Entering 'third_party/sleef' 2024-08-07T17:35:00.3326812Z Entering 'third_party/tensorpipe' 2024-08-07T17:35:00.3365622Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:35:00.3403390Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:35:00.3440912Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:35:00.3478354Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:35:00.3515824Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:35:00.3570998Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-07T17:35:00.3849479Z Entering 'android/libs/fbjni' 2024-08-07T17:35:00.3888122Z Entering 'third_party/FP16' 2024-08-07T17:35:00.3927174Z Entering 'third_party/FXdiv' 2024-08-07T17:35:00.3965575Z Entering 'third_party/NNPACK' 2024-08-07T17:35:00.4004719Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:35:00.4043664Z Entering 'third_party/XNNPACK' 2024-08-07T17:35:00.4099811Z Entering 'third_party/benchmark' 2024-08-07T17:35:00.4138641Z Entering 'third_party/cpp-httplib' 2024-08-07T17:35:00.4177525Z Entering 'third_party/cpuinfo' 2024-08-07T17:35:00.4216186Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:35:00.4254416Z Entering 'third_party/cutlass' 2024-08-07T17:35:00.4302871Z Entering 'third_party/eigen' 2024-08-07T17:35:00.4343514Z Entering 'third_party/fbgemm' 2024-08-07T17:35:00.4382975Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:35:00.4422383Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:35:00.4459827Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:35:00.4503581Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:35:00.4540531Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:35:00.4579979Z Entering 'third_party/flatbuffers' 2024-08-07T17:35:00.4621075Z Entering 'third_party/fmt' 2024-08-07T17:35:00.4659496Z Entering 'third_party/foxi' 2024-08-07T17:35:00.4698384Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:35:00.4737373Z Entering 'third_party/gloo' 2024-08-07T17:35:00.4775997Z Entering 'third_party/googletest' 2024-08-07T17:35:00.4814772Z Entering 'third_party/ideep' 2024-08-07T17:35:00.4852553Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:35:00.4898951Z Entering 'third_party/ittapi' 2024-08-07T17:35:00.4936982Z Entering 'third_party/kineto' 2024-08-07T17:35:00.4975382Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:35:00.5013585Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:35:00.5053491Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:35:00.5092444Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:35:00.5129917Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:35:00.5170128Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:35:00.5210802Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:35:00.5247960Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:35:00.5286105Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:35:00.5324795Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:35:00.5364694Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:35:00.5402821Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:35:00.5443051Z Entering 'third_party/mimalloc' 2024-08-07T17:35:00.5482730Z Entering 'third_party/nccl/nccl' 2024-08-07T17:35:00.5521894Z Entering 'third_party/nlohmann' 2024-08-07T17:35:00.5561624Z Entering 'third_party/onnx' 2024-08-07T17:35:00.5616318Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:35:00.5654793Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:35:00.5696164Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:35:00.5737526Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:35:00.5775481Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:35:00.5812862Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:35:00.5850034Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:35:00.5888198Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:35:00.5924817Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:35:00.5961656Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:35:00.5999106Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:35:00.6038984Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:35:00.6078219Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:35:00.6135338Z Entering 'third_party/pocketfft' 2024-08-07T17:35:00.6176834Z Entering 'third_party/protobuf' 2024-08-07T17:35:00.6219568Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:35:00.6256915Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:35:00.6297055Z Entering 'third_party/psimd' 2024-08-07T17:35:00.6336133Z Entering 'third_party/pthreadpool' 2024-08-07T17:35:00.6374508Z Entering 'third_party/pybind11' 2024-08-07T17:35:00.6413166Z Entering 'third_party/python-peachpy' 2024-08-07T17:35:00.6451252Z Entering 'third_party/sleef' 2024-08-07T17:35:00.6489814Z Entering 'third_party/tensorpipe' 2024-08-07T17:35:00.6528767Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:35:00.6566555Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:35:00.6603461Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:35:00.6640072Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:35:00.6678265Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:35:00.6729604Z ##[endgroup] 2024-08-07T17:35:00.6771462Z [command]/usr/bin/git log -1 --format='%H' 2024-08-07T17:35:00.6799467Z '016588f53c6904b840aa56aa86f95460b4d9c996' 2024-08-07T17:35:00.6946102Z Prepare all required actions 2024-08-07T17:35:00.6946717Z Getting action download info 2024-08-07T17:35:00.8317396Z ##[group]Run ./.github/actions/setup-linux 2024-08-07T17:35:00.8317772Z env: 2024-08-07T17:35:00.8317989Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:00.8318265Z ##[endgroup] 2024-08-07T17:35:00.8373784Z ##[group]Run set -euo pipefail 2024-08-07T17:35:00.8374152Z set -euo pipefail 2024-08-07T17:35:00.8374451Z function get_ec2_metadata() { 2024-08-07T17:35:00.8374851Z  # Pulled from instance metadata endpoint for EC2 2024-08-07T17:35:00.8375541Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-07T17:35:00.8376116Z  category=$1 2024-08-07T17:35:00.8376485Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-07T17:35:00.8376937Z  runner_name_str=i-036a6435dcb00f35e 2024-08-07T17:35:00.8377333Z  if [[ -f /.inarc ]]; then 2024-08-07T17:35:00.8377696Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-07T17:35:00.8378087Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-07T17:35:00.8378570Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-07T17:35:00.8379019Z  else 2024-08-07T17:35:00.8379375Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-07T17:35:00.8379804Z  fi 2024-08-07T17:35:00.8380029Z } 2024-08-07T17:35:00.8380292Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-07T17:35:00.8380735Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-07T17:35:00.8381229Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-07T17:35:00.8381644Z echo "system info $(uname -a)" 2024-08-07T17:35:00.8389611Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:00.8390013Z env: 2024-08-07T17:35:00.8390308Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:00.8390582Z ##[endgroup] 2024-08-07T17:35:00.8481055Z ami-id: ami-06c68f701d8090592 2024-08-07T17:35:00.8532476Z instance-id: i-036a6435dcb00f35e 2024-08-07T17:35:00.8581175Z instance-type: c5.2xlarge 2024-08-07T17:35:00.8591157Z system info Linux ip-10-0-41-45.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-07T17:35:00.8622820Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-07T17:35:00.8623530Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-07T17:35:00.8629564Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:00.8629958Z env: 2024-08-07T17:35:00.8630183Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:00.8630445Z ##[endgroup] 2024-08-07T17:35:00.8703867Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-07T17:35:00.8704360Z if systemctl is-active --quiet docker; then 2024-08-07T17:35:00.8704768Z  echo "Docker daemon is running..."; 2024-08-07T17:35:00.8705099Z else 2024-08-07T17:35:00.8705471Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-07T17:35:00.8705917Z fi 2024-08-07T17:35:00.8711290Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:00.8711685Z env: 2024-08-07T17:35:00.8711914Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:00.8712179Z ##[endgroup] 2024-08-07T17:35:00.8781640Z Docker daemon is running... 2024-08-07T17:35:00.8834811Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-07T17:35:00.8835243Z with: 2024-08-07T17:35:00.8835465Z shell: bash 2024-08-07T17:35:00.8835704Z timeout_minutes: 5 2024-08-07T17:35:00.8835948Z max_attempts: 3 2024-08-07T17:35:00.8836203Z retry_wait_seconds: 30 2024-08-07T17:35:00.8837506Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-08-07T17:35:00.8838627Z polling_interval_seconds: 1 2024-08-07T17:35:00.8838966Z warning_on_retry: true 2024-08-07T17:35:00.8839241Z continue_on_error: false 2024-08-07T17:35:00.8839508Z env: 2024-08-07T17:35:00.8839726Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:00.8840009Z AWS_RETRY_MODE: standard 2024-08-07T17:35:00.8840284Z AWS_MAX_ATTEMPTS: 5 2024-08-07T17:35:00.8840557Z AWS_DEFAULT_REGION: us-east-1 2024-08-07T17:35:00.8840845Z ##[endgroup] 2024-08-07T17:35:02.0725142Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-07T17:35:02.0725796Z Configure a credential helper to remove this warning. See 2024-08-07T17:35:02.0726396Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-07T17:35:02.0726793Z 2024-08-07T17:35:02.0726920Z Login Succeeded 2024-08-07T17:35:02.9401239Z Command completed after 1 attempt(s). 2024-08-07T17:35:02.9459690Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-07T17:35:02.9460254Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-07T17:35:02.9460720Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-07T17:35:02.9467365Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:02.9467766Z env: 2024-08-07T17:35:02.9467994Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:02.9468286Z ##[endgroup] 2024-08-07T17:35:02.9553873Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T17:35:02.9554585Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T17:35:02.9555020Z # shellcheck disable=SC2046 2024-08-07T17:35:02.9555377Z docker stop $(docker ps -q) || true 2024-08-07T17:35:02.9555759Z # Prune all of the docker images 2024-08-07T17:35:02.9556090Z docker system prune -af 2024-08-07T17:35:02.9561432Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:02.9561830Z env: 2024-08-07T17:35:02.9562049Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:02.9562320Z ##[endgroup] 2024-08-07T17:35:02.9840694Z "docker stop" requires at least 1 argument. 2024-08-07T17:35:02.9841128Z See 'docker stop --help'. 2024-08-07T17:35:02.9841309Z 2024-08-07T17:35:02.9841480Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-07T17:35:02.9841772Z 2024-08-07T17:35:02.9841916Z Stop one or more running containers 2024-08-07T17:35:03.0000811Z Total reclaimed space: 0B 2024-08-07T17:35:03.0071521Z ##[group]Run set +e 2024-08-07T17:35:03.0071824Z set +e 2024-08-07T17:35:03.0072074Z set -x 2024-08-07T17:35:03.0072280Z  2024-08-07T17:35:03.0072526Z PT_DOMAIN=download.pytorch.org 2024-08-07T17:35:03.0073133Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-07T17:35:03.0073901Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-07T17:35:03.0074531Z # one is returned at random 2024-08-07T17:35:03.0074940Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-07T17:35:03.0075321Z  2024-08-07T17:35:03.0075534Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-07T17:35:03.0075958Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-07T17:35:03.0076638Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-07T17:35:03.0077016Z  2024-08-07T17:35:03.0077242Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-07T17:35:03.0077623Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-07T17:35:03.0077975Z  exit 1 2024-08-07T17:35:03.0078221Z  fi 2024-08-07T17:35:03.0078562Z fi 2024-08-07T17:35:03.0078769Z  2024-08-07T17:35:03.0079125Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-07T17:35:03.0079504Z  # Clean up any old records first 2024-08-07T17:35:03.0079869Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-07T17:35:03.0080210Z fi 2024-08-07T17:35:03.0080429Z  2024-08-07T17:35:03.0080738Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-07T17:35:03.0081155Z cat /etc/hosts 2024-08-07T17:35:03.0086896Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:03.0087284Z env: 2024-08-07T17:35:03.0087510Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:03.0087783Z ##[endgroup] 2024-08-07T17:35:03.0110264Z + PT_DOMAIN=download.pytorch.org 2024-08-07T17:35:03.0115982Z ++ dig -4 +short download.pytorch.org 2024-08-07T17:35:03.0116373Z ++ tail -n1 2024-08-07T17:35:03.0641658Z + RESOLVED_IP=18.160.10.28 2024-08-07T17:35:03.0642220Z + '[' -z 18.160.10.28 ']' 2024-08-07T17:35:03.0642530Z + grep -r download.pytorch.org /etc/hosts 2024-08-07T17:35:03.0653333Z 3.162.125.77 download.pytorch.org 2024-08-07T17:35:03.0654035Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-08-07T17:35:03.2991884Z + echo '18.160.10.28 download.pytorch.org' 2024-08-07T17:35:03.2992556Z + sudo tee -a /etc/hosts 2024-08-07T17:35:03.3397280Z 18.160.10.28 download.pytorch.org 2024-08-07T17:35:03.3411412Z + cat /etc/hosts 2024-08-07T17:35:03.3421075Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-07T17:35:03.3427984Z ::1 localhost6 localhost6.localdomain6 2024-08-07T17:35:03.3428392Z 18.160.10.28 download.pytorch.org 2024-08-07T17:35:03.3567504Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-07T17:35:03.3567987Z with: 2024-08-07T17:35:03.3568660Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:03.3569446Z docker-build-dir: .ci/docker 2024-08-07T17:35:03.3569747Z working-directory: . 2024-08-07T17:35:03.3570108Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:03.3570505Z force-push: false 2024-08-07T17:35:03.3570744Z env: 2024-08-07T17:35:03.3570954Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:03.3571226Z ##[endgroup] 2024-08-07T17:35:03.3597058Z ##[group]Run set -ex 2024-08-07T17:35:03.3597377Z set -ex 2024-08-07T17:35:03.3597617Z  2024-08-07T17:35:03.3598028Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-07T17:35:03.3598766Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-07T17:35:03.3599343Z # job could then download the pre-built image as usual 2024-08-07T17:35:03.3599871Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-07T17:35:03.3600367Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:03.3600833Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:03.3601244Z  2024-08-07T17:35:03.3601615Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-07T17:35:03.3602073Z  exit 0 2024-08-07T17:35:03.3602298Z else 2024-08-07T17:35:03.3602572Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:03.3602918Z fi 2024-08-07T17:35:03.3603126Z  2024-08-07T17:35:03.3603621Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-07T17:35:03.3604251Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-07T17:35:03.3604792Z  # use it as it is, but first let's extract the tag 2024-08-07T17:35:03.3605294Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-07T17:35:03.3605820Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:03.3606316Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:03.3606739Z else 2024-08-07T17:35:03.3607064Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-07T17:35:03.3607530Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:03.3608191Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:03.3608773Z fi 2024-08-07T17:35:03.3616524Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:03.3616908Z env: 2024-08-07T17:35:03.3617141Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:03.3617414Z REPO_NAME: pytorch 2024-08-07T17:35:03.3618131Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:03.3618918Z DOCKER_BUILD_DIR: .ci/docker 2024-08-07T17:35:03.3619303Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:03.3619691Z ##[endgroup] 2024-08-07T17:35:03.3643239Z + [[ ! -d .ci/docker ]] 2024-08-07T17:35:03.3643546Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-07T17:35:03.3643849Z + echo skip=false 2024-08-07T17:35:03.3645174Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-07T17:35:03.3650462Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:03.3651208Z ++ awk -F '[:,]' '{print $2}' 2024-08-07T17:35:03.3697414Z + DOCKER_TAG=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:03.3698218Z + echo docker-tag=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:03.3699639Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:03.3733350Z ##[group]Run set +e 2024-08-07T17:35:03.3733684Z set +e 2024-08-07T17:35:03.3733919Z set -x 2024-08-07T17:35:03.3734157Z  2024-08-07T17:35:03.3734382Z login() { 2024-08-07T17:35:03.3734863Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-07T17:35:03.3735404Z } 2024-08-07T17:35:03.3735632Z  2024-08-07T17:35:03.3735839Z retry () { 2024-08-07T17:35:03.3736127Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-07T17:35:03.3736457Z } 2024-08-07T17:35:03.3736663Z  2024-08-07T17:35:03.3736905Z retry login "${DOCKER_REGISTRY}" 2024-08-07T17:35:03.3737211Z  2024-08-07T17:35:03.3737557Z # Check if image already exists, if it does then skip building it 2024-08-07T17:35:03.3738067Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-07T17:35:03.3738442Z  exit 0 2024-08-07T17:35:03.3738665Z fi 2024-08-07T17:35:03.3738881Z  2024-08-07T17:35:03.3739231Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-07T17:35:03.3739817Z # be empty. The default action would be to continue rebuild the image 2024-08-07T17:35:03.3740345Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-07T17:35:03.3740956Z  # if we're on the base branch then use the parent commit 2024-08-07T17:35:03.3741362Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-07T17:35:03.3741692Z else 2024-08-07T17:35:03.3742030Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-07T17:35:03.3742504Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-07T17:35:03.3742878Z fi 2024-08-07T17:35:03.3743097Z  2024-08-07T17:35:03.3743326Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-07T17:35:03.3743699Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:03.3744044Z  2024-08-07T17:35:03.3744511Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-07T17:35:03.3745077Z  exit 0 2024-08-07T17:35:03.3745311Z fi 2024-08-07T17:35:03.3745563Z  2024-08-07T17:35:03.3745877Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-07T17:35:03.3746586Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-07T17:35:03.3747177Z  exit 1 2024-08-07T17:35:03.3747409Z fi 2024-08-07T17:35:03.3747626Z  2024-08-07T17:35:03.3748001Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-07T17:35:03.3748673Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-07T17:35:03.3749281Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-07T17:35:03.3749978Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-07T17:35:03.3750756Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-07T17:35:03.3751230Z fi 2024-08-07T17:35:03.3751456Z  2024-08-07T17:35:03.3751715Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:03.3757652Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:03.3758096Z env: 2024-08-07T17:35:03.3758315Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:03.3758600Z DOCKER_BUILD_DIR: .ci/docker 2024-08-07T17:35:03.3758949Z BASE_REVISION: 6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T17:35:03.3759740Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:03.3760549Z DOCKER_TAG: 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:03.3761014Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:03.3761398Z ##[endgroup] 2024-08-07T17:35:03.3784080Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:03.3784758Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:03.3786720Z + aws ecr get-login-password --region us-east-1 2024-08-07T17:35:03.3787824Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:03.9425769Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-07T17:35:03.9426467Z Configure a credential helper to remove this warning. See 2024-08-07T17:35:03.9427223Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-07T17:35:03.9427646Z 2024-08-07T17:35:03.9427748Z Login Succeeded 2024-08-07T17:35:03.9439498Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:04.1722812Z { 2024-08-07T17:35:04.1723203Z "schemaVersion": 2, 2024-08-07T17:35:04.1723675Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-07T17:35:04.1724132Z "config": { 2024-08-07T17:35:04.1724617Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-07T17:35:04.1725601Z "size": 43802, 2024-08-07T17:35:04.1726307Z "digest": "sha256:eb2e8636b63e804e38c5e56190d8b860c4bf8a9e0794f88825eff958ed6b1e76" 2024-08-07T17:35:04.1727017Z }, 2024-08-07T17:35:04.1727319Z "layers": [ 2024-08-07T17:35:04.1727633Z { 2024-08-07T17:35:04.1728100Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1728725Z "size": 30439866, 2024-08-07T17:35:04.1729166Z "digest": "sha256:9b857f539cb142c9aa2201a17bb8e1cd5cf12edd4a65adf5732fe9f4343964cf" 2024-08-07T17:35:04.1729646Z }, 2024-08-07T17:35:04.1729853Z { 2024-08-07T17:35:04.1730201Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1730626Z "size": 1820, 2024-08-07T17:35:04.1731057Z "digest": "sha256:ceceb076946ee95300822daccb5d3e4e45e6857d34996ef89305d9a078410f8e" 2024-08-07T17:35:04.1731545Z }, 2024-08-07T17:35:04.1731791Z { 2024-08-07T17:35:04.1732131Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1732567Z "size": 313165371, 2024-08-07T17:35:04.1732990Z "digest": "sha256:042461771f00f834b40891f079173616ad761e9e3b2ec2eb25affa3041d71a3d" 2024-08-07T17:35:04.1733457Z }, 2024-08-07T17:35:04.1733661Z { 2024-08-07T17:35:04.1733992Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1734420Z "size": 802, 2024-08-07T17:35:04.1734848Z "digest": "sha256:bac001ac2fee5df3367ff9332bc1fcd706a8de3ad05a1885903f338197619520" 2024-08-07T17:35:04.1735325Z }, 2024-08-07T17:35:04.1735531Z { 2024-08-07T17:35:04.1735869Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1736320Z "size": 106, 2024-08-07T17:35:04.1736739Z "digest": "sha256:4a587220513dc642f8fe95f4ae1b2b643225db8ae8753fa25b68a4f84488c4ab" 2024-08-07T17:35:04.1737216Z }, 2024-08-07T17:35:04.1737434Z { 2024-08-07T17:35:04.1737771Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1738208Z "size": 542, 2024-08-07T17:35:04.1738795Z "digest": "sha256:af8081143f008c5cb1bd61f6d92fde22553cfc007b6a77c2fd2aaf6a9f0088a9" 2024-08-07T17:35:04.1739311Z }, 2024-08-07T17:35:04.1739504Z { 2024-08-07T17:35:04.1739852Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1740288Z "size": 1242, 2024-08-07T17:35:04.1740897Z "digest": "sha256:e7a423dcab0171df0317d2cf65153a533a002a33dce81ae5b0972b06d0838e26" 2024-08-07T17:35:04.1741718Z }, 2024-08-07T17:35:04.1742105Z { 2024-08-07T17:35:04.1742532Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1742961Z "size": 483, 2024-08-07T17:35:04.1743371Z "digest": "sha256:ad26542e0d01609840496a12691483f2c50eb8e23002a205becb5b695f9159fb" 2024-08-07T17:35:04.1743860Z }, 2024-08-07T17:35:04.1744049Z { 2024-08-07T17:35:04.1744411Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1744844Z "size": 110321946, 2024-08-07T17:35:04.1745273Z "digest": "sha256:9e1d4f8784a0033c6a3605f0126d93af125fed0dd0839965c97924ba177e350e" 2024-08-07T17:35:04.1745771Z }, 2024-08-07T17:35:04.1745987Z { 2024-08-07T17:35:04.1746318Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1746750Z "size": 3679, 2024-08-07T17:35:04.1747170Z "digest": "sha256:b1f815d162740bbe496657c7b85acd702ea1818a4d93554f8a822bd964920100" 2024-08-07T17:35:04.1747640Z }, 2024-08-07T17:35:04.1747868Z { 2024-08-07T17:35:04.1748210Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1748627Z "size": 1907, 2024-08-07T17:35:04.1749056Z "digest": "sha256:91f9dddffbee46981d9e74382704c1b53c87f1808350cf1dd518687805d3aa59" 2024-08-07T17:35:04.1749536Z }, 2024-08-07T17:35:04.1749726Z { 2024-08-07T17:35:04.1750043Z + exit 0 2024-08-07T17:35:04.1750377Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1750803Z "size": 699, 2024-08-07T17:35:04.1751331Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-07T17:35:04.1751822Z }, 2024-08-07T17:35:04.1752012Z { 2024-08-07T17:35:04.1752352Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1752766Z "size": 2627946145, 2024-08-07T17:35:04.1753202Z "digest": "sha256:8385f3c14a81343c1f329fe1449447035a2acfc7b7e7c552321cded4206f1e1d" 2024-08-07T17:35:04.1753674Z }, 2024-08-07T17:35:04.1753862Z { 2024-08-07T17:35:04.1754317Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1754750Z "size": 32, 2024-08-07T17:35:04.1755156Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:04.1755640Z }, 2024-08-07T17:35:04.1755847Z { 2024-08-07T17:35:04.1756171Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1756599Z "size": 378, 2024-08-07T17:35:04.1757024Z "digest": "sha256:202c71391ade27762d5e9cafde793db67b4d5e4d6869debe2643d5092101c461" 2024-08-07T17:35:04.1757498Z }, 2024-08-07T17:35:04.1757706Z { 2024-08-07T17:35:04.1758064Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1758493Z "size": 222877, 2024-08-07T17:35:04.1758924Z "digest": "sha256:240ee294d88b3c8decdc8b379b5bfae83e2224ed365151fe2e8026fca366d621" 2024-08-07T17:35:04.1759409Z }, 2024-08-07T17:35:04.1759598Z { 2024-08-07T17:35:04.1759934Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1760360Z "size": 231, 2024-08-07T17:35:04.1760770Z "digest": "sha256:7a823d3d7147e0cae46f715bfd2d9ffd953c6cadae532c88c8c857092e6456da" 2024-08-07T17:35:04.1761250Z }, 2024-08-07T17:35:04.1761453Z { 2024-08-07T17:35:04.1761782Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1762210Z "size": 2933351, 2024-08-07T17:35:04.1762646Z "digest": "sha256:a1b9e8856530ee2f7a1abe6b877bf49121fe149f66fbb6ba309902c35f139fb6" 2024-08-07T17:35:04.1763368Z }, 2024-08-07T17:35:04.1763576Z { 2024-08-07T17:35:04.1764060Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1764483Z "size": 1804, 2024-08-07T17:35:04.1764924Z "digest": "sha256:9c5b585a81167e4e7befcddfd54e7b522e6ed96b7c9a9773ef7099c23000ca20" 2024-08-07T17:35:04.1765414Z }, 2024-08-07T17:35:04.1765605Z { 2024-08-07T17:35:04.1765948Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1766364Z "size": 105, 2024-08-07T17:35:04.1766789Z "digest": "sha256:9cd1bcd9f117daa712ab4916cef270b8b4ecf401bed1df3784d55147457c5de5" 2024-08-07T17:35:04.1767279Z }, 2024-08-07T17:35:04.1767469Z { 2024-08-07T17:35:04.1767857Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1768286Z "size": 563, 2024-08-07T17:35:04.1768735Z "digest": "sha256:2b9731d295d60278e361d491fc274e5d558a89ec5adeb9fdf342d7a8cc17bfec" 2024-08-07T17:35:04.1769223Z }, 2024-08-07T17:35:04.1769428Z { 2024-08-07T17:35:04.1769757Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1770183Z "size": 32, 2024-08-07T17:35:04.1770646Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:04.1771119Z }, 2024-08-07T17:35:04.1771317Z { 2024-08-07T17:35:04.1771656Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1772072Z "size": 104, 2024-08-07T17:35:04.1772483Z "digest": "sha256:8d223c48f5429ef9db659237511d2d5f73345f1f217d316591adc008e8abb5a0" 2024-08-07T17:35:04.1772955Z }, 2024-08-07T17:35:04.1773143Z { 2024-08-07T17:35:04.1773480Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1773906Z "size": 502, 2024-08-07T17:35:04.1774358Z "digest": "sha256:49a0e418a175ac12f1b4b503e80e22679bc321e6163ed5b203af848204f0606a" 2024-08-07T17:35:04.1774831Z }, 2024-08-07T17:35:04.1775040Z { 2024-08-07T17:35:04.1775481Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1775912Z "size": 101109806, 2024-08-07T17:35:04.1776344Z "digest": "sha256:51ed474f6c3273503d24a4b965ff3b7e49e8684501e10413499ecc7249337a86" 2024-08-07T17:35:04.1776808Z }, 2024-08-07T17:35:04.1777012Z { 2024-08-07T17:35:04.1777346Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1777761Z "size": 109, 2024-08-07T17:35:04.1778175Z "digest": "sha256:1c01ecb575c64355bc7f0f73f60b8d212e7a83cd58347d573d0074c2a5926b57" 2024-08-07T17:35:04.1778653Z }, 2024-08-07T17:35:04.1778840Z { 2024-08-07T17:35:04.1779176Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1779602Z "size": 487, 2024-08-07T17:35:04.1780016Z "digest": "sha256:a2630e5fe166ae8ff146e4c14db0bdc1299ed2b6120108351c961adcd2375288" 2024-08-07T17:35:04.1780496Z }, 2024-08-07T17:35:04.1780686Z { 2024-08-07T17:35:04.1781028Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1781464Z "size": 340, 2024-08-07T17:35:04.1781871Z "digest": "sha256:554a42b2a2e61462bb2c2c08ed88f895679ad08a1e9295d040c239217102dc70" 2024-08-07T17:35:04.1782348Z }, 2024-08-07T17:35:04.1782551Z { 2024-08-07T17:35:04.1782882Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1783318Z "size": 103, 2024-08-07T17:35:04.1783750Z "digest": "sha256:c9a20a1fda169765309cb8a927a7ad545bddb403e1ed7db164678f1c0c3b5552" 2024-08-07T17:35:04.1784226Z }, 2024-08-07T17:35:04.1784432Z { 2024-08-07T17:35:04.1784834Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1785248Z "size": 1470, 2024-08-07T17:35:04.1785729Z "digest": "sha256:2c2e35b979f5fc0ecccef6efc157a97f33db01a77958fa33855dc367544a5c46" 2024-08-07T17:35:04.1786218Z }, 2024-08-07T17:35:04.1786416Z { 2024-08-07T17:35:04.1786760Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1787196Z "size": 629993175, 2024-08-07T17:35:04.1787705Z "digest": "sha256:0e3c02dff025006601de33f0303b8c2f62aefdee1d0c34db3a60419ebe1a7e30" 2024-08-07T17:35:04.1788197Z }, 2024-08-07T17:35:04.1788398Z { 2024-08-07T17:35:04.1788722Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1789150Z "size": 160, 2024-08-07T17:35:04.1789567Z "digest": "sha256:2c2cb9893903884e0cf968da484f3b97fb62e1a881f8d68b437c240c0f2f25d1" 2024-08-07T17:35:04.1790035Z }, 2024-08-07T17:35:04.1790237Z { 2024-08-07T17:35:04.1790574Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1790989Z "size": 2209, 2024-08-07T17:35:04.1791411Z "digest": "sha256:ec0a42784a49a934dcd7d8c99dbe058a3828080e6716a12cba391210891574f3" 2024-08-07T17:35:04.1791889Z }, 2024-08-07T17:35:04.1792078Z { 2024-08-07T17:35:04.1792415Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1792844Z "size": 150, 2024-08-07T17:35:04.1793251Z "digest": "sha256:a20886f4e5372f586ef3e2728653d1969c6d898d8eae0667de17908891ac278f" 2024-08-07T17:35:04.1793728Z }, 2024-08-07T17:35:04.1793915Z { 2024-08-07T17:35:04.1794362Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1794792Z "size": 643, 2024-08-07T17:35:04.1795206Z "digest": "sha256:af1fb4fa4eed1c385edc3660525fa1e46533ec917477c141d430c07c735ff8da" 2024-08-07T17:35:04.1795689Z }, 2024-08-07T17:35:04.1795891Z { 2024-08-07T17:35:04.1796215Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1796644Z "size": 32, 2024-08-07T17:35:04.1797066Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:04.1797533Z }, 2024-08-07T17:35:04.1797785Z { 2024-08-07T17:35:04.1798127Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1798546Z "size": 162, 2024-08-07T17:35:04.1799069Z "digest": "sha256:de13eba39cb5160281708ab7fda3d1c4e26a9edb7793e8fddc3355c37a2c2bb6" 2024-08-07T17:35:04.1799567Z }, 2024-08-07T17:35:04.1799756Z { 2024-08-07T17:35:04.1800101Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1800534Z "size": 109, 2024-08-07T17:35:04.1800946Z "digest": "sha256:ac851722c4ad33e4e6ad7a9580ff1f61e91dc5ec40f5e5ff5629cd32c8a6dff9" 2024-08-07T17:35:04.1801432Z }, 2024-08-07T17:35:04.1801634Z { 2024-08-07T17:35:04.1801960Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1802385Z "size": 102, 2024-08-07T17:35:04.1802801Z "digest": "sha256:225d8e1484f0c72016ef99e76b8309a3472d6fb528b34633eb25c00394e350d9" 2024-08-07T17:35:04.1803369Z }, 2024-08-07T17:35:04.1803574Z { 2024-08-07T17:35:04.1803979Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1804449Z "size": 421, 2024-08-07T17:35:04.1804918Z "digest": "sha256:0b92275d203788d5d990e589228173aa2fe9709737328fd78f2aa02bf159a285" 2024-08-07T17:35:04.1805452Z }, 2024-08-07T17:35:04.1805646Z { 2024-08-07T17:35:04.1805991Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1806527Z "size": 32, 2024-08-07T17:35:04.1806998Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:04.1807485Z }, 2024-08-07T17:35:04.1807738Z { 2024-08-07T17:35:04.1808066Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1808544Z "size": 111, 2024-08-07T17:35:04.1808967Z "digest": "sha256:86d521d46977bce4fd8bbc492e3f0310b7f26ffc5e36fafe97225a7ca8fa9ffa" 2024-08-07T17:35:04.1809509Z }, 2024-08-07T17:35:04.1809709Z { 2024-08-07T17:35:04.1810092Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1810519Z "size": 471, 2024-08-07T17:35:04.1810989Z "digest": "sha256:3914906be0cb7fd68abac1a779b275af42993ccb4ed1bd1015f467e170d0b1f4" 2024-08-07T17:35:04.1811466Z }, 2024-08-07T17:35:04.1811669Z { 2024-08-07T17:35:04.1812131Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1812550Z "size": 32, 2024-08-07T17:35:04.1812978Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:04.1813467Z }, 2024-08-07T17:35:04.1813658Z { 2024-08-07T17:35:04.1813995Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1814424Z "size": 112, 2024-08-07T17:35:04.1814828Z "digest": "sha256:193407c5262c69087b764527f63b3d7e47e029f4f0ddf8f6a3fe4baaa7f29c57" 2024-08-07T17:35:04.1815298Z }, 2024-08-07T17:35:04.1815524Z { 2024-08-07T17:35:04.1815849Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1816287Z "size": 562, 2024-08-07T17:35:04.1816709Z "digest": "sha256:adf9d181cfbca5b0544f23797995d4ea6500cc4a635db9f7e18954c9044e9f4a" 2024-08-07T17:35:04.1817187Z }, 2024-08-07T17:35:04.1817396Z { 2024-08-07T17:35:04.1817739Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1818163Z "size": 43163921, 2024-08-07T17:35:04.1818604Z "digest": "sha256:736c4af6f5277292080e42bfc2c7edaf8412ef1b795cf6fcbf2e95beffbb7ed2" 2024-08-07T17:35:04.1819090Z }, 2024-08-07T17:35:04.1819285Z { 2024-08-07T17:35:04.1819624Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1820055Z "size": 106, 2024-08-07T17:35:04.1820461Z "digest": "sha256:801521a42094555983e4a2674ddbb195243f6fe0d15ad1aeaab4aa88d81d22c4" 2024-08-07T17:35:04.1820940Z }, 2024-08-07T17:35:04.1821145Z { 2024-08-07T17:35:04.1821476Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1821904Z "size": 294, 2024-08-07T17:35:04.1822333Z "digest": "sha256:ed7b11c2ed84119d1bd1ebf8b7317c20e0f79da24ca754a02cb6775c00a17ae1" 2024-08-07T17:35:04.1822812Z }, 2024-08-07T17:35:04.1823013Z { 2024-08-07T17:35:04.1823406Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1823877Z "size": 32, 2024-08-07T17:35:04.1824308Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:04.1824786Z }, 2024-08-07T17:35:04.1824990Z { 2024-08-07T17:35:04.1825338Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1825757Z "size": 106, 2024-08-07T17:35:04.1826176Z "digest": "sha256:5c6f2f3fe7c213d4b989fac53b07197b9f9ab99fe04360200826bcb2ca7ac66e" 2024-08-07T17:35:04.1826655Z }, 2024-08-07T17:35:04.1826843Z { 2024-08-07T17:35:04.1827178Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1827603Z "size": 424, 2024-08-07T17:35:04.1828014Z "digest": "sha256:3886e8f57dc887a87bb710fbda080ef4fd521f1ed2e560a895b000c05f6e9b0a" 2024-08-07T17:35:04.1828494Z }, 2024-08-07T17:35:04.1828693Z { 2024-08-07T17:35:04.1829018Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1829453Z "size": 19308828, 2024-08-07T17:35:04.1829894Z "digest": "sha256:cc822658f193feb19cb990795b3114bca9c649bd221c0b1aab684e652069c295" 2024-08-07T17:35:04.1830364Z }, 2024-08-07T17:35:04.1830568Z { 2024-08-07T17:35:04.1830904Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1831319Z "size": 108, 2024-08-07T17:35:04.1831729Z "digest": "sha256:cca044783521353ee93922f8ea943e74964e2a9815b43751866ff494b8d4d8d9" 2024-08-07T17:35:04.1832226Z }, 2024-08-07T17:35:04.1832476Z { 2024-08-07T17:35:04.1832909Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1833476Z "size": 438, 2024-08-07T17:35:04.1834010Z "digest": "sha256:0eb80a0ac0006a45aa6b4581263cc1613ddf9545bd4a1d5a544310efe898ad81" 2024-08-07T17:35:04.1834774Z }, 2024-08-07T17:35:04.1835155Z { 2024-08-07T17:35:04.1835769Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1836445Z "size": 699, 2024-08-07T17:35:04.1837148Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-07T17:35:04.1837913Z }, 2024-08-07T17:35:04.1838219Z { 2024-08-07T17:35:04.1838832Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1839434Z "size": 141, 2024-08-07T17:35:04.1839864Z "digest": "sha256:a9f11b2b3622f5530c5cfefe973e96e028323fab1ff02e7a6337eb86fadebeb5" 2024-08-07T17:35:04.1840381Z }, 2024-08-07T17:35:04.1840581Z { 2024-08-07T17:35:04.1840920Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1841338Z "size": 135, 2024-08-07T17:35:04.1841751Z "digest": "sha256:d7e94279f8b2fe549e3f23637275d9dca8c831563ce84f1d44117eb297ddc871" 2024-08-07T17:35:04.1842226Z }, 2024-08-07T17:35:04.1842418Z { 2024-08-07T17:35:04.1842756Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1843184Z "size": 32, 2024-08-07T17:35:04.1843631Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:04.1844173Z }, 2024-08-07T17:35:04.1844375Z { 2024-08-07T17:35:04.1844705Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1845130Z "size": 190, 2024-08-07T17:35:04.1845552Z "digest": "sha256:073f669362431eb57b23c427bcb2898ba4bb050cf4f9d37c09e270ceb847ccd6" 2024-08-07T17:35:04.1846020Z }, 2024-08-07T17:35:04.1846217Z { 2024-08-07T17:35:04.1846544Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1846973Z "size": 1210, 2024-08-07T17:35:04.1847399Z "digest": "sha256:719a1c2abcfd7e4113aa2f03adeb54350a5e381c720688ea53bc0e18401ad33d" 2024-08-07T17:35:04.1847873Z }, 2024-08-07T17:35:04.1848073Z { 2024-08-07T17:35:04.1848413Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1848826Z "size": 699, 2024-08-07T17:35:04.1849306Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-07T17:35:04.1849894Z }, 2024-08-07T17:35:04.1850091Z { 2024-08-07T17:35:04.1850436Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1850866Z "size": 138, 2024-08-07T17:35:04.1851269Z "digest": "sha256:8e1355d516c5e0527c25402c664fc648a8a2b814090e6ee0fdd60f49a8826ac1" 2024-08-07T17:35:04.1851751Z }, 2024-08-07T17:35:04.1851954Z { 2024-08-07T17:35:04.1852278Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1870915Z "size": 1842834361, 2024-08-07T17:35:04.1871519Z "digest": "sha256:4d5ca8eafd7982e3b256257abe6dfce2dc314e52d34defe80c8298de227a5f19" 2024-08-07T17:35:04.1872021Z }, 2024-08-07T17:35:04.1872230Z { 2024-08-07T17:35:04.1872561Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1872995Z "size": 153, 2024-08-07T17:35:04.1873418Z "digest": "sha256:73831405501a21694a29eaa05a64c05dbc545d9c5ba676a5b5f204fa2300291a" 2024-08-07T17:35:04.1873900Z }, 2024-08-07T17:35:04.1874194Z { 2024-08-07T17:35:04.1874555Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1874977Z "size": 846, 2024-08-07T17:35:04.1875400Z "digest": "sha256:0b48a1547ae65f413b4be264aad75996efd1ab0fac4107687531843d1ee54e5c" 2024-08-07T17:35:04.1875883Z }, 2024-08-07T17:35:04.1876074Z { 2024-08-07T17:35:04.1876414Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1876836Z "size": 699, 2024-08-07T17:35:04.1877252Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-07T17:35:04.1877763Z }, 2024-08-07T17:35:04.1877962Z { 2024-08-07T17:35:04.1878289Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1878711Z "size": 139, 2024-08-07T17:35:04.1879131Z "digest": "sha256:794615be0ac3201bd2845c4cd2e2f9e9651f1fba0142b0c74dccfda79fee82b1" 2024-08-07T17:35:04.1879599Z }, 2024-08-07T17:35:04.1879807Z { 2024-08-07T17:35:04.1880312Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1880729Z "size": 32, 2024-08-07T17:35:04.1881151Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:04.1881636Z }, 2024-08-07T17:35:04.1881824Z { 2024-08-07T17:35:04.1882163Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1882588Z "size": 157, 2024-08-07T17:35:04.1882996Z "digest": "sha256:ad492c23051a703625b3e779faf07bba8b9bea291956c20e744bba89898af6be" 2024-08-07T17:35:04.1883477Z }, 2024-08-07T17:35:04.1883679Z { 2024-08-07T17:35:04.1884008Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1884451Z "size": 905, 2024-08-07T17:35:04.1884880Z "digest": "sha256:fc956bf4e5b49119cab7cc7bda867c61bed76c76c1e113dfa7c7b184251f8173" 2024-08-07T17:35:04.1885360Z }, 2024-08-07T17:35:04.1885561Z { 2024-08-07T17:35:04.1885892Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1886322Z "size": 699, 2024-08-07T17:35:04.1886749Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-07T17:35:04.1887219Z }, 2024-08-07T17:35:04.1887417Z { 2024-08-07T17:35:04.1887753Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1888166Z "size": 134, 2024-08-07T17:35:04.1888594Z "digest": "sha256:874491333fb5b81d9cbbd71520163a6baf07e6f011dc148b1634b94e301972b8" 2024-08-07T17:35:04.1889073Z }, 2024-08-07T17:35:04.1889259Z { 2024-08-07T17:35:04.1889599Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1890023Z "size": 32, 2024-08-07T17:35:04.1890431Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:04.1890920Z }, 2024-08-07T17:35:04.1891152Z { 2024-08-07T17:35:04.1891476Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1892016Z "size": 156, 2024-08-07T17:35:04.1892454Z "digest": "sha256:62fd235ebb62f15f85f93abd31bdfcd2445df1b40f97d0b474a4879e05a47b0e" 2024-08-07T17:35:04.1892926Z }, 2024-08-07T17:35:04.1893131Z { 2024-08-07T17:35:04.1893473Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1893937Z "size": 1577, 2024-08-07T17:35:04.1894353Z "digest": "sha256:42646dcb0889a48f164507344b56ed0254f1c64ca68b2f974c7fa9f2569f1925" 2024-08-07T17:35:04.1894829Z }, 2024-08-07T17:35:04.1895054Z { 2024-08-07T17:35:04.1895394Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1895819Z "size": 32, 2024-08-07T17:35:04.1896223Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:04.1896712Z }, 2024-08-07T17:35:04.1896910Z { 2024-08-07T17:35:04.1897231Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1897666Z "size": 128, 2024-08-07T17:35:04.1898089Z "digest": "sha256:5df8c6bd8177de4d452857fef1396a5c9574f50e37b342b1a192f89ec36fb4e5" 2024-08-07T17:35:04.1898554Z }, 2024-08-07T17:35:04.1898761Z { 2024-08-07T17:35:04.1899086Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1899539Z "size": 377, 2024-08-07T17:35:04.1899964Z "digest": "sha256:96381dfde5a802fe7b27ba3ae3b5f8589f88c170c5bae6863be97a37fa86688b" 2024-08-07T17:35:04.1900489Z }, 2024-08-07T17:35:04.1900678Z { 2024-08-07T17:35:04.1901020Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1901433Z "size": 32, 2024-08-07T17:35:04.1901857Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:04.1902342Z }, 2024-08-07T17:35:04.1902529Z { 2024-08-07T17:35:04.1902866Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1903294Z "size": 104, 2024-08-07T17:35:04.1903702Z "digest": "sha256:d0dba839d4c041ba853621262a7e730c3699058d43cfe7d6f31ec311708acc73" 2024-08-07T17:35:04.1904248Z }, 2024-08-07T17:35:04.1904456Z { 2024-08-07T17:35:04.1904805Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1905233Z "size": 1839, 2024-08-07T17:35:04.1905658Z "digest": "sha256:c1d4cac78640fc7d364011da807a363d6b248a2c46ccf99a6e1488c4974c17e3" 2024-08-07T17:35:04.1906126Z }, 2024-08-07T17:35:04.1906330Z { 2024-08-07T17:35:04.1906677Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1907089Z "size": 7529628, 2024-08-07T17:35:04.1907521Z "digest": "sha256:67cb12cd081ccde6047c7cbbdd4d99c4499d4b12d3db421c307ef6d901ff0c81" 2024-08-07T17:35:04.1908016Z }, 2024-08-07T17:35:04.1908206Z { 2024-08-07T17:35:04.1908543Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1908970Z "size": 106, 2024-08-07T17:35:04.1909379Z "digest": "sha256:b431724b7afe431951242ade99ee881ce85e51dc44dd1f8c0c1ed1710de62f68" 2024-08-07T17:35:04.1909859Z }, 2024-08-07T17:35:04.1910065Z { 2024-08-07T17:35:04.1910421Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1910845Z "size": 161, 2024-08-07T17:35:04.1911258Z "digest": "sha256:190dccfa687dc1e698f27965d8065a183168d7c66361505f368765456ff9f4bc" 2024-08-07T17:35:04.1911713Z }, 2024-08-07T17:35:04.1911913Z { 2024-08-07T17:35:04.1912250Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1912663Z "size": 7942, 2024-08-07T17:35:04.1913088Z "digest": "sha256:b01bea4b6634f47c51ede2b4a95ceade4e61356ca29551168b493ae6d4492f44" 2024-08-07T17:35:04.1913578Z }, 2024-08-07T17:35:04.1913772Z { 2024-08-07T17:35:04.1914210Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1914633Z "size": 8065, 2024-08-07T17:35:04.1915065Z "digest": "sha256:77fc3d1eca2c044fa3497f2e035f9ca2e5ffcd8f16b810f1173ae52af23b71a7" 2024-08-07T17:35:04.1915644Z }, 2024-08-07T17:35:04.1915831Z { 2024-08-07T17:35:04.1916172Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1916596Z "size": 302, 2024-08-07T17:35:04.1917002Z "digest": "sha256:96d6dcd01b9f6c5557ba405d028442d808dd329f4f93e75f2c3ba0554e703a52" 2024-08-07T17:35:04.1917481Z }, 2024-08-07T17:35:04.1917677Z { 2024-08-07T17:35:04.1918005Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1918474Z "size": 32, 2024-08-07T17:35:04.1918899Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:04.1919371Z }, 2024-08-07T17:35:04.1919570Z { 2024-08-07T17:35:04.1919916Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1920363Z "size": 108, 2024-08-07T17:35:04.1920778Z "digest": "sha256:79ef14b93271d6a0315533761cb955f79fba8ee77b9a73ba823923e7dd1613f7" 2024-08-07T17:35:04.1921261Z }, 2024-08-07T17:35:04.1921451Z { 2024-08-07T17:35:04.1921793Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1922216Z "size": 54145774, 2024-08-07T17:35:04.1922630Z "digest": "sha256:45103ff2548064a8f9e94bdb31d8bc89e396f9880773cc06e0540dac48d20e10" 2024-08-07T17:35:04.1923101Z }, 2024-08-07T17:35:04.1923299Z { 2024-08-07T17:35:04.1923626Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:04.1924050Z "size": 32, 2024-08-07T17:35:04.1924465Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:04.1924939Z } 2024-08-07T17:35:04.1925141Z ] 2024-08-07T17:35:04.1925341Z } 2024-08-07T17:35:04.1991529Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-07T17:35:04.1991927Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-07T17:35:04.1992345Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-07T17:35:04.1998934Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:04.1999330Z env: 2024-08-07T17:35:04.1999560Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:04.2000283Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:04.2001034Z ##[endgroup] 2024-08-07T17:35:04.2027256Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-jammy-py3.8-gcc11-02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:04.2077126Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-07T17:35:04.2077581Z with: 2024-08-07T17:35:04.2078356Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:04.2079209Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:04.2079604Z env: 2024-08-07T17:35:04.2079861Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:04.2080128Z ##[endgroup] 2024-08-07T17:35:04.2102636Z ##[group]Run set -x 2024-08-07T17:35:04.2102931Z set -x 2024-08-07T17:35:04.2103156Z set +e 2024-08-07T17:35:04.2103409Z  2024-08-07T17:35:04.2103630Z login() { 2024-08-07T17:35:04.2104114Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-07T17:35:04.2104650Z } 2024-08-07T17:35:04.2104866Z  2024-08-07T17:35:04.2105122Z retry () { 2024-08-07T17:35:04.2105391Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-07T17:35:04.2105717Z } 2024-08-07T17:35:04.2105934Z  2024-08-07T17:35:04.2106169Z retry login "${DOCKER_REGISTRY}" 2024-08-07T17:35:04.2106496Z  2024-08-07T17:35:04.2106715Z set -e 2024-08-07T17:35:04.2107074Z # ignore output since only exit code is used for conditional 2024-08-07T17:35:04.2107571Z # only pull docker image if it's not available locally 2024-08-07T17:35:04.2108263Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-07T17:35:04.2108796Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-07T17:35:04.2109119Z fi 2024-08-07T17:35:04.2114587Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:04.2114984Z env: 2024-08-07T17:35:04.2115204Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:04.2115935Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:04.2116787Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:04.2117179Z ##[endgroup] 2024-08-07T17:35:04.2138948Z + set +e 2024-08-07T17:35:04.2139357Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:04.2139799Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:04.2142215Z + aws ecr get-login-password --region us-east-1 2024-08-07T17:35:04.2143505Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:04.7527526Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-07T17:35:04.7528299Z Configure a credential helper to remove this warning. See 2024-08-07T17:35:04.7529183Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-07T17:35:04.7529736Z 2024-08-07T17:35:04.7529993Z Login Succeeded 2024-08-07T17:35:04.7542493Z + set -e 2024-08-07T17:35:04.7544023Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:04.7668035Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:04.7669252Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:05.0178572Z 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9: Pulling from pytorch/pytorch-linux-jammy-py3.8-gcc11 2024-08-07T17:35:05.0192451Z 9b857f539cb1: Pulling fs layer 2024-08-07T17:35:05.0193228Z ceceb076946e: Pulling fs layer 2024-08-07T17:35:05.0194197Z 042461771f00: Pulling fs layer 2024-08-07T17:35:05.0194731Z bac001ac2fee: Pulling fs layer 2024-08-07T17:35:05.0195208Z 4a587220513d: Pulling fs layer 2024-08-07T17:35:05.0196628Z af8081143f00: Pulling fs layer 2024-08-07T17:35:05.0197212Z e7a423dcab01: Pulling fs layer 2024-08-07T17:35:05.0197970Z ad26542e0d01: Pulling fs layer 2024-08-07T17:35:05.0198420Z 9e1d4f8784a0: Pulling fs layer 2024-08-07T17:35:05.0198959Z b1f815d16274: Pulling fs layer 2024-08-07T17:35:05.0199357Z 91f9dddffbee: Pulling fs layer 2024-08-07T17:35:05.0199804Z 03fb78f207e3: Pulling fs layer 2024-08-07T17:35:05.0200218Z 8385f3c14a81: Pulling fs layer 2024-08-07T17:35:05.0200661Z bac001ac2fee: Waiting 2024-08-07T17:35:05.0201056Z 4f4fb700ef54: Pulling fs layer 2024-08-07T17:35:05.0201521Z e7a423dcab01: Waiting 2024-08-07T17:35:05.0201922Z 202c71391ade: Pulling fs layer 2024-08-07T17:35:05.0202323Z 240ee294d88b: Pulling fs layer 2024-08-07T17:35:05.0202766Z 7a823d3d7147: Pulling fs layer 2024-08-07T17:35:05.0203205Z ad26542e0d01: Waiting 2024-08-07T17:35:05.0203578Z 4a587220513d: Waiting 2024-08-07T17:35:05.0203886Z 9e1d4f8784a0: Waiting 2024-08-07T17:35:05.0204135Z af8081143f00: Waiting 2024-08-07T17:35:05.0204393Z a1b9e8856530: Pulling fs layer 2024-08-07T17:35:05.0204678Z 91f9dddffbee: Waiting 2024-08-07T17:35:05.0204927Z b1f815d16274: Waiting 2024-08-07T17:35:05.0205170Z 9c5b585a8116: Pulling fs layer 2024-08-07T17:35:05.0205452Z 03fb78f207e3: Waiting 2024-08-07T17:35:05.0205701Z 7a823d3d7147: Waiting 2024-08-07T17:35:05.0205957Z 202c71391ade: Waiting 2024-08-07T17:35:05.0206402Z 8385f3c14a81: Waiting 2024-08-07T17:35:05.0206669Z 9cd1bcd9f117: Pulling fs layer 2024-08-07T17:35:05.0206942Z 240ee294d88b: Waiting 2024-08-07T17:35:05.0207196Z 4f4fb700ef54: Waiting 2024-08-07T17:35:05.0207862Z 2b9731d295d6: Pulling fs layer 2024-08-07T17:35:05.0208337Z a1b9e8856530: Waiting 2024-08-07T17:35:05.0208780Z 9cd1bcd9f117: Waiting 2024-08-07T17:35:05.0209269Z 8d223c48f542: Pulling fs layer 2024-08-07T17:35:05.0209647Z 9c5b585a8116: Waiting 2024-08-07T17:35:05.0209893Z 2b9731d295d6: Waiting 2024-08-07T17:35:05.0210181Z 49a0e418a175: Pulling fs layer 2024-08-07T17:35:05.0210476Z 51ed474f6c32: Pulling fs layer 2024-08-07T17:35:05.0210786Z 1c01ecb575c6: Pulling fs layer 2024-08-07T17:35:05.0211057Z 8d223c48f542: Waiting 2024-08-07T17:35:05.0211306Z 49a0e418a175: Waiting 2024-08-07T17:35:05.0211570Z a2630e5fe166: Pulling fs layer 2024-08-07T17:35:05.0211851Z 554a42b2a2e6: Pulling fs layer 2024-08-07T17:35:05.0212133Z 1c01ecb575c6: Waiting 2024-08-07T17:35:05.0212382Z a2630e5fe166: Waiting 2024-08-07T17:35:05.0212632Z c9a20a1fda16: Pulling fs layer 2024-08-07T17:35:05.0212913Z 554a42b2a2e6: Waiting 2024-08-07T17:35:05.0213168Z 2c2e35b979f5: Pulling fs layer 2024-08-07T17:35:05.0213583Z c9a20a1fda16: Waiting 2024-08-07T17:35:05.0213849Z 0e3c02dff025: Pulling fs layer 2024-08-07T17:35:05.0214142Z 2c2cb9893903: Pulling fs layer 2024-08-07T17:35:05.0214410Z 0e3c02dff025: Waiting 2024-08-07T17:35:05.0214660Z 2c2e35b979f5: Waiting 2024-08-07T17:35:05.0214916Z ec0a42784a49: Pulling fs layer 2024-08-07T17:35:05.0215233Z a20886f4e537: Pulling fs layer 2024-08-07T17:35:05.0215512Z 51ed474f6c32: Waiting 2024-08-07T17:35:05.0215844Z af1fb4fa4eed: Pulling fs layer 2024-08-07T17:35:05.0216117Z 2c2cb9893903: Waiting 2024-08-07T17:35:05.0216405Z ec0a42784a49: Waiting 2024-08-07T17:35:05.0216659Z de13eba39cb5: Pulling fs layer 2024-08-07T17:35:05.0216944Z a20886f4e537: Waiting 2024-08-07T17:35:05.0217193Z af1fb4fa4eed: Waiting 2024-08-07T17:35:05.0217444Z ac851722c4ad: Pulling fs layer 2024-08-07T17:35:05.0217734Z 225d8e1484f0: Pulling fs layer 2024-08-07T17:35:05.0218019Z ac851722c4ad: Waiting 2024-08-07T17:35:05.0218265Z 0b92275d2037: Pulling fs layer 2024-08-07T17:35:05.0218545Z 225d8e1484f0: Waiting 2024-08-07T17:35:05.0218807Z 86d521d46977: Pulling fs layer 2024-08-07T17:35:05.0219072Z 0b92275d2037: Waiting 2024-08-07T17:35:05.0219437Z 3914906be0cb: Pulling fs layer 2024-08-07T17:35:05.0219720Z 86d521d46977: Waiting 2024-08-07T17:35:05.0220055Z 193407c5262c: Pulling fs layer 2024-08-07T17:35:05.0220350Z adf9d181cfbc: Pulling fs layer 2024-08-07T17:35:05.0220643Z 3914906be0cb: Waiting 2024-08-07T17:35:05.0220893Z 736c4af6f527: Pulling fs layer 2024-08-07T17:35:05.0221341Z 801521a42094: Pulling fs layer 2024-08-07T17:35:05.0221625Z 193407c5262c: Waiting 2024-08-07T17:35:05.0221876Z ed7b11c2ed84: Pulling fs layer 2024-08-07T17:35:05.0222171Z adf9d181cfbc: Waiting 2024-08-07T17:35:05.0222409Z 801521a42094: Waiting 2024-08-07T17:35:05.0222664Z 736c4af6f527: Waiting 2024-08-07T17:35:05.0222923Z 5c6f2f3fe7c2: Pulling fs layer 2024-08-07T17:35:05.0223206Z 3886e8f57dc8: Pulling fs layer 2024-08-07T17:35:05.0223485Z 5c6f2f3fe7c2: Waiting 2024-08-07T17:35:05.0223742Z cc822658f193: Pulling fs layer 2024-08-07T17:35:05.0224024Z cca044783521: Pulling fs layer 2024-08-07T17:35:05.0224304Z 3886e8f57dc8: Waiting 2024-08-07T17:35:05.0224562Z 0eb80a0ac000: Pulling fs layer 2024-08-07T17:35:05.0224828Z cc822658f193: Waiting 2024-08-07T17:35:05.0225086Z a9f11b2b3622: Pulling fs layer 2024-08-07T17:35:05.0225394Z d7e94279f8b2: Pulling fs layer 2024-08-07T17:35:05.0225664Z 0eb80a0ac000: Waiting 2024-08-07T17:35:05.0225928Z 073f66936243: Pulling fs layer 2024-08-07T17:35:05.0226211Z cca044783521: Waiting 2024-08-07T17:35:05.0226463Z 719a1c2abcfd: Pulling fs layer 2024-08-07T17:35:05.0226809Z a9f11b2b3622: Waiting 2024-08-07T17:35:05.0227059Z d7e94279f8b2: Waiting 2024-08-07T17:35:05.0227309Z 8e1355d516c5: Pulling fs layer 2024-08-07T17:35:05.0227599Z 719a1c2abcfd: Waiting 2024-08-07T17:35:05.0227871Z 4d5ca8eafd79: Pulling fs layer 2024-08-07T17:35:05.0228147Z 073f66936243: Waiting 2024-08-07T17:35:05.0228406Z 73831405501a: Pulling fs layer 2024-08-07T17:35:05.0228701Z 0b48a1547ae6: Pulling fs layer 2024-08-07T17:35:05.0228972Z 8e1355d516c5: Waiting 2024-08-07T17:35:05.0229307Z 4d5ca8eafd79: Waiting 2024-08-07T17:35:05.0229573Z 794615be0ac3: Pulling fs layer 2024-08-07T17:35:05.0229844Z 73831405501a: Waiting 2024-08-07T17:35:05.0230107Z ad492c23051a: Pulling fs layer 2024-08-07T17:35:05.0230394Z 794615be0ac3: Waiting 2024-08-07T17:35:05.0230643Z fc956bf4e5b4: Pulling fs layer 2024-08-07T17:35:05.0230937Z 874491333fb5: Pulling fs layer 2024-08-07T17:35:05.0231222Z ad492c23051a: Waiting 2024-08-07T17:35:05.0231464Z fc956bf4e5b4: Waiting 2024-08-07T17:35:05.0231725Z 62fd235ebb62: Pulling fs layer 2024-08-07T17:35:05.0231991Z 874491333fb5: Waiting 2024-08-07T17:35:05.0232246Z 42646dcb0889: Pulling fs layer 2024-08-07T17:35:05.0232629Z 62fd235ebb62: Waiting 2024-08-07T17:35:05.0232878Z 5df8c6bd8177: Pulling fs layer 2024-08-07T17:35:05.0233170Z 96381dfde5a8: Pulling fs layer 2024-08-07T17:35:05.0233521Z 5df8c6bd8177: Waiting 2024-08-07T17:35:05.0233770Z d0dba839d4c0: Pulling fs layer 2024-08-07T17:35:05.0234069Z c1d4cac78640: Pulling fs layer 2024-08-07T17:35:05.0234490Z 67cb12cd081c: Pulling fs layer 2024-08-07T17:35:05.0234761Z 96381dfde5a8: Waiting 2024-08-07T17:35:05.0235023Z b431724b7afe: Pulling fs layer 2024-08-07T17:35:05.0235313Z d0dba839d4c0: Waiting 2024-08-07T17:35:05.0235554Z c1d4cac78640: Waiting 2024-08-07T17:35:05.0235815Z 190dccfa687d: Pulling fs layer 2024-08-07T17:35:05.0236101Z 67cb12cd081c: Waiting 2024-08-07T17:35:05.0236349Z b01bea4b6634: Pulling fs layer 2024-08-07T17:35:05.0236636Z b431724b7afe: Waiting 2024-08-07T17:35:05.0236894Z 77fc3d1eca2c: Pulling fs layer 2024-08-07T17:35:05.0237178Z 96d6dcd01b9f: Pulling fs layer 2024-08-07T17:35:05.0237458Z 77fc3d1eca2c: Waiting 2024-08-07T17:35:05.0237719Z 79ef14b93271: Pulling fs layer 2024-08-07T17:35:05.0237987Z 96d6dcd01b9f: Waiting 2024-08-07T17:35:05.0238246Z 45103ff25480: Pulling fs layer 2024-08-07T17:35:05.0238706Z 79ef14b93271: Waiting 2024-08-07T17:35:05.0239116Z 45103ff25480: Waiting 2024-08-07T17:35:05.1171510Z ceceb076946e: Download complete 2024-08-07T17:35:05.1930268Z bac001ac2fee: Verifying Checksum 2024-08-07T17:35:05.1930897Z bac001ac2fee: Download complete 2024-08-07T17:35:05.2632363Z 4a587220513d: Verifying Checksum 2024-08-07T17:35:05.2632895Z 4a587220513d: Download complete 2024-08-07T17:35:05.3339168Z af8081143f00: Verifying Checksum 2024-08-07T17:35:05.3339662Z af8081143f00: Download complete 2024-08-07T17:35:05.3755117Z 9b857f539cb1: Verifying Checksum 2024-08-07T17:35:05.3755506Z 9b857f539cb1: Download complete 2024-08-07T17:35:05.3976117Z e7a423dcab01: Verifying Checksum 2024-08-07T17:35:05.3976678Z e7a423dcab01: Download complete 2024-08-07T17:35:05.4654969Z ad26542e0d01: Verifying Checksum 2024-08-07T17:35:05.4655534Z ad26542e0d01: Download complete 2024-08-07T17:35:05.5315515Z b1f815d16274: Download complete 2024-08-07T17:35:05.6078240Z 91f9dddffbee: Verifying Checksum 2024-08-07T17:35:05.6922022Z 91f9dddffbee: Download complete 2024-08-07T17:35:05.6922579Z 03fb78f207e3: Verifying Checksum 2024-08-07T17:35:05.6923321Z 03fb78f207e3: Download complete 2024-08-07T17:35:06.3273892Z 9b857f539cb1: Pull complete 2024-08-07T17:35:06.3421620Z ceceb076946e: Pull complete 2024-08-07T17:35:06.5600997Z 9e1d4f8784a0: Verifying Checksum 2024-08-07T17:35:06.5601445Z 9e1d4f8784a0: Download complete 2024-08-07T17:35:06.5697750Z 4f4fb700ef54: Verifying Checksum 2024-08-07T17:35:06.5698140Z 4f4fb700ef54: Download complete 2024-08-07T17:35:06.6538023Z 202c71391ade: Verifying Checksum 2024-08-07T17:35:06.6538459Z 202c71391ade: Download complete 2024-08-07T17:35:06.7728651Z 240ee294d88b: Download complete 2024-08-07T17:35:06.8509835Z 7a823d3d7147: Download complete 2024-08-07T17:35:06.9489166Z a1b9e8856530: Download complete 2024-08-07T17:35:07.0135232Z 9c5b585a8116: Verifying Checksum 2024-08-07T17:35:07.0135777Z 9c5b585a8116: Download complete 2024-08-07T17:35:07.0966044Z 9cd1bcd9f117: Verifying Checksum 2024-08-07T17:35:07.0966497Z 9cd1bcd9f117: Download complete 2024-08-07T17:35:07.1687502Z 2b9731d295d6: Download complete 2024-08-07T17:35:07.2325648Z 8d223c48f542: Verifying Checksum 2024-08-07T17:35:07.2326544Z 8d223c48f542: Download complete 2024-08-07T17:35:07.2969833Z 49a0e418a175: Verifying Checksum 2024-08-07T17:35:07.2970415Z 49a0e418a175: Download complete 2024-08-07T17:35:08.2000244Z 042461771f00: Verifying Checksum 2024-08-07T17:35:08.2000847Z 042461771f00: Download complete 2024-08-07T17:35:08.2920014Z 1c01ecb575c6: Verifying Checksum 2024-08-07T17:35:08.2920616Z 1c01ecb575c6: Download complete 2024-08-07T17:35:08.3557222Z 51ed474f6c32: Verifying Checksum 2024-08-07T17:35:08.3557680Z 51ed474f6c32: Download complete 2024-08-07T17:35:08.3707218Z a2630e5fe166: Download complete 2024-08-07T17:35:08.4470989Z 554a42b2a2e6: Verifying Checksum 2024-08-07T17:35:08.4472154Z 554a42b2a2e6: Download complete 2024-08-07T17:35:08.4884128Z c9a20a1fda16: Download complete 2024-08-07T17:35:08.5245911Z 2c2e35b979f5: Verifying Checksum 2024-08-07T17:35:08.5246536Z 2c2e35b979f5: Download complete 2024-08-07T17:35:08.6339572Z 2c2cb9893903: Verifying Checksum 2024-08-07T17:35:08.6340174Z 2c2cb9893903: Download complete 2024-08-07T17:35:08.7179100Z ec0a42784a49: Verifying Checksum 2024-08-07T17:35:08.7179712Z ec0a42784a49: Download complete 2024-08-07T17:35:08.8021353Z a20886f4e537: Verifying Checksum 2024-08-07T17:35:08.8021966Z a20886f4e537: Download complete 2024-08-07T17:35:08.8839576Z af1fb4fa4eed: Verifying Checksum 2024-08-07T17:35:08.8840231Z af1fb4fa4eed: Download complete 2024-08-07T17:35:08.9554271Z de13eba39cb5: Verifying Checksum 2024-08-07T17:35:08.9554889Z de13eba39cb5: Download complete 2024-08-07T17:35:09.0344009Z ac851722c4ad: Verifying Checksum 2024-08-07T17:35:09.0344654Z ac851722c4ad: Download complete 2024-08-07T17:35:09.1033785Z 225d8e1484f0: Download complete 2024-08-07T17:35:09.1941814Z 0b92275d2037: Download complete 2024-08-07T17:35:09.2705507Z 86d521d46977: Verifying Checksum 2024-08-07T17:35:09.2706063Z 86d521d46977: Download complete 2024-08-07T17:35:09.3641191Z 3914906be0cb: Verifying Checksum 2024-08-07T17:35:09.3641644Z 3914906be0cb: Download complete 2024-08-07T17:35:09.4462588Z 193407c5262c: Download complete 2024-08-07T17:35:09.5069700Z adf9d181cfbc: Verifying Checksum 2024-08-07T17:35:09.5070279Z adf9d181cfbc: Download complete 2024-08-07T17:35:10.0007155Z 736c4af6f527: Verifying Checksum 2024-08-07T17:35:10.0007800Z 736c4af6f527: Download complete 2024-08-07T17:35:10.1023589Z 801521a42094: Download complete 2024-08-07T17:35:10.1757007Z ed7b11c2ed84: Download complete 2024-08-07T17:35:10.2539593Z 5c6f2f3fe7c2: Download complete 2024-08-07T17:35:10.3557271Z 3886e8f57dc8: Download complete 2024-08-07T17:35:10.6115199Z cc822658f193: Verifying Checksum 2024-08-07T17:35:10.6115588Z cc822658f193: Download complete 2024-08-07T17:35:10.6806567Z cca044783521: Verifying Checksum 2024-08-07T17:35:10.7453567Z 0eb80a0ac000: Download complete 2024-08-07T17:35:10.8291203Z a9f11b2b3622: Verifying Checksum 2024-08-07T17:35:10.8291815Z a9f11b2b3622: Download complete 2024-08-07T17:35:10.9030422Z d7e94279f8b2: Download complete 2024-08-07T17:35:10.9644119Z 073f66936243: Download complete 2024-08-07T17:35:11.0513416Z 719a1c2abcfd: Verifying Checksum 2024-08-07T17:35:11.0514056Z 719a1c2abcfd: Download complete 2024-08-07T17:35:11.1284994Z 8e1355d516c5: Verifying Checksum 2024-08-07T17:35:11.1285449Z 8e1355d516c5: Download complete 2024-08-07T17:35:14.8539650Z 0e3c02dff025: Verifying Checksum 2024-08-07T17:35:14.8540136Z 0e3c02dff025: Download complete 2024-08-07T17:35:14.9260609Z 73831405501a: Verifying Checksum 2024-08-07T17:35:14.9261104Z 73831405501a: Download complete 2024-08-07T17:35:15.1282664Z 0b48a1547ae6: Download complete 2024-08-07T17:35:15.2260165Z 794615be0ac3: Verifying Checksum 2024-08-07T17:35:15.2260667Z 794615be0ac3: Download complete 2024-08-07T17:35:15.2902025Z ad492c23051a: Download complete 2024-08-07T17:35:15.3617192Z fc956bf4e5b4: Verifying Checksum 2024-08-07T17:35:15.3617769Z fc956bf4e5b4: Download complete 2024-08-07T17:35:15.4453535Z 874491333fb5: Verifying Checksum 2024-08-07T17:35:15.4454107Z 874491333fb5: Download complete 2024-08-07T17:35:15.5067000Z 62fd235ebb62: Download complete 2024-08-07T17:35:15.5657216Z 42646dcb0889: Verifying Checksum 2024-08-07T17:35:15.5659532Z 42646dcb0889: Download complete 2024-08-07T17:35:15.6377436Z 5df8c6bd8177: Verifying Checksum 2024-08-07T17:35:15.6377857Z 5df8c6bd8177: Download complete 2024-08-07T17:35:15.7366866Z 96381dfde5a8: Verifying Checksum 2024-08-07T17:35:15.7367442Z 96381dfde5a8: Download complete 2024-08-07T17:35:15.7986997Z d0dba839d4c0: Download complete 2024-08-07T17:35:15.8635928Z c1d4cac78640: Verifying Checksum 2024-08-07T17:35:15.8636324Z c1d4cac78640: Download complete 2024-08-07T17:35:15.9871313Z 67cb12cd081c: Verifying Checksum 2024-08-07T17:35:15.9871713Z 67cb12cd081c: Download complete 2024-08-07T17:35:16.0597265Z b431724b7afe: Download complete 2024-08-07T17:35:16.1479275Z 190dccfa687d: Verifying Checksum 2024-08-07T17:35:16.1479911Z 190dccfa687d: Download complete 2024-08-07T17:35:16.2220206Z b01bea4b6634: Verifying Checksum 2024-08-07T17:35:16.2220590Z b01bea4b6634: Download complete 2024-08-07T17:35:16.2979268Z 77fc3d1eca2c: Verifying Checksum 2024-08-07T17:35:16.2979801Z 77fc3d1eca2c: Download complete 2024-08-07T17:35:16.3788830Z 96d6dcd01b9f: Verifying Checksum 2024-08-07T17:35:16.3789395Z 96d6dcd01b9f: Download complete 2024-08-07T17:35:16.4410706Z 79ef14b93271: Verifying Checksum 2024-08-07T17:35:16.4411196Z 79ef14b93271: Download complete 2024-08-07T17:35:18.2066951Z 45103ff25480: Verifying Checksum 2024-08-07T17:35:18.2067412Z 45103ff25480: Download complete 2024-08-07T17:35:19.1062799Z 042461771f00: Pull complete 2024-08-07T17:35:19.2187262Z bac001ac2fee: Pull complete 2024-08-07T17:35:19.3391366Z 4a587220513d: Pull complete 2024-08-07T17:35:19.4610931Z af8081143f00: Pull complete 2024-08-07T17:35:19.5328506Z e7a423dcab01: Pull complete 2024-08-07T17:35:19.5766109Z ad26542e0d01: Pull complete 2024-08-07T17:35:40.9640702Z 9e1d4f8784a0: Pull complete 2024-08-07T17:35:41.1124265Z b1f815d16274: Pull complete 2024-08-07T17:35:41.3156962Z 91f9dddffbee: Pull complete 2024-08-07T17:35:41.4932258Z 03fb78f207e3: Pull complete 2024-08-07T17:35:41.9806797Z 4d5ca8eafd79: Verifying Checksum 2024-08-07T17:35:41.9808172Z 4d5ca8eafd79: Download complete 2024-08-07T17:35:44.4983947Z 8385f3c14a81: Verifying Checksum 2024-08-07T17:35:44.4984339Z 8385f3c14a81: Download complete 2024-08-07T17:36:31.9602992Z 8385f3c14a81: Pull complete 2024-08-07T17:36:32.2153499Z 4f4fb700ef54: Pull complete 2024-08-07T17:36:32.4250341Z 202c71391ade: Pull complete 2024-08-07T17:36:32.6391882Z 240ee294d88b: Pull complete 2024-08-07T17:36:32.8048532Z 7a823d3d7147: Pull complete 2024-08-07T17:36:33.0422230Z a1b9e8856530: Pull complete 2024-08-07T17:36:33.2553205Z 9c5b585a8116: Pull complete 2024-08-07T17:36:33.4777089Z 9cd1bcd9f117: Pull complete 2024-08-07T17:36:33.7047536Z 2b9731d295d6: Pull complete 2024-08-07T17:36:34.2492755Z 8d223c48f542: Pull complete 2024-08-07T17:36:34.5009016Z 49a0e418a175: Pull complete 2024-08-07T17:36:37.2969709Z 51ed474f6c32: Pull complete 2024-08-07T17:36:37.5353752Z 1c01ecb575c6: Pull complete 2024-08-07T17:36:37.6187182Z a2630e5fe166: Pull complete 2024-08-07T17:36:37.8443337Z 554a42b2a2e6: Pull complete 2024-08-07T17:36:37.9932837Z c9a20a1fda16: Pull complete 2024-08-07T17:36:38.0615382Z 2c2e35b979f5: Pull complete 2024-08-07T17:36:46.3636529Z 0e3c02dff025: Pull complete 2024-08-07T17:36:46.3817226Z 2c2cb9893903: Pull complete 2024-08-07T17:36:46.3996234Z ec0a42784a49: Pull complete 2024-08-07T17:36:46.4166381Z a20886f4e537: Pull complete 2024-08-07T17:36:46.4341305Z af1fb4fa4eed: Pull complete 2024-08-07T17:36:46.4691969Z de13eba39cb5: Pull complete 2024-08-07T17:36:46.4895245Z ac851722c4ad: Pull complete 2024-08-07T17:36:46.5057851Z 225d8e1484f0: Pull complete 2024-08-07T17:36:46.5219509Z 0b92275d2037: Pull complete 2024-08-07T17:36:46.5578883Z 86d521d46977: Pull complete 2024-08-07T17:36:46.5756035Z 3914906be0cb: Pull complete 2024-08-07T17:36:46.6148363Z 193407c5262c: Pull complete 2024-08-07T17:36:46.6312942Z adf9d181cfbc: Pull complete 2024-08-07T17:36:48.4581371Z 736c4af6f527: Pull complete 2024-08-07T17:36:48.6302662Z 801521a42094: Pull complete 2024-08-07T17:36:48.7041685Z ed7b11c2ed84: Pull complete 2024-08-07T17:36:48.7967985Z 5c6f2f3fe7c2: Pull complete 2024-08-07T17:36:48.8407428Z 3886e8f57dc8: Pull complete 2024-08-07T17:36:49.1130212Z cc822658f193: Pull complete 2024-08-07T17:36:49.2278864Z cca044783521: Pull complete 2024-08-07T17:36:49.3109377Z 0eb80a0ac000: Pull complete 2024-08-07T17:36:49.4114577Z a9f11b2b3622: Pull complete 2024-08-07T17:36:49.4911518Z d7e94279f8b2: Pull complete 2024-08-07T17:36:49.7603702Z 073f66936243: Pull complete 2024-08-07T17:36:49.9046577Z 719a1c2abcfd: Pull complete 2024-08-07T17:36:50.1104674Z 8e1355d516c5: Pull complete 2024-08-07T17:37:20.8070284Z 4d5ca8eafd79: Pull complete 2024-08-07T17:37:21.0033269Z 73831405501a: Pull complete 2024-08-07T17:37:21.0749809Z 0b48a1547ae6: Pull complete 2024-08-07T17:37:21.3907803Z 794615be0ac3: Pull complete 2024-08-07T17:37:21.7247619Z ad492c23051a: Pull complete 2024-08-07T17:37:21.9133370Z fc956bf4e5b4: Pull complete 2024-08-07T17:37:22.0662054Z 874491333fb5: Pull complete 2024-08-07T17:37:22.1594391Z 62fd235ebb62: Pull complete 2024-08-07T17:37:22.1962037Z 42646dcb0889: Pull complete 2024-08-07T17:37:22.2570537Z 5df8c6bd8177: Pull complete 2024-08-07T17:37:22.3074027Z 96381dfde5a8: Pull complete 2024-08-07T17:37:22.4673808Z d0dba839d4c0: Pull complete 2024-08-07T17:37:22.5234964Z c1d4cac78640: Pull complete 2024-08-07T17:37:22.9034917Z 67cb12cd081c: Pull complete 2024-08-07T17:37:23.0219348Z b431724b7afe: Pull complete 2024-08-07T17:37:23.0861893Z 190dccfa687d: Pull complete 2024-08-07T17:37:23.2170688Z b01bea4b6634: Pull complete 2024-08-07T17:37:23.3830688Z 77fc3d1eca2c: Pull complete 2024-08-07T17:37:23.4394056Z 96d6dcd01b9f: Pull complete 2024-08-07T17:37:23.5060467Z 79ef14b93271: Pull complete 2024-08-07T17:37:25.9204707Z 45103ff25480: Pull complete 2024-08-07T17:37:26.1754376Z Digest: sha256:c75221497aa22766802555fccb76014708414eebd47578b213b43b3f8f4cddb9 2024-08-07T17:37:26.2166726Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:37:26.2393879Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:37:26.2439722Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-07T17:37:26.2440433Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-07T17:37:26.2447785Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:26.2448181Z env: 2024-08-07T17:37:26.2448409Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:26.2448669Z ##[endgroup] 2024-08-07T17:37:26.2714074Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-07T17:37:26.2714660Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-07T17:37:26.2715181Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-07T17:37:26.2715690Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:37:26.2721503Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:26.2721894Z env: 2024-08-07T17:37:26.2722121Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:26.2722380Z ##[endgroup] 2024-08-07T17:37:26.7281847Z Defaulting to user installation because normal site-packages is not writeable 2024-08-07T17:37:26.7524771Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.9/site-packages (5.9.1) 2024-08-07T17:37:26.7530578Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.9/site-packages (11.525.84) 2024-08-07T17:37:26.8809121Z Prepare all required actions 2024-08-07T17:37:26.8810041Z Getting action download info 2024-08-07T17:37:26.9763094Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-07T17:37:27.4034446Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-07T17:37:27.5804506Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-07T17:37:27.5805174Z with: 2024-08-07T17:37:27.5805587Z name: linux-jammy-py3.8-gcc11 2024-08-07T17:37:27.5806106Z s3-bucket: gha-artifacts 2024-08-07T17:37:27.5806577Z env: 2024-08-07T17:37:27.5806952Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:27.5807413Z ##[endgroup] 2024-08-07T17:37:27.5935817Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-07T17:37:27.5936437Z with: 2024-08-07T17:37:27.5936830Z name: linux-jammy-py3.8-gcc11 2024-08-07T17:37:27.5937361Z s3-bucket: gha-artifacts 2024-08-07T17:37:27.5937906Z region: us-east-1 2024-08-07T17:37:27.5938335Z env: 2024-08-07T17:37:27.5938678Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:27.5939134Z ##[endgroup] 2024-08-07T17:37:28.3028588Z (node:477296) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-07T17:37:28.3029555Z 2024-08-07T17:37:28.3029899Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-07T17:37:28.3030844Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-07T17:37:28.3031844Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-07T17:37:28.3734026Z Found 1 objects with prefix pytorch/pytorch/10288745067/linux-jammy-py3.8-gcc11/ 2024-08-07T17:37:28.3735278Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-07T17:37:32.3587446Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-07T17:37:32.3594856Z Artifact download has finished successfully 2024-08-07T17:37:32.3772594Z ##[group]Run unzip -o artifacts.zip 2024-08-07T17:37:32.3772970Z unzip -o artifacts.zip 2024-08-07T17:37:32.3779203Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:32.3779588Z env: 2024-08-07T17:37:32.3779813Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:32.3780097Z ##[endgroup] 2024-08-07T17:37:32.4097287Z Archive: artifacts.zip 2024-08-07T17:37:32.4098055Z creating: dist/ 2024-08-07T17:37:33.3559485Z inflating: dist/torch-2.5.0a0+git016588f-cp38-cp38-linux_x86_64.whl 2024-08-07T17:37:33.3560068Z creating: build/custom_test_artifacts/ 2024-08-07T17:37:33.3560471Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-07T17:37:33.3560982Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-07T17:37:33.3561596Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-07T17:37:33.3563376Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-07T17:37:33.3564172Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-07T17:37:33.3565351Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-07T17:37:33.3566096Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-07T17:37:33.3566931Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-07T17:37:33.3568043Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-07T17:37:33.3569504Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-07T17:37:33.3570251Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-07T17:37:33.3571141Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-07T17:37:33.3572606Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-07T17:37:33.3573924Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-07T17:37:33.3575234Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-07T17:37:33.3576672Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-07T17:37:33.3578333Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-07T17:37:33.3579887Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-07T17:37:33.3581141Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-07T17:37:33.3582355Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-07T17:37:33.3583688Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-07T17:37:33.3584457Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-07T17:37:33.3585317Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-07T17:37:33.3586168Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-07T17:37:33.3586950Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-07T17:37:33.3587742Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-07T17:37:33.3588553Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-07T17:37:33.3589364Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-07T17:37:33.3590170Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-07T17:37:33.3590967Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-07T17:37:33.3602488Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-07T17:37:33.3733089Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-07T17:37:33.3734508Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-07T17:37:33.3735972Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-07T17:37:33.3737746Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-07T17:37:33.3738891Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-07T17:37:33.3739707Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-07T17:37:33.3740550Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-07T17:37:33.3741404Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-07T17:37:33.3742258Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-07T17:37:33.3743129Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-07T17:37:33.3743966Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-07T17:37:33.3756985Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-07T17:37:33.3833240Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-07T17:37:33.3834899Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-07T17:37:33.3836374Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-07T17:37:33.3837519Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-07T17:37:33.3838700Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-07T17:37:33.3839613Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-07T17:37:33.3840256Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-07T17:37:33.3840821Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-07T17:37:33.3841412Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-07T17:37:33.3952156Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-07T17:37:33.4012183Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-07T17:37:33.4013192Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-07T17:37:33.4013736Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-07T17:37:33.4014330Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-07T17:37:33.4016210Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-07T17:37:33.4017616Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-07T17:37:33.4018391Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-07T17:37:33.4019103Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-07T17:37:33.4020200Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-07T17:37:33.4021375Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-07T17:37:33.4022464Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-07T17:37:33.4023197Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-07T17:37:33.4024318Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-07T17:37:33.4025555Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-07T17:37:33.4026690Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-07T17:37:33.4027900Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-07T17:37:33.4029159Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-07T17:37:33.4030772Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-07T17:37:33.4032424Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-07T17:37:33.4033776Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-07T17:37:33.4035062Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-07T17:37:33.4036070Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-07T17:37:33.4036864Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-07T17:37:33.4060710Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-07T17:37:33.4061619Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-07T17:37:33.4062672Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-07T17:37:33.4063742Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-07T17:37:33.4064615Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-07T17:37:33.4065584Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-07T17:37:33.4066421Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-07T17:37:33.4067253Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-07T17:37:33.4068163Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-07T17:37:33.4117448Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-07T17:37:33.4119158Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-07T17:37:33.4120505Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-07T17:37:33.4121365Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-07T17:37:33.4122462Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-07T17:37:33.4123695Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-07T17:37:33.4124387Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-07T17:37:33.4124950Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-07T17:37:33.4125521Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-07T17:37:33.4172464Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-07T17:37:33.4173405Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-07T17:37:33.4173950Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-07T17:37:33.4174595Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-07T17:37:33.4176465Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-07T17:37:33.4177812Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-07T17:37:33.4178552Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-07T17:37:33.4179468Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-07T17:37:33.4180539Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-07T17:37:33.4182039Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-07T17:37:33.4182947Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-07T17:37:33.4184141Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-07T17:37:33.4185187Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-07T17:37:33.4186468Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-07T17:37:33.4187821Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-07T17:37:33.4189121Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-07T17:37:33.4190747Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-07T17:37:33.4192911Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-07T17:37:33.4194158Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-07T17:37:33.4195642Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-07T17:37:33.4197097Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-07T17:37:33.4197857Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-07T17:37:33.4198716Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-07T17:37:33.4199674Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-07T17:37:33.4201005Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-07T17:37:33.4201878Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-07T17:37:33.4202767Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-07T17:37:33.4203676Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-07T17:37:33.4204587Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-07T17:37:33.4205493Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-07T17:37:33.4206376Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-07T17:37:33.4207336Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-07T17:37:33.4318494Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-07T17:37:33.4320193Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-07T17:37:33.4321868Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-07T17:37:33.4323609Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-07T17:37:33.4324579Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-07T17:37:33.4325491Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-07T17:37:33.4326469Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-07T17:37:33.4327421Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-07T17:37:33.4328386Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-07T17:37:33.4329328Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-07T17:37:33.4330276Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-07T17:37:33.4342234Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-07T17:37:33.4393768Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-07T17:37:33.4395630Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-07T17:37:33.4397024Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-07T17:37:33.4398424Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-07T17:37:33.4399718Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-07T17:37:33.4400432Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-07T17:37:33.4401298Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-07T17:37:33.4401917Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-07T17:37:33.4402533Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-07T17:37:33.4499404Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-07T17:37:33.4540116Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-07T17:37:33.4540879Z creating: build/lib/ 2024-08-07T17:37:33.4541409Z inflating: build/lib/libclog.a 2024-08-07T17:37:33.4550380Z inflating: build/lib/libpthreadpool.a 2024-08-07T17:37:33.4557905Z inflating: build/lib/libcpuinfo_internals.a 2024-08-07T17:37:33.4566132Z inflating: build/lib/libcpuinfo.a 2024-08-07T17:37:33.4629584Z inflating: build/lib/libgtest.a 2024-08-07T17:37:33.4709191Z inflating: build/lib/libprotobuf-lite.a 2024-08-07T17:37:33.4715953Z inflating: build/lib/libittnotify.a 2024-08-07T17:37:33.4786296Z inflating: build/lib/libbenchmark.a 2024-08-07T17:37:33.4812899Z inflating: build/lib/libtensorpipe_uv.a 2024-08-07T17:37:33.4874377Z inflating: build/lib/libasmjit.a 2024-08-07T17:37:33.5299035Z inflating: build/lib/libprotobuf.a 2024-08-07T17:37:33.5407581Z inflating: build/lib/libgloo.a 2024-08-07T17:37:33.5427014Z inflating: build/lib/libfmt.a 2024-08-07T17:37:33.5427631Z inflating: build/lib/libfoxi_loader.a 2024-08-07T17:37:33.5428841Z inflating: build/lib/libtorch_global_deps.so 2024-08-07T17:37:33.5521044Z inflating: build/lib/libc10.so 2024-08-07T17:37:33.5523045Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-07T17:37:33.5541335Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-07T17:37:33.6016324Z inflating: build/lib/libprotoc.a 2024-08-07T17:37:33.6032978Z inflating: build/lib/libgmock.a 2024-08-07T17:37:33.6033682Z inflating: build/lib/libbenchmark_main.a 2024-08-07T17:37:33.6034353Z inflating: build/lib/libgtest_main.a 2024-08-07T17:37:33.6051610Z inflating: build/lib/libnnpack.a 2024-08-07T17:37:34.5269434Z inflating: build/lib/libdnnl.a 2024-08-07T17:37:34.5796408Z inflating: build/lib/libtensorpipe.a 2024-08-07T17:37:34.5797044Z inflating: build/lib/libgmock_main.a 2024-08-07T17:37:34.6786720Z inflating: build/lib/libfbgemm.a 2024-08-07T17:37:34.7060993Z inflating: build/lib/libkineto.a 2024-08-07T17:37:34.7099842Z inflating: build/lib/libonnx_proto.a 2024-08-07T17:37:34.7276088Z inflating: build/lib/libXNNPACK.a 2024-08-07T17:37:34.7937492Z inflating: build/lib/libonnx.a 2024-08-07T17:37:37.0284791Z inflating: build/lib/libtorch_cpu.so 2024-08-07T17:37:37.0285412Z inflating: build/lib/libtorch.so 2024-08-07T17:37:37.0290158Z inflating: build/lib/libunbox_lib.a 2024-08-07T17:37:37.0293849Z inflating: build/lib/libshm.so 2024-08-07T17:37:37.0361898Z inflating: build/lib/libtorchbind_test.so 2024-08-07T17:37:37.0381349Z inflating: build/lib/libjitbackend_test.so 2024-08-07T17:37:37.0405281Z inflating: build/lib/libbackend_with_compiler.so 2024-08-07T17:37:37.0429318Z inflating: build/lib/libaoti_custom_ops.so 2024-08-07T17:37:37.2253248Z inflating: build/lib/libtorch_python.so 2024-08-07T17:37:37.2286888Z inflating: build/lib/libnnapi_backend.so 2024-08-07T17:37:37.2287337Z creating: build/bin/ 2024-08-07T17:37:37.2287786Z creating: build/bin/CMakeFiles/ 2024-08-07T17:37:37.2288599Z inflating: build/bin/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-07T17:37:37.2289485Z extracting: build/bin/CMakeFiles/progress.marks 2024-08-07T17:37:37.2289941Z inflating: build/bin/Makefile 2024-08-07T17:37:37.2290360Z inflating: build/bin/cmake_install.cmake 2024-08-07T17:37:37.2290789Z inflating: build/bin/CTestTestfile.cmake 2024-08-07T17:37:37.2342161Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-07T17:37:37.2394114Z inflating: build/bin/c10_DeviceGuard_test 2024-08-07T17:37:37.2444993Z inflating: build/bin/c10_Device_test 2024-08-07T17:37:37.2505078Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-07T17:37:37.2559695Z inflating: build/bin/c10_Scalar_test 2024-08-07T17:37:37.2609835Z inflating: build/bin/c10_StreamGuard_test 2024-08-07T17:37:37.2661691Z inflating: build/bin/c10_SymInt_test 2024-08-07T17:37:37.2716828Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-07T17:37:37.2773348Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-07T17:37:37.2829364Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-07T17:37:37.2879263Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-07T17:37:37.2933068Z inflating: build/bin/c10_Bitset_test 2024-08-07T17:37:37.3004227Z inflating: build/bin/c10_cow_test 2024-08-07T17:37:37.3054490Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-07T17:37:37.3105925Z inflating: build/bin/c10_Half_test 2024-08-07T17:37:37.3162279Z inflating: build/bin/c10_LeftRight_test 2024-08-07T17:37:37.3218441Z inflating: build/bin/c10_Metaprogramming_test 2024-08-07T17:37:37.3271187Z inflating: build/bin/c10_TypeIndex_test 2024-08-07T17:37:37.3322919Z inflating: build/bin/c10_TypeList_test 2024-08-07T17:37:37.3377311Z inflating: build/bin/c10_Synchronized_test 2024-08-07T17:37:37.3433263Z inflating: build/bin/c10_ThreadLocal_test 2024-08-07T17:37:37.3483596Z inflating: build/bin/c10_TypeTraits_test 2024-08-07T17:37:37.3535979Z inflating: build/bin/c10_accumulate_test 2024-08-07T17:37:37.3591993Z inflating: build/bin/c10_bfloat16_test 2024-08-07T17:37:37.3644515Z inflating: build/bin/c10_bit_cast_test 2024-08-07T17:37:37.3701723Z inflating: build/bin/c10_complex_math_test 2024-08-07T17:37:37.3754647Z inflating: build/bin/c10_exception_test 2024-08-07T17:37:37.3810207Z inflating: build/bin/c10_complex_test 2024-08-07T17:37:37.3860958Z inflating: build/bin/c10_flags_test 2024-08-07T17:37:37.3911923Z inflating: build/bin/c10_generic_math_test 2024-08-07T17:37:37.3963860Z inflating: build/bin/c10_irange_test 2024-08-07T17:37:37.4123863Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-07T17:37:37.4178117Z inflating: build/bin/c10_lazy_test 2024-08-07T17:37:37.4235931Z inflating: build/bin/c10_logging_test 2024-08-07T17:37:37.4310902Z inflating: build/bin/c10_optional_test 2024-08-07T17:37:37.4373417Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-07T17:37:37.4426927Z inflating: build/bin/c10_registry_test 2024-08-07T17:37:37.4574949Z inflating: build/bin/c10_small_vector_test 2024-08-07T17:37:37.4627251Z inflating: build/bin/c10_string_util_test 2024-08-07T17:37:37.4679815Z inflating: build/bin/c10_ssize_test 2024-08-07T17:37:37.4738987Z inflating: build/bin/c10_string_view_test 2024-08-07T17:37:37.4790118Z inflating: build/bin/c10_tempfile_test 2024-08-07T17:37:37.4837634Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-07T17:37:37.4894749Z inflating: build/bin/c10_typeid_test 2024-08-07T17:37:37.5340494Z inflating: build/bin/protoc-3.13.0.0 2024-08-07T17:37:37.5784397Z inflating: build/bin/protoc 2024-08-07T17:37:37.6110322Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-07T17:37:37.6453613Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-07T17:37:37.6807415Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-07T17:37:37.6860673Z inflating: build/bin/BackoffTest 2024-08-07T17:37:37.6915458Z inflating: build/bin/FileStoreTest 2024-08-07T17:37:37.6972351Z inflating: build/bin/TCPStoreTest 2024-08-07T17:37:37.7026397Z inflating: build/bin/HashStoreTest 2024-08-07T17:37:37.7028713Z inflating: build/bin/example_allreduce 2024-08-07T17:37:37.7095148Z inflating: build/bin/ProcessGroupGlooTest 2024-08-07T17:37:37.7149476Z inflating: build/bin/test_dist_autograd 2024-08-07T17:37:37.7151812Z inflating: build/bin/parallel_benchmark 2024-08-07T17:37:37.7220155Z inflating: build/bin/test_cpp_rpc 2024-08-07T17:37:37.7286839Z inflating: build/bin/test_mobile_nnc 2024-08-07T17:37:37.7295425Z inflating: build/bin/aot_model_compiler_test 2024-08-07T17:37:37.7353269Z inflating: build/bin/static_runtime_bench 2024-08-07T17:37:37.7686823Z inflating: build/bin/test_lazy 2024-08-07T17:37:37.7933054Z inflating: build/bin/static_runtime_test 2024-08-07T17:37:37.8006905Z inflating: build/bin/Dict_test 2024-08-07T17:37:37.8060083Z inflating: build/bin/Dimname_test 2024-08-07T17:37:37.8125914Z inflating: build/bin/MaybeOwned_test 2024-08-07T17:37:37.8183422Z inflating: build/bin/NamedTensor_test 2024-08-07T17:37:37.8243161Z inflating: build/bin/apply_utils_test 2024-08-07T17:37:37.8302688Z inflating: build/bin/atest 2024-08-07T17:37:37.8366427Z inflating: build/bin/basic 2024-08-07T17:37:37.8422264Z inflating: build/bin/broadcast_test 2024-08-07T17:37:37.8474204Z inflating: build/bin/cpu_allocator_test 2024-08-07T17:37:37.8533267Z inflating: build/bin/cpu_generator_test 2024-08-07T17:37:37.8586837Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-07T17:37:37.8686311Z inflating: build/bin/cpu_rng_test 2024-08-07T17:37:37.8737239Z inflating: build/bin/dispatch_key_set_test 2024-08-07T17:37:37.8788755Z inflating: build/bin/dlconvertor_test 2024-08-07T17:37:37.8847076Z inflating: build/bin/extension_backend_test 2024-08-07T17:37:37.8903168Z inflating: build/bin/half_test 2024-08-07T17:37:37.8957351Z inflating: build/bin/math_kernel_test 2024-08-07T17:37:37.9007863Z inflating: build/bin/lazy_tensor_test 2024-08-07T17:37:37.9102660Z inflating: build/bin/ivalue_test 2024-08-07T17:37:37.9156906Z inflating: build/bin/memory_format_test 2024-08-07T17:37:37.9210754Z inflating: build/bin/memory_overlapping_test 2024-08-07T17:37:37.9264449Z inflating: build/bin/mobile_memory_cleanup 2024-08-07T17:37:37.9320948Z inflating: build/bin/native_test 2024-08-07T17:37:37.9372374Z inflating: build/bin/operator_name_test 2024-08-07T17:37:37.9423710Z inflating: build/bin/operators_test 2024-08-07T17:37:37.9476710Z inflating: build/bin/packedtensoraccessor_test 2024-08-07T17:37:37.9543501Z inflating: build/bin/pow_test 2024-08-07T17:37:37.9601126Z inflating: build/bin/quantized_test 2024-08-07T17:37:38.0704577Z inflating: build/bin/test_api 2024-08-07T17:37:38.0754868Z inflating: build/bin/reduce_ops_test 2024-08-07T17:37:38.0807038Z inflating: build/bin/reportMemoryUsage_test 2024-08-07T17:37:38.0864141Z inflating: build/bin/scalar_tensor_test 2024-08-07T17:37:38.0916357Z inflating: build/bin/StorageUtils_test 2024-08-07T17:37:38.0975274Z inflating: build/bin/scalar_test 2024-08-07T17:37:38.1028075Z inflating: build/bin/stride_properties_test 2024-08-07T17:37:38.1083732Z inflating: build/bin/type_ptr_test 2024-08-07T17:37:38.1162354Z inflating: build/bin/tensor_iterator_test 2024-08-07T17:37:38.1216656Z inflating: build/bin/test_parallel 2024-08-07T17:37:38.1219188Z inflating: build/bin/thread_init_test 2024-08-07T17:37:38.1279482Z inflating: build/bin/type_test 2024-08-07T17:37:38.1280909Z inflating: build/bin/verify_api_visibility 2024-08-07T17:37:38.1333681Z inflating: build/bin/undefined_tensor_test 2024-08-07T17:37:38.1403526Z inflating: build/bin/legacy_vmap_test 2024-08-07T17:37:38.1455301Z inflating: build/bin/weakref_test 2024-08-07T17:37:38.1507442Z inflating: build/bin/wrapdim_test 2024-08-07T17:37:38.1613417Z inflating: build/bin/List_test 2024-08-07T17:37:38.1673308Z inflating: build/bin/IListRef_test 2024-08-07T17:37:38.1725202Z inflating: build/bin/xla_tensor_test 2024-08-07T17:37:38.1792372Z inflating: build/bin/KernelFunction_test 2024-08-07T17:37:38.1911608Z inflating: build/bin/kernel_function_legacy_test 2024-08-07T17:37:38.2006393Z inflating: build/bin/kernel_function_test 2024-08-07T17:37:38.2131956Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-07T17:37:38.2232583Z inflating: build/bin/kernel_lambda_test 2024-08-07T17:37:38.2284442Z inflating: build/bin/CppSignature_test 2024-08-07T17:37:38.2378995Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-07T17:37:38.2439500Z inflating: build/bin/kernel_stackbased_test 2024-08-07T17:37:38.2489291Z inflating: build/bin/op_allowlist_test 2024-08-07T17:37:38.2553251Z inflating: build/bin/inline_container_test 2024-08-07T17:37:38.2608779Z inflating: build/bin/backend_fallback_test 2024-08-07T17:37:38.2911263Z inflating: build/bin/op_registration_test 2024-08-07T17:37:38.2965452Z inflating: build/bin/test_edge_op_registration 2024-08-07T17:37:38.2969587Z inflating: build/bin/torch_shm_manager 2024-08-07T17:37:38.2982066Z inflating: build/bin/tutorial_tensorexpr 2024-08-07T17:37:38.3751724Z inflating: build/bin/test_tensorexpr 2024-08-07T17:37:38.4317262Z inflating: build/bin/test_jit 2024-08-07T17:37:38.4317654Z creating: .additional_ci_files/ 2024-08-07T17:37:38.4372931Z inflating: .additional_ci_files/test-times.json 2024-08-07T17:37:38.4597562Z inflating: .additional_ci_files/test-class-times.json 2024-08-07T17:37:38.4627490Z ##[group]Run rm artifacts.zip 2024-08-07T17:37:38.4627958Z rm artifacts.zip 2024-08-07T17:37:38.4634116Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:38.4634561Z env: 2024-08-07T17:37:38.4634885Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:38.4635292Z ##[endgroup] 2024-08-07T17:37:38.5057376Z ##[group]Run df -H 2024-08-07T17:37:38.5057666Z df -H 2024-08-07T17:37:38.5063625Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:38.5064035Z env: 2024-08-07T17:37:38.5064256Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:38.5064528Z ##[endgroup] 2024-08-07T17:37:38.5176052Z Filesystem Size Used Avail Use% Mounted on 2024-08-07T17:37:38.5176499Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-07T17:37:38.5176839Z tmpfs 8.2G 426k 8.2G 1% /dev/shm 2024-08-07T17:37:38.5177188Z tmpfs 3.3G 488k 3.3G 1% /run 2024-08-07T17:37:38.5177526Z /dev/nvme0n1p1 161G 26G 135G 17% / 2024-08-07T17:37:38.5177850Z tmpfs 8.2G 17k 8.2G 1% /tmp 2024-08-07T17:37:38.5178320Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-07T17:37:38.5257201Z Prepare all required actions 2024-08-07T17:37:38.5257596Z Getting action download info 2024-08-07T17:37:38.6549718Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-07T17:37:38.6550091Z with: 2024-08-07T17:37:38.6550289Z env: 2024-08-07T17:37:38.6550511Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:38.6550786Z ##[endgroup] 2024-08-07T17:37:38.6674460Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-07T17:37:38.6674830Z with: 2024-08-07T17:37:38.6675051Z name: td_results 2024-08-07T17:37:38.6675289Z s3-bucket: gha-artifacts 2024-08-07T17:37:38.6675587Z region: us-east-1 2024-08-07T17:37:38.6675828Z env: 2024-08-07T17:37:38.6676034Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:38.6676302Z ##[endgroup] 2024-08-07T17:37:39.1497189Z (node:477318) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-07T17:37:39.1497733Z 2024-08-07T17:37:39.1498018Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-07T17:37:39.1498572Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-07T17:37:39.1499152Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-07T17:37:39.2188435Z Found 1 objects with prefix pytorch/pytorch/10288745067/td_results/ 2024-08-07T17:37:39.2189128Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-07T17:37:39.2701129Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-07T17:37:39.2706883Z Artifact download has finished successfully 2024-08-07T17:37:39.3024223Z ##[group]Run mkdir -p .additional_ci_files 2024-08-07T17:37:39.3024635Z mkdir -p .additional_ci_files 2024-08-07T17:37:39.3025048Z mv td_results.json .additional_ci_files/td_results.json 2024-08-07T17:37:39.3031655Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:39.3032059Z env: 2024-08-07T17:37:39.3032300Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:39.3032579Z ##[endgroup] 2024-08-07T17:37:39.3453224Z ##[group]Run .github/scripts/parse_ref.py 2024-08-07T17:37:39.3453639Z .github/scripts/parse_ref.py 2024-08-07T17:37:39.3459265Z shell: /usr/bin/bash -e {0} 2024-08-07T17:37:39.3459541Z env: 2024-08-07T17:37:39.3459771Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:39.3460048Z ##[endgroup] 2024-08-07T17:37:39.3790330Z Prepare all required actions 2024-08-07T17:37:39.3983132Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-07T17:37:39.3983527Z with: 2024-08-07T17:37:39.3983991Z github-token: *** 2024-08-07T17:37:39.3984235Z env: 2024-08-07T17:37:39.3984462Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:39.3984722Z ##[endgroup] 2024-08-07T17:37:39.4109950Z ##[group]Run set -eux 2024-08-07T17:37:39.4110266Z set -eux 2024-08-07T17:37:39.4110716Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-07T17:37:39.4117087Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:39.4117487Z env: 2024-08-07T17:37:39.4117705Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:39.4118308Z GITHUB_TOKEN: *** 2024-08-07T17:37:39.4118563Z ##[endgroup] 2024-08-07T17:37:39.4142994Z + python3 .github/scripts/get_workflow_job_id.py 10288745067 i-036a6435dcb00f35e 2024-08-07T17:37:41.4935506Z setting job-id=28475634473 2024-08-07T17:37:41.4936158Z setting job-name=linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-07T17:37:41.5174367Z Prepare all required actions 2024-08-07T17:37:41.5174819Z Getting action download info 2024-08-07T17:37:41.6321990Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-07T17:37:41.6322357Z with: 2024-08-07T17:37:41.6322804Z github-token: *** 2024-08-07T17:37:41.6325585Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]} 2024-08-07T17:37:41.6328530Z job-name: linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-07T17:37:41.6329011Z env: 2024-08-07T17:37:41.6329249Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:41.6329575Z ##[endgroup] 2024-08-07T17:37:41.6423188Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-07T17:37:41.6423616Z with: 2024-08-07T17:37:41.6423844Z shell: bash 2024-08-07T17:37:41.6424082Z timeout_minutes: 10 2024-08-07T17:37:41.6424327Z max_attempts: 5 2024-08-07T17:37:41.6424576Z retry_wait_seconds: 30 2024-08-07T17:37:41.6425391Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-07T17:37:41.6426246Z polling_interval_seconds: 1 2024-08-07T17:37:41.6426541Z warning_on_retry: true 2024-08-07T17:37:41.6426812Z continue_on_error: false 2024-08-07T17:37:41.6427223Z env: 2024-08-07T17:37:41.6427448Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:41.6428060Z GITHUB_TOKEN: *** 2024-08-07T17:37:41.6428309Z ##[endgroup] 2024-08-07T17:37:41.7190480Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-07T17:37:41.9595489Z Defaulting to user installation because normal site-packages is not writeable 2024-08-07T17:37:41.9758451Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.9/site-packages (2.27.1) 2024-08-07T17:37:41.9762210Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.9/site-packages (6.0.1) 2024-08-07T17:37:41.9875327Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2024.7.4) 2024-08-07T17:37:41.9885505Z Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2.0.12) 2024-08-07T17:37:41.9894011Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-08-07T17:37:41.9899700Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-08-07T17:37:42.6962725Z Command completed after 1 attempt(s). 2024-08-07T17:37:42.7069794Z ##[group]Run set -x 2024-08-07T17:37:42.7070080Z set -x 2024-08-07T17:37:42.7070321Z  2024-08-07T17:37:42.7070718Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-07T17:37:42.7071221Z # in runner workspace 2024-08-07T17:37:42.7071625Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-07T17:37:42.7077422Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:42.7077822Z env: 2024-08-07T17:37:42.7078061Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:42.7078505Z ##[endgroup] 2024-08-07T17:37:42.7101259Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-07T17:37:42.7422779Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-07T17:37:42.7423264Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-07T17:37:42.7423612Z echo "Job name: ${JOB_NAME}" 2024-08-07T17:37:42.7423923Z  2024-08-07T17:37:42.7424319Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-07T17:37:42.7424804Z # in runner workspace 2024-08-07T17:37:42.7425250Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-07T17:37:42.7425748Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-07T17:37:42.7426107Z  --job-name "${JOB_NAME}" \ 2024-08-07T17:37:42.7428901Z  --test-matrix "{"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]}" \ 2024-08-07T17:37:42.7431724Z  --selected-test-configs "" \ 2024-08-07T17:37:42.7432067Z  --pr-number "${PR_NUMBER}" \ 2024-08-07T17:37:42.7432393Z  --tag "${TAG}" \ 2024-08-07T17:37:42.7432702Z  --event-name "${EVENT_NAME}" \ 2024-08-07T17:37:42.7433138Z  --schedule "${SCHEDULE}" \ 2024-08-07T17:37:42.7433465Z  --branch "${HEAD_BRANCH}" 2024-08-07T17:37:42.7438986Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:42.7439365Z env: 2024-08-07T17:37:42.7439649Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:42.7440151Z GITHUB_TOKEN: *** 2024-08-07T17:37:42.7440559Z JOB_NAME: linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-07T17:37:42.7441048Z PR_NUMBER: 131248 2024-08-07T17:37:42.7441296Z TAG: 2024-08-07T17:37:42.7441514Z EVENT_NAME: pull_request 2024-08-07T17:37:42.7441793Z SCHEDULE: 2024-08-07T17:37:42.7442027Z HEAD_BRANCH: 2024-08-07T17:37:42.7442250Z ##[endgroup] 2024-08-07T17:37:42.7463738Z Workflow: pull 2024-08-07T17:37:42.7464465Z Job name: linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-07T17:37:43.0349831Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-07T17:37:43.2341906Z ##[group]Run echo "Filtered matrix:" 2024-08-07T17:37:43.2342298Z echo "Filtered matrix:" 2024-08-07T17:37:43.2345080Z echo "{"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]}" 2024-08-07T17:37:43.2347806Z  2024-08-07T17:37:43.2348157Z echo 2024-08-07T17:37:43.2348452Z echo "Is the current job unstable? False" 2024-08-07T17:37:43.2348816Z  2024-08-07T17:37:43.2349026Z echo 2024-08-07T17:37:43.2349305Z echo "Is keep-going label set? False" 2024-08-07T17:37:43.2349647Z  2024-08-07T17:37:43.2349858Z echo 2024-08-07T17:37:43.2350111Z echo "Renabled issues? " 2024-08-07T17:37:43.2355834Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:43.2356217Z env: 2024-08-07T17:37:43.2356451Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:43.2356730Z ##[endgroup] 2024-08-07T17:37:43.2379017Z Filtered matrix: 2024-08-07T17:37:43.2382330Z {include: [{config: default, shard: 1, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: default, shard: 2, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: default, shard: 3, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: default, shard: 4, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: docs_test, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: jit_legacy, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: backwards_compat, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: distributed, shard: 1, num_shards: 2, runner: amz2023.linux.2xlarge}, {config: distributed, shard: 2, num_shards: 2, runner: amz2023.linux.2xlarge}]} 2024-08-07T17:37:43.2384903Z 2024-08-07T17:37:43.2385032Z Is the current job unstable? False 2024-08-07T17:37:43.2385253Z 2024-08-07T17:37:43.2385364Z Is keep-going label set? False 2024-08-07T17:37:43.2385554Z 2024-08-07T17:37:43.2385659Z Renabled issues? 2024-08-07T17:37:43.2515270Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-07T17:37:43.2515822Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-07T17:37:43.2521371Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:43.2521760Z env: 2024-08-07T17:37:43.2521987Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:43.2522272Z JOB_TIMEOUT: 240 2024-08-07T17:37:43.2522502Z ##[endgroup] 2024-08-07T17:37:43.2644195Z ##[group]Run set -x 2024-08-07T17:37:43.2644524Z set -x 2024-08-07T17:37:43.2644748Z  2024-08-07T17:37:43.2645017Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-07T17:37:43.2645429Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-07T17:37:43.2645832Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-07T17:37:43.2646214Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-07T17:37:43.2646532Z else 2024-08-07T17:37:43.2646784Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-07T17:37:43.2647111Z fi 2024-08-07T17:37:43.2647333Z  2024-08-07T17:37:43.2647675Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-07T17:37:43.2648233Z # TODO: Stop building test binaries as part of the build phase 2024-08-07T17:37:43.2648722Z # Used for GPU_FLAG since that doesn't play nice 2024-08-07T17:37:43.2649162Z # shellcheck disable=SC2086,SC2090 2024-08-07T17:37:43.2649516Z container_name=$(docker run \ 2024-08-07T17:37:43.2649843Z  ${GPU_FLAG:-} \ 2024-08-07T17:37:43.2650142Z  -e BUILD_ENVIRONMENT \ 2024-08-07T17:37:43.2650444Z  -e PR_NUMBER \ 2024-08-07T17:37:43.2650728Z  -e GITHUB_ACTIONS \ 2024-08-07T17:37:43.2651029Z  -e GITHUB_REPOSITORY \ 2024-08-07T17:37:43.2651330Z  -e GITHUB_WORKFLOW \ 2024-08-07T17:37:43.2651628Z  -e GITHUB_JOB \ 2024-08-07T17:37:43.2651909Z  -e GITHUB_RUN_ID \ 2024-08-07T17:37:43.2652189Z  -e GITHUB_RUN_NUMBER \ 2024-08-07T17:37:43.2652499Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-07T17:37:43.2652801Z  -e JOB_ID \ 2024-08-07T17:37:43.2653051Z  -e JOB_NAME \ 2024-08-07T17:37:43.2653322Z  -e BASE_SHA \ 2024-08-07T17:37:43.2653589Z  -e BRANCH \ 2024-08-07T17:37:43.2653834Z  -e SHA1 \ 2024-08-07T17:37:43.2654104Z  -e AWS_DEFAULT_REGION \ 2024-08-07T17:37:43.2654414Z  -e IN_WHEEL_TEST \ 2024-08-07T17:37:43.2654691Z  -e SHARD_NUMBER \ 2024-08-07T17:37:43.2654976Z  -e TEST_CONFIG \ 2024-08-07T17:37:43.2655267Z  -e NUM_TEST_SHARDS \ 2024-08-07T17:37:43.2655556Z  -e REENABLED_ISSUES \ 2024-08-07T17:37:43.2655871Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-07T17:37:43.2656197Z  -e VERBOSE_TEST_LOGS \ 2024-08-07T17:37:43.2656493Z  -e TEST_SHOWLOCALS \ 2024-08-07T17:37:43.2656792Z  -e NO_TEST_TIMEOUT \ 2024-08-07T17:37:43.2657079Z  -e NO_TD \ 2024-08-07T17:37:43.2657332Z  -e TD_DISTRIBUTED \ 2024-08-07T17:37:43.2657626Z  -e PR_LABELS \ 2024-08-07T17:37:43.2657932Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-07T17:37:43.2658267Z  -e SCCACHE_BUCKET \ 2024-08-07T17:37:43.2658570Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-07T17:37:43.2658882Z  -e XLA_CUDA \ 2024-08-07T17:37:43.2659181Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-07T17:37:43.2659558Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-07T17:37:43.2659945Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-07T17:37:43.2660324Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-07T17:37:43.2660684Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-07T17:37:43.2661014Z  -e DASHBOARD_TAG \ 2024-08-07T17:37:43.2661352Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-07T17:37:43.2661764Z  --security-opt seccomp=unconfined \ 2024-08-07T17:37:43.2662122Z  --cap-add=SYS_PTRACE \ 2024-08-07T17:37:43.2662417Z  --ipc=host \ 2024-08-07T17:37:43.2662698Z  --shm-size="${SHM_SIZE}" \ 2024-08-07T17:37:43.2663232Z  --tty \ 2024-08-07T17:37:43.2663473Z  --detach \ 2024-08-07T17:37:43.2663756Z  --name="${container_name}" \ 2024-08-07T17:37:43.2664252Z  --user jenkins \ 2024-08-07T17:37:43.2664710Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-07T17:37:43.2665137Z  -w /var/lib/jenkins/workspace \ 2024-08-07T17:37:43.2665478Z  "${DOCKER_IMAGE}" 2024-08-07T17:37:43.2665743Z ) 2024-08-07T17:37:43.2666051Z # Propagate download.pytorch.org IP to container 2024-08-07T17:37:43.2666745Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-07T17:37:43.2667474Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-07T17:37:43.2668152Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-07T17:37:43.2673547Z shell: /usr/bin/bash -e {0} 2024-08-07T17:37:43.2673817Z env: 2024-08-07T17:37:43.2674038Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:43.2674357Z BUILD_ENVIRONMENT: linux-jammy-py3.8-gcc11 2024-08-07T17:37:43.2674701Z PR_NUMBER: 131248 2024-08-07T17:37:43.2674968Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-07T17:37:43.2675286Z GITHUB_WORKFLOW: pull 2024-08-07T17:37:43.2675533Z GITHUB_JOB: test 2024-08-07T17:37:43.2675776Z GITHUB_RUN_ID: 10288745067 2024-08-07T17:37:43.2676059Z GITHUB_RUN_NUMBER: 234358 2024-08-07T17:37:43.2676324Z GITHUB_RUN_ATTEMPT: 1 2024-08-07T17:37:43.2676578Z JOB_ID: 28475634473 2024-08-07T17:37:43.2676997Z JOB_NAME: linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-07T17:37:43.2677469Z BRANCH: pull/131248 2024-08-07T17:37:43.2677756Z SHA1: 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:37:43.2678143Z BASE_SHA: 6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T17:37:43.2678494Z TEST_CONFIG: distributed 2024-08-07T17:37:43.2678752Z SHARD_NUMBER: 2 2024-08-07T17:37:43.2678990Z NUM_TEST_SHARDS: 2 2024-08-07T17:37:43.2679244Z REENABLED_ISSUES: 2024-08-07T17:37:43.2679495Z CONTINUE_THROUGH_ERROR: False 2024-08-07T17:37:43.2679801Z VERBOSE_TEST_LOGS: False 2024-08-07T17:37:43.2680077Z TEST_SHOWLOCALS: False 2024-08-07T17:37:43.2680337Z NO_TEST_TIMEOUT: False 2024-08-07T17:37:43.2680597Z NO_TD: False 2024-08-07T17:37:43.2680835Z TD_DISTRIBUTED: False 2024-08-07T17:37:43.2681149Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-07T17:37:43.2681527Z SCCACHE_S3_KEY_PREFIX: pull 2024-08-07T17:37:43.2681810Z SHM_SIZE: 1g 2024-08-07T17:37:43.2682468Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:37:43.2683208Z XLA_CUDA: 2024-08-07T17:37:43.2683577Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-07T17:37:43.2684036Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-07T17:37:43.2684372Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-07T17:37:43.2684685Z DASHBOARD_TAG: 2024-08-07T17:37:43.2684920Z HUGGING_FACE_HUB_TOKEN: 2024-08-07T17:37:43.2685183Z ##[endgroup] 2024-08-07T17:37:43.2706871Z + [[ distributed == \m\u\l\t\i\g\p\u ]] 2024-08-07T17:37:43.2707304Z + [[ linux-jammy-py3.8-gcc11 == *onnx* ]] 2024-08-07T17:37:43.2707631Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-07T17:37:43.2714503Z +++ nproc --ignore=2 2024-08-07T17:37:43.2793972Z ++ docker run -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=6 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10288745067 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=1g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:37:54.2167997Z + container_name=75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T17:37:54.2172588Z + grep download.pytorch.org /etc/hosts 2024-08-07T17:37:54.2173662Z + docker exec -i 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-07T17:37:54.3619027Z + echo DOCKER_CONTAINER_ID=75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T17:37:54.3622729Z ++ echo dist/torch-2.5.0a0+git016588f-cp38-cp38-linux_x86_64.whl 2024-08-07T17:37:54.3626645Z + docker exec -t 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa sh -c 'pip install dist/torch-2.5.0a0+git016588f-cp38-cp38-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-07T17:37:55.2265870Z Processing ./dist/torch-2.5.0a0+git016588f-cp38-cp38-linux_x86_64.whl (from torch==2.5.0a0+git016588f) 2024-08-07T17:37:55.6592436Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (3.13.1) 2024-08-07T17:37:55.6598300Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (4.12.2) 2024-08-07T17:37:55.6601606Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (2.8.8) 2024-08-07T17:37:55.6605093Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (3.1.4) 2024-08-07T17:37:55.6608639Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (2024.6.1) 2024-08-07T17:37:55.6617706Z Requirement already satisfied: sympy==1.12.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (1.12.1) 2024-08-07T17:37:55.6639242Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy==1.12.1->torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (1.3.0) 2024-08-07T17:37:55.6664853Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (3.3.0) 2024-08-07T17:37:55.6726850Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (1.21.2) 2024-08-07T17:37:55.7578351Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (2.1.5) 2024-08-07T17:37:56.9940742Z Installing collected packages: torch 2024-08-07T17:38:04.7875706Z Successfully installed torch-2.5.0a0+git016588f 2024-08-07T17:38:04.8591336Z ++ dirname .ci/pytorch/test.sh 2024-08-07T17:38:04.8607151Z + source .ci/pytorch/common.sh 2024-08-07T17:38:04.8614133Z +++ dirname .ci/pytorch/common.sh 2024-08-07T17:38:04.8620194Z ++ source .ci/pytorch/common_utils.sh 2024-08-07T17:38:04.8624801Z +++ declare -f -t trap_add 2024-08-07T17:38:04.8628390Z ++ set -ex 2024-08-07T17:38:04.8628746Z ++ [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-07T17:38:04.8629188Z ++ BUILD_TEST_LIBTORCH=0 2024-08-07T17:38:04.8629817Z + [[ linux-jammy-py3.8-gcc11 != *rocm* ]] 2024-08-07T17:38:04.8632724Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-07T17:38:04.8673948Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-07T17:38:04.8674738Z + trap_add cleanup_workspace EXIT 2024-08-07T17:38:04.8675207Z + trap_add_cmd=cleanup_workspace 2024-08-07T17:38:04.8675591Z + shift 2024-08-07T17:38:04.8675828Z + for trap_add_name in "$@" 2024-08-07T17:38:04.8680455Z +++ trap -p EXIT 2024-08-07T17:38:04.8682496Z ++ eval 'extract_trap_cmd ' 2024-08-07T17:38:04.8682802Z +++ extract_trap_cmd 2024-08-07T17:38:04.8683067Z +++ printf '%s\n' '' 2024-08-07T17:38:04.8683404Z ++ printf '%s\n' cleanup_workspace 2024-08-07T17:38:04.8684264Z + trap -- ' 2024-08-07T17:38:04.8684510Z cleanup_workspace' EXIT 2024-08-07T17:38:04.8684839Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-07T17:38:05.3686642Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-07T17:38:05.3982576Z + echo 'Environment variables:' 2024-08-07T17:38:05.3983161Z Environment variables: 2024-08-07T17:38:05.3983442Z + env 2024-08-07T17:38:05.4001662Z INSTALLED_DB=yes 2024-08-07T17:38:05.4002357Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:38:05.4003106Z CONTINUE_THROUGH_ERROR=False 2024-08-07T17:38:05.4003489Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11 2024-08-07T17:38:05.4003894Z HOSTNAME=75b1949dfde8 2024-08-07T17:38:05.4004993Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_781867e6-5949-4f09-baa3-a008a87d0e71 2024-08-07T17:38:05.4005642Z GITHUB_ACTION=__self 2024-08-07T17:38:05.4005922Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-07T17:38:05.4006226Z GITHUB_RUN_NUMBER=234358 2024-08-07T17:38:05.4006497Z TEST_CONFIG=distributed 2024-08-07T17:38:05.4006883Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-07T17:38:05.4007272Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-07T17:38:05.4007750Z GITHUB_TRIGGERING_ACTOR=zdevito 2024-08-07T17:38:05.4008032Z GITHUB_REF_TYPE=branch 2024-08-07T17:38:05.4008300Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-07T17:38:05.4008624Z BASE_SHA=6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T17:38:05.4008947Z XLA_CUDA= 2024-08-07T17:38:05.4009228Z HUGGING_FACE_HUB_TOKEN= 2024-08-07T17:38:05.4009738Z *** 2024-08-07T17:38:05.4009950Z GITHUB_REPOSITORY_ID=65600975 2024-08-07T17:38:05.4010242Z GITHUB_ACTIONS=true 2024-08-07T17:38:05.4010560Z SHA1=016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:38:05.4010924Z GITHUB_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T17:38:05.4011480Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/131248/merge 2024-08-07T17:38:05.4012071Z UCC_HOME=/usr 2024-08-07T17:38:05.4012330Z VERBOSE_TEST_LOGS=False 2024-08-07T17:38:05.4012612Z GITHUB_REF=refs/pull/131248/merge 2024-08-07T17:38:05.4012898Z SHARD_NUMBER=2 2024-08-07T17:38:05.4013150Z GITHUB_REF_PROTECTED=false 2024-08-07T17:38:05.4013419Z HOME=/var/lib/jenkins 2024-08-07T17:38:05.4013716Z GITHUB_API_URL=https://api.github.com 2024-08-07T17:38:05.4014062Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-07T17:38:05.4014358Z UCX_COMMIT= 2024-08-07T17:38:05.4014601Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-07T17:38:05.4014882Z NUM_TEST_SHARDS=2 2024-08-07T17:38:05.4015125Z UCX_HOME=/usr 2024-08-07T17:38:05.4015711Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_781867e6-5949-4f09-baa3-a008a87d0e71 2024-08-07T17:38:05.4016534Z JOB_NAME=linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-07T17:38:05.4017321Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_781867e6-5949-4f09-baa3-a008a87d0e71 2024-08-07T17:38:05.4018151Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-07T17:38:05.4018710Z GITHUB_EVENT_NAME=pull_request 2024-08-07T17:38:05.4019004Z DASHBOARD_TAG= 2024-08-07T17:38:05.4019241Z GITHUB_RUN_ID=10288745067 2024-08-07T17:38:05.4019894Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_781867e6-5949-4f09-baa3-a008a87d0e71 2024-08-07T17:38:05.4020586Z GITHUB_ACTOR=zdevito 2024-08-07T17:38:05.4020828Z PR_NUMBER=131248 2024-08-07T17:38:05.4021072Z DESIRED_CUDA= 2024-08-07T17:38:05.4022123Z GITHUB_RUN_ATTEMPT=1 2024-08-07T17:38:05.4022380Z ANACONDA_PYTHON_VERSION=3.8 2024-08-07T17:38:05.4022725Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-07T17:38:05.4023086Z TERM=xterm 2024-08-07T17:38:05.4023302Z INSTALLED_VISION=yes 2024-08-07T17:38:05.4023556Z BRANCH=pull/131248 2024-08-07T17:38:05.4023811Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-07T17:38:05.4024095Z CUDA_PATH=/usr/local/cuda 2024-08-07T17:38:05.4024633Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-07T17:38:05.4025228Z GITHUB_SERVER_URL=https://github.com 2024-08-07T17:38:05.4025533Z UCC_COMMIT= 2024-08-07T17:38:05.4025760Z REENABLED_ISSUES= 2024-08-07T17:38:05.4025990Z DOCS=yes 2024-08-07T17:38:05.4026213Z INSTALLED_ANDROID= 2024-08-07T17:38:05.4026454Z SHLVL=1 2024-08-07T17:38:05.4026650Z MAX_JOBS=6 2024-08-07T17:38:05.4026878Z GITHUB_ACTOR_ID=370202 2024-08-07T17:38:05.4027227Z GITHUB_WORKFLOW_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T17:38:05.4027618Z GITHUB_REF_NAME=131248/merge 2024-08-07T17:38:05.4028037Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-07T17:38:05.4028479Z GITHUB_JOB=test 2024-08-07T17:38:05.4028710Z NO_TEST_TIMEOUT=False 2024-08-07T17:38:05.4028971Z TD_DISTRIBUTED=False 2024-08-07T17:38:05.4029245Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-07T17:38:05.4029543Z GITHUB_RETENTION_DAYS=90 2024-08-07T17:38:05.4029821Z OPENSSL_DIR=/opt/openssl 2024-08-07T17:38:05.4030101Z GITHUB_ACTION_REPOSITORY= 2024-08-07T17:38:05.4030888Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T17:38:05.4031731Z GITHUB_BASE_REF=gh/zdevito/267/base 2024-08-07T17:38:05.4032041Z INSTALLED_ACL= 2024-08-07T17:38:05.4032257Z CI=true 2024-08-07T17:38:05.4032494Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-07T17:38:05.4032781Z JOB_ID=28475634473 2024-08-07T17:38:05.4033121Z INSTALLED_PROTOBUF=yes 2024-08-07T17:38:05.4033415Z GITHUB_HEAD_REF=gh/zdevito/267/head 2024-08-07T17:38:05.4033726Z GITHUB_ACTION_REF= 2024-08-07T17:38:05.4034019Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-07T17:38:05.4034390Z TEST_SHOWLOCALS=False 2024-08-07T17:38:05.4034641Z GITHUB_WORKFLOW=pull 2024-08-07T17:38:05.4034917Z DEBIAN_FRONTEND=noninteractive 2024-08-07T17:38:05.4035559Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_781867e6-5949-4f09-baa3-a008a87d0e71 2024-08-07T17:38:05.4036215Z NO_TD=False 2024-08-07T17:38:05.4036443Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-07T17:38:05.4036731Z _=/usr/bin/env 2024-08-07T17:38:05.4037046Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-07T17:38:05.4315633Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch 2024-08-07T17:38:05.4316506Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-08-07T17:38:05.4317290Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib 2024-08-07T17:38:05.4317968Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/test 2024-08-07T17:38:05.4318489Z + BUILD_DIR=build 2024-08-07T17:38:05.4318751Z + BUILD_RENAMED_DIR=build_renamed 2024-08-07T17:38:05.4319047Z + BUILD_BIN_DIR=build/bin 2024-08-07T17:38:05.4319373Z + SHARD_NUMBER=2 2024-08-07T17:38:05.4319611Z + NUM_TEST_SHARDS=2 2024-08-07T17:38:05.4319843Z + export VALGRIND=ON 2024-08-07T17:38:05.4320155Z + VALGRIND=ON 2024-08-07T17:38:05.4320450Z + [[ linux-jammy-py3.8-gcc11 == *clang9* ]] 2024-08-07T17:38:05.4320923Z + [[ 0 == \1 ]] 2024-08-07T17:38:05.4321174Z + [[ False == \1 ]] 2024-08-07T17:38:05.4321538Z + [[ linux-jammy-py3.8-gcc11 != *bazel* ]] 2024-08-07T17:38:05.4321951Z ++ realpath build/custom_test_artifacts 2024-08-07T17:38:05.4339718Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-07T17:38:05.4340238Z + [[ -n '' ]] 2024-08-07T17:38:05.4340677Z + echo 'Environment variables' 2024-08-07T17:38:05.4341097Z Environment variables 2024-08-07T17:38:05.4341349Z + env 2024-08-07T17:38:05.4346350Z INSTALLED_DB=yes 2024-08-07T17:38:05.4346998Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:38:05.4347835Z CONTINUE_THROUGH_ERROR=False 2024-08-07T17:38:05.4348356Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11 2024-08-07T17:38:05.4348924Z HOSTNAME=75b1949dfde8 2024-08-07T17:38:05.4350003Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_781867e6-5949-4f09-baa3-a008a87d0e71 2024-08-07T17:38:05.4351140Z GITHUB_ACTION=__self 2024-08-07T17:38:05.4351622Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-07T17:38:05.4352181Z GITHUB_RUN_NUMBER=234358 2024-08-07T17:38:05.4352652Z TEST_CONFIG=distributed 2024-08-07T17:38:05.4353137Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-07T17:38:05.4353484Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-07T17:38:05.4353816Z GITHUB_TRIGGERING_ACTOR=zdevito 2024-08-07T17:38:05.4354172Z GITHUB_REF_TYPE=branch 2024-08-07T17:38:05.4354447Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-07T17:38:05.4354873Z BASE_SHA=6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T17:38:05.4355216Z XLA_CUDA= 2024-08-07T17:38:05.4355451Z HUGGING_FACE_HUB_TOKEN= 2024-08-07T17:38:05.4355841Z *** 2024-08-07T17:38:05.4356071Z GITHUB_REPOSITORY_ID=65600975 2024-08-07T17:38:05.4356367Z GITHUB_ACTIONS=true 2024-08-07T17:38:05.4356635Z SHA1=016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:38:05.4357012Z GITHUB_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T17:38:05.4357567Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/131248/merge 2024-08-07T17:38:05.4358061Z UCC_HOME=/usr 2024-08-07T17:38:05.4358303Z VERBOSE_TEST_LOGS=False 2024-08-07T17:38:05.4358573Z GITHUB_REF=refs/pull/131248/merge 2024-08-07T17:38:05.4358907Z SHARD_NUMBER=2 2024-08-07T17:38:05.4359328Z GITHUB_REF_PROTECTED=false 2024-08-07T17:38:05.4359612Z HOME=/var/lib/jenkins 2024-08-07T17:38:05.4359896Z GITHUB_API_URL=https://api.github.com 2024-08-07T17:38:05.4360322Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-07T17:38:05.4360623Z UCX_COMMIT= 2024-08-07T17:38:05.4360847Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-07T17:38:05.4361124Z NUM_TEST_SHARDS=2 2024-08-07T17:38:05.4361361Z UCX_HOME=/usr 2024-08-07T17:38:05.4361929Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_781867e6-5949-4f09-baa3-a008a87d0e71 2024-08-07T17:38:05.4362742Z JOB_NAME=linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-07T17:38:05.4363738Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_781867e6-5949-4f09-baa3-a008a87d0e71 2024-08-07T17:38:05.4364551Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-07T17:38:05.4365078Z GITHUB_EVENT_NAME=pull_request 2024-08-07T17:38:05.4365374Z DASHBOARD_TAG= 2024-08-07T17:38:05.4365606Z GITHUB_RUN_ID=10288745067 2024-08-07T17:38:05.4366256Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_781867e6-5949-4f09-baa3-a008a87d0e71 2024-08-07T17:38:05.4366951Z GITHUB_ACTOR=zdevito 2024-08-07T17:38:05.4367200Z PR_NUMBER=131248 2024-08-07T17:38:05.4367439Z DESIRED_CUDA= 2024-08-07T17:38:05.4367674Z GITHUB_RUN_ATTEMPT=1 2024-08-07T17:38:05.4367914Z VALGRIND=ON 2024-08-07T17:38:05.4368152Z ANACONDA_PYTHON_VERSION=3.8 2024-08-07T17:38:05.4368495Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-07T17:38:05.4368845Z TERM=xterm 2024-08-07T17:38:05.4369073Z INSTALLED_VISION=yes 2024-08-07T17:38:05.4369326Z BRANCH=pull/131248 2024-08-07T17:38:05.4369577Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-07T17:38:05.4369872Z CUDA_PATH=/usr/local/cuda 2024-08-07T17:38:05.4370409Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-07T17:38:05.4371042Z GITHUB_SERVER_URL=https://github.com 2024-08-07T17:38:05.4371418Z UCC_COMMIT= 2024-08-07T17:38:05.4371635Z REENABLED_ISSUES= 2024-08-07T17:38:05.4372169Z DOCS=yes 2024-08-07T17:38:05.4372393Z INSTALLED_ANDROID= 2024-08-07T17:38:05.4372621Z SHLVL=1 2024-08-07T17:38:05.4372833Z MAX_JOBS=6 2024-08-07T17:38:05.4373064Z GITHUB_ACTOR_ID=370202 2024-08-07T17:38:05.4373400Z GITHUB_WORKFLOW_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T17:38:05.4373805Z GITHUB_REF_NAME=131248/merge 2024-08-07T17:38:05.4374225Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-07T17:38:05.4374655Z GITHUB_JOB=test 2024-08-07T17:38:05.4374899Z NO_TEST_TIMEOUT=False 2024-08-07T17:38:05.4375162Z TD_DISTRIBUTED=False 2024-08-07T17:38:05.4375426Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-07T17:38:05.4375741Z GITHUB_RETENTION_DAYS=90 2024-08-07T17:38:05.4376020Z OPENSSL_DIR=/opt/openssl 2024-08-07T17:38:05.4376286Z GITHUB_ACTION_REPOSITORY= 2024-08-07T17:38:05.4377083Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T17:38:05.4377931Z GITHUB_BASE_REF=gh/zdevito/267/base 2024-08-07T17:38:05.4378230Z INSTALLED_ACL= 2024-08-07T17:38:05.4378458Z CI=true 2024-08-07T17:38:05.4378687Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-07T17:38:05.4378960Z JOB_ID=28475634473 2024-08-07T17:38:05.4379211Z INSTALLED_PROTOBUF=yes 2024-08-07T17:38:05.4379485Z GITHUB_HEAD_REF=gh/zdevito/267/head 2024-08-07T17:38:05.4379779Z GITHUB_ACTION_REF= 2024-08-07T17:38:05.4380083Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-07T17:38:05.4380448Z TEST_SHOWLOCALS=False 2024-08-07T17:38:05.4380697Z GITHUB_WORKFLOW=pull 2024-08-07T17:38:05.4380971Z DEBIAN_FRONTEND=noninteractive 2024-08-07T17:38:05.4381602Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_781867e6-5949-4f09-baa3-a008a87d0e71 2024-08-07T17:38:05.4382253Z NO_TD=False 2024-08-07T17:38:05.4382505Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-07T17:38:05.4382787Z _=/usr/bin/env 2024-08-07T17:38:05.4383035Z + echo 'Testing pytorch' 2024-08-07T17:38:05.4383323Z Testing pytorch 2024-08-07T17:38:05.4383577Z + export LANG=C.UTF-8 2024-08-07T17:38:05.4383837Z + LANG=C.UTF-8 2024-08-07T17:38:05.4407948Z + PR_NUMBER=131248 2024-08-07T17:38:05.4408452Z + [[ distributed == \d\e\f\a\u\l\t ]] 2024-08-07T17:38:05.4409029Z + [[ distributed == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-07T17:38:05.4409588Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-07T17:38:05.4410039Z + [[ distributed == \s\l\o\w ]] 2024-08-07T17:38:05.4410510Z + [[ linux-jammy-py3.8-gcc11 == *slow-gradcheck* ]] 2024-08-07T17:38:05.4410943Z + [[ linux-jammy-py3.8-gcc11 == *cuda* ]] 2024-08-07T17:38:05.4411304Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-07T17:38:05.4411653Z + [[ linux-jammy-py3.8-gcc11 == *xpu* ]] 2024-08-07T17:38:05.4411984Z + [[ distributed == *crossref* ]] 2024-08-07T17:38:05.4412312Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-07T17:38:05.4412645Z + [[ linux-jammy-py3.8-gcc11 == *xpu* ]] 2024-08-07T17:38:05.4413115Z + [[ linux-jammy-py3.8-gcc11 != *-bazel-* ]] 2024-08-07T17:38:05.4413502Z + pip_install --user ninja==1.10.2 2024-08-07T17:38:05.4413865Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-07T17:38:07.3760211Z Collecting ninja==1.10.2 2024-08-07T17:38:07.3930834Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-07T17:38:07.4491079Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-07T17:38:08.7001559Z Installing collected packages: ninja 2024-08-07T17:38:08.7096491Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-07T17:38:08.7097488Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-07T17:38:08.7148144Z Successfully installed ninja-1.10.2 2024-08-07T17:38:08.8176015Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T17:38:08.8177803Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T17:38:08.8178775Z + [[ linux-jammy-py3.8-gcc11 == *aarch64* ]] 2024-08-07T17:38:08.8179116Z + install_tlparse 2024-08-07T17:38:08.8179389Z + pip_install --user tlparse==0.3.7 2024-08-07T17:38:08.8179761Z + pip install --progress-bar off --user tlparse==0.3.7 2024-08-07T17:38:09.2504848Z Collecting tlparse==0.3.7 2024-08-07T17:38:09.2652708Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-08-07T17:38:09.2736889Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-07T17:38:10.4953743Z Installing collected packages: tlparse 2024-08-07T17:38:10.5357191Z Successfully installed tlparse-0.3.7 2024-08-07T17:38:10.6117321Z ++ python -m site --user-base 2024-08-07T17:38:10.6341302Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T17:38:10.6342428Z + [[ linux-jammy-py3.8-gcc11 == *asan* ]] 2024-08-07T17:38:10.6342790Z + [[ linux-jammy-py3.8-gcc11 == *-debug* ]] 2024-08-07T17:38:10.6343152Z + [[ linux-jammy-py3.8-gcc11 != *-bazel-* ]] 2024-08-07T17:38:10.6343737Z + echo 'We are not in debug mode: linux-jammy-py3.8-gcc11. Expect the assertion to pass' 2024-08-07T17:38:10.6344619Z We are not in debug mode: linux-jammy-py3.8-gcc11. Expect the assertion to pass 2024-08-07T17:38:10.6345177Z + cd test 2024-08-07T17:38:10.6345526Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-07T17:38:12.1909485Z + [[ distributed == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-07T17:38:12.1910204Z + [[ distributed == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-07T17:38:12.1911159Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-07T17:38:12.1911700Z + [[ distributed == *dynamo_eager* ]] 2024-08-07T17:38:12.1912291Z + [[ distributed == *aot_eager* ]] 2024-08-07T17:38:12.1912879Z + [[ distributed == *aot_inductor* ]] 2024-08-07T17:38:12.1913550Z + [[ distributed == *inductor* ]] 2024-08-07T17:38:12.1914123Z + [[ distributed == *dynamic* ]] 2024-08-07T17:38:12.1914671Z + [[ distributed == *cpu* ]] 2024-08-07T17:38:12.1915279Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-07T17:38:12.1935586Z + [[ linux-jammy-py3.8-gcc11 == *libtorch* ]] 2024-08-07T17:38:12.1936016Z + [[ linux-jammy-py3.8-gcc11 == *-bazel-* ]] 2024-08-07T17:38:12.1938317Z + cd test 2024-08-07T17:38:12.1938891Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-07T17:38:13.3851463Z PyTorch built with: 2024-08-07T17:38:13.3851771Z - GCC 11.4 2024-08-07T17:38:13.3852016Z - C++ Version: 201703 2024-08-07T17:38:13.3852626Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-07T17:38:13.3853389Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-07T17:38:13.3853848Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-07T17:38:13.3854206Z - LAPACK is enabled (usually provided by MKL) 2024-08-07T17:38:13.3854542Z - NNPACK is enabled 2024-08-07T17:38:13.3854817Z - CPU capability usage: AVX512 2024-08-07T17:38:13.3860241Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-07T17:38:13.3865966Z 2024-08-07T17:38:13.5909228Z + cd test 2024-08-07T17:38:13.5909646Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-07T17:38:14.7772914Z ATen/Parallel: 2024-08-07T17:38:14.7773401Z at::get_num_threads() : 4 2024-08-07T17:38:14.7773731Z at::get_num_interop_threads() : 4 2024-08-07T17:38:14.7774076Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-07T17:38:14.7774395Z omp_get_max_threads() : 4 2024-08-07T17:38:14.7774983Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-07T17:38:14.7775666Z mkl_get_max_threads() : 4 2024-08-07T17:38:14.7776068Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-07T17:38:14.7776530Z std::thread::hardware_concurrency() : 8 2024-08-07T17:38:14.7776848Z Environment variables: 2024-08-07T17:38:14.7777121Z OMP_NUM_THREADS : [not set] 2024-08-07T17:38:14.7777408Z MKL_NUM_THREADS : [not set] 2024-08-07T17:38:14.7777689Z ATen parallel backend: OpenMP 2024-08-07T17:38:14.7777892Z 2024-08-07T17:38:14.9954723Z + [[ linux-jammy-py3.8-gcc11 == *aarch64* ]] 2024-08-07T17:38:14.9955135Z + [[ distributed == *backward* ]] 2024-08-07T17:38:14.9955451Z + [[ distributed == *xla* ]] 2024-08-07T17:38:14.9955768Z + [[ distributed == *executorch* ]] 2024-08-07T17:38:14.9956111Z + [[ distributed == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-07T17:38:14.9956486Z + [[ linux-jammy-py3.8-gcc11 == *libtorch* ]] 2024-08-07T17:38:14.9956844Z + [[ distributed == distributed ]] 2024-08-07T17:38:14.9957141Z + test_distributed 2024-08-07T17:38:14.9957422Z + echo 'Testing distributed python tests' 2024-08-07T17:38:14.9957763Z Testing distributed python tests 2024-08-07T17:38:14.9958198Z + python test/run_test.py --distributed-tests --shard 2 2 --verbose 2024-08-07T17:38:15.1122928Z test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-07T17:38:15.1123672Z import pkg_resources 2024-08-07T17:38:18.5809645Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-07T17:38:18.6502220Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-07T17:38:18.7004438Z Ignoring disabled issues: [''] 2024-08-07T17:38:18.7115458Z Found test times from artifacts 2024-08-07T17:38:18.7531215Z Found test times from artifacts 2024-08-07T17:38:18.7546512Z Running all tests 2024-08-07T17:38:18.7680510Z Running parallel tests on 3 processes 2024-08-07T17:38:18.7686850Z Name: tests to run (est. time: 62.4min) 2024-08-07T17:38:18.7687313Z Serial tests (110): 2024-08-07T17:38:18.7687652Z distributed/_tools/test_mod_tracker 1/1 2024-08-07T17:38:18.7688043Z distributed/_tensor/debug/test_op_coverage 1/1 2024-08-07T17:38:18.7688437Z distributed/_tensor/test_dtensor_ops 1/1 2024-08-07T17:38:18.7688791Z distributed/_tools/test_mem_tracker 1/1 2024-08-07T17:38:18.7689161Z distributed/_tensor/test_convolution_ops 1/1 2024-08-07T17:38:18.7689536Z distributed/_tensor/test_attention 1/1 2024-08-07T17:38:18.7689874Z distributed/_tensor/test_dtensor 1/1 2024-08-07T17:38:18.7690215Z distributed/_tensor/test_api 1/1 2024-08-07T17:38:18.7690773Z distributed/pipelining/test_pipe 1/1 2024-08-07T17:38:18.7691283Z distributed/test_c10d_functional_native 1/1 2024-08-07T17:38:18.7691653Z distributed/_tensor/test_matrix_ops 1/1 2024-08-07T17:38:18.7692008Z distributed/fsdp/test_fsdp_fx 1/1 2024-08-07T17:38:18.7692363Z distributed/_tensor/test_experimental_ops 1/1 2024-08-07T17:38:18.7692732Z distributed/test_c10d_pypg 1/1 2024-08-07T17:38:18.7693071Z distributed/_tensor/test_common_rules 1/1 2024-08-07T17:38:18.7693440Z distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 2024-08-07T17:38:18.7693835Z distributed/checkpoint/test_dedup_tensors 1/1 2024-08-07T17:38:18.7694217Z distributed/_tools/test_memory_tracker 1/1 2024-08-07T17:38:18.7694561Z distributed/_tensor/test_init 1/1 2024-08-07T17:38:18.7694898Z distributed/pipelining/test_stage 1/1 2024-08-07T17:38:18.7695260Z distributed/pipelining/test_unflatten 1/1 2024-08-07T17:38:18.7695681Z distributed/checkpoint/e2e/test_fsdp_ep 1/1 2024-08-07T17:38:18.7696069Z distributed/_tensor/test_xla_integration 1/1 2024-08-07T17:38:18.7696463Z distributed/checkpoint/test_save_load_api 1/1 2024-08-07T17:38:18.7696838Z distributed/pipelining/test_microbatch 1/1 2024-08-07T17:38:18.7697250Z distributed/_tensor/debug/test_comm_mode_features 1/1 2024-08-07T17:38:18.7697717Z distributed/_composable/fsdp/test_fully_shard_state 1/1 2024-08-07T17:38:18.7698130Z distributed/_composable/test_checkpoint 1/1 2024-08-07T17:38:18.7698518Z distributed/tensor/parallel/test_tp_style 1/1 2024-08-07T17:38:18.7698902Z distributed/checkpoint/test_traverse 1/1 2024-08-07T17:38:18.7699313Z distributed/_composable/fsdp/test_fully_shard_logging 1/1 2024-08-07T17:38:18.7699752Z distributed/test_compute_comm_reordering 1/1 2024-08-07T17:38:18.7700174Z distributed/tensor/parallel/test_tp_random_state 1/1 2024-08-07T17:38:18.7700578Z distributed/_tools/test_fsdp2_mem_tracker 1/1 2024-08-07T17:38:18.7700960Z distributed/test_inductor_collectives 1/1 2024-08-07T17:38:18.7701377Z distributed/checkpoint/e2e/test_e2e_save_and_load 1/1 2024-08-07T17:38:18.7701761Z distributed/launcher/test_run 1/1 2024-08-07T17:38:18.7702162Z distributed/_composable/fsdp/test_fully_shard_compile 1/1 2024-08-07T17:38:18.7702644Z distributed/_composable/fsdp/test_fully_shard_overlap 1/1 2024-08-07T17:38:18.7703046Z distributed/_tensor/test_view_ops 1/1 2024-08-07T17:38:18.7703419Z distributed/checkpoint/e2e/test_fine_tuning 1/1 2024-08-07T17:38:18.7703810Z distributed/elastic/test_control_plane 1/1 2024-08-07T17:38:18.7704167Z distributed/checkpoint/test_planner 1/1 2024-08-07T17:38:18.7704562Z distributed/checkpoint/test_dtensor_checkpoint 1/1 2024-08-07T17:38:18.7704977Z distributed/fsdp/test_fsdp_tp_integration 1/1 2024-08-07T17:38:18.7705402Z distributed/_composable/fsdp/test_fully_shard_memory 1/1 2024-08-07T17:38:18.7705835Z distributed/checkpoint/e2e/test_pipeline 1/1 2024-08-07T17:38:18.7706214Z distributed/_tensor/test_op_strategy 1/1 2024-08-07T17:38:18.7706593Z distributed/checkpoint/test_compatibility 1/1 2024-08-07T17:38:18.7706983Z distributed/_tensor/test_embedding_ops 1/1 2024-08-07T17:38:18.7707375Z distributed/fsdp/test_fsdp_ignored_modules 1/1 2024-08-07T17:38:18.7707774Z distributed/checkpoint/test_hsdp_checkpoint 1/1 2024-08-07T17:38:18.7708165Z distributed/pipelining/test_schedule 1/1 2024-08-07T17:38:18.7708520Z distributed/test_functional_api 1/1 2024-08-07T17:38:18.7708948Z distributed/_composable/fully_shard/test_fully_shard_compile 1/1 2024-08-07T17:38:18.7709431Z distributed/checkpoint/test_state_dict_utils 1/1 2024-08-07T17:38:18.7709827Z distributed/_composable/test_replicate 1/1 2024-08-07T17:38:18.7710240Z distributed/optim/test_apply_optimizer_in_backward 1/1 2024-08-07T17:38:18.7710709Z distributed/_composable/fsdp/test_fully_shard_frozen 1/1 2024-08-07T17:38:18.7711128Z distributed/_tensor/test_optimizers 1/1 2024-08-07T17:38:18.7711478Z distributed/test_distributed_spawn 2/3 2024-08-07T17:38:18.7712125Z distributed/_composable/fully_shard/test_fully_shard_mixed_precision 1/1 2024-08-07T17:38:18.7712696Z distributed/_composable/fully_shard/test_fully_shard_util 1/1 2024-08-07T17:38:18.7713235Z distributed/fsdp/test_fsdp_hybrid_shard 1/1 2024-08-07T17:38:18.7713643Z distributed/fsdp/test_fsdp_dtensor_state_dict 1/1 2024-08-07T17:38:18.7714042Z distributed/_composable/test_compose 1/1 2024-08-07T17:38:18.7714427Z distributed/fsdp/test_distributed_checkpoint 1/1 2024-08-07T17:38:18.7714839Z distributed/fsdp/test_fsdp_clip_grad_norm 1/1 2024-08-07T17:38:18.7715232Z distributed/checkpoint/test_checkpoint 1/1 2024-08-07T17:38:18.7715627Z distributed/fsdp/test_fsdp_sharded_grad_scaler 1/1 2024-08-07T17:38:18.7716092Z distributed/_composable/fsdp/test_fully_shard_state_dict 1/1 2024-08-07T17:38:18.7716521Z distributed/_tensor/test_utils 1/1 2024-08-07T17:38:18.7716897Z distributed/fsdp/test_fsdp_pure_fp16 1/1 2024-08-07T17:38:18.7717367Z distributed/_composable/test_composability/test_2d_composability 1/1 2024-08-07T17:38:18.7717845Z distributed/fsdp/test_fsdp_overlap 1/1 2024-08-07T17:38:18.7718224Z distributed/fsdp/test_fsdp_unshard_params 1/1 2024-08-07T17:38:18.7718687Z distributed/_composable/fsdp/test_fully_shard_mixed_precision 1/1 2024-08-07T17:38:18.7719200Z distributed/checkpoint/test_file_system_checkpoint 1/1 2024-08-07T17:38:18.7719659Z distributed/fsdp/test_fsdp_mixed_precision 1/1 2024-08-07T17:38:18.7720036Z distributed/nn/jit/test_instantiator 1/1 2024-08-07T17:38:18.7720407Z distributed/fsdp/test_fsdp_state_dict 1/1 2024-08-07T17:38:18.7720835Z distributed/_composable/test_replicate_with_compiler 1/1 2024-08-07T17:38:18.7721375Z distributed/_composable/fully_shard/test_fully_shard_model_checkpoint 1/1 2024-08-07T17:38:18.7721893Z distributed/fsdp/test_fsdp_use_orig_params 1/1 2024-08-07T17:38:18.7722298Z distributed/fsdp/test_fsdp_flatten_params 1/1 2024-08-07T17:38:18.7722751Z distributed/_composable/fully_shard/test_fully_shard_init 1/1 2024-08-07T17:38:18.7723185Z distributed/test_pg_wrapper 1/1 2024-08-07T17:38:18.7723530Z distributed/test_multi_threaded_pg 1/1 2024-08-07T17:38:18.7723870Z distributed/fsdp/test_fsdp_memory 1/1 2024-08-07T17:38:18.7724224Z distributed/fsdp/test_fsdp_grad_acc 1/1 2024-08-07T17:38:18.7724607Z distributed/test_c10d_object_collectives 1/1 2024-08-07T17:38:18.7725050Z distributed/_composable/fsdp/test_fully_shard_extensions 1/1 2024-08-07T17:38:18.7725521Z distributed/fsdp/test_hsdp_dtensor_state_dict 1/1 2024-08-07T17:38:18.7726012Z distributed/_composable/fully_shard/test_fully_shard_runtime 1/1 2024-08-07T17:38:18.7726519Z distributed/algorithms/ddp_comm_hooks/test_ddp_hooks 1/1 2024-08-07T17:38:18.7727018Z distributed/checkpoint/test_file_system_checkpoint_cpu 1/1 2024-08-07T17:38:18.7727445Z distributed/fsdp/test_fsdp_apply 1/1 2024-08-07T17:38:18.7727801Z distributed/fsdp/test_fsdp_comm_hooks 1/1 2024-08-07T17:38:18.7728146Z distributed/fsdp/test_wrap 1/1 2024-08-07T17:38:18.7728465Z distributed/test_c10d_ucc 1/1 2024-08-07T17:38:18.7728782Z distributed/fsdp/test_fsdp_traversal 1/1 2024-08-07T17:38:18.7729159Z distributed/rpc/cuda/test_tensorpipe_agent 1/1 2024-08-07T17:38:18.7729613Z distributed/_shard/sharded_tensor/ops/test_binary_cmp 1/1 2024-08-07T17:38:18.7730076Z distributed/_shard/sharding_spec/test_sharding_spec 1/1 2024-08-07T17:38:18.7730616Z distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint 1/1 2024-08-07T17:38:18.7731113Z distributed/rpc/test_tensorpipe_agent 1/4 2024-08-07T17:38:18.7731482Z distributed/rpc/test_tensorpipe_agent 3/4 2024-08-07T17:38:18.7731857Z distributed/fsdp/test_fsdp_multiple_forward 1/1 2024-08-07T17:38:18.7732306Z distributed/_shard/sharded_tensor/ops/test_tensor_ops 1/1 2024-08-07T17:38:18.7732719Z distributed/test_c10d_gloo 1/1 2024-08-07T17:38:18.7733013Z Parallel tests (0): 2024-08-07T17:38:18.7733475Z Name: excluded (est. time: 0.0min) 2024-08-07T17:38:18.7733764Z Serial tests (0): 2024-08-07T17:38:18.7734015Z Parallel tests (0): 2024-08-07T17:38:18.7734406Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-07T17:38:18.7923481Z Running distributed/_tools/test_mod_tracker 1/1 ... [2024-08-07 17:38:18.791971] 2024-08-07T17:38:18.7929713Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tools/test_mod_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:38:18.792567] 2024-08-07T17:38:22.4705358Z 2024-08-07T17:38:22.4706712Z distributed/_tools/test_mod_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_mod_tracker_1.1_03e39533a79d5d40_.log 2024-08-07T17:38:22.4709340Z Running 4 items in this shard: test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_ac, test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_bw_detection, test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_module_hierarchy, test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_user_hooks 2024-08-07T17:38:22.4710818Z 2024-08-07T17:38:22.4714960Z Running distributed/_tensor/debug/test_op_coverage 1/1 ... [2024-08-07 17:38:22.471296] 2024-08-07T17:38:22.4719854Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/debug/test_op_coverage.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:38:22.471742] 2024-08-07T17:38:25.3959855Z 2024-08-07T17:38:25.3961243Z distributed/_tensor/debug/test_op_coverage 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.debug.test_op_coverage_1.1_10235cd5176eed36_.log 2024-08-07T17:38:25.3962666Z Running 1 items in this shard: test/distributed/_tensor/debug/test_op_coverage.py::TestOpCoverage::test_trace_with_inductor_decomp 2024-08-07T17:38:25.3963385Z 2024-08-07T17:38:25.3966532Z Running distributed/_tensor/test_dtensor_ops 1/1 ... [2024-08-07 17:38:25.396454] 2024-08-07T17:38:25.3972007Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_dtensor_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:38:25.396874] 2024-08-07T17:39:42.1674494Z 2024-08-07T17:39:42.1675830Z distributed/_tensor/test_dtensor_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_dtensor_ops_1.1_437f35955c6a12d3_.log 2024-08-07T17:39:42.2440579Z Running 666 items in this shard: test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_H_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_T_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___getitem___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___radd___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rdiv___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rmatmul___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rmod___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rmul___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rpow___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rsub___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__batch_norm_with_update_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__chunk_cat_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__native_batch_norm_legit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__segment_reduce_lengths_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__segment_reduce_offsets_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__softmax_backward_data_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__unsafe_masked_index_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__unsafe_masked_index_put_accumulate_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__upsample_bilinear2d_aa_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_abs_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_acos_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_acosh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_add_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addbmm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addcdiv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addcmul_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addmm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addmm_decomposed_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addmv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_alias_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_all_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_allclose_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_amax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_amin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_aminmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_angle_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_any_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_arange_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argmin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argsort_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argwhere_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_partial_views_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_asin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_asinh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atan2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atan_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atanh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atleast_1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atleast_2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atleast_3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_baddbmm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bernoulli_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bfloat16_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_block_diag_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bmm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bool_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_broadcast_shapes_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_broadcast_tensors_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_broadcast_to_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bucketize_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_byte_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cartesian_prod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cat_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cauchy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cdist_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cdouble_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ceil_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cfloat_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_chalf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_char_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cholesky_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cholesky_inverse_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cholesky_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_chunk_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clamp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clamp_max_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clamp_min_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clone_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_column_stack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_combinations_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_complex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_conj_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_conj_physical_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_constant_pad_nd_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_contiguous_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_copysign_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_corrcoef_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cos_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cosh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_count_nonzero_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cov_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cross_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cummax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cummin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cumprod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cumsum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cumulative_trapezoid_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_deg2rad_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diag_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diag_embed_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagflat_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagonal_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagonal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagonal_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diff_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_digamma_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dist_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_div_floor_rounding_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_div_no_rounding_mode_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_div_trunc_rounding_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_double_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dsplit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dstack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_einsum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_permuted_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_strided_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_eq_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_equal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_erf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_erfc_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_erfinv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_exp2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_exp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_expand_as_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_expand_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_expand_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_expm1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_exponential_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_eye_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fftshift_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_hfft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_hfft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_hfftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifftshift_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ihfft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ihfft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ihfftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_irfft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_irfft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_irfftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_rfft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_rfft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_rfftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fill_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_flatten_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_flip_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fliplr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_flipud_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_float_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_float_power_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_floor_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_floor_divide_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fmin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fmod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_frac_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_frexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_full_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_full_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_gather_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ge_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_geometric_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_geqrf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_gradient_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_grid_sampler_2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_gt_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_half_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_heaviside_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_histc_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_histogram_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_histogramdd_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_hsplit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_hstack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_hypot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_i0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_igamma_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_igammac_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_add_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_fill_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_put_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_amax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_amin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_prod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_select_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_inner_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_int_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isclose_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isfinite_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isinf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isnan_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isneginf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isposinf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isreal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_item_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_2inputs_2outputs_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_4inputs_with_extra_args_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_binary_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_binary_return_by_ref_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_unary_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_kron_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_kthvalue_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ldexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_le_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lerp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lgamma_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cholesky_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cholesky_ex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cond_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cross_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_det_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_det_singular_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_diagonal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eig_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eigh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eigvals_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eigvalsh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_householder_product_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_inv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_inv_ex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_ldl_factor_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_ldl_factor_ex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_ldl_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lstsq_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lstsq_grad_oriented_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_factor_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_factor_ex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_power_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_rank_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_rank_hermitian_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_multi_dot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_norm_subgradients_at_zero_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_pinv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_pinv_hermitian_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_pinv_singular_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_qr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_slogdet_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_solve_ex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_solve_triangular_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_svd_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_svdvals_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_tensorinv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_tensorsolve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_vander_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_vecdot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_vector_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linspace_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linspace_tensor_overload_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log10_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log1p_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_normal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_softmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_softmax_with_dtype_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logaddexp2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logaddexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logcumsumexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logdet_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_and_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_not_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_or_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_xor_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logspace_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logspace_tensor_overload_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logsumexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_long_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lt_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lu_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lu_unpack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mH_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mT_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_amax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_amin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_argmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_argmin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_cumprod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_cumsum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_fill_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_log_softmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_logaddexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_logsumexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_median_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_normalize_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_prod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_select_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_softmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_softmin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_std_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_sum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_var_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_matmul_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_matrix_exp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_binary_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_pool2d_with_indices_backward_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_reduction_no_dim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_reduction_with_dim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_maximum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_median_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_meshgrid_list_of_tensors_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_meshgrid_variadic_tensors_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_min_binary_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_min_reduction_no_dim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_min_reduction_with_dim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_minimum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mode_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_movedim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_msort_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mul_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_multinomial_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mvlgamma_mvlgamma_p_1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mvlgamma_mvlgamma_p_3_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mvlgamma_mvlgamma_p_5_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nan_to_num_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nanmean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nanmedian_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nanquantile_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nansum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_narrow_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_narrow_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_native_batch_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_native_dropout_backward_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_native_layer_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ne_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_neg_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_empty_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_empty_strided_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_full_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_ones_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_zeros_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nextafter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_avg_pool1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_avg_pool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_avg_pool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_max_pool1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_max_pool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_max_pool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_alpha_dropout_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_avg_pool1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_avg_pool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_avg_pool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_batch_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_bilinear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_binary_cross_entropy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_binary_cross_entropy_with_logits_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_celu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_channel_shuffle_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv_transpose1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv_transpose2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv_transpose3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_cosine_embedding_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_cosine_similarity_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_cross_entropy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_ctc_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_dropout2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_dropout3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_dropout_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_elu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_embedding_bag_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_embedding_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_feature_alpha_dropout_with_train_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_feature_alpha_dropout_without_train_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_fractional_max_pool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_fractional_max_pool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_gaussian_nll_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_gelu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_glu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_grid_sample_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_group_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardshrink_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardsigmoid_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardswish_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardtanh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hinge_embedding_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_huber_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_instance_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_area_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_bicubic_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_bilinear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_linear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_nearest-exact_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_nearest_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_trilinear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_kl_div_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_l1_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_layer_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_leaky_relu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_linear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_local_response_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_logsigmoid_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_margin_ranking_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_pool1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_pool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_pool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool1d_grad_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool2d_grad_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool3d_grad_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_mish_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_mse_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multi_head_attention_forward_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multi_margin_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multilabel_margin_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multilabel_soft_margin_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_nll_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_normalize_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_circular_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_constant_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_reflect_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_replicate_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_replicate_negative_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pairwise_distance_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pdist_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pixel_shuffle_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pixel_unshuffle_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_poisson_nll_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_prelu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_relu6_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_relu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_rms_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_rrelu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_scaled_dot_product_attention_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_selu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_silu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_smooth_l1_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_soft_margin_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softmin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softmin_with_dtype_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softplus_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softshrink_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softsign_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_tanhshrink_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_threshold_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_triplet_margin_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_triplet_margin_with_distance_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_unfold_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_upsample_bilinear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_upsample_nearest_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nonzero_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nonzero_static_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_fro_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_inf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_nuc_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_normal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_normal_in_place_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_normal_number_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ones_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ones_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ormqr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_outer_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_pca_lowrank_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_permute_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_pinverse_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polar_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_3_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_4_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_positive_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_pow_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_prod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_put_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_qr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_quantile_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rad2deg_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rand_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randint_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randint_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randn_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ravel_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_real_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_reciprocal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_remainder_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_renorm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_repeat_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_repeat_interleave_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_reshape_as_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_reshape_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resize__cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resize_as__cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resolve_conj_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resolve_neg_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_roll_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rot90_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_decimals_0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_decimals_3_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_decimals_neg_3_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rsqrt_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rsub_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scalar_tensor_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_add_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_amax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_amin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_prod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_sum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_searchsorted_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_select_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_select_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sgn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_short_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sigmoid_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sign_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_bartlett_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_blackman_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_cosine_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_exponential_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_gaussian_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_general_cosine_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_general_hamming_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_hamming_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_hann_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_kaiser_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_nuttall_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signbit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sinc_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sinh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_slice_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_slice_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_softmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_softmax_with_dtype_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sort_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sparse_mm_reduce_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sparse_sampled_addmm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_airy_ai_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_j0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_j1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_y0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_y1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_t_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_u_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_v_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_w_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_entr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_erfcx_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_hermite_polynomial_h_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_hermite_polynomial_he_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_i0e_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_i1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_i1e_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_laguerre_polynomial_l_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_legendre_polynomial_p_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_log_ndtr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_i0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_i1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_k0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_k1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_ndtr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_ndtri_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_polygamma_special_polygamma_n_0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_scaled_modified_bessel_k0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_scaled_modified_bessel_k1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_t_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_u_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_v_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_w_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_spherical_bessel_j0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_xlog1py_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_zeta_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_list_args_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_with_sizes_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_with_sizes_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sqrt_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_square_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_squeeze_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_squeeze_multiple_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_stack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_mean_unbiased_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_unbiased_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_stft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sub_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sum_to_size_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_svd_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_svd_lowrank_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_t_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_t_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_take_along_dim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_take_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tan_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tanh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tensor_split_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tensordot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tile_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_to_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_to_sparse_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_topk_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trace_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_transpose_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trapezoid_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trapz_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_triangular_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tril_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_triu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_true_divide_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trunc_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unbind_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unflatten_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unfold_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unfold_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_uniform_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unique_consecutive_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unique_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unsafe_chunk_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unsafe_split_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unsqueeze_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unsqueeze_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_mean_unbiased_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_unbiased_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_vdot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_as_complex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_as_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_vsplit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_vstack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_where_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_xlogy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_zero__cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_zeros_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_zeros_like_cpu_float32 2024-08-07T17:39:42.2744044Z 2024-08-07T17:39:42.2760903Z Running distributed/_tools/test_mem_tracker 1/1 ... [2024-08-07 17:39:42.275848] 2024-08-07T17:39:42.2765952Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tools/test_mem_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:39:42.276322] 2024-08-07T17:39:45.0506627Z 2024-08-07T17:39:45.0507946Z distributed/_tools/test_mem_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_mem_tracker_1.1_06f3da25b294a9b1_.log 2024-08-07T17:39:45.0510007Z Running 3 items in this shard: test/distributed/_tools/test_mem_tracker.py::TestMemTracker::test_cuda_tracker_equivalence, test/distributed/_tools/test_mem_tracker.py::TestMemTracker::test_tracker_attribution, test/distributed/_tools/test_mem_tracker.py::TestMemTracker::test_tracker_with_activation_checkpointing 2024-08-07T17:39:45.0511407Z 2024-08-07T17:39:45.0522715Z Running distributed/_tensor/test_convolution_ops 1/1 ... [2024-08-07 17:39:45.051088] 2024-08-07T17:39:45.0524599Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_convolution_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:39:45.051499] 2024-08-07T17:40:08.6209942Z 2024-08-07T17:40:08.6212707Z distributed/_tensor/test_convolution_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_convolution_ops_1.1_88ded636f87325d0_.log 2024-08-07T17:40:08.6214558Z Running 2 items in this shard: test/distributed/_tensor/test_convolution_ops.py::DistConvolutionOpsTest::test_depthwise_convolution, test/distributed/_tensor/test_convolution_ops.py::DistConvolutionOpsTest::test_downsampling_convolution 2024-08-07T17:40:08.6215657Z 2024-08-07T17:40:08.6222007Z Running distributed/_tensor/test_attention 1/1 ... [2024-08-07 17:40:08.621481] 2024-08-07T17:40:08.6227842Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_attention.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:40:08.622530] 2024-08-07T17:40:14.4011457Z 2024-08-07T17:40:14.4013105Z distributed/_tensor/test_attention 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_attention_1.1_7376c0f65c051eda_.log 2024-08-07T17:40:14.4017964Z Running 8 items in this shard: test/distributed/_tensor/test_attention.py::RingAttentionTest::test_is_causal_behavior, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_compile_attention_fn0, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_compile_attention_fn1, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_custom_transformer, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_native_transformer_is_causal_False, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_native_transformer_is_causal_True, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_sdpa_is_causal_False, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_sdpa_is_causal_True 2024-08-07T17:40:14.4021938Z 2024-08-07T17:40:14.4022197Z Running distributed/_tensor/test_dtensor 1/1 ... [2024-08-07 17:40:14.401570] 2024-08-07T17:40:14.4023475Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_dtensor.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:40:14.401995] 2024-08-07T17:42:00.5589562Z 2024-08-07T17:42:00.5590723Z distributed/_tensor/test_dtensor 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_dtensor_1.1_49622edfae5ad27e_.log 2024-08-07T17:42:00.5603785Z Running 32 items in this shard: test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_async_output, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_constructor, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_new_empty_strided, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_properties, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_save_load, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_spec_hash, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_spec_read_only_after_set, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_stride, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_from_local, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_from_local_negative_dim, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_from_local_then_to_local, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_from_local_uneven_sharding, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_from_local_uneven_sharding_raise_error, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_full_tensor_grad_hint, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_full_tensor_sync, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_meta_dtensor, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_modules_w_meta_dtensor, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_to_local, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_to_local_grad_hint, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_auto_implicit_replication, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_default_value_sub_mesh, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_device_mesh_nd, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_dtensor_2d_mesh, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_dtensor_api_device_mesh_context_manager, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_dtensor_device_mesh_device_conversion, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_dtensor_spec_local_shard_offset, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_from_local_sub_mesh, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_implicit_replication, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_metadata_consistency_check, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_redistribute_sub_mesh, test/distributed/_tensor/test_dtensor.py::TestDTensorPlacementTypes::test_split_tensor_1D, test/distributed/_tensor/test_dtensor.py::DTensorLogTest::test_dtensor_log 2024-08-07T17:42:00.5615349Z 2024-08-07T17:42:00.5615606Z Running distributed/_tensor/test_api 1/1 ... [2024-08-07 17:42:00.559463] 2024-08-07T17:42:00.5616830Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_api.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:42:00.559890] 2024-08-07T17:42:27.2366898Z 2024-08-07T17:42:27.2368530Z distributed/_tensor/test_api 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_api_1.1_2cf5e21a5ddca3c0_.log 2024-08-07T17:42:27.2372649Z Running 8 items in this shard: test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_module, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_module_casting, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_module_input_fn_output_fn, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_module_input_fn_output_fn_warning, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_module_meta, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_tensor, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_tensor_errors, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_tensor_uneven_sharding 2024-08-07T17:42:27.2376445Z 2024-08-07T17:42:27.2377226Z Running distributed/pipelining/test_pipe 1/1 ... [2024-08-07 17:42:27.237390] 2024-08-07T17:42:27.2381585Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_pipe.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:42:27.237870] 2024-08-07T17:42:31.3166118Z 2024-08-07T17:42:31.3167629Z distributed/pipelining/test_pipe 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_pipe_1.1_87f59d15033e9578_.log 2024-08-07T17:42:31.3169631Z Running 3 items in this shard: test/distributed/pipelining/test_pipe.py::PipeTests::test_model_split_ModelClass0, test/distributed/pipelining/test_pipe.py::PipeTests::test_model_split_ModelClass1, test/distributed/pipelining/test_pipe.py::PipeTests::test_model_split_ModelClass2 2024-08-07T17:42:31.3170888Z 2024-08-07T17:42:31.3172501Z Running distributed/test_c10d_functional_native 1/1 ... [2024-08-07 17:42:31.317061] 2024-08-07T17:42:31.3177426Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_functional_native.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:42:31.317505] 2024-08-07T17:42:34.2935029Z 2024-08-07T17:42:34.2936527Z distributed/test_c10d_functional_native 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_functional_native_1.1_13fb73fe420082e7_.log 2024-08-07T17:42:34.2948285Z Running 25 items in this shard: test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_gather_into_tensor_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_gather_into_tensor_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_coalesced_, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_single_, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_to_all_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_broadcast, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_inductor_dtypeview_memory_leak, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_py_work, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_reduce_scatter_tensor_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_threading, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_unwaited, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_gather_into_tensor_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_gather_into_tensor_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_reduce_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_reduce_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_to_all_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_broadcast, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_inplace_op_on_view, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reduce_scatter_tensor_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reuse_buffer_after_inplace_collective, test/distributed/test_c10d_functional_native.py::CompileTest::test_ranks_and_tag 2024-08-07T17:42:34.2958356Z 2024-08-07T17:42:34.2959048Z Running distributed/_tensor/test_matrix_ops 1/1 ... [2024-08-07 17:42:34.293978] 2024-08-07T17:42:34.2960332Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_matrix_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:42:34.294448] 2024-08-07T17:43:04.3288863Z 2024-08-07T17:43:04.3290292Z distributed/_tensor/test_matrix_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_matrix_ops_1.1_e5cb3ad308fc5bdb_.log 2024-08-07T17:43:04.3294631Z Running 9 items in this shard: test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_addmm, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_addmm_auto_redistribute, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_addmm_empty_operand, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_baddbmm, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_bmm, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_mm, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_scaled_dot_product_attention, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_t, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_t_partial 2024-08-07T17:43:04.3298147Z 2024-08-07T17:43:04.3298442Z Running distributed/fsdp/test_fsdp_fx 1/1 ... [2024-08-07 17:43:04.329486] 2024-08-07T17:43:04.3301773Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_fx.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:43:04.329905] 2024-08-07T17:43:07.3549312Z 2024-08-07T17:43:07.3550345Z distributed/fsdp/test_fsdp_fx 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_fx_1.1_178c6c32c80c85d3_.log 2024-08-07T17:43:07.3551585Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_fx.py::TestSymbolicTracing::test_symbolic_tracing_outputs 2024-08-07T17:43:07.3552126Z 2024-08-07T17:43:07.3555165Z Running distributed/_tensor/test_experimental_ops 1/1 ... [2024-08-07 17:43:07.355345] 2024-08-07T17:43:07.3560079Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_experimental_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:43:07.355790] 2024-08-07T17:43:18.9456506Z 2024-08-07T17:43:18.9457945Z distributed/_tensor/test_experimental_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_experimental_ops_1.1_d348b7969cc9ca08_.log 2024-08-07T17:43:18.9460467Z Running 3 items in this shard: test/distributed/_tensor/test_experimental_ops.py::DistOtherOpsTest::test_bernoulli, test/distributed/_tensor/test_experimental_ops.py::DistOtherOpsTest::test_nll, test/distributed/_tensor/test_experimental_ops.py::DistOtherOpsTest::test_slice 2024-08-07T17:43:18.9462034Z 2024-08-07T17:43:18.9462421Z Running distributed/test_c10d_pypg 1/1 ... [2024-08-07 17:43:18.945966] 2024-08-07T17:43:18.9466810Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_pypg.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:43:18.946418] 2024-08-07T17:45:15.0468948Z 2024-08-07T17:45:15.0472133Z distributed/test_c10d_pypg 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_pypg_1.1_903f24460f6ee61b_.log 2024-08-07T17:45:15.0492658Z Running 42 items in this shard: test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_dataclass_output, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_dataclass_output_unused_param, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_dynamic_module, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_dynamic_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_once_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_once_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_static_graph_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_static_graph_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_unused_params_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_unused_params_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_weight_sharing_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_weight_sharing_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_invoke_work_object, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_with_pypg, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_with_pypg_with_grad_views, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_invalid_powerSGD_state, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_sync_batch_norm_empty_input, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_sync_batch_norm_only_empty_input, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_dataclass_output, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_dataclass_output_unused_param, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_dynamic_module, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_dynamic_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_once_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_once_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_static_graph_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_static_graph_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_unused_params_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_unused_params_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_weight_sharing_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_weight_sharing_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_invoke_work_object, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_with_pypg, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_with_pypg_with_grad_views, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_invalid_powerSGD_state, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_sync_batch_norm_empty_input, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_sync_batch_norm_only_empty_input 2024-08-07T17:45:15.0511683Z 2024-08-07T17:45:15.0511958Z Running distributed/_tensor/test_common_rules 1/1 ... [2024-08-07 17:45:15.047672] 2024-08-07T17:45:15.0513325Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_common_rules.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:45:15.048151] 2024-08-07T17:45:47.7374422Z 2024-08-07T17:45:47.7375532Z distributed/_tensor/test_common_rules 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_common_rules_1.1_f956e4fb7cad15e5_.log 2024-08-07T17:45:47.7380765Z Running 10 items in this shard: test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_basic_propagation, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_errors, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_linearity, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_merge_sharding, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_multi_sharding_on_mesh_dim, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_pointwise_propagation, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_enforce_sharding_multi_sharding_on_mesh_dim, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_multi_sharding_on_mesh_dim, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_rules_broadcasting, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_rules_suggestion 2024-08-07T17:45:47.7385200Z 2024-08-07T17:45:47.7385525Z Running distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 ... [2024-08-07 17:45:47.737826] 2024-08-07T17:45:47.7386906Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/fsdp/test_fsdp_dsd.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:45:47.738226] 2024-08-07T17:45:50.8632490Z 2024-08-07T17:45:50.8633693Z distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.fsdp.test_fsdp_dsd_1.1_8bb5d7fad196763f_.log 2024-08-07T17:45:50.8636484Z Running 3 items in this shard: test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_1d_fsdp_cpu_offload_full_model_state_dict, test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_1d_fsdp_get_model_state_dict, test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_save_with_fsdp1_and_load_with_fsdp2 2024-08-07T17:45:50.8638817Z 2024-08-07T17:45:50.8639130Z Running distributed/checkpoint/test_dedup_tensors 1/1 ... [2024-08-07 17:45:50.863669] 2024-08-07T17:45:50.8642948Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_dedup_tensors.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:45:50.864071] 2024-08-07T17:45:53.9403184Z 2024-08-07T17:45:53.9404537Z distributed/checkpoint/test_dedup_tensors 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_dedup_tensors_1.1_32869f4aa790eba3_.log 2024-08-07T17:45:53.9405885Z Running 1 items in this shard: test/distributed/checkpoint/test_dedup_tensors.py::TestDedupTensor::test_dedup_shards 2024-08-07T17:45:53.9406440Z 2024-08-07T17:45:53.9410210Z Running distributed/_tools/test_memory_tracker 1/1 ... [2024-08-07 17:45:53.940738] 2024-08-07T17:45:53.9413936Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tools/test_memory_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:45:53.941158] 2024-08-07T17:45:56.8669648Z 2024-08-07T17:45:56.8671433Z distributed/_tools/test_memory_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_memory_tracker_1.1_607828d618a64a37_.log 2024-08-07T17:45:56.8673882Z Running 1 items in this shard: test/distributed/_tools/test_memory_tracker.py::TestMemoryTracker::test_local_model 2024-08-07T17:45:56.8674860Z 2024-08-07T17:45:56.8678859Z Running distributed/_tensor/test_init 1/1 ... [2024-08-07 17:45:56.867668] 2024-08-07T17:45:56.8684901Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_init.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:45:56.868151] 2024-08-07T17:46:20.6387457Z 2024-08-07T17:46:20.6390551Z distributed/_tensor/test_init 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_init_1.1_22e3ca6ed7fcb653_.log 2024-08-07T17:46:20.6394628Z Running 7 items in this shard: test/distributed/_tensor/test_init.py::DTensorInitOpsTest::test_init_ops, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_empty, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_full, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_ones, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_zeros, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_zeros_full_mesh, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_zeros_submesh 2024-08-07T17:46:20.6397647Z 2024-08-07T17:46:20.6397959Z Running distributed/pipelining/test_stage 1/1 ... [2024-08-07 17:46:20.639161] 2024-08-07T17:46:20.6399278Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_stage.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:46:20.639561] 2024-08-07T17:46:23.6281822Z 2024-08-07T17:46:23.6283040Z distributed/pipelining/test_stage 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_stage_1.1_04f11a14b14e747b_.log 2024-08-07T17:46:23.6283853Z 2024-08-07T17:46:23.6287951Z Running distributed/pipelining/test_unflatten 1/1 ... [2024-08-07 17:46:23.628613] 2024-08-07T17:46:23.6292837Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_unflatten.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:46:23.629056] 2024-08-07T17:46:27.1549509Z 2024-08-07T17:46:27.1550874Z distributed/pipelining/test_unflatten 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_unflatten_1.1_e18830072ddc1fb0_.log 2024-08-07T17:46:27.1552184Z Running 1 items in this shard: test/distributed/pipelining/test_unflatten.py::UnflattenTests::test_unflatten 2024-08-07T17:46:27.1552766Z 2024-08-07T17:46:27.1556575Z Running distributed/checkpoint/e2e/test_fsdp_ep 1/1 ... [2024-08-07 17:46:27.155444] 2024-08-07T17:46:27.1561487Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/e2e/test_fsdp_ep.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:46:27.155910] 2024-08-07T17:46:30.1810797Z 2024-08-07T17:46:30.1812563Z distributed/checkpoint/e2e/test_fsdp_ep 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.e2e.test_fsdp_ep_1.1_c43f1ba888c0a73d_.log 2024-08-07T17:46:30.1814943Z Running 1 items in this shard: test/distributed/checkpoint/e2e/test_fsdp_ep.py::TestFSDPWithEP::test_e2e 2024-08-07T17:46:30.1815859Z 2024-08-07T17:46:30.1819674Z Running distributed/_tensor/test_xla_integration 1/1 ... [2024-08-07 17:46:30.181758] 2024-08-07T17:46:30.1825632Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_xla_integration.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:46:30.182243] 2024-08-07T17:46:33.3081185Z 2024-08-07T17:46:33.3082376Z distributed/_tensor/test_xla_integration 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_xla_integration_1.1_8c57478fd0ba1683_.log 2024-08-07T17:46:33.3084790Z Running 3 items in this shard: test/distributed/_tensor/test_xla_integration.py::DTensorXLAIntegrationTest::test_xla_distribute_tensor_1d_replicate, test/distributed/_tensor/test_xla_integration.py::DTensorXLAIntegrationTest::test_xla_distribute_tensor_1d_shard, test/distributed/_tensor/test_xla_integration.py::DTensorXLAIntegrationTest::test_xla_distribute_tensor_2d 2024-08-07T17:46:33.3086681Z 2024-08-07T17:46:33.3087427Z Running distributed/checkpoint/test_save_load_api 1/1 ... [2024-08-07 17:46:33.308567] 2024-08-07T17:46:33.3092295Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_save_load_api.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:46:33.309000] 2024-08-07T17:46:39.1390981Z 2024-08-07T17:46:39.1392184Z distributed/checkpoint/test_save_load_api 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_save_load_api_1.1_b987cf693df673de_.log 2024-08-07T17:46:39.1393600Z Running 1 items in this shard: test/distributed/checkpoint/test_save_load_api.py::TestSaveAndLoadAPI::test_auto_detect 2024-08-07T17:46:39.1394139Z 2024-08-07T17:46:39.1397422Z Running distributed/pipelining/test_microbatch 1/1 ... [2024-08-07 17:46:39.139539] 2024-08-07T17:46:39.1401975Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_microbatch.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:46:39.139966] 2024-08-07T17:46:42.5177129Z 2024-08-07T17:46:42.5178283Z distributed/pipelining/test_microbatch 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_microbatch_1.1_4c3c8fd929e4ea78_.log 2024-08-07T17:46:42.5180031Z Running 2 items in this shard: test/distributed/pipelining/test_microbatch.py::MicrobatchTests::test_chunk_spec, test/distributed/pipelining/test_microbatch.py::MicrobatchTests::test_split_and_merge 2024-08-07T17:46:42.5180924Z 2024-08-07T17:46:42.5183307Z Running distributed/_tensor/debug/test_comm_mode_features 1/1 ... [2024-08-07 17:46:42.518143] 2024-08-07T17:46:42.5188501Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/debug/test_comm_mode_features.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:46:42.518559] 2024-08-07T17:46:57.2691789Z 2024-08-07T17:46:57.2693155Z distributed/_tensor/debug/test_comm_mode_features 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.debug.test_comm_mode_features_1.1_d3bb2389cd8c4476_.log 2024-08-07T17:46:57.2696431Z Running 4 items in this shard: test/distributed/_tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_MLPStacked_distributed_sharding_display, test/distributed/_tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_MLP_distributed_sharding_display, test/distributed/_tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_MLP_module_tracing, test/distributed/_tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_transformer_module_tracing 2024-08-07T17:46:57.2698890Z 2024-08-07T17:46:57.2699237Z Running distributed/_composable/fsdp/test_fully_shard_state 1/1 ... [2024-08-07 17:46:57.269582] 2024-08-07T17:46:57.2702221Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:46:57.269995] 2024-08-07T17:47:00.4455178Z 2024-08-07T17:47:00.4456408Z distributed/_composable/fsdp/test_fully_shard_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_state_1.1_194bd2c3d531e64c_.log 2024-08-07T17:47:00.4459741Z Running 5 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_state.py::TestFullyShardState::test_fully_shard_cls, test/distributed/_composable/fsdp/test_fully_shard_state.py::TestFullyShardState::test_fully_shard_deepcopy, test/distributed/_composable/fsdp/test_fully_shard_state.py::TestFullyShardState::test_fully_shard_reapply, test/distributed/_composable/fsdp/test_fully_shard_state.py::TestFullyShardState::test_fully_shard_state, test/distributed/_composable/fsdp/test_fully_shard_state.py::TestFullyShardState::test_fully_shard_unsupported_module_cls 2024-08-07T17:47:00.4462493Z 2024-08-07T17:47:00.4462787Z Running distributed/_composable/test_checkpoint 1/1 ... [2024-08-07 17:47:00.445925] 2024-08-07T17:47:00.4466522Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:47:00.446393] 2024-08-07T17:47:03.5216739Z 2024-08-07T17:47:03.5217877Z distributed/_composable/test_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_checkpoint_1.1_ab802b5b200b2edd_.log 2024-08-07T17:47:03.5221158Z Running 6 items in this shard: test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_checkpoint_kwargs, test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_clears_state_on_error_in_forward, test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_multi_args, test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_random_cpu, test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_tensor_only_cpu, test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_tensor_only_gpu 2024-08-07T17:47:03.5223849Z 2024-08-07T17:47:03.5224225Z Running distributed/tensor/parallel/test_tp_style 1/1 ... [2024-08-07 17:47:03.522114] 2024-08-07T17:47:03.5228313Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_tp_style.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:47:03.522586] 2024-08-07T17:47:33.7069231Z 2024-08-07T17:47:33.7070966Z distributed/tensor/parallel/test_tp_style 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_tp_style_1.1_9b52ffca6d02e276_.log 2024-08-07T17:47:33.7080471Z Running 9 items in this shard: test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_colwise_parallel_embedding, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_colwise_parallel_style, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_prepare_module_input, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_prepare_module_input_multiple_inputs, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_prepare_module_kwargs_input, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_prepare_module_output, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_rowwise_parallel_embedding, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_rowwise_parallel_style, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_sequence_parallel_style 2024-08-07T17:47:33.7085401Z 2024-08-07T17:47:33.7085713Z Running distributed/checkpoint/test_traverse 1/1 ... [2024-08-07 17:47:33.707490] 2024-08-07T17:47:33.7087049Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_traverse.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:47:33.707965] 2024-08-07T17:47:36.7341948Z 2024-08-07T17:47:36.7343058Z distributed/checkpoint/test_traverse 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_traverse_1.1_76d12ce56ff6070a_.log 2024-08-07T17:47:36.7346742Z Running 7 items in this shard: test/distributed/checkpoint/test_traverse.py::TestTraverse::test_get_element, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_set_element, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_doesnt_ignore_intermediate_collections, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_nested_dict, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_nested_list, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_shallow, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_with_ordered_dict 2024-08-07T17:47:36.7349682Z 2024-08-07T17:47:36.7350024Z Running distributed/_composable/fsdp/test_fully_shard_logging 1/1 ... [2024-08-07 17:47:36.734669] 2024-08-07T17:47:36.7353352Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_logging.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:47:36.735097] 2024-08-07T17:47:39.5043839Z 2024-08-07T17:47:39.5045351Z distributed/_composable/fsdp/test_fully_shard_logging 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_logging_1.1_bdb6a0478797f6a2_.log 2024-08-07T17:47:39.5046567Z Running 0 items in this shard: 2024-08-07T17:47:39.5046765Z 2024-08-07T17:47:39.5049166Z Running distributed/test_compute_comm_reordering 1/1 ... [2024-08-07 17:47:39.504709] 2024-08-07T17:47:39.5054135Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_compute_comm_reordering.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:47:39.505143] 2024-08-07T17:47:42.5814332Z 2024-08-07T17:47:42.5815635Z distributed/test_compute_comm_reordering 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_compute_comm_reordering_1.1_8e3f53dc4769e27e_.log 2024-08-07T17:47:42.5820484Z Running 7 items in this shard: test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_grouped_scheduler_node, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_nccl_heuristics, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_raise_comms, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_reorder_compute_for_overlap, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_reorder_compute_for_overlap_custom_runtime_estimation, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_sink_waits, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_sink_waits_raise_comms 2024-08-07T17:47:42.5824572Z 2024-08-07T17:47:42.5824928Z Running distributed/tensor/parallel/test_tp_random_state 1/1 ... [2024-08-07 17:47:42.581926] 2024-08-07T17:47:42.5827602Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_tp_random_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:47:42.582385] 2024-08-07T17:47:48.5146897Z 2024-08-07T17:47:48.5148271Z distributed/tensor/parallel/test_tp_random_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_tp_random_state_1.1_ec148801e3d0c5d9_.log 2024-08-07T17:47:48.5149784Z Running 1 items in this shard: test/distributed/tensor/parallel/test_tp_random_state.py::TensorParallelRandomStateTests::test_model_init 2024-08-07T17:47:48.5150440Z 2024-08-07T17:47:48.5152182Z Running distributed/_tools/test_fsdp2_mem_tracker 1/1 ... [2024-08-07 17:47:48.515016] 2024-08-07T17:47:48.5157020Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tools/test_fsdp2_mem_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:47:48.515475] 2024-08-07T17:47:51.5418618Z 2024-08-07T17:47:51.5420098Z distributed/_tools/test_fsdp2_mem_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_fsdp2_mem_tracker_1.1_1d7ab7a8c32f70d5_.log 2024-08-07T17:47:51.5422741Z Running 3 items in this shard: test/distributed/_tools/test_fsdp2_mem_tracker.py::TestTrackerFullyShard1DTrainingCore::test_tracker_multi_group_eager, test/distributed/_tools/test_fsdp2_mem_tracker.py::TestTrackerFullyShard1DTrainingCore::test_tracker_non_root_forward_backward, test/distributed/_tools/test_fsdp2_mem_tracker.py::TestTrackerFullyShard1DTrainingCompose::test_tracker_with_activation_checkpointing 2024-08-07T17:47:51.5424838Z 2024-08-07T17:47:51.5425667Z Running distributed/test_inductor_collectives 1/1 ... [2024-08-07 17:47:51.542321] 2024-08-07T17:47:51.5430074Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_inductor_collectives.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:47:51.542746] 2024-08-07T17:47:55.6193933Z 2024-08-07T17:47:55.6195855Z distributed/test_inductor_collectives 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_inductor_collectives_1.1_4f688a1e553d8707_.log 2024-08-07T17:47:55.6247924Z Running 47 items in this shard: test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_all_to_all_single_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_all_to_all_single_inductor_split_sizes_none, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allgather_contiguous_input, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allgather_into_tensor_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allgather_output_buffer_reuse, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allreduce_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allreduce_inductor_cudagraph_trees, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allreduce_input_buffer_reuse, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_broadcast_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_c10d_functional_tagged_pt2_compliant, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_eager_allreduce_inductor_wait, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_inductor_allreduce_eager_wait, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_permute_tensor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_reduce_scatter_tensor_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_backwards, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_get_world_group_source_GroupMember_WORLD, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_get_world_group_source__get_default_group, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_get_world_group_source_group_WORLD, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_graphbreaks_unsupported_async_op, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_pg_var, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_gather, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_gather_args_match, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_gather_list, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_to_all_single, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_kwargs, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_kwargs_none, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_positional, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_positional_none, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_unspecified, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op0, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op1, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op2, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op3, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op4, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_reduce_scatter, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_support_collective_op_with_async_op_False, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_all_gather_tensor, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_all_gather_tensor_pg, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_allgather_coalesced, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_allreduce, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_reduce_scatter_tensor, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_all_gather_coalesced, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_doesnt_mutate_shared, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_reduce_scatter_coalesced, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_single_op, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_steal_buffer, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_meta 2024-08-07T17:47:55.6298786Z 2024-08-07T17:47:55.6299351Z Running distributed/checkpoint/e2e/test_e2e_save_and_load 1/1 ... [2024-08-07 17:47:55.619910] 2024-08-07T17:47:55.6301539Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/e2e/test_e2e_save_and_load.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:47:55.620396] 2024-08-07T17:48:39.6348729Z 2024-08-07T17:48:39.6350672Z distributed/checkpoint/e2e/test_e2e_save_and_load 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.e2e.test_e2e_save_and_load_1.1_e6800c43e1d40a6f_.log 2024-08-07T17:48:39.6364898Z Running 14 items in this shard: test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_different_ordered_state_dict_keys, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_async_cached_cache_staged_state_dict_False, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_async_cached_cache_staged_state_dict_True, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_False_model_type0, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_False_model_type1, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_False_model_type2, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_True_model_type0, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_True_model_type1, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_True_model_type2, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_no_dist, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_overwrite, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_partial_load, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestNoCPU::test_no_cpu, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestInitStateDict::test_init_state_dict 2024-08-07T17:48:39.6375903Z 2024-08-07T17:48:39.6376293Z Running distributed/launcher/test_run 1/1 ... [2024-08-07 17:48:39.635858] 2024-08-07T17:48:39.6378120Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/launcher/test_run.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:48:39.636426] 2024-08-07T17:49:18.7724437Z 2024-08-07T17:49:18.7725659Z distributed/launcher/test_run 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.launcher.test_run_1.1_f04b5f308e297745_.log 2024-08-07T17:49:18.7737025Z Running 24 items in this shard: test/distributed/launcher/test_run.py::ElasticLaunchTest::test_capture_logs_using_default_logs_specs, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_init_method_env_with_torchelastic, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_init_method_tcp_with_torchelastic, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_is_not_torchelastic_launched, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_is_torchelastic_launched, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_is_torchelastic_launched_with_logs_spec_defined, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic_agent_raise_exception, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic_multiple_agents, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic_worker_raise_exception, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_run_path, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_shutdown, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_standalone, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_bash, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_default_nproc, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_python, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_python_caffe2_bc, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_with_env_vars, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_logs_logs_spec_entrypoint_must_be_defined, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_min_max_nodes_parse, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_gpu_launch_configurations, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_launch_auto_configurations, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_launch_number_configurations, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_launch_unknown_configurations 2024-08-07T17:49:18.7747476Z 2024-08-07T17:49:18.7747830Z Running distributed/_composable/fsdp/test_fully_shard_compile 1/1 ... [2024-08-07 17:49:18.773421] 2024-08-07T17:49:18.7749277Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_compile.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:49:18.773884] 2024-08-07T17:49:28.6621609Z 2024-08-07T17:49:28.6623700Z distributed/_composable/fsdp/test_fully_shard_compile 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_compile_1.1_8e2569d7affe3e80_.log 2024-08-07T17:49:28.6638954Z Running 12 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompileCompute::test_disable_compiling_hooks, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_dynamo_trace_use_training_state, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_nested_fully_shard_backend_aot_eager, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_nested_fully_shard_backend_aot_eager_decomp_partition, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_nested_fully_shard_backend_inductor, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_simple_mlp_fullgraph_backend_aot_eager, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_simple_mlp_fullgraph_backend_aot_eager_decomp_partition, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_simple_mlp_fullgraph_backend_inductor, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_trace_fsdp_set_, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_transformer_backend_aot_eager, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_transformer_backend_aot_eager_decomp_partition, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_transformer_backend_inductor 2024-08-07T17:49:28.6652087Z 2024-08-07T17:49:28.6652747Z Running distributed/_composable/fsdp/test_fully_shard_overlap 1/1 ... [2024-08-07 17:49:28.662570] 2024-08-07T17:49:28.6655300Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_overlap.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:49:28.663013] 2024-08-07T17:49:31.9412308Z 2024-08-07T17:49:31.9413585Z distributed/_composable/fsdp/test_fully_shard_overlap 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_overlap_1.1_75a174308a3c950c_.log 2024-08-07T17:49:31.9415727Z Running 2 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_overlap.py::TestFullyShardOverlap::test_fully_shard_post_optim_event_overlap, test/distributed/_composable/fsdp/test_fully_shard_overlap.py::TestFullyShardOverlap::test_fully_shard_training_overlap 2024-08-07T17:49:31.9417003Z 2024-08-07T17:49:31.9418655Z Running distributed/_tensor/test_view_ops 1/1 ... [2024-08-07 17:49:31.941668] 2024-08-07T17:49:31.9423582Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_view_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:49:31.942137] 2024-08-07T17:50:04.8836883Z 2024-08-07T17:50:04.8840002Z distributed/_tensor/test_view_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_view_ops_1.1_e6297a8bbc299d28_.log 2024-08-07T17:50:04.8841947Z Running 3 items in this shard: test/distributed/_tensor/test_view_ops.py::TestViewOps::test_complex_view_ops, test/distributed/_tensor/test_view_ops.py::TestViewOps::test_view_groups, test/distributed/_tensor/test_view_ops.py::TestViewOps::test_view_ops 2024-08-07T17:50:04.8843429Z 2024-08-07T17:50:04.8844204Z Running distributed/checkpoint/e2e/test_fine_tuning 1/1 ... [2024-08-07 17:50:04.884176] 2024-08-07T17:50:04.8848746Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/e2e/test_fine_tuning.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:50:04.884620] 2024-08-07T17:50:07.9105972Z 2024-08-07T17:50:07.9107188Z distributed/checkpoint/e2e/test_fine_tuning 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.e2e.test_fine_tuning_1.1_4d469253ed32f39e_.log 2024-08-07T17:50:07.9108556Z Running 1 items in this shard: test/distributed/checkpoint/e2e/test_fine_tuning.py::TestFineTuning::test_fine_tuning 2024-08-07T17:50:07.9109106Z 2024-08-07T17:50:07.9112446Z Running distributed/elastic/test_control_plane 1/1 ... [2024-08-07 17:50:07.911041] 2024-08-07T17:50:07.9118020Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/test_control_plane.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:50:07.911527] 2024-08-07T17:50:14.1422785Z 2024-08-07T17:50:14.1423942Z distributed/elastic/test_control_plane 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.test_control_plane_1.1_93e7aaf2656b96f2_.log 2024-08-07T17:50:14.1428829Z Running 9 items in this shard: test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_nccl_trace_pickle, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_nccl_trace_pickle_with_json, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_nccl_trace_pickle_with_params, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_traceback, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_get_handler_names, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_get_handler_nonexistant, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_run_handler, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_tcp, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_worker_server 2024-08-07T17:50:14.1432982Z 2024-08-07T17:50:14.1433291Z Running distributed/checkpoint/test_planner 1/1 ... [2024-08-07 17:50:14.142765] 2024-08-07T17:50:14.1434751Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_planner.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:50:14.143208] 2024-08-07T17:50:17.2686793Z 2024-08-07T17:50:17.2688011Z distributed/checkpoint/test_planner 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_planner_1.1_5f575488ed3709df_.log 2024-08-07T17:50:17.2691834Z Running 7 items in this shard: test/distributed/checkpoint/test_planner.py::TestSavePlan::test_global_plan, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_load_with_resharding, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_load_with_world_size_diff_by_one, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_local_load_plan, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_local_plan, test/distributed/checkpoint/test_planner.py::TestPlannerHelpers::test_create_read_item_from_chunks, test/distributed/checkpoint/test_planner.py::TestLoadPlanner::test_strict 2024-08-07T17:50:17.2694757Z 2024-08-07T17:50:17.2695102Z Running distributed/checkpoint/test_dtensor_checkpoint 1/1 ... [2024-08-07 17:50:17.269121] 2024-08-07T17:50:17.2698094Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_dtensor_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:50:17.269549] 2024-08-07T17:50:23.4515817Z 2024-08-07T17:50:23.4517863Z distributed/checkpoint/test_dtensor_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_dtensor_checkpoint_1.1_3c66b27e3ac3056d_.log 2024-08-07T17:50:23.4521014Z Running 1 items in this shard: test/distributed/checkpoint/test_dtensor_checkpoint.py::DTensorPlanner::test_distributed_tensor_planner 2024-08-07T17:50:23.4522156Z 2024-08-07T17:50:23.4525365Z Running distributed/fsdp/test_fsdp_tp_integration 1/1 ... [2024-08-07 17:50:23.452277] 2024-08-07T17:50:23.4531748Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_tp_integration.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:50:23.452807] 2024-08-07T17:50:35.2473710Z 2024-08-07T17:50:35.2474830Z distributed/fsdp/test_fsdp_tp_integration 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_tp_integration_1.1_0020767bc8e6acc2_.log 2024-08-07T17:50:35.2477117Z Running 3 items in this shard: test/distributed/fsdp/test_fsdp_tp_integration.py::TestTPFSDPIntegration::test_fsdp_tp_extension_grad, test/distributed/fsdp/test_fsdp_tp_integration.py::TestTPFSDPIntegration::test_fsdp_tp_integration, test/distributed/fsdp/test_fsdp_tp_integration.py::TestTPFSDPIntegration::test_fsdp_tp_sync_module_state 2024-08-07T17:50:35.2478913Z 2024-08-07T17:50:35.2479627Z Running distributed/_composable/fsdp/test_fully_shard_memory 1/1 ... [2024-08-07 17:50:35.247732] 2024-08-07T17:50:35.2485761Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_memory.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:50:35.248136] 2024-08-07T17:50:38.2733458Z 2024-08-07T17:50:38.2734851Z distributed/_composable/fsdp/test_fully_shard_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_memory_1.1_453f3c698f7d7572_.log 2024-08-07T17:50:38.2736397Z Running 1 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_memory.py::TestFullyShardMemory::test_fully_shard_training_memory 2024-08-07T17:50:38.2737068Z 2024-08-07T17:50:38.2739805Z Running distributed/checkpoint/e2e/test_pipeline 1/1 ... [2024-08-07 17:50:38.273783] 2024-08-07T17:50:38.2744440Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/e2e/test_pipeline.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:50:38.274204] 2024-08-07T17:50:41.3493059Z 2024-08-07T17:50:41.3494273Z distributed/checkpoint/e2e/test_pipeline 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.e2e.test_pipeline_1.1_0747a1c5770b7eb5_.log 2024-08-07T17:50:41.3495595Z Running 1 items in this shard: test/distributed/checkpoint/e2e/test_pipeline.py::TestPipeline::test_pipeline 2024-08-07T17:50:41.3496120Z 2024-08-07T17:50:41.3499369Z Running distributed/_tensor/test_op_strategy 1/1 ... [2024-08-07 17:50:41.349737] 2024-08-07T17:50:41.3504405Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_op_strategy.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:50:41.350200] 2024-08-07T17:50:44.5265729Z 2024-08-07T17:50:44.5266893Z distributed/_tensor/test_op_strategy 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_op_strategy_1.1_244c2a6459dd8010_.log 2024-08-07T17:50:44.5273401Z Running 15 items in this shard: test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_batch_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_bmm_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_free_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_mm_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_bmm_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_bmm_2d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_linearity_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_mm_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_mm_2d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_pointwise_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_bmm_strategies, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_mm_strategies, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_redistribute_cost_latency, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_redistribute_cost_mesh_1d, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_redistribute_cost_mesh_2d 2024-08-07T17:50:44.5279190Z 2024-08-07T17:50:44.5279499Z Running distributed/checkpoint/test_compatibility 1/1 ... [2024-08-07 17:50:44.526954] 2024-08-07T17:50:44.5280877Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_compatibility.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:50:44.527396] 2024-08-07T17:50:47.5545461Z 2024-08-07T17:50:47.5547357Z distributed/checkpoint/test_compatibility 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_compatibility_1.1_f2c3c7fc8e2fe9fb_.log 2024-08-07T17:50:47.5549595Z Running 3 items in this shard: test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_metadata, test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_sharded_tensor_dependency, test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_storage_meta 2024-08-07T17:50:47.5551381Z 2024-08-07T17:50:47.5551976Z Running distributed/_tensor/test_embedding_ops 1/1 ... [2024-08-07 17:50:47.554984] 2024-08-07T17:50:47.5556836Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_embedding_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:50:47.555433] 2024-08-07T17:51:03.1631604Z 2024-08-07T17:51:03.1633029Z distributed/_tensor/test_embedding_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_embedding_ops_1.1_345116dddc823fdc_.log 2024-08-07T17:51:03.1635680Z Running 4 items in this shard: test/distributed/_tensor/test_embedding_ops.py::TestEmbeddingOp::test_multiple_embeddings_rowwise, test/distributed/_tensor/test_embedding_ops.py::TestEmbeddingOp::test_sharded_embedding_colwise, test/distributed/_tensor/test_embedding_ops.py::TestEmbeddingOp::test_sharded_embedding_colwise_max_norm_errors, test/distributed/_tensor/test_embedding_ops.py::TestEmbeddingOp::test_sharded_embedding_rowwise 2024-08-07T17:51:03.1637692Z 2024-08-07T17:51:03.1638176Z Running distributed/fsdp/test_fsdp_ignored_modules 1/1 ... [2024-08-07 17:51:03.163625] 2024-08-07T17:51:03.1643187Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_ignored_modules.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:51:03.164071] 2024-08-07T17:51:06.2414650Z 2024-08-07T17:51:06.2415980Z distributed/fsdp/test_fsdp_ignored_modules 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_ignored_modules_1.1_6b22c27396e6aca0_.log 2024-08-07T17:51:06.2424703Z Running 13 items in this shard: test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_diff_ignored_modules_across_ranks, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_invalid_composable_False, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_invalid_composable_True, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_nested, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_nested_composable, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_False_composable_False, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_False_composable_True, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_True_composable_False, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_True_composable_True, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_transformer, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_transformer_composable, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_states_auto_wrap, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_states_check 2024-08-07T17:51:06.2432553Z 2024-08-07T17:51:06.2432933Z Running distributed/checkpoint/test_hsdp_checkpoint 1/1 ... [2024-08-07 17:51:06.241904] 2024-08-07T17:51:06.2434333Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_hsdp_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:51:06.242400] 2024-08-07T17:51:20.1926481Z 2024-08-07T17:51:20.1928713Z distributed/checkpoint/test_hsdp_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_hsdp_checkpoint_1.1_fc5612e41458e053_.log 2024-08-07T17:51:20.1934709Z Running 4 items in this shard: test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_checkpoint_is_even_sharded_model_False, test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_checkpoint_is_even_sharded_model_True, test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_fsdp_checkpoint_conversion_is_even_sharded_model_False, test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_fsdp_checkpoint_conversion_is_even_sharded_model_True 2024-08-07T17:51:20.1938961Z 2024-08-07T17:51:20.1939482Z Running distributed/pipelining/test_schedule 1/1 ... [2024-08-07 17:51:20.193380] 2024-08-07T17:51:20.1942282Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_schedule.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:51:20.193936] 2024-08-07T17:52:28.0529974Z 2024-08-07T17:52:28.0531178Z distributed/pipelining/test_schedule 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_schedule_1.1_986cab85da7df3fc_.log 2024-08-07T17:52:28.0538142Z Running 12 items in this shard: test/distributed/pipelining/test_schedule.py::TestSchedulePlan::test_pipeline_order_ScheduleClass0, test/distributed/pipelining/test_schedule.py::TestSchedulePlan::test_pipeline_order_ScheduleClass1, test/distributed/pipelining/test_schedule.py::TestSchedulePlan::test_pipeline_order_flex_and_zero_bubble_ScheduleClass0, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_action_parse_action_str_and_ref0, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_action_parse_action_str_and_ref1, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_action_parse_action_str_and_ref2, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_action_parse_action_str_and_ref3, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_action_parse_action_str_and_ref4, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_action_parse_action_str_and_ref5, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_action_parse_action_str_and_ref6, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_send_recv_test_info0, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_unshard_reshard_test_info0 2024-08-07T17:52:28.0544574Z 2024-08-07T17:52:28.0544835Z Running distributed/test_functional_api 1/1 ... [2024-08-07 17:52:28.053532] 2024-08-07T17:52:28.0546099Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_functional_api.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:52:28.053992] 2024-08-07T17:53:02.7296401Z 2024-08-07T17:53:02.7297812Z distributed/test_functional_api 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_functional_api_1.1_687ff3d99d83df63_.log 2024-08-07T17:53:02.7321285Z Running 50 items in this shard: test/distributed/test_functional_api.py::TestExpand::test_expand_1d_rank_list, test/distributed/test_functional_api.py::TestExpand::test_expand_2d_rank_list, test/distributed/test_functional_api.py::TestExpand::test_expand_device_mesh, test/distributed/test_functional_api.py::TestExpand::test_expand_device_mesh_tuple, test/distributed/test_functional_api.py::TestExpand::test_expand_process_group, test/distributed/test_functional_api.py::TestPgTag::test_find_or_create_pg, test/distributed/test_functional_api.py::TestPgTag::test_find_root_pg, test/distributed/test_functional_api.py::TestPgTag::test_pg_creation_with_tag, test/distributed/test_functional_api.py::TestPgTag::test_pg_lookup_roundtrip, test/distributed/test_functional_api.py::TestPgTag::test_pg_lookup_with_tag, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_gather_into_tensor_coalesced_device_cpu, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_gather_into_tensor_coalesced_device_cuda, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_gather_tensor_device_cpu, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_gather_tensor_device_cuda, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_reduce_coalesced_eager_device_cpu, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_reduce_coalesced_eager_device_cuda, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_reduce_eager_device_cpu, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_reduce_eager_device_cuda, test/distributed/test_functional_api.py::TestTraceableCollectives::test_broadcast_device_cpu, test/distributed/test_functional_api.py::TestTraceableCollectives::test_broadcast_device_cuda, test/distributed/test_functional_api.py::TestTraceableCollectives::test_reduce_scatter_into_tensor_coalesced_device_cpu, test/distributed/test_functional_api.py::TestTraceableCollectives::test_reduce_scatter_into_tensor_coalesced_device_cuda, test/distributed/test_functional_api.py::TestTraceableCollectives::test_reduce_scatter_tensor_device_cpu, test/distributed/test_functional_api.py::TestTraceableCollectives::test_reduce_scatter_tensor_device_cuda, test/distributed/test_functional_api.py::TestMetaCollectives::test_all_reduce, test/distributed/test_functional_api.py::TestGradCollectives::test_all_reduce, test/distributed/test_functional_api.py::TestMakeFx::test_all_reduce_tracing, test/distributed/test_functional_api.py::TestCollectivesWithNCCL::test_all_gather_into_tensor_coalesced, test/distributed/test_functional_api.py::TestCollectivesWithNCCL::test_all_to_all_single, test/distributed/test_functional_api.py::TestCollectivesWithNCCL::test_all_to_all_single_1d_input, test/distributed/test_functional_api.py::TestCollectivesWithNCCL::test_all_to_all_single_split_sizes_none, test/distributed/test_functional_api.py::TestCollectivesWithNCCL::test_tracing, test/distributed/test_functional_api.py::TestCollectivesWithNCCL::test_tracing_with_dce_code, test/distributed/test_functional_api.py::TestCollectivesWithNCCL::test_tracing_with_fakepg, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_all_gather_into_tensor_coalesced, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_all_to_all_single, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_all_to_all_single_1d_input, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_all_to_all_single_split_sizes_none, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_permute_tensor_with_sub_group, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_tracing, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_tracing_with_dce_code, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_tracing_with_fakepg, test/distributed/test_functional_api.py::TestFunctionalAutograd::test_all_gather_tensor_compile_False, test/distributed/test_functional_api.py::TestFunctionalAutograd::test_all_gather_tensor_compile_True, test/distributed/test_functional_api.py::TestFunctionalAutograd::test_all_to_all_single_compile_False, test/distributed/test_functional_api.py::TestFunctionalAutograd::test_all_to_all_single_compile_True, test/distributed/test_functional_api.py::TestFunctionalAutograd::test_all_to_all_single_inductor, test/distributed/test_functional_api.py::TestFunctionalAutograd::test_reduce_scatter_tensor_compile_False, test/distributed/test_functional_api.py::TestFunctionalAutograd::test_reduce_scatter_tensor_compile_True, test/distributed/test_functional_api.py::TestFunctionalAutogradWithNCCL::test_all_to_all_single 2024-08-07T17:53:02.7342077Z 2024-08-07T17:53:02.7342455Z Running distributed/_composable/fully_shard/test_fully_shard_compile 1/1 ... [2024-08-07 17:53:02.730198] 2024-08-07T17:53:02.7343970Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_compile.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:53:02.730649] 2024-08-07T17:53:05.2042904Z 2024-08-07T17:53:05.2045043Z distributed/_composable/fully_shard/test_fully_shard_compile 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_compile_1.1_970b30e3954589da_.log 2024-08-07T17:53:05.2047974Z Running 1 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_compile.py::TestCompile::test_compile 2024-08-07T17:53:05.2049078Z 2024-08-07T17:53:05.2052382Z Running distributed/checkpoint/test_state_dict_utils 1/1 ... [2024-08-07 17:53:05.205018] 2024-08-07T17:53:05.2058879Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_state_dict_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:53:05.205532] 2024-08-07T17:53:07.8795866Z 2024-08-07T17:53:07.8797348Z distributed/checkpoint/test_state_dict_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_state_dict_utils_1.1_b561fcc064884f20_.log 2024-08-07T17:53:07.8801116Z Running 5 items in this shard: test/distributed/checkpoint/test_state_dict_utils.py::TestStateDictUtils::test_complicated_dict, test/distributed/checkpoint/test_state_dict_utils.py::TestStateDictUtils::test_cpu_and_ranks_only, test/distributed/checkpoint/test_state_dict_utils.py::TestStateDictUtils::test_create_cpu_state_dict, test/distributed/checkpoint/test_state_dict_utils.py::TestStateDictUtils::test_gather_state_dict_dtensor, test/distributed/checkpoint/test_state_dict_utils.py::TestStateDictUtils::test_gather_with_cpu_and_ranks_only 2024-08-07T17:53:07.8803954Z 2024-08-07T17:53:07.8804232Z Running distributed/_composable/test_replicate 1/1 ... [2024-08-07 17:53:07.879977] 2024-08-07T17:53:07.8806631Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_replicate.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:53:07.880406] 2024-08-07T17:53:34.6942554Z 2024-08-07T17:53:34.6943656Z distributed/_composable/test_replicate 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_replicate_1.1_004b78bb4139cee0_.log 2024-08-07T17:53:34.6948625Z Running 9 items in this shard: test/distributed/_composable/test_replicate.py::ReplicateStateDictTest::test_replicate_non_root_multiple_save_load, test/distributed/_composable/test_replicate.py::ReplicateStateDictTest::test_replicate_single_module_save_load, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_device_id, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_ignore_module, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_move_args_kwargs_to_device, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_multi_module, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_single_module, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_with_kwargs, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_wrong_device_id_type 2024-08-07T17:53:34.6953416Z 2024-08-07T17:53:34.6953865Z Running distributed/optim/test_apply_optimizer_in_backward 1/1 ... [2024-08-07 17:53:34.694792] 2024-08-07T17:53:34.6955437Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/optim/test_apply_optimizer_in_backward.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:53:34.695254] 2024-08-07T17:53:36.7393012Z 2024-08-07T17:53:36.7394309Z distributed/optim/test_apply_optimizer_in_backward 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.optim.test_apply_optimizer_in_backward_1.1_ae19531074ad7647_.log 2024-08-07T17:53:36.7395257Z 2024-08-07T17:53:36.7399774Z Running distributed/_composable/fsdp/test_fully_shard_frozen 1/1 ... [2024-08-07 17:53:36.739788] 2024-08-07T17:53:36.7404624Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_frozen.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:53:36.740234] 2024-08-07T17:53:39.5141626Z 2024-08-07T17:53:39.5142972Z distributed/_composable/fsdp/test_fully_shard_frozen 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_frozen_1.1_234f36918ffc22a5_.log 2024-08-07T17:53:39.5145823Z Running 3 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_frozen.py::TestFullyShardFrozen::test_multi_forward_mixed_requires_grad, test/distributed/_composable/fsdp/test_fully_shard_frozen.py::TestFullyShardFrozen::test_train_mixed_requires_grad_across_groups, test/distributed/_composable/fsdp/test_fully_shard_frozen.py::TestFullyShardFrozen::test_train_mixed_requires_grad_per_group 2024-08-07T17:53:39.5147848Z 2024-08-07T17:53:39.5148528Z Running distributed/_tensor/test_optimizers 1/1 ... [2024-08-07 17:53:39.514623] 2024-08-07T17:53:39.5153175Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_optimizers.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:53:39.515055] 2024-08-07T17:54:18.1741249Z 2024-08-07T17:54:18.1742488Z distributed/_tensor/test_optimizers 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_optimizers_1.1_8b69f7ec634d46b8_.log 2024-08-07T17:54:18.1749852Z Running 11 items in this shard: test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_RMSprop_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_adadelta_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_adagrad_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_adam_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_adamax_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_adamw_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_asgd_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_nadam_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_optimizer_foreach_supported_types_include_DTensor, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_radam_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_sgd_1d_sharding 2024-08-07T17:54:18.1755080Z 2024-08-07T17:54:18.1755346Z Running distributed/test_distributed_spawn 2/3 ... [2024-08-07 17:54:18.175200] 2024-08-07T17:54:18.1820524Z MPI not available -- MPI backend tests will be skipped 2024-08-07T17:54:18.1822022Z Running distributed tests for the test backend with env init_method 2024-08-07T17:54:18.1829903Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=2', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:54:18.182765] 2024-08-07T17:54:20.8047351Z 2024-08-07T17:54:20.8049086Z distributed/test_distributed_spawn 2/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_2.3_da21afe04599d41a_.log 2024-08-07T17:54:20.8050688Z Running 0 items in this shard: 2024-08-07T17:54:20.8051009Z 2024-08-07T17:54:20.8057479Z Running distributed tests for the test backend with file init_method 2024-08-07T17:54:20.8063252Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=2', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:54:20.806051] 2024-08-07T17:54:23.5171715Z 2024-08-07T17:54:23.5172805Z distributed/test_distributed_spawn 2/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_2.3_81c92bafc3a0f3d5_.log 2024-08-07T17:54:23.5173733Z Running 0 items in this shard: 2024-08-07T17:54:23.5173964Z 2024-08-07T17:54:23.5180988Z Running distributed tests for the gloo backend with env init_method 2024-08-07T17:54:23.5186963Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=2', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:54:23.518466] 2024-08-07T18:02:13.7614310Z 2024-08-07T18:02:13.7617147Z distributed/test_distributed_spawn 2/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_2.3_c11380512544f2d7_.log 2024-08-07T18:02:13.7672107Z Running 92 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Single_Input_Per_Process, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_half, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedSampler_padding, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_SyncBatchNorm_process_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_with_then_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_simple, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_with_empty, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_v_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_complex_unsupported_ops, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_global, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_ignored_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce_return_future, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_comm_hook_logging, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device_mesh_initialization, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_grad_div_uneven_inputs, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_has_finalized, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_powerSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_inference, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_join_model_equivalence, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_err_ignore_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_no_set_grad_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_set_grad_to_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sink_noclone, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_bn_training_vs_eval, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_exception, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_unused_params_rebuild_buckets_exception, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_zero_output_features, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_detect_ddp_is_actually_static, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_dump_DDP_relevant_env_vars, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_backend, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_grads_same_across_ranks_with_no_sync, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang_wait_all_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_rank_0_timeout, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_broadcast, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_input_rank_exceeds_world_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_overlap_not_allowed, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_dict_module, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_step_reload, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_v_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_twice, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_stateless_api_with_ddp, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sync_bn_logged 2024-08-07T18:02:13.7714453Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel 2024-08-07T18:02:13.7715606Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm 2024-08-07T18:02:13.7717078Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Single_Input_Per_Process 2024-08-07T18:02:13.7718408Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_half 2024-08-07T18:02:13.7719581Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedSampler_padding 2024-08-07T18:02:13.7720703Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_SyncBatchNorm_process_group 2024-08-07T18:02:13.7721918Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_with_then_hook 2024-08-07T18:02:13.7723140Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_full_group 2024-08-07T18:02:13.7724259Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_simple 2024-08-07T18:02:13.7725375Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_with_empty 2024-08-07T18:02:13.7726445Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_complex 2024-08-07T18:02:13.7727470Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda_complex 2024-08-07T18:02:13.7728502Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_v_cuda 2024-08-07T18:02:13.7729555Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_product 2024-08-07T18:02:13.7730670Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_complex_unsupported_ops 2024-08-07T18:02:13.7731785Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_sum 2024-08-07T18:02:13.7732828Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_min 2024-08-07T18:02:13.7733872Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_complex 2024-08-07T18:02:13.7734885Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda 2024-08-07T18:02:13.7735857Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all 2024-08-07T18:02:13.7736828Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_complex 2024-08-07T18:02:13.7737830Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda 2024-08-07T18:02:13.7738894Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_complex 2024-08-07T18:02:13.7740061Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda 2024-08-07T18:02:13.7741385Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split 2024-08-07T18:02:13.7742542Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_complex 2024-08-07T18:02:13.7743779Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda_complex 2024-08-07T18:02:13.7745036Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group_cuda 2024-08-07T18:02:13.7746255Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group 2024-08-07T18:02:13.7747340Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_group 2024-08-07T18:02:13.7748309Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group 2024-08-07T18:02:13.7749322Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_global 2024-08-07T18:02:13.7750371Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_nccl 2024-08-07T18:02:13.7751425Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_err 2024-08-07T18:02:13.7752429Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_cuda 2024-08-07T18:02:13.7753485Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_group 2024-08-07T18:02:13.7754539Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward 2024-08-07T18:02:13.7755724Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_ignored_params 2024-08-07T18:02:13.7756843Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer 2024-08-07T18:02:13.7758033Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce_return_future 2024-08-07T18:02:13.7759163Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_comm_hook_logging 2024-08-07T18:02:13.7760232Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device_mesh_initialization 2024-08-07T18:02:13.7761351Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_grad_div_uneven_inputs 2024-08-07T18:02:13.7762402Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_has_finalized 2024-08-07T18:02:13.7763616Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce 2024-08-07T18:02:13.7764673Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_powerSGD 2024-08-07T18:02:13.7765689Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_inference 2024-08-07T18:02:13.7766720Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_join_model_equivalence 2024-08-07T18:02:13.7767918Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_err_ignore_params 2024-08-07T18:02:13.7769356Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_no_set_grad_none 2024-08-07T18:02:13.7770837Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_set_grad_to_none 2024-08-07T18:02:13.7772114Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_autograd_profiler 2024-08-07T18:02:13.7773245Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_torch_profiler 2024-08-07T18:02:13.7774274Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sink_noclone 2024-08-07T18:02:13.7775322Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_bn_training_vs_eval 2024-08-07T18:02:13.7776425Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_exception 2024-08-07T18:02:13.7777575Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_unused_params_rebuild_buckets_exception 2024-08-07T18:02:13.7778731Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_zero_output_features 2024-08-07T18:02:13.7779773Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_full_group 2024-08-07T18:02:13.7780761Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_group 2024-08-07T18:02:13.7781794Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_detect_ddp_is_actually_static 2024-08-07T18:02:13.7782898Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_dump_DDP_relevant_env_vars 2024-08-07T18:02:13.7783929Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_cuda 2024-08-07T18:02:13.7784915Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object_subgroup 2024-08-07T18:02:13.7785912Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_backend 2024-08-07T18:02:13.7786923Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_full_group 2024-08-07T18:02:13.7788033Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_grads_same_across_ranks_with_no_sync 2024-08-07T18:02:13.7789062Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend 2024-08-07T18:02:13.7790032Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_torch_profiler 2024-08-07T18:02:13.7791178Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang_wait_all_ranks 2024-08-07T18:02:13.7792403Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_rank_0_timeout 2024-08-07T18:02:13.7793610Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_broadcast 2024-08-07T18:02:13.7794715Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration 2024-08-07T18:02:13.7795952Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_input_rank_exceeds_world_size 2024-08-07T18:02:13.7797322Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_overlap_not_allowed 2024-08-07T18:02:13.7798460Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_dict_module 2024-08-07T18:02:13.7799574Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager 2024-08-07T18:02:13.7800763Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd 2024-08-07T18:02:13.7801988Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_step_reload 2024-08-07T18:02:13.7803095Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_max 2024-08-07T18:02:13.7804289Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_product 2024-08-07T18:02:13.7805300Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_sum 2024-08-07T18:02:13.7806256Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_max 2024-08-07T18:02:13.7807243Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_v_cuda 2024-08-07T18:02:13.7808264Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_twice 2024-08-07T18:02:13.7809244Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_group 2024-08-07T18:02:13.7810222Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source 2024-08-07T18:02:13.7811317Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_torch_profiler 2024-08-07T18:02:13.7812442Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_autograd_profiler 2024-08-07T18:02:13.7813504Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum 2024-08-07T18:02:13.7814544Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_stateless_api_with_ddp 2024-08-07T18:02:13.7815554Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sync_bn_logged 2024-08-07T18:02:13.7816098Z 2024-08-07T18:02:13.7816336Z Running distributed tests for the gloo backend with file init_method 2024-08-07T18:02:13.7817653Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=2', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:02:13.765746] 2024-08-07T18:10:03.4485914Z 2024-08-07T18:10:03.4486990Z distributed/test_distributed_spawn 2/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_2.3_cbb3550ae9f426c8_.log 2024-08-07T18:10:03.4545085Z Running 92 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Single_Input_Per_Process, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_half, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedSampler_padding, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_SyncBatchNorm_process_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_with_then_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_simple, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_with_empty, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_v_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_complex_unsupported_ops, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_global, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_ignored_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce_return_future, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_comm_hook_logging, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device_mesh_initialization, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_grad_div_uneven_inputs, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_has_finalized, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_powerSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_inference, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_join_model_equivalence, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_err_ignore_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_no_set_grad_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_set_grad_to_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sink_noclone, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_bn_training_vs_eval, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_exception, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_unused_params_rebuild_buckets_exception, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_zero_output_features, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_detect_ddp_is_actually_static, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_dump_DDP_relevant_env_vars, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_backend, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_grads_same_across_ranks_with_no_sync, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang_wait_all_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_rank_0_timeout, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_broadcast, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_input_rank_exceeds_world_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_overlap_not_allowed, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_dict_module, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_step_reload, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_v_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_twice, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_stateless_api_with_ddp, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sync_bn_logged 2024-08-07T18:10:03.4587874Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel 2024-08-07T18:10:03.4589026Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm 2024-08-07T18:10:03.4590345Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Single_Input_Per_Process 2024-08-07T18:10:03.4591691Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_half 2024-08-07T18:10:03.4592941Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedSampler_padding 2024-08-07T18:10:03.4594046Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_SyncBatchNorm_process_group 2024-08-07T18:10:03.4595261Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_with_then_hook 2024-08-07T18:10:03.4596483Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_full_group 2024-08-07T18:10:03.4597602Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_simple 2024-08-07T18:10:03.4598712Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_with_empty 2024-08-07T18:10:03.4599782Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_complex 2024-08-07T18:10:03.4600800Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda_complex 2024-08-07T18:10:03.4601833Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_v_cuda 2024-08-07T18:10:03.4602891Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_product 2024-08-07T18:10:03.4604011Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_complex_unsupported_ops 2024-08-07T18:10:03.4605140Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_sum 2024-08-07T18:10:03.4606188Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_min 2024-08-07T18:10:03.4607227Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_complex 2024-08-07T18:10:03.4608801Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda 2024-08-07T18:10:03.4609897Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all 2024-08-07T18:10:03.4610881Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_complex 2024-08-07T18:10:03.4611878Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda 2024-08-07T18:10:03.4612949Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_complex 2024-08-07T18:10:03.4614125Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda 2024-08-07T18:10:03.4615266Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split 2024-08-07T18:10:03.4616428Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_complex 2024-08-07T18:10:03.4617657Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda_complex 2024-08-07T18:10:03.4618911Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group_cuda 2024-08-07T18:10:03.4620130Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group 2024-08-07T18:10:03.4621192Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_group 2024-08-07T18:10:03.4622163Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group 2024-08-07T18:10:03.4623176Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_global 2024-08-07T18:10:03.4624223Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_nccl 2024-08-07T18:10:03.4625268Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_err 2024-08-07T18:10:03.4626288Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_cuda 2024-08-07T18:10:03.4627275Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_group 2024-08-07T18:10:03.4628306Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward 2024-08-07T18:10:03.4629478Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_ignored_params 2024-08-07T18:10:03.4630620Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer 2024-08-07T18:10:03.4631730Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce_return_future 2024-08-07T18:10:03.4632928Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_comm_hook_logging 2024-08-07T18:10:03.4634010Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device_mesh_initialization 2024-08-07T18:10:03.4635124Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_grad_div_uneven_inputs 2024-08-07T18:10:03.4636168Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_has_finalized 2024-08-07T18:10:03.4637316Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce 2024-08-07T18:10:03.4638382Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_powerSGD 2024-08-07T18:10:03.4639401Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_inference 2024-08-07T18:10:03.4640415Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_join_model_equivalence 2024-08-07T18:10:03.4641607Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_err_ignore_params 2024-08-07T18:10:03.4642953Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_no_set_grad_none 2024-08-07T18:10:03.4644358Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_set_grad_to_none 2024-08-07T18:10:03.4645623Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_autograd_profiler 2024-08-07T18:10:03.4646744Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_torch_profiler 2024-08-07T18:10:03.4647793Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sink_noclone 2024-08-07T18:10:03.4648842Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_bn_training_vs_eval 2024-08-07T18:10:03.4649921Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_exception 2024-08-07T18:10:03.4651095Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_unused_params_rebuild_buckets_exception 2024-08-07T18:10:03.4652250Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_zero_output_features 2024-08-07T18:10:03.4653287Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_full_group 2024-08-07T18:10:03.4654260Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_group 2024-08-07T18:10:03.4655306Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_detect_ddp_is_actually_static 2024-08-07T18:10:03.4656403Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_dump_DDP_relevant_env_vars 2024-08-07T18:10:03.4657405Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_cuda 2024-08-07T18:10:03.4658414Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object_subgroup 2024-08-07T18:10:03.4659407Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_backend 2024-08-07T18:10:03.4660407Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_full_group 2024-08-07T18:10:03.4661512Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_grads_same_across_ranks_with_no_sync 2024-08-07T18:10:03.4662549Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend 2024-08-07T18:10:03.4663717Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_torch_profiler 2024-08-07T18:10:03.4664972Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang_wait_all_ranks 2024-08-07T18:10:03.4666268Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_rank_0_timeout 2024-08-07T18:10:03.4667409Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_broadcast 2024-08-07T18:10:03.4668510Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration 2024-08-07T18:10:03.4669736Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_input_rank_exceeds_world_size 2024-08-07T18:10:03.4670983Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_overlap_not_allowed 2024-08-07T18:10:03.4672150Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_dict_module 2024-08-07T18:10:03.4673317Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager 2024-08-07T18:10:03.4674494Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd 2024-08-07T18:10:03.4675739Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_step_reload 2024-08-07T18:10:03.4676960Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_max 2024-08-07T18:10:03.4677988Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_product 2024-08-07T18:10:03.4678986Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_sum 2024-08-07T18:10:03.4679950Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_max 2024-08-07T18:10:03.4680937Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_v_cuda 2024-08-07T18:10:03.4681947Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_twice 2024-08-07T18:10:03.4682913Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_group 2024-08-07T18:10:03.4683907Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source 2024-08-07T18:10:03.4685008Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_torch_profiler 2024-08-07T18:10:03.4686124Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_autograd_profiler 2024-08-07T18:10:03.4687354Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum 2024-08-07T18:10:03.4688401Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_stateless_api_with_ddp 2024-08-07T18:10:03.4689405Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sync_bn_logged 2024-08-07T18:10:03.4689941Z 2024-08-07T18:10:03.4690336Z Running distributed/_composable/fully_shard/test_fully_shard_mixed_precision 1/1 ... [2024-08-07 18:10:03.452693] 2024-08-07T18:10:03.4692030Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_mixed_precision.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:10:03.453070] 2024-08-07T18:10:08.9310143Z 2024-08-07T18:10:08.9311535Z distributed/_composable/fully_shard/test_fully_shard_mixed_precision 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_mixed_precision_1.1_e77e74eff81e23c0_.log 2024-08-07T18:10:08.9313326Z Running 1 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_mixed_precision.py::TestMixedPrecision::test_float16_cast_forward 2024-08-07T18:10:08.9314028Z 2024-08-07T18:10:08.9316532Z Running distributed/_composable/fully_shard/test_fully_shard_util 1/1 ... [2024-08-07 18:10:08.931466] 2024-08-07T18:10:08.9322053Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_util.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:10:08.931971] 2024-08-07T18:10:14.2600541Z 2024-08-07T18:10:14.2601927Z distributed/_composable/fully_shard/test_fully_shard_util 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_util_1.1_e483b7de3346e9f8_.log 2024-08-07T18:10:14.2603562Z Running 1 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_util.py::TestUtils::test_get_sharded_module_tree_with_module_name_to_fqns 2024-08-07T18:10:14.2604279Z 2024-08-07T18:10:14.2606332Z Running distributed/fsdp/test_fsdp_hybrid_shard 1/1 ... [2024-08-07 18:10:14.260453] 2024-08-07T18:10:14.2610857Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_hybrid_shard.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:10:14.260858] 2024-08-07T18:10:32.7611272Z 2024-08-07T18:10:32.7613054Z distributed/fsdp/test_fsdp_hybrid_shard 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_hybrid_shard_1.1_ac2273a1a0ab8443_.log 2024-08-07T18:10:32.7619800Z Running 6 items in this shard: test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_fsdp_hybrid_shard_basic_setup, test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_fsdp_hybrid_shard_parity, test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_hsdp_save_load_state_dict, test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_hsdp_sync_module_state, test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_invalid_pg_specification_raises, test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_raises_manual_wrap_hybrid_shard_when_none_policy 2024-08-07T18:10:32.7624888Z 2024-08-07T18:10:32.7625510Z Running distributed/fsdp/test_fsdp_dtensor_state_dict 1/1 ... [2024-08-07 18:10:32.761914] 2024-08-07T18:10:32.7627981Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_dtensor_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:10:32.762472] 2024-08-07T18:11:17.2712024Z 2024-08-07T18:11:17.2713600Z distributed/fsdp/test_fsdp_dtensor_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_dtensor_state_dict_1.1_b945de4c43c8d9e6_.log 2024-08-07T18:11:17.2727388Z Running 15 items in this shard: test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_False_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_False_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_True_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_True_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_False_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_False_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_True_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_True_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_False_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_False_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_True_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_True_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_fsdp_init_with_device_mesh_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_fsdp_init_with_device_mesh_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_raises_warning_or_errors 2024-08-07T18:11:17.2739753Z 2024-08-07T18:11:17.2740085Z Running distributed/_composable/test_compose 1/1 ... [2024-08-07 18:11:17.271780] 2024-08-07T18:11:17.2741449Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_compose.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:11:17.272247] 2024-08-07T18:11:43.4335739Z 2024-08-07T18:11:43.4338611Z distributed/_composable/test_compose 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_compose_1.1_0fa164432eedc3ab_.log 2024-08-07T18:11:43.4344603Z Running 9 items in this shard: test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fsdp_submodules_non_reentrant, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fsdp_submodules_with_param, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fsdp_submodules_with_param_no_shard, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fully_shard_cast_forward_inputs, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_composable_fsdp_replicate, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_fully_shard_replicate_composability, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_fully_shard_replicate_correct_replicate_params, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_state_dict_fsdp_submodules, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_wrap_same_submodule 2024-08-07T18:11:43.4349785Z 2024-08-07T18:11:43.4350169Z Running distributed/fsdp/test_distributed_checkpoint 1/1 ... [2024-08-07 18:11:43.433974] 2024-08-07T18:11:43.4351676Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_distributed_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:11:43.434421] 2024-08-07T18:11:46.1585921Z 2024-08-07T18:11:46.1587325Z distributed/fsdp/test_distributed_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_distributed_checkpoint_1.1_18e3fe2b676bf48b_.log 2024-08-07T18:11:46.1589459Z Running 2 items in this shard: test/distributed/fsdp/test_distributed_checkpoint.py::TestDistributedCheckpoint::test_distributed_checkpoint_state_dict_type0, test/distributed/fsdp/test_distributed_checkpoint.py::TestDistributedCheckpoint::test_distributed_checkpoint_state_dict_type1 2024-08-07T18:11:46.1590839Z 2024-08-07T18:11:46.1593050Z Running distributed/fsdp/test_fsdp_clip_grad_norm 1/1 ... [2024-08-07 18:11:46.159048] 2024-08-07T18:11:46.1596764Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_clip_grad_norm.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:11:46.159467] 2024-08-07T18:12:00.4063649Z 2024-08-07T18:12:00.4065424Z distributed/fsdp/test_fsdp_clip_grad_norm 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_clip_grad_norm_1.1_e2379f5948c911f5_.log 2024-08-07T18:12:00.4070050Z Running 4 items in this shard: test/distributed/fsdp/test_fsdp_clip_grad_norm.py::TestClipGradNorm::test_ddp_parity, test/distributed/fsdp/test_fsdp_clip_grad_norm.py::TestClipGradNorm::test_low_precision_grads, test/distributed/fsdp/test_fsdp_clip_grad_norm.py::TestClipGradNorm::test_no_gradients, test/distributed/fsdp/test_fsdp_clip_grad_norm.py::TestClipGradNorm::test_non_root 2024-08-07T18:12:00.4073002Z 2024-08-07T18:12:00.4073640Z Running distributed/checkpoint/test_checkpoint 1/1 ... [2024-08-07 18:12:00.407151] 2024-08-07T18:12:00.4080261Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:12:00.407684] 2024-08-07T18:12:08.8438505Z 2024-08-07T18:12:08.8440345Z distributed/checkpoint/test_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_checkpoint_1.1_0e55c4721113d9c6_.log 2024-08-07T18:12:08.8448925Z Running 8 items in this shard: test/distributed/checkpoint/test_checkpoint.py::TestDistributedCheckpointing::test_default_metadata, test/distributed/checkpoint/test_checkpoint.py::TestDistributedCheckpointing::test_tensor_metadata_with_missing_rank_spec, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_dummy_reader_works, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_dummy_writer_works, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_load_error_handling, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_load_error_handling_no_dist, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_save_error_handling, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_save_error_handling_no_dist 2024-08-07T18:12:08.8455649Z 2024-08-07T18:12:08.8456120Z Running distributed/fsdp/test_fsdp_sharded_grad_scaler 1/1 ... [2024-08-07 18:12:08.844551] 2024-08-07T18:12:08.8458387Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_sharded_grad_scaler.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:12:08.845109] 2024-08-07T18:12:59.3674107Z 2024-08-07T18:12:59.3675462Z distributed/fsdp/test_fsdp_sharded_grad_scaler 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_sharded_grad_scaler_1.1_1c3fd8228698b117_.log 2024-08-07T18:12:59.3690511Z Running 20 items in this shard: test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardGradScaler::test_grad_scaling, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardGradScaler::test_inf_gradients_skip_optim_step, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardGradScaler::test_scaling_unscaling_sparse, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_none_mixed_precision_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_none_mixed_precision_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_none_none_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_none_none_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_shard_grad_op_mixed_precision_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_shard_grad_op_mixed_precision_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_shard_grad_op_none_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_shard_grad_op_none_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_none_mixed_precision_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_none_mixed_precision_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_none_none_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_none_none_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_shard_grad_op_mixed_precision_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_shard_grad_op_mixed_precision_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_shard_grad_op_none_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_shard_grad_op_none_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_sharded_grad_scaler_found_inf 2024-08-07T18:12:59.3704461Z 2024-08-07T18:12:59.3704831Z Running distributed/_composable/fsdp/test_fully_shard_state_dict 1/1 ... [2024-08-07 18:12:59.367971] 2024-08-07T18:12:59.3706526Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:12:59.368384] 2024-08-07T18:13:02.2426841Z 2024-08-07T18:13:02.2428244Z distributed/_composable/fsdp/test_fully_shard_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_state_dict_1.1_e727a6fc39ff7893_.log 2024-08-07T18:13:02.2431056Z Running 3 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_dp_state_dict_cpu_offload, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_dp_state_dict_save_load, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiThread::test_rank0_offload_full_state_dict 2024-08-07T18:13:02.2433314Z 2024-08-07T18:13:02.2433567Z Running distributed/_tensor/test_utils 1/1 ... [2024-08-07 18:13:02.243115] 2024-08-07T18:13:02.2438133Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:13:02.243583] 2024-08-07T18:13:25.7182399Z 2024-08-07T18:13:25.7183517Z distributed/_tensor/test_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_utils_1.1_5f293d0ca2099ff9_.log 2024-08-07T18:13:25.7186682Z Running 5 items in this shard: test/distributed/_tensor/test_utils.py::UtilTest::test_compute_local_shape_2d_uneven, test/distributed/_tensor/test_utils.py::UtilTest::test_compute_local_shape_and_global_offset_1D, test/distributed/_tensor/test_utils.py::UtilTest::test_compute_local_shape_and_global_offset_2D, test/distributed/_tensor/test_utils.py::Test2DStridedLocalShard::test_fsdp1_tp_2d_dtensor_local_shards_and_offsets, test/distributed/_tensor/test_utils.py::Test2DStridedLocalShard::test_fsdp2_tp_2d_dtensor_local_shards_and_offsets 2024-08-07T18:13:25.7189114Z 2024-08-07T18:13:25.7192053Z Running distributed/fsdp/test_fsdp_pure_fp16 1/1 ... [2024-08-07 18:13:25.719008] 2024-08-07T18:13:25.7196926Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_pure_fp16.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:13:25.719470] 2024-08-07T18:13:34.3056857Z 2024-08-07T18:13:34.3058183Z distributed/fsdp/test_fsdp_pure_fp16 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_pure_fp16_1.1_55909142b5732c67_.log 2024-08-07T18:13:34.3059850Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_pure_fp16.py::TestPureFP16::test_fp16_dtypes, test/distributed/fsdp/test_fsdp_pure_fp16.py::TestPureFP16::test_pure_fp16_training 2024-08-07T18:13:34.3060909Z 2024-08-07T18:13:34.3061308Z Running distributed/_composable/test_composability/test_2d_composability 1/1 ... [2024-08-07 18:13:34.305880] 2024-08-07T18:13:34.3065965Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_composability/test_2d_composability.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:13:34.306345] 2024-08-07T18:13:37.0310833Z 2024-08-07T18:13:37.0312437Z distributed/_composable/test_composability/test_2d_composability 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_composability.test_2d_composability_1.1_0f881e2356d6427c_.log 2024-08-07T18:13:37.0315588Z Running 3 items in this shard: test/distributed/_composable/test_composability/test_2d_composability.py::TestFullyShard2DTraining::test_raise_not_implemented_state_dict_if_2d, test/distributed/_composable/test_composability/test_2d_composability.py::TestFullyShard2DTraining::test_tp_with_fsdp_offloading, test/distributed/_composable/test_composability/test_2d_composability.py::TestFullyShard2DTraining::test_train_parity_2d_mlp 2024-08-07T18:13:37.0317798Z 2024-08-07T18:13:37.0318088Z Running distributed/fsdp/test_fsdp_overlap 1/1 ... [2024-08-07 18:13:37.031594] 2024-08-07T18:13:37.0322952Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_overlap.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:13:37.032054] 2024-08-07T18:13:44.8133226Z 2024-08-07T18:13:44.8134254Z distributed/fsdp/test_fsdp_overlap 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_overlap_1.1_3fa97357064c368b_.log 2024-08-07T18:13:44.8136162Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_overlap.py::TestForwardOverlapWorldSizeOne::test_forward_overlap, test/distributed/fsdp/test_fsdp_overlap.py::TestForwardOverlapWorldSizeTwo::test_forward_overlap 2024-08-07T18:13:44.8137278Z 2024-08-07T18:13:44.8139104Z Running distributed/fsdp/test_fsdp_unshard_params 1/1 ... [2024-08-07 18:13:44.813743] 2024-08-07T18:13:44.8144231Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_unshard_params.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:13:44.814175] 2024-08-07T18:14:26.1956977Z 2024-08-07T18:14:26.1958102Z distributed/fsdp/test_fsdp_unshard_params 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_unshard_params_1.1_568ae831facfb107_.log 2024-08-07T18:14:26.1966668Z Running 15 items in this shard: test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_named_parameters_and_buffers, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_param_data, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_recurse, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_respects_reshard, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_writeback, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_singleton_param_writeback, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_submodule, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_with_grads_core, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_with_grads_none_grads, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsNoShard::test_unshard_params_param_data_no_shard, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsNoShard::test_unshard_params_writeback_no_shard, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_offload_to_cpu_no_shard_raises, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_rank0_only_with_writeback_raises, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_unshard_params_from_backward_raises, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_unshard_params_from_forward_raises 2024-08-07T18:14:26.1974459Z 2024-08-07T18:14:26.1974836Z Running distributed/_composable/fsdp/test_fully_shard_mixed_precision 1/1 ... [2024-08-07 18:14:26.196155] 2024-08-07T18:14:26.1976467Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_mixed_precision.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:14:26.196595] 2024-08-07T18:14:29.0208987Z 2024-08-07T18:14:29.0210429Z distributed/_composable/fsdp/test_fully_shard_mixed_precision 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_mixed_precision_1.1_0062007bfc625135_.log 2024-08-07T18:14:29.0216102Z Running 7 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionTraining::test_compute_dtype, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionTraining::test_grad_acc_with_reduce_dtype, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionTraining::test_reduce_dtype, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_float16_on_one_submodule, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_norm_modules_bf16, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_norm_modules_fp16, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_submodules_with_external_inputs 2024-08-07T18:14:29.0220728Z 2024-08-07T18:14:29.0221091Z Running distributed/checkpoint/test_file_system_checkpoint 1/1 ... [2024-08-07 18:14:29.021284] 2024-08-07T18:14:29.0222642Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_file_system_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:14:29.021702] 2024-08-07T18:14:31.7958054Z 2024-08-07T18:14:31.7960058Z distributed/checkpoint/test_file_system_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_file_system_checkpoint_1.1_777de1a626d5c628_.log 2024-08-07T18:14:31.7968588Z Running 6 items in this shard: test/distributed/checkpoint/test_file_system_checkpoint.py::TestDistributedStateDictSaveLoad::test_read_write_only_tensor, test/distributed/checkpoint/test_file_system_checkpoint.py::TestDistributedStateDictSaveLoadWithSharedTensor::test_read_write_shard_tensor, test/distributed/checkpoint/test_file_system_checkpoint.py::TestDistributedReshardOnLoad::test_load_rowwise_to_colwise, test/distributed/checkpoint/test_file_system_checkpoint.py::TestDistributedReshardOnLoad::test_load_with_different_shard_plan, test/distributed/checkpoint/test_file_system_checkpoint.py::TestDistributedReshardOnLoad::test_save_load_bytes, test/distributed/checkpoint/test_file_system_checkpoint.py::TestDistributedReshardOnLoad::test_switch_between_sharded_tensor_to_tensor 2024-08-07T18:14:31.7974929Z 2024-08-07T18:14:31.7975460Z Running distributed/fsdp/test_fsdp_mixed_precision 1/1 ... [2024-08-07 18:14:31.796611] 2024-08-07T18:14:31.7977367Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_mixed_precision.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:14:31.797165] 2024-08-07T18:16:59.0365597Z 2024-08-07T18:16:59.0367012Z distributed/fsdp/test_fsdp_mixed_precision 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_mixed_precision_1.1_2015c497cb9d0f55_.log 2024-08-07T18:16:59.0407575Z Running 57 items in this shard: test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_eval_root_cast_inputs, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_full_precision_in_eval, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_full_precision_in_eval_buffers, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_full_precision_in_eval_comm, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_grads_reduced_precision, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_input_grads_with_param_mixed_precision, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_false_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_false_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_false_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_false_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_true_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_true_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_true_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_true_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_false_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_false_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_false_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_false_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_true_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_true_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_true_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_true_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_false_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_false_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_false_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_false_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_true_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_true_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_true_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_true_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_false_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_false_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_false_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_false_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_true_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_true_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_true_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_true_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_no_reshard_after_forward, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_resnet, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_batchnorm_convert_sync_bn_False, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_batchnorm_convert_sync_bn_True, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_embedding_default, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_embedding_only_params_and_bufs, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_embedding_params_and_reduce_diff, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_embedding_reduce, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionUnsharded::test_grads_reduced_precision, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionUnsharded::test_mixed_precision_e2e_full_shard, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionUnsharded::test_mixed_precision_no_reshard_after_forward, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionIgnoredModules::test_mixed_precision_with_ignored_module, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_float16_on_one_submodule, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_float16_on_one_submodule_skip_inputs, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_float16_on_one_submodule_skip_inputs_error, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_submodules_with_different_precisions, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_submodules_with_different_precisions_error, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_submodules_with_external_inputs, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPTrainEval::test_train_ema_eval_flow 2024-08-07T18:16:59.0443936Z 2024-08-07T18:16:59.0444219Z Running distributed/nn/jit/test_instantiator 1/1 ... [2024-08-07 18:16:59.037256] 2024-08-07T18:16:59.0445612Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/nn/jit/test_instantiator.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:16:59.037700] 2024-08-07T18:17:01.8118137Z 2024-08-07T18:17:01.8119302Z distributed/nn/jit/test_instantiator 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.nn.jit.test_instantiator_1.1_4141844af58a19f9_.log 2024-08-07T18:17:01.8121851Z Running 3 items in this shard: test/distributed/nn/jit/test_instantiator.py::TestInstantiator::test_get_arg_return_types_from_interface, test/distributed/nn/jit/test_instantiator.py::TestInstantiator::test_instantiate_non_scripted_remote_module_template, test/distributed/nn/jit/test_instantiator.py::TestInstantiator::test_instantiate_scripted_remote_module_template 2024-08-07T18:17:01.8123821Z 2024-08-07T18:17:01.8124115Z Running distributed/fsdp/test_fsdp_state_dict 1/1 ... [2024-08-07 18:17:01.812220] 2024-08-07T18:17:01.8128493Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:17:01.812632] 2024-08-07T18:17:05.1374088Z 2024-08-07T18:17:05.1375760Z distributed/fsdp/test_fsdp_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_state_dict_1.1_92b7942f53dabc59_.log 2024-08-07T18:17:05.1546272Z Running 179 items in this shard: test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_keys_state_dict_type_local_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_keys_state_dict_type_sharded_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_keys_state_dict_type_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_full_state_dict_missing_unexpected_keys_cleaned, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_local_state_dict_with_empty_ranks, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_False_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_False_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_True_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_True_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_False_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_False_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_True_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_True_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_False_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_False_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_True_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_True_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_sharded_load_multi_backend_pg, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_shared_module_and_shared_parameter, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_False_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_False_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_True_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_True_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_False_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_False_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_True_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_True_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_rank0_offload_save_load_flow_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_rank0_offload_save_load_flow_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_save_load_flow_state_dict_type_local_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_save_load_flow_state_dict_type_sharded_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_save_load_flow_state_dict_type_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_skip_module_state_dict_type_local_state_dict_double_nest_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_skip_module_state_dict_type_sharded_state_dict_double_nest_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_skip_module_state_dict_type_state_dict_double_nest_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_type, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_sharded_state_dict_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_sharded_state_dict_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_state_dict_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_state_dict_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_shared_parameters_state_dict_type_local_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_shared_parameters_state_dict_type_sharded_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_shared_parameters_state_dict_type_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_torch_save_load, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_world_size_one, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_wrong_state_dict_config, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict4GPUs::test_local_state_dict_reshard 2024-08-07T18:17:05.1686565Z 2024-08-07T18:17:05.1686931Z Running distributed/_composable/test_replicate_with_compiler 1/1 ... [2024-08-07 18:17:05.139057] 2024-08-07T18:17:05.1688379Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_replicate_with_compiler.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:17:05.139621] 2024-08-07T18:17:07.9149744Z 2024-08-07T18:17:07.9150991Z distributed/_composable/test_replicate_with_compiler 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_replicate_with_compiler_1.1_e527452dabf21da8_.log 2024-08-07T18:17:07.9156547Z Running 10 items in this shard: test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_bucketing_coalesced_op, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_bucketing_concat_op, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_backward_only, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_bf16, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_cpu, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_cpu_no_sync, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_fp16, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_gpu, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_gpu_ac, test/distributed/_composable/test_replicate_with_compiler.py::DDP_TP_Test::test_ddp_tp 2024-08-07T18:17:07.9161274Z 2024-08-07T18:17:07.9161691Z Running distributed/_composable/fully_shard/test_fully_shard_model_checkpoint 1/1 ... [2024-08-07 18:17:07.915414] 2024-08-07T18:17:07.9163618Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_model_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:17:07.915849] 2024-08-07T18:17:20.9048382Z 2024-08-07T18:17:20.9050691Z distributed/_composable/fully_shard/test_fully_shard_model_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_model_checkpoint_1.1_1a80d15c7b26abf7_.log 2024-08-07T18:17:20.9057475Z Running 4 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_model_checkpoint.py::TestModelCheckpointing::test_full_state_dict_save_load_mixed_sharding, test/distributed/_composable/fully_shard/test_fully_shard_model_checkpoint.py::TestModelCheckpointing::test_state_dict_save_load_flow, test/distributed/_composable/fully_shard/test_fully_shard_model_checkpoint.py::TestModelCheckpointing::test_state_dict_save_load_root_fully_shard, test/distributed/_composable/fully_shard/test_fully_shard_model_checkpoint.py::TestModelCheckpointing::test_state_dict_save_load_submodule_fully_shard 2024-08-07T18:17:20.9062038Z 2024-08-07T18:17:20.9062596Z Running distributed/fsdp/test_fsdp_use_orig_params 1/1 ... [2024-08-07 18:17:20.905567] 2024-08-07T18:17:20.9065199Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_use_orig_params.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:17:20.906134] 2024-08-07T18:18:18.8190476Z 2024-08-07T18:18:18.8192586Z distributed/fsdp/test_fsdp_use_orig_params 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_use_orig_params_1.1_c954796d90552525_.log 2024-08-07T18:18:18.8211199Z Running 25 items in this shard: test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_cpu_offload_sharding_strategy_str_full_shard, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_cpu_offload_sharding_strategy_str_no_shard, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_cpu_offload_sharding_strategy_str_shard_grad_op, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_sharding_strategy_str_full_shard, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_sharding_strategy_str_no_shard, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_sharding_strategy_str_shard_grad_op, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_trainability, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_fsdp_compile, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_multiple_optimizers, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsUnshardReshard::test_multiple_forward_offload_params_False, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsUnshardReshard::test_multiple_forward_offload_params_True, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsUnshardReshard::test_summon_between_two_forwards_offload_params_False, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsUnshardReshard::test_summon_between_two_forwards_offload_params_True, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsParamAccess::test_access_params_after_forward, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsWriteback::test_grad_writeback, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsWriteback::test_no_reshard_and_mixed_precision, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsWriteback::test_param_writeback, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsWriteback::test_writeback_between_fwd_and_bwd_for_no_reshard_raises, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsWriteback::test_writeback_shape_mismatch, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsFQNs::test_named_parameters_in_forward, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsNoSync::test_no_sync_correctness, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsNoSync::test_no_sync_mixed_precision, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsInit::test_non_uniform_requires_grad, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestMultiTensorApply::test_multi_tensor_apply_size0_tensors_cpu, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestMultiTensorApply::test_multi_tensor_apply_size0_tensors_cuda 2024-08-07T18:18:18.8225526Z 2024-08-07T18:18:18.8225838Z Running distributed/fsdp/test_fsdp_flatten_params 1/1 ... [2024-08-07 18:18:18.819611] 2024-08-07T18:18:18.8227186Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_flatten_params.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:18:18.820064] 2024-08-07T18:18:48.3809093Z 2024-08-07T18:18:48.3810886Z distributed/fsdp/test_fsdp_flatten_params 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_flatten_params_1.1_b661b8924f4f352a_.log 2024-08-07T18:18:48.3817204Z Running 11 items in this shard: test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_empty_module, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_flat_param_shard_metadata_aligned_full_precision, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_flat_param_shard_metadata_aligned_mixed_precision, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_flat_param_shard_metadata_unaligned, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_flatten_nothing, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_numel_with_shared_params, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_numel_without_shared_params, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_output_with_shared_params, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_output_without_shared_params, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_partial_flattening, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_pnorm_after_step_with_shared_params 2024-08-07T18:18:48.3822842Z 2024-08-07T18:18:48.3823209Z Running distributed/_composable/fully_shard/test_fully_shard_init 1/1 ... [2024-08-07 18:18:48.381378] 2024-08-07T18:18:48.3824688Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_init.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:18:48.381795] 2024-08-07T18:19:06.7797814Z 2024-08-07T18:19:06.7800239Z distributed/_composable/fully_shard/test_fully_shard_init 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_init_1.1_d31cbb66502b1dc2_.log 2024-08-07T18:19:06.7807432Z Running 6 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_device_id, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_manual_fully_shard, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_materialize_meta_module, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_nested_fully_shard_shared_state, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_policy, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_sync_module_states 2024-08-07T18:19:06.7812511Z 2024-08-07T18:19:06.7812990Z Running distributed/test_pg_wrapper 1/1 ... [2024-08-07 18:19:06.780496] 2024-08-07T18:19:06.7815022Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_pg_wrapper.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:19:06.781041] 2024-08-07T18:20:23.0158051Z 2024-08-07T18:20:23.0159524Z distributed/test_pg_wrapper 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_pg_wrapper_1.1_d1b32ffd407d7859_.log 2024-08-07T18:20:23.0171899Z Running 17 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_coalescing_manager_debug_mode_detail, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_hang, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_detail, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_off, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_debug_level_detail_no_gloo, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_new_group_no_gloo, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_hang, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode_off, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_debug_mode 2024-08-07T18:20:23.0180649Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_coalescing_manager_debug_mode_detail 2024-08-07T18:20:23.0181721Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_hang 2024-08-07T18:20:23.0182811Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_detail 2024-08-07T18:20:23.0184021Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_off 2024-08-07T18:20:23.0185119Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch 2024-08-07T18:20:23.0186209Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch_debug_mode 2024-08-07T18:20:23.0187307Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_debug_level_detail_no_gloo 2024-08-07T18:20:23.0188309Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_new_group_no_gloo 2024-08-07T18:20:23.0189274Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_hang 2024-08-07T18:20:23.0190393Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda 2024-08-07T18:20:23.0191545Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda_debug_mode 2024-08-07T18:20:23.0192794Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode 2024-08-07T18:20:23.0193965Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode_off 2024-08-07T18:20:23.0195073Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch 2024-08-07T18:20:23.0196125Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda 2024-08-07T18:20:23.0197242Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda_debug_mode 2024-08-07T18:20:23.0198400Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_debug_mode 2024-08-07T18:20:23.0199022Z 2024-08-07T18:20:23.0199442Z Running distributed/test_multi_threaded_pg 1/1 ... [2024-08-07 18:20:23.016615] 2024-08-07T18:20:23.0200728Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_multi_threaded_pg.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:20:23.017050] 2024-08-07T18:20:25.9416885Z 2024-08-07T18:20:25.9418079Z distributed/test_multi_threaded_pg 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_multi_threaded_pg_1.1_ecead697f38ef48d_.log 2024-08-07T18:20:25.9428927Z Running 22 items in this shard: test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_all_to_all_single_list, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_all_to_all_single_none, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_all_to_all_single_tensor, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_broadcast_object_list, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_collective_error_on_rank_non_zero, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_collective_error_on_rank_non_zero_all, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_collective_error_on_rank_zero, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_skip, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_reduce, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_reduce_coalesced, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_reduce_ops, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_to_all, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_allgather, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_assert_equal_on_rank, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_broadcast, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_broadcast_object_list, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_bwd_sees_fwd_pg, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_gather, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_reduce_scatter, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_scatter, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_subpg, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_using_pg_from_another_thread 2024-08-07T18:20:25.9438530Z 2024-08-07T18:20:25.9438805Z Running distributed/fsdp/test_fsdp_memory 1/1 ... [2024-08-07 18:20:25.942217] 2024-08-07T18:20:25.9440080Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_memory.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:20:25.942626] 2024-08-07T18:20:33.7757940Z 2024-08-07T18:20:33.7759003Z distributed/fsdp/test_fsdp_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_memory_1.1_07041a5640bb86f6_.log 2024-08-07T18:20:33.7760612Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_memory.py::TestFSDPMemory::test_fsdp_memory_ckpt_ckpt, test/distributed/fsdp/test_fsdp_memory.py::TestFSDPMemory::test_fsdp_memory_ckpt_no_ckpt 2024-08-07T18:20:33.7761499Z 2024-08-07T18:20:33.7764101Z Running distributed/fsdp/test_fsdp_grad_acc 1/1 ... [2024-08-07 18:20:33.776204] 2024-08-07T18:20:33.7769725Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_grad_acc.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:20:33.776705] 2024-08-07T18:20:51.8810648Z 2024-08-07T18:20:51.8812099Z distributed/fsdp/test_fsdp_grad_acc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_grad_acc_1.1_4c927a02eb799cf0_.log 2024-08-07T18:20:51.8815524Z Running 6 items in this shard: test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_configs0_use_orig_params_False, test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_configs0_use_orig_params_True, test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_configs1_use_orig_params_False, test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_configs1_use_orig_params_True, test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_cpu_offload_use_orig_params_False, test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_cpu_offload_use_orig_params_True 2024-08-07T18:20:51.8818561Z 2024-08-07T18:20:51.8818865Z Running distributed/test_c10d_object_collectives 1/1 ... [2024-08-07 18:20:51.881585] 2024-08-07T18:20:51.8822821Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_object_collectives.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:20:51.882033] 2024-08-07T18:21:18.3091099Z 2024-08-07T18:21:18.3092295Z distributed/test_c10d_object_collectives 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_object_collectives_1.1_9cca1cb0db85cf13_.log 2024-08-07T18:21:18.3097542Z Running 9 items in this shard: test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_all_gather_object, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_broadcast_object_list, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_gather_object, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_scatter_object_list, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_send_recv_object_list, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_subpg_all_gather_object, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_subpg_broadcast_object, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_subpg_gather_object, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_subpg_scatter_object 2024-08-07T18:21:18.3101935Z 2024-08-07T18:21:18.3102302Z Running distributed/_composable/fsdp/test_fully_shard_extensions 1/1 ... [2024-08-07 18:21:18.309535] 2024-08-07T18:21:18.3103905Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_extensions.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:21:18.309959] 2024-08-07T18:21:24.2913632Z 2024-08-07T18:21:24.2914887Z distributed/_composable/fsdp/test_fully_shard_extensions 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_extensions_1.1_50fb5dc53a18c6b8_.log 2024-08-07T18:21:24.2918019Z Running 3 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_extensions.py::TestFullyShardAllGatherExtensionsMultiProcess::test_all_gather_extensions_train_parity, test/distributed/_composable/fsdp/test_fully_shard_extensions.py::TestFullyShardAllGatherExtensionsMultiThread::test_all_gather_extensions_end_to_end, test/distributed/_composable/fsdp/test_fully_shard_extensions.py::TestFullyShardAllGatherExtensionsMultiThread::test_all_gather_extensions_monkey_patch 2024-08-07T18:21:24.2920488Z 2024-08-07T18:21:24.2921206Z Running distributed/fsdp/test_hsdp_dtensor_state_dict 1/1 ... [2024-08-07 18:21:24.291840] 2024-08-07T18:21:24.2925455Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_hsdp_dtensor_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:21:24.292306] 2024-08-07T18:21:49.1115786Z 2024-08-07T18:21:49.1116961Z distributed/fsdp/test_hsdp_dtensor_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_hsdp_dtensor_state_dict_1.1_14ac140802f74c7f_.log 2024-08-07T18:21:49.1123064Z Running 8 items in this shard: test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_False, test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_True, test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_False, test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_True, test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_False, test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_True, test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_hsdp_init_with_device_mesh, test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_root_module_is_not_FSDP 2024-08-07T18:21:49.1128508Z 2024-08-07T18:21:49.1128902Z Running distributed/_composable/fully_shard/test_fully_shard_runtime 1/1 ... [2024-08-07 18:21:49.111988] 2024-08-07T18:21:49.1130403Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_runtime.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:21:49.112403] 2024-08-07T18:21:51.8865264Z 2024-08-07T18:21:51.8867405Z distributed/_composable/fully_shard/test_fully_shard_runtime 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_runtime_1.1_8571869f1c2efdb0_.log 2024-08-07T18:21:51.8871174Z Running 2 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_runtime.py::TestRuntime::test_training, test/distributed/_composable/fully_shard/test_fully_shard_runtime.py::TestRuntime::test_unshard_reshard_order 2024-08-07T18:21:51.8873035Z 2024-08-07T18:21:51.8874915Z Running distributed/algorithms/ddp_comm_hooks/test_ddp_hooks 1/1 ... [2024-08-07 18:21:51.887264] 2024-08-07T18:21:51.8881595Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:21:51.887809] 2024-08-07T18:21:54.4116259Z 2024-08-07T18:21:54.4118453Z distributed/algorithms/ddp_comm_hooks/test_ddp_hooks 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.algorithms.ddp_comm_hooks.test_ddp_hooks_1.1_2f6a3c55d4185a69_.log 2024-08-07T18:21:54.4127344Z Running 6 items in this shard: test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_ddp_comm_hook_allreduce_hook, test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_ddp_comm_hook_fp16compress_hook, test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_ddp_comm_hook_noop_hook, test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_ddp_comm_hook_quantize_per_channel_hook, test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_ddp_comm_hook_quantize_per_tensor_hook, test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_is_last_hook 2024-08-07T18:21:54.4134010Z 2024-08-07T18:21:54.4134562Z Running distributed/checkpoint/test_file_system_checkpoint_cpu 1/1 ... [2024-08-07 18:21:54.412389] 2024-08-07T18:21:54.4136626Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_file_system_checkpoint_cpu.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:21:54.412957] 2024-08-07T18:22:23.6775977Z 2024-08-07T18:22:23.6777926Z distributed/checkpoint/test_file_system_checkpoint_cpu 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_file_system_checkpoint_cpu_1.1_2a88ee43e3b44bb8_.log 2024-08-07T18:22:23.6789662Z Running 12 items in this shard: test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoad::test_read_write_only_tensor_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoad::test_read_write_only_tensor_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoadWithSharedTensor::test_read_write_shard_tensor_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoadWithSharedTensor::test_read_write_shard_tensor_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_rowwise_to_colwise_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_rowwise_to_colwise_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_with_different_shard_plan_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_with_different_shard_plan_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_save_load_bytes_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_save_load_bytes_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_switch_between_sharded_tensor_to_tensor_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_switch_between_sharded_tensor_to_tensor_thread_count_2 2024-08-07T18:22:23.6797810Z 2024-08-07T18:22:23.6798080Z Running distributed/fsdp/test_fsdp_apply 1/1 ... [2024-08-07 18:22:23.678347] 2024-08-07T18:22:23.6799367Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_apply.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:22:23.678829] 2024-08-07T18:22:34.3657023Z 2024-08-07T18:22:34.3658210Z distributed/fsdp/test_fsdp_apply 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_apply_1.1_4a35827969bc828b_.log 2024-08-07T18:22:34.3660605Z Running 3 items in this shard: test/distributed/fsdp/test_fsdp_apply.py::TestApply::test_apply_in_summon_raises_error, test/distributed/fsdp/test_fsdp_apply.py::TestApply::test_nested_module_apply, test/distributed/fsdp/test_fsdp_apply.py::TestApply::test_transformer_module_apply 2024-08-07T18:22:34.3664058Z 2024-08-07T18:22:34.3664347Z Running distributed/fsdp/test_fsdp_comm_hooks 1/1 ... [2024-08-07 18:22:34.366147] 2024-08-07T18:22:34.3668835Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_comm_hooks.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:22:34.366627] 2024-08-07T18:23:21.9320274Z 2024-08-07T18:23:21.9321618Z distributed/fsdp/test_fsdp_comm_hooks 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_comm_hooks_1.1_fbcd39eaa3831936_.log 2024-08-07T18:23:21.9340084Z Running 28 items in this shard: test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_False_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_True_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_behavior_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_behavior_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_behavior_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_False_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_True_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_False_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_True_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_hybrid_strategy, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_non_root_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_non_root_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_non_root_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_submodules_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_submodules_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_submodules_sharding_strategy2 2024-08-07T18:23:21.9356310Z 2024-08-07T18:23:21.9356550Z Running distributed/fsdp/test_wrap 1/1 ... [2024-08-07 18:23:21.932593] 2024-08-07T18:23:21.9357780Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_wrap.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:23:21.933052] 2024-08-07T18:24:36.1030990Z 2024-08-07T18:24:36.1033052Z distributed/fsdp/test_wrap 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_wrap_1.1_203ddc14df198fd7_.log 2024-08-07T18:24:36.1072067Z Running 51 items in this shard: test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_bn_always_wrapped_individually, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_False_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_False_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_True_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_True_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_False_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_False_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_True_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_True_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_False_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_False_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_True_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_True_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_False_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_False_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_True_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_True_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_False_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_False_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_True_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_True_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_wrap_batchnorm_individually_use_or_policy_False, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_wrap_batchnorm_individually_use_or_policy_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_always_wrap, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_always_wrap_with_ignored_modules_wrap_method0, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_always_wrap_with_ignored_modules_wrap_method1, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_api, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_exclude_wrap, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_exclude_wrap_include_children, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_force_leaf, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_force_leaf_custom, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode0_cpu_offload0_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode0_cpu_offload0_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode0_cpu_offload1_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode0_cpu_offload1_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode1_cpu_offload0_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode1_cpu_offload0_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode1_cpu_offload1_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode1_cpu_offload1_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_with_ignored_modules_wrap_method0, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_with_ignored_modules_wrap_method1, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_custom_policy, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_frozen_params, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_module_wrap_policy, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_module_wrap_policy_callable, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_transformer_auto_wrap_policy, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_disabled_outside_context, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_override_defaults, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_wrap_method0, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_wrap_method1, test/distributed/fsdp/test_wrap.py::TestWrapUtils::test_validate_frozen_params 2024-08-07T18:24:36.1103205Z 2024-08-07T18:24:36.1103461Z Running distributed/test_c10d_ucc 1/1 ... [2024-08-07 18:24:36.103971] 2024-08-07T18:24:36.1104750Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_ucc.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:24:36.104527] 2024-08-07T18:24:37.4819788Z 2024-08-07T18:24:37.4820783Z distributed/test_c10d_ucc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_ucc_1.1_e41a54995ed06ff9_.log 2024-08-07T18:24:37.4821489Z 2024-08-07T18:24:37.4825664Z Running distributed/fsdp/test_fsdp_traversal 1/1 ... [2024-08-07 18:24:37.482399] 2024-08-07T18:24:37.4830112Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_traversal.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:24:37.482799] 2024-08-07T18:24:42.8619939Z 2024-08-07T18:24:42.8621761Z distributed/fsdp/test_fsdp_traversal 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_traversal_1.1_45902b8b9a587f9d_.log 2024-08-07T18:24:42.8624402Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_traversal.py::TestTraversal::test_fsdp_modules 2024-08-07T18:24:42.8625478Z 2024-08-07T18:24:42.8629242Z Running distributed/rpc/cuda/test_tensorpipe_agent 1/1 ... [2024-08-07 18:24:42.862681] 2024-08-07T18:24:42.8633912Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/cuda/test_tensorpipe_agent.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:24:42.863160] 2024-08-07T18:34:56.8222400Z 2024-08-07T18:34:56.8223534Z distributed/rpc/cuda/test_tensorpipe_agent 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.cuda.test_tensorpipe_agent_1.1_c0c9eef275b77b4f_.log 2024-08-07T18:34:56.8324719Z Running 99 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRpcTest::test_profiler_remote_cuda, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_simple, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation_gpu_root, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device_script, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_invalid_devices, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_valid_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_nested_with_cuda_future, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_with_cuda_future, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_callback_changes_devices, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_sparse_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_sparse_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_sparse_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_int, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_str, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_not_cuda, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_modify_tensor_inplace, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_replace_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_value_on_bad_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_multi, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested_multi, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default_to_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_5, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_6, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_7, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_8, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_5, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_6, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_7, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_8, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default_to_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_in_options, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_local_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_remote_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_min_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_many_to_one, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_loop, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_not_timeout, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote_response, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response_loop, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu_self, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_one_to_many, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_remote, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu_self, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_wrong_worker_name, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_mismatch, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch_reverse, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization5, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_with_unpickleable_attributes, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_tensor_view_as_return_value, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_device_maps_backward_pass, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_dist_autograd_sync_streams, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_gradients_synchronizations 2024-08-07T18:34:56.8382751Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRpcTest::test_profiler_remote_cuda 2024-08-07T18:34:56.8383865Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_simple 2024-08-07T18:34:56.8385113Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation 2024-08-07T18:34:56.8386362Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation_gpu_root 2024-08-07T18:34:56.8387631Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device 2024-08-07T18:34:56.8388905Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device_script 2024-08-07T18:34:56.8390127Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_invalid_devices 2024-08-07T18:34:56.8391234Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_valid_device 2024-08-07T18:34:56.8392456Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote_gpu 2024-08-07T18:34:56.8393937Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_nested_with_cuda_future 2024-08-07T18:34:56.8395336Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_with_cuda_future 2024-08-07T18:34:56.8396709Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_callback_changes_devices 2024-08-07T18:34:56.8398146Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_sparse_tensor 2024-08-07T18:34:56.8399564Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_tensor 2024-08-07T18:34:56.8401047Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_sparse_tensor 2024-08-07T18:34:56.8402606Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_tensor 2024-08-07T18:34:56.8404297Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_sparse_tensor 2024-08-07T18:34:56.8405909Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_tensor 2024-08-07T18:34:56.8407316Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_device 2024-08-07T18:34:56.8408619Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_int 2024-08-07T18:34:56.8409921Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_str 2024-08-07T18:34:56.8411236Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_not_cuda 2024-08-07T18:34:56.8412590Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_modify_tensor_inplace 2024-08-07T18:34:56.8413928Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_replace_tensor 2024-08-07T18:34:56.8415260Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_value_on_bad_device 2024-08-07T18:34:56.8416525Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream 2024-08-07T18:34:56.8417745Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_multi 2024-08-07T18:34:56.8418980Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested 2024-08-07T18:34:56.8420265Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested_multi 2024-08-07T18:34:56.8421508Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu 2024-08-07T18:34:56.8422771Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_default 2024-08-07T18:34:56.8424106Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_non_default 2024-08-07T18:34:56.8425518Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default 2024-08-07T18:34:56.8426863Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default_to_non_default 2024-08-07T18:34:56.8428202Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_1 2024-08-07T18:34:56.8429461Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_2 2024-08-07T18:34:56.8430728Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_3 2024-08-07T18:34:56.8431999Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_4 2024-08-07T18:34:56.8433332Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_5 2024-08-07T18:34:56.8434694Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_6 2024-08-07T18:34:56.8436029Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_7 2024-08-07T18:34:56.8437310Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_8 2024-08-07T18:34:56.8438601Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_1 2024-08-07T18:34:56.8439927Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_2 2024-08-07T18:34:56.8441247Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_3 2024-08-07T18:34:56.8442569Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_4 2024-08-07T18:34:56.8443874Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_5 2024-08-07T18:34:56.8445184Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_6 2024-08-07T18:34:56.8446502Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_7 2024-08-07T18:34:56.8447816Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_8 2024-08-07T18:34:56.8449115Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default 2024-08-07T18:34:56.8450474Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default_to_default 2024-08-07T18:34:56.8451852Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_default 2024-08-07T18:34:56.8453211Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_non_default 2024-08-07T18:34:56.8454491Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_gpu 2024-08-07T18:34:56.8455742Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_in_options 2024-08-07T18:34:56.8457082Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_local_device 2024-08-07T18:34:56.8458493Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_remote_device 2024-08-07T18:34:56.8459868Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_min_device 2024-08-07T18:34:56.8461197Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_many_to_one 2024-08-07T18:34:56.8462492Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config 2024-08-07T18:34:56.8464091Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_loop 2024-08-07T18:34:56.8465544Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_not_timeout 2024-08-07T18:34:56.8466927Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote 2024-08-07T18:34:56.8468343Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote_response 2024-08-07T18:34:56.8469757Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response 2024-08-07T18:34:56.8471163Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response_loop 2024-08-07T18:34:56.8472519Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu 2024-08-07T18:34:56.8473871Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu_self 2024-08-07T18:34:56.8475166Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_one_to_many 2024-08-07T18:34:56.8476412Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_remote 2024-08-07T18:34:56.8477681Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu 2024-08-07T18:34:56.8479007Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu_self 2024-08-07T18:34:56.8480352Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_wrong_worker_name 2024-08-07T18:34:56.8481612Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_mismatch 2024-08-07T18:34:56.8482856Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch 2024-08-07T18:34:56.8484171Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch_reverse 2024-08-07T18:34:56.8485553Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization1 2024-08-07T18:34:56.8486935Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization2 2024-08-07T18:34:56.8488325Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization3 2024-08-07T18:34:56.8489713Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization4 2024-08-07T18:34:56.8491072Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization1 2024-08-07T18:34:56.8492388Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization2 2024-08-07T18:34:56.8493774Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization3 2024-08-07T18:34:56.8495154Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization4 2024-08-07T18:34:56.8496490Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization5 2024-08-07T18:34:56.8497816Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization1 2024-08-07T18:34:56.8499163Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization2 2024-08-07T18:34:56.8500503Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization3 2024-08-07T18:34:56.8501835Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization4 2024-08-07T18:34:56.8503175Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization1 2024-08-07T18:34:56.8504516Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization2 2024-08-07T18:34:56.8505963Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization3 2024-08-07T18:34:56.8507296Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization4 2024-08-07T18:34:56.8508652Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_with_unpickleable_attributes 2024-08-07T18:34:56.8509991Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_tensor_view_as_return_value 2024-08-07T18:34:56.8511323Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_device_maps_backward_pass 2024-08-07T18:34:56.8512724Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_dist_autograd_sync_streams 2024-08-07T18:34:56.8514084Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_gradients_synchronizations 2024-08-07T18:34:56.8514834Z 2024-08-07T18:34:56.8515171Z Running distributed/_shard/sharded_tensor/ops/test_binary_cmp 1/1 ... [2024-08-07 18:34:56.827002] 2024-08-07T18:34:56.8516622Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_binary_cmp.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:34:56.827562] 2024-08-07T18:34:59.3025584Z 2024-08-07T18:34:59.3028150Z distributed/_shard/sharded_tensor/ops/test_binary_cmp 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_binary_cmp_1.1_93daaeb5f57d2743_.log 2024-08-07T18:34:59.3033814Z Running 4 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_binary_cmp.py::TestShardedTensorBinaryOps::test_torch_allclose, test/distributed/_shard/sharded_tensor/ops/test_binary_cmp.py::TestShardedTensorBinaryOps::test_torch_allclose_tensor_specs, test/distributed/_shard/sharded_tensor/ops/test_binary_cmp.py::TestShardedTensorBinaryOps::test_torch_equal, test/distributed/_shard/sharded_tensor/ops/test_binary_cmp.py::TestShardedTensorBinaryOps::test_torch_equal_tensor_specs 2024-08-07T18:34:59.3038146Z 2024-08-07T18:34:59.3038739Z Running distributed/_shard/sharding_spec/test_sharding_spec 1/1 ... [2024-08-07 18:34:59.303307] 2024-08-07T18:34:59.3041807Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharding_spec/test_sharding_spec.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:34:59.303880] 2024-08-07T18:35:05.2847305Z 2024-08-07T18:35:05.2848527Z distributed/_shard/sharding_spec/test_sharding_spec 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharding_spec.test_sharding_spec_1.1_553d445d9821125d_.log 2024-08-07T18:35:05.2854888Z Running 11 items in this shard: test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_check_overlapping, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_chunked_sharding_spec, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_device_placement, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_enumerable_sharding_spec, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_get_chunk_sharding_params, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_get_chunked_dim_size, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_get_split_size, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_infer_sharding_spec_from_shards_metadata, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestCustomShardingSpec::test_custom_sharding_spec, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestCustomShardingSpec::test_custom_sharding_spec_shard_tensor, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestCustomShardingSpec::test_custom_sharding_spec_tensor_ctor 2024-08-07T18:35:05.2860866Z 2024-08-07T18:35:05.2861278Z Running distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint 1/1 ... [2024-08-07 18:35:05.285202] 2024-08-07T18:35:05.2863059Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:35:05.285634] 2024-08-07T18:35:13.4179832Z 2024-08-07T18:35:13.4181216Z distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint_1.1_8a46dd5800c05e9b_.log 2024-08-07T18:35:13.4183736Z Running 2 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint.py::TestOptimStateCheckpointing::test_optim_state_dict_save_load, test/distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint.py::TestOptimStateCheckpointing::test_optim_state_dict_submodule_fully_shard 2024-08-07T18:35:13.4185413Z 2024-08-07T18:35:13.4185704Z Running distributed/rpc/test_tensorpipe_agent 1/4 ... [2024-08-07 18:35:13.418298] 2024-08-07T18:35:13.4189705Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_tensorpipe_agent.py', '--shard-id=1', '--num-shards=4', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:35:13.418731] 2024-08-07T18:45:10.6421207Z 2024-08-07T18:45:10.6422318Z distributed/rpc/test_tensorpipe_agent 1/4 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_tensorpipe_agent_1.4_307f52cc79e6fb1e_.log 2024-08-07T18:45:10.6481534Z Running 85 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_add_done_callback, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_chained, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_chained_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_call_method_on_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_chain, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_multi, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_with_ret, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_custom_exception_throw_during_reconstruction, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_debug_info, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_nested_callback, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_int_callee, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_rref_no_fork, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_layer_nested_async_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_py_udf_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_rpc_key_names, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_udf_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_remote_rpc_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_static_method, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_register_rpc_backend_and_set_and_start_rpc_backend, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_remote_same_worker, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_partial_subset, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_return_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_forward_chain, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_class, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_non_exist, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_with_error_non_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_py_udf_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_rpc_arg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_stress_heavy_rpc_torchscript, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_exit_early_builtin, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_exit_early_script_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wrong_types, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_accumulate_grads, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_autograd_engine_error, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_tensor_dims, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_nested, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_python_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_unused_send_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_without_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_clean_context_during_backward, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_tensor_with_grad, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_error_in_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_copy_sparse_indices_extra_ref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_only_on_return_value_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_remote_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_grad_copy, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_graph_with_tensors_not_require_grad, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_rpc_complex_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_thread_local_context_id, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_all_kwargs_are_populated_by_defaults, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_wrong_return_type, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_with_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_create_local_script_class_rref_in_py, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_create_local_script_module_rref_in_py, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_kwargs_not_passed, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_module, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_return_local_script_class_rref_in_py_and_use_in_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_python_annotation, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_unexepected_kwarg_is_specified, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_user_rrefs_confirmed_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_send_remote_module_with_a_new_attribute_not_pickled_over_the_wire, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_create_remote_module_from_module_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_comparison, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_existing_rank_can_communicate_with_new_rank, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_infer_backend_from_options, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_mismatched_type_for_options, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_builtin_remote_ret_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_rpc_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_remote_arg_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_self_rpc_arg_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_send_to_rank_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_stress_heavy_rpc_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_different_dtypes_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_call_itself_sparse 2024-08-07T18:45:10.6520902Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_add_done_callback 2024-08-07T18:45:10.6521905Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy 2024-08-07T18:45:10.6522951Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy_async 2024-08-07T18:45:10.6523970Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_chained 2024-08-07T18:45:10.6525004Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_chained_remote 2024-08-07T18:45:10.6526084Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type 2024-08-07T18:45:10.6527118Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_call_method_on_rref 2024-08-07T18:45:10.6528064Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_chain 2024-08-07T18:45:10.6529010Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_multi 2024-08-07T18:45:10.6529969Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_with_ret 2024-08-07T18:45:10.6531048Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_custom_exception_throw_during_reconstruction 2024-08-07T18:45:10.6532110Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_debug_info 2024-08-07T18:45:10.6533074Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_nested_callback 2024-08-07T18:45:10.6534026Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_int_callee 2024-08-07T18:45:10.6534949Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_rref_no_fork 2024-08-07T18:45:10.6535962Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_layer_nested_async_rpc 2024-08-07T18:45:10.6536985Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_py_udf_remote 2024-08-07T18:45:10.6537961Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_rpc_key_names 2024-08-07T18:45:10.6539008Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin 2024-08-07T18:45:10.6540164Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin_single_threaded 2024-08-07T18:45:10.6541304Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_udf 2024-08-07T18:45:10.6542324Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_udf 2024-08-07T18:45:10.6543543Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_udf_single_threaded 2024-08-07T18:45:10.6544736Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_remote_rpc_single_threaded 2024-08-07T18:45:10.6545894Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin 2024-08-07T18:45:10.6546919Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_static_method 2024-08-07T18:45:10.6548044Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_register_rpc_backend_and_set_and_start_rpc_backend 2024-08-07T18:45:10.6549152Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_remote_same_worker 2024-08-07T18:45:10.6550165Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_partial_subset 2024-08-07T18:45:10.6551148Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_return_rref 2024-08-07T18:45:10.6552104Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_forward_chain 2024-08-07T18:45:10.6553125Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_class 2024-08-07T18:45:10.6554080Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_non_exist 2024-08-07T18:45:10.6555140Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_with_error_non_blocking 2024-08-07T18:45:10.6556181Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_py_udf_remote 2024-08-07T18:45:10.6557189Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_rpc_arg 2024-08-07T18:45:10.6558234Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_stress_heavy_rpc_torchscript 2024-08-07T18:45:10.6559285Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_exit_early_builtin 2024-08-07T18:45:10.6560364Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_exit_early_script_function 2024-08-07T18:45:10.6561377Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wrong_types 2024-08-07T18:45:10.6562402Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_accumulate_grads 2024-08-07T18:45:10.6563735Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_autograd_engine_error 2024-08-07T18:45:10.6564926Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_tensor_dims 2024-08-07T18:45:10.6566040Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_nested 2024-08-07T18:45:10.6567149Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_python_udf 2024-08-07T18:45:10.6568309Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_unused_send_function 2024-08-07T18:45:10.6569452Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_without_context 2024-08-07T18:45:10.6570712Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_clean_context_during_backward 2024-08-07T18:45:10.6571966Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_tensor_with_grad 2024-08-07T18:45:10.6573083Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_error_in_context 2024-08-07T18:45:10.6574218Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_copy_sparse_indices_extra_ref 2024-08-07T18:45:10.6575403Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_only_on_return_value_remote 2024-08-07T18:45:10.6576561Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_call 2024-08-07T18:45:10.6577721Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_remote_call 2024-08-07T18:45:10.6578807Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_grad_copy 2024-08-07T18:45:10.6579930Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_graph_with_tensors_not_require_grad 2024-08-07T18:45:10.6581075Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_rpc_complex_args 2024-08-07T18:45:10.6582150Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_thread_local_context_id 2024-08-07T18:45:10.6583280Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_all_kwargs_are_populated_by_defaults 2024-08-07T18:45:10.6584425Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_wrong_return_type 2024-08-07T18:45:10.6585512Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_with_exception 2024-08-07T18:45:10.6586617Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_create_local_script_class_rref_in_py 2024-08-07T18:45:10.6587769Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_create_local_script_module_rref_in_py 2024-08-07T18:45:10.6588848Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_kwargs_not_passed 2024-08-07T18:45:10.6589852Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_module 2024-08-07T18:45:10.6591016Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_return_local_script_class_rref_in_py_and_use_in_script 2024-08-07T18:45:10.6592173Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_python_annotation 2024-08-07T18:45:10.6593278Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_function 2024-08-07T18:45:10.6594349Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_unexepected_kwarg_is_specified 2024-08-07T18:45:10.6595436Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_user_rrefs_confirmed_remote 2024-08-07T18:45:10.6596482Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_async 2024-08-07T18:45:10.6597725Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_send_remote_module_with_a_new_attribute_not_pickled_over_the_wire 2024-08-07T18:45:10.6599684Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_create_remote_module_from_module_rref 2024-08-07T18:45:10.6600882Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_comparison 2024-08-07T18:45:10.6602132Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_existing_rank_can_communicate_with_new_rank 2024-08-07T18:45:10.6603469Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_infer_backend_from_options 2024-08-07T18:45:10.6604681Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_mismatched_type_for_options 2024-08-07T18:45:10.6605912Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_builtin_remote_ret_sparse 2024-08-07T18:45:10.6607104Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_rpc_sparse 2024-08-07T18:45:10.6608309Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_remote_arg_sparse 2024-08-07T18:45:10.6609626Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_self_rpc_arg_sparse 2024-08-07T18:45:10.6610845Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_send_to_rank_sparse 2024-08-07T18:45:10.6611999Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_stress_heavy_rpc_sparse 2024-08-07T18:45:10.6613275Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_different_dtypes_sparse 2024-08-07T18:45:10.6614666Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_call_itself_sparse 2024-08-07T18:45:10.6615429Z 2024-08-07T18:45:10.6615695Z Running distributed/rpc/test_tensorpipe_agent 3/4 ... [2024-08-07 18:45:10.647033] 2024-08-07T18:45:10.6617070Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_tensorpipe_agent.py', '--shard-id=3', '--num-shards=4', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:45:10.647493] 2024-08-07T18:58:32.4695450Z 2024-08-07T18:58:32.4696833Z distributed/rpc/test_tensorpipe_agent 3/4 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_tensorpipe_agent_3.4_bd612aa1aba76d1e_.log 2024-08-07T18:58:32.4779454Z Running 117 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_add, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_add_with_id, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_chained, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_fanout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_fanout_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_with_future_ctor, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_cbs_jit_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_build_rpc_profiling_key, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_builtin_remote_ret, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_in_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_none, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_with_error, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_deadlock, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_disable_gil_profiling, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_dist_init_decorator, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_duplicate_name_2, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_in_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_wait_twice, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_ignore_rref_leak, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_rpc_then_pg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_shutdown_with_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_my_parameter_server, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_non_garbage_collected_user_rref_due_to_local_circular_dependency, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nonzero, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_remote_events_profiled_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_autograd_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_builtin_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_async_rpc_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_remote_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_sync_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_constructor, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_instance_method, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_multi_async_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rpc_rref_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rref_args_user_share, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_udf_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_remote_record_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_get_future, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_non_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_slow_init, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_with_error_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rrefs_confirmed, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rrefs_confirmed_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_raise_in_body, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_with_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_with_partial_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_twice_dense, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_world_size_one, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_autograd_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_dtypes, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_invalid_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_roots, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_round_trips, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_no_grad_on_tensor, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_node_failure, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_node_failure_python_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_python_udf_error, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_script_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_verify_hooks, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backwards_nested_python_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_nested_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_tensor_no_grad, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_builtin_remote_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_remote_call_itself, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_python_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_mixed_requires_grad, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_nested_backward_accumulate_grads, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_grad_copy_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_post_hooks, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_trainer_ps, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_exception_on_constructor, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_add_done_callback, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_args_and_kwargs_contain_different_types, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_args_kwargs_are_neither_passed, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote_multi, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_fork_in_jit_with_profiling, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_rpc_with_profiling, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_future_python_annotation, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_local_rref_local_value, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_more_than_needed_args_are_specified, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_my_script_module_with_rrefs, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_throw, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_return_local_script_module_rref_in_py_and_use_in_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_some_kwargs_are_populated_by_defaults, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_restore_context_after_swtich_to_jit_thread, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_bad_module, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_async_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_train_eval, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_unsupported_methods, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeReinforcementLearningRpcTest::test_rl_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_builtin_remote_ret_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_builtin_remote_self_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_py_udf_remote_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rpc_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_get_type_timeout_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_get_type_timeout_non_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_py_udf_remote_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_self_remote_arg_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_tensorpipe_options_throw_on_timedelta_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_wait_all_workers_twice_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_world_size_one_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_multi_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backwards_nested_python_udf_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_nested_rpc_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_no_grad_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_embedding_bag_with_no_grad_tensors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_multiple_backward_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_no_graph_with_tensors_not_require_grad_remote_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_remote_complex_args_sparse 2024-08-07T18:58:32.4832900Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_add 2024-08-07T18:58:32.4833892Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_add_with_id 2024-08-07T18:58:32.4834889Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_chained 2024-08-07T18:58:32.4835925Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_fanout 2024-08-07T18:58:32.4837002Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_fanout_async 2024-08-07T18:58:32.4838053Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_simple 2024-08-07T18:58:32.4839101Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_with_future_ctor 2024-08-07T18:58:32.4840249Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_cbs_jit_call 2024-08-07T18:58:32.4841383Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_build_rpc_profiling_key 2024-08-07T18:58:32.4842383Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_builtin_remote_ret 2024-08-07T18:58:32.4843338Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_in_rpc 2024-08-07T18:58:32.4844294Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_none 2024-08-07T18:58:32.4845256Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_with_error 2024-08-07T18:58:32.4846193Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_deadlock 2024-08-07T18:58:32.4847140Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_disable_gil_profiling 2024-08-07T18:58:32.4848133Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_dist_init_decorator 2024-08-07T18:58:32.4849102Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_duplicate_name_2 2024-08-07T18:58:32.4850040Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_in_rpc 2024-08-07T18:58:32.4850989Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_wait_twice 2024-08-07T18:58:32.4851940Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_ignore_rref_leak 2024-08-07T18:58:32.4852895Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_rpc_then_pg 2024-08-07T18:58:32.4853873Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_shutdown_with_rpc 2024-08-07T18:58:32.4854874Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_my_parameter_server 2024-08-07T18:58:32.4855828Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_remote 2024-08-07T18:58:32.4856968Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_non_garbage_collected_user_rref_due_to_local_circular_dependency 2024-08-07T18:58:32.4858079Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nonzero 2024-08-07T18:58:32.4859136Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_remote_events_profiled_single_threaded 2024-08-07T18:58:32.4860289Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_autograd_context 2024-08-07T18:58:32.4861414Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_builtin_single_threaded 2024-08-07T18:58:32.4862622Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_async_rpc_single_threaded 2024-08-07T18:58:32.4863962Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_remote_rpc 2024-08-07T18:58:32.4865045Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_sync_rpc 2024-08-07T18:58:32.4866064Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_constructor 2024-08-07T18:58:32.4867197Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_instance_method 2024-08-07T18:58:32.4868273Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_multi_async_call 2024-08-07T18:58:32.4869240Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rpc_rref_args 2024-08-07T18:58:32.4870213Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rref_args_user_share 2024-08-07T18:58:32.4871177Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors 2024-08-07T18:58:32.4872099Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_udf_remote 2024-08-07T18:58:32.4873123Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_remote_record_function 2024-08-07T18:58:32.4874258Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_get_future 2024-08-07T18:58:32.4875240Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_non_blocking 2024-08-07T18:58:32.4876232Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_slow_init 2024-08-07T18:58:32.4877238Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_with_error_blocking 2024-08-07T18:58:32.4878268Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rrefs_confirmed 2024-08-07T18:58:32.4879291Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rrefs_confirmed_remote 2024-08-07T18:58:32.4880251Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all 2024-08-07T18:58:32.4881211Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_raise_in_body 2024-08-07T18:58:32.4882228Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_with_exception 2024-08-07T18:58:32.4883285Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_with_partial_exception 2024-08-07T18:58:32.4884343Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_twice_dense 2024-08-07T18:58:32.4885345Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_world_size_one 2024-08-07T18:58:32.4886358Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_autograd_context 2024-08-07T18:58:32.4887454Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_dtypes 2024-08-07T18:58:32.4888557Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_invalid_args 2024-08-07T18:58:32.4889659Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_roots 2024-08-07T18:58:32.4890798Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_round_trips 2024-08-07T18:58:32.4891935Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_no_grad_on_tensor 2024-08-07T18:58:32.4893045Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_node_failure 2024-08-07T18:58:32.4894260Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_node_failure_python_udf 2024-08-07T18:58:32.4895477Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_python_udf_error 2024-08-07T18:58:32.4896614Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_script_call 2024-08-07T18:58:32.4897740Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_verify_hooks 2024-08-07T18:58:32.4898867Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backwards_nested_python_udf 2024-08-07T18:58:32.4900016Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_nested_rpc 2024-08-07T18:58:32.4901170Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_tensor_no_grad 2024-08-07T18:58:32.4902359Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_builtin_remote_call 2024-08-07T18:58:32.4903574Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_remote_call_itself 2024-08-07T18:58:32.4904731Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_python_call 2024-08-07T18:58:32.4905819Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_mixed_requires_grad 2024-08-07T18:58:32.4906955Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_nested_backward_accumulate_grads 2024-08-07T18:58:32.4908092Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_grad_copy_sparse 2024-08-07T18:58:32.4909116Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_post_hooks 2024-08-07T18:58:32.4910108Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_trainer_ps 2024-08-07T18:58:32.4911150Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_exception 2024-08-07T18:58:32.4912315Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_exception_on_constructor 2024-08-07T18:58:32.4913419Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_add_done_callback 2024-08-07T18:58:32.4914587Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_args_and_kwargs_contain_different_types 2024-08-07T18:58:32.4915742Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_args_kwargs_are_neither_passed 2024-08-07T18:58:32.4916844Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote_multi 2024-08-07T18:58:32.4917885Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_simple 2024-08-07T18:58:32.4918963Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_fork_in_jit_with_profiling 2024-08-07T18:58:32.4920044Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_rpc_with_profiling 2024-08-07T18:58:32.4921039Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_simple 2024-08-07T18:58:32.4922114Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_future_python_annotation 2024-08-07T18:58:32.4923209Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_local_rref_local_value 2024-08-07T18:58:32.4924305Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_more_than_needed_args_are_specified 2024-08-07T18:58:32.4925422Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_my_script_module_with_rrefs 2024-08-07T18:58:32.4926472Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_throw 2024-08-07T18:58:32.4927642Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_return_local_script_module_rref_in_py_and_use_in_script 2024-08-07T18:58:32.4928890Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_some_kwargs_are_populated_by_defaults 2024-08-07T18:58:32.4930125Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_restore_context_after_swtich_to_jit_thread 2024-08-07T18:58:32.4931274Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_bad_module 2024-08-07T18:58:32.4932307Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_async_script 2024-08-07T18:58:32.4933326Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_train_eval 2024-08-07T18:58:32.4934357Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_unsupported_methods 2024-08-07T18:58:32.4935437Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeReinforcementLearningRpcTest::test_rl_rpc 2024-08-07T18:58:32.4936578Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_builtin_remote_ret_sparse 2024-08-07T18:58:32.4937773Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_builtin_remote_self_sparse 2024-08-07T18:58:32.4939069Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_py_udf_remote_sparse 2024-08-07T18:58:32.4940239Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rpc_sparse 2024-08-07T18:58:32.4941422Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_get_type_timeout_blocking 2024-08-07T18:58:32.4942676Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_get_type_timeout_non_blocking 2024-08-07T18:58:32.4943930Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_py_udf_remote_sparse 2024-08-07T18:58:32.4945205Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_self_remote_arg_sparse 2024-08-07T18:58:32.4946571Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_tensorpipe_options_throw_on_timedelta_timeout 2024-08-07T18:58:32.4947859Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_wait_all_workers_twice_sparse 2024-08-07T18:58:32.4949069Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_world_size_one_sparse 2024-08-07T18:58:32.4950388Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_multi_sparse 2024-08-07T18:58:32.4952158Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_sparse 2024-08-07T18:58:32.4953480Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backwards_nested_python_udf_sparse 2024-08-07T18:58:32.4954954Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_nested_rpc_sparse 2024-08-07T18:58:32.4956351Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_no_grad_sparse 2024-08-07T18:58:32.4957753Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_embedding_bag_with_no_grad_tensors 2024-08-07T18:58:32.4959082Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_multiple_backward_sparse 2024-08-07T18:58:32.4960499Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_no_graph_with_tensors_not_require_grad_remote_sparse 2024-08-07T18:58:32.4961920Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_remote_complex_args_sparse 2024-08-07T18:58:32.4962630Z 2024-08-07T18:58:32.4963120Z Running distributed/fsdp/test_fsdp_multiple_forward 1/1 ... [2024-08-07 18:58:32.476606] 2024-08-07T18:58:32.4964486Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_multiple_forward.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:58:32.477027] 2024-08-07T18:58:38.2068788Z 2024-08-07T18:58:38.2070104Z distributed/fsdp/test_fsdp_multiple_forward 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_multiple_forward_1.1_82efef342db19c0d_.log 2024-08-07T18:58:38.2071441Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_multiple_forward.py::TestMultiForward::test_multi_forward 2024-08-07T18:58:38.2071992Z 2024-08-07T18:58:38.2074809Z Running distributed/_shard/sharded_tensor/ops/test_tensor_ops 1/1 ... [2024-08-07 18:58:38.207273] 2024-08-07T18:58:38.2079679Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_tensor_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:58:38.207729] 2024-08-07T18:58:40.7316852Z 2024-08-07T18:58:40.7318857Z distributed/_shard/sharded_tensor/ops/test_tensor_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_tensor_ops_1.1_fca5509c87e0a4a0_.log 2024-08-07T18:58:40.7324578Z Running 5 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_clone, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_deep_copy, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_detach, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_inplace_copy, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_set_requires_grad 2024-08-07T18:58:40.7328330Z 2024-08-07T18:58:40.7328739Z Running distributed/test_c10d_gloo 1/1 ... [2024-08-07 18:58:40.732479] 2024-08-07T18:58:40.7334259Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_gloo.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:58:40.733051] 2024-08-07T19:09:39.6024594Z 2024-08-07T19:09:39.6027364Z distributed/test_c10d_gloo 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_gloo_1.1_3e56c660a385de83_.log 2024-08-07T19:09:39.6096453Z Running 122 items in this shard: test/distributed/test_c10d_gloo.py::RendezvousEnvTest::test_logging_init, test/distributed/test_c10d_gloo.py::TimeoutTest::test_default_store_timeout_gloo, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_async, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_into_tensor_coalesced, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_noncontiguous_input, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_async, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_barrier_implies_wait, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_empty_tensors, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_noncontiguous_input, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_multi_device_constructor, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_round_robin, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_round_robin_create_destroy, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_send_recv_all_to_all, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_set_gloo_pg_timeout, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_cuda_dispatched, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output_unused_param, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_weight_sharing, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_weight_sharing, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_cpu, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_gpu_gloo, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_register_just_once, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_sparse_gradients, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_init, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_return_type, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_find_unused_parameters_when_unused_parameters_empty, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_grad_is_view, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_static_graph, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_integer_list, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_torch_device_list, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_2gpu_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_4gpu_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module_grad_is_view, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output_with_unused_parameters, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_sharded_tensor, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_invalid_powerSGD_state, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_save_load_checkpoint, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients_grad_is_view, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_empty_input, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_only_empty_input, test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward, test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_optimizer, test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_unused_parameters, test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_multi_bucket, test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_single_bucket, test/distributed/test_c10d_gloo.py::ReducerTest::test_single_dtype_single_bucket, test/distributed/test_c10d_gloo.py::CommTest::test_bool_tensors, test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cpu, test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cuda, test/distributed/test_c10d_gloo.py::CommTest::test_gloo_rank_membership, test/distributed/test_c10d_gloo.py::CommTest::test_gloo_warn_not_in_group, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_default, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_subgroup, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_default_pg_gloo, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_gloo_new_group, test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_complex, test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_mismatch, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_all_to_all_single, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allgather_coalesced, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allreduce_coalesced, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_collectives, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_for_all_backends, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_optional_backend, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_monitored_barrier, test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync, test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_duplicate_pg, test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_sanity_check 2024-08-07T19:09:39.6145949Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::RendezvousEnvTest::test_logging_init 2024-08-07T19:09:39.6146814Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::TimeoutTest::test_default_store_timeout_gloo 2024-08-07T19:09:39.6147689Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics 2024-08-07T19:09:39.6148634Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics_cuda 2024-08-07T19:09:39.6149584Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_checks 2024-08-07T19:09:39.6150491Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_async 2024-08-07T19:09:39.6151436Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_checks 2024-08-07T19:09:39.6152419Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_into_tensor_coalesced 2024-08-07T19:09:39.6153487Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_noncontiguous_input 2024-08-07T19:09:39.6154417Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress 2024-08-07T19:09:39.6155309Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress_cuda 2024-08-07T19:09:39.6156207Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics 2024-08-07T19:09:39.6157095Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics_cuda 2024-08-07T19:09:39.6157987Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_checks 2024-08-07T19:09:39.6158881Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_async 2024-08-07T19:09:39.6159832Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_basics 2024-08-07T19:09:39.6160793Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks 2024-08-07T19:09:39.6161761Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks_cuda 2024-08-07T19:09:39.6162744Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_stress 2024-08-07T19:09:39.6163848Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress 2024-08-07T19:09:39.6164752Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress_cuda 2024-08-07T19:09:39.6165654Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_barrier_implies_wait 2024-08-07T19:09:39.6166545Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics 2024-08-07T19:09:39.6167449Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics_cuda 2024-08-07T19:09:39.6168341Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_checks 2024-08-07T19:09:39.6169209Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress 2024-08-07T19:09:39.6170099Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress_cuda 2024-08-07T19:09:39.6170987Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_empty_tensors 2024-08-07T19:09:39.6171825Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics 2024-08-07T19:09:39.6172690Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics_cuda 2024-08-07T19:09:39.6173561Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_checks 2024-08-07T19:09:39.6174450Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_noncontiguous_input 2024-08-07T19:09:39.6175564Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress 2024-08-07T19:09:39.6176435Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress_cuda 2024-08-07T19:09:39.6177366Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_multi_device_constructor 2024-08-07T19:09:39.6178262Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics 2024-08-07T19:09:39.6179138Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics_cuda 2024-08-07T19:09:39.6180000Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_checks 2024-08-07T19:09:39.6180884Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor 2024-08-07T19:09:39.6181849Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor_coalesced 2024-08-07T19:09:39.6182771Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress 2024-08-07T19:09:39.6183644Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress_cuda 2024-08-07T19:09:39.6184502Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_round_robin 2024-08-07T19:09:39.6185399Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_round_robin_create_destroy 2024-08-07T19:09:39.6186298Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics 2024-08-07T19:09:39.6187171Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics_cuda 2024-08-07T19:09:39.6188059Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_checks 2024-08-07T19:09:39.6188912Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress 2024-08-07T19:09:39.6189786Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress_cuda 2024-08-07T19:09:39.6190688Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_send_recv_all_to_all 2024-08-07T19:09:39.6191589Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_set_gloo_pg_timeout 2024-08-07T19:09:39.6192491Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics 2024-08-07T19:09:39.6193515Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics_cuda 2024-08-07T19:09:39.6194474Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_checks 2024-08-07T19:09:39.6195457Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_cuda_dispatched 2024-08-07T19:09:39.6196422Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output 2024-08-07T19:09:39.6197429Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output_unused_param 2024-08-07T19:09:39.6198510Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_module 2024-08-07T19:09:39.6199633Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_weight_sharing 2024-08-07T19:09:39.6200819Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_False 2024-08-07T19:09:39.6202650Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_True 2024-08-07T19:09:39.6203890Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_False 2024-08-07T19:09:39.6205164Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_True 2024-08-07T19:09:39.6206407Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_False 2024-08-07T19:09:39.6207594Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_True 2024-08-07T19:09:39.6208761Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_weight_sharing 2024-08-07T19:09:39.6209951Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_False 2024-08-07T19:09:39.6211187Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_True 2024-08-07T19:09:39.6212434Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_False 2024-08-07T19:09:39.6213682Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_True 2024-08-07T19:09:39.6214846Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_cpu 2024-08-07T19:09:39.6215968Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_gpu_gloo 2024-08-07T19:09:39.6217089Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_register_just_once 2024-08-07T19:09:39.6218153Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_sparse_gradients 2024-08-07T19:09:39.6219210Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_init 2024-08-07T19:09:39.6220277Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_return_type 2024-08-07T19:09:39.6221446Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_find_unused_parameters_when_unused_parameters_empty 2024-08-07T19:09:39.6222609Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad 2024-08-07T19:09:39.6223780Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_grad_is_view 2024-08-07T19:09:39.6225009Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_static_graph 2024-08-07T19:09:39.6226243Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_integer_list 2024-08-07T19:09:39.6227473Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_torch_device_list 2024-08-07T19:09:39.6228608Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_2gpu_module 2024-08-07T19:09:39.6229688Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_4gpu_module 2024-08-07T19:09:39.6230762Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module 2024-08-07T19:09:39.6231815Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module_grad_is_view 2024-08-07T19:09:39.6232955Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output 2024-08-07T19:09:39.6233996Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output_with_unused_parameters 2024-08-07T19:09:39.6235058Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_sharded_tensor 2024-08-07T19:09:39.6236060Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_invalid_powerSGD_state 2024-08-07T19:09:39.6237047Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_save_load_checkpoint 2024-08-07T19:09:39.6238012Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients 2024-08-07T19:09:39.6238999Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients_grad_is_view 2024-08-07T19:09:39.6240052Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_empty_input 2024-08-07T19:09:39.6241119Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_only_empty_input 2024-08-07T19:09:39.6242069Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward 2024-08-07T19:09:39.6242898Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_optimizer 2024-08-07T19:09:39.6243820Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_unused_parameters 2024-08-07T19:09:39.6244715Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_multi_bucket 2024-08-07T19:09:39.6245558Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_single_bucket 2024-08-07T19:09:39.6246417Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_single_dtype_single_bucket 2024-08-07T19:09:39.6247224Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_bool_tensors 2024-08-07T19:09:39.6248011Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cpu 2024-08-07T19:09:39.6248870Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cuda 2024-08-07T19:09:39.6249711Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_gloo_rank_membership 2024-08-07T19:09:39.6250522Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_gloo_warn_not_in_group 2024-08-07T19:09:39.6251378Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_default 2024-08-07T19:09:39.6252308Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_subgroup 2024-08-07T19:09:39.6253218Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_default_pg_gloo 2024-08-07T19:09:39.6254093Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_gloo_new_group 2024-08-07T19:09:39.6254941Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_complex 2024-08-07T19:09:39.6255838Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_mismatch 2024-08-07T19:09:39.6256897Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_all_to_all_single 2024-08-07T19:09:39.6258078Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allgather_coalesced 2024-08-07T19:09:39.6259275Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allreduce_coalesced 2024-08-07T19:09:39.6260437Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_collectives 2024-08-07T19:09:39.6261666Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_for_all_backends 2024-08-07T19:09:39.6263186Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_optional_backend 2024-08-07T19:09:39.6264461Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_monitored_barrier 2024-08-07T19:09:39.6265475Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync 2024-08-07T19:09:39.6266359Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_duplicate_pg 2024-08-07T19:09:39.6267308Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_sanity_check 2024-08-07T19:09:39.6267847Z 2024-08-07T19:09:40.4515241Z 2024-08-07T19:09:40.4515798Z real 91m25.456s 2024-08-07T19:09:40.4516222Z user 188m11.753s 2024-08-07T19:09:40.4516571Z sys 36m21.458s 2024-08-07T19:09:40.4516923Z + assert_git_not_dirty 2024-08-07T19:09:40.4517377Z + [[ linux-jammy-py3.8-gcc11 != *rocm* ]] 2024-08-07T19:09:40.4517930Z + [[ linux-jammy-py3.8-gcc11 != *xla* ]] 2024-08-07T19:09:40.4521745Z ++ git status --porcelain 2024-08-07T19:09:40.4522253Z ++ grep -v '?? third_party' 2024-08-07T19:10:03.1624878Z ++ true 2024-08-07T19:10:03.1625626Z + git_status= 2024-08-07T19:10:03.1626012Z + [[ -n '' ]] 2024-08-07T19:10:03.1626400Z + [[ linux-jammy-py3.8-gcc11 == *cuda* ]] 2024-08-07T19:10:03.1627007Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-07T19:10:03.1627497Z + [[ 2 == 1 ]] 2024-08-07T19:10:03.1627835Z + cleanup_workspace 2024-08-07T19:10:03.1628664Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-07T19:10:03.1629959Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-07T19:10:03.1630970Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-07T19:10:03.1631463Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-07T19:10:03.1632063Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-07T19:10:03.1632680Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-07T19:10:03.1633298Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-07T19:10:06.5768052Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-07T19:10:06.5768460Z cat test/**/*_toprint.log || true 2024-08-07T19:10:06.5801715Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:10:06.5802115Z env: 2024-08-07T19:10:06.5802336Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:10:06.5802820Z DOCKER_CONTAINER_ID: 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:06.5803333Z ##[endgroup] 2024-08-07T19:10:06.5874662Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-07T19:10:06.5920214Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-07T19:10:06.5920590Z kill "$MONITOR_SCRIPT_PID" 2024-08-07T19:10:06.5926202Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:10:06.5926746Z env: 2024-08-07T19:10:06.5926977Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:10:06.5927441Z DOCKER_CONTAINER_ID: 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:06.5927969Z MONITOR_SCRIPT_PID: 477288 2024-08-07T19:10:06.5928249Z ##[endgroup] 2024-08-07T19:10:06.6150371Z Prepare all required actions 2024-08-07T19:10:06.6150818Z Getting action download info 2024-08-07T19:10:06.8739373Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-07T19:10:07.1010962Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-07T19:10:07.1011320Z with: 2024-08-07T19:10:07.1011663Z file-suffix: test-distributed-2-2-amz2023.linux.2xlarge_28475634473 2024-08-07T19:10:07.1012107Z s3-bucket: gha-artifacts 2024-08-07T19:10:07.1012375Z env: 2024-08-07T19:10:07.1012583Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:10:07.1013050Z DOCKER_CONTAINER_ID: 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:07.1013570Z ##[endgroup] 2024-08-07T19:10:07.1044264Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-07T19:10:07.1044749Z # Remove any previous test jsons if they exist 2024-08-07T19:10:07.1045126Z rm -f test-jsons-*.zip 2024-08-07T19:10:07.1045571Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-07T19:10:07.1051160Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:10:07.1051548Z env: 2024-08-07T19:10:07.1051779Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:10:07.1052249Z DOCKER_CONTAINER_ID: 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:07.1052890Z FILE_SUFFIX: test-distributed-2-2-amz2023.linux.2xlarge_28475634473 2024-08-07T19:10:07.1053330Z ##[endgroup] 2024-08-07T19:10:07.1385929Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-07T19:10:07.1413387Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-07T19:10:07.1413924Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-07T19:10:07.1419922Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-07T19:10:07.1428284Z adding: test/test-reports/td_exclusions-a85436a93b36321efb9c.json (deflated 86%) 2024-08-07T19:10:07.1433406Z adding: test/.pytorch-slow-tests.json (deflated 81%) 2024-08-07T19:10:07.1446022Z adding: test/.pytorch-disabled-tests.json (deflated 89%) 2024-08-07T19:10:07.1479124Z ##[group]Run # Remove any previous test reports if they exist 2024-08-07T19:10:07.1479620Z # Remove any previous test reports if they exist 2024-08-07T19:10:07.1480013Z rm -f test-reports-*.zip 2024-08-07T19:10:07.1480464Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-07T19:10:07.1489448Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:10:07.1489833Z env: 2024-08-07T19:10:07.1490064Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:10:07.1490544Z DOCKER_CONTAINER_ID: 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:07.1491196Z FILE_SUFFIX: test-distributed-2-2-amz2023.linux.2xlarge_28475634473 2024-08-07T19:10:07.1491626Z ##[endgroup] 2024-08-07T19:10:07.1748252Z adding: test/test-reports/python-pytest/distributed._tools.test_mod_tracker/distributed._tools.test_mod_tracker-130f076f79dcde0c.xml (deflated 64%) 2024-08-07T19:10:07.1752697Z adding: test/test-reports/python-pytest/distributed._tensor.debug.test_op_coverage/distributed._tensor.debug.test_op_coverage-1bf3311968c22126.xml (deflated 34%) 2024-08-07T19:10:07.1786717Z adding: test/test-reports/python-pytest/distributed._tensor.test_dtensor_ops/distributed._tensor.test_dtensor_ops-5f497f1736cbe8af.xml (deflated 96%) 2024-08-07T19:10:07.1788029Z adding: test/test-reports/python-pytest/distributed._tools.test_mem_tracker/distributed._tools.test_mem_tracker-ff4741d433499d69.xml (deflated 65%) 2024-08-07T19:10:07.1789415Z adding: test/test-reports/python-pytest/distributed._tensor.test_convolution_ops/distributed._tensor.test_convolution_ops-f4738c0ee66d9a2f.xml (deflated 54%) 2024-08-07T19:10:07.1790998Z adding: test/test-reports/python-pytest/distributed._tensor.test_attention/distributed._tensor.test_attention-ac868f4354d673b6.xml (deflated 85%) 2024-08-07T19:10:07.1792441Z adding: test/test-reports/python-pytest/distributed._tensor.test_dtensor/distributed._tensor.test_dtensor-2d818b69e6892ff1.xml (deflated 82%) 2024-08-07T19:10:07.1793767Z adding: test/test-reports/python-pytest/distributed._tensor.test_api/distributed._tensor.test_api-f9a52f108667de14.xml (deflated 77%) 2024-08-07T19:10:07.1795026Z adding: test/test-reports/python-pytest/distributed.pipelining.test_pipe/distributed.pipelining.test_pipe-2f5cf36c9a51c2a9.xml (deflated 65%) 2024-08-07T19:10:07.1796429Z adding: test/test-reports/python-pytest/distributed.test_c10d_functional_native/distributed.test_c10d_functional_native-d1d311a75a956942.xml (deflated 93%) 2024-08-07T19:10:07.1797851Z adding: test/test-reports/python-pytest/distributed._tensor.test_matrix_ops/distributed._tensor.test_matrix_ops-fb6d18b3fae17384.xml (deflated 77%) 2024-08-07T19:10:07.1799147Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_fx/distributed.fsdp.test_fsdp_fx-1c7577a3c35481d3.xml (deflated 37%) 2024-08-07T19:10:07.1801332Z adding: test/test-reports/python-pytest/distributed._tensor.test_experimental_ops/distributed._tensor.test_experimental_ops-0f338605c66d2bd2.xml (deflated 60%) 2024-08-07T19:10:07.1802669Z adding: test/test-reports/python-pytest/distributed.test_c10d_pypg/distributed.test_c10d_pypg-047ba4e7b4c20db2.xml (deflated 94%) 2024-08-07T19:10:07.1804009Z adding: test/test-reports/python-pytest/distributed._tensor.test_common_rules/distributed._tensor.test_common_rules-b5c4cc77204cf5c8.xml (deflated 79%) 2024-08-07T19:10:07.1805488Z adding: test/test-reports/python-pytest/distributed.checkpoint.fsdp.test_fsdp_dsd/distributed.checkpoint.fsdp.test_fsdp_dsd-b2cce3734fab1360.xml (deflated 62%) 2024-08-07T19:10:07.1806950Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_dedup_tensors/distributed.checkpoint.test_dedup_tensors-0362b788ea7f39ca.xml (deflated 38%) 2024-08-07T19:10:07.1808343Z adding: test/test-reports/python-pytest/distributed._tools.test_memory_tracker/distributed._tools.test_memory_tracker-6463328f2d971e8b.xml (deflated 44%) 2024-08-07T19:10:07.1809637Z adding: test/test-reports/python-pytest/distributed._tensor.test_init/distributed._tensor.test_init-a68666d2deede744.xml (deflated 74%) 2024-08-07T19:10:07.1810965Z adding: test/test-reports/python-pytest/distributed.pipelining.test_unflatten/distributed.pipelining.test_unflatten-5f3f4adfa373b6d5.xml (deflated 41%) 2024-08-07T19:10:07.1812460Z adding: test/test-reports/python-pytest/distributed.checkpoint.e2e.test_fsdp_ep/distributed.checkpoint.e2e.test_fsdp_ep-6d508e66036ccbbb.xml (deflated 34%) 2024-08-07T19:10:07.1813862Z adding: test/test-reports/python-pytest/distributed._tensor.test_xla_integration/distributed._tensor.test_xla_integration-a66ed02856221b4a.xml (deflated 74%) 2024-08-07T19:10:07.1815367Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_save_load_api/distributed.checkpoint.test_save_load_api-bc9d3acd4166f4dc.xml (deflated 43%) 2024-08-07T19:10:07.1816820Z adding: test/test-reports/python-pytest/distributed.pipelining.test_microbatch/distributed.pipelining.test_microbatch-cf68ee8f1e010757.xml (deflated 53%) 2024-08-07T19:10:07.1818372Z adding: test/test-reports/python-pytest/distributed._tensor.debug.test_comm_mode_features/distributed._tensor.debug.test_comm_mode_features-078914ca4504b517.xml (deflated 65%) 2024-08-07T19:10:07.1819980Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_state/distributed._composable.fsdp.test_fully_shard_state-52cc439a4d8ff2d1.xml (deflated 81%) 2024-08-07T19:10:07.1821478Z adding: test/test-reports/python-pytest/distributed._composable.test_checkpoint/distributed._composable.test_checkpoint-b04991736e407157.xml (deflated 70%) 2024-08-07T19:10:07.1822981Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_tp_style/distributed.tensor.parallel.test_tp_style-3872a347bcb87660.xml (deflated 80%) 2024-08-07T19:10:07.1824538Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_traverse/distributed.checkpoint.test_traverse-e2136b5a1503481f.xml (deflated 75%) 2024-08-07T19:10:07.1826080Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_logging/distributed._composable.fsdp.test_fully_shard_logging-3b618e8973db8505.xml (deflated 28%) 2024-08-07T19:10:07.1827630Z adding: test/test-reports/python-pytest/distributed.test_compute_comm_reordering/distributed.test_compute_comm_reordering-93f6016c3149a5f3.xml (deflated 86%) 2024-08-07T19:10:07.1829160Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_tp_random_state/distributed.tensor.parallel.test_tp_random_state-702dd130272855d3.xml (deflated 46%) 2024-08-07T19:10:07.1830664Z adding: test/test-reports/python-pytest/distributed._tools.test_fsdp2_mem_tracker/distributed._tools.test_fsdp2_mem_tracker-49cd9b9882ea0ed3.xml (deflated 60%) 2024-08-07T19:10:07.1832093Z adding: test/test-reports/python-pytest/distributed.test_inductor_collectives/distributed.test_inductor_collectives-a162f9a93197b3a7.xml (deflated 94%) 2024-08-07T19:10:07.1833655Z adding: test/test-reports/python-pytest/distributed.checkpoint.e2e.test_e2e_save_and_load/distributed.checkpoint.e2e.test_e2e_save_and_load-71d70c490430ddaa.xml (deflated 89%) 2024-08-07T19:10:07.1835089Z adding: test/test-reports/python-pytest/distributed.launcher.test_run/distributed.launcher.test_run-4d6a821a921e37d0.xml (deflated 84%) 2024-08-07T19:10:07.1836548Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_compile/distributed._composable.fsdp.test_fully_shard_compile-31f484442dc5a781.xml (deflated 89%) 2024-08-07T19:10:07.1838223Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_overlap/distributed._composable.fsdp.test_fully_shard_overlap-fda0a41a1fcf80a2.xml (deflated 55%) 2024-08-07T19:10:07.1839682Z adding: test/test-reports/python-pytest/distributed._tensor.test_view_ops/distributed._tensor.test_view_ops-3293cef06ccda560.xml (deflated 58%) 2024-08-07T19:10:07.1841050Z adding: test/test-reports/python-pytest/distributed.checkpoint.e2e.test_fine_tuning/distributed.checkpoint.e2e.test_fine_tuning-8962be1f7d702fa2.xml (deflated 38%) 2024-08-07T19:10:07.1842462Z adding: test/test-reports/python-pytest/distributed.elastic.test_control_plane/distributed.elastic.test_control_plane-35108931e672564c.xml (deflated 78%) 2024-08-07T19:10:07.1843791Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_planner/distributed.checkpoint.test_planner-c76975eb0aa346ff.xml (deflated 67%) 2024-08-07T19:10:07.1845295Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_dtensor_checkpoint/distributed.checkpoint.test_dtensor_checkpoint-bf4d458046a913ef.xml (deflated 47%) 2024-08-07T19:10:07.1846777Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_tp_integration/distributed.fsdp.test_fsdp_tp_integration-41ddd67f34de0de8.xml (deflated 72%) 2024-08-07T19:10:07.1848315Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_memory/distributed._composable.fsdp.test_fully_shard_memory-bc14a649e029aea6.xml (deflated 38%) 2024-08-07T19:10:07.1849887Z adding: test/test-reports/python-pytest/distributed.checkpoint.e2e.test_pipeline/distributed.checkpoint.e2e.test_pipeline-06059dd7e29f7f47.xml (deflated 38%) 2024-08-07T19:10:07.1851343Z adding: test/test-reports/python-pytest/distributed._tensor.test_op_strategy/distributed._tensor.test_op_strategy-331783ce1363d436.xml (deflated 84%) 2024-08-07T19:10:07.1852785Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_compatibility/distributed.checkpoint.test_compatibility-ff54cfeaafde8dbf.xml (deflated 66%) 2024-08-07T19:10:07.1854349Z adding: test/test-reports/python-pytest/distributed._tensor.test_embedding_ops/distributed._tensor.test_embedding_ops-519a21e2a8e6f106.xml (deflated 67%) 2024-08-07T19:10:07.1855879Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_ignored_modules/distributed.fsdp.test_fsdp_ignored_modules-d3aa5735d4846548.xml (deflated 86%) 2024-08-07T19:10:07.1857428Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_hsdp_checkpoint/distributed.checkpoint.test_hsdp_checkpoint-39d2be4897356dae.xml (deflated 79%) 2024-08-07T19:10:07.1858911Z adding: test/test-reports/python-pytest/distributed.pipelining.test_schedule/distributed.pipelining.test_schedule-1385a837f9cf07ff.xml (deflated 83%) 2024-08-07T19:10:07.1860254Z adding: test/test-reports/python-pytest/distributed.test_functional_api/distributed.test_functional_api-d341b4b0ba47784d.xml (deflated 87%) 2024-08-07T19:10:07.1861830Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_compile/distributed._composable.fully_shard.test_fully_shard_compile-5257b865138b7b2a.xml (deflated 49%) 2024-08-07T19:10:07.1863691Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_state_dict_utils/distributed.checkpoint.test_state_dict_utils-5b970f1a5e3971c4.xml (deflated 71%) 2024-08-07T19:10:07.1865151Z adding: test/test-reports/python-pytest/distributed._composable.test_replicate/distributed._composable.test_replicate-2db6bcf2f622f88f.xml (deflated 79%) 2024-08-07T19:10:07.1866678Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_frozen/distributed._composable.fsdp.test_fully_shard_frozen-71f4e6d1a5278f36.xml (deflated 62%) 2024-08-07T19:10:07.1868148Z adding: test/test-reports/python-pytest/distributed._tensor.test_optimizers/distributed._tensor.test_optimizers-2c8cb050ff9f51d0.xml (deflated 81%) 2024-08-07T19:10:07.1869784Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_mixed_precision/distributed._composable.fully_shard.test_fully_shard_mixed_precision-231f8a5c4860fb31.xml (deflated 47%) 2024-08-07T19:10:07.1871642Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_util/distributed._composable.fully_shard.test_fully_shard_util-05fe67cf23d6e290.xml (deflated 47%) 2024-08-07T19:10:07.1873293Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_hybrid_shard/distributed.fsdp.test_fsdp_hybrid_shard-fa3ff7e582e89dba.xml (deflated 81%) 2024-08-07T19:10:07.1874825Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_dtensor_state_dict/distributed.fsdp.test_fsdp_dtensor_state_dict-ac159fa35398366f.xml (deflated 92%) 2024-08-07T19:10:07.1876276Z adding: test/test-reports/python-pytest/distributed._composable.test_compose/distributed._composable.test_compose-c064b69b77e4f727.xml (deflated 86%) 2024-08-07T19:10:07.1877780Z adding: test/test-reports/python-pytest/distributed.fsdp.test_distributed_checkpoint/distributed.fsdp.test_distributed_checkpoint-740c2ad569e138e2.xml (deflated 77%) 2024-08-07T19:10:07.1879270Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_clip_grad_norm/distributed.fsdp.test_fsdp_clip_grad_norm-7c072129eab77a2f.xml (deflated 77%) 2024-08-07T19:10:07.1880736Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_checkpoint/distributed.checkpoint.test_checkpoint-8e9b3c7b484b6562.xml (deflated 84%) 2024-08-07T19:10:07.1882228Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_sharded_grad_scaler/distributed.fsdp.test_fsdp_sharded_grad_scaler-0191130ef74ab3c7.xml (deflated 93%) 2024-08-07T19:10:07.1883894Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_state_dict/distributed._composable.fsdp.test_fully_shard_state_dict-d0ddbd92d2ab7783.xml (deflated 62%) 2024-08-07T19:10:07.1885433Z adding: test/test-reports/python-pytest/distributed._tensor.test_utils/distributed._tensor.test_utils-a9ffba31b33ff4be.xml (deflated 68%) 2024-08-07T19:10:07.1886893Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_pure_fp16/distributed.fsdp.test_fsdp_pure_fp16-a11576987d3d8bbf.xml (deflated 63%) 2024-08-07T19:10:07.1888657Z adding: test/test-reports/python-pytest/distributed._composable.test_composability.test_2d_composability/distributed._composable.test_composability.test_2d_composability-0cdc1b58048d476c.xml (deflated 62%) 2024-08-07T19:10:07.1890318Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_overlap/distributed.fsdp.test_fsdp_overlap-3edee5ef75719945.xml (deflated 64%) 2024-08-07T19:10:07.1891721Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_unshard_params/distributed.fsdp.test_fsdp_unshard_params-5ede35988e28a35f.xml (deflated 89%) 2024-08-07T19:10:07.1893375Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_mixed_precision/distributed._composable.fsdp.test_fully_shard_mixed_precision-257862b52408a1c0.xml (deflated 83%) 2024-08-07T19:10:07.1895141Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_file_system_checkpoint/distributed.checkpoint.test_file_system_checkpoint-46df29794e22f2bd.xml (deflated 79%) 2024-08-07T19:10:07.1896749Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_mixed_precision/distributed.fsdp.test_fsdp_mixed_precision-84319df0cc898f88.xml (deflated 95%) 2024-08-07T19:10:07.1898206Z adding: test/test-reports/python-pytest/distributed.nn.jit.test_instantiator/distributed.nn.jit.test_instantiator-3550f2c5fae2c255.xml (deflated 61%) 2024-08-07T19:10:07.1899578Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_state_dict/distributed.fsdp.test_fsdp_state_dict-0f4a5c6affcd512b.xml (deflated 97%) 2024-08-07T19:10:07.1901140Z adding: test/test-reports/python-pytest/distributed._composable.test_replicate_with_compiler/distributed._composable.test_replicate_with_compiler-74f5b093e0553ebe.xml (deflated 86%) 2024-08-07T19:10:07.1902976Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_model_checkpoint/distributed._composable.fully_shard.test_fully_shard_model_checkpoint-aedfd4838da961ef.xml (deflated 80%) 2024-08-07T19:10:07.1904730Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_use_orig_params/distributed.fsdp.test_fsdp_use_orig_params-716e3bee9ab228f9.xml (deflated 90%) 2024-08-07T19:10:07.1906201Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_flatten_params/distributed.fsdp.test_fsdp_flatten_params-f62218687030c420.xml (deflated 84%) 2024-08-07T19:10:07.1907816Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_init/distributed._composable.fully_shard.test_fully_shard_init-be807239dfdf61a8.xml (deflated 84%) 2024-08-07T19:10:07.1909322Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-9ec8b21e353aac88.xml (deflated 44%) 2024-08-07T19:10:07.1910550Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-b1adbb6f158d92ee.xml (deflated 44%) 2024-08-07T19:10:07.1911773Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-73f8d48da6e4f6f1.xml (deflated 43%) 2024-08-07T19:10:07.1913063Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-fd9c59a8391166da.xml (deflated 43%) 2024-08-07T19:10:07.1914282Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-023eaaac50647eb7.xml (deflated 44%) 2024-08-07T19:10:07.1915519Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-926ea8b427ed0e10.xml (deflated 44%) 2024-08-07T19:10:07.1916721Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-81496ffd050987fb.xml (deflated 44%) 2024-08-07T19:10:07.1917933Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-25aa074166de7efa.xml (deflated 44%) 2024-08-07T19:10:07.1919220Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-df47e20134bfa660.xml (deflated 35%) 2024-08-07T19:10:07.1920493Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-4657b93059a3ed58.xml (deflated 42%) 2024-08-07T19:10:07.1921731Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-0ad801c3c5ada3f2.xml (deflated 42%) 2024-08-07T19:10:07.1922960Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-4c1da6be7f53534f.xml (deflated 35%) 2024-08-07T19:10:07.1924178Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-04195308c56ce82e.xml (deflated 34%) 2024-08-07T19:10:07.1925354Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-23fbeee9112406f6.xml (deflated 35%) 2024-08-07T19:10:07.1926578Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-13ae10be5173accd.xml (deflated 42%) 2024-08-07T19:10:07.1927781Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-5c239c213ad88ef0.xml (deflated 42%) 2024-08-07T19:10:07.1929000Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-dacb6a104c493206.xml (deflated 34%) 2024-08-07T19:10:07.1930304Z adding: test/test-reports/python-pytest/distributed.test_multi_threaded_pg/distributed.test_multi_threaded_pg-7aaf16bc5cf0214b.xml (deflated 80%) 2024-08-07T19:10:07.1931644Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_memory/distributed.fsdp.test_fsdp_memory-c8f5f3a24680300b.xml (deflated 64%) 2024-08-07T19:10:07.1933005Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_grad_acc/distributed.fsdp.test_fsdp_grad_acc-0b824021944d404a.xml (deflated 84%) 2024-08-07T19:10:07.1934442Z adding: test/test-reports/python-pytest/distributed.test_c10d_object_collectives/distributed.test_c10d_object_collectives-7974d6535a565a4a.xml (deflated 81%) 2024-08-07T19:10:07.1936105Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_extensions/distributed._composable.fsdp.test_fully_shard_extensions-2c1cdbc22917a32d.xml (deflated 71%) 2024-08-07T19:10:07.1937761Z adding: test/test-reports/python-pytest/distributed.fsdp.test_hsdp_dtensor_state_dict/distributed.fsdp.test_hsdp_dtensor_state_dict-47da1bf190bde4d0.xml (deflated 86%) 2024-08-07T19:10:07.1939437Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_runtime/distributed._composable.fully_shard.test_fully_shard_runtime-4ce5cef745b5faf5.xml (deflated 53%) 2024-08-07T19:10:07.1941238Z adding: test/test-reports/python-pytest/distributed.algorithms.ddp_comm_hooks.test_ddp_hooks/distributed.algorithms.ddp_comm_hooks.test_ddp_hooks-c62610b302a3ed92.xml (deflated 85%) 2024-08-07T19:10:07.1942964Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_file_system_checkpoint_cpu/distributed.checkpoint.test_file_system_checkpoint_cpu-07d351960f73b1ce.xml (deflated 79%) 2024-08-07T19:10:07.1944550Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_apply/distributed.fsdp.test_fsdp_apply-e76407703a63f40a.xml (deflated 71%) 2024-08-07T19:10:07.1945889Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_comm_hooks/distributed.fsdp.test_fsdp_comm_hooks-b9c598ee3f4cda89.xml (deflated 94%) 2024-08-07T19:10:07.1947218Z adding: test/test-reports/python-pytest/distributed.fsdp.test_wrap/distributed.fsdp.test_wrap-d40e6c301d89ddb3.xml (deflated 94%) 2024-08-07T19:10:07.1948549Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_traversal/distributed.fsdp.test_fsdp_traversal-98c26622a621b884.xml (deflated 45%) 2024-08-07T19:10:07.1950010Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-7519141e515f5744.xml (deflated 41%) 2024-08-07T19:10:07.1951638Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-fb984b7b4562e278.xml (deflated 41%) 2024-08-07T19:10:07.1953237Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-8a9b490950cd910c.xml (deflated 41%) 2024-08-07T19:10:07.1954756Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-06546b416e505d5c.xml (deflated 41%) 2024-08-07T19:10:07.1956265Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-87114aa8496014d4.xml (deflated 41%) 2024-08-07T19:10:07.1957785Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-4343837162816c62.xml (deflated 40%) 2024-08-07T19:10:07.1959306Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-99d811274ce40fc8.xml (deflated 41%) 2024-08-07T19:10:07.1960831Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0df1e38710bb2304.xml (deflated 40%) 2024-08-07T19:10:07.1962389Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-ec8ca1275aa87bc9.xml (deflated 43%) 2024-08-07T19:10:07.1964034Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-46edaf2102dc0334.xml (deflated 41%) 2024-08-07T19:10:07.1965552Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-e0423dbafdffab79.xml (deflated 41%) 2024-08-07T19:10:07.1967073Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-57db13036c5ea66f.xml (deflated 42%) 2024-08-07T19:10:07.1968597Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-13bdda36e617a7d6.xml (deflated 42%) 2024-08-07T19:10:07.1970135Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-5e4672f9f1728f77.xml (deflated 42%) 2024-08-07T19:10:07.1971672Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-a93eb3883d277593.xml (deflated 42%) 2024-08-07T19:10:07.1973182Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-6b3bdfc57dca54b5.xml (deflated 42%) 2024-08-07T19:10:07.1974713Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-faa56048bd7a311a.xml (deflated 42%) 2024-08-07T19:10:07.1976239Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-fc0d60853a0651ea.xml (deflated 42%) 2024-08-07T19:10:07.1977761Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-ac79dce8af287f0e.xml (deflated 42%) 2024-08-07T19:10:07.1979311Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-281bcb318313b822.xml (deflated 42%) 2024-08-07T19:10:07.1980836Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-020fe8eb94a1a2ee.xml (deflated 42%) 2024-08-07T19:10:07.1982343Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-1b10844395119798.xml (deflated 42%) 2024-08-07T19:10:07.1984071Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-01ed81a9ac6ec80e.xml (deflated 42%) 2024-08-07T19:10:07.1985620Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-17fe1fcd4371bbd3.xml (deflated 42%) 2024-08-07T19:10:07.1987147Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-57f70aef29c7e152.xml (deflated 42%) 2024-08-07T19:10:07.1988675Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-03ef80e73dbd6d9a.xml (deflated 42%) 2024-08-07T19:10:07.1990189Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-04b869144915932f.xml (deflated 42%) 2024-08-07T19:10:07.1991729Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-7d1b7d45abdde675.xml (deflated 42%) 2024-08-07T19:10:07.1993336Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-78aaa7694752b41d.xml (deflated 42%) 2024-08-07T19:10:07.1994855Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-3cb9709317e0b388.xml (deflated 36%) 2024-08-07T19:10:07.1996377Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-f9864e8c345a37d3.xml (deflated 42%) 2024-08-07T19:10:07.1997884Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-46a12e297e634d0b.xml (deflated 42%) 2024-08-07T19:10:07.1999411Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-15064e68a80f3fc0.xml (deflated 42%) 2024-08-07T19:10:07.2000934Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-81b48b1dc1187c58.xml (deflated 42%) 2024-08-07T19:10:07.2002459Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-363a35c83f1b9446.xml (deflated 42%) 2024-08-07T19:10:07.2004005Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-97ad07e674b2971a.xml (deflated 42%) 2024-08-07T19:10:07.2005524Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-bef6238e0ec39f2a.xml (deflated 42%) 2024-08-07T19:10:07.2007046Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-2e6eb096f2c1d1ea.xml (deflated 41%) 2024-08-07T19:10:07.2008577Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-456db4e67610169f.xml (deflated 41%) 2024-08-07T19:10:07.2010082Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b25070621443193b.xml (deflated 41%) 2024-08-07T19:10:07.2011602Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-6bc6ec4d6c379af2.xml (deflated 41%) 2024-08-07T19:10:07.2013143Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-9a3152bdf1e6624f.xml (deflated 42%) 2024-08-07T19:10:07.2014671Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0f819a33db9388c1.xml (deflated 42%) 2024-08-07T19:10:07.2016282Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-6b257daee4718b17.xml (deflated 41%) 2024-08-07T19:10:07.2017862Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-02cd5e26945b8cf0.xml (deflated 41%) 2024-08-07T19:10:07.2019396Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-9ec76e351bb7aa1b.xml (deflated 42%) 2024-08-07T19:10:07.2020920Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-5a8f4ad60941187d.xml (deflated 42%) 2024-08-07T19:10:07.2022432Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d81707fcfa01c7e9.xml (deflated 41%) 2024-08-07T19:10:07.2023974Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b51de598270a8afb.xml (deflated 42%) 2024-08-07T19:10:07.2025489Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-e082a57a17428669.xml (deflated 41%) 2024-08-07T19:10:07.2027015Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d3d95a2dc54fa99c.xml (deflated 42%) 2024-08-07T19:10:07.2028612Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-819e232d09b075e4.xml (deflated 42%) 2024-08-07T19:10:07.2031286Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-6554ad48a0fdb20e.xml (deflated 41%) 2024-08-07T19:10:07.2034031Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-37b8ca67491c20d6.xml (deflated 42%) 2024-08-07T19:10:07.2036722Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-69774e8cf888958d.xml (deflated 42%) 2024-08-07T19:10:07.2039407Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-01452a0787427e07.xml (deflated 42%) 2024-08-07T19:10:07.2042044Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-5a21a8b41e362345.xml (deflated 42%) 2024-08-07T19:10:07.2044712Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-2315df13f632c50c.xml (deflated 42%) 2024-08-07T19:10:07.2047397Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b84d53ff4bb2107b.xml (deflated 42%) 2024-08-07T19:10:07.2050090Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-65f790a8b3ba9a73.xml (deflated 42%) 2024-08-07T19:10:07.2052740Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-1041701a03212d98.xml (deflated 42%) 2024-08-07T19:10:07.2055414Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-5082f93e103ae347.xml (deflated 41%) 2024-08-07T19:10:07.2058093Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-ebe121884f730d7b.xml (deflated 42%) 2024-08-07T19:10:07.2060782Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-96142f005981b000.xml (deflated 42%) 2024-08-07T19:10:07.2063707Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-c189315ed9be4bde.xml (deflated 42%) 2024-08-07T19:10:07.2066542Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-bd9663db55211999.xml (deflated 41%) 2024-08-07T19:10:07.2069235Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-8e4068d16a663a6e.xml (deflated 42%) 2024-08-07T19:10:07.2071932Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-365e28e37a063750.xml (deflated 42%) 2024-08-07T19:10:07.2074638Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-5580ec617d1f9b77.xml (deflated 42%) 2024-08-07T19:10:07.2077325Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-debd938b770f18de.xml (deflated 42%) 2024-08-07T19:10:07.2080051Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-8aab60d4c4ebaa1a.xml (deflated 42%) 2024-08-07T19:10:07.2082737Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-e3136b7c25beafd4.xml (deflated 42%) 2024-08-07T19:10:07.2085396Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-2e5f5f9730359fa9.xml (deflated 42%) 2024-08-07T19:10:07.2088095Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-1f59dc66932fa1aa.xml (deflated 42%) 2024-08-07T19:10:07.2090797Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-65f9825ed49f9ba3.xml (deflated 42%) 2024-08-07T19:10:07.2093491Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-f98b605c0a1a43a5.xml (deflated 42%) 2024-08-07T19:10:07.2096178Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-14d5d19764b5858b.xml (deflated 42%) 2024-08-07T19:10:07.2098867Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-e5e69d91c3976c04.xml (deflated 41%) 2024-08-07T19:10:07.2101555Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-fb6f424a3b59d2cf.xml (deflated 41%) 2024-08-07T19:10:07.2104242Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-e4361f874d799173.xml (deflated 41%) 2024-08-07T19:10:07.2106905Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-128cb9cfe53bf69c.xml (deflated 41%) 2024-08-07T19:10:07.2109598Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-052abc1a529ff651.xml (deflated 41%) 2024-08-07T19:10:07.2112279Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-af8e50e7508d7c41.xml (deflated 41%) 2024-08-07T19:10:07.2115030Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-98622629e54d7489.xml (deflated 41%) 2024-08-07T19:10:07.2117680Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b904b43e48675a9e.xml (deflated 41%) 2024-08-07T19:10:07.2120493Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-17a8c9cc131524c2.xml (deflated 41%) 2024-08-07T19:10:07.2123288Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d20d86b4005bd077.xml (deflated 41%) 2024-08-07T19:10:07.2125957Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-578c2f1d63560029.xml (deflated 41%) 2024-08-07T19:10:07.2128642Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-31c8a4a8fdb18f29.xml (deflated 41%) 2024-08-07T19:10:07.2131343Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-1c678f87d32933a5.xml (deflated 41%) 2024-08-07T19:10:07.2134003Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-9d64bdba0d56e207.xml (deflated 41%) 2024-08-07T19:10:07.2136713Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-858a75bba82e6382.xml (deflated 41%) 2024-08-07T19:10:07.2139360Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-94961ab9450cf216.xml (deflated 41%) 2024-08-07T19:10:07.2142035Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b5b4fd7e0f60cdee.xml (deflated 41%) 2024-08-07T19:10:07.2144746Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-43f227db8375284f.xml (deflated 42%) 2024-08-07T19:10:07.2147416Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-8835af1de8c16792.xml (deflated 42%) 2024-08-07T19:10:07.2150075Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-44ed77d786aaeaa0.xml (deflated 42%) 2024-08-07T19:10:07.2151705Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d229536146893a1a.xml (deflated 43%) 2024-08-07T19:10:07.2153155Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-bef9ee8d73600428.xml (deflated 42%) 2024-08-07T19:10:07.2154637Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_binary_cmp/distributed._shard.sharded_tensor.ops.test_binary_cmp-e036268dc230d83d.xml (deflated 80%) 2024-08-07T19:10:07.2156422Z adding: test/test-reports/python-pytest/distributed._shard.sharding_spec.test_sharding_spec/distributed._shard.sharding_spec.test_sharding_spec-77142c5dc11074f5.xml (deflated 83%) 2024-08-07T19:10:07.2159123Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint-00fedcfc7b042bd3.xml (deflated 67%) 2024-08-07T19:10:07.2160743Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6cb4fba6ee3dd364.xml (deflated 35%) 2024-08-07T19:10:07.2162061Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e77fa29f8ad25e47.xml (deflated 36%) 2024-08-07T19:10:07.2163530Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4dbbde0aab1aab9e.xml (deflated 36%) 2024-08-07T19:10:07.2164854Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-efba77e648c36be2.xml (deflated 35%) 2024-08-07T19:10:07.2166306Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b4baed5152e1ee9d.xml (deflated 35%) 2024-08-07T19:10:07.2167712Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-866b3c1433ec6040.xml (deflated 35%) 2024-08-07T19:10:07.2169010Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1ba521205cc48741.xml (deflated 35%) 2024-08-07T19:10:07.2170337Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cf730a57116c7a7d.xml (deflated 35%) 2024-08-07T19:10:07.2171654Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-84fa846b88a26a28.xml (deflated 35%) 2024-08-07T19:10:07.2172968Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9f42b2021ce75600.xml (deflated 34%) 2024-08-07T19:10:07.2174270Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d78a3b8c56ce333f.xml (deflated 35%) 2024-08-07T19:10:07.2175594Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7d2388d706e5c995.xml (deflated 36%) 2024-08-07T19:10:07.2176907Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-488cfd9ea762dae5.xml (deflated 35%) 2024-08-07T19:10:07.2178216Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-233d15184bd3be7b.xml (deflated 35%) 2024-08-07T19:10:07.2179505Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b6cc0186decf4c95.xml (deflated 35%) 2024-08-07T19:10:07.2180820Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d37d85db3cfc27a2.xml (deflated 35%) 2024-08-07T19:10:07.2182132Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-395cb27ff08aedfa.xml (deflated 35%) 2024-08-07T19:10:07.2183446Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2208455dccb3313f.xml (deflated 56%) 2024-08-07T19:10:07.2184738Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-10baa480281be8d9.xml (deflated 35%) 2024-08-07T19:10:07.2186049Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a6b2dcf0d2e4bacf.xml (deflated 35%) 2024-08-07T19:10:07.2187373Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-81bbee7ad9cb59be.xml (deflated 35%) 2024-08-07T19:10:07.2188692Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f9723e32aae22715.xml (deflated 35%) 2024-08-07T19:10:07.2189989Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e1ce6d7d5432420a.xml (deflated 35%) 2024-08-07T19:10:07.2191303Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-44b6f3f786968c68.xml (deflated 35%) 2024-08-07T19:10:07.2192604Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5592b96c55d6999e.xml (deflated 35%) 2024-08-07T19:10:07.2193982Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1bd618438da8f4cb.xml (deflated 36%) 2024-08-07T19:10:07.2195287Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3ff99a72369d3c6c.xml (deflated 38%) 2024-08-07T19:10:07.2196700Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-61ee46550012aa72.xml (deflated 35%) 2024-08-07T19:10:07.2198075Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e0f26ee1c0943ffb.xml (deflated 36%) 2024-08-07T19:10:07.2199394Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b04fea1b26fc4eb5.xml (deflated 36%) 2024-08-07T19:10:07.2200702Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d45d907b7ac8c28c.xml (deflated 35%) 2024-08-07T19:10:07.2202010Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c216167a90a06f4e.xml (deflated 35%) 2024-08-07T19:10:07.2203321Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-04b83c1c0f7bd231.xml (deflated 35%) 2024-08-07T19:10:07.2204627Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e6f1d5ab27b273b4.xml (deflated 35%) 2024-08-07T19:10:07.2205943Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0acba1fdf541b665.xml (deflated 36%) 2024-08-07T19:10:07.2207253Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-57903baf976a965f.xml (deflated 35%) 2024-08-07T19:10:07.2208560Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c4623ea0a45dbe00.xml (deflated 35%) 2024-08-07T19:10:07.2209865Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a259ce28b4205b6b.xml (deflated 35%) 2024-08-07T19:10:07.2211182Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-675285ce5f9fdd57.xml (deflated 35%) 2024-08-07T19:10:07.2212502Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0d8be48b20616775.xml (deflated 35%) 2024-08-07T19:10:07.2213813Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-874d470d626a8b1c.xml (deflated 34%) 2024-08-07T19:10:07.2215107Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2f803d87c6a38585.xml (deflated 36%) 2024-08-07T19:10:07.2216418Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6cfc673270f03cae.xml (deflated 35%) 2024-08-07T19:10:07.2217737Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-250b797dcad2e0e1.xml (deflated 34%) 2024-08-07T19:10:07.2219072Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-efcc6d2c17d1f306.xml (deflated 34%) 2024-08-07T19:10:07.2220373Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5b26b1a7e28b3519.xml (deflated 35%) 2024-08-07T19:10:07.2221682Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d42a99142f04669d.xml (deflated 34%) 2024-08-07T19:10:07.2222989Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4b45429e708a587d.xml (deflated 35%) 2024-08-07T19:10:07.2224296Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9fa9a95dc934097f.xml (deflated 35%) 2024-08-07T19:10:07.2225651Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e880fe58ee025f19.xml (deflated 35%) 2024-08-07T19:10:07.2227011Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-445af6ce3efd0474.xml (deflated 35%) 2024-08-07T19:10:07.2228328Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4b50d1465e3a22da.xml (deflated 35%) 2024-08-07T19:10:07.2229637Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c55a70ad1a61cb73.xml (deflated 35%) 2024-08-07T19:10:07.2230954Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8bdd75ee6fb0ceda.xml (deflated 36%) 2024-08-07T19:10:07.2232271Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c37b2853a81c31ea.xml (deflated 35%) 2024-08-07T19:10:07.2233675Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8732902a69ec8cd9.xml (deflated 35%) 2024-08-07T19:10:07.2234996Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e2dd8485cb4dd6e1.xml (deflated 34%) 2024-08-07T19:10:07.2236308Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e189f2fb7f2be546.xml (deflated 34%) 2024-08-07T19:10:07.2237617Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e36c894b02706fe6.xml (deflated 35%) 2024-08-07T19:10:07.2238931Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d8ae927f433e0d66.xml (deflated 34%) 2024-08-07T19:10:07.2240246Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7f81129f774a73ca.xml (deflated 34%) 2024-08-07T19:10:07.2241551Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b23fb19679945d3e.xml (deflated 35%) 2024-08-07T19:10:07.2242874Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fc53b2a6210ff6eb.xml (deflated 35%) 2024-08-07T19:10:07.2244195Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e1dfcba01e3775a3.xml (deflated 35%) 2024-08-07T19:10:07.2245520Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-075a20d9fc466e9e.xml (deflated 35%) 2024-08-07T19:10:07.2246819Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e68a84c4755f80d9.xml (deflated 36%) 2024-08-07T19:10:07.2248151Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-aabba6a2c3aab76c.xml (deflated 35%) 2024-08-07T19:10:07.2249481Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4e78a89adf6cfac8.xml (deflated 35%) 2024-08-07T19:10:07.2250905Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cd1356503d4511f2.xml (deflated 35%) 2024-08-07T19:10:07.2252200Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-05e8d417132746a1.xml (deflated 35%) 2024-08-07T19:10:07.2253509Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-66c1fec91b59c98e.xml (deflated 35%) 2024-08-07T19:10:07.2254818Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-db43a679aecbd014.xml (deflated 36%) 2024-08-07T19:10:07.2256245Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f9683666a8789075.xml (deflated 36%) 2024-08-07T19:10:07.2257540Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-01b0e95f0be4770c.xml (deflated 36%) 2024-08-07T19:10:07.2258860Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0178afdcabb67dbd.xml (deflated 36%) 2024-08-07T19:10:07.2260185Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5a432b0d07d18aa4.xml (deflated 36%) 2024-08-07T19:10:07.2261496Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3876debe788b9f4b.xml (deflated 36%) 2024-08-07T19:10:07.2262974Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-360915e6d7b35919.xml (deflated 37%) 2024-08-07T19:10:07.2264314Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d0ed02b979775fe7.xml (deflated 37%) 2024-08-07T19:10:07.2265634Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-453b6ff7295818fb.xml (deflated 38%) 2024-08-07T19:10:07.2266936Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c49edb00b3dc3870.xml (deflated 38%) 2024-08-07T19:10:07.2268242Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-adba6d73750cf512.xml (deflated 36%) 2024-08-07T19:10:07.2269547Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-67bbca37616efc31.xml (deflated 37%) 2024-08-07T19:10:07.2270873Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9a2c03ab5cc4aedb.xml (deflated 36%) 2024-08-07T19:10:07.2272181Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-248018b0d025a8e1.xml (deflated 36%) 2024-08-07T19:10:07.2273536Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-579d331d171f34c5.xml (deflated 35%) 2024-08-07T19:10:07.2274852Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-319cf9d9ee33a37c.xml (deflated 35%) 2024-08-07T19:10:07.2276163Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b741d9d953cc391d.xml (deflated 35%) 2024-08-07T19:10:07.2277467Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8402e85faf2ff374.xml (deflated 35%) 2024-08-07T19:10:07.2278791Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e3a9e92b67d2d772.xml (deflated 36%) 2024-08-07T19:10:07.2280114Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cc1cb7556df1c5ed.xml (deflated 35%) 2024-08-07T19:10:07.2281425Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1c4408ff991213c9.xml (deflated 35%) 2024-08-07T19:10:07.2282725Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fd2e7fc1b47d9568.xml (deflated 35%) 2024-08-07T19:10:07.2284044Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5e1ba7fae2448870.xml (deflated 35%) 2024-08-07T19:10:07.2285456Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-80cc2850e6761257.xml (deflated 36%) 2024-08-07T19:10:07.2286853Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a902655d2d3205c4.xml (deflated 35%) 2024-08-07T19:10:07.2288158Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-17ebfd2c09308fa0.xml (deflated 35%) 2024-08-07T19:10:07.2289472Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cb12b921f3d093d1.xml (deflated 35%) 2024-08-07T19:10:07.2290789Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f4c39a148a6d00d7.xml (deflated 35%) 2024-08-07T19:10:07.2292103Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-42d81eff164a3f5b.xml (deflated 36%) 2024-08-07T19:10:07.2293412Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e71ba827b8da76b9.xml (deflated 36%) 2024-08-07T19:10:07.2294731Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-439fa5c1bbed85ae.xml (deflated 35%) 2024-08-07T19:10:07.2296047Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-80e67e8238c1efb2.xml (deflated 35%) 2024-08-07T19:10:07.2297354Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-324145fa37c3ebdc.xml (deflated 35%) 2024-08-07T19:10:07.2298645Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-16e00151f041bc08.xml (deflated 35%) 2024-08-07T19:10:07.2299957Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ca4ed00b304582d2.xml (deflated 35%) 2024-08-07T19:10:07.2301256Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-88520500b3ff6e36.xml (deflated 35%) 2024-08-07T19:10:07.2302564Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3d11e1a00086f73c.xml (deflated 36%) 2024-08-07T19:10:07.2303858Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c7ab8e56fd649018.xml (deflated 36%) 2024-08-07T19:10:07.2305166Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-eac440da9562a8fe.xml (deflated 35%) 2024-08-07T19:10:07.2306477Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-399420e26db98a69.xml (deflated 35%) 2024-08-07T19:10:07.2307794Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6ba11603e7ff2f64.xml (deflated 36%) 2024-08-07T19:10:07.2309091Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-74629b0aa68db85e.xml (deflated 35%) 2024-08-07T19:10:07.2310400Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e35945a05d44b15f.xml (deflated 35%) 2024-08-07T19:10:07.2311705Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-440733af95c8d265.xml (deflated 35%) 2024-08-07T19:10:07.2313102Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-47918eb374cfd74c.xml (deflated 35%) 2024-08-07T19:10:07.2314469Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ebb8b01c6bd52453.xml (deflated 35%) 2024-08-07T19:10:07.2315842Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5eaec3b3e70d1b5d.xml (deflated 36%) 2024-08-07T19:10:07.2317149Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4054cfef510742b6.xml (deflated 36%) 2024-08-07T19:10:07.2318469Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8bdcd9bc1b09a2f2.xml (deflated 35%) 2024-08-07T19:10:07.2319767Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7320f513b33438d7.xml (deflated 35%) 2024-08-07T19:10:07.2321072Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1ea6db5652d412f6.xml (deflated 36%) 2024-08-07T19:10:07.2322379Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6525e5f984838a71.xml (deflated 36%) 2024-08-07T19:10:07.2323681Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-20d1d06287f3b826.xml (deflated 35%) 2024-08-07T19:10:07.2324978Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e0712ed6e71bf075.xml (deflated 35%) 2024-08-07T19:10:07.2326297Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-57e4aff6e8deb0c8.xml (deflated 36%) 2024-08-07T19:10:07.2327614Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-491bcec9a8323c36.xml (deflated 35%) 2024-08-07T19:10:07.2328922Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-41e49ac0f33520e9.xml (deflated 35%) 2024-08-07T19:10:07.2330216Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5055c2445b70895e.xml (deflated 36%) 2024-08-07T19:10:07.2331523Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2e78cb385b51fd20.xml (deflated 35%) 2024-08-07T19:10:07.2332833Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b8e133698259b0b5.xml (deflated 35%) 2024-08-07T19:10:07.2334128Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-dc9aaeff2728d8ba.xml (deflated 35%) 2024-08-07T19:10:07.2335437Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7bdb33d21e035475.xml (deflated 35%) 2024-08-07T19:10:07.2336751Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f420fc6a05d131d9.xml (deflated 35%) 2024-08-07T19:10:07.2338072Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ee5eaa48b0dbbdaa.xml (deflated 35%) 2024-08-07T19:10:07.2339380Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-576f0dba2694c470.xml (deflated 35%) 2024-08-07T19:10:07.2340685Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a793a0841f847e3a.xml (deflated 35%) 2024-08-07T19:10:07.2341995Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2ca7f3b9784d2c6b.xml (deflated 35%) 2024-08-07T19:10:07.2343304Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-96b5033ef8171f78.xml (deflated 34%) 2024-08-07T19:10:07.2344725Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c4e5806b6310770e.xml (deflated 34%) 2024-08-07T19:10:07.2346043Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-55b5523b68673aad.xml (deflated 34%) 2024-08-07T19:10:07.2347358Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5c5072e1bbe05d05.xml (deflated 34%) 2024-08-07T19:10:07.2348673Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-eadb233e80b3778a.xml (deflated 35%) 2024-08-07T19:10:07.2349972Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1476e2f1db119522.xml (deflated 35%) 2024-08-07T19:10:07.2351283Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-803443b9d0f59618.xml (deflated 35%) 2024-08-07T19:10:07.2352598Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3f4395d136f29cda.xml (deflated 34%) 2024-08-07T19:10:07.2353989Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5ad1bb8d71f9d898.xml (deflated 34%) 2024-08-07T19:10:07.2355297Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-73c4000ebbce4080.xml (deflated 34%) 2024-08-07T19:10:07.2356608Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ae49b7610b5de2e3.xml (deflated 35%) 2024-08-07T19:10:07.2357931Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7dc8a9e8587e3baf.xml (deflated 35%) 2024-08-07T19:10:07.2359255Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2b98f3f03687791a.xml (deflated 35%) 2024-08-07T19:10:07.2360565Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b25438b94db81cd0.xml (deflated 34%) 2024-08-07T19:10:07.2361877Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1970ca455e3b7397.xml (deflated 35%) 2024-08-07T19:10:07.2363564Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-dce83cfbbe22d914.xml (deflated 34%) 2024-08-07T19:10:07.2364901Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fe459aaac27fe11e.xml (deflated 34%) 2024-08-07T19:10:07.2366201Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ddc7e834185634cb.xml (deflated 35%) 2024-08-07T19:10:07.2367525Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-95de23bc88360eb0.xml (deflated 35%) 2024-08-07T19:10:07.2368834Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-164875fc9073e975.xml (deflated 34%) 2024-08-07T19:10:07.2370146Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8c1ac01fffbc3c1c.xml (deflated 34%) 2024-08-07T19:10:07.2371444Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8260e84760c98154.xml (deflated 35%) 2024-08-07T19:10:07.2372758Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-353b7ba9e8e5eef2.xml (deflated 35%) 2024-08-07T19:10:07.2374270Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-77a8e52064c17cb3.xml (deflated 35%) 2024-08-07T19:10:07.2375677Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ef2d052c0fa7382c.xml (deflated 35%) 2024-08-07T19:10:07.2376980Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7ac72fa76ad16f0b.xml (deflated 35%) 2024-08-07T19:10:07.2378295Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0b85af0a1744d860.xml (deflated 35%) 2024-08-07T19:10:07.2379605Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-029b4ba0f0efd5bc.xml (deflated 35%) 2024-08-07T19:10:07.2380922Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7fe14e18da520274.xml (deflated 34%) 2024-08-07T19:10:07.2382230Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-88ca1d7d5c2578bf.xml (deflated 34%) 2024-08-07T19:10:07.2383555Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a1b91cfccf2d1ddf.xml (deflated 34%) 2024-08-07T19:10:07.2384874Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9f503fa659b70b3e.xml (deflated 35%) 2024-08-07T19:10:07.2386179Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-22358722b01b4600.xml (deflated 35%) 2024-08-07T19:10:07.2387477Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-552ed376de47578a.xml (deflated 35%) 2024-08-07T19:10:07.2388786Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a6955ebd953772fd.xml (deflated 34%) 2024-08-07T19:10:07.2407555Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-13fb820db722468d.xml (deflated 34%) 2024-08-07T19:10:07.2408890Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f8dfd4918ca76907.xml (deflated 36%) 2024-08-07T19:10:07.2410202Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a57a3d198ba8cd9d.xml (deflated 35%) 2024-08-07T19:10:07.2411497Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-30bca433bb066b1f.xml (deflated 34%) 2024-08-07T19:10:07.2412797Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4f24941d02473792.xml (deflated 35%) 2024-08-07T19:10:07.2414113Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2ddad79cf8c53e75.xml (deflated 35%) 2024-08-07T19:10:07.2415423Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7e2c876fa4bf7bb5.xml (deflated 35%) 2024-08-07T19:10:07.2416717Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-363d62a434bd9d33.xml (deflated 35%) 2024-08-07T19:10:07.2418019Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-54958e8daffef3b2.xml (deflated 35%) 2024-08-07T19:10:07.2419312Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ff2456b51089d06a.xml (deflated 36%) 2024-08-07T19:10:07.2420726Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d4bf7b69b9cf701d.xml (deflated 36%) 2024-08-07T19:10:07.2422101Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9578a5aeb53e15e4.xml (deflated 36%) 2024-08-07T19:10:07.2423403Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-daf29728f1e2c6c3.xml (deflated 37%) 2024-08-07T19:10:07.2424695Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ff01cc555a92ccb8.xml (deflated 36%) 2024-08-07T19:10:07.2426003Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e9ed335c07b49620.xml (deflated 36%) 2024-08-07T19:10:07.2427280Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b0907a2bc688185a.xml (deflated 36%) 2024-08-07T19:10:07.2428584Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-38f6282d844ecb36.xml (deflated 38%) 2024-08-07T19:10:07.2429884Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c047ddad08908715.xml (deflated 38%) 2024-08-07T19:10:07.2431176Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-02d74fa24382d98a.xml (deflated 36%) 2024-08-07T19:10:07.2432471Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5a17b6c29778e6da.xml (deflated 36%) 2024-08-07T19:10:07.2433850Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-25176d73627867e4.xml (deflated 36%) 2024-08-07T19:10:07.2435148Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e239d54cec59371b.xml (deflated 36%) 2024-08-07T19:10:07.2436449Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4fd801404a8fea4d.xml (deflated 36%) 2024-08-07T19:10:07.2437743Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-28641a3b8357d301.xml (deflated 36%) 2024-08-07T19:10:07.2439037Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7fd9d56540a2e573.xml (deflated 37%) 2024-08-07T19:10:07.2440336Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e8e2b242d76f0d16.xml (deflated 37%) 2024-08-07T19:10:07.2441614Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7725a5510d0934cf.xml (deflated 36%) 2024-08-07T19:10:07.2442914Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9b0f242bb86935b7.xml (deflated 37%) 2024-08-07T19:10:07.2444224Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8d5c7ba0d0b2e33f.xml (deflated 36%) 2024-08-07T19:10:07.2445574Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_multiple_forward/distributed.fsdp.test_fsdp_multiple_forward-3800041feeccc7b0.xml (deflated 46%) 2024-08-07T19:10:07.2447026Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_tensor_ops/distributed._shard.sharded_tensor.ops.test_tensor_ops-182763aa26616508.xml (deflated 83%) 2024-08-07T19:10:07.2448345Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-35fbd03b63dbea89.xml (deflated 34%) 2024-08-07T19:10:07.2449447Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a18ffeba7c163b5c.xml (deflated 37%) 2024-08-07T19:10:07.2450610Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-aa40e1f0f4d3c1db.xml (deflated 34%) 2024-08-07T19:10:07.2451760Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-d2fc821b9ff72a12.xml (deflated 42%) 2024-08-07T19:10:07.2452861Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-eb6012a9a6013f76.xml (deflated 35%) 2024-08-07T19:10:07.2453964Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f83aed9b621eb0b0.xml (deflated 35%) 2024-08-07T19:10:07.2455079Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-14debc268aac9f1b.xml (deflated 36%) 2024-08-07T19:10:07.2456166Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e4cab9000597aff0.xml (deflated 35%) 2024-08-07T19:10:07.2457272Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2f3649a1f69b19a9.xml (deflated 35%) 2024-08-07T19:10:07.2458380Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-ed320aa9c15cb88a.xml (deflated 35%) 2024-08-07T19:10:07.2459474Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-1345d2e039985e26.xml (deflated 43%) 2024-08-07T19:10:07.2460582Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-aef94c4b9c7de1ff.xml (deflated 35%) 2024-08-07T19:10:07.2461687Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-700a6fadb1fd4d82.xml (deflated 43%) 2024-08-07T19:10:07.2462788Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-d3ba115b5f79a41d.xml (deflated 35%) 2024-08-07T19:10:07.2464094Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-549abab25f7b1ed1.xml (deflated 36%) 2024-08-07T19:10:07.2465215Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c5efee772e663a11.xml (deflated 36%) 2024-08-07T19:10:07.2466320Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0970c7681f7ea7a0.xml (deflated 35%) 2024-08-07T19:10:07.2467426Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-db8eff2d5e3da727.xml (deflated 43%) 2024-08-07T19:10:07.2468516Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-41bd836627ffc7e3.xml (deflated 35%) 2024-08-07T19:10:07.2469627Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e50c7fba1312cd9c.xml (deflated 35%) 2024-08-07T19:10:07.2470728Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b8fa778c1550c8c0.xml (deflated 43%) 2024-08-07T19:10:07.2471817Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-606bce507c84d1d4.xml (deflated 35%) 2024-08-07T19:10:07.2472989Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b1624a6fcc8a592e.xml (deflated 36%) 2024-08-07T19:10:07.2474095Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9dad0ed75712cc12.xml (deflated 42%) 2024-08-07T19:10:07.2475202Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-02a41654cf5935dd.xml (deflated 35%) 2024-08-07T19:10:07.2476287Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-634387843d283ed9.xml (deflated 35%) 2024-08-07T19:10:07.2477383Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9cea7501afa5b856.xml (deflated 43%) 2024-08-07T19:10:07.2478595Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-58c35f868c4c2999.xml (deflated 35%) 2024-08-07T19:10:07.2479769Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b13224d2a2985954.xml (deflated 36%) 2024-08-07T19:10:07.2480855Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a320b7a8a42e1eaa.xml (deflated 42%) 2024-08-07T19:10:07.2481966Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-11492e3d062cec91.xml (deflated 35%) 2024-08-07T19:10:07.2483063Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-648bb2212676fa7a.xml (deflated 35%) 2024-08-07T19:10:07.2484147Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-047e4703a9fc773b.xml (deflated 35%) 2024-08-07T19:10:07.2485244Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-952e980cfec8a5b8.xml (deflated 43%) 2024-08-07T19:10:07.2486342Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4cd1992b10800c04.xml (deflated 35%) 2024-08-07T19:10:07.2487451Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e9fe6754e68f75f0.xml (deflated 35%) 2024-08-07T19:10:07.2488546Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-5508b6aadc163c9a.xml (deflated 43%) 2024-08-07T19:10:07.2489644Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f9f0e829237e6a66.xml (deflated 35%) 2024-08-07T19:10:07.2490741Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-19170d37e4c1227c.xml (deflated 36%) 2024-08-07T19:10:07.2491837Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4352f3480be2ba00.xml (deflated 35%) 2024-08-07T19:10:07.2492928Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3e422e13b44e4ec5.xml (deflated 35%) 2024-08-07T19:10:07.2494032Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9c8e136b4c31abd5.xml (deflated 42%) 2024-08-07T19:10:07.2495137Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3ee401f05b4197ab.xml (deflated 35%) 2024-08-07T19:10:07.2496232Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8ec4db956eaaec6e.xml (deflated 36%) 2024-08-07T19:10:07.2497330Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8459184190b65ac9.xml (deflated 36%) 2024-08-07T19:10:07.2498423Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-1c21b130afc162f1.xml (deflated 43%) 2024-08-07T19:10:07.2499525Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f61d7774e663849b.xml (deflated 35%) 2024-08-07T19:10:07.2500618Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a9fa6eaa8d051a1f.xml (deflated 36%) 2024-08-07T19:10:07.2501723Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4ee5d0842349a70e.xml (deflated 47%) 2024-08-07T19:10:07.2502815Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-5bb11f65eb88a519.xml (deflated 35%) 2024-08-07T19:10:07.2503917Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-008e6ff50a93263c.xml (deflated 57%) 2024-08-07T19:10:07.2505009Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-1dd9cadd4ccbe5e9.xml (deflated 35%) 2024-08-07T19:10:07.2506121Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-bf6c8a8f8f3b0e79.xml (deflated 43%) 2024-08-07T19:10:07.2507287Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e3e13b48a23ec312.xml (deflated 35%) 2024-08-07T19:10:07.2508436Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e45367c2fa49ac65.xml (deflated 43%) 2024-08-07T19:10:07.2509525Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-d79d41a03e7131a5.xml (deflated 43%) 2024-08-07T19:10:07.2510626Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f7c66c0138c5af53.xml (deflated 43%) 2024-08-07T19:10:07.2511728Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8b9057c74dc5390e.xml (deflated 42%) 2024-08-07T19:10:07.2512875Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-599d548ecb5f518d.xml (deflated 42%) 2024-08-07T19:10:07.2514048Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9910de89f06f6afc.xml (deflated 42%) 2024-08-07T19:10:07.2515146Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-610ca85686f90586.xml (deflated 42%) 2024-08-07T19:10:07.2516252Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9e00e78ed30daa5f.xml (deflated 42%) 2024-08-07T19:10:07.2517349Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-56cc2098f88f7a26.xml (deflated 42%) 2024-08-07T19:10:07.2518453Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e00a7e6d49482683.xml (deflated 42%) 2024-08-07T19:10:07.2519557Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-12e9265048d3d04f.xml (deflated 42%) 2024-08-07T19:10:07.2520660Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-48481f41d20172ba.xml (deflated 42%) 2024-08-07T19:10:07.2521750Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2408c96fd95565e6.xml (deflated 42%) 2024-08-07T19:10:07.2522859Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a9e09ac37d52f1fb.xml (deflated 42%) 2024-08-07T19:10:07.2523961Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3dea424242f9ced7.xml (deflated 42%) 2024-08-07T19:10:07.2525051Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9865bf6950479eba.xml (deflated 42%) 2024-08-07T19:10:07.2526152Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4b837e69446a51c6.xml (deflated 36%) 2024-08-07T19:10:07.2527258Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-ed48bb0be9a2cf47.xml (deflated 43%) 2024-08-07T19:10:07.2528366Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-34e93412736ac719.xml (deflated 36%) 2024-08-07T19:10:07.2529458Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-006c2a942a2c2e9f.xml (deflated 36%) 2024-08-07T19:10:07.2530559Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-fe91803279067ec9.xml (deflated 36%) 2024-08-07T19:10:07.2531675Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-db0c0dd3410805e2.xml (deflated 36%) 2024-08-07T19:10:07.2532782Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-cfd855a37e8de5bb.xml (deflated 45%) 2024-08-07T19:10:07.2533873Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-385963c25c9cb041.xml (deflated 43%) 2024-08-07T19:10:07.2535056Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a712f53af5473a0f.xml (deflated 44%) 2024-08-07T19:10:07.2536207Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-78965d7f28cd8d7c.xml (deflated 43%) 2024-08-07T19:10:07.2537296Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c1f50eb5358431ee.xml (deflated 44%) 2024-08-07T19:10:07.2538404Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8e5422938e0f9f28.xml (deflated 44%) 2024-08-07T19:10:07.2539496Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b8353f1037f7668e.xml (deflated 44%) 2024-08-07T19:10:07.2540589Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-847a533dc2e88076.xml (deflated 44%) 2024-08-07T19:10:07.2541678Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-fc36f93bbb2401c0.xml (deflated 36%) 2024-08-07T19:10:07.2542778Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-5f2f2c4c73286ce6.xml (deflated 36%) 2024-08-07T19:10:07.2543880Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e06bf688059dd651.xml (deflated 36%) 2024-08-07T19:10:07.2545001Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-5f9b1b72e7bacca1.xml (deflated 36%) 2024-08-07T19:10:07.2546087Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-67766de36a18038a.xml (deflated 44%) 2024-08-07T19:10:07.2547187Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3300e89a8cf68baa.xml (deflated 36%) 2024-08-07T19:10:07.2548294Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-30bfe50d71cc69fa.xml (deflated 44%) 2024-08-07T19:10:07.2549386Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-ff716ac593614090.xml (deflated 37%) 2024-08-07T19:10:07.2550487Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-06ef892254c87b1e.xml (deflated 37%) 2024-08-07T19:10:07.2551589Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a24540c9362471fc.xml (deflated 42%) 2024-08-07T19:10:07.2552683Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-84315ca6c3da8d70.xml (deflated 42%) 2024-08-07T19:10:07.2553846Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-dac51b3256baf241.xml (deflated 36%) 2024-08-07T19:10:07.2554954Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2cdb525a776f157a.xml (deflated 36%) 2024-08-07T19:10:07.2556054Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-1e69c0b438d78a58.xml (deflated 36%) 2024-08-07T19:10:07.2557160Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f0a0f17d1a20307d.xml (deflated 36%) 2024-08-07T19:10:07.2558254Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-93ae567b58fe66b5.xml (deflated 36%) 2024-08-07T19:10:07.2559364Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-77f5d0dcb38ddfde.xml (deflated 37%) 2024-08-07T19:10:07.2560472Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4daa952da3aa73c0.xml (deflated 36%) 2024-08-07T19:10:07.2561570Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2d4ed7abdd8108b9.xml (deflated 37%) 2024-08-07T19:10:07.2562672Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c70d64e9c18f9b01.xml (deflated 44%) 2024-08-07T19:10:07.2564098Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-94c61c60e638bca2.xml (deflated 44%) 2024-08-07T19:10:07.2565213Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-1b443c79253bc25b.xml (deflated 43%) 2024-08-07T19:10:07.2566300Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-dd186d7b7f012e70.xml (deflated 43%) 2024-08-07T19:10:07.2567408Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8677a509c2f4aa7b.xml (deflated 43%) 2024-08-07T19:10:07.2568517Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c2db0d4636912920.xml (deflated 44%) 2024-08-07T19:10:07.2569621Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3a749b6056556346.xml (deflated 43%) 2024-08-07T19:10:07.2570709Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-fe2534933f312a34.xml (deflated 43%) 2024-08-07T19:10:07.2571810Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0d5c84236abbfaf3.xml (deflated 43%) 2024-08-07T19:10:07.2572924Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-675b891442819118.xml (deflated 34%) 2024-08-07T19:10:07.2574410Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-513594f5f1a3b667.xml (deflated 34%) 2024-08-07T19:10:07.2575561Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f014c56869ff57bd.xml (deflated 34%) 2024-08-07T19:10:07.2576707Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-205d0b9a986c3a2d.xml (deflated 35%) 2024-08-07T19:10:07.2577813Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-01a92848c7480c6e.xml (deflated 35%) 2024-08-07T19:10:07.2578896Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-14e5de28b0059364.xml (deflated 34%) 2024-08-07T19:10:07.2579997Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8ed346a429e96328.xml (deflated 33%) 2024-08-07T19:10:07.2581096Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0ef253485a3d7f5b.xml (deflated 34%) 2024-08-07T19:10:07.2582201Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2ff9b1030fe696ee.xml (deflated 34%) 2024-08-07T19:10:07.2583290Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b9899a140f0920b1.xml (deflated 34%) 2024-08-07T19:10:07.2584456Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0d0b0bed44f091aa.xml (deflated 57%) 2024-08-07T19:10:07.2585676Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-76118cefc91275bd.xml (deflated 42%) 2024-08-07T19:10:07.2586882Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f0a7b43713759760.xml (deflated 41%) 2024-08-07T19:10:07.2588086Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b8456d13cabeec27.xml (deflated 42%) 2024-08-07T19:10:07.2589290Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9f35309f7d3d43a1.xml (deflated 43%) 2024-08-07T19:10:07.2590492Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-564d966322862c7e.xml (deflated 42%) 2024-08-07T19:10:07.2591682Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3376a59ce9b24bc8.xml (deflated 35%) 2024-08-07T19:10:07.2593086Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-928ad594565fc856.xml (deflated 36%) 2024-08-07T19:10:07.2594290Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0ce2336358106bad.xml (deflated 35%) 2024-08-07T19:10:07.2595499Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-35a088159ecfa3af.xml (deflated 35%) 2024-08-07T19:10:07.2596699Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fb1990bdfdc98d67.xml (deflated 35%) 2024-08-07T19:10:07.2597909Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7ad7476d49be1a71.xml (deflated 43%) 2024-08-07T19:10:07.2599139Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1bbd177f0f2f1bba.xml (deflated 44%) 2024-08-07T19:10:07.2600361Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c5c85e8b25e545b7.xml (deflated 36%) 2024-08-07T19:10:07.2601566Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e4fa85d4d068f182.xml (deflated 35%) 2024-08-07T19:10:07.2602786Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-752c17d6d5c74594.xml (deflated 35%) 2024-08-07T19:10:07.2603992Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d1af58d459eeff1f.xml (deflated 35%) 2024-08-07T19:10:07.2605203Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-06cb8096a40a205b.xml (deflated 35%) 2024-08-07T19:10:07.2606407Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c614e1417a18f587.xml (deflated 41%) 2024-08-07T19:10:07.2607620Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0a19f5fc981d194e.xml (deflated 43%) 2024-08-07T19:10:07.2608833Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4e49bf9e5ccbf71a.xml (deflated 43%) 2024-08-07T19:10:07.2610041Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7aba6e658507b576.xml (deflated 43%) 2024-08-07T19:10:07.2611236Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8619d502079eb4ec.xml (deflated 44%) 2024-08-07T19:10:07.2612447Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2f545cd529a8c5dc.xml (deflated 44%) 2024-08-07T19:10:07.2613669Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b6c3840edbae5434.xml (deflated 44%) 2024-08-07T19:10:07.2614881Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8cf5eb6737bc00a8.xml (deflated 44%) 2024-08-07T19:10:07.2616097Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b57711e6a6431cbc.xml (deflated 44%) 2024-08-07T19:10:07.2617310Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ad2ce550d6965bc3.xml (deflated 44%) 2024-08-07T19:10:07.2618517Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b2dc3eb497164d81.xml (deflated 44%) 2024-08-07T19:10:07.2619719Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ddb0e99d52aa0cd9.xml (deflated 42%) 2024-08-07T19:10:07.2620975Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1e70c008eca47480.xml (deflated 35%) 2024-08-07T19:10:07.2622218Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1956cd5ae4878913.xml (deflated 50%) 2024-08-07T19:10:07.2623414Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1246ebaf77f12582.xml (deflated 42%) 2024-08-07T19:10:07.2624607Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-50c96aeaf0c69f7e.xml (deflated 42%) 2024-08-07T19:10:07.2625803Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-99f91c183ae69ddd.xml (deflated 42%) 2024-08-07T19:10:07.2627009Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-531cea136e298f7f.xml (deflated 35%) 2024-08-07T19:10:07.2628208Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c3759d4321e435e4.xml (deflated 42%) 2024-08-07T19:10:07.2629401Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5f63dfd4030c898a.xml (deflated 56%) 2024-08-07T19:10:07.2630600Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9fd848861006fee4.xml (deflated 42%) 2024-08-07T19:10:07.2631807Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4b12b9570b19f20a.xml (deflated 42%) 2024-08-07T19:10:07.2633066Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cbdadb4035b97205.xml (deflated 42%) 2024-08-07T19:10:07.2634264Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e223f9bc4370f41f.xml (deflated 56%) 2024-08-07T19:10:07.2635465Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-737318a99ab5f37b.xml (deflated 42%) 2024-08-07T19:10:07.2636666Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d18490a693552b35.xml (deflated 48%) 2024-08-07T19:10:07.2637867Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c659cda27f6483e3.xml (deflated 42%) 2024-08-07T19:10:07.2639058Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-91730d723e515ab4.xml (deflated 42%) 2024-08-07T19:10:07.2640250Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-54f9e32aa598c857.xml (deflated 42%) 2024-08-07T19:10:07.2641455Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a30791993e995409.xml (deflated 41%) 2024-08-07T19:10:07.2642649Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0df8aa2ba58ddbae.xml (deflated 42%) 2024-08-07T19:10:07.2643849Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d079588d6f676459.xml (deflated 42%) 2024-08-07T19:10:07.2645049Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b415320428a4c2be.xml (deflated 42%) 2024-08-07T19:10:07.2646231Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8c0a77ea043707aa.xml (deflated 44%) 2024-08-07T19:10:07.2647430Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3fbbb65baa9a1acb.xml (deflated 43%) 2024-08-07T19:10:07.2648639Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b17d3f5a117e3366.xml (deflated 42%) 2024-08-07T19:10:07.2649910Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a140e7c3f2702a11.xml (deflated 47%) 2024-08-07T19:10:07.2651198Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-654f74887ae6ba79.xml (deflated 42%) 2024-08-07T19:10:07.2652383Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e937e32a4067186d.xml (deflated 41%) 2024-08-07T19:10:07.2653585Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1199dfa2e50876a8.xml (deflated 42%) 2024-08-07T19:10:07.2654787Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-900e6ce768e71f72.xml (deflated 35%) 2024-08-07T19:10:07.2655976Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f92dc19fc441267a.xml (deflated 36%) 2024-08-07T19:10:07.2657178Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bdaee3bb68d18c4f.xml (deflated 42%) 2024-08-07T19:10:07.2658383Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-61a879d1c44a2ef7.xml (deflated 35%) 2024-08-07T19:10:07.2659578Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3ace409f3a95127e.xml (deflated 43%) 2024-08-07T19:10:07.2660768Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5f3754078644f91d.xml (deflated 35%) 2024-08-07T19:10:07.2661958Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d0e30a09d9848733.xml (deflated 36%) 2024-08-07T19:10:07.2663343Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-22d3f0564def8f2d.xml (deflated 34%) 2024-08-07T19:10:07.2664539Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-84431532435a1aa9.xml (deflated 36%) 2024-08-07T19:10:07.2665737Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f86fda4b86e3b81e.xml (deflated 37%) 2024-08-07T19:10:07.2666941Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-26b96b8847d78980.xml (deflated 35%) 2024-08-07T19:10:07.2668134Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1c499c29f71d8822.xml (deflated 42%) 2024-08-07T19:10:07.2669326Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2c9b783d97c4f219.xml (deflated 35%) 2024-08-07T19:10:07.2670508Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3d6dff20707921a1.xml (deflated 45%) 2024-08-07T19:10:07.2671706Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2d729514ee9b5d4c.xml (deflated 43%) 2024-08-07T19:10:07.2672957Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-12d6d04eac7bf9f4.xml (deflated 43%) 2024-08-07T19:10:07.2674152Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8c3c7a41d41d42e1.xml (deflated 43%) 2024-08-07T19:10:07.2675333Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e9d8f715c56a640d.xml (deflated 42%) 2024-08-07T19:10:07.2676538Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c4daf9fe4fd4d934.xml (deflated 42%) 2024-08-07T19:10:07.2677747Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7dd025faad0ac963.xml (deflated 41%) 2024-08-07T19:10:07.2679109Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-30b6834a759e28c7.xml (deflated 56%) 2024-08-07T19:10:07.2680305Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9ddcb95eeab04c63.xml (deflated 35%) 2024-08-07T19:10:07.2681511Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0f2e9deaa02c3eaa.xml (deflated 36%) 2024-08-07T19:10:07.2682716Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-399dddd03249feca.xml (deflated 35%) 2024-08-07T19:10:07.2683916Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-56941ba8543e0057.xml (deflated 36%) 2024-08-07T19:10:07.2685111Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-001dfb6fc1e489d4.xml (deflated 45%) 2024-08-07T19:10:07.2686323Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4dd499f2422a42e7.xml (deflated 36%) 2024-08-07T19:10:07.2687519Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-30c7be559ba24a86.xml (deflated 36%) 2024-08-07T19:10:07.2688713Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-33becc4a9c7b9cac.xml (deflated 35%) 2024-08-07T19:10:07.2689920Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-abd6052b283ae127.xml (deflated 35%) 2024-08-07T19:10:07.2691120Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-05759808bc993d88.xml (deflated 35%) 2024-08-07T19:10:07.2692312Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2521baf1f87cc98b.xml (deflated 35%) 2024-08-07T19:10:07.2693507Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3d8104d4ff31612d.xml (deflated 42%) 2024-08-07T19:10:07.2694699Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-73e8096a3492bbee.xml (deflated 42%) 2024-08-07T19:10:07.2695896Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-950b9c1367ad34b6.xml (deflated 57%) 2024-08-07T19:10:07.2697097Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-12718872b68b369e.xml (deflated 42%) 2024-08-07T19:10:07.2698284Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b0ae4453dd193306.xml (deflated 41%) 2024-08-07T19:10:07.2699484Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e1b17cc3fc263e21.xml (deflated 42%) 2024-08-07T19:10:07.2700683Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0b3bac6f97602864.xml (deflated 43%) 2024-08-07T19:10:07.2701882Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f3f73e3d81bf543a.xml (deflated 42%) 2024-08-07T19:10:07.2703077Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7c1f243ed9322150.xml (deflated 35%) 2024-08-07T19:10:07.2704284Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ee40bbf127ff44ab.xml (deflated 35%) 2024-08-07T19:10:07.2705491Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6170ad32319fb751.xml (deflated 35%) 2024-08-07T19:10:07.2706734Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-eea63c7bcfac45fa.xml (deflated 35%) 2024-08-07T19:10:07.2707977Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-18b4c775070c6ebb.xml (deflated 35%) 2024-08-07T19:10:07.2709173Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7eb900f78ae9c396.xml (deflated 43%) 2024-08-07T19:10:07.2710377Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d626abe9a34d0a5b.xml (deflated 44%) 2024-08-07T19:10:07.2711566Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7755a20924b09328.xml (deflated 36%) 2024-08-07T19:10:07.2712760Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-baa7e4e559d7e2ed.xml (deflated 35%) 2024-08-07T19:10:07.2714028Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8a86e56939a7bf31.xml (deflated 35%) 2024-08-07T19:10:07.2715230Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-37edc79271ade527.xml (deflated 35%) 2024-08-07T19:10:07.2716425Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e382b1ce4d2cc3c0.xml (deflated 35%) 2024-08-07T19:10:07.2717623Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-59332d70a44ac2a6.xml (deflated 42%) 2024-08-07T19:10:07.2718819Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c03fb90965fd139c.xml (deflated 43%) 2024-08-07T19:10:07.2720024Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0471a46f67a29ade.xml (deflated 43%) 2024-08-07T19:10:07.2721209Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d7e42b7c4420adb1.xml (deflated 43%) 2024-08-07T19:10:07.2722419Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d27a666bb2f137e8.xml (deflated 44%) 2024-08-07T19:10:07.2723624Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4d54a0f9bca4a492.xml (deflated 45%) 2024-08-07T19:10:07.2724815Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-72977cea40f4ae8e.xml (deflated 44%) 2024-08-07T19:10:07.2725997Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cceda99809e59d6d.xml (deflated 44%) 2024-08-07T19:10:07.2727192Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-961a150c45eea3d2.xml (deflated 44%) 2024-08-07T19:10:07.2728389Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-10e3911453c46012.xml (deflated 44%) 2024-08-07T19:10:07.2729585Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9d48de661a0861c0.xml (deflated 44%) 2024-08-07T19:10:07.2730772Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a3b852c0deb1a0b0.xml (deflated 43%) 2024-08-07T19:10:07.2731978Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-dce47131e6ae073c.xml (deflated 35%) 2024-08-07T19:10:07.2733168Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f85087794852aa6c.xml (deflated 50%) 2024-08-07T19:10:07.2734559Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-06b0337a2c5f6348.xml (deflated 42%) 2024-08-07T19:10:07.2735905Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a5da7274c9db8e6a.xml (deflated 42%) 2024-08-07T19:10:07.2737174Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ad3544c36b1f3eb1.xml (deflated 42%) 2024-08-07T19:10:07.2738372Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f6a79122c255e1a5.xml (deflated 36%) 2024-08-07T19:10:07.2739558Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4cd0bead9127b598.xml (deflated 42%) 2024-08-07T19:10:07.2740760Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c88d6a16c527c979.xml (deflated 56%) 2024-08-07T19:10:07.2741956Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-15bab8d6c82a4093.xml (deflated 42%) 2024-08-07T19:10:07.2743155Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-207820370855f78f.xml (deflated 42%) 2024-08-07T19:10:07.2744343Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-baea6eee950f5dd3.xml (deflated 42%) 2024-08-07T19:10:07.2745545Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f02c940538813e53.xml (deflated 56%) 2024-08-07T19:10:07.2746747Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-375f0a10cc57827b.xml (deflated 42%) 2024-08-07T19:10:07.2747946Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5be734c1aab2936c.xml (deflated 48%) 2024-08-07T19:10:07.2749134Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0f5798852c2b798b.xml (deflated 42%) 2024-08-07T19:10:07.2750337Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c910f14807a4b663.xml (deflated 42%) 2024-08-07T19:10:07.2751540Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e707ba0d75e670e2.xml (deflated 42%) 2024-08-07T19:10:07.2752736Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bd569f1716e84d11.xml (deflated 42%) 2024-08-07T19:10:07.2753990Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-06fd201eff54fbe3.xml (deflated 42%) 2024-08-07T19:10:07.2755190Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f508f777cf5964ca.xml (deflated 42%) 2024-08-07T19:10:07.2756399Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-44cc470df9e5ee39.xml (deflated 42%) 2024-08-07T19:10:07.2757602Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ed52823931be64a6.xml (deflated 44%) 2024-08-07T19:10:07.2758792Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-33f1457779b5af22.xml (deflated 43%) 2024-08-07T19:10:07.2759991Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-df32cbad0347708f.xml (deflated 42%) 2024-08-07T19:10:07.2761182Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-04f228eed13168af.xml (deflated 47%) 2024-08-07T19:10:07.2762369Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-91b0b93136507772.xml (deflated 42%) 2024-08-07T19:10:07.2763762Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-565685c267261436.xml (deflated 41%) 2024-08-07T19:10:07.2765121Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5b94f953fe56e308.xml (deflated 42%) 2024-08-07T19:10:07.2766340Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a0373dc65ed72ab8.xml (deflated 35%) 2024-08-07T19:10:07.2767538Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-804e5fee8da89bdb.xml (deflated 36%) 2024-08-07T19:10:07.2768747Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8da67fa63394a05f.xml (deflated 42%) 2024-08-07T19:10:07.2769959Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-379d791ff74c641c.xml (deflated 35%) 2024-08-07T19:10:07.2771160Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5aead967b1c4e71a.xml (deflated 43%) 2024-08-07T19:10:07.2772364Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5539a6f2385911b4.xml (deflated 36%) 2024-08-07T19:10:07.2773574Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-123366146d0e775a.xml (deflated 36%) 2024-08-07T19:10:07.2774793Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cc2fdaf0e5dccdb0.xml (deflated 35%) 2024-08-07T19:10:07.2776014Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-494ec98f7957fad6.xml (deflated 35%) 2024-08-07T19:10:07.2777213Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c0c86f1014509aac.xml (deflated 36%) 2024-08-07T19:10:07.2778432Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f6d06a6991d0aa08.xml (deflated 35%) 2024-08-07T19:10:07.2779639Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-629f5cb0690bb568.xml (deflated 42%) 2024-08-07T19:10:07.2780839Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ccfdd4ab6e39a6fd.xml (deflated 35%) 2024-08-07T19:10:07.2782055Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-22c9cd86711ac3c1.xml (deflated 45%) 2024-08-07T19:10:07.2783262Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cded252024f66c9d.xml (deflated 43%) 2024-08-07T19:10:07.2784469Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e79378586603ddcc.xml (deflated 42%) 2024-08-07T19:10:07.2785666Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bdcfe93ea8329c63.xml (deflated 43%) 2024-08-07T19:10:07.2786881Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f0bf3513ef2cf2e6.xml (deflated 42%) 2024-08-07T19:10:07.2788094Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c98928868650fa4d.xml (deflated 42%) 2024-08-07T19:10:07.2789301Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a3fe9306cb1d9369.xml (deflated 41%) 2024-08-07T19:10:07.2790502Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3fef22a8569b173a.xml (deflated 56%) 2024-08-07T19:10:07.2791720Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4a1ae2a81af5527a.xml (deflated 35%) 2024-08-07T19:10:07.2793035Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4d5ac8537311b73a.xml (deflated 36%) 2024-08-07T19:10:07.2794291Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5802ee735a1c0b7c.xml (deflated 35%) 2024-08-07T19:10:07.2795494Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1b298bfba4bd2789.xml (deflated 36%) 2024-08-07T19:10:07.2796716Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ecda950c5648e835.xml (deflated 44%) 2024-08-07T19:10:07.2797934Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e729e5472b116da4.xml (deflated 35%) 2024-08-07T19:10:07.2799138Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-25215e4e21e9d11f.xml (deflated 35%) 2024-08-07T19:10:07.2800336Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7c77476644d43238.xml (deflated 35%) 2024-08-07T19:10:07.2801556Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fb8e271c24bcac0b.xml (deflated 35%) 2024-08-07T19:10:07.2802762Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fc97e7ce3863a5c0.xml (deflated 35%) 2024-08-07T19:10:07.2803961Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c578e0a9e227f93d.xml (deflated 35%) 2024-08-07T19:10:07.2805179Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-52d22d3dbcb565f9.xml (deflated 42%) 2024-08-07T19:10:07.2806384Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-77068944d672591f.xml (deflated 42%) 2024-08-07T19:10:07.2835007Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-07T19:10:07.2835479Z # Remove any previous usage logs if they exist 2024-08-07T19:10:07.2835850Z rm -f logs-*.zip 2024-08-07T19:10:07.2836331Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-07T19:10:07.2836883Z # so check to see if the file exists first 2024-08-07T19:10:07.2837245Z if [ -f 'usage_log.txt' ]; then 2024-08-07T19:10:07.2837607Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-07T19:10:07.2837951Z fi 2024-08-07T19:10:07.2838213Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-07T19:10:07.2838603Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-07T19:10:07.2838957Z fi 2024-08-07T19:10:07.2844179Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:10:07.2844559Z env: 2024-08-07T19:10:07.2844779Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:10:07.2845249Z DOCKER_CONTAINER_ID: 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:07.2845869Z FILE_SUFFIX: test-distributed-2-2-amz2023.linux.2xlarge_28475634473 2024-08-07T19:10:07.2846307Z ##[endgroup] 2024-08-07T19:10:07.2936076Z adding: usage_log.txt (deflated 92%) 2024-08-07T19:10:07.3193314Z adding: test/test-reports/distributed._tools.test_mod_tracker_1.1_03e39533a79d5d40_.log (deflated 64%) 2024-08-07T19:10:07.3198197Z adding: test/test-reports/distributed._tensor.debug.test_op_coverage_1.1_10235cd5176eed36_.log (deflated 55%) 2024-08-07T19:10:07.4504381Z adding: test/test-reports/distributed._tensor.test_dtensor_ops_1.1_437f35955c6a12d3_.log (deflated 96%) 2024-08-07T19:10:07.4505328Z adding: test/test-reports/distributed._tools.test_mem_tracker_1.1_06f3da25b294a9b1_.log (deflated 62%) 2024-08-07T19:10:07.4506283Z adding: test/test-reports/distributed._tensor.test_convolution_ops_1.1_88ded636f87325d0_.log (deflated 62%) 2024-08-07T19:10:07.4507357Z adding: test/test-reports/distributed._tensor.test_attention_1.1_7376c0f65c051eda_.log (deflated 76%) 2024-08-07T19:10:07.4508573Z adding: test/test-reports/distributed._tensor.test_dtensor_1.1_49622edfae5ad27e_.log (deflated 90%) 2024-08-07T19:10:07.4509489Z adding: test/test-reports/distributed._tensor.test_api_1.1_2cf5e21a5ddca3c0_.log (deflated 89%) 2024-08-07T19:10:07.4510389Z adding: test/test-reports/distributed.pipelining.test_pipe_1.1_87f59d15033e9578_.log (deflated 62%) 2024-08-07T19:10:07.4511347Z adding: test/test-reports/distributed.test_c10d_functional_native_1.1_13fb73fe420082e7_.log (deflated 86%) 2024-08-07T19:10:07.4512349Z adding: test/test-reports/distributed._tensor.test_matrix_ops_1.1_e5cb3ad308fc5bdb_.log (deflated 88%) 2024-08-07T19:10:07.4513339Z adding: test/test-reports/distributed.fsdp.test_fsdp_fx_1.1_178c6c32c80c85d3_.log (deflated 53%) 2024-08-07T19:10:07.4514185Z adding: test/test-reports/distributed._tensor.test_experimental_ops_1.1_d348b7969cc9ca08_.log (deflated 74%) 2024-08-07T19:10:07.4515114Z adding: test/test-reports/distributed.test_c10d_pypg_1.1_903f24460f6ee61b_.log (deflated 90%) 2024-08-07T19:10:07.4516119Z adding: test/test-reports/distributed._tensor.test_common_rules_1.1_f956e4fb7cad15e5_.log (deflated 76%) 2024-08-07T19:10:07.4517109Z adding: test/test-reports/distributed.checkpoint.fsdp.test_fsdp_dsd_1.1_8bb5d7fad196763f_.log (deflated 67%) 2024-08-07T19:10:07.4518106Z adding: test/test-reports/distributed.checkpoint.test_dedup_tensors_1.1_32869f4aa790eba3_.log (deflated 52%) 2024-08-07T19:10:07.4519107Z adding: test/test-reports/distributed._tools.test_memory_tracker_1.1_607828d618a64a37_.log (deflated 54%) 2024-08-07T19:10:07.4519984Z adding: test/test-reports/distributed._tensor.test_init_1.1_22e3ca6ed7fcb653_.log (deflated 70%) 2024-08-07T19:10:07.4520890Z adding: test/test-reports/distributed.pipelining.test_stage_1.1_04f11a14b14e747b_.log (deflated 2%) 2024-08-07T19:10:07.4521814Z adding: test/test-reports/distributed.pipelining.test_unflatten_1.1_e18830072ddc1fb0_.log (deflated 54%) 2024-08-07T19:10:07.4522753Z adding: test/test-reports/distributed.checkpoint.e2e.test_fsdp_ep_1.1_c43f1ba888c0a73d_.log (deflated 54%) 2024-08-07T19:10:07.4523730Z adding: test/test-reports/distributed._tensor.test_xla_integration_1.1_8c57478fd0ba1683_.log (deflated 66%) 2024-08-07T19:10:07.4524713Z adding: test/test-reports/distributed.checkpoint.test_save_load_api_1.1_b987cf693df673de_.log (deflated 53%) 2024-08-07T19:10:07.4525675Z adding: test/test-reports/distributed.pipelining.test_microbatch_1.1_4c3c8fd929e4ea78_.log (deflated 58%) 2024-08-07T19:10:07.4526700Z adding: test/test-reports/distributed._tensor.debug.test_comm_mode_features_1.1_d3bb2389cd8c4476_.log (deflated 82%) 2024-08-07T19:10:07.4527718Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_state_1.1_194bd2c3d531e64c_.log (deflated 72%) 2024-08-07T19:10:07.4528698Z adding: test/test-reports/distributed._composable.test_checkpoint_1.1_ab802b5b200b2edd_.log (deflated 69%) 2024-08-07T19:10:07.4529703Z adding: test/test-reports/distributed.tensor.parallel.test_tp_style_1.1_9b52ffca6d02e276_.log (deflated 90%) 2024-08-07T19:10:07.4530614Z adding: test/test-reports/distributed.checkpoint.test_traverse_1.1_76d12ce56ff6070a_.log (deflated 71%) 2024-08-07T19:10:07.4531539Z adding: test/test-reports/distributed.algorithms.ddp_comm_hooks.test_ddp_hooks_1.1_2f6a3c55d4185a69_.log (deflated 76%) 2024-08-07T19:10:07.4532597Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_logging_1.1_bdb6a0478797f6a2_.log (deflated 53%) 2024-08-07T19:10:07.4533547Z adding: test/test-reports/distributed.test_compute_comm_reordering_1.1_8e3f53dc4769e27e_.log (deflated 76%) 2024-08-07T19:10:07.4534465Z adding: test/test-reports/distributed.tensor.parallel.test_tp_random_state_1.1_ec148801e3d0c5d9_.log (deflated 55%) 2024-08-07T19:10:07.4535452Z adding: test/test-reports/distributed._tools.test_fsdp2_mem_tracker_1.1_1d7ab7a8c32f70d5_.log (deflated 67%) 2024-08-07T19:10:07.4536451Z adding: test/test-reports/distributed.test_inductor_collectives_1.1_4f688a1e553d8707_.log (deflated 90%) 2024-08-07T19:10:07.4537529Z adding: test/test-reports/distributed.checkpoint.e2e.test_e2e_save_and_load_1.1_e6800c43e1d40a6f_.log (deflated 94%) 2024-08-07T19:10:07.4538466Z adding: test/test-reports/distributed.launcher.test_run_1.1_f04b5f308e297745_.log (deflated 89%) 2024-08-07T19:10:07.4539452Z adding: test/test-reports/distributed.checkpoint.test_file_system_checkpoint_cpu_1.1_2a88ee43e3b44bb8_.log (deflated 94%) 2024-08-07T19:10:07.4540596Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_compile_1.1_8e2569d7affe3e80_.log (deflated 82%) 2024-08-07T19:10:07.4541575Z adding: test/test-reports/distributed.fsdp.test_fsdp_apply_1.1_4a35827969bc828b_.log (deflated 63%) 2024-08-07T19:10:07.4542580Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_overlap_1.1_75a174308a3c950c_.log (deflated 63%) 2024-08-07T19:10:07.4543588Z adding: test/test-reports/distributed._tensor.test_view_ops_1.1_e6297a8bbc299d28_.log (deflated 81%) 2024-08-07T19:10:07.4544463Z adding: test/test-reports/distributed.checkpoint.e2e.test_fine_tuning_1.1_4d469253ed32f39e_.log (deflated 53%) 2024-08-07T19:10:07.4545458Z adding: test/test-reports/distributed.elastic.test_control_plane_1.1_93e7aaf2656b96f2_.log (deflated 73%) 2024-08-07T19:10:07.4546382Z adding: test/test-reports/distributed.checkpoint.test_planner_1.1_5f575488ed3709df_.log (deflated 67%) 2024-08-07T19:10:07.4547430Z adding: test/test-reports/distributed.checkpoint.test_dtensor_checkpoint_1.1_3c66b27e3ac3056d_.log (deflated 55%) 2024-08-07T19:10:07.4548415Z adding: test/test-reports/distributed.fsdp.test_fsdp_tp_integration_1.1_0020767bc8e6acc2_.log (deflated 67%) 2024-08-07T19:10:07.4549328Z adding: test/test-reports/distributed.fsdp.test_fsdp_comm_hooks_1.1_fbcd39eaa3831936_.log (deflated 89%) 2024-08-07T19:10:07.4550314Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_memory_1.1_453f3c698f7d7572_.log (deflated 55%) 2024-08-07T19:10:07.4551303Z adding: test/test-reports/distributed.checkpoint.e2e.test_pipeline_1.1_0747a1c5770b7eb5_.log (deflated 53%) 2024-08-07T19:10:07.4552229Z adding: test/test-reports/distributed._tensor.test_op_strategy_1.1_244c2a6459dd8010_.log (deflated 79%) 2024-08-07T19:10:07.4553279Z adding: test/test-reports/distributed.checkpoint.test_compatibility_1.1_f2c3c7fc8e2fe9fb_.log (deflated 63%) 2024-08-07T19:10:07.4554248Z adding: test/test-reports/distributed._tensor.test_embedding_ops_1.1_345116dddc823fdc_.log (deflated 84%) 2024-08-07T19:10:07.4555224Z adding: test/test-reports/distributed.fsdp.test_fsdp_ignored_modules_1.1_6b22c27396e6aca0_.log (deflated 85%) 2024-08-07T19:10:07.4556172Z adding: test/test-reports/distributed.checkpoint.test_hsdp_checkpoint_1.1_fc5612e41458e053_.log (deflated 71%) 2024-08-07T19:10:07.4557080Z adding: test/test-reports/distributed.pipelining.test_schedule_1.1_986cab85da7df3fc_.log (deflated 79%) 2024-08-07T19:10:07.4558091Z adding: test/test-reports/distributed.test_functional_api_1.1_687ff3d99d83df63_.log (deflated 86%) 2024-08-07T19:10:07.4558964Z adding: test/test-reports/distributed.fsdp.test_hsdp_dtensor_state_dict_1.1_14ac140802f74c7f_.log (deflated 79%) 2024-08-07T19:10:07.4559939Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_compile_1.1_970b30e3954589da_.log (deflated 56%) 2024-08-07T19:10:07.4560937Z adding: test/test-reports/distributed.checkpoint.test_state_dict_utils_1.1_b561fcc064884f20_.log (deflated 73%) 2024-08-07T19:10:07.4561842Z adding: test/test-reports/distributed._composable.test_replicate_1.1_004b78bb4139cee0_.log (deflated 74%) 2024-08-07T19:10:07.4562761Z adding: test/test-reports/distributed.optim.test_apply_optimizer_in_backward_1.1_ae19531074ad7647_.log (deflated 32%) 2024-08-07T19:10:07.4563799Z adding: test/test-reports/distributed.fsdp.test_wrap_1.1_203ddc14df198fd7_.log (deflated 90%) 2024-08-07T19:10:07.4564820Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_frozen_1.1_234f36918ffc22a5_.log (deflated 68%) 2024-08-07T19:10:07.4568042Z adding: test/test-reports/distributed._tensor.test_optimizers_1.1_8b69f7ec634d46b8_.log (deflated 95%) 2024-08-07T19:10:07.4568887Z adding: test/test-reports/distributed.test_distributed_spawn_2.3_da21afe04599d41a_.log (deflated 37%) 2024-08-07T19:10:07.4569732Z adding: test/test-reports/distributed.test_distributed_spawn_2.3_81c92bafc3a0f3d5_.log (deflated 37%) 2024-08-07T19:10:07.4584773Z adding: test/test-reports/distributed.test_distributed_spawn_2.3_c11380512544f2d7_.log (deflated 97%) 2024-08-07T19:10:07.4600986Z adding: test/test-reports/distributed.test_distributed_spawn_2.3_cbb3550ae9f426c8_.log (deflated 97%) 2024-08-07T19:10:07.4602035Z adding: test/test-reports/distributed.test_c10d_object_collectives_1.1_9cca1cb0db85cf13_.log (deflated 77%) 2024-08-07T19:10:07.4603116Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_mixed_precision_1.1_e77e74eff81e23c0_.log (deflated 58%) 2024-08-07T19:10:07.4604141Z adding: test/test-reports/distributed.test_c10d_ucc_1.1_e41a54995ed06ff9_.log (stored 0%) 2024-08-07T19:10:07.4605080Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_util_1.1_e483b7de3346e9f8_.log (deflated 57%) 2024-08-07T19:10:07.4606170Z adding: test/test-reports/distributed.fsdp.test_fsdp_hybrid_shard_1.1_ac2273a1a0ab8443_.log (deflated 73%) 2024-08-07T19:10:07.4607125Z adding: test/test-reports/distributed.fsdp.test_fsdp_dtensor_state_dict_1.1_b945de4c43c8d9e6_.log (deflated 86%) 2024-08-07T19:10:07.4608076Z adding: test/test-reports/distributed._composable.test_compose_1.1_0fa164432eedc3ab_.log (deflated 78%) 2024-08-07T19:10:07.4609055Z adding: test/test-reports/distributed.fsdp.test_distributed_checkpoint_1.1_18e3fe2b676bf48b_.log (deflated 65%) 2024-08-07T19:10:07.4610063Z adding: test/test-reports/distributed.fsdp.test_fsdp_clip_grad_norm_1.1_e2379f5948c911f5_.log (deflated 69%) 2024-08-07T19:10:07.4611013Z adding: test/test-reports/distributed.checkpoint.test_checkpoint_1.1_0e55c4721113d9c6_.log (deflated 81%) 2024-08-07T19:10:07.4611968Z adding: test/test-reports/distributed.fsdp.test_fsdp_sharded_grad_scaler_1.1_1c3fd8228698b117_.log (deflated 89%) 2024-08-07T19:10:07.4612854Z adding: test/test-reports/distributed.fsdp.test_fsdp_traversal_1.1_45902b8b9a587f9d_.log (deflated 53%) 2024-08-07T19:10:07.4613788Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_state_dict_1.1_e727a6fc39ff7893_.log (deflated 66%) 2024-08-07T19:10:07.4614703Z adding: test/test-reports/distributed._tensor.test_utils_1.1_5f293d0ca2099ff9_.log (deflated 87%) 2024-08-07T19:10:07.4615516Z adding: test/test-reports/distributed.fsdp.test_fsdp_pure_fp16_1.1_55909142b5732c67_.log (deflated 60%) 2024-08-07T19:10:07.4627275Z adding: test/test-reports/distributed.test_c10d_gloo_1.1_3e56c660a385de83_.log (deflated 96%) 2024-08-07T19:10:07.4628378Z adding: test/test-reports/distributed._composable.test_composability.test_2d_composability_1.1_0f881e2356d6427c_.log (deflated 68%) 2024-08-07T19:10:07.4629447Z adding: test/test-reports/distributed.fsdp.test_fsdp_overlap_1.1_3fa97357064c368b_.log (deflated 59%) 2024-08-07T19:10:07.4630343Z adding: test/test-reports/distributed.fsdp.test_fsdp_unshard_params_1.1_568ae831facfb107_.log (deflated 83%) 2024-08-07T19:10:07.4631469Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_runtime_1.1_8571869f1c2efdb0_.log (deflated 62%) 2024-08-07T19:10:07.4632526Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_mixed_precision_1.1_0062007bfc625135_.log (deflated 74%) 2024-08-07T19:10:07.4633722Z adding: test/test-reports/distributed.checkpoint.test_file_system_checkpoint_1.1_777de1a626d5c628_.log (deflated 73%) 2024-08-07T19:10:07.4634700Z adding: test/test-reports/distributed.fsdp.test_fsdp_mixed_precision_1.1_2015c497cb9d0f55_.log (deflated 92%) 2024-08-07T19:10:07.4635686Z adding: test/test-reports/distributed.nn.jit.test_instantiator_1.1_4141844af58a19f9_.log (deflated 64%) 2024-08-07T19:10:07.4642893Z adding: test/test-reports/distributed.fsdp.test_fsdp_state_dict_1.1_92b7942f53dabc59_.log (deflated 96%) 2024-08-07T19:10:07.4662484Z adding: test/test-reports/distributed.rpc.cuda.test_tensorpipe_agent_1.1_c0c9eef275b77b4f_.log (deflated 98%) 2024-08-07T19:10:07.4663861Z adding: test/test-reports/distributed._composable.test_replicate_with_compiler_1.1_e527452dabf21da8_.log (deflated 78%) 2024-08-07T19:10:07.4664922Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_extensions_1.1_50fb5dc53a18c6b8_.log (deflated 67%) 2024-08-07T19:10:07.4666126Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_model_checkpoint_1.1_1a80d15c7b26abf7_.log (deflated 73%) 2024-08-07T19:10:07.4667251Z adding: test/test-reports/distributed.fsdp.test_fsdp_use_orig_params_1.1_c954796d90552525_.log (deflated 86%) 2024-08-07T19:10:07.4668233Z adding: test/test-reports/distributed.fsdp.test_fsdp_flatten_params_1.1_b661b8924f4f352a_.log (deflated 79%) 2024-08-07T19:10:07.4669186Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_binary_cmp_1.1_93daaeb5f57d2743_.log (deflated 71%) 2024-08-07T19:10:07.4670308Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_init_1.1_d31cbb66502b1dc2_.log (deflated 75%) 2024-08-07T19:10:07.4671213Z adding: test/test-reports/distributed.test_pg_wrapper_1.1_d1b32ffd407d7859_.log (deflated 93%) 2024-08-07T19:10:07.4672193Z adding: test/test-reports/distributed.test_multi_threaded_pg_1.1_ecead697f38ef48d_.log (deflated 85%) 2024-08-07T19:10:07.4673145Z adding: test/test-reports/distributed.fsdp.test_fsdp_memory_1.1_07041a5640bb86f6_.log (deflated 59%) 2024-08-07T19:10:07.4674039Z adding: test/test-reports/distributed.fsdp.test_fsdp_grad_acc_1.1_4c927a02eb799cf0_.log (deflated 74%) 2024-08-07T19:10:07.4675062Z adding: test/test-reports/distributed._shard.sharding_spec.test_sharding_spec_1.1_553d445d9821125d_.log (deflated 78%) 2024-08-07T19:10:07.4676046Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_tensor_ops_1.1_fca5509c87e0a4a0_.log (deflated 71%) 2024-08-07T19:10:07.4677209Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint_1.1_8a46dd5800c05e9b_.log (deflated 65%) 2024-08-07T19:10:07.4700256Z adding: test/test-reports/distributed.rpc.test_tensorpipe_agent_1.4_307f52cc79e6fb1e_.log (deflated 97%) 2024-08-07T19:10:07.4739520Z adding: test/test-reports/distributed.rpc.test_tensorpipe_agent_3.4_bd612aa1aba76d1e_.log (deflated 97%) 2024-08-07T19:10:07.4740503Z adding: test/test-reports/distributed.fsdp.test_fsdp_multiple_forward_1.1_82efef342db19c0d_.log (deflated 56%) 2024-08-07T19:10:07.4776999Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-07T19:10:07.4777584Z # Remove any previous debugging artifacts if they exist 2024-08-07T19:10:07.4777995Z rm -f debug-*.zip 2024-08-07T19:10:07.4778293Z if [ -d 'test/debug' ]; then 2024-08-07T19:10:07.4778679Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-07T19:10:07.4779029Z fi 2024-08-07T19:10:07.4784586Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:10:07.4784976Z env: 2024-08-07T19:10:07.4785196Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:10:07.4785682Z DOCKER_CONTAINER_ID: 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:07.4786335Z FILE_SUFFIX: test-distributed-2-2-amz2023.linux.2xlarge_28475634473 2024-08-07T19:10:07.4786758Z ##[endgroup] 2024-08-07T19:10:07.4871325Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T19:10:07.4871684Z with: 2024-08-07T19:10:07.4871905Z s3-bucket: gha-artifacts 2024-08-07T19:10:07.4872244Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T19:10:07.4872723Z retention-days: 14 2024-08-07T19:10:07.4873075Z if-no-files-found: warn 2024-08-07T19:10:07.4873363Z path: test-jsons-*.zip 2024-08-07T19:10:07.4873634Z name: artifact 2024-08-07T19:10:07.4873861Z region: us-east-1 2024-08-07T19:10:07.4874100Z env: 2024-08-07T19:10:07.4874426Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:10:07.4874892Z DOCKER_CONTAINER_ID: 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:07.4875405Z ##[endgroup] 2024-08-07T19:10:07.8693021Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-07T19:10:07.8693564Z With the provided path, there will be 1 file uploaded 2024-08-07T19:10:07.8694038Z Uploading to s3 prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T19:10:07.9772751Z Starting upload of test-jsons-test-distributed-2-2-amz2023.linux.2xlarge_28475634473.zip 2024-08-07T19:10:08.1190510Z Finished upload of test-jsons-test-distributed-2-2-amz2023.linux.2xlarge_28475634473.zip 2024-08-07T19:10:08.1401678Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T19:10:08.1402055Z with: 2024-08-07T19:10:08.1402290Z s3-bucket: gha-artifacts 2024-08-07T19:10:08.1402614Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T19:10:08.1402977Z retention-days: 14 2024-08-07T19:10:08.1403257Z if-no-files-found: error 2024-08-07T19:10:08.1403533Z path: test-reports-*.zip 2024-08-07T19:10:08.1403808Z name: artifact 2024-08-07T19:10:08.1404042Z region: us-east-1 2024-08-07T19:10:08.1404268Z env: 2024-08-07T19:10:08.1404489Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:10:08.1404961Z DOCKER_CONTAINER_ID: 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:08.1405461Z ##[endgroup] 2024-08-07T19:10:08.4827676Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-07T19:10:08.4828476Z With the provided path, there will be 1 file uploaded 2024-08-07T19:10:08.4829203Z Uploading to s3 prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T19:10:08.4869661Z Starting upload of test-reports-test-distributed-2-2-amz2023.linux.2xlarge_28475634473.zip 2024-08-07T19:10:08.6678233Z Finished upload of test-reports-test-distributed-2-2-amz2023.linux.2xlarge_28475634473.zip 2024-08-07T19:10:08.6866998Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T19:10:08.6867377Z with: 2024-08-07T19:10:08.6867599Z s3-bucket: gha-artifacts 2024-08-07T19:10:08.6867929Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T19:10:08.6868291Z retention-days: 14 2024-08-07T19:10:08.6868548Z if-no-files-found: ignore 2024-08-07T19:10:08.6868827Z path: logs-*.zip 2024-08-07T19:10:08.6869070Z name: artifact 2024-08-07T19:10:08.6869294Z region: us-east-1 2024-08-07T19:10:08.6869533Z env: 2024-08-07T19:10:08.6869756Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:10:08.6870223Z DOCKER_CONTAINER_ID: 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:08.6870735Z ##[endgroup] 2024-08-07T19:10:09.0323230Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-07T19:10:09.0323910Z With the provided path, there will be 1 file uploaded 2024-08-07T19:10:09.0324375Z Uploading to s3 prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T19:10:09.0363796Z Starting upload of logs-test-distributed-2-2-amz2023.linux.2xlarge_28475634473.zip 2024-08-07T19:10:09.2885042Z Finished upload of logs-test-distributed-2-2-amz2023.linux.2xlarge_28475634473.zip 2024-08-07T19:10:09.3074037Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T19:10:09.3074380Z with: 2024-08-07T19:10:09.3074617Z s3-bucket: gha-artifacts 2024-08-07T19:10:09.3074951Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T19:10:09.3075303Z retention-days: 14 2024-08-07T19:10:09.3075573Z if-no-files-found: ignore 2024-08-07T19:10:09.3075857Z path: debug-*.zip 2024-08-07T19:10:09.3076091Z name: artifact 2024-08-07T19:10:09.3076336Z region: us-east-1 2024-08-07T19:10:09.3076575Z env: 2024-08-07T19:10:09.3076790Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:10:09.3077274Z DOCKER_CONTAINER_ID: 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:09.3077965Z ##[endgroup] 2024-08-07T19:10:09.6485099Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-07T19:10:09.6674036Z ##[group]Run # shellcheck disable=SC2156 2024-08-07T19:10:09.6674423Z # shellcheck disable=SC2156 2024-08-07T19:10:09.6675047Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-07T19:10:09.6680912Z shell: /usr/bin/bash -e {0} 2024-08-07T19:10:09.6681205Z env: 2024-08-07T19:10:09.6681440Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:10:09.6681914Z DOCKER_CONTAINER_ID: 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:09.6682431Z ##[endgroup] 2024-08-07T19:10:09.8914337Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-07T19:10:09.8914791Z with: 2024-08-07T19:10:09.8914999Z env: 2024-08-07T19:10:09.8915224Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:10:09.8915723Z DOCKER_CONTAINER_ID: 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:09.8916222Z ##[endgroup] 2024-08-07T19:10:09.8942450Z ##[group]Run set -eou pipefail 2024-08-07T19:10:09.8942784Z set -eou pipefail 2024-08-07T19:10:09.8943049Z  2024-08-07T19:10:09.8943428Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-07T19:10:09.8943906Z for _ in $(seq 1440); do 2024-08-07T19:10:09.8944237Z  # Break if no ssh session exists anymore 2024-08-07T19:10:09.8944599Z  if [ "$(who)" = "" ]; then 2024-08-07T19:10:09.8944911Z  break 2024-08-07T19:10:09.8945146Z  fi 2024-08-07T19:10:09.8945427Z  echo "." 2024-08-07T19:10:09.8945671Z  sleep 5 2024-08-07T19:10:09.8945923Z done 2024-08-07T19:10:09.8951373Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:10:09.8951766Z env: 2024-08-07T19:10:09.8951992Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:10:09.8952468Z DOCKER_CONTAINER_ID: 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:09.8952968Z ##[endgroup] 2024-08-07T19:10:09.8975515Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-07T19:10:09.9056991Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T19:10:09.9057569Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T19:10:09.9057999Z # shellcheck disable=SC2046 2024-08-07T19:10:09.9058346Z docker stop $(docker ps -q) || true 2024-08-07T19:10:09.9058704Z # Prune all of the docker images 2024-08-07T19:10:09.9059029Z docker system prune -af 2024-08-07T19:10:09.9064627Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:10:09.9065020Z env: 2024-08-07T19:10:09.9065232Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:10:09.9065721Z DOCKER_CONTAINER_ID: 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:09.9066233Z ##[endgroup] 2024-08-07T19:10:20.5932231Z 75b1949dfde8 2024-08-07T19:10:21.0800077Z Deleted Containers: 2024-08-07T19:10:21.0800692Z 75b1949dfde8441c8e68b37da9fc3b103cad3d95d51ec6f6a780fd71659f28aa 2024-08-07T19:10:21.0801165Z 2024-08-07T19:10:28.5606692Z Deleted Images: 2024-08-07T19:10:28.5607719Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T19:10:28.5609586Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11@sha256:c75221497aa22766802555fccb76014708414eebd47578b213b43b3f8f4cddb9 2024-08-07T19:10:28.5610647Z deleted: sha256:eb2e8636b63e804e38c5e56190d8b860c4bf8a9e0794f88825eff958ed6b1e76 2024-08-07T19:10:28.5611309Z deleted: sha256:3a75aa289a82638e31ac4b969360c49c8013f3546a449692cb458fcde3935f22 2024-08-07T19:10:28.5611975Z deleted: sha256:bdfa3e02d65d776f3bb70b6bf71d852ddca4c1dfbdca2356544996f07fef2ed8 2024-08-07T19:10:28.5612871Z deleted: sha256:6a85078ea3791ab81c962cdfc07c05a8b7eb4c5576721c3b56b45f4adf3fd0d1 2024-08-07T19:10:28.5613614Z deleted: sha256:935ccaf7b02ef4b0aabe934f8ed13eaf6bbc24da0954bdc3eea688aafcf5aea6 2024-08-07T19:10:28.5614289Z deleted: sha256:eef572047e96af9ee6bb247ba9acc9c558d7468ff89e2297aa4ad37d1f6043bb 2024-08-07T19:10:28.5614944Z deleted: sha256:4cde019d0b2c20c0af3f1631adfd033c260533dfeefc25dad09f2900dd488b69 2024-08-07T19:10:28.5615611Z deleted: sha256:d2e3c7156b77f135b6bfcbd686f30b96a9fdd1cf7c08b62815572e2d2f7c68be 2024-08-07T19:10:28.5616271Z deleted: sha256:9cd733cd617c46bdc2cb513ead975cec6d449f22a08e36d803cbcef61fa1ef01 2024-08-07T19:10:28.5617170Z deleted: sha256:64744261caf2f507a0f6160868b1d4f0a4a435fb546701053735b889023799eb 2024-08-07T19:10:28.5617816Z deleted: sha256:a2047a4af9745168d52ba90b39a780bc22dc15918425324774e959c0755645a5 2024-08-07T19:10:28.5618474Z deleted: sha256:7d8b4203c0f94a0eccdf0a3ee5819d3be23e8acb9412c3330f66669f03d23e4a 2024-08-07T19:10:28.5619140Z deleted: sha256:7c66ee01d3a6002bf7c53c998d9320f1fb14268af96da395dc28488d5e28001e 2024-08-07T19:10:28.5619782Z deleted: sha256:c1821ba8bdf74b922560615bc28d42e36b63152f7f8e7b07fead450cf78c03b3 2024-08-07T19:10:28.5620570Z deleted: sha256:738da9c7930e061cb698fdaff5ab4d9019fd00adeeacbc24c07eeef3b9a1e8f3 2024-08-07T19:10:28.5621237Z deleted: sha256:b035793979cb8d15291d75f7ceb86be4c7e50b0b55c3ad2de59633310f8eb2ad 2024-08-07T19:10:28.5621872Z deleted: sha256:8caa23c592f4ec71096f15d741c7f74b3674d14f201069b87058d486ee469d80 2024-08-07T19:10:28.5622513Z deleted: sha256:5ec471cacf0cf9317535522156613c2b647715813f96552a7756767caa05b5bc 2024-08-07T19:10:28.5623156Z deleted: sha256:3ddc946b392989b8e7e8086bc23fdab9391ec27042559856bb40fd155a0626b0 2024-08-07T19:10:28.5623805Z deleted: sha256:8bb9b390febb36e87a303534a0672e799f14263107dccecb8fb816711b19b197 2024-08-07T19:10:28.5624609Z deleted: sha256:7636da8a45668f13fc8bdd44f4bcbdfe08506e1222f2e6bf0b6173ff1ebc9d73 2024-08-07T19:10:28.5625290Z deleted: sha256:db811dce25acbc1220f0b64f6b6378af1a8715aadf36172b548ccf95a8a10f5c 2024-08-07T19:10:28.5626006Z deleted: sha256:ea9f1876d9bef2b9737a8084aa575ae2ccf54320762796b1a25b2fd7a1f9a842 2024-08-07T19:10:28.5626653Z deleted: sha256:ee9308661047f85a0cb1e065673e3e34a223a0fabc0d758ca482d1caf72bc156 2024-08-07T19:10:28.5627316Z deleted: sha256:83af056623fdf4bdd8eb6689dd830c618fc706f07f5745771bd23dee1a9df53c 2024-08-07T19:10:28.5627975Z deleted: sha256:1697cd38e32034c99390abaf80e0ef262175ceb38304c2a33a2391ea089625ce 2024-08-07T19:10:28.5628632Z deleted: sha256:2c941f6435258652fe85b0aa3fe7dfd6446f8ad4b90ff57b75cff5173ca901d1 2024-08-07T19:10:28.5629271Z deleted: sha256:de31781f10299152d97a453b662468e0fe812bf2c80e40d7e87d21f1aeebe0de 2024-08-07T19:10:28.5629935Z deleted: sha256:df51025351b2d7b823ae2da119ef42f7a8bcb65067852d58b58793b87a4020cd 2024-08-07T19:10:28.5630591Z deleted: sha256:dea53edb79ae100c832ff7d8e52aa46514a05f85b18820e9798e14cfca8d2b0a 2024-08-07T19:10:28.5631245Z deleted: sha256:30a0b14b371bbf3393cc59e86ebddb2c2bf5ec6e14a2623357e0e8522004373a 2024-08-07T19:10:28.5631905Z deleted: sha256:86275d2a7f4ab08c4db777a2d58601c8e9a33f085073ee419cecc0f80706ba0a 2024-08-07T19:10:28.5632580Z deleted: sha256:be527c207999e42b06dd2b66329fee5905a775d5a617576a8822937689e181b2 2024-08-07T19:10:28.5633257Z deleted: sha256:b726bfe9f806f36a474aff38d139841ad1f3062ad40b2644b38a070efa891fc4 2024-08-07T19:10:28.5634008Z deleted: sha256:79621bf70363b2e574b56de08889a123c0aafdf331b6d403b1227d601196878f 2024-08-07T19:10:28.5634662Z deleted: sha256:23da9a238173789a5e64303a1d5b580bdc2ff4ee4adb1726a937e74fccdb3304 2024-08-07T19:10:28.5635300Z deleted: sha256:6ae72960efb5326803e5d131f2b788bea7d945094d6b4154fbb0d23a1724bc9d 2024-08-07T19:10:28.5635959Z deleted: sha256:4b522ca122bee088c81b99d32935d0c7f798acd919e4787f71f60860f47039c8 2024-08-07T19:10:28.5636603Z deleted: sha256:8099c7b0342e10a71c3957854138df7a86715582a05dfeb507862407dda38fc3 2024-08-07T19:10:28.5637237Z deleted: sha256:9a3099fdec71a88416fee7bf46f4a7692b473fb9684dd93aaa0370840bb1ec2a 2024-08-07T19:10:28.5637996Z deleted: sha256:c7ffa0248132fad86167206fd29b1b8f49bf20a42406063d47df97a13d8f198a 2024-08-07T19:10:28.5638646Z deleted: sha256:496001e36192545049eca8ecc830251c2fa1eb6daf21421c8dafe0acb1f81538 2024-08-07T19:10:28.5639298Z deleted: sha256:9813a6d93cac15977f4540b3e5b3fe849269f35658ed780bdda563ee74b15b3f 2024-08-07T19:10:28.5639928Z deleted: sha256:e29ab5e9037a9838843863a372ef1748406bc95406369a70955134a7cae06d63 2024-08-07T19:10:28.5640889Z deleted: sha256:27d3f1ba6f4aaf25f597a8dcf5d90168631cbbaa2283d5d82528bb267dbc9aa4 2024-08-07T19:10:28.5641765Z deleted: sha256:e7b742f0c6815a00a368e84b648d3d3707bb9f50fc1111d2c99506f5e446552f 2024-08-07T19:10:28.5642526Z deleted: sha256:fd8caf0a273887946a30471b6d8565942faac80f5294f4b3f60779ba4fb1589c 2024-08-07T19:10:28.5643633Z deleted: sha256:6b4ae34804843731029058254b2a78ec9bf655c143179356562fb6792a6d387e 2024-08-07T19:10:28.5644650Z deleted: sha256:5eb13a2c08fccfe8a68cb2eedd5374ca8533971837acdecf1a6241b970be84d4 2024-08-07T19:10:28.5645819Z deleted: sha256:d782a8ac99055b7a7b36960eadce2aa26f0702a2e9de910d9a282b4ad4cf0409 2024-08-07T19:10:28.5647059Z deleted: sha256:32b90a12e0e9cbec20f720d2dd8dc1fe94d05f62a3106ccbe443c7674bcc4ce0 2024-08-07T19:10:28.5648191Z deleted: sha256:ec714b229d3a0dff80ffbf136428ab5a5271e5df7b61198f3fefcda6287ccb6c 2024-08-07T19:10:28.5649205Z deleted: sha256:2f5754399f29badb5803e0b433f6704280c4b79475b44de6728af1dfbcbb2323 2024-08-07T19:10:28.5650283Z deleted: sha256:7bf42df28ece50b780ec31bab2ecea6382f1fe26a88816cc6b9718265cf3465e 2024-08-07T19:10:28.5651370Z deleted: sha256:9a6aa7079f47ded223c38219eed1824b2ffb668af29fa9bcb485254f7e83ab21 2024-08-07T19:10:28.5652438Z deleted: sha256:e9724bcc7ec56c5acb95e352940c94b1c2a9ee52109ad8aab536930bc9728176 2024-08-07T19:10:28.5653601Z deleted: sha256:8f8ef3805ce4283573b6bf58f161a9b3331f50a60ce8d81d4ac659f1fff055c0 2024-08-07T19:10:28.5654728Z deleted: sha256:0729580f4cbe15e55af6aad9352c76c40a739f10bb9b43550680d486bd5c076b 2024-08-07T19:10:28.5655886Z deleted: sha256:4a0b4eb049da56b51a7fdfa4d8206378bdbb1b016dfc6e0bca982791a12fd5eb 2024-08-07T19:10:28.5656751Z deleted: sha256:cf4eeeecbbf6221e1ef64a38d6ba85c73e0bbf10869125ebf081e18f5ef4e9e0 2024-08-07T19:10:28.5657416Z deleted: sha256:f3f2f0324ca26243445f66f17828f234c7a77640dc5d9c44b7a90746c31a2c69 2024-08-07T19:10:28.5658071Z deleted: sha256:014362212fc6cec487fa443cf40098dfadb1562fbd6c17eaa1d987bcac2005d5 2024-08-07T19:10:28.5658730Z deleted: sha256:9d8bd07fc065e2f3faa82c6be619acdbe2a0ac5c84fa4d34564dd9ca50c1abb4 2024-08-07T19:10:28.5659588Z deleted: sha256:be515201f04cb68521fbdaf149a170e1cfee851eb7cf0d0e21cf44788d2d2738 2024-08-07T19:10:28.5660241Z deleted: sha256:a80708e23e0af5c08681444d907936033a760a3707ff65145db0d08946092e6c 2024-08-07T19:10:28.5660905Z deleted: sha256:a12ed841d341f49b467e30c6f384e0cba615d2dc38e7c23f7a3e926f8218fee0 2024-08-07T19:10:28.5661568Z deleted: sha256:508edf28b5543e1a9d07f1caeb9dddf6e77d34ef0d416dc734399f3f49335923 2024-08-07T19:10:28.5662224Z deleted: sha256:0e101de6946d43b7137acf4e038ab8d10e32dbf0c2480641c24be8ed6086abc1 2024-08-07T19:10:28.5663216Z deleted: sha256:c1c19d0e8afe1ecbbde109ac71c1d3f17a00bfcf3d50834770b8d5464ee9c8b2 2024-08-07T19:10:28.5664112Z deleted: sha256:9dc9d6055307261f20c9b3b25bff6eba44e06a05a5b285d733599015160c7dbc 2024-08-07T19:10:28.5665242Z deleted: sha256:7b94babd6975875afe9642a2b9bcf59e402dc13a48cc8be6fd65c2a5fef3ca7c 2024-08-07T19:10:28.5666012Z deleted: sha256:3873bca3dc5064ab463ae26e0a2ab4e9372bea50dd21aa9d60f740676dad7486 2024-08-07T19:10:28.5666683Z deleted: sha256:cbf23c83ce2d413d3afd8790f223a461b296a546765135e62cd690dad638d411 2024-08-07T19:10:28.5667356Z deleted: sha256:bf83fffdacd881f54312039756eae0f29ee91313edb99240a21b46a7a8a52112 2024-08-07T19:10:28.5668005Z deleted: sha256:f8a51c6f429112858d9fa585e8865b12aef6f984426a6b249a9c2405bd6bc617 2024-08-07T19:10:28.5668684Z deleted: sha256:a5f3bdddd25f3ff04aeee3425b589034eea5abb8f3356a23da937bdfae388f96 2024-08-07T19:10:28.5669368Z deleted: sha256:a63fb4f23633e92cf452b4d42c11fabe08bc5941a7b3ccfdb514d9f5df78ace9 2024-08-07T19:10:28.5670199Z deleted: sha256:55935042e44f44818cd67bfb9dbce2a9b57e383adb3caee10c52a60bb4ca2907 2024-08-07T19:10:28.5670863Z deleted: sha256:4e77f30e7a210eadab33be7d5a8e30b207fe0595cc887f660034793669ad4882 2024-08-07T19:10:28.5671531Z deleted: sha256:ff95c9e3cc791feac6467bb1a86bd910894d5aa807dbc7ef8d0656f4c313fb55 2024-08-07T19:10:28.5672190Z deleted: sha256:2d609ec5d6f0f513dc06ab2d069d1f3705dbb888c1c33b153d4d7b402a3a8445 2024-08-07T19:10:28.5672848Z deleted: sha256:6871dd9601fad505a94c55343d42d89a15ba2f3a15352e4264e4aef36be2d3b2 2024-08-07T19:10:28.5673599Z deleted: sha256:e7aab1aad704a9457fcc4018383183080c494fdd601a6d38dc2a06489e3819a6 2024-08-07T19:10:28.5674256Z deleted: sha256:77b1d18f5457391c44c5bf9bfbbdac615becb4cf2149d82856165b77ad19132e 2024-08-07T19:10:28.5675018Z deleted: sha256:5ad805ac6b7297997e85c12ead27e5b34775f55d66c96af2c9c643752c231060 2024-08-07T19:10:28.5675667Z deleted: sha256:f169358745a88fef9997e00330b991a45ceb3553651bbc391a938a896ddeaebf 2024-08-07T19:10:28.5676332Z deleted: sha256:51b3d62ea293d971aac7f5c6e7a8cac80e68e53419b75c0f225b3e14c1b78b1b 2024-08-07T19:10:28.5676971Z deleted: sha256:380d79122c2fdff1088471ef238b65b189372fe3d073caf4500c53935a53a6f5 2024-08-07T19:10:28.5677617Z deleted: sha256:cf4619277942bc0846de485114cca961fa146acbd85711f3a6ee5160681d0a8a 2024-08-07T19:10:28.5678270Z deleted: sha256:331c0a0477e8aa63e33aba884616ecd6cf5dafa7fe512374b8e8d01fb50bdf6f 2024-08-07T19:10:28.5678918Z deleted: sha256:052665ffd1620d190087d835e6fae9b128aaea33ff929767ce2a7d6b5c9c77f2 2024-08-07T19:10:28.5679564Z deleted: sha256:4534c8036770494ac61772b0ffe2379aaa62dbeb0d1307fff0164be698430f2c 2024-08-07T19:10:28.5680215Z deleted: sha256:931b7ff0cb6f494b27d31a4cbec3efe62ac54676add9c7469560302f1541ecaf 2024-08-07T19:10:28.5680602Z 2024-08-07T19:10:28.5680735Z Total reclaimed space: 15.25GB 2024-08-07T19:10:28.5952318Z Post job cleanup. 2024-08-07T19:10:28.6016248Z Post job cleanup. 2024-08-07T19:10:28.6881125Z [command]/usr/bin/git version 2024-08-07T19:10:28.7128929Z git version 2.40.1 2024-08-07T19:10:28.7166071Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/efa40b64-df71-4d57-ae91-ff2c9637547c' before making global git config changes 2024-08-07T19:10:28.7167058Z Adding repository directory to the temporary git global config as a safe directory 2024-08-07T19:10:28.7171003Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T19:10:28.7218405Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-07T19:10:28.7250102Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-07T19:10:28.7628998Z Entering 'android/libs/fbjni' 2024-08-07T19:10:28.7680123Z Entering 'third_party/FP16' 2024-08-07T19:10:28.7730266Z Entering 'third_party/FXdiv' 2024-08-07T19:10:28.7780180Z Entering 'third_party/NNPACK' 2024-08-07T19:10:28.7829764Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T19:10:28.7879457Z Entering 'third_party/XNNPACK' 2024-08-07T19:10:28.7944893Z Entering 'third_party/benchmark' 2024-08-07T19:10:28.7995626Z Entering 'third_party/cpp-httplib' 2024-08-07T19:10:28.8043917Z Entering 'third_party/cpuinfo' 2024-08-07T19:10:28.8094648Z Entering 'third_party/cudnn_frontend' 2024-08-07T19:10:28.8142973Z Entering 'third_party/cutlass' 2024-08-07T19:10:28.8200028Z Entering 'third_party/eigen' 2024-08-07T19:10:28.8250845Z Entering 'third_party/fbgemm' 2024-08-07T19:10:28.8300691Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T19:10:28.8348406Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T19:10:28.8397094Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T19:10:28.8451433Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T19:10:28.8499992Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T19:10:28.8549176Z Entering 'third_party/flatbuffers' 2024-08-07T19:10:28.8601779Z Entering 'third_party/fmt' 2024-08-07T19:10:28.8650072Z Entering 'third_party/foxi' 2024-08-07T19:10:28.8699241Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T19:10:28.8747658Z Entering 'third_party/gloo' 2024-08-07T19:10:28.8797062Z Entering 'third_party/googletest' 2024-08-07T19:10:28.8845632Z Entering 'third_party/ideep' 2024-08-07T19:10:28.8893908Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T19:10:28.8949863Z Entering 'third_party/ittapi' 2024-08-07T19:10:28.8999142Z Entering 'third_party/kineto' 2024-08-07T19:10:28.9047718Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T19:10:28.9096179Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T19:10:28.9146227Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T19:10:28.9194943Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T19:10:28.9244232Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T19:10:28.9291863Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T19:10:28.9343470Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T19:10:28.9392889Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T19:10:28.9441127Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T19:10:28.9492301Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T19:10:28.9541727Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T19:10:28.9590745Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T19:10:28.9640264Z Entering 'third_party/mimalloc' 2024-08-07T19:10:28.9690499Z Entering 'third_party/nccl/nccl' 2024-08-07T19:10:28.9739919Z Entering 'third_party/nlohmann' 2024-08-07T19:10:28.9790528Z Entering 'third_party/onnx' 2024-08-07T19:10:28.9854509Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T19:10:28.9903730Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T19:10:28.9954584Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T19:10:29.0005278Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T19:10:29.0053112Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T19:10:29.0100731Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T19:10:29.0148368Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T19:10:29.0198609Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T19:10:29.0245951Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T19:10:29.0294003Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T19:10:29.0342455Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T19:10:29.0392612Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T19:10:29.0442488Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T19:10:29.0511585Z Entering 'third_party/pocketfft' 2024-08-07T19:10:29.0561011Z Entering 'third_party/protobuf' 2024-08-07T19:10:29.0613172Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T19:10:29.0660770Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T19:10:29.0711521Z Entering 'third_party/psimd' 2024-08-07T19:10:29.0761129Z Entering 'third_party/pthreadpool' 2024-08-07T19:10:29.0810545Z Entering 'third_party/pybind11' 2024-08-07T19:10:29.0859854Z Entering 'third_party/python-peachpy' 2024-08-07T19:10:29.0908845Z Entering 'third_party/sleef' 2024-08-07T19:10:29.0957688Z Entering 'third_party/tensorpipe' 2024-08-07T19:10:29.1006366Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T19:10:29.1053784Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T19:10:29.1102584Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T19:10:29.1150311Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T19:10:29.1197475Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T19:10:29.1262636Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-07T19:10:29.1289584Z http.https://github.com/.extraheader 2024-08-07T19:10:29.1297641Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-07T19:10:29.1331262Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-07T19:10:29.1598182Z Entering 'android/libs/fbjni' 2024-08-07T19:10:29.1631588Z http.https://github.com/.extraheader 2024-08-07T19:10:29.1662772Z Entering 'third_party/FP16' 2024-08-07T19:10:29.1695440Z http.https://github.com/.extraheader 2024-08-07T19:10:29.1725974Z Entering 'third_party/FXdiv' 2024-08-07T19:10:29.1758252Z http.https://github.com/.extraheader 2024-08-07T19:10:29.1788160Z Entering 'third_party/NNPACK' 2024-08-07T19:10:29.1821375Z http.https://github.com/.extraheader 2024-08-07T19:10:29.1851827Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T19:10:29.1885400Z http.https://github.com/.extraheader 2024-08-07T19:10:29.1916564Z Entering 'third_party/XNNPACK' 2024-08-07T19:10:29.1949167Z http.https://github.com/.extraheader 2024-08-07T19:10:29.1996485Z Entering 'third_party/benchmark' 2024-08-07T19:10:29.2029539Z http.https://github.com/.extraheader 2024-08-07T19:10:29.2060105Z Entering 'third_party/cpp-httplib' 2024-08-07T19:10:29.2092713Z http.https://github.com/.extraheader 2024-08-07T19:10:29.2122574Z Entering 'third_party/cpuinfo' 2024-08-07T19:10:29.2155065Z http.https://github.com/.extraheader 2024-08-07T19:10:29.2186193Z Entering 'third_party/cudnn_frontend' 2024-08-07T19:10:29.2218583Z http.https://github.com/.extraheader 2024-08-07T19:10:29.2248691Z Entering 'third_party/cutlass' 2024-08-07T19:10:29.2281486Z http.https://github.com/.extraheader 2024-08-07T19:10:29.2319274Z Entering 'third_party/eigen' 2024-08-07T19:10:29.2351780Z http.https://github.com/.extraheader 2024-08-07T19:10:29.2384719Z Entering 'third_party/fbgemm' 2024-08-07T19:10:29.2417760Z http.https://github.com/.extraheader 2024-08-07T19:10:29.2447986Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T19:10:29.2481391Z http.https://github.com/.extraheader 2024-08-07T19:10:29.2511788Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T19:10:29.2544231Z http.https://github.com/.extraheader 2024-08-07T19:10:29.2574589Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T19:10:29.2607199Z http.https://github.com/.extraheader 2024-08-07T19:10:29.2641880Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T19:10:29.2674291Z http.https://github.com/.extraheader 2024-08-07T19:10:29.2703732Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T19:10:29.2736496Z http.https://github.com/.extraheader 2024-08-07T19:10:29.2767130Z Entering 'third_party/flatbuffers' 2024-08-07T19:10:29.2800568Z http.https://github.com/.extraheader 2024-08-07T19:10:29.2832968Z Entering 'third_party/fmt' 2024-08-07T19:10:29.2867496Z http.https://github.com/.extraheader 2024-08-07T19:10:29.2896581Z Entering 'third_party/foxi' 2024-08-07T19:10:29.2929525Z http.https://github.com/.extraheader 2024-08-07T19:10:29.2959909Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T19:10:29.2992757Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3022816Z Entering 'third_party/gloo' 2024-08-07T19:10:29.3054978Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3085376Z Entering 'third_party/googletest' 2024-08-07T19:10:29.3117643Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3147555Z Entering 'third_party/ideep' 2024-08-07T19:10:29.3180406Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3209276Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T19:10:29.3241659Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3279239Z Entering 'third_party/ittapi' 2024-08-07T19:10:29.3312872Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3342856Z Entering 'third_party/kineto' 2024-08-07T19:10:29.3377981Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3407773Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T19:10:29.3440172Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3469395Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T19:10:29.3502462Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3534857Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T19:10:29.3568268Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3599064Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T19:10:29.3631411Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3662035Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T19:10:29.3694352Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3724145Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T19:10:29.3757183Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3789219Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T19:10:29.3821406Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3851808Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T19:10:29.3884570Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3915242Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T19:10:29.3948073Z http.https://github.com/.extraheader 2024-08-07T19:10:29.3980867Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T19:10:29.4013429Z http.https://github.com/.extraheader 2024-08-07T19:10:29.4045265Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T19:10:29.4077670Z http.https://github.com/.extraheader 2024-08-07T19:10:29.4107411Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T19:10:29.4139296Z http.https://github.com/.extraheader 2024-08-07T19:10:29.4171278Z Entering 'third_party/mimalloc' 2024-08-07T19:10:29.4204103Z http.https://github.com/.extraheader 2024-08-07T19:10:29.4234601Z Entering 'third_party/nccl/nccl' 2024-08-07T19:10:29.4268183Z http.https://github.com/.extraheader 2024-08-07T19:10:29.4301112Z Entering 'third_party/nlohmann' 2024-08-07T19:10:29.4334333Z http.https://github.com/.extraheader 2024-08-07T19:10:29.4365932Z Entering 'third_party/onnx' 2024-08-07T19:10:29.4398969Z http.https://github.com/.extraheader 2024-08-07T19:10:29.4444550Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T19:10:29.4478678Z http.https://github.com/.extraheader 2024-08-07T19:10:29.4508490Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T19:10:29.4541224Z http.https://github.com/.extraheader 2024-08-07T19:10:29.4574701Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T19:10:29.4608660Z http.https://github.com/.extraheader 2024-08-07T19:10:29.4641343Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T19:10:29.4674934Z http.https://github.com/.extraheader 2024-08-07T19:10:29.4705936Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T19:10:29.4737966Z http.https://github.com/.extraheader 2024-08-07T19:10:29.4769047Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T19:10:29.4801004Z http.https://github.com/.extraheader 2024-08-07T19:10:29.4830957Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T19:10:29.4863426Z http.https://github.com/.extraheader 2024-08-07T19:10:29.4895094Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T19:10:29.4927109Z http.https://github.com/.extraheader 2024-08-07T19:10:29.4957660Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T19:10:29.4990496Z http.https://github.com/.extraheader 2024-08-07T19:10:29.5020312Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T19:10:29.5051871Z http.https://github.com/.extraheader 2024-08-07T19:10:29.5082935Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T19:10:29.5115979Z http.https://github.com/.extraheader 2024-08-07T19:10:29.5148090Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T19:10:29.5181118Z http.https://github.com/.extraheader 2024-08-07T19:10:29.5214669Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T19:10:29.5247320Z http.https://github.com/.extraheader 2024-08-07T19:10:29.5297602Z Entering 'third_party/pocketfft' 2024-08-07T19:10:29.5330395Z http.https://github.com/.extraheader 2024-08-07T19:10:29.5361648Z Entering 'third_party/protobuf' 2024-08-07T19:10:29.5394988Z http.https://github.com/.extraheader 2024-08-07T19:10:29.5428507Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T19:10:29.5461003Z http.https://github.com/.extraheader 2024-08-07T19:10:29.5490432Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T19:10:29.5522308Z http.https://github.com/.extraheader 2024-08-07T19:10:29.5554942Z Entering 'third_party/psimd' 2024-08-07T19:10:29.5588925Z http.https://github.com/.extraheader 2024-08-07T19:10:29.5619034Z Entering 'third_party/pthreadpool' 2024-08-07T19:10:29.5652771Z http.https://github.com/.extraheader 2024-08-07T19:10:29.5685362Z Entering 'third_party/pybind11' 2024-08-07T19:10:29.5718430Z http.https://github.com/.extraheader 2024-08-07T19:10:29.5747781Z Entering 'third_party/python-peachpy' 2024-08-07T19:10:29.5781558Z http.https://github.com/.extraheader 2024-08-07T19:10:29.5810854Z Entering 'third_party/sleef' 2024-08-07T19:10:29.5844133Z http.https://github.com/.extraheader 2024-08-07T19:10:29.5876305Z Entering 'third_party/tensorpipe' 2024-08-07T19:10:29.5909614Z http.https://github.com/.extraheader 2024-08-07T19:10:29.5939775Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T19:10:29.5972760Z http.https://github.com/.extraheader 2024-08-07T19:10:29.6002757Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T19:10:29.6034481Z http.https://github.com/.extraheader 2024-08-07T19:10:29.6064563Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T19:10:29.6096636Z http.https://github.com/.extraheader 2024-08-07T19:10:29.6126851Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T19:10:29.6158916Z http.https://github.com/.extraheader 2024-08-07T19:10:29.6188737Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T19:10:29.6221191Z http.https://github.com/.extraheader 2024-08-07T19:10:29.6329938Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-07T19:10:29.6351722Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-07T19:10:29.6356842Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:10:29.6357235Z ##[endgroup] 2024-08-07T19:10:37.0660498Z Cleaning up orphan processes