2024-08-06T20:21:40.6318525Z Current runner version: '2.318.0' 2024-08-06T20:21:40.6326015Z Runner name: 'i-064ea195702d7f670' 2024-08-06T20:21:40.6326982Z Runner group name: 'Default' 2024-08-06T20:21:40.6327997Z Machine name: 'ip-10-0-33-59' 2024-08-06T20:21:40.6332923Z ##[group]GITHUB_TOKEN Permissions 2024-08-06T20:21:40.6336023Z Actions: read 2024-08-06T20:21:40.6336700Z Attestations: read 2024-08-06T20:21:40.6337283Z Checks: read 2024-08-06T20:21:40.6337899Z Contents: read 2024-08-06T20:21:40.6338534Z Deployments: read 2024-08-06T20:21:40.6339126Z Discussions: read 2024-08-06T20:21:40.6339749Z Issues: read 2024-08-06T20:21:40.6340368Z Metadata: read 2024-08-06T20:21:40.6340945Z Packages: read 2024-08-06T20:21:40.6341564Z Pages: read 2024-08-06T20:21:40.6342176Z PullRequests: read 2024-08-06T20:21:40.6342856Z RepositoryProjects: read 2024-08-06T20:21:40.6343550Z SecurityEvents: read 2024-08-06T20:21:40.6344205Z Statuses: read 2024-08-06T20:21:40.6344778Z ##[endgroup] 2024-08-06T20:21:40.6348298Z Secret source: Actions 2024-08-06T20:21:40.6349240Z Prepare workflow directory 2024-08-06T20:21:40.9470544Z Prepare all required actions 2024-08-06T20:21:40.9515093Z Getting action download info 2024-08-06T20:21:41.1249266Z Download action repository 'pytorch/test-infra@main' (SHA:a1f5a89251fc4258ab59806094fe3108f7d6741a) 2024-08-06T20:21:42.4601440Z Download action repository 'pytorch/pytorch@main' (SHA:de00c7958301ce81b9716bdef5731ed40d4d14ca) 2024-08-06T20:21:53.4969841Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-06T20:21:53.7346129Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-06T20:21:54.0226490Z Getting action download info 2024-08-06T20:21:54.1205333Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-06T20:21:54.4037427Z Getting action download info 2024-08-06T20:21:54.5095252Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-06T20:21:54.6585230Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/pull/132710/merge (bf5bb5a1585a03379137fab341e87c02c77e76cd) 2024-08-06T20:21:54.6587459Z ##[group] Inputs 2024-08-06T20:21:54.6587805Z build-environment: linux-jammy-py3.8-gcc11 2024-08-06T20:21:54.6590587Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]} 2024-08-06T20:21:54.6593706Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:21:54.6594440Z sync-tag: 2024-08-06T20:21:54.6595256Z timeout-minutes: 240 2024-08-06T20:21:54.6595529Z use-gha: 2024-08-06T20:21:54.6595750Z dashboard-tag: 2024-08-06T20:21:54.6596014Z s3-bucket: gha-artifacts 2024-08-06T20:21:54.6596299Z aws-role-to-assume: 2024-08-06T20:21:54.6596556Z ##[endgroup] 2024-08-06T20:21:54.6596996Z Complete job name: linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-06T20:21:54.7054873Z A job started hook has been configured by the self-hosted runner administrator 2024-08-06T20:21:54.7164696Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-06T20:21:54.7174392Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:21:54.7175086Z ##[endgroup] 2024-08-06T20:21:56.3521871Z Runner Type: amz2023.linux.2xlarge 2024-08-06T20:21:56.3522590Z Instance Type: c5.2xlarge 2024-08-06T20:21:56.3523136Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-06T20:21:56.3523749Z AMI ID: ami-06c68f701d8090592 2024-08-06T20:22:02.2751257Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-06T20:22:02.2751727Z with: 2024-08-06T20:22:02.2752395Z github-secret: *** 2024-08-06T20:22:02.2753132Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-06T20:22:02.2753927Z activate-with-label: false 2024-08-06T20:22:02.2754222Z label: with-ssh 2024-08-06T20:22:02.2754467Z remove-existing-keys: true 2024-08-06T20:22:02.2754763Z fail-silently: true 2024-08-06T20:22:02.2755021Z env: 2024-08-06T20:22:02.2755263Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:22:02.2755546Z ##[endgroup] 2024-08-06T20:22:02.3603298Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-06T20:22:02.7120873Z Grabbing public ssh keys from https://github.com/drisspg.keys 2024-08-06T20:22:02.7812327Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-06T20:22:02.7827134Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-06T20:22:02.7853199Z Login using: ssh ec2-user@ec2-44-202-139-132.compute-1.amazonaws.com 2024-08-06T20:22:02.7854089Z All testing is done inside the container, to start an interactive session run: 2024-08-06T20:22:02.7854840Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-06T20:22:02.7968515Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-06T20:22:02.7968985Z with: 2024-08-06T20:22:02.7969232Z submodules: recursive 2024-08-06T20:22:02.7969517Z fetch-depth: 0 2024-08-06T20:22:02.7969764Z env: 2024-08-06T20:22:02.7969992Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:22:02.7970255Z ##[endgroup] 2024-08-06T20:22:02.8051998Z ##[group]Run retry () { 2024-08-06T20:22:02.8052334Z retry () { 2024-08-06T20:22:02.8052679Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-06T20:22:02.8053079Z } 2024-08-06T20:22:02.8053351Z echo "${GITHUB_WORKSPACE}" 2024-08-06T20:22:02.8053701Z if [ -z "${NO_SUDO}" ]; then 2024-08-06T20:22:02.8054062Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-06T20:22:02.8054523Z else 2024-08-06T20:22:02.8054780Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-06T20:22:02.8055111Z fi 2024-08-06T20:22:02.8055410Z mkdir "${GITHUB_WORKSPACE}" 2024-08-06T20:22:02.8063961Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:22:02.8064360Z env: 2024-08-06T20:22:02.8064593Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:22:02.8064871Z NO_SUDO: 2024-08-06T20:22:02.8065096Z ##[endgroup] 2024-08-06T20:22:02.8090178Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:22:05.0098242Z ##[group]Run malfet/checkout@silent-checkout 2024-08-06T20:22:05.0098580Z with: 2024-08-06T20:22:05.0098839Z ref: b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:22:05.0099179Z fetch-depth: 0 2024-08-06T20:22:05.0099418Z submodules: recursive 2024-08-06T20:22:05.0099690Z quiet-checkout: true 2024-08-06T20:22:05.0099960Z repository: pytorch/pytorch 2024-08-06T20:22:05.0100362Z token: *** 2024-08-06T20:22:05.0100590Z ssh-strict: true 2024-08-06T20:22:05.0100847Z persist-credentials: true 2024-08-06T20:22:05.0101113Z clean: true 2024-08-06T20:22:05.0101381Z sparse-checkout-cone-mode: true 2024-08-06T20:22:05.0101682Z lfs: false 2024-08-06T20:22:05.0101906Z set-safe-directory: true 2024-08-06T20:22:05.0102173Z env: 2024-08-06T20:22:05.0102378Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:22:05.0102853Z ##[endgroup] 2024-08-06T20:22:05.1049035Z Syncing repository: pytorch/pytorch 2024-08-06T20:22:05.1050430Z ##[group]Getting Git version info 2024-08-06T20:22:05.1050937Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-06T20:22:05.1051636Z [command]/usr/bin/git version 2024-08-06T20:22:05.1051942Z git version 2.40.1 2024-08-06T20:22:05.1059439Z ##[endgroup] 2024-08-06T20:22:05.1072436Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/ac607744-13e2-4651-912a-c1b6c2155fb5' before making global git config changes 2024-08-06T20:22:05.1073415Z Adding repository directory to the temporary git global config as a safe directory 2024-08-06T20:22:05.1076959Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:22:05.1117797Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-06T20:22:05.1121602Z ##[group]Initializing the repository 2024-08-06T20:22:05.1124268Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:22:05.1249469Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-06T20:22:05.1250129Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-06T20:22:05.1250722Z hint: of your new repositories, which will suppress this warning, call: 2024-08-06T20:22:05.1251156Z hint: 2024-08-06T20:22:05.1251732Z hint: git config --global init.defaultBranch 2024-08-06T20:22:05.1252148Z hint: 2024-08-06T20:22:05.1252498Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-06T20:22:05.1253082Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-06T20:22:05.1253803Z hint: 2024-08-06T20:22:05.1254046Z hint: git branch -m 2024-08-06T20:22:05.1254656Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-06T20:22:05.1260471Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-06T20:22:05.1299581Z ##[endgroup] 2024-08-06T20:22:05.1300028Z ##[group]Disabling automatic garbage collection 2024-08-06T20:22:05.1302753Z [command]/usr/bin/git config --local gc.auto 0 2024-08-06T20:22:05.1331821Z ##[endgroup] 2024-08-06T20:22:05.1332221Z ##[group]Setting up auth 2024-08-06T20:22:05.1338130Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-06T20:22:05.1367482Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-06T20:22:05.1626246Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-06T20:22:05.1661819Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-06T20:22:05.1923071Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-06T20:22:05.1972248Z ##[endgroup] 2024-08-06T20:22:05.1972794Z ##[group]Fetching the repository 2024-08-06T20:22:05.1978532Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-06T20:22:07.5539190Z remote: Enumerating objects: 1008384 2024-08-06T20:22:07.5539652Z remote: Enumerating objects: 1008673, done. 2024-08-06T20:22:07.5540275Z remote: Counting objects: 0% (1/289) 2024-08-06T20:22:07.5540772Z remote: Counting objects: 1% (3/289) 2024-08-06T20:22:07.5541324Z remote: Counting objects: 2% (6/289) 2024-08-06T20:22:07.5541894Z remote: Counting objects: 3% (9/289) 2024-08-06T20:22:07.5542545Z remote: Counting objects: 4% (12/289) 2024-08-06T20:22:07.5543520Z remote: Counting objects: 5% (15/289) 2024-08-06T20:22:07.5544024Z remote: Counting objects: 6% (18/289) 2024-08-06T20:22:07.5544522Z remote: Counting objects: 7% (21/289) 2024-08-06T20:22:07.5545079Z remote: Counting objects: 8% (24/289) 2024-08-06T20:22:07.5545454Z remote: Counting objects: 9% (27/289) 2024-08-06T20:22:07.5545982Z remote: Counting objects: 10% (29/289) 2024-08-06T20:22:07.5546341Z remote: Counting objects: 11% (32/289) 2024-08-06T20:22:07.5546715Z remote: Counting objects: 12% (35/289) 2024-08-06T20:22:07.5547087Z remote: Counting objects: 13% (38/289) 2024-08-06T20:22:07.5547447Z remote: Counting objects: 14% (41/289) 2024-08-06T20:22:07.5547832Z remote: Counting objects: 15% (44/289) 2024-08-06T20:22:07.5548200Z remote: Counting objects: 16% (47/289) 2024-08-06T20:22:07.5548562Z remote: Counting objects: 17% (50/289) 2024-08-06T20:22:07.5548997Z remote: Counting objects: 18% (53/289) 2024-08-06T20:22:07.5549361Z remote: Counting objects: 19% (55/289) 2024-08-06T20:22:07.5549719Z remote: Counting objects: 20% (58/289) 2024-08-06T20:22:07.5550084Z remote: Counting objects: 21% (61/289) 2024-08-06T20:22:07.5550448Z remote: Counting objects: 22% (64/289) 2024-08-06T20:22:07.5550808Z remote: Counting objects: 23% (67/289) 2024-08-06T20:22:07.5551173Z remote: Counting objects: 24% (70/289) 2024-08-06T20:22:07.5551543Z remote: Counting objects: 25% (73/289) 2024-08-06T20:22:07.5551901Z remote: Counting objects: 26% (76/289) 2024-08-06T20:22:07.5552374Z remote: Counting objects: 27% (79/289) 2024-08-06T20:22:07.5552933Z remote: Counting objects: 28% (81/289) 2024-08-06T20:22:07.5553295Z remote: Counting objects: 29% (84/289) 2024-08-06T20:22:07.5553663Z remote: Counting objects: 30% (87/289) 2024-08-06T20:22:07.5554036Z remote: Counting objects: 31% (90/289) 2024-08-06T20:22:07.5554404Z remote: Counting objects: 32% (93/289) 2024-08-06T20:22:07.5554777Z remote: Counting objects: 33% (96/289) 2024-08-06T20:22:07.5555148Z remote: Counting objects: 34% (99/289) 2024-08-06T20:22:07.5555509Z remote: Counting objects: 35% (102/289) 2024-08-06T20:22:07.5555886Z remote: Counting objects: 36% (105/289) 2024-08-06T20:22:07.5556260Z remote: Counting objects: 37% (107/289) 2024-08-06T20:22:07.5556620Z remote: Counting objects: 38% (110/289) 2024-08-06T20:22:07.5556992Z remote: Counting objects: 39% (113/289) 2024-08-06T20:22:07.5557371Z remote: Counting objects: 40% (116/289) 2024-08-06T20:22:07.5557738Z remote: Counting objects: 41% (119/289) 2024-08-06T20:22:07.5558115Z remote: Counting objects: 42% (122/289) 2024-08-06T20:22:07.5558488Z remote: Counting objects: 43% (125/289) 2024-08-06T20:22:07.5558848Z remote: Counting objects: 44% (128/289) 2024-08-06T20:22:07.5559227Z remote: Counting objects: 45% (131/289) 2024-08-06T20:22:07.5559602Z remote: Counting objects: 46% (133/289) 2024-08-06T20:22:07.5559964Z remote: Counting objects: 47% (136/289) 2024-08-06T20:22:07.5560336Z remote: Counting objects: 48% (139/289) 2024-08-06T20:22:07.5560709Z remote: Counting objects: 49% (142/289) 2024-08-06T20:22:07.5561070Z remote: Counting objects: 50% (145/289) 2024-08-06T20:22:07.5561441Z remote: Counting objects: 51% (148/289) 2024-08-06T20:22:07.5561833Z remote: Counting objects: 52% (151/289) 2024-08-06T20:22:07.5562194Z remote: Counting objects: 53% (154/289) 2024-08-06T20:22:07.5562565Z remote: Counting objects: 54% (157/289) 2024-08-06T20:22:07.5562971Z remote: Counting objects: 55% (159/289) 2024-08-06T20:22:07.5563339Z remote: Counting objects: 56% (162/289) 2024-08-06T20:22:07.5563715Z remote: Counting objects: 57% (165/289) 2024-08-06T20:22:07.5564178Z remote: Counting objects: 58% (168/289) 2024-08-06T20:22:07.5564553Z remote: Counting objects: 59% (171/289) 2024-08-06T20:22:07.5564919Z remote: Counting objects: 60% (174/289) 2024-08-06T20:22:07.5565298Z remote: Counting objects: 61% (177/289) 2024-08-06T20:22:07.5565676Z remote: Counting objects: 62% (180/289) 2024-08-06T20:22:07.5566072Z remote: Counting objects: 63% (183/289) 2024-08-06T20:22:07.5566455Z remote: Counting objects: 64% (185/289) 2024-08-06T20:22:07.5566837Z remote: Counting objects: 65% (188/289) 2024-08-06T20:22:07.5567200Z remote: Counting objects: 66% (191/289) 2024-08-06T20:22:07.5567576Z remote: Counting objects: 67% (194/289) 2024-08-06T20:22:07.5567957Z remote: Counting objects: 68% (197/289) 2024-08-06T20:22:07.5568317Z remote: Counting objects: 69% (200/289) 2024-08-06T20:22:07.5568692Z remote: Counting objects: 70% (203/289) 2024-08-06T20:22:07.5569071Z remote: Counting objects: 71% (206/289) 2024-08-06T20:22:07.5569429Z remote: Counting objects: 72% (209/289) 2024-08-06T20:22:07.5569804Z remote: Counting objects: 73% (211/289) 2024-08-06T20:22:07.5570173Z remote: Counting objects: 74% (214/289) 2024-08-06T20:22:07.5570544Z remote: Counting objects: 75% (217/289) 2024-08-06T20:22:07.5570905Z remote: Counting objects: 76% (220/289) 2024-08-06T20:22:07.5571277Z remote: Counting objects: 77% (223/289) 2024-08-06T20:22:07.5571645Z remote: Counting objects: 78% (226/289) 2024-08-06T20:22:07.5572005Z remote: Counting objects: 79% (229/289) 2024-08-06T20:22:07.5572374Z remote: Counting objects: 80% (232/289) 2024-08-06T20:22:07.5572822Z remote: Counting objects: 81% (235/289) 2024-08-06T20:22:07.5573378Z remote: Counting objects: 82% (237/289) 2024-08-06T20:22:07.5573901Z remote: Counting objects: 83% (240/289) 2024-08-06T20:22:07.5574465Z remote: Counting objects: 84% (243/289) 2024-08-06T20:22:07.5574829Z remote: Counting objects: 85% (246/289) 2024-08-06T20:22:07.5575197Z remote: Counting objects: 86% (249/289) 2024-08-06T20:22:07.5575572Z remote: Counting objects: 87% (252/289) 2024-08-06T20:22:07.5575934Z remote: Counting objects: 88% (255/289) 2024-08-06T20:22:07.5576311Z remote: Counting objects: 89% (258/289) 2024-08-06T20:22:07.5576684Z remote: Counting objects: 90% (261/289) 2024-08-06T20:22:07.5577093Z remote: Counting objects: 91% (263/289) 2024-08-06T20:22:07.5577669Z remote: Counting objects: 92% (266/289) 2024-08-06T20:22:07.5578147Z remote: Counting objects: 93% (269/289) 2024-08-06T20:22:07.5578576Z remote: Counting objects: 94% (272/289) 2024-08-06T20:22:07.5578951Z remote: Counting objects: 95% (275/289) 2024-08-06T20:22:07.5579327Z remote: Counting objects: 96% (278/289) 2024-08-06T20:22:07.5579694Z remote: Counting objects: 97% (281/289) 2024-08-06T20:22:07.5580065Z remote: Counting objects: 98% (284/289) 2024-08-06T20:22:07.5580438Z remote: Counting objects: 99% (287/289) 2024-08-06T20:22:07.5580800Z remote: Counting objects: 100% (289/289) 2024-08-06T20:22:07.5581198Z remote: Counting objects: 100% (289/289), done. 2024-08-06T20:22:07.5581610Z remote: Compressing objects: 0% (1/161) 2024-08-06T20:22:07.5581989Z remote: Compressing objects: 1% (2/161) 2024-08-06T20:22:07.5582405Z remote: Compressing objects: 2% (4/161) 2024-08-06T20:22:07.5582794Z remote: Compressing objects: 3% (5/161) 2024-08-06T20:22:07.5583166Z remote: Compressing objects: 4% (7/161) 2024-08-06T20:22:07.5583556Z remote: Compressing objects: 5% (9/161) 2024-08-06T20:22:07.5583943Z remote: Compressing objects: 6% (10/161) 2024-08-06T20:22:07.5584318Z remote: Compressing objects: 7% (12/161) 2024-08-06T20:22:07.5629909Z remote: Compressing objects: 8% (13/161) 2024-08-06T20:22:07.5641611Z remote: Compressing objects: 9% (15/161) 2024-08-06T20:22:07.5817568Z remote: Compressing objects: 10% (17/161) 2024-08-06T20:22:07.5817977Z remote: Compressing objects: 11% (18/161) 2024-08-06T20:22:07.5818375Z remote: Compressing objects: 12% (20/161) 2024-08-06T20:22:07.5818866Z remote: Compressing objects: 13% (21/161) 2024-08-06T20:22:07.5819362Z remote: Compressing objects: 14% (23/161) 2024-08-06T20:22:07.5819775Z remote: Compressing objects: 15% (25/161) 2024-08-06T20:22:07.5820159Z remote: Compressing objects: 16% (26/161) 2024-08-06T20:22:07.5820528Z remote: Compressing objects: 17% (28/161) 2024-08-06T20:22:07.5820930Z remote: Compressing objects: 18% (29/161) 2024-08-06T20:22:07.5821312Z remote: Compressing objects: 19% (31/161) 2024-08-06T20:22:07.5821683Z remote: Compressing objects: 20% (33/161) 2024-08-06T20:22:07.5822076Z remote: Compressing objects: 21% (34/161) 2024-08-06T20:22:07.5822456Z remote: Compressing objects: 22% (36/161) 2024-08-06T20:22:07.5822978Z remote: Compressing objects: 23% (38/161) 2024-08-06T20:22:07.5823367Z remote: Compressing objects: 24% (39/161) 2024-08-06T20:22:07.5823750Z remote: Compressing objects: 25% (41/161) 2024-08-06T20:22:07.5824119Z remote: Compressing objects: 26% (42/161) 2024-08-06T20:22:07.5824502Z remote: Compressing objects: 27% (44/161) 2024-08-06T20:22:07.5824887Z remote: Compressing objects: 28% (46/161) 2024-08-06T20:22:07.5825257Z remote: Compressing objects: 29% (47/161) 2024-08-06T20:22:07.5825640Z remote: Compressing objects: 30% (49/161) 2024-08-06T20:22:07.5826379Z remote: Compressing objects: 31% (50/161) 2024-08-06T20:22:07.5826802Z remote: Compressing objects: 32% (52/161) 2024-08-06T20:22:07.5827892Z remote: Compressing objects: 33% (54/161) 2024-08-06T20:22:07.5830981Z remote: Compressing objects: 34% (55/161) 2024-08-06T20:22:07.5831381Z remote: Compressing objects: 35% (57/161) 2024-08-06T20:22:07.5835288Z remote: Compressing objects: 36% (58/161) 2024-08-06T20:22:07.5837082Z remote: Compressing objects: 37% (60/161) 2024-08-06T20:22:07.5837476Z remote: Compressing objects: 38% (62/161) 2024-08-06T20:22:07.5868041Z remote: Compressing objects: 39% (63/161) 2024-08-06T20:22:07.5868618Z remote: Compressing objects: 40% (65/161) 2024-08-06T20:22:07.5869019Z remote: Compressing objects: 41% (67/161) 2024-08-06T20:22:07.5869396Z remote: Compressing objects: 42% (68/161) 2024-08-06T20:22:07.5869806Z remote: Compressing objects: 43% (70/161) 2024-08-06T20:22:07.5870199Z remote: Compressing objects: 44% (71/161) 2024-08-06T20:22:07.5870574Z remote: Compressing objects: 45% (73/161) 2024-08-06T20:22:07.5870968Z remote: Compressing objects: 46% (75/161) 2024-08-06T20:22:07.5871377Z remote: Compressing objects: 47% (76/161) 2024-08-06T20:22:07.5871752Z remote: Compressing objects: 48% (78/161) 2024-08-06T20:22:07.5872141Z remote: Compressing objects: 49% (79/161) 2024-08-06T20:22:07.5872526Z remote: Compressing objects: 50% (81/161) 2024-08-06T20:22:07.5872899Z remote: Compressing objects: 51% (83/161) 2024-08-06T20:22:07.5873287Z remote: Compressing objects: 52% (84/161) 2024-08-06T20:22:07.5873677Z remote: Compressing objects: 53% (86/161) 2024-08-06T20:22:07.5874052Z remote: Compressing objects: 54% (87/161) 2024-08-06T20:22:07.5874441Z remote: Compressing objects: 55% (89/161) 2024-08-06T20:22:07.5874836Z remote: Compressing objects: 56% (91/161) 2024-08-06T20:22:07.5875211Z remote: Compressing objects: 57% (92/161) 2024-08-06T20:22:07.5875596Z remote: Compressing objects: 58% (94/161) 2024-08-06T20:22:07.5876210Z remote: Compressing objects: 59% (95/161) 2024-08-06T20:22:07.5876582Z remote: Compressing objects: 60% (97/161) 2024-08-06T20:22:07.5876974Z remote: Compressing objects: 61% (99/161) 2024-08-06T20:22:07.5877471Z remote: Compressing objects: 62% (100/161) 2024-08-06T20:22:07.5877854Z remote: Compressing objects: 63% (102/161) 2024-08-06T20:22:07.5878277Z remote: Compressing objects: 64% (104/161) 2024-08-06T20:22:07.5878685Z remote: Compressing objects: 65% (105/161) 2024-08-06T20:22:07.5879080Z remote: Compressing objects: 66% (107/161) 2024-08-06T20:22:07.5879467Z remote: Compressing objects: 67% (108/161) 2024-08-06T20:22:07.5879854Z remote: Compressing objects: 68% (110/161) 2024-08-06T20:22:07.5880253Z remote: Compressing objects: 69% (112/161) 2024-08-06T20:22:07.5880626Z remote: Compressing objects: 70% (113/161) 2024-08-06T20:22:07.5881015Z remote: Compressing objects: 71% (115/161) 2024-08-06T20:22:07.5881408Z remote: Compressing objects: 72% (116/161) 2024-08-06T20:22:07.5881781Z remote: Compressing objects: 73% (118/161) 2024-08-06T20:22:07.5882174Z remote: Compressing objects: 74% (120/161) 2024-08-06T20:22:07.5882563Z remote: Compressing objects: 75% (121/161) 2024-08-06T20:22:07.5882937Z remote: Compressing objects: 76% (123/161) 2024-08-06T20:22:07.5883323Z remote: Compressing objects: 77% (124/161) 2024-08-06T20:22:07.5883714Z remote: Compressing objects: 78% (126/161) 2024-08-06T20:22:07.5884093Z remote: Compressing objects: 79% (128/161) 2024-08-06T20:22:07.5884479Z remote: Compressing objects: 80% (129/161) 2024-08-06T20:22:07.5884863Z remote: Compressing objects: 81% (131/161) 2024-08-06T20:22:07.5885549Z remote: Compressing objects: 82% (133/161) 2024-08-06T20:22:07.5886260Z remote: Compressing objects: 83% (134/161) 2024-08-06T20:22:07.5886968Z remote: Compressing objects: 84% (136/161) 2024-08-06T20:22:07.5887673Z remote: Compressing objects: 85% (137/161) 2024-08-06T20:22:07.5888383Z remote: Compressing objects: 86% (139/161) 2024-08-06T20:22:07.5889107Z remote: Compressing objects: 87% (141/161) 2024-08-06T20:22:07.5889813Z remote: Compressing objects: 88% (142/161) 2024-08-06T20:22:07.5890610Z remote: Compressing objects: 89% (144/161) 2024-08-06T20:22:07.5891319Z remote: Compressing objects: 90% (145/161) 2024-08-06T20:22:07.5892010Z remote: Compressing objects: 91% (147/161) 2024-08-06T20:22:07.5892708Z remote: Compressing objects: 92% (149/161) 2024-08-06T20:22:07.5893413Z remote: Compressing objects: 93% (150/161) 2024-08-06T20:22:07.5894137Z remote: Compressing objects: 94% (152/161) 2024-08-06T20:22:07.5894943Z remote: Compressing objects: 95% (153/161) 2024-08-06T20:22:07.5895660Z remote: Compressing objects: 96% (155/161) 2024-08-06T20:22:07.5896388Z remote: Compressing objects: 97% (157/161) 2024-08-06T20:22:07.5897094Z remote: Compressing objects: 98% (158/161) 2024-08-06T20:22:07.5897795Z remote: Compressing objects: 99% (160/161) 2024-08-06T20:22:07.5898498Z remote: Compressing objects: 100% (161/161) 2024-08-06T20:22:07.5899225Z remote: Compressing objects: 100% (161/161), done. 2024-08-06T20:22:27.9907262Z remote: Total 1008673 (delta 139), reused 248 (delta 126), pack-reused 1008384 2024-08-06T20:22:55.0922080Z [command]/usr/bin/git rev-parse --verify --quiet b9d86fa89636e301796d4201f36d86c73f6e49bc^{object} 2024-08-06T20:22:55.0951722Z b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:22:55.0955952Z ##[endgroup] 2024-08-06T20:22:55.0956444Z ##[group]Determining the checkout info 2024-08-06T20:22:55.0957289Z ##[endgroup] 2024-08-06T20:22:55.0957728Z ##[group]Checking out the ref 2024-08-06T20:22:55.0960769Z [command]/usr/bin/git checkout --quiet --force b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:22:56.4707973Z ##[endgroup] 2024-08-06T20:22:56.4708460Z ##[group]Setting up auth for fetching submodules 2024-08-06T20:22:56.4712163Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-06T20:22:56.4764187Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-06T20:22:56.4795030Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-06T20:22:56.4823971Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-06T20:22:56.4849818Z ##[endgroup] 2024-08-06T20:22:56.4850226Z ##[group]Fetching submodules 2024-08-06T20:22:56.4853252Z [command]/usr/bin/git submodule sync --recursive 2024-08-06T20:22:56.5136162Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-06T20:22:56.5415784Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-06T20:22:56.5417144Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-06T20:22:56.5418708Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-06T20:22:56.5421207Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-06T20:22:56.5424136Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-06T20:22:56.5426847Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-06T20:22:56.5429390Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-06T20:22:56.5432790Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-06T20:22:56.5436271Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-06T20:22:56.5439456Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-06T20:22:56.5442540Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-06T20:22:56.5445832Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-06T20:22:56.5449272Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-06T20:22:56.5452833Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-06T20:22:56.5456346Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-06T20:22:56.5460611Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-08-06T20:22:56.5466116Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:22:56.5469965Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-06T20:22:56.5473971Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-06T20:22:56.5478106Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-06T20:22:56.5482293Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-06T20:22:56.5486530Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-06T20:22:56.5490893Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-06T20:22:56.5495340Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-06T20:22:56.5499838Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-06T20:22:56.5504296Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-06T20:22:56.5509290Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-06T20:22:56.5513934Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-06T20:22:56.5518881Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-06T20:22:56.5523889Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-06T20:22:56.5529039Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-06T20:22:56.5534418Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-06T20:22:56.5539863Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-06T20:22:56.5546836Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-06T20:22:56.5552270Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-06T20:22:56.5581771Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-06T20:22:56.8423408Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-06T20:22:57.0684625Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-06T20:22:57.2582264Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-06T20:22:57.5050333Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-06T20:22:59.6327461Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-06T20:23:13.1841364Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-06T20:23:13.6156384Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-06T20:23:14.1123833Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-06T20:23:14.7430913Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-06T20:23:15.9063641Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-06T20:23:17.7902369Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-06T20:23:23.1644175Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-06T20:23:24.5194273Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-06T20:23:26.5595574Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-06T20:23:27.9758547Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-08-06T20:23:28.1663782Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-06T20:23:28.6918481Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-06T20:23:29.0755669Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-06T20:23:30.2579502Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-06T20:23:30.7177886Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-06T20:23:30.9981660Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-06T20:23:32.4990226Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-06T20:23:33.2538465Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-06T20:23:33.9318739Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-06T20:23:40.2415674Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-06T20:23:42.4154380Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-06T20:23:48.2783092Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-06T20:23:48.5127921Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-06T20:23:57.8472750Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-06T20:23:58.0260934Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-06T20:23:58.2422247Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-06T20:23:59.1062755Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-06T20:23:59.4166304Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-06T20:24:00.1734797Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-06T20:24:00.5771536Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-06T20:24:00.5884720Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-06T20:24:00.5971149Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-06T20:24:00.6204387Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-06T20:24:00.6579024Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-06T20:24:01.5933388Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-06T20:24:01.6167374Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-06T20:24:01.6617743Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-06T20:24:01.7574854Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-06T20:24:01.7903879Z Submodule path 'third_party/cudnn_frontend': checked out '98ca4e1941fe3263f128f74f10063a3ea35c7019' 2024-08-06T20:24:02.2795481Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-06T20:24:02.5266639Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-06T20:24:02.6060268Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-06T20:24:02.6076864Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:24:02.6079108Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:24:02.6081471Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:24:02.6083974Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:24:02.6086620Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:24:02.6112599Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-06T20:24:03.7535498Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-06T20:24:04.3875463Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-06T20:24:06.3285922Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-06T20:24:07.5189731Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-06T20:24:07.8429628Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-06T20:24:07.9374564Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-06T20:24:08.3284213Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-06T20:24:08.3897752Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-06T20:24:08.4023689Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-06T20:24:08.5170603Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-06T20:24:08.5577740Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-06T20:24:08.5672327Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-08-06T20:24:08.6069246Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-06T20:24:08.6323519Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-06T20:24:08.6774875Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-06T20:24:08.6900556Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-06T20:24:08.6915816Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-06T20:24:08.6940643Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-06T20:24:22.7298318Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-06T20:24:22.7472465Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-06T20:24:22.8309968Z Submodule path 'third_party/kineto': checked out 'da2f2682cabaf95d601fa2a9b7e0979f84fe7667' 2024-08-06T20:24:22.8329308Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:24:22.8331969Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:24:22.8335612Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:24:22.8361165Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-06T20:24:23.3970286Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-06T20:24:24.7233310Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-06T20:24:26.1149537Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-06T20:24:26.1164731Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:24:26.1166912Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:24:26.1169390Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:24:26.1171931Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:24:26.1174553Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:24:26.1177448Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:24:26.1180817Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:24:26.1183888Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:24:26.1210064Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-06T20:24:26.9862615Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-06T20:24:27.5098044Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-06T20:24:28.8164605Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-06T20:24:29.1407761Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-06T20:24:29.6997501Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-06T20:24:30.9615943Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-06T20:24:38.7009806Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-06T20:24:39.1656900Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-06T20:24:39.1837804Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-06T20:24:39.2198607Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-06T20:24:39.2328153Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-06T20:24:39.2342266Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:24:39.2367343Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-06T20:24:39.6459933Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-06T20:24:39.6642666Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-06T20:24:39.7047397Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-06T20:24:39.8069493Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-06T20:24:39.8228560Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-06T20:24:39.8630360Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-06T20:24:39.9218546Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-06T20:24:39.9592508Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-06T20:24:39.9826583Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-06T20:24:40.0833399Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-06T20:24:40.4243650Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-06T20:24:40.4278133Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-06T20:24:40.4280202Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-06T20:24:40.4307470Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-06T20:24:40.9045849Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-06T20:24:41.8006344Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-06T20:24:41.8338023Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-06T20:24:41.9003842Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-06T20:24:41.9024003Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:24:41.9025550Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:24:41.9027606Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:24:41.9030701Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:24:41.9033900Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:24:41.9036415Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:24:41.9039131Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:24:41.9042000Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:24:41.9067710Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-06T20:24:42.3015446Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-06T20:24:43.5312211Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-06T20:24:44.7409035Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-06T20:24:51.0195651Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-06T20:24:51.2917018Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-06T20:24:51.4892357Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-06T20:24:51.7747069Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-06T20:24:57.8743148Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-06T20:24:57.9144625Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-06T20:24:57.9299273Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-06T20:24:58.0267714Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-06T20:24:58.0396786Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-06T20:24:58.0540333Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-06T20:24:58.0693925Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-06T20:24:58.0708632Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:24:58.0710774Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:24:58.0737388Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-06T20:25:00.2948232Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-06T20:25:01.7402808Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-06T20:25:01.7861648Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-06T20:25:02.2372695Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-06T20:25:02.2488216Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-06T20:25:02.5066857Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-06T20:25:02.5087549Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:25:02.5089898Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-06T20:25:02.5116194Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-06T20:25:03.0077160Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-06T20:25:04.2644661Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-06T20:25:04.3351401Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-06T20:25:04.3441380Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-06T20:25:04.3560028Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-06T20:25:04.3922534Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-06T20:25:04.4203162Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-06T20:25:04.4605331Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-06T20:25:04.4865528Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-06T20:25:04.4882051Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:25:04.4884232Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:25:04.4886527Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:25:04.4888833Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:25:04.4914495Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-06T20:25:05.6673872Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-06T20:25:05.9022307Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-06T20:25:07.0909546Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-06T20:25:08.0657568Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-06T20:25:08.0815976Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-06T20:25:08.1497718Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-06T20:25:08.1788129Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-06T20:25:08.1803652Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:25:08.1832175Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-06T20:25:08.3723789Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-06T20:25:08.3755949Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-06T20:25:08.4042879Z Entering 'android/libs/fbjni' 2024-08-06T20:25:08.4083905Z Entering 'third_party/FP16' 2024-08-06T20:25:08.4123819Z Entering 'third_party/FXdiv' 2024-08-06T20:25:08.4163795Z Entering 'third_party/NNPACK' 2024-08-06T20:25:08.4203323Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:25:08.4243062Z Entering 'third_party/XNNPACK' 2024-08-06T20:25:08.4300713Z Entering 'third_party/benchmark' 2024-08-06T20:25:08.4341322Z Entering 'third_party/cpp-httplib' 2024-08-06T20:25:08.4380786Z Entering 'third_party/cpuinfo' 2024-08-06T20:25:08.4420515Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:25:08.4460599Z Entering 'third_party/cutlass' 2024-08-06T20:25:08.4509161Z Entering 'third_party/eigen' 2024-08-06T20:25:08.4552541Z Entering 'third_party/fbgemm' 2024-08-06T20:25:08.4593723Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:25:08.4634169Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:25:08.4673738Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:25:08.4719490Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:25:08.4758910Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:25:08.4798935Z Entering 'third_party/flatbuffers' 2024-08-06T20:25:08.4842308Z Entering 'third_party/fmt' 2024-08-06T20:25:08.4880846Z Entering 'third_party/foxi' 2024-08-06T20:25:08.4920378Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:25:08.4960390Z Entering 'third_party/gloo' 2024-08-06T20:25:08.4998923Z Entering 'third_party/googletest' 2024-08-06T20:25:08.5038226Z Entering 'third_party/ideep' 2024-08-06T20:25:08.5077146Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:25:08.5124413Z Entering 'third_party/ittapi' 2024-08-06T20:25:08.5165417Z Entering 'third_party/kineto' 2024-08-06T20:25:08.5205091Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:25:08.5245779Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:25:08.5286888Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:25:08.5325465Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:25:08.5364487Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:25:08.5402630Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:25:08.5444199Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:25:08.5483290Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:25:08.5523057Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:25:08.5563675Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:25:08.5605201Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:25:08.5644466Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:25:08.5684793Z Entering 'third_party/mimalloc' 2024-08-06T20:25:08.5724183Z Entering 'third_party/nccl/nccl' 2024-08-06T20:25:08.5764716Z Entering 'third_party/nlohmann' 2024-08-06T20:25:08.5804919Z Entering 'third_party/onnx' 2024-08-06T20:25:08.5861203Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:25:08.5902297Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:25:08.5944262Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:25:08.5986335Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:25:08.6026088Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:25:08.6065368Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:25:08.6104049Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:25:08.6143753Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:25:08.6182645Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:25:08.6220991Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:25:08.6260019Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:25:08.6299671Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:25:08.6342048Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:25:08.6402086Z Entering 'third_party/pocketfft' 2024-08-06T20:25:08.6442005Z Entering 'third_party/protobuf' 2024-08-06T20:25:08.6484956Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:25:08.6523686Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:25:08.6564003Z Entering 'third_party/psimd' 2024-08-06T20:25:08.6602460Z Entering 'third_party/pthreadpool' 2024-08-06T20:25:08.6641039Z Entering 'third_party/pybind11' 2024-08-06T20:25:08.6679551Z Entering 'third_party/python-peachpy' 2024-08-06T20:25:08.6717339Z Entering 'third_party/sleef' 2024-08-06T20:25:08.6756119Z Entering 'third_party/tensorpipe' 2024-08-06T20:25:08.6794624Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:25:08.6833672Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:25:08.6872816Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:25:08.6912652Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:25:08.6952778Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:25:08.7005219Z ##[endgroup] 2024-08-06T20:25:08.7006452Z ##[group]Persisting credentials for submodules 2024-08-06T20:25:08.7011356Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-06T20:25:08.7300493Z Entering 'android/libs/fbjni' 2024-08-06T20:25:08.7355073Z Entering 'third_party/FP16' 2024-08-06T20:25:08.7404719Z Entering 'third_party/FXdiv' 2024-08-06T20:25:08.7454645Z Entering 'third_party/NNPACK' 2024-08-06T20:25:08.7504644Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:25:08.7554391Z Entering 'third_party/XNNPACK' 2024-08-06T20:25:08.7621869Z Entering 'third_party/benchmark' 2024-08-06T20:25:08.7672898Z Entering 'third_party/cpp-httplib' 2024-08-06T20:25:08.7724319Z Entering 'third_party/cpuinfo' 2024-08-06T20:25:08.7775289Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:25:08.7825277Z Entering 'third_party/cutlass' 2024-08-06T20:25:08.7884675Z Entering 'third_party/eigen' 2024-08-06T20:25:08.7938666Z Entering 'third_party/fbgemm' 2024-08-06T20:25:08.7989085Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:25:08.8039761Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:25:08.8089815Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:25:08.8147128Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:25:08.8197958Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:25:08.8251706Z Entering 'third_party/flatbuffers' 2024-08-06T20:25:08.8307176Z Entering 'third_party/fmt' 2024-08-06T20:25:08.8358197Z Entering 'third_party/foxi' 2024-08-06T20:25:08.8408564Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:25:08.8459103Z Entering 'third_party/gloo' 2024-08-06T20:25:08.8510126Z Entering 'third_party/googletest' 2024-08-06T20:25:08.8561053Z Entering 'third_party/ideep' 2024-08-06T20:25:08.8609900Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:25:08.8667474Z Entering 'third_party/ittapi' 2024-08-06T20:25:08.8717331Z Entering 'third_party/kineto' 2024-08-06T20:25:08.8768163Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:25:08.8819263Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:25:08.8872664Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:25:08.8922757Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:25:08.8973106Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:25:08.9021351Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:25:08.9076138Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:25:08.9127887Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:25:08.9180980Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:25:08.9232461Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:25:08.9284066Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:25:08.9335686Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:25:08.9388882Z Entering 'third_party/mimalloc' 2024-08-06T20:25:08.9439899Z Entering 'third_party/nccl/nccl' 2024-08-06T20:25:08.9490944Z Entering 'third_party/nlohmann' 2024-08-06T20:25:08.9545991Z Entering 'third_party/onnx' 2024-08-06T20:25:08.9613397Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:25:08.9665519Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:25:08.9719373Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:25:08.9772677Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:25:08.9823975Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:25:08.9876556Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:25:08.9927120Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:25:08.9981886Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:25:09.0033734Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:25:09.0085015Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:25:09.0136025Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:25:09.0190060Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:25:09.0242874Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:25:09.0314590Z Entering 'third_party/pocketfft' 2024-08-06T20:25:09.0368885Z Entering 'third_party/protobuf' 2024-08-06T20:25:09.0423831Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:25:09.0476110Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:25:09.0528552Z Entering 'third_party/psimd' 2024-08-06T20:25:09.0582704Z Entering 'third_party/pthreadpool' 2024-08-06T20:25:09.0639691Z Entering 'third_party/pybind11' 2024-08-06T20:25:09.0690197Z Entering 'third_party/python-peachpy' 2024-08-06T20:25:09.0743304Z Entering 'third_party/sleef' 2024-08-06T20:25:09.0794561Z Entering 'third_party/tensorpipe' 2024-08-06T20:25:09.0846558Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:25:09.0897504Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:25:09.0947413Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:25:09.0997573Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:25:09.1048398Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:25:09.1116614Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-06T20:25:09.1405959Z Entering 'android/libs/fbjni' 2024-08-06T20:25:09.1455004Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-06T20:25:09.1470528Z Entering 'third_party/FP16' 2024-08-06T20:25:09.1519483Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-06T20:25:09.1535217Z Entering 'third_party/FXdiv' 2024-08-06T20:25:09.1583874Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-06T20:25:09.1597678Z Entering 'third_party/NNPACK' 2024-08-06T20:25:09.1646027Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-06T20:25:09.1663000Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:25:09.1710772Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-06T20:25:09.1726002Z Entering 'third_party/XNNPACK' 2024-08-06T20:25:09.1773510Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-06T20:25:09.1805867Z Entering 'third_party/benchmark' 2024-08-06T20:25:09.1853689Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-06T20:25:09.1869346Z Entering 'third_party/cpp-httplib' 2024-08-06T20:25:09.1917148Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-06T20:25:09.1935905Z Entering 'third_party/cpuinfo' 2024-08-06T20:25:09.1984788Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-06T20:25:09.2001043Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:25:09.2050171Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-06T20:25:09.2065656Z Entering 'third_party/cutlass' 2024-08-06T20:25:09.2113579Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-06T20:25:09.2138642Z Entering 'third_party/eigen' 2024-08-06T20:25:09.2187844Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-06T20:25:09.2205317Z Entering 'third_party/fbgemm' 2024-08-06T20:25:09.2254585Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-06T20:25:09.2270059Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:25:09.2318403Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-06T20:25:09.2333500Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:25:09.2381055Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-06T20:25:09.2396078Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:25:09.2444353Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-06T20:25:09.2466412Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:25:09.2513446Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:25:09.2528082Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:25:09.2575461Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-06T20:25:09.2592078Z Entering 'third_party/flatbuffers' 2024-08-06T20:25:09.2643902Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-06T20:25:09.2661286Z Entering 'third_party/fmt' 2024-08-06T20:25:09.2708894Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-06T20:25:09.2725108Z Entering 'third_party/foxi' 2024-08-06T20:25:09.2774592Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-08-06T20:25:09.2789692Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:25:09.2839148Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-06T20:25:09.2854368Z Entering 'third_party/gloo' 2024-08-06T20:25:09.2904179Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-06T20:25:09.2920392Z Entering 'third_party/googletest' 2024-08-06T20:25:09.2971470Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:25:09.2987133Z Entering 'third_party/ideep' 2024-08-06T20:25:09.3036095Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-06T20:25:09.3050519Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:25:09.3100675Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-06T20:25:09.3124968Z Entering 'third_party/ittapi' 2024-08-06T20:25:09.3177914Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-06T20:25:09.3194058Z Entering 'third_party/kineto' 2024-08-06T20:25:09.3245625Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-06T20:25:09.3261601Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:25:09.3312391Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-06T20:25:09.3326600Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:25:09.3378233Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-06T20:25:09.3394317Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:25:09.3443754Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-06T20:25:09.3457885Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:25:09.3506644Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-06T20:25:09.3521334Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:25:09.3571542Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-06T20:25:09.3585303Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:25:09.3634394Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-06T20:25:09.3650752Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:25:09.3699216Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-06T20:25:09.3713829Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:25:09.3762356Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:25:09.3777159Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:25:09.3825672Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-06T20:25:09.3841717Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:25:09.3890181Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-06T20:25:09.3906755Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:25:09.3955150Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-06T20:25:09.3969743Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:25:09.4016893Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-06T20:25:09.4034280Z Entering 'third_party/mimalloc' 2024-08-06T20:25:09.4083068Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-06T20:25:09.4098421Z Entering 'third_party/nccl/nccl' 2024-08-06T20:25:09.4147478Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-06T20:25:09.4163952Z Entering 'third_party/nlohmann' 2024-08-06T20:25:09.4213260Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-06T20:25:09.4230025Z Entering 'third_party/onnx' 2024-08-06T20:25:09.4278451Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-06T20:25:09.4310118Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:25:09.4363298Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-06T20:25:09.4378742Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:25:09.4426373Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-06T20:25:09.4445181Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:25:09.4494507Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-06T20:25:09.4511676Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:25:09.4560870Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-06T20:25:09.4575795Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:25:09.4622366Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:25:09.4637477Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:25:09.4684184Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-06T20:25:09.4698702Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:25:09.4746365Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-06T20:25:09.4762558Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:25:09.4809514Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-06T20:25:09.4824028Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:25:09.4871524Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-06T20:25:09.4886586Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:25:09.4934339Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-06T20:25:09.4948088Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:25:09.4996307Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-06T20:25:09.5013307Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:25:09.5062685Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-06T20:25:09.5079372Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:25:09.5126577Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-06T20:25:09.5162454Z Entering 'third_party/pocketfft' 2024-08-06T20:25:09.5211298Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-06T20:25:09.5227703Z Entering 'third_party/protobuf' 2024-08-06T20:25:09.5276633Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-06T20:25:09.5294365Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:25:09.5342191Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-06T20:25:09.5357022Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:25:09.5403929Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:25:09.5421342Z Entering 'third_party/psimd' 2024-08-06T20:25:09.5470145Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-06T20:25:09.5486863Z Entering 'third_party/pthreadpool' 2024-08-06T20:25:09.5533710Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-06T20:25:09.5549288Z Entering 'third_party/pybind11' 2024-08-06T20:25:09.5597322Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-06T20:25:09.5612872Z Entering 'third_party/python-peachpy' 2024-08-06T20:25:09.5661267Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-06T20:25:09.5677711Z Entering 'third_party/sleef' 2024-08-06T20:25:09.5724995Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-06T20:25:09.5741091Z Entering 'third_party/tensorpipe' 2024-08-06T20:25:09.5791344Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-06T20:25:09.5807893Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:25:09.5855868Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-06T20:25:09.5870766Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:25:09.5918552Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-06T20:25:09.5933269Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:25:09.5980051Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-06T20:25:09.5994854Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:25:09.6042402Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-06T20:25:09.6056292Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:25:09.6105771Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-06T20:25:09.6935843Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-06T20:25:09.7230893Z Entering 'android/libs/fbjni' 2024-08-06T20:25:09.7271347Z Entering 'third_party/FP16' 2024-08-06T20:25:09.7312703Z Entering 'third_party/FXdiv' 2024-08-06T20:25:09.7352725Z Entering 'third_party/NNPACK' 2024-08-06T20:25:09.7393198Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:25:09.7433395Z Entering 'third_party/XNNPACK' 2024-08-06T20:25:09.7491279Z Entering 'third_party/benchmark' 2024-08-06T20:25:09.7532264Z Entering 'third_party/cpp-httplib' 2024-08-06T20:25:09.7572911Z Entering 'third_party/cpuinfo' 2024-08-06T20:25:09.7612992Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:25:09.7654007Z Entering 'third_party/cutlass' 2024-08-06T20:25:09.7701486Z Entering 'third_party/eigen' 2024-08-06T20:25:09.7743915Z Entering 'third_party/fbgemm' 2024-08-06T20:25:09.7783457Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:25:09.7822743Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:25:09.7861265Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:25:09.7907456Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:25:09.7945828Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:25:09.7986978Z Entering 'third_party/flatbuffers' 2024-08-06T20:25:09.8029992Z Entering 'third_party/fmt' 2024-08-06T20:25:09.8069212Z Entering 'third_party/foxi' 2024-08-06T20:25:09.8109490Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:25:09.8149157Z Entering 'third_party/gloo' 2024-08-06T20:25:09.8188426Z Entering 'third_party/googletest' 2024-08-06T20:25:09.8227466Z Entering 'third_party/ideep' 2024-08-06T20:25:09.8265966Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:25:09.8314159Z Entering 'third_party/ittapi' 2024-08-06T20:25:09.8355053Z Entering 'third_party/kineto' 2024-08-06T20:25:09.8395573Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:25:09.8436612Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:25:09.8477813Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:25:09.8516853Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:25:09.8556366Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:25:09.8595958Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:25:09.8640124Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:25:09.8679613Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:25:09.8719199Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:25:09.8763946Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:25:09.8807460Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:25:09.8848284Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:25:09.8891336Z Entering 'third_party/mimalloc' 2024-08-06T20:25:09.8933781Z Entering 'third_party/nccl/nccl' 2024-08-06T20:25:09.8974511Z Entering 'third_party/nlohmann' 2024-08-06T20:25:09.9016531Z Entering 'third_party/onnx' 2024-08-06T20:25:09.9074187Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:25:09.9115237Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:25:09.9158798Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:25:09.9202876Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:25:09.9244571Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:25:09.9284576Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:25:09.9324826Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:25:09.9365789Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:25:09.9404678Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:25:09.9443290Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:25:09.9482401Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:25:09.9524412Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:25:09.9566713Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:25:09.9626227Z Entering 'third_party/pocketfft' 2024-08-06T20:25:09.9666785Z Entering 'third_party/protobuf' 2024-08-06T20:25:09.9710297Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:25:09.9750341Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:25:09.9790562Z Entering 'third_party/psimd' 2024-08-06T20:25:09.9832573Z Entering 'third_party/pthreadpool' 2024-08-06T20:25:09.9871090Z Entering 'third_party/pybind11' 2024-08-06T20:25:09.9911088Z Entering 'third_party/python-peachpy' 2024-08-06T20:25:09.9950923Z Entering 'third_party/sleef' 2024-08-06T20:25:09.9991844Z Entering 'third_party/tensorpipe' 2024-08-06T20:25:10.0031881Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:25:10.0071130Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:25:10.0108956Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:25:10.0148245Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:25:10.0186238Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:25:10.0240564Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-06T20:25:10.0527399Z Entering 'android/libs/fbjni' 2024-08-06T20:25:10.0568793Z Entering 'third_party/FP16' 2024-08-06T20:25:10.0609469Z Entering 'third_party/FXdiv' 2024-08-06T20:25:10.0653202Z Entering 'third_party/NNPACK' 2024-08-06T20:25:10.0693425Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T20:25:10.0733326Z Entering 'third_party/XNNPACK' 2024-08-06T20:25:10.0790693Z Entering 'third_party/benchmark' 2024-08-06T20:25:10.0831806Z Entering 'third_party/cpp-httplib' 2024-08-06T20:25:10.0871906Z Entering 'third_party/cpuinfo' 2024-08-06T20:25:10.0912937Z Entering 'third_party/cudnn_frontend' 2024-08-06T20:25:10.0953193Z Entering 'third_party/cutlass' 2024-08-06T20:25:10.1001659Z Entering 'third_party/eigen' 2024-08-06T20:25:10.1044919Z Entering 'third_party/fbgemm' 2024-08-06T20:25:10.1085709Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T20:25:10.1124832Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T20:25:10.1166763Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T20:25:10.1213007Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T20:25:10.1252373Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T20:25:10.1292845Z Entering 'third_party/flatbuffers' 2024-08-06T20:25:10.1337931Z Entering 'third_party/fmt' 2024-08-06T20:25:10.1377272Z Entering 'third_party/foxi' 2024-08-06T20:25:10.1416277Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T20:25:10.1457357Z Entering 'third_party/gloo' 2024-08-06T20:25:10.1497295Z Entering 'third_party/googletest' 2024-08-06T20:25:10.1537800Z Entering 'third_party/ideep' 2024-08-06T20:25:10.1576723Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T20:25:10.1622816Z Entering 'third_party/ittapi' 2024-08-06T20:25:10.1663540Z Entering 'third_party/kineto' 2024-08-06T20:25:10.1702966Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T20:25:10.1742181Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T20:25:10.1782512Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T20:25:10.1822177Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T20:25:10.1863688Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T20:25:10.1899681Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T20:25:10.1942270Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T20:25:10.1980694Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T20:25:10.2019543Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T20:25:10.2059995Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T20:25:10.2100618Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T20:25:10.2139378Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T20:25:10.2180574Z Entering 'third_party/mimalloc' 2024-08-06T20:25:10.2222855Z Entering 'third_party/nccl/nccl' 2024-08-06T20:25:10.2263314Z Entering 'third_party/nlohmann' 2024-08-06T20:25:10.2304547Z Entering 'third_party/onnx' 2024-08-06T20:25:10.2361659Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T20:25:10.2402286Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T20:25:10.2446924Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T20:25:10.2490269Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T20:25:10.2529610Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T20:25:10.2569557Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T20:25:10.2607801Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T20:25:10.2647906Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T20:25:10.2687313Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T20:25:10.2726101Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T20:25:10.2765488Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T20:25:10.2806288Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T20:25:10.2848245Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T20:25:10.2908419Z Entering 'third_party/pocketfft' 2024-08-06T20:25:10.2948513Z Entering 'third_party/protobuf' 2024-08-06T20:25:10.2991336Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T20:25:10.3029318Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T20:25:10.3069215Z Entering 'third_party/psimd' 2024-08-06T20:25:10.3108813Z Entering 'third_party/pthreadpool' 2024-08-06T20:25:10.3149463Z Entering 'third_party/pybind11' 2024-08-06T20:25:10.3189979Z Entering 'third_party/python-peachpy' 2024-08-06T20:25:10.3230669Z Entering 'third_party/sleef' 2024-08-06T20:25:10.3272313Z Entering 'third_party/tensorpipe' 2024-08-06T20:25:10.3311753Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T20:25:10.3350367Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T20:25:10.3387963Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T20:25:10.3426065Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T20:25:10.3463142Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T20:25:10.3513650Z ##[endgroup] 2024-08-06T20:25:10.3556614Z [command]/usr/bin/git log -1 --format='%H' 2024-08-06T20:25:10.3585687Z 'b9d86fa89636e301796d4201f36d86c73f6e49bc' 2024-08-06T20:25:10.3743834Z Prepare all required actions 2024-08-06T20:25:10.3744431Z Getting action download info 2024-08-06T20:25:10.5377897Z ##[group]Run ./.github/actions/setup-linux 2024-08-06T20:25:10.5378240Z env: 2024-08-06T20:25:10.5378461Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:25:10.5378738Z ##[endgroup] 2024-08-06T20:25:10.5428363Z ##[group]Run set -euo pipefail 2024-08-06T20:25:10.5428727Z set -euo pipefail 2024-08-06T20:25:10.5429019Z function get_ec2_metadata() { 2024-08-06T20:25:10.5429412Z  # Pulled from instance metadata endpoint for EC2 2024-08-06T20:25:10.5430063Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-06T20:25:10.5430847Z  category=$1 2024-08-06T20:25:10.5431223Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-06T20:25:10.5431673Z  runner_name_str=i-064ea195702d7f670 2024-08-06T20:25:10.5432095Z  if [[ -f /.inarc ]]; then 2024-08-06T20:25:10.5432453Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-06T20:25:10.5432840Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-06T20:25:10.5433323Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-06T20:25:10.5433765Z  else 2024-08-06T20:25:10.5434107Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-06T20:25:10.5434530Z  fi 2024-08-06T20:25:10.5434770Z } 2024-08-06T20:25:10.5435045Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-06T20:25:10.5435472Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-06T20:25:10.5435960Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-06T20:25:10.5436388Z echo "system info $(uname -a)" 2024-08-06T20:25:10.5443686Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:25:10.5444102Z env: 2024-08-06T20:25:10.5444343Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:25:10.5444606Z ##[endgroup] 2024-08-06T20:25:10.5537721Z ami-id: ami-06c68f701d8090592 2024-08-06T20:25:10.5587485Z instance-id: i-064ea195702d7f670 2024-08-06T20:25:10.5634015Z instance-type: c5.2xlarge 2024-08-06T20:25:10.5644242Z system info Linux ip-10-0-33-59.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-06T20:25:10.5675341Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-06T20:25:10.5676008Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-06T20:25:10.5681920Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:25:10.5682310Z env: 2024-08-06T20:25:10.5682530Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:25:10.5682802Z ##[endgroup] 2024-08-06T20:25:10.5756167Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-06T20:25:10.5756686Z if systemctl is-active --quiet docker; then 2024-08-06T20:25:10.5757070Z  echo "Docker daemon is running..."; 2024-08-06T20:25:10.5757405Z else 2024-08-06T20:25:10.5757768Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-06T20:25:10.5758187Z fi 2024-08-06T20:25:10.5763539Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:25:10.5763925Z env: 2024-08-06T20:25:10.5764135Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:25:10.5764405Z ##[endgroup] 2024-08-06T20:25:10.5832640Z Docker daemon is running... 2024-08-06T20:25:10.5882197Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-06T20:25:10.5882623Z with: 2024-08-06T20:25:10.5882849Z shell: bash 2024-08-06T20:25:10.5883091Z timeout_minutes: 5 2024-08-06T20:25:10.5883336Z max_attempts: 3 2024-08-06T20:25:10.5883589Z retry_wait_seconds: 30 2024-08-06T20:25:10.5884838Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-08-06T20:25:10.5886057Z polling_interval_seconds: 1 2024-08-06T20:25:10.5886345Z warning_on_retry: true 2024-08-06T20:25:10.5886623Z continue_on_error: false 2024-08-06T20:25:10.5886891Z env: 2024-08-06T20:25:10.5887097Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:25:10.5887373Z AWS_RETRY_MODE: standard 2024-08-06T20:25:10.5887643Z AWS_MAX_ATTEMPTS: 5 2024-08-06T20:25:10.5887895Z AWS_DEFAULT_REGION: us-east-1 2024-08-06T20:25:10.5888177Z ##[endgroup] 2024-08-06T20:25:11.8108500Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-06T20:25:11.8109152Z Configure a credential helper to remove this warning. See 2024-08-06T20:25:11.8109736Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-06T20:25:11.8110196Z 2024-08-06T20:25:11.8110316Z Login Succeeded 2024-08-06T20:25:12.6450379Z Command completed after 1 attempt(s). 2024-08-06T20:25:12.6506903Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-06T20:25:12.6507472Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-06T20:25:12.6507927Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-06T20:25:12.6514614Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:25:12.6515013Z env: 2024-08-06T20:25:12.6515232Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:25:12.6515510Z ##[endgroup] 2024-08-06T20:25:12.6599763Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-06T20:25:12.6600364Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-06T20:25:12.6600815Z # shellcheck disable=SC2046 2024-08-06T20:25:12.6601159Z docker stop $(docker ps -q) || true 2024-08-06T20:25:12.6601551Z # Prune all of the docker images 2024-08-06T20:25:12.6601887Z docker system prune -af 2024-08-06T20:25:12.6607747Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:25:12.6608148Z env: 2024-08-06T20:25:12.6608378Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:25:12.6608639Z ##[endgroup] 2024-08-06T20:25:12.6871294Z "docker stop" requires at least 1 argument. 2024-08-06T20:25:12.6871845Z See 'docker stop --help'. 2024-08-06T20:25:12.6872024Z 2024-08-06T20:25:12.6872327Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-06T20:25:12.6872896Z 2024-08-06T20:25:12.6873101Z Stop one or more running containers 2024-08-06T20:25:12.7024182Z Total reclaimed space: 0B 2024-08-06T20:25:12.7091076Z ##[group]Run set +e 2024-08-06T20:25:12.7091391Z set +e 2024-08-06T20:25:12.7091632Z set -x 2024-08-06T20:25:12.7091866Z  2024-08-06T20:25:12.7092107Z PT_DOMAIN=download.pytorch.org 2024-08-06T20:25:12.7092732Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-06T20:25:12.7093512Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-06T20:25:12.7094038Z # one is returned at random 2024-08-06T20:25:12.7094566Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-06T20:25:12.7094957Z  2024-08-06T20:25:12.7095184Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-06T20:25:12.7095624Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-06T20:25:12.7096157Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-06T20:25:12.7096547Z  2024-08-06T20:25:12.7096788Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-06T20:25:12.7097180Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-06T20:25:12.7097542Z  exit 1 2024-08-06T20:25:12.7097784Z  fi 2024-08-06T20:25:12.7098306Z fi 2024-08-06T20:25:12.7098519Z  2024-08-06T20:25:12.7098796Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-06T20:25:12.7099176Z  # Clean up any old records first 2024-08-06T20:25:12.7099537Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-06T20:25:12.7099883Z fi 2024-08-06T20:25:12.7100105Z  2024-08-06T20:25:12.7100412Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-06T20:25:12.7100825Z cat /etc/hosts 2024-08-06T20:25:12.7106799Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:25:12.7107184Z env: 2024-08-06T20:25:12.7107432Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:25:12.7107712Z ##[endgroup] 2024-08-06T20:25:12.7132029Z + PT_DOMAIN=download.pytorch.org 2024-08-06T20:25:12.7138011Z ++ dig -4 +short download.pytorch.org 2024-08-06T20:25:12.7138662Z ++ tail -n1 2024-08-06T20:25:12.7696849Z + RESOLVED_IP=18.160.10.36 2024-08-06T20:25:12.7697256Z + '[' -z 18.160.10.36 ']' 2024-08-06T20:25:12.7697564Z + grep -r download.pytorch.org /etc/hosts 2024-08-06T20:25:12.7708721Z 18.160.10.76 download.pytorch.org 2024-08-06T20:25:12.7709821Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-08-06T20:25:13.0210047Z + echo '18.160.10.36 download.pytorch.org' 2024-08-06T20:25:13.0211035Z + sudo tee -a /etc/hosts 2024-08-06T20:25:13.0640460Z 18.160.10.36 download.pytorch.org 2024-08-06T20:25:13.0657387Z + cat /etc/hosts 2024-08-06T20:25:13.0668155Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-06T20:25:13.0675890Z ::1 localhost6 localhost6.localdomain6 2024-08-06T20:25:13.0676288Z 18.160.10.36 download.pytorch.org 2024-08-06T20:25:13.0821763Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-06T20:25:13.0822255Z with: 2024-08-06T20:25:13.0822929Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:13.0823726Z docker-build-dir: .ci/docker 2024-08-06T20:25:13.0824022Z working-directory: . 2024-08-06T20:25:13.0824382Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:25:13.0824782Z force-push: false 2024-08-06T20:25:13.0825023Z env: 2024-08-06T20:25:13.0825234Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:25:13.0825513Z ##[endgroup] 2024-08-06T20:25:13.0854998Z ##[group]Run set -ex 2024-08-06T20:25:13.0855342Z set -ex 2024-08-06T20:25:13.0855588Z  2024-08-06T20:25:13.0856001Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-06T20:25:13.0856737Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-06T20:25:13.0857308Z # job could then download the pre-built image as usual 2024-08-06T20:25:13.0857842Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-06T20:25:13.0858407Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-06T20:25:13.0859018Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:25:13.0859555Z  2024-08-06T20:25:13.0859984Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-06T20:25:13.0860443Z  exit 0 2024-08-06T20:25:13.0860671Z else 2024-08-06T20:25:13.0860952Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-06T20:25:13.0861290Z fi 2024-08-06T20:25:13.0861500Z  2024-08-06T20:25:13.0861850Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-06T20:25:13.0862474Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-06T20:25:13.0863009Z  # use it as it is, but first let's extract the tag 2024-08-06T20:25:13.0863500Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-06T20:25:13.0864241Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:25:13.0864728Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:25:13.0865153Z else 2024-08-06T20:25:13.0865474Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-06T20:25:13.0865939Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:25:13.0866597Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:25:13.0867171Z fi 2024-08-06T20:25:13.0875783Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:25:13.0876173Z env: 2024-08-06T20:25:13.0876407Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:25:13.0876676Z REPO_NAME: pytorch 2024-08-06T20:25:13.0877377Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:13.0878164Z DOCKER_BUILD_DIR: .ci/docker 2024-08-06T20:25:13.0878546Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:25:13.0878933Z ##[endgroup] 2024-08-06T20:25:13.0907138Z + [[ ! -d .ci/docker ]] 2024-08-06T20:25:13.0907658Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-06T20:25:13.0908130Z + echo skip=false 2024-08-06T20:25:13.0909621Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-06T20:25:13.0914980Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:13.0915869Z ++ awk -F '[:,]' '{print $2}' 2024-08-06T20:25:13.0987305Z + DOCKER_TAG=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:13.0987819Z + echo docker-tag=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:13.0988692Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:13.1026146Z ##[group]Run set +e 2024-08-06T20:25:13.1026497Z set +e 2024-08-06T20:25:13.1026729Z set -x 2024-08-06T20:25:13.1026966Z  2024-08-06T20:25:13.1027188Z login() { 2024-08-06T20:25:13.1027666Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-06T20:25:13.1028199Z } 2024-08-06T20:25:13.1028422Z  2024-08-06T20:25:13.1028628Z retry () { 2024-08-06T20:25:13.1028922Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-06T20:25:13.1029253Z } 2024-08-06T20:25:13.1029458Z  2024-08-06T20:25:13.1029702Z retry login "${DOCKER_REGISTRY}" 2024-08-06T20:25:13.1030008Z  2024-08-06T20:25:13.1032318Z # Check if image already exists, if it does then skip building it 2024-08-06T20:25:13.1032847Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-06T20:25:13.1033228Z  exit 0 2024-08-06T20:25:13.1033450Z fi 2024-08-06T20:25:13.1033668Z  2024-08-06T20:25:13.1034057Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-06T20:25:13.1034653Z # be empty. The default action would be to continue rebuild the image 2024-08-06T20:25:13.1035176Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-06T20:25:13.1035646Z  # if we're on the base branch then use the parent commit 2024-08-06T20:25:13.1036053Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-06T20:25:13.1036382Z else 2024-08-06T20:25:13.1036718Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-06T20:25:13.1037190Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-06T20:25:13.1037776Z fi 2024-08-06T20:25:13.1037996Z  2024-08-06T20:25:13.1038224Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-06T20:25:13.1038592Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-06T20:25:13.1038938Z  2024-08-06T20:25:13.1039400Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-06T20:25:13.1039971Z  exit 0 2024-08-06T20:25:13.1040210Z fi 2024-08-06T20:25:13.1040414Z  2024-08-06T20:25:13.1040731Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-06T20:25:13.1041433Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-06T20:25:13.1042015Z  exit 1 2024-08-06T20:25:13.1042248Z fi 2024-08-06T20:25:13.1042468Z  2024-08-06T20:25:13.1042833Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-06T20:25:13.1043508Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-06T20:25:13.1044111Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-06T20:25:13.1044802Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-06T20:25:13.1045568Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-06T20:25:13.1046043Z fi 2024-08-06T20:25:13.1046270Z  2024-08-06T20:25:13.1046525Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-06T20:25:13.1053827Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:25:13.1054404Z env: 2024-08-06T20:25:13.1054626Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:25:13.1054916Z DOCKER_BUILD_DIR: .ci/docker 2024-08-06T20:25:13.1055268Z BASE_REVISION: 1736af7cf736184c356be1bb00f59fb2feea6d7d 2024-08-06T20:25:13.1056075Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:13.1056872Z DOCKER_TAG: 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:13.1057332Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:25:13.1057716Z ##[endgroup] 2024-08-06T20:25:13.1083509Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:25:13.1084166Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:25:13.1087010Z + aws ecr get-login-password --region us-east-1 2024-08-06T20:25:13.1088090Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:25:13.7035810Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-06T20:25:13.7036448Z Configure a credential helper to remove this warning. See 2024-08-06T20:25:13.7037070Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-06T20:25:13.7037533Z 2024-08-06T20:25:13.7037817Z Login Succeeded 2024-08-06T20:25:13.7050287Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:13.9424531Z { 2024-08-06T20:25:13.9424895Z "schemaVersion": 2, 2024-08-06T20:25:13.9425478Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-06T20:25:13.9426159Z "config": { 2024-08-06T20:25:13.9426609Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-06T20:25:13.9427100Z "size": 43802, 2024-08-06T20:25:13.9427836Z "digest": "sha256:eb2e8636b63e804e38c5e56190d8b860c4bf8a9e0794f88825eff958ed6b1e76" 2024-08-06T20:25:13.9428408Z }, 2024-08-06T20:25:13.9428619Z "layers": [ 2024-08-06T20:25:13.9428884Z { 2024-08-06T20:25:13.9429491Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9430536Z "size": 30439866, 2024-08-06T20:25:13.9430968Z "digest": "sha256:9b857f539cb142c9aa2201a17bb8e1cd5cf12edd4a65adf5732fe9f4343964cf" 2024-08-06T20:25:13.9431502Z }, 2024-08-06T20:25:13.9431823Z { 2024-08-06T20:25:13.9432380Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9432815Z "size": 1820, 2024-08-06T20:25:13.9433223Z "digest": "sha256:ceceb076946ee95300822daccb5d3e4e45e6857d34996ef89305d9a078410f8e" 2024-08-06T20:25:13.9433707Z }, 2024-08-06T20:25:13.9433910Z { 2024-08-06T20:25:13.9434235Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9434664Z "size": 313165371, 2024-08-06T20:25:13.9435100Z "digest": "sha256:042461771f00f834b40891f079173616ad761e9e3b2ec2eb25affa3041d71a3d" 2024-08-06T20:25:13.9435555Z }, 2024-08-06T20:25:13.9435759Z { 2024-08-06T20:25:13.9436096Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9436509Z "size": 802, 2024-08-06T20:25:13.9436934Z "digest": "sha256:bac001ac2fee5df3367ff9332bc1fcd706a8de3ad05a1885903f338197619520" 2024-08-06T20:25:13.9437423Z }, 2024-08-06T20:25:13.9437611Z { 2024-08-06T20:25:13.9437942Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9438366Z "size": 106, 2024-08-06T20:25:13.9438765Z "digest": "sha256:4a587220513dc642f8fe95f4ae1b2b643225db8ae8753fa25b68a4f84488c4ab" 2024-08-06T20:25:13.9439236Z }, 2024-08-06T20:25:13.9439432Z { 2024-08-06T20:25:13.9439749Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9440171Z "size": 542, 2024-08-06T20:25:13.9440738Z "digest": "sha256:af8081143f008c5cb1bd61f6d92fde22553cfc007b6a77c2fd2aaf6a9f0088a9" 2024-08-06T20:25:13.9441210Z }, 2024-08-06T20:25:13.9441532Z { 2024-08-06T20:25:13.9441864Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9442273Z "size": 1242, 2024-08-06T20:25:13.9442690Z "digest": "sha256:e7a423dcab0171df0317d2cf65153a533a002a33dce81ae5b0972b06d0838e26" 2024-08-06T20:25:13.9443181Z }, 2024-08-06T20:25:13.9443374Z { 2024-08-06T20:25:13.9443850Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9444552Z "size": 483, 2024-08-06T20:25:13.9445093Z "digest": "sha256:ad26542e0d01609840496a12691483f2c50eb8e23002a205becb5b695f9159fb" 2024-08-06T20:25:13.9445788Z }, 2024-08-06T20:25:13.9446005Z { 2024-08-06T20:25:13.9446328Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9446756Z "size": 110321946, 2024-08-06T20:25:13.9447185Z "digest": "sha256:9e1d4f8784a0033c6a3605f0126d93af125fed0dd0839965c97924ba177e350e" 2024-08-06T20:25:13.9447653Z }, 2024-08-06T20:25:13.9447857Z { 2024-08-06T20:25:13.9448188Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9448601Z "size": 3679, 2024-08-06T20:25:13.9449012Z "digest": "sha256:b1f815d162740bbe496657c7b85acd702ea1818a4d93554f8a822bd964920100" 2024-08-06T20:25:13.9449499Z }, 2024-08-06T20:25:13.9449686Z { 2024-08-06T20:25:13.9450017Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9450440Z "size": 1907, 2024-08-06T20:25:13.9450842Z "digest": "sha256:91f9dddffbee46981d9e74382704c1b53c87f1808350cf1dd518687805d3aa59" 2024-08-06T20:25:13.9451316Z }, 2024-08-06T20:25:13.9451516Z { 2024-08-06T20:25:13.9451834Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9452257Z "size": 699, 2024-08-06T20:25:13.9452674Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-06T20:25:13.9453140Z }, 2024-08-06T20:25:13.9453346Z { 2024-08-06T20:25:13.9453666Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9454210Z "size": 2627946145, 2024-08-06T20:25:13.9454647Z "digest": "sha256:8385f3c14a81343c1f329fe1449447035a2acfc7b7e7c552321cded4206f1e1d" 2024-08-06T20:25:13.9455277Z }, 2024-08-06T20:25:13.9455486Z { 2024-08-06T20:25:13.9456115Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9456533Z "size": 32, 2024-08-06T20:25:13.9456952Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:25:13.9457440Z }, 2024-08-06T20:25:13.9457627Z { 2024-08-06T20:25:13.9457963Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9458388Z "size": 378, 2024-08-06T20:25:13.9458923Z "digest": "sha256:202c71391ade27762d5e9cafde793db67b4d5e4d6869debe2643d5092101c461" 2024-08-06T20:25:13.9459403Z }, 2024-08-06T20:25:13.9459604Z { 2024-08-06T20:25:13.9459938Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9460361Z "size": 222877, 2024-08-06T20:25:13.9460795Z "digest": "sha256:240ee294d88b3c8decdc8b379b5bfae83e2224ed365151fe2e8026fca366d621" 2024-08-06T20:25:13.9461268Z }, 2024-08-06T20:25:13.9461476Z { 2024-08-06T20:25:13.9461812Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9462227Z "size": 231, 2024-08-06T20:25:13.9462652Z "digest": "sha256:7a823d3d7147e0cae46f715bfd2d9ffd953c6cadae532c88c8c857092e6456da" 2024-08-06T20:25:13.9463142Z }, 2024-08-06T20:25:13.9463331Z { 2024-08-06T20:25:13.9463667Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9464096Z "size": 2933351, 2024-08-06T20:25:13.9464511Z "digest": "sha256:a1b9e8856530ee2f7a1abe6b877bf49121fe149f66fbb6ba309902c35f139fb6" 2024-08-06T20:25:13.9464988Z }, 2024-08-06T20:25:13.9465189Z { 2024-08-06T20:25:13.9465609Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9466046Z "size": 1804, 2024-08-06T20:25:13.9466476Z "digest": "sha256:9c5b585a81167e4e7befcddfd54e7b522e6ed96b7c9a9773ef7099c23000ca20" 2024-08-06T20:25:13.9466952Z }, 2024-08-06T20:25:13.9467160Z { 2024-08-06T20:25:13.9467506Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9467918Z "size": 105, 2024-08-06T20:25:13.9468350Z "digest": "sha256:9cd1bcd9f117daa712ab4916cef270b8b4ecf401bed1df3784d55147457c5de5" 2024-08-06T20:25:13.9468838Z }, 2024-08-06T20:25:13.9469032Z { 2024-08-06T20:25:13.9469372Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9469788Z "size": 563, 2024-08-06T20:25:13.9470204Z "digest": "sha256:2b9731d295d60278e361d491fc274e5d558a89ec5adeb9fdf342d7a8cc17bfec" 2024-08-06T20:25:13.9470685Z }, 2024-08-06T20:25:13.9470880Z { 2024-08-06T20:25:13.9471214Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9471653Z "size": 32, 2024-08-06T20:25:13.9472061Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:25:13.9472547Z }, 2024-08-06T20:25:13.9472756Z { 2024-08-06T20:25:13.9473078Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9473510Z "size": 104, 2024-08-06T20:25:13.9473921Z "digest": "sha256:8d223c48f5429ef9db659237511d2d5f73345f1f217d316591adc008e8abb5a0" 2024-08-06T20:25:13.9474380Z }, 2024-08-06T20:25:13.9474584Z { 2024-08-06T20:25:13.9474915Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9475325Z "size": 502, 2024-08-06T20:25:13.9475744Z "digest": "sha256:49a0e418a175ac12f1b4b503e80e22679bc321e6163ed5b203af848204f0606a" 2024-08-06T20:25:13.9476220Z }, 2024-08-06T20:25:13.9476407Z { 2024-08-06T20:25:13.9476740Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9477167Z "size": 101109806, 2024-08-06T20:25:13.9477586Z "digest": "sha256:51ed474f6c3273503d24a4b965ff3b7e49e8684501e10413499ecc7249337a86" 2024-08-06T20:25:13.9478060Z }, 2024-08-06T20:25:13.9478261Z { 2024-08-06T20:25:13.9478580Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9479089Z "size": 109, 2024-08-06T20:25:13.9479504Z "digest": "sha256:1c01ecb575c64355bc7f0f73f60b8d212e7a83cd58347d573d0074c2a5926b57" 2024-08-06T20:25:13.9479973Z }, 2024-08-06T20:25:13.9480175Z { 2024-08-06T20:25:13.9480508Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9480921Z "size": 487, 2024-08-06T20:25:13.9481340Z "digest": "sha256:a2630e5fe166ae8ff146e4c14db0bdc1299ed2b6120108351c961adcd2375288" 2024-08-06T20:25:13.9481822Z }, 2024-08-06T20:25:13.9482008Z { 2024-08-06T20:25:13.9482341Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9482752Z "size": 340, 2024-08-06T20:25:13.9483175Z "digest": "sha256:554a42b2a2e61462bb2c2c08ed88f895679ad08a1e9295d040c239217102dc70" 2024-08-06T20:25:13.9483646Z }, 2024-08-06T20:25:13.9483834Z { 2024-08-06T20:25:13.9484161Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9484584Z "size": 103, 2024-08-06T20:25:13.9484993Z "digest": "sha256:c9a20a1fda169765309cb8a927a7ad545bddb403e1ed7db164678f1c0c3b5552" 2024-08-06T20:25:13.9485475Z }, 2024-08-06T20:25:13.9485679Z { 2024-08-06T20:25:13.9485999Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9486421Z "size": 1470, 2024-08-06T20:25:13.9486844Z "digest": "sha256:2c2e35b979f5fc0ecccef6efc157a97f33db01a77958fa33855dc367544a5c46" 2024-08-06T20:25:13.9487313Z }, 2024-08-06T20:25:13.9487518Z { 2024-08-06T20:25:13.9487853Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9488265Z "size": 629993175, 2024-08-06T20:25:13.9488779Z "digest": "sha256:0e3c02dff025006601de33f0303b8c2f62aefdee1d0c34db3a60419ebe1a7e30" 2024-08-06T20:25:13.9489268Z }, 2024-08-06T20:25:13.9489459Z { 2024-08-06T20:25:13.9489792Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9490223Z "size": 160, 2024-08-06T20:25:13.9490621Z "digest": "sha256:2c2cb9893903884e0cf968da484f3b97fb62e1a881f8d68b437c240c0f2f25d1" 2024-08-06T20:25:13.9491104Z }, 2024-08-06T20:25:13.9491305Z { 2024-08-06T20:25:13.9491624Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9492047Z "size": 2209, 2024-08-06T20:25:13.9492465Z "digest": "sha256:ec0a42784a49a934dcd7d8c99dbe058a3828080e6716a12cba391210891574f3" 2024-08-06T20:25:13.9492930Z }, 2024-08-06T20:25:13.9493133Z { 2024-08-06T20:25:13.9493463Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9493875Z "size": 150, 2024-08-06T20:25:13.9494377Z "digest": "sha256:a20886f4e5372f586ef3e2728653d1969c6d898d8eae0667de17908891ac278f" 2024-08-06T20:25:13.9494854Z }, 2024-08-06T20:25:13.9495039Z { 2024-08-06T20:25:13.9495376Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9495801Z "size": 643, 2024-08-06T20:25:13.9496207Z "digest": "sha256:af1fb4fa4eed1c385edc3660525fa1e46533ec917477c141d430c07c735ff8da" 2024-08-06T20:25:13.9496690Z }, 2024-08-06T20:25:13.9496877Z { 2024-08-06T20:25:13.9497213Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9497635Z "size": 32, 2024-08-06T20:25:13.9498041Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:25:13.9498521Z }, 2024-08-06T20:25:13.9498728Z { 2024-08-06T20:25:13.9499046Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9499475Z "size": 162, 2024-08-06T20:25:13.9499896Z "digest": "sha256:de13eba39cb5160281708ab7fda3d1c4e26a9edb7793e8fddc3355c37a2c2bb6" 2024-08-06T20:25:13.9500374Z }, 2024-08-06T20:25:13.9500572Z { 2024-08-06T20:25:13.9500908Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9501322Z "size": 109, 2024-08-06T20:25:13.9501740Z "digest": "sha256:ac851722c4ad33e4e6ad7a9580ff1f61e91dc5ec40f5e5ff5629cd32c8a6dff9" 2024-08-06T20:25:13.9502360Z }, 2024-08-06T20:25:13.9502569Z { 2024-08-06T20:25:13.9502911Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9503343Z "size": 102, 2024-08-06T20:25:13.9503741Z "digest": "sha256:225d8e1484f0c72016ef99e76b8309a3472d6fb528b34633eb25c00394e350d9" 2024-08-06T20:25:13.9504214Z }, 2024-08-06T20:25:13.9504413Z { 2024-08-06T20:25:13.9504734Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9505161Z "size": 421, 2024-08-06T20:25:13.9505569Z "digest": "sha256:0b92275d203788d5d990e589228173aa2fe9709737328fd78f2aa02bf159a285" 2024-08-06T20:25:13.9506025Z }, 2024-08-06T20:25:13.9506235Z { 2024-08-06T20:25:13.9506574Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9506989Z "size": 32, 2024-08-06T20:25:13.9507408Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:25:13.9507896Z }, 2024-08-06T20:25:13.9508088Z { 2024-08-06T20:25:13.9508423Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9508849Z "size": 111, 2024-08-06T20:25:13.9509262Z "digest": "sha256:86d521d46977bce4fd8bbc492e3f0310b7f26ffc5e36fafe97225a7ca8fa9ffa" 2024-08-06T20:25:13.9509752Z }, 2024-08-06T20:25:13.9509943Z { 2024-08-06T20:25:13.9510274Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9510702Z "size": 471, 2024-08-06T20:25:13.9511107Z "digest": "sha256:3914906be0cb7fd68abac1a779b275af42993ccb4ed1bd1015f467e170d0b1f4" 2024-08-06T20:25:13.9511587Z }, 2024-08-06T20:25:13.9511784Z { 2024-08-06T20:25:13.9512190Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9512616Z "size": 32, 2024-08-06T20:25:13.9513034Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:25:13.9513505Z }, 2024-08-06T20:25:13.9513705Z { 2024-08-06T20:25:13.9514035Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9514453Z "size": 112, 2024-08-06T20:25:13.9514866Z "digest": "sha256:193407c5262c69087b764527f63b3d7e47e029f4f0ddf8f6a3fe4baaa7f29c57" 2024-08-06T20:25:13.9515338Z }, 2024-08-06T20:25:13.9515527Z { 2024-08-06T20:25:13.9515866Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9516290Z "size": 562, 2024-08-06T20:25:13.9516695Z "digest": "sha256:adf9d181cfbca5b0544f23797995d4ea6500cc4a635db9f7e18954c9044e9f4a" 2024-08-06T20:25:13.9517177Z }, 2024-08-06T20:25:13.9517378Z { 2024-08-06T20:25:13.9517698Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9518125Z "size": 43163921, 2024-08-06T20:25:13.9518555Z "digest": "sha256:736c4af6f5277292080e42bfc2c7edaf8412ef1b795cf6fcbf2e95beffbb7ed2" 2024-08-06T20:25:13.9519022Z }, 2024-08-06T20:25:13.9519220Z { 2024-08-06T20:25:13.9519549Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9519968Z "size": 106, 2024-08-06T20:25:13.9520378Z "digest": "sha256:801521a42094555983e4a2674ddbb195243f6fe0d15ad1aeaab4aa88d81d22c4" 2024-08-06T20:25:13.9520849Z }, 2024-08-06T20:25:13.9521042Z { 2024-08-06T20:25:13.9521380Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9521806Z "size": 294, 2024-08-06T20:25:13.9522273Z "digest": "sha256:ed7b11c2ed84119d1bd1ebf8b7317c20e0f79da24ca754a02cb6775c00a17ae1" 2024-08-06T20:25:13.9522763Z }, 2024-08-06T20:25:13.9522966Z { 2024-08-06T20:25:13.9523291Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9523711Z "size": 32, 2024-08-06T20:25:13.9524134Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:25:13.9524602Z }, 2024-08-06T20:25:13.9524803Z { 2024-08-06T20:25:13.9525122Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9525549Z "size": 106, 2024-08-06T20:25:13.9526057Z "digest": "sha256:5c6f2f3fe7c213d4b989fac53b07197b9f9ab99fe04360200826bcb2ca7ac66e" 2024-08-06T20:25:13.9526525Z }, 2024-08-06T20:25:13.9526734Z { 2024-08-06T20:25:13.9527070Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9527488Z "size": 424, 2024-08-06T20:25:13.9527910Z "digest": "sha256:3886e8f57dc887a87bb710fbda080ef4fd521f1ed2e560a895b000c05f6e9b0a" 2024-08-06T20:25:13.9528394Z }, 2024-08-06T20:25:13.9528580Z { 2024-08-06T20:25:13.9528907Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9529328Z "size": 19308828, 2024-08-06T20:25:13.9529746Z "digest": "sha256:cc822658f193feb19cb990795b3114bca9c649bd221c0b1aab684e652069c295" 2024-08-06T20:25:13.9530483Z }, 2024-08-06T20:25:13.9530691Z { 2024-08-06T20:25:13.9531011Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9531434Z "size": 108, 2024-08-06T20:25:13.9531838Z "digest": "sha256:cca044783521353ee93922f8ea943e74964e2a9815b43751866ff494b8d4d8d9" 2024-08-06T20:25:13.9532299Z }, 2024-08-06T20:25:13.9532500Z { 2024-08-06T20:25:13.9532830Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9533240Z "size": 438, 2024-08-06T20:25:13.9533658Z "digest": "sha256:0eb80a0ac0006a45aa6b4581263cc1613ddf9545bd4a1d5a544310efe898ad81" 2024-08-06T20:25:13.9534199Z }, 2024-08-06T20:25:13.9534391Z { 2024-08-06T20:25:13.9534731Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9535156Z "size": 699, 2024-08-06T20:25:13.9535565Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-06T20:25:13.9536242Z }, 2024-08-06T20:25:13.9536446Z { 2024-08-06T20:25:13.9536766Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9537190Z "size": 141, 2024-08-06T20:25:13.9537612Z "digest": "sha256:a9f11b2b3622f5530c5cfefe973e96e028323fab1ff02e7a6337eb86fadebeb5" 2024-08-06T20:25:13.9538089Z }, 2024-08-06T20:25:13.9538300Z { 2024-08-06T20:25:13.9538620Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9539048Z "size": 135, 2024-08-06T20:25:13.9539457Z "digest": "sha256:d7e94279f8b2fe549e3f23637275d9dca8c831563ce84f1d44117eb297ddc871" 2024-08-06T20:25:13.9539920Z }, 2024-08-06T20:25:13.9540123Z { 2024-08-06T20:25:13.9540456Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9540866Z "size": 32, 2024-08-06T20:25:13.9541289Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:25:13.9541774Z }, 2024-08-06T20:25:13.9541967Z { 2024-08-06T20:25:13.9542304Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9542725Z "size": 190, 2024-08-06T20:25:13.9543123Z "digest": "sha256:073f669362431eb57b23c427bcb2898ba4bb050cf4f9d37c09e270ceb847ccd6" 2024-08-06T20:25:13.9543601Z }, 2024-08-06T20:25:13.9543804Z { 2024-08-06T20:25:13.9544125Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9544556Z "size": 1210, 2024-08-06T20:25:13.9544976Z "digest": "sha256:719a1c2abcfd7e4113aa2f03adeb54350a5e381c720688ea53bc0e18401ad33d" 2024-08-06T20:25:13.9545448Z }, 2024-08-06T20:25:13.9545648Z { 2024-08-06T20:25:13.9545982Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9546393Z "size": 699, 2024-08-06T20:25:13.9546808Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-06T20:25:13.9547295Z }, 2024-08-06T20:25:13.9547542Z { 2024-08-06T20:25:13.9547980Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9548478Z "size": 138, 2024-08-06T20:25:13.9549182Z "digest": "sha256:8e1355d516c5e0527c25402c664fc648a8a2b814090e6ee0fdd60f49a8826ac1" 2024-08-06T20:25:13.9549837Z }, 2024-08-06T20:25:13.9550211Z { 2024-08-06T20:25:13.9550697Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9551233Z "size": 1842834361, 2024-08-06T20:25:13.9551713Z "digest": "sha256:4d5ca8eafd7982e3b256257abe6dfce2dc314e52d34defe80c8298de227a5f19" 2024-08-06T20:25:13.9552352Z }, 2024-08-06T20:25:13.9552659Z { 2024-08-06T20:25:13.9553026Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9553603Z "size": 153, 2024-08-06T20:25:13.9554121Z "digest": "sha256:73831405501a21694a29eaa05a64c05dbc545d9c5ba676a5b5f204fa2300291a" 2024-08-06T20:25:13.9554623Z }, 2024-08-06T20:25:13.9554972Z { 2024-08-06T20:25:13.9555465Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9555925Z "size": 846, 2024-08-06T20:25:13.9556487Z "digest": "sha256:0b48a1547ae65f413b4be264aad75996efd1ab0fac4107687531843d1ee54e5c" 2024-08-06T20:25:13.9557067Z }, 2024-08-06T20:25:13.9582517Z { 2024-08-06T20:25:13.9583060Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9583520Z "size": 699, 2024-08-06T20:25:13.9583937Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-06T20:25:13.9584412Z }, 2024-08-06T20:25:13.9584597Z { 2024-08-06T20:25:13.9584907Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9585314Z "size": 139, 2024-08-06T20:25:13.9585716Z "digest": "sha256:794615be0ac3201bd2845c4cd2e2f9e9651f1fba0142b0c74dccfda79fee82b1" 2024-08-06T20:25:13.9586187Z }, 2024-08-06T20:25:13.9586375Z { 2024-08-06T20:25:13.9586691Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9587278Z "size": 32, 2024-08-06T20:25:13.9587689Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:25:13.9588159Z }, 2024-08-06T20:25:13.9588332Z { 2024-08-06T20:25:13.9588643Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9589058Z "size": 157, 2024-08-06T20:25:13.9589443Z "digest": "sha256:ad492c23051a703625b3e779faf07bba8b9bea291956c20e744bba89898af6be" 2024-08-06T20:25:13.9589899Z }, 2024-08-06T20:25:13.9590080Z { 2024-08-06T20:25:13.9590391Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9590810Z "size": 905, 2024-08-06T20:25:13.9591219Z "digest": "sha256:fc956bf4e5b49119cab7cc7bda867c61bed76c76c1e113dfa7c7b184251f8173" 2024-08-06T20:25:13.9591686Z }, 2024-08-06T20:25:13.9591880Z { 2024-08-06T20:25:13.9592204Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9592609Z "size": 699, 2024-08-06T20:25:13.9593026Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-06T20:25:13.9593508Z }, 2024-08-06T20:25:13.9593694Z { 2024-08-06T20:25:13.9594022Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9594443Z "size": 134, 2024-08-06T20:25:13.9594838Z "digest": "sha256:874491333fb5b81d9cbbd71520163a6baf07e6f011dc148b1634b94e301972b8" 2024-08-06T20:25:13.9595309Z }, 2024-08-06T20:25:13.9595495Z { 2024-08-06T20:25:13.9595810Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9596234Z "size": 32, 2024-08-06T20:25:13.9596644Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:25:13.9597113Z }, 2024-08-06T20:25:13.9597306Z { 2024-08-06T20:25:13.9597622Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9598040Z "size": 156, 2024-08-06T20:25:13.9598456Z "digest": "sha256:62fd235ebb62f15f85f93abd31bdfcd2445df1b40f97d0b474a4879e05a47b0e" 2024-08-06T20:25:13.9598934Z }, 2024-08-06T20:25:13.9599120Z { 2024-08-06T20:25:13.9599447Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9599859Z "size": 1577, 2024-08-06T20:25:13.9600355Z "digest": "sha256:42646dcb0889a48f164507344b56ed0254f1c64ca68b2f974c7fa9f2569f1925" 2024-08-06T20:25:13.9600833Z }, 2024-08-06T20:25:13.9601023Z { 2024-08-06T20:25:13.9601358Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9601778Z "size": 32, 2024-08-06T20:25:13.9602184Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:25:13.9602661Z }, 2024-08-06T20:25:13.9602859Z { 2024-08-06T20:25:13.9603178Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9603602Z "size": 128, 2024-08-06T20:25:13.9604014Z "digest": "sha256:5df8c6bd8177de4d452857fef1396a5c9574f50e37b342b1a192f89ec36fb4e5" 2024-08-06T20:25:13.9604487Z }, 2024-08-06T20:25:13.9604686Z { 2024-08-06T20:25:13.9605021Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9605434Z "size": 377, 2024-08-06T20:25:13.9605850Z "digest": "sha256:96381dfde5a802fe7b27ba3ae3b5f8589f88c170c5bae6863be97a37fa86688b" 2024-08-06T20:25:13.9606339Z }, 2024-08-06T20:25:13.9606526Z { 2024-08-06T20:25:13.9606854Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9607276Z "size": 32, 2024-08-06T20:25:13.9607678Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:25:13.9608153Z }, 2024-08-06T20:25:13.9608351Z { 2024-08-06T20:25:13.9608668Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9609090Z "size": 104, 2024-08-06T20:25:13.9609498Z "digest": "sha256:d0dba839d4c041ba853621262a7e730c3699058d43cfe7d6f31ec311708acc73" 2024-08-06T20:25:13.9609960Z }, 2024-08-06T20:25:13.9610258Z { 2024-08-06T20:25:13.9610596Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9611009Z "size": 1839, 2024-08-06T20:25:13.9611427Z "digest": "sha256:c1d4cac78640fc7d364011da807a363d6b248a2c46ccf99a6e1488c4974c17e3" 2024-08-06T20:25:13.9611910Z }, 2024-08-06T20:25:13.9612096Z { 2024-08-06T20:25:13.9612431Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9612847Z "size": 7529628, 2024-08-06T20:25:13.9613279Z "digest": "sha256:67cb12cd081ccde6047c7cbbdd4d99c4499d4b12d3db421c307ef6d901ff0c81" 2024-08-06T20:25:13.9613761Z }, 2024-08-06T20:25:13.9613951Z { 2024-08-06T20:25:13.9614445Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9614873Z "size": 106, 2024-08-06T20:25:13.9615278Z "digest": "sha256:b431724b7afe431951242ade99ee881ce85e51dc44dd1f8c0c1ed1710de62f68" 2024-08-06T20:25:13.9615762Z }, 2024-08-06T20:25:13.9615966Z { 2024-08-06T20:25:13.9616290Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9616718Z "size": 161, 2024-08-06T20:25:13.9617127Z "digest": "sha256:190dccfa687dc1e698f27965d8065a183168d7c66361505f368765456ff9f4bc" 2024-08-06T20:25:13.9617580Z }, 2024-08-06T20:25:13.9617786Z { 2024-08-06T20:25:13.9618118Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9618535Z "size": 7942, 2024-08-06T20:25:13.9618962Z "digest": "sha256:b01bea4b6634f47c51ede2b4a95ceade4e61356ca29551168b493ae6d4492f44" 2024-08-06T20:25:13.9619444Z }, 2024-08-06T20:25:13.9619633Z { 2024-08-06T20:25:13.9619962Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9620390Z "size": 8065, 2024-08-06T20:25:13.9620804Z "digest": "sha256:77fc3d1eca2c044fa3497f2e035f9ca2e5ffcd8f16b810f1173ae52af23b71a7" 2024-08-06T20:25:13.9621292Z }, 2024-08-06T20:25:13.9621495Z { 2024-08-06T20:25:13.9621820Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9622247Z "size": 302, 2024-08-06T20:25:13.9622663Z "digest": "sha256:96d6dcd01b9f6c5557ba405d028442d808dd329f4f93e75f2c3ba0554e703a52" 2024-08-06T20:25:13.9623129Z }, 2024-08-06T20:25:13.9623330Z { 2024-08-06T20:25:13.9623751Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9624163Z "size": 32, 2024-08-06T20:25:13.9624578Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:25:13.9625058Z }, 2024-08-06T20:25:13.9625245Z { 2024-08-06T20:25:13.9625573Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9625986Z "size": 108, 2024-08-06T20:25:13.9626392Z "digest": "sha256:79ef14b93271d6a0315533761cb955f79fba8ee77b9a73ba823923e7dd1613f7" 2024-08-06T20:25:13.9626862Z }, 2024-08-06T20:25:13.9627047Z { 2024-08-06T20:25:13.9627375Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9627803Z "size": 54145774, 2024-08-06T20:25:13.9628217Z "digest": "sha256:45103ff2548064a8f9e94bdb31d8bc89e396f9880773cc06e0540dac48d20e10" 2024-08-06T20:25:13.9628689Z }, 2024-08-06T20:25:13.9628890Z { 2024-08-06T20:25:13.9629210Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-06T20:25:13.9629638Z "size": 32, 2024-08-06T20:25:13.9630055Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-06T20:25:13.9630786Z } 2024-08-06T20:25:13.9630987Z ] 2024-08-06T20:25:13.9631185Z } 2024-08-06T20:25:13.9631404Z + exit 0 2024-08-06T20:25:13.9699136Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-06T20:25:13.9699530Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-06T20:25:13.9699942Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-06T20:25:13.9706640Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:25:13.9707043Z env: 2024-08-06T20:25:13.9707268Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:25:13.9707979Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:13.9708721Z ##[endgroup] 2024-08-06T20:25:13.9735971Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-jammy-py3.8-gcc11-02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:13.9789195Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-06T20:25:13.9789657Z with: 2024-08-06T20:25:13.9790332Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:13.9791176Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:25:13.9791562Z env: 2024-08-06T20:25:13.9791786Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:25:13.9792066Z ##[endgroup] 2024-08-06T20:25:13.9815521Z ##[group]Run set -x 2024-08-06T20:25:13.9815830Z set -x 2024-08-06T20:25:13.9816053Z set +e 2024-08-06T20:25:13.9816286Z  2024-08-06T20:25:13.9816508Z login() { 2024-08-06T20:25:13.9816983Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-06T20:25:13.9817515Z } 2024-08-06T20:25:13.9817745Z  2024-08-06T20:25:13.9818007Z retry () { 2024-08-06T20:25:13.9818275Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-06T20:25:13.9818603Z } 2024-08-06T20:25:13.9818817Z  2024-08-06T20:25:13.9819053Z retry login "${DOCKER_REGISTRY}" 2024-08-06T20:25:13.9819367Z  2024-08-06T20:25:13.9819583Z set -e 2024-08-06T20:25:13.9819919Z # ignore output since only exit code is used for conditional 2024-08-06T20:25:13.9820423Z # only pull docker image if it's not available locally 2024-08-06T20:25:13.9821096Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-06T20:25:13.9821600Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-06T20:25:13.9821931Z fi 2024-08-06T20:25:13.9827827Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:25:13.9828220Z env: 2024-08-06T20:25:13.9828446Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:25:13.9829306Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:13.9830400Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:25:13.9830808Z ##[endgroup] 2024-08-06T20:25:13.9854388Z + set +e 2024-08-06T20:25:13.9855082Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:25:13.9855550Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:25:13.9857815Z + aws ecr get-login-password --region us-east-1 2024-08-06T20:25:13.9859300Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-06T20:25:14.5674332Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-06T20:25:14.5674996Z Configure a credential helper to remove this warning. See 2024-08-06T20:25:14.5675756Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-06T20:25:14.5676186Z 2024-08-06T20:25:14.5676303Z Login Succeeded 2024-08-06T20:25:14.5686660Z + set -e 2024-08-06T20:25:14.5687777Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:14.5821699Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:14.5822920Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:25:14.8867351Z 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9: Pulling from pytorch/pytorch-linux-jammy-py3.8-gcc11 2024-08-06T20:25:14.8876631Z 9b857f539cb1: Pulling fs layer 2024-08-06T20:25:14.8877207Z ceceb076946e: Pulling fs layer 2024-08-06T20:25:14.8877638Z 042461771f00: Pulling fs layer 2024-08-06T20:25:14.8878096Z bac001ac2fee: Pulling fs layer 2024-08-06T20:25:14.8878521Z 4a587220513d: Pulling fs layer 2024-08-06T20:25:14.8879261Z af8081143f00: Pulling fs layer 2024-08-06T20:25:14.8879747Z e7a423dcab01: Pulling fs layer 2024-08-06T20:25:14.8880246Z ad26542e0d01: Pulling fs layer 2024-08-06T20:25:14.8880713Z 9e1d4f8784a0: Pulling fs layer 2024-08-06T20:25:14.8881188Z b1f815d16274: Pulling fs layer 2024-08-06T20:25:14.8881623Z 91f9dddffbee: Pulling fs layer 2024-08-06T20:25:14.8882070Z 03fb78f207e3: Pulling fs layer 2024-08-06T20:25:14.8882402Z 8385f3c14a81: Pulling fs layer 2024-08-06T20:25:14.8882694Z 4f4fb700ef54: Pulling fs layer 2024-08-06T20:25:14.8883112Z 4a587220513d: Waiting 2024-08-06T20:25:14.8883368Z 202c71391ade: Pulling fs layer 2024-08-06T20:25:14.8883673Z 240ee294d88b: Pulling fs layer 2024-08-06T20:25:14.8884056Z af8081143f00: Waiting 2024-08-06T20:25:14.8884303Z e7a423dcab01: Waiting 2024-08-06T20:25:14.8884612Z 7a823d3d7147: Pulling fs layer 2024-08-06T20:25:14.8884903Z 9e1d4f8784a0: Waiting 2024-08-06T20:25:14.8885154Z ad26542e0d01: Waiting 2024-08-06T20:25:14.8885546Z a1b9e8856530: Pulling fs layer 2024-08-06T20:25:14.8885864Z b1f815d16274: Waiting 2024-08-06T20:25:14.8886126Z 91f9dddffbee: Waiting 2024-08-06T20:25:14.8886442Z 9c5b585a8116: Pulling fs layer 2024-08-06T20:25:14.8886746Z 03fb78f207e3: Waiting 2024-08-06T20:25:14.8887160Z 9cd1bcd9f117: Pulling fs layer 2024-08-06T20:25:14.8887812Z 8385f3c14a81: Waiting 2024-08-06T20:25:14.8888119Z 2b9731d295d6: Pulling fs layer 2024-08-06T20:25:14.8888418Z 202c71391ade: Waiting 2024-08-06T20:25:14.8888687Z 7a823d3d7147: Waiting 2024-08-06T20:25:14.8888943Z 8d223c48f542: Pulling fs layer 2024-08-06T20:25:14.8889254Z 4f4fb700ef54: Waiting 2024-08-06T20:25:14.8889502Z 240ee294d88b: Waiting 2024-08-06T20:25:14.8889768Z 49a0e418a175: Pulling fs layer 2024-08-06T20:25:14.8890094Z bac001ac2fee: Waiting 2024-08-06T20:25:14.8890345Z a1b9e8856530: Waiting 2024-08-06T20:25:14.8890651Z 2b9731d295d6: Waiting 2024-08-06T20:25:14.8890891Z 51ed474f6c32: Pulling fs layer 2024-08-06T20:25:14.8891168Z 9cd1bcd9f117: Waiting 2024-08-06T20:25:14.8891663Z 1c01ecb575c6: Pulling fs layer 2024-08-06T20:25:14.8891943Z 9c5b585a8116: Waiting 2024-08-06T20:25:14.8892254Z 8d223c48f542: Waiting 2024-08-06T20:25:14.8892495Z 51ed474f6c32: Waiting 2024-08-06T20:25:14.8892734Z a2630e5fe166: Pulling fs layer 2024-08-06T20:25:14.8893009Z 49a0e418a175: Waiting 2024-08-06T20:25:14.8893359Z 1c01ecb575c6: Waiting 2024-08-06T20:25:14.8893618Z 554a42b2a2e6: Pulling fs layer 2024-08-06T20:25:14.8894033Z c9a20a1fda16: Pulling fs layer 2024-08-06T20:25:14.8894439Z 554a42b2a2e6: Waiting 2024-08-06T20:25:14.8894759Z 2c2e35b979f5: Pulling fs layer 2024-08-06T20:25:14.8895092Z a2630e5fe166: Waiting 2024-08-06T20:25:14.8895337Z 0e3c02dff025: Pulling fs layer 2024-08-06T20:25:14.8895621Z c9a20a1fda16: Waiting 2024-08-06T20:25:14.8895875Z 2c2cb9893903: Pulling fs layer 2024-08-06T20:25:14.8896192Z ec0a42784a49: Pulling fs layer 2024-08-06T20:25:14.8896465Z a20886f4e537: Pulling fs layer 2024-08-06T20:25:14.8896779Z 2c2e35b979f5: Waiting 2024-08-06T20:25:14.8897022Z 0e3c02dff025: Waiting 2024-08-06T20:25:14.8897278Z 2c2cb9893903: Waiting 2024-08-06T20:25:14.8897585Z af1fb4fa4eed: Pulling fs layer 2024-08-06T20:25:14.8897868Z ec0a42784a49: Waiting 2024-08-06T20:25:14.8898155Z af1fb4fa4eed: Waiting 2024-08-06T20:25:14.8898417Z de13eba39cb5: Pulling fs layer 2024-08-06T20:25:14.8898708Z ac851722c4ad: Pulling fs layer 2024-08-06T20:25:14.8899043Z 225d8e1484f0: Pulling fs layer 2024-08-06T20:25:14.8899330Z 0b92275d2037: Pulling fs layer 2024-08-06T20:25:14.8899644Z 86d521d46977: Pulling fs layer 2024-08-06T20:25:14.8900029Z 3914906be0cb: Pulling fs layer 2024-08-06T20:25:14.8900318Z de13eba39cb5: Waiting 2024-08-06T20:25:14.8900603Z 0b92275d2037: Waiting 2024-08-06T20:25:14.8900848Z ac851722c4ad: Waiting 2024-08-06T20:25:14.8901110Z 193407c5262c: Pulling fs layer 2024-08-06T20:25:14.8901497Z adf9d181cfbc: Pulling fs layer 2024-08-06T20:25:14.8901781Z 225d8e1484f0: Waiting 2024-08-06T20:25:14.8902080Z 3914906be0cb: Waiting 2024-08-06T20:25:14.8902324Z 736c4af6f527: Pulling fs layer 2024-08-06T20:25:14.8902685Z 801521a42094: Pulling fs layer 2024-08-06T20:25:14.8903275Z ed7b11c2ed84: Pulling fs layer 2024-08-06T20:25:14.8903608Z 736c4af6f527: Waiting 2024-08-06T20:25:14.8903863Z adf9d181cfbc: Waiting 2024-08-06T20:25:14.8904128Z 5c6f2f3fe7c2: Pulling fs layer 2024-08-06T20:25:14.8904464Z 3886e8f57dc8: Pulling fs layer 2024-08-06T20:25:14.8904742Z 5c6f2f3fe7c2: Waiting 2024-08-06T20:25:14.8905329Z 801521a42094: Waiting 2024-08-06T20:25:14.8905578Z cc822658f193: Pulling fs layer 2024-08-06T20:25:14.8905923Z cca044783521: Pulling fs layer 2024-08-06T20:25:14.8906202Z 3886e8f57dc8: Waiting 2024-08-06T20:25:14.8906448Z 0eb80a0ac000: Pulling fs layer 2024-08-06T20:25:14.8906794Z a9f11b2b3622: Pulling fs layer 2024-08-06T20:25:14.8907072Z d7e94279f8b2: Pulling fs layer 2024-08-06T20:25:14.8907399Z cc822658f193: Waiting 2024-08-06T20:25:14.8907652Z 073f66936243: Pulling fs layer 2024-08-06T20:25:14.8907911Z cca044783521: Waiting 2024-08-06T20:25:14.8908209Z a9f11b2b3622: Waiting 2024-08-06T20:25:14.8908463Z 0eb80a0ac000: Waiting 2024-08-06T20:25:14.8908714Z 719a1c2abcfd: Pulling fs layer 2024-08-06T20:25:14.8909045Z 073f66936243: Waiting 2024-08-06T20:25:14.8909288Z 719a1c2abcfd: Waiting 2024-08-06T20:25:14.8909547Z 8e1355d516c5: Pulling fs layer 2024-08-06T20:25:14.8909876Z 4d5ca8eafd79: Pulling fs layer 2024-08-06T20:25:14.8910151Z 8e1355d516c5: Waiting 2024-08-06T20:25:14.8910530Z 73831405501a: Pulling fs layer 2024-08-06T20:25:14.8910865Z 0b48a1547ae6: Pulling fs layer 2024-08-06T20:25:14.8911202Z 794615be0ac3: Pulling fs layer 2024-08-06T20:25:14.8911464Z 0b48a1547ae6: Waiting 2024-08-06T20:25:14.8911721Z ad492c23051a: Pulling fs layer 2024-08-06T20:25:14.8912064Z fc956bf4e5b4: Pulling fs layer 2024-08-06T20:25:14.8912336Z 874491333fb5: Pulling fs layer 2024-08-06T20:25:14.8912623Z ad492c23051a: Waiting 2024-08-06T20:25:14.8912903Z 62fd235ebb62: Pulling fs layer 2024-08-06T20:25:14.8913181Z fc956bf4e5b4: Waiting 2024-08-06T20:25:14.8913493Z 42646dcb0889: Pulling fs layer 2024-08-06T20:25:14.8913915Z 5df8c6bd8177: Pulling fs layer 2024-08-06T20:25:14.8914239Z 73831405501a: Waiting 2024-08-06T20:25:14.8914503Z 96381dfde5a8: Pulling fs layer 2024-08-06T20:25:14.8914788Z 62fd235ebb62: Waiting 2024-08-06T20:25:14.8915093Z d0dba839d4c0: Pulling fs layer 2024-08-06T20:25:14.8915374Z 42646dcb0889: Waiting 2024-08-06T20:25:14.8915629Z c1d4cac78640: Pulling fs layer 2024-08-06T20:25:14.8915966Z 67cb12cd081c: Pulling fs layer 2024-08-06T20:25:14.8916258Z b431724b7afe: Pulling fs layer 2024-08-06T20:25:14.8916599Z 190dccfa687d: Pulling fs layer 2024-08-06T20:25:14.8916879Z b01bea4b6634: Pulling fs layer 2024-08-06T20:25:14.8917174Z 77fc3d1eca2c: Pulling fs layer 2024-08-06T20:25:14.8917518Z 190dccfa687d: Waiting 2024-08-06T20:25:14.8917793Z 67cb12cd081c: Waiting 2024-08-06T20:25:14.8918221Z 96d6dcd01b9f: Pulling fs layer 2024-08-06T20:25:14.8918664Z b01bea4b6634: Waiting 2024-08-06T20:25:14.8919060Z 79ef14b93271: Pulling fs layer 2024-08-06T20:25:14.8919540Z 96d6dcd01b9f: Waiting 2024-08-06T20:25:14.8919983Z 45103ff25480: Pulling fs layer 2024-08-06T20:25:14.8920470Z 77fc3d1eca2c: Waiting 2024-08-06T20:25:14.8920893Z 79ef14b93271: Waiting 2024-08-06T20:25:14.8921307Z 874491333fb5: Waiting 2024-08-06T20:25:14.9734720Z ceceb076946e: Verifying Checksum 2024-08-06T20:25:14.9735439Z ceceb076946e: Download complete 2024-08-06T20:25:15.0441008Z bac001ac2fee: Verifying Checksum 2024-08-06T20:25:15.0441486Z bac001ac2fee: Download complete 2024-08-06T20:25:15.1229085Z 4a587220513d: Verifying Checksum 2024-08-06T20:25:15.1229543Z 4a587220513d: Download complete 2024-08-06T20:25:15.2037949Z af8081143f00: Verifying Checksum 2024-08-06T20:25:15.2038316Z af8081143f00: Download complete 2024-08-06T20:25:15.2495548Z 9b857f539cb1: Download complete 2024-08-06T20:25:15.2911355Z e7a423dcab01: Verifying Checksum 2024-08-06T20:25:15.2912002Z e7a423dcab01: Download complete 2024-08-06T20:25:15.3241973Z ad26542e0d01: Download complete 2024-08-06T20:25:15.4091715Z b1f815d16274: Download complete 2024-08-06T20:25:15.4868417Z 91f9dddffbee: Download complete 2024-08-06T20:25:15.5639643Z 03fb78f207e3: Verifying Checksum 2024-08-06T20:25:15.5640030Z 03fb78f207e3: Download complete 2024-08-06T20:25:16.2079049Z 9b857f539cb1: Pull complete 2024-08-06T20:25:16.2270570Z ceceb076946e: Pull complete 2024-08-06T20:25:16.4534221Z 9e1d4f8784a0: Verifying Checksum 2024-08-06T20:25:16.4534862Z 9e1d4f8784a0: Download complete 2024-08-06T20:25:16.4608191Z 4f4fb700ef54: Verifying Checksum 2024-08-06T20:25:16.4608562Z 4f4fb700ef54: Download complete 2024-08-06T20:25:16.5429127Z 202c71391ade: Verifying Checksum 2024-08-06T20:25:16.5429718Z 202c71391ade: Download complete 2024-08-06T20:25:16.6420502Z 240ee294d88b: Download complete 2024-08-06T20:25:16.8178298Z a1b9e8856530: Verifying Checksum 2024-08-06T20:25:16.8178827Z a1b9e8856530: Download complete 2024-08-06T20:25:16.9211175Z 9c5b585a8116: Download complete 2024-08-06T20:25:16.9904199Z 9cd1bcd9f117: Download complete 2024-08-06T20:25:17.0721141Z 2b9731d295d6: Verifying Checksum 2024-08-06T20:25:17.0721678Z 2b9731d295d6: Download complete 2024-08-06T20:25:17.1484105Z 8d223c48f542: Verifying Checksum 2024-08-06T20:25:17.1484546Z 8d223c48f542: Download complete 2024-08-06T20:25:17.2345339Z 49a0e418a175: Verifying Checksum 2024-08-06T20:25:17.2345876Z 49a0e418a175: Download complete 2024-08-06T20:25:18.0708179Z 042461771f00: Verifying Checksum 2024-08-06T20:25:18.0711250Z 042461771f00: Download complete 2024-08-06T20:25:18.1327936Z 1c01ecb575c6: Verifying Checksum 2024-08-06T20:25:18.1328340Z 1c01ecb575c6: Download complete 2024-08-06T20:25:18.2057018Z a2630e5fe166: Verifying Checksum 2024-08-06T20:25:18.2057386Z a2630e5fe166: Download complete 2024-08-06T20:25:18.2778711Z 554a42b2a2e6: Download complete 2024-08-06T20:25:18.3081734Z 51ed474f6c32: Verifying Checksum 2024-08-06T20:25:18.3082713Z 51ed474f6c32: Download complete 2024-08-06T20:25:18.3504585Z c9a20a1fda16: Verifying Checksum 2024-08-06T20:25:18.3505147Z c9a20a1fda16: Download complete 2024-08-06T20:25:18.3830403Z 2c2e35b979f5: Download complete 2024-08-06T20:25:18.4603304Z 2c2cb9893903: Verifying Checksum 2024-08-06T20:25:18.4603905Z 2c2cb9893903: Download complete 2024-08-06T20:25:18.5349054Z ec0a42784a49: Download complete 2024-08-06T20:25:18.6016474Z a20886f4e537: Verifying Checksum 2024-08-06T20:25:18.6016998Z a20886f4e537: Download complete 2024-08-06T20:25:18.6808456Z af1fb4fa4eed: Download complete 2024-08-06T20:25:18.7451176Z de13eba39cb5: Download complete 2024-08-06T20:25:18.8273108Z ac851722c4ad: Verifying Checksum 2024-08-06T20:25:18.8273664Z ac851722c4ad: Download complete 2024-08-06T20:25:18.8846889Z 225d8e1484f0: Verifying Checksum 2024-08-06T20:25:18.8847449Z 225d8e1484f0: Download complete 2024-08-06T20:25:18.9552467Z 0b92275d2037: Verifying Checksum 2024-08-06T20:25:18.9553056Z 0b92275d2037: Download complete 2024-08-06T20:25:19.0292406Z 86d521d46977: Verifying Checksum 2024-08-06T20:25:19.0292940Z 86d521d46977: Download complete 2024-08-06T20:25:19.1092867Z 3914906be0cb: Verifying Checksum 2024-08-06T20:25:19.1093489Z 3914906be0cb: Download complete 2024-08-06T20:25:19.1884519Z 193407c5262c: Verifying Checksum 2024-08-06T20:25:19.1885087Z 193407c5262c: Download complete 2024-08-06T20:25:19.2824184Z adf9d181cfbc: Download complete 2024-08-06T20:25:19.7626705Z 736c4af6f527: Verifying Checksum 2024-08-06T20:25:19.7627299Z 736c4af6f527: Download complete 2024-08-06T20:25:19.8263318Z 801521a42094: Verifying Checksum 2024-08-06T20:25:19.8263957Z 801521a42094: Download complete 2024-08-06T20:25:19.9270886Z ed7b11c2ed84: Verifying Checksum 2024-08-06T20:25:19.9271496Z ed7b11c2ed84: Download complete 2024-08-06T20:25:20.0426425Z 5c6f2f3fe7c2: Download complete 2024-08-06T20:25:20.1155648Z 3886e8f57dc8: Verifying Checksum 2024-08-06T20:25:20.1156258Z 3886e8f57dc8: Download complete 2024-08-06T20:25:20.3684927Z cc822658f193: Verifying Checksum 2024-08-06T20:25:20.3685551Z cc822658f193: Download complete 2024-08-06T20:25:20.4511956Z cca044783521: Download complete 2024-08-06T20:25:20.5546800Z 0eb80a0ac000: Download complete 2024-08-06T20:25:20.6267912Z a9f11b2b3622: Download complete 2024-08-06T20:25:20.7227040Z d7e94279f8b2: Verifying Checksum 2024-08-06T20:25:20.7227485Z d7e94279f8b2: Download complete 2024-08-06T20:25:20.8049900Z 073f66936243: Verifying Checksum 2024-08-06T20:25:20.8050402Z 073f66936243: Download complete 2024-08-06T20:25:20.8842577Z 719a1c2abcfd: Verifying Checksum 2024-08-06T20:25:20.8843206Z 719a1c2abcfd: Download complete 2024-08-06T20:25:20.9568600Z 8e1355d516c5: Verifying Checksum 2024-08-06T20:25:20.9568980Z 8e1355d516c5: Download complete 2024-08-06T20:25:24.7001064Z 0e3c02dff025: Verifying Checksum 2024-08-06T20:25:24.7001559Z 0e3c02dff025: Download complete 2024-08-06T20:25:24.7654779Z 73831405501a: Verifying Checksum 2024-08-06T20:25:24.7655292Z 73831405501a: Download complete 2024-08-06T20:25:24.8591734Z 0b48a1547ae6: Verifying Checksum 2024-08-06T20:25:24.8592340Z 0b48a1547ae6: Download complete 2024-08-06T20:25:24.9298046Z 794615be0ac3: Verifying Checksum 2024-08-06T20:25:24.9298650Z 794615be0ac3: Download complete 2024-08-06T20:25:25.0010952Z ad492c23051a: Verifying Checksum 2024-08-06T20:25:25.0011564Z ad492c23051a: Download complete 2024-08-06T20:25:25.0738285Z fc956bf4e5b4: Download complete 2024-08-06T20:25:25.1448074Z 874491333fb5: Verifying Checksum 2024-08-06T20:25:25.1448665Z 874491333fb5: Download complete 2024-08-06T20:25:25.2079875Z 62fd235ebb62: Download complete 2024-08-06T20:25:25.2805025Z 42646dcb0889: Verifying Checksum 2024-08-06T20:25:25.2805538Z 42646dcb0889: Download complete 2024-08-06T20:25:25.3558601Z 5df8c6bd8177: Verifying Checksum 2024-08-06T20:25:25.3559076Z 5df8c6bd8177: Download complete 2024-08-06T20:25:25.4235985Z 96381dfde5a8: Verifying Checksum 2024-08-06T20:25:25.4236476Z 96381dfde5a8: Download complete 2024-08-06T20:25:25.4903478Z d0dba839d4c0: Verifying Checksum 2024-08-06T20:25:25.4903988Z d0dba839d4c0: Download complete 2024-08-06T20:25:25.5654879Z c1d4cac78640: Download complete 2024-08-06T20:25:25.6929443Z 67cb12cd081c: Verifying Checksum 2024-08-06T20:25:25.6929989Z 67cb12cd081c: Download complete 2024-08-06T20:25:25.7671507Z b431724b7afe: Verifying Checksum 2024-08-06T20:25:25.7672165Z b431724b7afe: Download complete 2024-08-06T20:25:25.8446624Z 190dccfa687d: Verifying Checksum 2024-08-06T20:25:25.8447197Z 190dccfa687d: Download complete 2024-08-06T20:25:25.9349243Z b01bea4b6634: Verifying Checksum 2024-08-06T20:25:25.9349816Z b01bea4b6634: Download complete 2024-08-06T20:25:26.0023116Z 77fc3d1eca2c: Download complete 2024-08-06T20:25:26.0630306Z 96d6dcd01b9f: Verifying Checksum 2024-08-06T20:25:26.0630889Z 96d6dcd01b9f: Download complete 2024-08-06T20:25:26.1262711Z 79ef14b93271: Verifying Checksum 2024-08-06T20:25:26.1263198Z 79ef14b93271: Download complete 2024-08-06T20:25:27.4537519Z 45103ff25480: Verifying Checksum 2024-08-06T20:25:27.4537964Z 45103ff25480: Download complete 2024-08-06T20:25:27.9754979Z 042461771f00: Pull complete 2024-08-06T20:25:28.1218029Z bac001ac2fee: Pull complete 2024-08-06T20:25:28.2113110Z 4a587220513d: Pull complete 2024-08-06T20:25:28.3662071Z af8081143f00: Pull complete 2024-08-06T20:25:28.4445201Z e7a423dcab01: Pull complete 2024-08-06T20:25:28.5198026Z ad26542e0d01: Pull complete 2024-08-06T20:25:50.2298543Z 9e1d4f8784a0: Pull complete 2024-08-06T20:25:50.3204639Z 4d5ca8eafd79: Verifying Checksum 2024-08-06T20:25:50.3205877Z 4d5ca8eafd79: Download complete 2024-08-06T20:25:50.4069203Z b1f815d16274: Pull complete 2024-08-06T20:25:50.5829893Z 91f9dddffbee: Pull complete 2024-08-06T20:25:50.7161398Z 03fb78f207e3: Pull complete 2024-08-06T20:25:54.0150952Z 8385f3c14a81: Verifying Checksum 2024-08-06T20:25:54.0151430Z 8385f3c14a81: Download complete 2024-08-06T20:26:41.8062051Z 8385f3c14a81: Pull complete 2024-08-06T20:26:41.9995298Z 4f4fb700ef54: Pull complete 2024-08-06T20:26:42.1903386Z 202c71391ade: Pull complete 2024-08-06T20:26:42.2845219Z 240ee294d88b: Pull complete 2024-08-06T20:26:42.3276747Z 7a823d3d7147: Pull complete 2024-08-06T20:26:42.5928918Z a1b9e8856530: Pull complete 2024-08-06T20:26:42.6675291Z 9c5b585a8116: Pull complete 2024-08-06T20:26:42.7110181Z 9cd1bcd9f117: Pull complete 2024-08-06T20:26:42.8088767Z 2b9731d295d6: Pull complete 2024-08-06T20:26:43.1732948Z 8d223c48f542: Pull complete 2024-08-06T20:26:43.3891159Z 49a0e418a175: Pull complete 2024-08-06T20:26:47.2008349Z 51ed474f6c32: Pull complete 2024-08-06T20:26:47.3271599Z 1c01ecb575c6: Pull complete 2024-08-06T20:26:47.4303188Z a2630e5fe166: Pull complete 2024-08-06T20:26:47.6362682Z 554a42b2a2e6: Pull complete 2024-08-06T20:26:47.7492695Z c9a20a1fda16: Pull complete 2024-08-06T20:26:47.8757036Z 2c2e35b979f5: Pull complete 2024-08-06T20:26:56.5949772Z 0e3c02dff025: Pull complete 2024-08-06T20:26:56.6209000Z 2c2cb9893903: Pull complete 2024-08-06T20:26:56.6441964Z ec0a42784a49: Pull complete 2024-08-06T20:26:56.6633521Z a20886f4e537: Pull complete 2024-08-06T20:26:56.6821916Z af1fb4fa4eed: Pull complete 2024-08-06T20:26:56.7314909Z de13eba39cb5: Pull complete 2024-08-06T20:26:56.7503832Z ac851722c4ad: Pull complete 2024-08-06T20:26:56.7738042Z 225d8e1484f0: Pull complete 2024-08-06T20:26:56.7992457Z 0b92275d2037: Pull complete 2024-08-06T20:26:56.8435130Z 86d521d46977: Pull complete 2024-08-06T20:26:56.8641324Z 3914906be0cb: Pull complete 2024-08-06T20:26:56.9196780Z 193407c5262c: Pull complete 2024-08-06T20:26:56.9395349Z adf9d181cfbc: Pull complete 2024-08-06T20:26:58.5213296Z 736c4af6f527: Pull complete 2024-08-06T20:26:58.5495098Z 801521a42094: Pull complete 2024-08-06T20:26:58.5676817Z ed7b11c2ed84: Pull complete 2024-08-06T20:26:58.6151061Z 5c6f2f3fe7c2: Pull complete 2024-08-06T20:26:58.6359468Z 3886e8f57dc8: Pull complete 2024-08-06T20:26:58.8563913Z cc822658f193: Pull complete 2024-08-06T20:26:58.8821529Z cca044783521: Pull complete 2024-08-06T20:26:58.9058306Z 0eb80a0ac000: Pull complete 2024-08-06T20:26:58.9472353Z a9f11b2b3622: Pull complete 2024-08-06T20:26:58.9658744Z d7e94279f8b2: Pull complete 2024-08-06T20:26:59.0203179Z 073f66936243: Pull complete 2024-08-06T20:26:59.0850454Z 719a1c2abcfd: Pull complete 2024-08-06T20:26:59.1810740Z 8e1355d516c5: Pull complete 2024-08-06T20:27:30.8766716Z 4d5ca8eafd79: Pull complete 2024-08-06T20:27:31.0302788Z 73831405501a: Pull complete 2024-08-06T20:27:31.1074521Z 0b48a1547ae6: Pull complete 2024-08-06T20:27:31.4057544Z 794615be0ac3: Pull complete 2024-08-06T20:27:31.5746669Z ad492c23051a: Pull complete 2024-08-06T20:27:31.6590923Z fc956bf4e5b4: Pull complete 2024-08-06T20:27:31.8480462Z 874491333fb5: Pull complete 2024-08-06T20:27:32.0901818Z 62fd235ebb62: Pull complete 2024-08-06T20:27:32.1437625Z 42646dcb0889: Pull complete 2024-08-06T20:27:32.5023531Z 5df8c6bd8177: Pull complete 2024-08-06T20:27:32.6005170Z 96381dfde5a8: Pull complete 2024-08-06T20:27:32.7732740Z d0dba839d4c0: Pull complete 2024-08-06T20:27:32.9387732Z c1d4cac78640: Pull complete 2024-08-06T20:27:33.4421551Z 67cb12cd081c: Pull complete 2024-08-06T20:27:33.5657709Z b431724b7afe: Pull complete 2024-08-06T20:27:33.6113653Z 190dccfa687d: Pull complete 2024-08-06T20:27:33.7145813Z b01bea4b6634: Pull complete 2024-08-06T20:27:33.7882851Z 77fc3d1eca2c: Pull complete 2024-08-06T20:27:33.8777642Z 96d6dcd01b9f: Pull complete 2024-08-06T20:27:34.2663746Z 79ef14b93271: Pull complete 2024-08-06T20:27:36.6569620Z 45103ff25480: Pull complete 2024-08-06T20:27:36.9855768Z Digest: sha256:c75221497aa22766802555fccb76014708414eebd47578b213b43b3f8f4cddb9 2024-08-06T20:27:37.0144911Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:27:37.0432992Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:27:37.0479042Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-06T20:27:37.0479771Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-06T20:27:37.0488100Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:27:37.0488499Z env: 2024-08-06T20:27:37.0488717Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:37.0489015Z ##[endgroup] 2024-08-06T20:27:37.0887469Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-06T20:27:37.0888083Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-06T20:27:37.0888605Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-06T20:27:37.0889091Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-06T20:27:37.0895240Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:27:37.0895634Z env: 2024-08-06T20:27:37.0895862Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:37.0896123Z ##[endgroup] 2024-08-06T20:27:37.9635167Z Defaulting to user installation because normal site-packages is not writeable 2024-08-06T20:27:38.0100312Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.9/site-packages (5.9.1) 2024-08-06T20:27:38.0105873Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.9/site-packages (11.525.84) 2024-08-06T20:27:38.1426982Z Prepare all required actions 2024-08-06T20:27:38.1427792Z Getting action download info 2024-08-06T20:27:38.2373457Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-06T20:27:39.1911956Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-06T20:27:39.3562095Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-06T20:27:39.3562479Z with: 2024-08-06T20:27:39.3562720Z name: linux-jammy-py3.8-gcc11 2024-08-06T20:27:39.3563008Z s3-bucket: gha-artifacts 2024-08-06T20:27:39.3563275Z env: 2024-08-06T20:27:39.3563497Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:39.3563751Z ##[endgroup] 2024-08-06T20:27:39.3704018Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-06T20:27:39.3704379Z with: 2024-08-06T20:27:39.3704605Z name: linux-jammy-py3.8-gcc11 2024-08-06T20:27:39.3704907Z s3-bucket: gha-artifacts 2024-08-06T20:27:39.3705441Z region: us-east-1 2024-08-06T20:27:39.3705683Z env: 2024-08-06T20:27:39.3705889Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:39.3706162Z ##[endgroup] 2024-08-06T20:27:40.0826738Z (node:343974) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-06T20:27:40.0827266Z 2024-08-06T20:27:40.0827462Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-06T20:27:40.0828020Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-06T20:27:40.0828773Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-06T20:27:40.1601630Z Found 1 objects with prefix pytorch/pytorch/10273124344/linux-jammy-py3.8-gcc11/ 2024-08-06T20:27:40.1602373Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-06T20:27:43.8844579Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-06T20:27:43.8853132Z Artifact download has finished successfully 2024-08-06T20:27:43.9024130Z ##[group]Run unzip -o artifacts.zip 2024-08-06T20:27:43.9024508Z unzip -o artifacts.zip 2024-08-06T20:27:43.9030913Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:27:43.9031315Z env: 2024-08-06T20:27:43.9031529Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:43.9031805Z ##[endgroup] 2024-08-06T20:27:43.9177746Z Archive: artifacts.zip 2024-08-06T20:27:43.9178892Z creating: dist/ 2024-08-06T20:27:44.8527809Z inflating: dist/torch-2.5.0a0+gitb9d86fa-cp38-cp38-linux_x86_64.whl 2024-08-06T20:27:44.8528662Z creating: build/custom_test_artifacts/ 2024-08-06T20:27:44.8529337Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-06T20:27:44.8530469Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-06T20:27:44.8531466Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-06T20:27:44.8532901Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-06T20:27:44.8533630Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-06T20:27:44.8534971Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-06T20:27:44.8535755Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-06T20:27:44.8536464Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-06T20:27:44.8537292Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-06T20:27:44.8539047Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-06T20:27:44.8539970Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-06T20:27:44.8540751Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-06T20:27:44.8543398Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-06T20:27:44.8544684Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-06T20:27:44.8546524Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-06T20:27:44.8547386Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-06T20:27:44.8549187Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-06T20:27:44.8550375Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-06T20:27:44.8551167Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-06T20:27:44.8551910Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-06T20:27:44.8552890Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-06T20:27:44.8554313Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-06T20:27:44.8555817Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-06T20:27:44.8556742Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-06T20:27:44.8557995Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-06T20:27:44.8559233Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-06T20:27:44.8560020Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-06T20:27:44.8561264Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-06T20:27:44.8562081Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-06T20:27:44.8562863Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-06T20:27:44.8577777Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-06T20:27:44.8706290Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-06T20:27:44.8707751Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-06T20:27:44.8709197Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-06T20:27:44.8710914Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-06T20:27:44.8712164Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-06T20:27:44.8712994Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-06T20:27:44.8713817Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-06T20:27:44.8714655Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-06T20:27:44.8715494Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-06T20:27:44.8716323Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-06T20:27:44.8717131Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-06T20:27:44.8730730Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-06T20:27:44.8806056Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-06T20:27:44.8807608Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-06T20:27:44.8808782Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-06T20:27:44.8809897Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-06T20:27:44.8810701Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-06T20:27:44.8811860Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-06T20:27:44.8849972Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-06T20:27:44.8850731Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-06T20:27:44.8851332Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-06T20:27:44.8923634Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-06T20:27:44.8982749Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-06T20:27:44.8983679Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-06T20:27:44.8984172Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-06T20:27:44.8984757Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-06T20:27:44.8986521Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-06T20:27:44.8987794Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-06T20:27:44.8988492Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-06T20:27:44.8989212Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-06T20:27:44.8990266Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-06T20:27:44.8991573Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-06T20:27:44.8992505Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-06T20:27:44.8993231Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-06T20:27:44.8994313Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-06T20:27:44.8995538Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-06T20:27:44.8996748Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-06T20:27:44.8997851Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-06T20:27:44.8999180Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-06T20:27:44.9000798Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-06T20:27:44.9002417Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-06T20:27:44.9003708Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-06T20:27:44.9004978Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-06T20:27:44.9005771Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-06T20:27:44.9006533Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-06T20:27:44.9007416Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-06T20:27:44.9008261Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-06T20:27:44.9009271Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-06T20:27:44.9010098Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-06T20:27:44.9010925Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-06T20:27:44.9011739Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-06T20:27:44.9012546Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-06T20:27:44.9013358Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-06T20:27:44.9023732Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-06T20:27:44.9083069Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-06T20:27:44.9085026Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-06T20:27:44.9086489Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-06T20:27:44.9087433Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-06T20:27:44.9088672Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-06T20:27:44.9089448Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-06T20:27:44.9090073Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-06T20:27:44.9090619Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-06T20:27:44.9091192Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-06T20:27:44.9136207Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-06T20:27:44.9137223Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-06T20:27:44.9137752Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-06T20:27:44.9138400Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-06T20:27:44.9139997Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-06T20:27:44.9141352Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-06T20:27:44.9142090Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-06T20:27:44.9142960Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-06T20:27:44.9143995Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-06T20:27:44.9145500Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-06T20:27:44.9146371Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-06T20:27:44.9147534Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-06T20:27:44.9148515Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-06T20:27:44.9149731Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-06T20:27:44.9151108Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-06T20:27:44.9152380Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-06T20:27:44.9154061Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-06T20:27:44.9156076Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-06T20:27:44.9157816Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-06T20:27:44.9159011Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-06T20:27:44.9159725Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-06T20:27:44.9160467Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-06T20:27:44.9161284Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-06T20:27:44.9162625Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-06T20:27:44.9163538Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-06T20:27:44.9164502Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-06T20:27:44.9165425Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-06T20:27:44.9166349Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-06T20:27:44.9167235Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-06T20:27:44.9168112Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-06T20:27:44.9168983Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-06T20:27:44.9169919Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-06T20:27:44.9277924Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-06T20:27:44.9279634Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-06T20:27:44.9281246Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-06T20:27:44.9282764Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-06T20:27:44.9283719Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-06T20:27:44.9284605Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-06T20:27:44.9285534Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-06T20:27:44.9286469Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-06T20:27:44.9287400Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-06T20:27:44.9288334Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-06T20:27:44.9289251Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-06T20:27:44.9301146Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-06T20:27:44.9351729Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-06T20:27:44.9353600Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-06T20:27:44.9355000Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-06T20:27:44.9356443Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-06T20:27:44.9357684Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-06T20:27:44.9358389Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-06T20:27:44.9359063Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-06T20:27:44.9359657Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-06T20:27:44.9360278Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-06T20:27:44.9456001Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-06T20:27:44.9496088Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-06T20:27:44.9496840Z creating: build/lib/ 2024-08-06T20:27:44.9497370Z inflating: build/lib/libclog.a 2024-08-06T20:27:44.9506290Z inflating: build/lib/libpthreadpool.a 2024-08-06T20:27:44.9569347Z inflating: build/lib/libgtest.a 2024-08-06T20:27:44.9577297Z inflating: build/lib/libcpuinfo.a 2024-08-06T20:27:44.9584802Z inflating: build/lib/libcpuinfo_internals.a 2024-08-06T20:27:44.9591752Z inflating: build/lib/libittnotify.a 2024-08-06T20:27:44.9670890Z inflating: build/lib/libprotobuf-lite.a 2024-08-06T20:27:44.9740284Z inflating: build/lib/libbenchmark.a 2024-08-06T20:27:44.9766618Z inflating: build/lib/libtensorpipe_uv.a 2024-08-06T20:27:44.9827261Z inflating: build/lib/libasmjit.a 2024-08-06T20:27:45.0248854Z inflating: build/lib/libprotobuf.a 2024-08-06T20:27:45.0356510Z inflating: build/lib/libgloo.a 2024-08-06T20:27:45.0376227Z inflating: build/lib/libfmt.a 2024-08-06T20:27:45.0377028Z inflating: build/lib/libfoxi_loader.a 2024-08-06T20:27:45.0378434Z inflating: build/lib/libtorch_global_deps.so 2024-08-06T20:27:45.0468863Z inflating: build/lib/libc10.so 2024-08-06T20:27:45.0471054Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-06T20:27:45.0489017Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-06T20:27:45.0961163Z inflating: build/lib/libprotoc.a 2024-08-06T20:27:45.0977178Z inflating: build/lib/libgmock.a 2024-08-06T20:27:45.0977943Z inflating: build/lib/libgtest_main.a 2024-08-06T20:27:45.0978611Z inflating: build/lib/libbenchmark_main.a 2024-08-06T20:27:45.0995569Z inflating: build/lib/libnnpack.a 2024-08-06T20:27:46.0104168Z inflating: build/lib/libdnnl.a 2024-08-06T20:27:46.0627014Z inflating: build/lib/libtensorpipe.a 2024-08-06T20:27:46.1611804Z inflating: build/lib/libfbgemm.a 2024-08-06T20:27:46.1612473Z inflating: build/lib/libgmock_main.a 2024-08-06T20:27:46.1886614Z inflating: build/lib/libkineto.a 2024-08-06T20:27:46.1925088Z inflating: build/lib/libonnx_proto.a 2024-08-06T20:27:46.2099219Z inflating: build/lib/libXNNPACK.a 2024-08-06T20:27:46.2753988Z inflating: build/lib/libonnx.a 2024-08-06T20:27:48.4906729Z inflating: build/lib/libtorch_cpu.so 2024-08-06T20:27:48.4907470Z inflating: build/lib/libtorch.so 2024-08-06T20:27:48.4912076Z inflating: build/lib/libunbox_lib.a 2024-08-06T20:27:48.4915306Z inflating: build/lib/libshm.so 2024-08-06T20:27:48.4934669Z inflating: build/lib/libjitbackend_test.so 2024-08-06T20:27:48.5002016Z inflating: build/lib/libtorchbind_test.so 2024-08-06T20:27:48.5025852Z inflating: build/lib/libaoti_custom_ops.so 2024-08-06T20:27:48.5049914Z inflating: build/lib/libbackend_with_compiler.so 2024-08-06T20:27:48.6852500Z inflating: build/lib/libtorch_python.so 2024-08-06T20:27:48.6885584Z inflating: build/lib/libnnapi_backend.so 2024-08-06T20:27:48.6886313Z creating: build/bin/ 2024-08-06T20:27:48.6887195Z creating: build/bin/CMakeFiles/ 2024-08-06T20:27:48.6888099Z inflating: build/bin/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-06T20:27:48.6889372Z extracting: build/bin/CMakeFiles/progress.marks 2024-08-06T20:27:48.6889918Z inflating: build/bin/Makefile 2024-08-06T20:27:48.6890368Z inflating: build/bin/cmake_install.cmake 2024-08-06T20:27:48.6890810Z inflating: build/bin/CTestTestfile.cmake 2024-08-06T20:27:48.6939626Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-06T20:27:48.6991314Z inflating: build/bin/c10_DeviceGuard_test 2024-08-06T20:27:48.7042179Z inflating: build/bin/c10_Device_test 2024-08-06T20:27:48.7101813Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-06T20:27:48.7156213Z inflating: build/bin/c10_Scalar_test 2024-08-06T20:27:48.7204952Z inflating: build/bin/c10_StreamGuard_test 2024-08-06T20:27:48.7256704Z inflating: build/bin/c10_SymInt_test 2024-08-06T20:27:48.7312709Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-06T20:27:48.7366831Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-06T20:27:48.7422683Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-06T20:27:48.7476565Z inflating: build/bin/c10_Bitset_test 2024-08-06T20:27:48.7547233Z inflating: build/bin/c10_cow_test 2024-08-06T20:27:48.7595939Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-06T20:27:48.7646392Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-06T20:27:48.7696812Z inflating: build/bin/c10_Half_test 2024-08-06T20:27:48.7752884Z inflating: build/bin/c10_LeftRight_test 2024-08-06T20:27:48.7808271Z inflating: build/bin/c10_Metaprogramming_test 2024-08-06T20:27:48.7858133Z inflating: build/bin/c10_Synchronized_test 2024-08-06T20:27:48.7913343Z inflating: build/bin/c10_ThreadLocal_test 2024-08-06T20:27:48.7965289Z inflating: build/bin/c10_TypeIndex_test 2024-08-06T20:27:48.8016495Z inflating: build/bin/c10_TypeList_test 2024-08-06T20:27:48.8065528Z inflating: build/bin/c10_TypeTraits_test 2024-08-06T20:27:48.8117550Z inflating: build/bin/c10_accumulate_test 2024-08-06T20:27:48.8168520Z inflating: build/bin/c10_bit_cast_test 2024-08-06T20:27:48.8223639Z inflating: build/bin/c10_bfloat16_test 2024-08-06T20:27:48.8279691Z inflating: build/bin/c10_complex_math_test 2024-08-06T20:27:48.8335032Z inflating: build/bin/c10_complex_test 2024-08-06T20:27:48.8387529Z inflating: build/bin/c10_exception_test 2024-08-06T20:27:48.8438168Z inflating: build/bin/c10_flags_test 2024-08-06T20:27:48.8488484Z inflating: build/bin/c10_generic_math_test 2024-08-06T20:27:48.8647554Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-06T20:27:48.8698363Z inflating: build/bin/c10_irange_test 2024-08-06T20:27:48.8752095Z inflating: build/bin/c10_lazy_test 2024-08-06T20:27:48.8809402Z inflating: build/bin/c10_logging_test 2024-08-06T20:27:48.8884100Z inflating: build/bin/c10_optional_test 2024-08-06T20:27:48.8946164Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-06T20:27:48.8999629Z inflating: build/bin/c10_registry_test 2024-08-06T20:27:48.9147288Z inflating: build/bin/c10_small_vector_test 2024-08-06T20:27:48.9199528Z inflating: build/bin/c10_string_util_test 2024-08-06T20:27:48.9251589Z inflating: build/bin/c10_ssize_test 2024-08-06T20:27:48.9302107Z inflating: build/bin/c10_tempfile_test 2024-08-06T20:27:48.9361099Z inflating: build/bin/c10_string_view_test 2024-08-06T20:27:48.9417417Z inflating: build/bin/c10_typeid_test 2024-08-06T20:27:48.9464588Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-06T20:27:48.9904524Z inflating: build/bin/protoc-3.13.0.0 2024-08-06T20:27:49.0343830Z inflating: build/bin/protoc 2024-08-06T20:27:49.0665273Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-06T20:27:49.1003616Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-06T20:27:49.1355299Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-06T20:27:49.1407515Z inflating: build/bin/BackoffTest 2024-08-06T20:27:49.1461066Z inflating: build/bin/FileStoreTest 2024-08-06T20:27:49.1517076Z inflating: build/bin/TCPStoreTest 2024-08-06T20:27:49.1570623Z inflating: build/bin/HashStoreTest 2024-08-06T20:27:49.1572814Z inflating: build/bin/example_allreduce 2024-08-06T20:27:49.1638187Z inflating: build/bin/ProcessGroupGlooTest 2024-08-06T20:27:49.1692358Z inflating: build/bin/test_dist_autograd 2024-08-06T20:27:49.1694731Z inflating: build/bin/parallel_benchmark 2024-08-06T20:27:49.1762199Z inflating: build/bin/test_cpp_rpc 2024-08-06T20:27:49.1827744Z inflating: build/bin/test_mobile_nnc 2024-08-06T20:27:49.1836692Z inflating: build/bin/aot_model_compiler_test 2024-08-06T20:27:49.2166458Z inflating: build/bin/test_lazy 2024-08-06T20:27:49.2223225Z inflating: build/bin/static_runtime_bench 2024-08-06T20:27:49.2296446Z inflating: build/bin/Dict_test 2024-08-06T20:27:49.2349276Z inflating: build/bin/Dimname_test 2024-08-06T20:27:49.2413655Z inflating: build/bin/MaybeOwned_test 2024-08-06T20:27:49.2656312Z inflating: build/bin/static_runtime_test 2024-08-06T20:27:49.2715272Z inflating: build/bin/apply_utils_test 2024-08-06T20:27:49.2771826Z inflating: build/bin/NamedTensor_test 2024-08-06T20:27:49.2830284Z inflating: build/bin/atest 2024-08-06T20:27:49.2893377Z inflating: build/bin/basic 2024-08-06T20:27:49.2948656Z inflating: build/bin/broadcast_test 2024-08-06T20:27:49.2999440Z inflating: build/bin/cpu_allocator_test 2024-08-06T20:27:49.3058089Z inflating: build/bin/cpu_generator_test 2024-08-06T20:27:49.3108472Z inflating: build/bin/dlconvertor_test 2024-08-06T20:27:49.3161810Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-06T20:27:49.3212274Z inflating: build/bin/dispatch_key_set_test 2024-08-06T20:27:49.3303336Z inflating: build/bin/cpu_rng_test 2024-08-06T20:27:49.3361167Z inflating: build/bin/extension_backend_test 2024-08-06T20:27:49.3415726Z inflating: build/bin/half_test 2024-08-06T20:27:49.3465718Z inflating: build/bin/lazy_tensor_test 2024-08-06T20:27:49.3559148Z inflating: build/bin/ivalue_test 2024-08-06T20:27:49.3612786Z inflating: build/bin/math_kernel_test 2024-08-06T20:27:49.3666465Z inflating: build/bin/memory_format_test 2024-08-06T20:27:49.3719383Z inflating: build/bin/memory_overlapping_test 2024-08-06T20:27:49.3772644Z inflating: build/bin/mobile_memory_cleanup 2024-08-06T20:27:49.3823477Z inflating: build/bin/operator_name_test 2024-08-06T20:27:49.3879404Z inflating: build/bin/native_test 2024-08-06T20:27:49.3930610Z inflating: build/bin/operators_test 2024-08-06T20:27:49.3982579Z inflating: build/bin/packedtensoraccessor_test 2024-08-06T20:27:49.5072616Z inflating: build/bin/test_api 2024-08-06T20:27:49.5138627Z inflating: build/bin/pow_test 2024-08-06T20:27:49.5195695Z inflating: build/bin/quantized_test 2024-08-06T20:27:49.5247229Z inflating: build/bin/reportMemoryUsage_test 2024-08-06T20:27:49.5297256Z inflating: build/bin/reduce_ops_test 2024-08-06T20:27:49.5354125Z inflating: build/bin/scalar_tensor_test 2024-08-06T20:27:49.5412193Z inflating: build/bin/scalar_test 2024-08-06T20:27:49.5463893Z inflating: build/bin/StorageUtils_test 2024-08-06T20:27:49.5516408Z inflating: build/bin/stride_properties_test 2024-08-06T20:27:49.5593870Z inflating: build/bin/tensor_iterator_test 2024-08-06T20:27:49.5648847Z inflating: build/bin/type_ptr_test 2024-08-06T20:27:49.5651382Z inflating: build/bin/thread_init_test 2024-08-06T20:27:49.5705143Z inflating: build/bin/test_parallel 2024-08-06T20:27:49.5764733Z inflating: build/bin/type_test 2024-08-06T20:27:49.5816859Z inflating: build/bin/undefined_tensor_test 2024-08-06T20:27:49.5818222Z inflating: build/bin/verify_api_visibility 2024-08-06T20:27:49.5886971Z inflating: build/bin/legacy_vmap_test 2024-08-06T20:27:49.5938934Z inflating: build/bin/weakref_test 2024-08-06T20:27:49.5990100Z inflating: build/bin/wrapdim_test 2024-08-06T20:27:49.6041974Z inflating: build/bin/xla_tensor_test 2024-08-06T20:27:49.6146553Z inflating: build/bin/List_test 2024-08-06T20:27:49.6205411Z inflating: build/bin/IListRef_test 2024-08-06T20:27:49.6271348Z inflating: build/bin/KernelFunction_test 2024-08-06T20:27:49.6390130Z inflating: build/bin/kernel_function_legacy_test 2024-08-06T20:27:49.6484173Z inflating: build/bin/kernel_function_test 2024-08-06T20:27:49.6607258Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-06T20:27:49.6706529Z inflating: build/bin/kernel_lambda_test 2024-08-06T20:27:49.6758089Z inflating: build/bin/CppSignature_test 2024-08-06T20:27:49.6818357Z inflating: build/bin/kernel_stackbased_test 2024-08-06T20:27:49.6912151Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-06T20:27:49.6960846Z inflating: build/bin/op_allowlist_test 2024-08-06T20:27:49.7015619Z inflating: build/bin/backend_fallback_test 2024-08-06T20:27:49.7079455Z inflating: build/bin/inline_container_test 2024-08-06T20:27:49.7379935Z inflating: build/bin/op_registration_test 2024-08-06T20:27:49.7433122Z inflating: build/bin/test_edge_op_registration 2024-08-06T20:27:49.7437153Z inflating: build/bin/torch_shm_manager 2024-08-06T20:27:49.7449851Z inflating: build/bin/tutorial_tensorexpr 2024-08-06T20:27:49.8212661Z inflating: build/bin/test_tensorexpr 2024-08-06T20:27:49.8771388Z inflating: build/bin/test_jit 2024-08-06T20:27:49.8771817Z creating: .additional_ci_files/ 2024-08-06T20:27:49.8826434Z inflating: .additional_ci_files/test-times.json 2024-08-06T20:27:49.9048386Z inflating: .additional_ci_files/test-class-times.json 2024-08-06T20:27:49.9081900Z ##[group]Run rm artifacts.zip 2024-08-06T20:27:49.9082236Z rm artifacts.zip 2024-08-06T20:27:49.9087878Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:27:49.9088277Z env: 2024-08-06T20:27:49.9088504Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:49.9088761Z ##[endgroup] 2024-08-06T20:27:49.9473916Z ##[group]Run df -H 2024-08-06T20:27:49.9474227Z df -H 2024-08-06T20:27:49.9479888Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:27:49.9480278Z env: 2024-08-06T20:27:49.9480494Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:49.9480764Z ##[endgroup] 2024-08-06T20:27:49.9605967Z Filesystem Size Used Avail Use% Mounted on 2024-08-06T20:27:49.9606385Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-06T20:27:49.9606729Z tmpfs 8.2G 4.1M 8.2G 1% /dev/shm 2024-08-06T20:27:49.9607073Z tmpfs 3.3G 488k 3.3G 1% /run 2024-08-06T20:27:49.9607410Z /dev/nvme0n1p1 161G 27G 135G 17% / 2024-08-06T20:27:49.9607733Z tmpfs 8.2G 21k 8.2G 1% /tmp 2024-08-06T20:27:49.9608243Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-06T20:27:49.9690799Z Prepare all required actions 2024-08-06T20:27:49.9691454Z Getting action download info 2024-08-06T20:27:50.1812357Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-06T20:27:50.1812730Z with: 2024-08-06T20:27:50.1812948Z env: 2024-08-06T20:27:50.1813170Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:50.1813442Z ##[endgroup] 2024-08-06T20:27:50.2006095Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-06T20:27:50.2006455Z with: 2024-08-06T20:27:50.2006674Z name: td_results 2024-08-06T20:27:50.2006914Z s3-bucket: gha-artifacts 2024-08-06T20:27:50.2007191Z region: us-east-1 2024-08-06T20:27:50.2007429Z env: 2024-08-06T20:27:50.2007640Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:50.2007909Z ##[endgroup] 2024-08-06T20:27:50.6828655Z (node:343993) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-06T20:27:50.6829183Z 2024-08-06T20:27:50.6829463Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-06T20:27:50.6830004Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-06T20:27:50.6830928Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-06T20:27:50.7615573Z Found 1 objects with prefix pytorch/pytorch/10273124344/td_results/ 2024-08-06T20:27:50.7616287Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-06T20:27:50.8119917Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-06T20:27:50.8126448Z Artifact download has finished successfully 2024-08-06T20:27:50.8427450Z ##[group]Run mkdir -p .additional_ci_files 2024-08-06T20:27:50.8427848Z mkdir -p .additional_ci_files 2024-08-06T20:27:50.8428255Z mv td_results.json .additional_ci_files/td_results.json 2024-08-06T20:27:50.8434854Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:27:50.8435252Z env: 2024-08-06T20:27:50.8435490Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:50.8435768Z ##[endgroup] 2024-08-06T20:27:50.8865684Z ##[group]Run .github/scripts/parse_ref.py 2024-08-06T20:27:50.8866323Z .github/scripts/parse_ref.py 2024-08-06T20:27:50.8872734Z shell: /usr/bin/bash -e {0} 2024-08-06T20:27:50.8873279Z env: 2024-08-06T20:27:50.8873627Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:50.8874001Z ##[endgroup] 2024-08-06T20:27:50.9302583Z Prepare all required actions 2024-08-06T20:27:50.9425337Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-06T20:27:50.9425724Z with: 2024-08-06T20:27:50.9426148Z github-token: *** 2024-08-06T20:27:50.9426398Z env: 2024-08-06T20:27:50.9426626Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:50.9426885Z ##[endgroup] 2024-08-06T20:27:50.9543566Z ##[group]Run set -eux 2024-08-06T20:27:50.9543852Z set -eux 2024-08-06T20:27:50.9544293Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-06T20:27:50.9550128Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:27:50.9550521Z env: 2024-08-06T20:27:50.9550742Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:50.9551300Z GITHUB_TOKEN: *** 2024-08-06T20:27:50.9551550Z ##[endgroup] 2024-08-06T20:27:50.9575001Z + python3 .github/scripts/get_workflow_job_id.py 10273124344 i-064ea195702d7f670 2024-08-06T20:27:53.4305377Z setting job-id=28427732833 2024-08-06T20:27:53.4306027Z setting job-name=linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-06T20:27:53.4507734Z Prepare all required actions 2024-08-06T20:27:53.4508254Z Getting action download info 2024-08-06T20:27:53.5792375Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-06T20:27:53.5792733Z with: 2024-08-06T20:27:53.5793182Z github-token: *** 2024-08-06T20:27:53.5795853Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]} 2024-08-06T20:27:53.5798691Z job-name: linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-06T20:27:53.5799177Z env: 2024-08-06T20:27:53.5799393Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:53.5799669Z ##[endgroup] 2024-08-06T20:27:53.6023335Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-06T20:27:53.6023763Z with: 2024-08-06T20:27:53.6023983Z shell: bash 2024-08-06T20:27:53.6024221Z timeout_minutes: 10 2024-08-06T20:27:53.6024469Z max_attempts: 5 2024-08-06T20:27:53.6024715Z retry_wait_seconds: 30 2024-08-06T20:27:53.6025523Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-06T20:27:53.6026382Z polling_interval_seconds: 1 2024-08-06T20:27:53.6026676Z warning_on_retry: true 2024-08-06T20:27:53.6026959Z continue_on_error: false 2024-08-06T20:27:53.6027215Z env: 2024-08-06T20:27:53.6027437Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:53.6028071Z GITHUB_TOKEN: *** 2024-08-06T20:27:53.6028322Z ##[endgroup] 2024-08-06T20:27:53.6892188Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-06T20:27:53.9322704Z Defaulting to user installation because normal site-packages is not writeable 2024-08-06T20:27:53.9490261Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.9/site-packages (2.27.1) 2024-08-06T20:27:53.9494043Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.9/site-packages (6.0.1) 2024-08-06T20:27:53.9616653Z Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2.0.12) 2024-08-06T20:27:53.9620424Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-08-06T20:27:53.9624308Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2024.7.4) 2024-08-06T20:27:53.9634889Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-08-06T20:27:54.6559258Z Command completed after 1 attempt(s). 2024-08-06T20:27:54.6613203Z ##[group]Run set -x 2024-08-06T20:27:54.6613489Z set -x 2024-08-06T20:27:54.6613712Z  2024-08-06T20:27:54.6614226Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-06T20:27:54.6614725Z # in runner workspace 2024-08-06T20:27:54.6615107Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-06T20:27:54.6621085Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:27:54.6621476Z env: 2024-08-06T20:27:54.6621699Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:54.6622176Z ##[endgroup] 2024-08-06T20:27:54.6646562Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-06T20:27:54.6900608Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-06T20:27:54.6901050Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-06T20:27:54.6901391Z echo "Job name: ${JOB_NAME}" 2024-08-06T20:27:54.6901697Z  2024-08-06T20:27:54.6902090Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-06T20:27:54.6902571Z # in runner workspace 2024-08-06T20:27:54.6903005Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-06T20:27:54.6903496Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-06T20:27:54.6903859Z  --job-name "${JOB_NAME}" \ 2024-08-06T20:27:54.6906552Z  --test-matrix "{"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]}" \ 2024-08-06T20:27:54.6909251Z  --selected-test-configs "" \ 2024-08-06T20:27:54.6909589Z  --pr-number "${PR_NUMBER}" \ 2024-08-06T20:27:54.6909914Z  --tag "${TAG}" \ 2024-08-06T20:27:54.6910222Z  --event-name "${EVENT_NAME}" \ 2024-08-06T20:27:54.6910558Z  --schedule "${SCHEDULE}" \ 2024-08-06T20:27:54.6910884Z  --branch "${HEAD_BRANCH}" 2024-08-06T20:27:54.6916632Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:27:54.6917013Z env: 2024-08-06T20:27:54.6917241Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:54.6917738Z GITHUB_TOKEN: *** 2024-08-06T20:27:54.6918146Z JOB_NAME: linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-06T20:27:54.6918633Z PR_NUMBER: 132710 2024-08-06T20:27:54.6918876Z TAG: 2024-08-06T20:27:54.6919088Z EVENT_NAME: pull_request 2024-08-06T20:27:54.6919360Z SCHEDULE: 2024-08-06T20:27:54.6919586Z HEAD_BRANCH: 2024-08-06T20:27:54.6919808Z ##[endgroup] 2024-08-06T20:27:54.6942607Z Workflow: pull 2024-08-06T20:27:54.6943224Z Job name: linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-06T20:27:54.9212019Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-06T20:27:55.1927666Z ##[group]Run echo "Filtered matrix:" 2024-08-06T20:27:55.1928035Z echo "Filtered matrix:" 2024-08-06T20:27:55.1931193Z echo "{"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]}" 2024-08-06T20:27:55.1933882Z  2024-08-06T20:27:55.1934333Z echo 2024-08-06T20:27:55.1934638Z echo "Is the current job unstable? False" 2024-08-06T20:27:55.1934994Z  2024-08-06T20:27:55.1935199Z echo 2024-08-06T20:27:55.1935474Z echo "Is keep-going label set? False" 2024-08-06T20:27:55.1935797Z  2024-08-06T20:27:55.1936012Z echo 2024-08-06T20:27:55.1936255Z echo "Renabled issues? " 2024-08-06T20:27:55.1941909Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:27:55.1942305Z env: 2024-08-06T20:27:55.1942531Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:55.1942791Z ##[endgroup] 2024-08-06T20:27:55.1966203Z Filtered matrix: 2024-08-06T20:27:55.1970329Z {include: [{config: default, shard: 1, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: default, shard: 2, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: default, shard: 3, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: default, shard: 4, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: docs_test, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: jit_legacy, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: backwards_compat, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: distributed, shard: 1, num_shards: 2, runner: amz2023.linux.2xlarge}, {config: distributed, shard: 2, num_shards: 2, runner: amz2023.linux.2xlarge}]} 2024-08-06T20:27:55.1972863Z 2024-08-06T20:27:55.1973006Z Is the current job unstable? False 2024-08-06T20:27:55.1973216Z 2024-08-06T20:27:55.1973326Z Is keep-going label set? False 2024-08-06T20:27:55.1973533Z 2024-08-06T20:27:55.1973628Z Renabled issues? 2024-08-06T20:27:55.2034793Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-06T20:27:55.2035340Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-06T20:27:55.2040734Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T20:27:55.2041129Z env: 2024-08-06T20:27:55.2041344Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:55.2041621Z JOB_TIMEOUT: 240 2024-08-06T20:27:55.2041860Z ##[endgroup] 2024-08-06T20:27:55.2113696Z ##[group]Run set -x 2024-08-06T20:27:55.2114032Z set -x 2024-08-06T20:27:55.2114262Z  2024-08-06T20:27:55.2114531Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-06T20:27:55.2114937Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-06T20:27:55.2115335Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-06T20:27:55.2115705Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-06T20:27:55.2116022Z else 2024-08-06T20:27:55.2116272Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-06T20:27:55.2116593Z fi 2024-08-06T20:27:55.2116812Z  2024-08-06T20:27:55.2117283Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-06T20:27:55.2117834Z # TODO: Stop building test binaries as part of the build phase 2024-08-06T20:27:55.2118317Z # Used for GPU_FLAG since that doesn't play nice 2024-08-06T20:27:55.2118753Z # shellcheck disable=SC2086,SC2090 2024-08-06T20:27:55.2119089Z container_name=$(docker run \ 2024-08-06T20:27:55.2119410Z  ${GPU_FLAG:-} \ 2024-08-06T20:27:55.2119697Z  -e BUILD_ENVIRONMENT \ 2024-08-06T20:27:55.2119994Z  -e PR_NUMBER \ 2024-08-06T20:27:55.2120275Z  -e GITHUB_ACTIONS \ 2024-08-06T20:27:55.2120575Z  -e GITHUB_REPOSITORY \ 2024-08-06T20:27:55.2120871Z  -e GITHUB_WORKFLOW \ 2024-08-06T20:27:55.2121166Z  -e GITHUB_JOB \ 2024-08-06T20:27:55.2121446Z  -e GITHUB_RUN_ID \ 2024-08-06T20:27:55.2121722Z  -e GITHUB_RUN_NUMBER \ 2024-08-06T20:27:55.2122029Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-06T20:27:55.2122334Z  -e JOB_ID \ 2024-08-06T20:27:55.2122584Z  -e JOB_NAME \ 2024-08-06T20:27:55.2122857Z  -e BASE_SHA \ 2024-08-06T20:27:55.2123123Z  -e BRANCH \ 2024-08-06T20:27:55.2123368Z  -e SHA1 \ 2024-08-06T20:27:55.2123638Z  -e AWS_DEFAULT_REGION \ 2024-08-06T20:27:55.2123943Z  -e IN_WHEEL_TEST \ 2024-08-06T20:27:55.2124217Z  -e SHARD_NUMBER \ 2024-08-06T20:27:55.2124499Z  -e TEST_CONFIG \ 2024-08-06T20:27:55.2124785Z  -e NUM_TEST_SHARDS \ 2024-08-06T20:27:55.2125074Z  -e REENABLED_ISSUES \ 2024-08-06T20:27:55.2125387Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-06T20:27:55.2125709Z  -e VERBOSE_TEST_LOGS \ 2024-08-06T20:27:55.2126001Z  -e TEST_SHOWLOCALS \ 2024-08-06T20:27:55.2126298Z  -e NO_TEST_TIMEOUT \ 2024-08-06T20:27:55.2126583Z  -e NO_TD \ 2024-08-06T20:27:55.2126831Z  -e TD_DISTRIBUTED \ 2024-08-06T20:27:55.2127127Z  -e PR_LABELS \ 2024-08-06T20:27:55.2127431Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-06T20:27:55.2127764Z  -e SCCACHE_BUCKET \ 2024-08-06T20:27:55.2128064Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-06T20:27:55.2128376Z  -e XLA_CUDA \ 2024-08-06T20:27:55.2128665Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-06T20:27:55.2129360Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-06T20:27:55.2129752Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-06T20:27:55.2130354Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-06T20:27:55.2130730Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-06T20:27:55.2131040Z  -e DASHBOARD_TAG \ 2024-08-06T20:27:55.2131387Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-06T20:27:55.2131802Z  --security-opt seccomp=unconfined \ 2024-08-06T20:27:55.2132144Z  --cap-add=SYS_PTRACE \ 2024-08-06T20:27:55.2132458Z  --ipc=host \ 2024-08-06T20:27:55.2132740Z  --shm-size="${SHM_SIZE}" \ 2024-08-06T20:27:55.2133035Z  --tty \ 2024-08-06T20:27:55.2133281Z  --detach \ 2024-08-06T20:27:55.2133552Z  --name="${container_name}" \ 2024-08-06T20:27:55.2134029Z  --user jenkins \ 2024-08-06T20:27:55.2134540Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-06T20:27:55.2134955Z  -w /var/lib/jenkins/workspace \ 2024-08-06T20:27:55.2135276Z  "${DOCKER_IMAGE}" 2024-08-06T20:27:55.2135551Z ) 2024-08-06T20:27:55.2135857Z # Propagate download.pytorch.org IP to container 2024-08-06T20:27:55.2136525Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-06T20:27:55.2137253Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-06T20:27:55.2137932Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-06T20:27:55.2143652Z shell: /usr/bin/bash -e {0} 2024-08-06T20:27:55.2143934Z env: 2024-08-06T20:27:55.2144156Z GIT_DEFAULT_BRANCH: main 2024-08-06T20:27:55.2144455Z BUILD_ENVIRONMENT: linux-jammy-py3.8-gcc11 2024-08-06T20:27:55.2144799Z PR_NUMBER: 132710 2024-08-06T20:27:55.2145063Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-06T20:27:55.2145367Z GITHUB_WORKFLOW: pull 2024-08-06T20:27:55.2145622Z GITHUB_JOB: test 2024-08-06T20:27:55.2145869Z GITHUB_RUN_ID: 10273124344 2024-08-06T20:27:55.2146136Z GITHUB_RUN_NUMBER: 233985 2024-08-06T20:27:55.2146410Z GITHUB_RUN_ATTEMPT: 1 2024-08-06T20:27:55.2146661Z JOB_ID: 28427732833 2024-08-06T20:27:55.2147062Z JOB_NAME: linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-06T20:27:55.2147542Z BRANCH: pull/132710 2024-08-06T20:27:55.2147823Z SHA1: b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:27:55.2148193Z BASE_SHA: 1736af7cf736184c356be1bb00f59fb2feea6d7d 2024-08-06T20:27:55.2148555Z TEST_CONFIG: distributed 2024-08-06T20:27:55.2148830Z SHARD_NUMBER: 2 2024-08-06T20:27:55.2149055Z NUM_TEST_SHARDS: 2 2024-08-06T20:27:55.2149308Z REENABLED_ISSUES: 2024-08-06T20:27:55.2149567Z CONTINUE_THROUGH_ERROR: False 2024-08-06T20:27:55.2149854Z VERBOSE_TEST_LOGS: False 2024-08-06T20:27:55.2150128Z TEST_SHOWLOCALS: False 2024-08-06T20:27:55.2150395Z NO_TEST_TIMEOUT: False 2024-08-06T20:27:55.2150637Z NO_TD: False 2024-08-06T20:27:55.2150869Z TD_DISTRIBUTED: False 2024-08-06T20:27:55.2151186Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-06T20:27:55.2151550Z SCCACHE_S3_KEY_PREFIX: pull 2024-08-06T20:27:55.2151828Z SHM_SIZE: 1g 2024-08-06T20:27:55.2152492Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:27:55.2153207Z XLA_CUDA: 2024-08-06T20:27:55.2153574Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-06T20:27:55.2154042Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-06T20:27:55.2154361Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-06T20:27:55.2154672Z DASHBOARD_TAG: 2024-08-06T20:27:55.2154917Z HUGGING_FACE_HUB_TOKEN: 2024-08-06T20:27:55.2155171Z ##[endgroup] 2024-08-06T20:27:55.2178136Z + [[ distributed == \m\u\l\t\i\g\p\u ]] 2024-08-06T20:27:55.2178749Z + [[ linux-jammy-py3.8-gcc11 == *onnx* ]] 2024-08-06T20:27:55.2179086Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-06T20:27:55.2186001Z +++ nproc --ignore=2 2024-08-06T20:27:55.2205898Z ++ docker run -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=6 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10273124344 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=1g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T20:28:04.7156183Z + container_name=3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T20:28:04.7158033Z + grep download.pytorch.org /etc/hosts 2024-08-06T20:28:04.7159568Z + docker exec -i 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-06T20:28:04.8907895Z + echo DOCKER_CONTAINER_ID=3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T20:28:04.8911632Z ++ echo dist/torch-2.5.0a0+gitb9d86fa-cp38-cp38-linux_x86_64.whl 2024-08-06T20:28:04.8914261Z + docker exec -t 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 sh -c 'pip install dist/torch-2.5.0a0+gitb9d86fa-cp38-cp38-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-06T20:28:06.0607034Z Processing ./dist/torch-2.5.0a0+gitb9d86fa-cp38-cp38-linux_x86_64.whl (from torch==2.5.0a0+gitb9d86fa) 2024-08-06T20:28:06.5017157Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (3.13.1) 2024-08-06T20:28:06.5022941Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (4.12.2) 2024-08-06T20:28:06.5026593Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (2.8.8) 2024-08-06T20:28:06.5030077Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (3.1.4) 2024-08-06T20:28:06.5033898Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (2024.6.1) 2024-08-06T20:28:06.5043818Z Requirement already satisfied: sympy==1.12.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (1.12.1) 2024-08-06T20:28:06.5066556Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy==1.12.1->torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (1.3.0) 2024-08-06T20:28:06.5092813Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (3.3.0) 2024-08-06T20:28:06.5157228Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (1.21.2) 2024-08-06T20:28:06.6028644Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch==2.5.0a0+gitb9d86fa->torch==2.5.0a0+gitb9d86fa) (2.1.5) 2024-08-06T20:28:07.9120503Z Installing collected packages: torch 2024-08-06T20:28:16.0678188Z Successfully installed torch-2.5.0a0+gitb9d86fa 2024-08-06T20:28:16.1585050Z ++ dirname .ci/pytorch/test.sh 2024-08-06T20:28:16.1601250Z + source .ci/pytorch/common.sh 2024-08-06T20:28:16.1609293Z +++ dirname .ci/pytorch/common.sh 2024-08-06T20:28:16.1616190Z ++ source .ci/pytorch/common_utils.sh 2024-08-06T20:28:16.1622051Z +++ declare -f -t trap_add 2024-08-06T20:28:16.1626293Z ++ set -ex 2024-08-06T20:28:16.1626738Z ++ [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-06T20:28:16.1627121Z ++ BUILD_TEST_LIBTORCH=0 2024-08-06T20:28:16.1627439Z + [[ linux-jammy-py3.8-gcc11 != *rocm* ]] 2024-08-06T20:28:16.1629363Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-06T20:28:16.1686224Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-06T20:28:16.1686986Z + trap_add cleanup_workspace EXIT 2024-08-06T20:28:16.1687313Z + trap_add_cmd=cleanup_workspace 2024-08-06T20:28:16.1687615Z + shift 2024-08-06T20:28:16.1687851Z + for trap_add_name in "$@" 2024-08-06T20:28:16.1692763Z +++ trap -p EXIT 2024-08-06T20:28:16.1694886Z ++ eval 'extract_trap_cmd ' 2024-08-06T20:28:16.1695469Z +++ extract_trap_cmd 2024-08-06T20:28:16.1695879Z +++ printf '%s\n' '' 2024-08-06T20:28:16.1696159Z ++ printf '%s\n' cleanup_workspace 2024-08-06T20:28:16.1696926Z + trap -- ' 2024-08-06T20:28:16.1697272Z cleanup_workspace' EXIT 2024-08-06T20:28:16.1697611Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-06T20:28:16.7023523Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-06T20:28:16.7636090Z + echo 'Environment variables:' 2024-08-06T20:28:16.7636465Z Environment variables: 2024-08-06T20:28:16.7636725Z + env 2024-08-06T20:28:16.7683341Z INSTALLED_DB=yes 2024-08-06T20:28:16.7684035Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:28:16.7684822Z CONTINUE_THROUGH_ERROR=False 2024-08-06T20:28:16.7685145Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11 2024-08-06T20:28:16.7685493Z HOSTNAME=3a8f9a9b0555 2024-08-06T20:28:16.7686151Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_026c3442-fea7-4e0c-87f6-c52381528af5 2024-08-06T20:28:16.7687313Z GITHUB_ACTION=__self 2024-08-06T20:28:16.7687758Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-06T20:28:16.7688281Z GITHUB_RUN_NUMBER=233985 2024-08-06T20:28:16.7688716Z TEST_CONFIG=distributed 2024-08-06T20:28:16.7689136Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-06T20:28:16.7689754Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-06T20:28:16.7690319Z GITHUB_TRIGGERING_ACTOR=drisspg 2024-08-06T20:28:16.7690793Z GITHUB_REF_TYPE=branch 2024-08-06T20:28:16.7691202Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-06T20:28:16.7691520Z BASE_SHA=1736af7cf736184c356be1bb00f59fb2feea6d7d 2024-08-06T20:28:16.7691861Z XLA_CUDA= 2024-08-06T20:28:16.7692111Z HUGGING_FACE_HUB_TOKEN= 2024-08-06T20:28:16.7692582Z *** 2024-08-06T20:28:16.7692796Z GITHUB_REPOSITORY_ID=65600975 2024-08-06T20:28:16.7693087Z GITHUB_ACTIONS=true 2024-08-06T20:28:16.7693350Z SHA1=b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:28:16.7693727Z GITHUB_SHA=bf5bb5a1585a03379137fab341e87c02c77e76cd 2024-08-06T20:28:16.7694359Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/132710/merge 2024-08-06T20:28:16.7694871Z UCC_HOME=/usr 2024-08-06T20:28:16.7695096Z VERBOSE_TEST_LOGS=False 2024-08-06T20:28:16.7695376Z GITHUB_REF=refs/pull/132710/merge 2024-08-06T20:28:16.7695660Z SHARD_NUMBER=2 2024-08-06T20:28:16.7695909Z GITHUB_REF_PROTECTED=false 2024-08-06T20:28:16.7696207Z HOME=/var/lib/jenkins 2024-08-06T20:28:16.7696487Z GITHUB_API_URL=https://api.github.com 2024-08-06T20:28:16.7696833Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-06T20:28:16.7697138Z UCX_COMMIT= 2024-08-06T20:28:16.7697359Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-06T20:28:16.7697696Z NUM_TEST_SHARDS=2 2024-08-06T20:28:16.7698005Z UCX_HOME=/usr 2024-08-06T20:28:16.7698566Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_026c3442-fea7-4e0c-87f6-c52381528af5 2024-08-06T20:28:16.7699378Z JOB_NAME=linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-06T20:28:16.7700170Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_026c3442-fea7-4e0c-87f6-c52381528af5 2024-08-06T20:28:16.7700973Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-06T20:28:16.7701491Z GITHUB_EVENT_NAME=pull_request 2024-08-06T20:28:16.7701780Z DASHBOARD_TAG= 2024-08-06T20:28:16.7702015Z GITHUB_RUN_ID=10273124344 2024-08-06T20:28:16.7702657Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_026c3442-fea7-4e0c-87f6-c52381528af5 2024-08-06T20:28:16.7703347Z GITHUB_ACTOR=drisspg 2024-08-06T20:28:16.7703589Z PR_NUMBER=132710 2024-08-06T20:28:16.7703831Z DESIRED_CUDA= 2024-08-06T20:28:16.7704310Z GITHUB_RUN_ATTEMPT=1 2024-08-06T20:28:16.7704572Z ANACONDA_PYTHON_VERSION=3.8 2024-08-06T20:28:16.7704924Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-06T20:28:16.7705285Z TERM=xterm 2024-08-06T20:28:16.7705499Z INSTALLED_VISION=yes 2024-08-06T20:28:16.7705754Z BRANCH=pull/132710 2024-08-06T20:28:16.7706011Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-06T20:28:16.7706290Z CUDA_PATH=/usr/local/cuda 2024-08-06T20:28:16.7706820Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-06T20:28:16.7707415Z GITHUB_SERVER_URL=https://github.com 2024-08-06T20:28:16.7707712Z UCC_COMMIT= 2024-08-06T20:28:16.7708111Z REENABLED_ISSUES= 2024-08-06T20:28:16.7708351Z DOCS=yes 2024-08-06T20:28:16.7708564Z INSTALLED_ANDROID= 2024-08-06T20:28:16.7708807Z SHLVL=1 2024-08-06T20:28:16.7709009Z MAX_JOBS=6 2024-08-06T20:28:16.7709283Z GITHUB_ACTOR_ID=32754868 2024-08-06T20:28:16.7709668Z GITHUB_WORKFLOW_SHA=bf5bb5a1585a03379137fab341e87c02c77e76cd 2024-08-06T20:28:16.7710055Z GITHUB_REF_NAME=132710/merge 2024-08-06T20:28:16.7710469Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-06T20:28:16.7710908Z GITHUB_JOB=test 2024-08-06T20:28:16.7711138Z NO_TEST_TIMEOUT=False 2024-08-06T20:28:16.7711399Z TD_DISTRIBUTED=False 2024-08-06T20:28:16.7711673Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-06T20:28:16.7711968Z GITHUB_RETENTION_DAYS=90 2024-08-06T20:28:16.7712244Z OPENSSL_DIR=/opt/openssl 2024-08-06T20:28:16.7712526Z GITHUB_ACTION_REPOSITORY= 2024-08-06T20:28:16.7713368Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-06T20:28:16.7714207Z GITHUB_BASE_REF=gh/drisspg/23/base 2024-08-06T20:28:16.7714513Z INSTALLED_ACL= 2024-08-06T20:28:16.7714732Z CI=true 2024-08-06T20:28:16.7714966Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-06T20:28:16.7715263Z JOB_ID=28427732833 2024-08-06T20:28:16.7715511Z INSTALLED_PROTOBUF=yes 2024-08-06T20:28:16.7715791Z GITHUB_HEAD_REF=gh/drisspg/23/head 2024-08-06T20:28:16.7716094Z GITHUB_ACTION_REF= 2024-08-06T20:28:16.7716382Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-06T20:28:16.7716745Z TEST_SHOWLOCALS=False 2024-08-06T20:28:16.7717003Z GITHUB_WORKFLOW=pull 2024-08-06T20:28:16.7717259Z DEBIAN_FRONTEND=noninteractive 2024-08-06T20:28:16.7717893Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_026c3442-fea7-4e0c-87f6-c52381528af5 2024-08-06T20:28:16.7718542Z NO_TD=False 2024-08-06T20:28:16.7718768Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-06T20:28:16.7719052Z _=/usr/bin/env 2024-08-06T20:28:16.7719400Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-06T20:28:16.8592900Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch 2024-08-06T20:28:16.8593829Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-08-06T20:28:16.8594454Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib 2024-08-06T20:28:16.8595195Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/test 2024-08-06T20:28:16.8595693Z + BUILD_DIR=build 2024-08-06T20:28:16.8595954Z + BUILD_RENAMED_DIR=build_renamed 2024-08-06T20:28:16.8596249Z + BUILD_BIN_DIR=build/bin 2024-08-06T20:28:16.8596517Z + SHARD_NUMBER=2 2024-08-06T20:28:16.8596795Z + NUM_TEST_SHARDS=2 2024-08-06T20:28:16.8597076Z + export VALGRIND=ON 2024-08-06T20:28:16.8597330Z + VALGRIND=ON 2024-08-06T20:28:16.8597592Z + [[ linux-jammy-py3.8-gcc11 == *clang9* ]] 2024-08-06T20:28:16.8597905Z + [[ 0 == \1 ]] 2024-08-06T20:28:16.8598134Z + [[ False == \1 ]] 2024-08-06T20:28:16.8598418Z + [[ linux-jammy-py3.8-gcc11 != *bazel* ]] 2024-08-06T20:28:16.8598755Z ++ realpath build/custom_test_artifacts 2024-08-06T20:28:16.8631963Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-06T20:28:16.8633012Z + [[ -n '' ]] 2024-08-06T20:28:16.8633700Z + echo 'Environment variables' 2024-08-06T20:28:16.8634034Z Environment variables 2024-08-06T20:28:16.8634293Z + env 2024-08-06T20:28:16.8640658Z INSTALLED_DB=yes 2024-08-06T20:28:16.8641268Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T20:28:16.8641887Z CONTINUE_THROUGH_ERROR=False 2024-08-06T20:28:16.8642315Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11 2024-08-06T20:28:16.8642754Z HOSTNAME=3a8f9a9b0555 2024-08-06T20:28:16.8643545Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_026c3442-fea7-4e0c-87f6-c52381528af5 2024-08-06T20:28:16.8644415Z GITHUB_ACTION=__self 2024-08-06T20:28:16.8644763Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-06T20:28:16.8645439Z GITHUB_RUN_NUMBER=233985 2024-08-06T20:28:16.8645808Z TEST_CONFIG=distributed 2024-08-06T20:28:16.8646165Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-06T20:28:16.8646613Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-06T20:28:16.8647069Z GITHUB_TRIGGERING_ACTOR=drisspg 2024-08-06T20:28:16.8647471Z GITHUB_REF_TYPE=branch 2024-08-06T20:28:16.8647831Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-06T20:28:16.8648275Z BASE_SHA=1736af7cf736184c356be1bb00f59fb2feea6d7d 2024-08-06T20:28:16.8648717Z XLA_CUDA= 2024-08-06T20:28:16.8649008Z HUGGING_FACE_HUB_TOKEN= 2024-08-06T20:28:16.8649532Z *** 2024-08-06T20:28:16.8649812Z GITHUB_REPOSITORY_ID=65600975 2024-08-06T20:28:16.8650206Z GITHUB_ACTIONS=true 2024-08-06T20:28:16.8650704Z SHA1=b9d86fa89636e301796d4201f36d86c73f6e49bc 2024-08-06T20:28:16.8651203Z GITHUB_SHA=bf5bb5a1585a03379137fab341e87c02c77e76cd 2024-08-06T20:28:16.8651950Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/132710/merge 2024-08-06T20:28:16.8652641Z UCC_HOME=/usr 2024-08-06T20:28:16.8652955Z VERBOSE_TEST_LOGS=False 2024-08-06T20:28:16.8653314Z GITHUB_REF=refs/pull/132710/merge 2024-08-06T20:28:16.8653690Z SHARD_NUMBER=2 2024-08-06T20:28:16.8654020Z GITHUB_REF_PROTECTED=false 2024-08-06T20:28:16.8654448Z HOME=/var/lib/jenkins 2024-08-06T20:28:16.8654830Z GITHUB_API_URL=https://api.github.com 2024-08-06T20:28:16.8655305Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-06T20:28:16.8655714Z UCX_COMMIT= 2024-08-06T20:28:16.8656011Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-06T20:28:16.8656384Z NUM_TEST_SHARDS=2 2024-08-06T20:28:16.8656693Z UCX_HOME=/usr 2024-08-06T20:28:16.8657463Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_026c3442-fea7-4e0c-87f6-c52381528af5 2024-08-06T20:28:16.8658597Z JOB_NAME=linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-06T20:28:16.8659676Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_026c3442-fea7-4e0c-87f6-c52381528af5 2024-08-06T20:28:16.8660794Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-06T20:28:16.8661503Z GITHUB_EVENT_NAME=pull_request 2024-08-06T20:28:16.8661875Z DASHBOARD_TAG= 2024-08-06T20:28:16.8662169Z GITHUB_RUN_ID=10273124344 2024-08-06T20:28:16.8663059Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_026c3442-fea7-4e0c-87f6-c52381528af5 2024-08-06T20:28:16.8664000Z GITHUB_ACTOR=drisspg 2024-08-06T20:28:16.8664320Z PR_NUMBER=132710 2024-08-06T20:28:16.8664642Z DESIRED_CUDA= 2024-08-06T20:28:16.8664947Z GITHUB_RUN_ATTEMPT=1 2024-08-06T20:28:16.8665258Z VALGRIND=ON 2024-08-06T20:28:16.8665566Z ANACONDA_PYTHON_VERSION=3.8 2024-08-06T20:28:16.8666017Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-06T20:28:16.8666490Z TERM=xterm 2024-08-06T20:28:16.8666790Z INSTALLED_VISION=yes 2024-08-06T20:28:16.8667121Z BRANCH=pull/132710 2024-08-06T20:28:16.8667446Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-06T20:28:16.8667836Z CUDA_PATH=/usr/local/cuda 2024-08-06T20:28:16.8668568Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-06T20:28:16.8669359Z GITHUB_SERVER_URL=https://github.com 2024-08-06T20:28:16.8669771Z UCC_COMMIT= 2024-08-06T20:28:16.8670067Z REENABLED_ISSUES= 2024-08-06T20:28:16.8670533Z DOCS=yes 2024-08-06T20:28:16.8670833Z INSTALLED_ANDROID= 2024-08-06T20:28:16.8671141Z SHLVL=1 2024-08-06T20:28:16.8671401Z MAX_JOBS=6 2024-08-06T20:28:16.8671703Z GITHUB_ACTOR_ID=32754868 2024-08-06T20:28:16.8672168Z GITHUB_WORKFLOW_SHA=bf5bb5a1585a03379137fab341e87c02c77e76cd 2024-08-06T20:28:16.8672710Z GITHUB_REF_NAME=132710/merge 2024-08-06T20:28:16.8673269Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-06T20:28:16.8673857Z GITHUB_JOB=test 2024-08-06T20:28:16.8674167Z NO_TEST_TIMEOUT=False 2024-08-06T20:28:16.8674522Z TD_DISTRIBUTED=False 2024-08-06T20:28:16.8674870Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-06T20:28:16.8675306Z GITHUB_RETENTION_DAYS=90 2024-08-06T20:28:16.8675791Z OPENSSL_DIR=/opt/openssl 2024-08-06T20:28:16.8676152Z GITHUB_ACTION_REPOSITORY= 2024-08-06T20:28:16.8677251Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-06T20:28:16.8678425Z GITHUB_BASE_REF=gh/drisspg/23/base 2024-08-06T20:28:16.8678831Z INSTALLED_ACL= 2024-08-06T20:28:16.8679116Z CI=true 2024-08-06T20:28:16.8679420Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-06T20:28:16.8679791Z JOB_ID=28427732833 2024-08-06T20:28:16.8680113Z INSTALLED_PROTOBUF=yes 2024-08-06T20:28:16.8680478Z GITHUB_HEAD_REF=gh/drisspg/23/head 2024-08-06T20:28:16.8680873Z GITHUB_ACTION_REF= 2024-08-06T20:28:16.8681278Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-06T20:28:16.8681780Z TEST_SHOWLOCALS=False 2024-08-06T20:28:16.8682116Z GITHUB_WORKFLOW=pull 2024-08-06T20:28:16.8682479Z DEBIAN_FRONTEND=noninteractive 2024-08-06T20:28:16.8683351Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_026c3442-fea7-4e0c-87f6-c52381528af5 2024-08-06T20:28:16.8684250Z NO_TD=False 2024-08-06T20:28:16.8684573Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-06T20:28:16.8684955Z _=/usr/bin/env 2024-08-06T20:28:16.8685253Z + echo 'Testing pytorch' 2024-08-06T20:28:16.8685610Z Testing pytorch 2024-08-06T20:28:16.8685935Z + export LANG=C.UTF-8 2024-08-06T20:28:16.8686174Z + LANG=C.UTF-8 2024-08-06T20:28:16.8764738Z + PR_NUMBER=132710 2024-08-06T20:28:16.8765221Z + [[ distributed == \d\e\f\a\u\l\t ]] 2024-08-06T20:28:16.8765741Z + [[ distributed == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-06T20:28:16.8766338Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-06T20:28:16.8766881Z + [[ distributed == \s\l\o\w ]] 2024-08-06T20:28:16.8767402Z + [[ linux-jammy-py3.8-gcc11 == *slow-gradcheck* ]] 2024-08-06T20:28:16.8767995Z + [[ linux-jammy-py3.8-gcc11 == *cuda* ]] 2024-08-06T20:28:16.8768533Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-06T20:28:16.8769073Z + [[ linux-jammy-py3.8-gcc11 == *xpu* ]] 2024-08-06T20:28:16.8769424Z + [[ distributed == *crossref* ]] 2024-08-06T20:28:16.8769744Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-06T20:28:16.8770068Z + [[ linux-jammy-py3.8-gcc11 == *xpu* ]] 2024-08-06T20:28:16.8772081Z + [[ linux-jammy-py3.8-gcc11 != *-bazel-* ]] 2024-08-06T20:28:16.8772496Z + pip_install --user ninja==1.10.2 2024-08-06T20:28:16.8772851Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-06T20:28:19.3260555Z Collecting ninja==1.10.2 2024-08-06T20:28:19.3426939Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-06T20:28:19.3584784Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-06T20:28:20.5741489Z Installing collected packages: ninja 2024-08-06T20:28:20.5837224Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-06T20:28:20.5838217Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-06T20:28:20.5888119Z Successfully installed ninja-1.10.2 2024-08-06T20:28:20.6818676Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-06T20:28:20.6820311Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-06T20:28:20.6821276Z + [[ linux-jammy-py3.8-gcc11 == *aarch64* ]] 2024-08-06T20:28:20.6821607Z + install_tlparse 2024-08-06T20:28:20.6821870Z + pip_install --user tlparse==0.3.7 2024-08-06T20:28:20.6822248Z + pip install --progress-bar off --user tlparse==0.3.7 2024-08-06T20:28:21.1034787Z Collecting tlparse==0.3.7 2024-08-06T20:28:21.1194345Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-08-06T20:28:21.1282227Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-06T20:28:22.3588449Z Installing collected packages: tlparse 2024-08-06T20:28:22.3992024Z Successfully installed tlparse-0.3.7 2024-08-06T20:28:22.4929455Z ++ python -m site --user-base 2024-08-06T20:28:22.5156823Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-06T20:28:22.5158190Z + [[ linux-jammy-py3.8-gcc11 == *asan* ]] 2024-08-06T20:28:22.5158552Z + [[ linux-jammy-py3.8-gcc11 == *-debug* ]] 2024-08-06T20:28:22.5158893Z + [[ linux-jammy-py3.8-gcc11 != *-bazel-* ]] 2024-08-06T20:28:22.5159399Z + echo 'We are not in debug mode: linux-jammy-py3.8-gcc11. Expect the assertion to pass' 2024-08-06T20:28:22.5160103Z We are not in debug mode: linux-jammy-py3.8-gcc11. Expect the assertion to pass 2024-08-06T20:28:22.5160566Z + cd test 2024-08-06T20:28:22.5160920Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-06T20:28:24.2062998Z + [[ distributed == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-06T20:28:24.2063443Z + [[ distributed == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-06T20:28:24.2065736Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-06T20:28:24.2066147Z + [[ distributed == *dynamo_eager* ]] 2024-08-06T20:28:24.2066549Z + [[ distributed == *aot_eager* ]] 2024-08-06T20:28:24.2066878Z + [[ distributed == *aot_inductor* ]] 2024-08-06T20:28:24.2067197Z + [[ distributed == *inductor* ]] 2024-08-06T20:28:24.2067500Z + [[ distributed == *dynamic* ]] 2024-08-06T20:28:24.2067790Z + [[ distributed == *cpu* ]] 2024-08-06T20:28:24.2068109Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-06T20:28:24.2088946Z + [[ linux-jammy-py3.8-gcc11 == *libtorch* ]] 2024-08-06T20:28:24.2089363Z + [[ linux-jammy-py3.8-gcc11 == *-bazel-* ]] 2024-08-06T20:28:24.2092670Z + cd test 2024-08-06T20:28:24.2093291Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-06T20:28:25.4330633Z PyTorch built with: 2024-08-06T20:28:25.4330962Z - GCC 11.4 2024-08-06T20:28:25.4331203Z - C++ Version: 201703 2024-08-06T20:28:25.4331808Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-06T20:28:25.4332539Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-06T20:28:25.4332987Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-06T20:28:25.4333329Z - LAPACK is enabled (usually provided by MKL) 2024-08-06T20:28:25.4333724Z - NNPACK is enabled 2024-08-06T20:28:25.4334028Z - CPU capability usage: AVX512 2024-08-06T20:28:25.4339550Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-06T20:28:25.4344817Z 2024-08-06T20:28:25.6601612Z + cd test 2024-08-06T20:28:25.6602205Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-06T20:28:26.8788633Z ATen/Parallel: 2024-08-06T20:28:26.8789021Z at::get_num_threads() : 4 2024-08-06T20:28:26.8789344Z at::get_num_interop_threads() : 4 2024-08-06T20:28:26.8789678Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-06T20:28:26.8789983Z omp_get_max_threads() : 4 2024-08-06T20:28:26.8790550Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-06T20:28:26.8791155Z mkl_get_max_threads() : 4 2024-08-06T20:28:26.8791550Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-06T20:28:26.8792009Z std::thread::hardware_concurrency() : 8 2024-08-06T20:28:26.8792329Z Environment variables: 2024-08-06T20:28:26.8792598Z OMP_NUM_THREADS : [not set] 2024-08-06T20:28:26.8792869Z MKL_NUM_THREADS : [not set] 2024-08-06T20:28:26.8793160Z ATen parallel backend: OpenMP 2024-08-06T20:28:26.8793372Z 2024-08-06T20:28:27.1094850Z + [[ linux-jammy-py3.8-gcc11 == *aarch64* ]] 2024-08-06T20:28:27.1095256Z + [[ distributed == *backward* ]] 2024-08-06T20:28:27.1095643Z + [[ distributed == *xla* ]] 2024-08-06T20:28:27.1095944Z + [[ distributed == *executorch* ]] 2024-08-06T20:28:27.1096333Z + [[ distributed == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-06T20:28:27.1096682Z + [[ linux-jammy-py3.8-gcc11 == *libtorch* ]] 2024-08-06T20:28:27.1097068Z + [[ distributed == distributed ]] 2024-08-06T20:28:27.1097364Z + test_distributed 2024-08-06T20:28:27.1097645Z + echo 'Testing distributed python tests' 2024-08-06T20:28:27.1098018Z Testing distributed python tests 2024-08-06T20:28:27.1098474Z + python test/run_test.py --distributed-tests --shard 2 2 --verbose 2024-08-06T20:28:27.2294234Z test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-06T20:28:27.2294975Z import pkg_resources 2024-08-06T20:28:30.7441531Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-06T20:28:30.8051227Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-06T20:28:30.8526387Z Ignoring disabled issues: [''] 2024-08-06T20:28:30.8639815Z Found test times from artifacts 2024-08-06T20:28:30.9060785Z Found test times from artifacts 2024-08-06T20:28:30.9078394Z Running all tests 2024-08-06T20:28:30.9207281Z Running parallel tests on 3 processes 2024-08-06T20:28:30.9213611Z Name: tests to run (est. time: 58.41min) 2024-08-06T20:28:30.9214257Z Serial tests (106): 2024-08-06T20:28:30.9214698Z distributed/checkpoint/test_nested_dict 1/1 2024-08-06T20:28:30.9215151Z distributed/_tensor/debug/test_op_coverage 1/1 2024-08-06T20:28:30.9215525Z distributed/_tools/test_mod_tracker 1/1 2024-08-06T20:28:30.9215886Z distributed/pipelining/test_backward 1/1 2024-08-06T20:28:30.9216330Z distributed/tensor/parallel/test_micro_pipeline_tp 1/1 2024-08-06T20:28:30.9216746Z distributed/pipelining/test_transformer 1/1 2024-08-06T20:28:30.9217145Z distributed/_shard/sharded_tensor/test_logger 1/1 2024-08-06T20:28:30.9217525Z distributed/test_symmetric_memory 1/1 2024-08-06T20:28:30.9218108Z distributed/_tensor/test_dtensor_compile 1/1 2024-08-06T20:28:30.9218485Z distributed/_tensor/test_attention 1/1 2024-08-06T20:28:30.9218848Z distributed/_composable/test_contract 1/1 2024-08-06T20:28:30.9219190Z distributed/_tensor/test_dtensor 1/1 2024-08-06T20:28:30.9219559Z distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 2024-08-06T20:28:30.9219949Z distributed/_tensor/test_xla_integration 1/1 2024-08-06T20:28:30.9220353Z distributed/_tensor/experimental/test_tp_transform 1/1 2024-08-06T20:28:30.9220782Z distributed/checkpoint/test_dedup_tensors 1/1 2024-08-06T20:28:30.9221166Z distributed/pipelining/test_unflatten 1/1 2024-08-06T20:28:30.9221673Z distributed/_tensor/test_convolution_ops 1/1 2024-08-06T20:28:30.9222044Z distributed/_tensor/test_tensor_ops 1/1 2024-08-06T20:28:30.9222417Z distributed/pipelining/test_composability 1/1 2024-08-06T20:28:30.9222794Z distributed/pipelining/test_microbatch 1/1 2024-08-06T20:28:30.9223198Z distributed/tensor/parallel/test_tp_examples 1/1 2024-08-06T20:28:30.9223590Z distributed/checkpoint/test_traverse 1/1 2024-08-06T20:28:30.9223923Z distributed/test_device_mesh 1/1 2024-08-06T20:28:30.9224253Z distributed/pipelining/test_stage 1/1 2024-08-06T20:28:30.9224600Z distributed/checkpoint/test_planner 1/1 2024-08-06T20:28:30.9224941Z distributed/checkpoint/test_fsspec 1/1 2024-08-06T20:28:30.9225295Z distributed/_tensor/test_op_strategy 1/1 2024-08-06T20:28:30.9225694Z distributed/tensor/parallel/test_ddp_2d_parallel 1/1 2024-08-06T20:28:30.9226087Z distributed/test_inductor_collectives 1/1 2024-08-06T20:28:30.9226482Z distributed/checkpoint/test_dtensor_checkpoint 1/1 2024-08-06T20:28:30.9226871Z distributed/launcher/test_run 1/1 2024-08-06T20:28:30.9227220Z distributed/checkpoint/e2e/test_fine_tuning 1/1 2024-08-06T20:28:30.9227627Z distributed/checkpoint/test_compatibility 1/1 2024-08-06T20:28:30.9228030Z distributed/checkpoint/test_hsdp_checkpoint 1/1 2024-08-06T20:28:30.9228513Z distributed/_composable/test_composability/test_2d_composability 1/1 2024-08-06T20:28:30.9228982Z distributed/test_collective_utils 1/1 2024-08-06T20:28:30.9229386Z distributed/_composable/fsdp/test_fully_shard_memory 1/1 2024-08-06T20:28:30.9229834Z distributed/_composable/fsdp/test_fully_shard_init 1/1 2024-08-06T20:28:30.9230454Z distributed/fsdp/test_fsdp_optim_state 1/1 2024-08-06T20:28:30.9230817Z distributed/checkpoint/test_utils 1/1 2024-08-06T20:28:30.9231171Z distributed/checkpoint/e2e/test_pipeline 1/1 2024-08-06T20:28:30.9231552Z distributed/elastic/test_control_plane 1/1 2024-08-06T20:28:30.9231922Z distributed/test_c10d_pypg 1/1 2024-08-06T20:28:30.9232352Z distributed/_composable/fsdp/test_fully_shard_mixed_precision 1/1 2024-08-06T20:28:30.9232816Z distributed/fsdp/test_fsdp_ignored_modules 1/1 2024-08-06T20:28:30.9233185Z distributed/test_nccl 1/1 2024-08-06T20:28:30.9233503Z distributed/_composable/test_compose 1/1 2024-08-06T20:28:30.9233929Z distributed/_composable/fully_shard/test_fully_shard_util 1/1 2024-08-06T20:28:30.9234461Z distributed/_composable/fsdp/test_fully_shard_training 1/1 2024-08-06T20:28:30.9234968Z distributed/fsdp/test_fsdp_tp_integration 1/1 2024-08-06T20:28:30.9235362Z distributed/fsdp/test_fsdp_dtensor_state_dict 1/1 2024-08-06T20:28:30.9235832Z distributed/_composable/fsdp/test_fully_shard_state_dict 1/1 2024-08-06T20:28:30.9236276Z distributed/fsdp/test_fsdp_pure_fp16 1/1 2024-08-06T20:28:30.9236633Z distributed/checkpoint/test_checkpoint 1/1 2024-08-06T20:28:30.9237088Z distributed/_composable/fsdp/test_fully_shard_frozen 1/1 2024-08-06T20:28:30.9237543Z distributed/fsdp/test_distributed_checkpoint 1/1 2024-08-06T20:28:30.9237998Z distributed/_composable/fsdp/test_fully_shard_autograd 1/1 2024-08-06T20:28:30.9238399Z distributed/test_fake_pg 1/1 2024-08-06T20:28:30.9238722Z distributed/rpc/test_share_memory 1/1 2024-08-06T20:28:30.9239238Z distributed/fsdp/test_fsdp_state_dict 1/1 2024-08-06T20:28:30.9239587Z distributed/test_distributed_spawn 1/3 2024-08-06T20:28:30.9239944Z distributed/test_distributed_spawn 3/3 2024-08-06T20:28:30.9240291Z distributed/fsdp/test_fsdp_memory 1/1 2024-08-06T20:28:30.9240712Z distributed/_composable/fully_shard/test_fully_shard_runtime 1/1 2024-08-06T20:28:30.9241237Z distributed/checkpoint/test_file_system_checkpoint_cpu 1/1 2024-08-06T20:28:30.9241660Z distributed/test_data_parallel 1/1 2024-08-06T20:28:30.9242047Z distributed/algorithms/ddp_comm_hooks/test_ddp_hooks 1/1 2024-08-06T20:28:30.9242467Z distributed/fsdp/test_fsdp_comm_hooks 1/1 2024-08-06T20:28:30.9242910Z distributed/test_multi_threaded_pg 1/1 2024-08-06T20:28:30.9243243Z distributed/fsdp/test_fsdp_apply 1/1 2024-08-06T20:28:30.9243610Z distributed/fsdp/test_fsdp_mixed_precision 1/1 2024-08-06T20:28:30.9243987Z distributed/fsdp/test_fsdp_input 1/1 2024-08-06T20:28:30.9244354Z distributed/fsdp/test_fsdp_multiple_wrapping 1/1 2024-08-06T20:28:30.9244734Z distributed/fsdp/test_fsdp_uneven 1/1 2024-08-06T20:28:30.9245089Z distributed/fsdp/test_fsdp_traversal 1/1 2024-08-06T20:28:30.9245430Z distributed/fsdp/test_fsdp_misc 1/1 2024-08-06T20:28:30.9245762Z distributed/test_pg_wrapper 1/1 2024-08-06T20:28:30.9246101Z distributed/rpc/test_tensorpipe_agent 2/3 2024-08-06T20:28:30.9246570Z distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint 1/1 2024-08-06T20:28:30.9247079Z distributed/rpc/cuda/test_tensorpipe_agent 1/1 2024-08-06T20:28:30.9247508Z distributed/_shard/sharded_optim/test_sharded_optim 1/1 2024-08-06T20:28:30.9247928Z distributed/fsdp/test_fsdp_multiple_forward 1/1 2024-08-06T20:28:30.9248303Z distributed/fsdp/test_fsdp_comm 1/1 2024-08-06T20:28:30.9248634Z distributed/test_c10d_common 1/1 2024-08-06T20:28:30.9248988Z distributed/_shard/sharded_tensor/ops/test_init 1/1 2024-08-06T20:28:30.9249373Z distributed/rpc/test_faulty_agent 1/1 2024-08-06T20:28:30.9249809Z distributed/_shard/sharded_tensor/test_sharded_tensor_reshard 1/1 2024-08-06T20:28:30.9250304Z distributed/_shard/sharded_tensor/test_sharded_tensor 1/1 2024-08-06T20:28:30.9250769Z distributed/_shard/sharded_tensor/ops/test_embedding 1/1 2024-08-06T20:28:30.9251247Z distributed/_shard/sharded_tensor/ops/test_embedding_bag 1/1 2024-08-06T20:28:30.9251653Z distributed/test_c10d_nccl 1/1 2024-08-06T20:28:30.9251972Z distributed/test_c10d_spawn_gloo 1/1 2024-08-06T20:28:30.9252307Z distributed/test_c10d_spawn_ucc 1/1 2024-08-06T20:28:30.9252627Z distributed/test_c10d_spawn_nccl 1/1 2024-08-06T20:28:30.9252957Z distributed/test_launcher 1/1 2024-08-06T20:28:30.9253271Z distributed/test_store 1/1 2024-08-06T20:28:30.9253570Z distributed/elastic/events/lib_test 1/1 2024-08-06T20:28:30.9253925Z distributed/elastic/metrics/api_test 1/1 2024-08-06T20:28:30.9254403Z distributed/elastic/multiprocessing/api_test 1/1 2024-08-06T20:28:30.9254788Z distributed/elastic/timer/api_test 1/1 2024-08-06T20:28:30.9255173Z distributed/elastic/timer/local_timer_example 1/1 2024-08-06T20:28:30.9255580Z distributed/elastic/timer/local_timer_test 1/1 2024-08-06T20:28:30.9255963Z distributed/elastic/utils/distributed_test 1/1 2024-08-06T20:28:30.9256351Z distributed/elastic/utils/logging_test 1/1 2024-08-06T20:28:30.9256712Z distributed/elastic/utils/util_test 1/1 2024-08-06T20:28:30.9257030Z Parallel tests (0): 2024-08-06T20:28:30.9257302Z Name: excluded (est. time: 0.0min) 2024-08-06T20:28:30.9257604Z Serial tests (0): 2024-08-06T20:28:30.9257839Z Parallel tests (0): 2024-08-06T20:28:30.9258221Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-06T20:28:30.9482523Z Running distributed/checkpoint/test_nested_dict 1/1 ... [2024-08-06 20:28:30.947798] 2024-08-06T20:28:30.9489002Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_nested_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:28:30.948443] 2024-08-06T20:28:34.7278422Z 2024-08-06T20:28:34.7279856Z distributed/checkpoint/test_nested_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_nested_dict_1.1_3982453ef09520ae_.log 2024-08-06T20:28:34.7282104Z Running 2 items in this shard: test/distributed/checkpoint/test_nested_dict.py::TestFlattening::test_flattening_round_trip, test/distributed/checkpoint/test_nested_dict.py::TestFlattening::test_mapping 2024-08-06T20:28:34.7283001Z 2024-08-06T20:28:34.7287369Z Running distributed/_tensor/debug/test_op_coverage 1/1 ... [2024-08-06 20:28:34.728543] 2024-08-06T20:28:34.7292572Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/debug/test_op_coverage.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:28:34.728992] 2024-08-06T20:28:37.9540065Z 2024-08-06T20:28:37.9541496Z distributed/_tensor/debug/test_op_coverage 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.debug.test_op_coverage_1.1_35b1de57236a0d7a_.log 2024-08-06T20:28:37.9542870Z Running 1 items in this shard: test/distributed/_tensor/debug/test_op_coverage.py::TestOpCoverage::test_trace_with_inductor_decomp 2024-08-06T20:28:37.9543457Z 2024-08-06T20:28:37.9546622Z Running distributed/_tools/test_mod_tracker 1/1 ... [2024-08-06 20:28:37.954472] 2024-08-06T20:28:37.9551634Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tools/test_mod_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:28:37.954943] 2024-08-06T20:28:41.0299147Z 2024-08-06T20:28:41.0300829Z distributed/_tools/test_mod_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_mod_tracker_1.1_abf692734e99f766_.log 2024-08-06T20:28:41.0304895Z Running 4 items in this shard: test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_ac, test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_bw_detection, test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_module_hierarchy, test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_user_hooks 2024-08-06T20:28:41.0307322Z 2024-08-06T20:28:41.0308405Z Running distributed/pipelining/test_backward 1/1 ... [2024-08-06 20:28:41.030626] 2024-08-06T20:28:41.0314679Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_backward.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:28:41.031179] 2024-08-06T20:28:44.5567796Z 2024-08-06T20:28:44.5569043Z distributed/pipelining/test_backward 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_backward_1.1_706e065e140c5a02_.log 2024-08-06T20:28:44.5570352Z Running 1 items in this shard: test/distributed/pipelining/test_backward.py::StageBackwardTests::test_stage_backward 2024-08-06T20:28:44.5570880Z 2024-08-06T20:28:44.5574092Z Running distributed/tensor/parallel/test_micro_pipeline_tp 1/1 ... [2024-08-06 20:28:44.557211] 2024-08-06T20:28:44.5579045Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_micro_pipeline_tp.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:28:44.557690] 2024-08-06T20:28:48.4843569Z 2024-08-06T20:28:48.4845947Z distributed/tensor/parallel/test_micro_pipeline_tp 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_micro_pipeline_tp_1.1_d4d4bf780fa701ad_.log 2024-08-06T20:28:48.4873832Z Running 29 items in this shard: test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_dtensor_seq_par_shard_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_dtensor_seq_par_shard_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_find_all_gather_patterns, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_find_reduce_scatter_patterns, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_2_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_2_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_2_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_3_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_3_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_3_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_2_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_2_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_2_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_3_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_3_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_3_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_2_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_2_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_2_scatter_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_3_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_3_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_3_scatter_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_2_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_2_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_2_scatter_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_3_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_3_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_3_scatter_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_get_unexposed_collectives 2024-08-06T20:28:48.4910965Z 2024-08-06T20:28:48.4911315Z Running distributed/pipelining/test_transformer 1/1 ... [2024-08-06 20:28:48.484955] 2024-08-06T20:28:48.4912672Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_transformer.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:28:48.485546] 2024-08-06T20:28:52.5637707Z 2024-08-06T20:28:52.5639225Z distributed/pipelining/test_transformer 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_transformer_1.1_c9dca16cc8b1d03e_.log 2024-08-06T20:28:52.5640978Z Running 1 items in this shard: test/distributed/pipelining/test_transformer.py::TransformerTests::test_ir 2024-08-06T20:28:52.5641532Z 2024-08-06T20:28:52.5644203Z Running distributed/_shard/sharded_tensor/test_logger 1/1 ... [2024-08-06 20:28:52.564232] 2024-08-06T20:28:52.5649030Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/test_logger.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:28:52.564664] 2024-08-06T20:28:55.5391914Z 2024-08-06T20:28:55.5393299Z distributed/_shard/sharded_tensor/test_logger 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.test_logger_1.1_7a62f46a4ac1757a_.log 2024-08-06T20:28:55.5394724Z Running 1 items in this shard: test/distributed/_shard/sharded_tensor/test_logger.py::ShardingSpecLoggerTest::test_get_or_create_logger 2024-08-06T20:28:55.5395351Z 2024-08-06T20:28:55.5398206Z Running distributed/test_symmetric_memory 1/1 ... [2024-08-06 20:28:55.539625] 2024-08-06T20:28:55.5403109Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_symmetric_memory.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:28:55.540058] 2024-08-06T20:28:58.2643504Z 2024-08-06T20:28:58.2644947Z distributed/test_symmetric_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_symmetric_memory_1.1_538ea45d724dfe9e_.log 2024-08-06T20:28:58.2656405Z Running 20 items in this shard: test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_cuda_nvlink_connectivity_detection, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_empty_strided_p2p, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_empty_strided_p2p_persistent, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_matmul_gather_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_matmul_gather_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_scaled_matmul_gather_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_scaled_matmul_gather_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_matmul_reduce_scatter_scatter_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_matmul_reduce_scatter_scatter_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_scaled_matmul_reduce_scatter_scatter_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_scaled_matmul_reduce_scatter_scatter_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_all_gather_symm_mem_input_False, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_all_gather_symm_mem_input_True, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_avg_symm_mem_input_False, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_avg_symm_mem_input_True, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_sum_symm_mem_input_False, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_sum_symm_mem_input_True, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_optimal_layout_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_optimal_layout_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_optimal_layout_dim_2 2024-08-06T20:28:58.2665775Z 2024-08-06T20:28:58.2666059Z Running distributed/_tensor/test_dtensor_compile 1/1 ... [2024-08-06 20:28:58.264826] 2024-08-06T20:28:58.2667485Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_dtensor_compile.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:28:58.265270] 2024-08-06T20:29:25.4418263Z 2024-08-06T20:29:25.4419496Z distributed/_tensor/test_dtensor_compile 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_dtensor_compile_1.1_7d26bc3deb116800_.log 2024-08-06T20:29:25.4433864Z Running 25 items in this shard: test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_device_mesh_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_attribute_access_on_intermediate, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_constructor_w_dynamo_disable, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_constructor_w_graph_break, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_contiguous_dtensor_noncontiguous_local_as_tangent, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_different_gradient_placement, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_dynamo_device_mesh_attrs, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_noncontiguous_output, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_partial_placement_graph_output, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_partial_placement_redistribute_unbalanced_correct_strides, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor_from_local, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor_from_local_redistribute, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor_recompile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_to_local_kwargs, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_to_local_kwargs_forward_hook, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_fakify_dtensor, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_graph_input_is_async, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_placement_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_tp_compile_comm_reordering, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_2d_fsdp_tp_ac_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_2d_fsdp_tp_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_compile_dtensor_redistribute_backward, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_tp_compile_fullgraph_is_seq_parallel_False, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_tp_compile_fullgraph_is_seq_parallel_True 2024-08-06T20:29:25.4446147Z 2024-08-06T20:29:25.4446416Z Running distributed/_tensor/test_attention 1/1 ... [2024-08-06 20:29:25.442335] 2024-08-06T20:29:25.4447657Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_attention.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:29:25.442761] 2024-08-06T20:29:31.5734494Z 2024-08-06T20:29:31.5735531Z distributed/_tensor/test_attention 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_attention_1.1_6c7430925d044677_.log 2024-08-06T20:29:31.5740397Z Running 8 items in this shard: test/distributed/_tensor/test_attention.py::RingAttentionTest::test_is_causal_behavior, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_compile_attention_fn0, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_compile_attention_fn1, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_custom_transformer, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_native_transformer_is_causal_False, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_native_transformer_is_causal_True, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_sdpa_is_causal_False, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_sdpa_is_causal_True 2024-08-06T20:29:31.5744276Z 2024-08-06T20:29:31.5744606Z Running distributed/_composable/test_contract 1/1 ... [2024-08-06 20:29:31.573871] 2024-08-06T20:29:31.5745905Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_contract.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:29:31.574282] 2024-08-06T20:29:34.6011980Z 2024-08-06T20:29:34.6013070Z distributed/_composable/test_contract 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_contract_1.1_bcc7cf669987594b_.log 2024-08-06T20:29:34.6015624Z Running 5 items in this shard: test/distributed/_composable/test_contract.py::TestContract::test_add_hooks, test/distributed/_composable/test_contract.py::TestContract::test_modify_fqn, test/distributed/_composable/test_contract.py::TestContract::test_multi_module_api, test/distributed/_composable/test_contract.py::TestContract::test_registry, test/distributed/_composable/test_contract.py::TestContract::test_state 2024-08-06T20:29:34.6017408Z 2024-08-06T20:29:34.6018618Z Running distributed/_tensor/test_dtensor 1/1 ... [2024-08-06 20:29:34.601701] 2024-08-06T20:29:34.6023871Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_dtensor.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:29:34.602168] 2024-08-06T20:31:22.7865520Z 2024-08-06T20:31:22.7867088Z distributed/_tensor/test_dtensor 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_dtensor_1.1_554229b9a54ce713_.log 2024-08-06T20:31:22.7880599Z Running 32 items in this shard: test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_async_output, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_constructor, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_new_empty_strided, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_properties, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_save_load, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_spec_hash, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_spec_read_only_after_set, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_stride, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_from_local, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_from_local_negative_dim, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_from_local_then_to_local, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_from_local_uneven_sharding, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_from_local_uneven_sharding_raise_error, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_full_tensor_grad_hint, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_full_tensor_sync, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_meta_dtensor, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_modules_w_meta_dtensor, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_to_local, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_to_local_grad_hint, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_auto_implicit_replication, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_default_value_sub_mesh, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_device_mesh_nd, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_dtensor_2d_mesh, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_dtensor_api_device_mesh_context_manager, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_dtensor_device_mesh_device_conversion, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_dtensor_spec_local_shard_offset, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_from_local_sub_mesh, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_implicit_replication, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_metadata_consistency_check, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_redistribute_sub_mesh, test/distributed/_tensor/test_dtensor.py::TestDTensorPlacementTypes::test_split_tensor_1D, test/distributed/_tensor/test_dtensor.py::DTensorLogTest::test_dtensor_log 2024-08-06T20:31:22.7892150Z 2024-08-06T20:31:22.7892465Z Running distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 ... [2024-08-06 20:31:22.787167] 2024-08-06T20:31:22.7893884Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/fsdp/test_fsdp_dsd.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:31:22.787627] 2024-08-06T20:31:25.8127468Z 2024-08-06T20:31:25.8129237Z distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.fsdp.test_fsdp_dsd_1.1_85b4ecdbcd97bf0e_.log 2024-08-06T20:31:25.8134707Z Running 3 items in this shard: test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_1d_fsdp_cpu_offload_full_model_state_dict, test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_1d_fsdp_get_model_state_dict, test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_save_with_fsdp1_and_load_with_fsdp2 2024-08-06T20:31:25.8138076Z 2024-08-06T20:31:25.8138549Z Running distributed/_tensor/test_xla_integration 1/1 ... [2024-08-06 20:31:25.813507] 2024-08-06T20:31:25.8144302Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_xla_integration.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:31:25.814086] 2024-08-06T20:31:28.8392274Z 2024-08-06T20:31:28.8394053Z distributed/_tensor/test_xla_integration 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_xla_integration_1.1_fcb656dce4b7076e_.log 2024-08-06T20:31:28.8398855Z Running 3 items in this shard: test/distributed/_tensor/test_xla_integration.py::DTensorXLAIntegrationTest::test_xla_distribute_tensor_1d_replicate, test/distributed/_tensor/test_xla_integration.py::DTensorXLAIntegrationTest::test_xla_distribute_tensor_1d_shard, test/distributed/_tensor/test_xla_integration.py::DTensorXLAIntegrationTest::test_xla_distribute_tensor_2d 2024-08-06T20:31:28.8401787Z 2024-08-06T20:31:28.8402326Z Running distributed/_tensor/experimental/test_tp_transform 1/1 ... [2024-08-06 20:31:28.839881] 2024-08-06T20:31:28.8408088Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/experimental/test_tp_transform.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:31:28.840458] 2024-08-06T20:31:42.9894007Z 2024-08-06T20:31:42.9895264Z distributed/_tensor/experimental/test_tp_transform 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.experimental.test_tp_transform_1.1_427c20fdd6b0a402_.log 2024-08-06T20:31:42.9897679Z Running 3 items in this shard: test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_e2e, test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_no_bias, test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_with_uncovered_op 2024-08-06T20:31:42.9899264Z 2024-08-06T20:31:42.9899560Z Running distributed/checkpoint/test_dedup_tensors 1/1 ... [2024-08-06 20:31:42.989686] 2024-08-06T20:31:42.9903942Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_dedup_tensors.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:31:42.990161] 2024-08-06T20:31:45.9654245Z 2024-08-06T20:31:45.9655569Z distributed/checkpoint/test_dedup_tensors 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_dedup_tensors_1.1_19c6bb14af1a40b5_.log 2024-08-06T20:31:45.9656918Z Running 1 items in this shard: test/distributed/checkpoint/test_dedup_tensors.py::TestDedupTensor::test_dedup_shards 2024-08-06T20:31:45.9657454Z 2024-08-06T20:31:45.9662062Z Running distributed/pipelining/test_unflatten 1/1 ... [2024-08-06 20:31:45.965987] 2024-08-06T20:31:45.9667501Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_unflatten.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:31:45.966505] 2024-08-06T20:31:49.6921371Z 2024-08-06T20:31:49.6922550Z distributed/pipelining/test_unflatten 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_unflatten_1.1_59ecf857e03f6768_.log 2024-08-06T20:31:49.6923837Z Running 1 items in this shard: test/distributed/pipelining/test_unflatten.py::UnflattenTests::test_unflatten 2024-08-06T20:31:49.6924338Z 2024-08-06T20:31:49.6927620Z Running distributed/_tensor/test_convolution_ops 1/1 ... [2024-08-06 20:31:49.692595] 2024-08-06T20:31:49.6933098Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_convolution_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:31:49.693061] 2024-08-06T20:32:14.2164033Z 2024-08-06T20:32:14.2165682Z distributed/_tensor/test_convolution_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_convolution_ops_1.1_57bf8d653013d9cd_.log 2024-08-06T20:32:14.2172518Z Running 2 items in this shard: test/distributed/_tensor/test_convolution_ops.py::DistConvolutionOpsTest::test_depthwise_convolution, test/distributed/_tensor/test_convolution_ops.py::DistConvolutionOpsTest::test_downsampling_convolution 2024-08-06T20:32:14.2173919Z 2024-08-06T20:32:14.2174186Z Running distributed/_tensor/test_tensor_ops 1/1 ... [2024-08-06 20:32:14.216205] 2024-08-06T20:32:14.2175603Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_tensor_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:32:14.217119] 2024-08-06T20:33:37.0397127Z 2024-08-06T20:33:37.0398376Z distributed/_tensor/test_tensor_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_tensor_ops_1.1_2e51d5de20662031_.log 2024-08-06T20:33:37.0408650Z Running 25 items in this shard: test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_aten_contiguous, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_clone, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_contiguous, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_detach, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_dtensor_dtype_conversion, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_empty_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_equal, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_fill_inplace, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_fill_inplace_partial_sum, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_full_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_gather, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_index, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_inplace_op, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_new_empty_strided, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_new_full, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_ones_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_ones_like_partial_sum, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_op_out_variant, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_scatter, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_slice, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_stack, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_where_type_promotion, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_zero_inplace, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_zeros_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_zeros_like_partial_sum 2024-08-06T20:33:37.0417532Z 2024-08-06T20:33:37.0417831Z Running distributed/pipelining/test_composability 1/1 ... [2024-08-06 20:33:37.040588] 2024-08-06T20:33:37.0419187Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_composability.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:33:37.041017] 2024-08-06T20:33:40.3372112Z 2024-08-06T20:33:40.3373234Z distributed/pipelining/test_composability 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_composability_1.1_62f32717b5b569f3_.log 2024-08-06T20:33:40.3374136Z 2024-08-06T20:33:40.3378371Z Running distributed/pipelining/test_microbatch 1/1 ... [2024-08-06 20:33:40.337656] 2024-08-06T20:33:40.3383548Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_microbatch.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:33:40.338094] 2024-08-06T20:33:44.0642705Z 2024-08-06T20:33:44.0643815Z distributed/pipelining/test_microbatch 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_microbatch_1.1_e8e184ca756f82c6_.log 2024-08-06T20:33:44.0645489Z Running 2 items in this shard: test/distributed/pipelining/test_microbatch.py::MicrobatchTests::test_chunk_spec, test/distributed/pipelining/test_microbatch.py::MicrobatchTests::test_split_and_merge 2024-08-06T20:33:44.0646384Z 2024-08-06T20:33:44.0648636Z Running distributed/tensor/parallel/test_tp_examples 1/1 ... [2024-08-06 20:33:44.064692] 2024-08-06T20:33:44.0653483Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_tp_examples.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:33:44.065114] 2024-08-06T20:34:13.5970743Z 2024-08-06T20:34:13.5973776Z distributed/tensor/parallel/test_tp_examples 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_tp_examples_1.1_327e0e6708f6aa5d_.log 2024-08-06T20:34:13.5980400Z Running 9 items in this shard: test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_loss_parallel, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_mlp_inference, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_mlp_training_is_seq_parallel_False_recompute_activation_False, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_mlp_training_is_seq_parallel_True_recompute_activation_False, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_False_float32, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_False_float64, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_True_float32, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_True_float64, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_weight_tying 2024-08-06T20:34:13.5986243Z 2024-08-06T20:34:13.5986515Z Running distributed/checkpoint/test_traverse 1/1 ... [2024-08-06 20:34:13.597564] 2024-08-06T20:34:13.5987924Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_traverse.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:34:13.598021] 2024-08-06T20:34:16.6228011Z 2024-08-06T20:34:16.6229800Z distributed/checkpoint/test_traverse 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_traverse_1.1_6be42eacf814c4fd_.log 2024-08-06T20:34:16.6236576Z Running 7 items in this shard: test/distributed/checkpoint/test_traverse.py::TestTraverse::test_get_element, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_set_element, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_doesnt_ignore_intermediate_collections, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_nested_dict, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_nested_list, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_shallow, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_with_ordered_dict 2024-08-06T20:34:16.6241591Z 2024-08-06T20:34:16.6242005Z Running distributed/test_device_mesh 1/1 ... [2024-08-06 20:34:16.623579] 2024-08-06T20:34:16.6244544Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_device_mesh.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:34:16.624154] 2024-08-06T20:36:39.8166382Z 2024-08-06T20:36:39.8167392Z distributed/test_device_mesh 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_device_mesh_1.1_6902e9689b8f6306_.log 2024-08-06T20:36:39.8182752Z Running 39 items in this shard: test/distributed/test_device_mesh.py::DeviceMeshTest::test_assert_invalid_mesh_tensor, test/distributed/test_device_mesh.py::DeviceMeshTest::test_device_mesh_2d, test/distributed/test_device_mesh.py::DeviceMeshTest::test_device_mesh_init_backend, test/distributed/test_device_mesh.py::DeviceMeshTest::test_fake_pg_device_mesh, test/distributed/test_device_mesh.py::DeviceMeshTest::test_from_group_with_global_pg, test/distributed/test_device_mesh.py::DeviceMeshTest::test_from_group_with_invalid_mesh, test/distributed/test_device_mesh.py::DeviceMeshTest::test_get_group_and_get_all_groups, test/distributed/test_device_mesh.py::DeviceMeshTest::test_get_local_rank, test/distributed/test_device_mesh.py::DeviceMeshTest::test_get_local_rank_raises_exception, test/distributed/test_device_mesh.py::DeviceMeshTest::test_init_process_group, test/distributed/test_device_mesh.py::DeviceMeshTest::test_raises_invalid_device_type, test/distributed/test_device_mesh.py::DeviceMeshTest::test_set_mesh_dim_group_options, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_device_mesh_hash, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_device_mesh_nd, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_device_mesh_parent_child_hash, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_from_group_with_mesh_shape, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_get_local_rank_3d, test/distributed/test_device_mesh.py::InitDeviceMeshTest::test_init_device_mesh, test/distributed/test_device_mesh.py::InitDeviceMeshTest::test_raises_duplicate_mesh_dim_names, test/distributed/test_device_mesh.py::InitDeviceMeshTest::test_raises_mesh_shape_mesh_dim_names_mismatch, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_cache_and_reuse_submesh_slice_result, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_1d, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_2d, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_3d, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_3d_noncontinuous_slicing, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_raises_invalid_mesh_dim_name, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_raises_no_mesh_dim_found, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_mesh_dim_by_name, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_parent_mesh, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_parent_mesh_dim_exist, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_parent_mesh_dim_not_exist, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_all_gather_uneven, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_broadcast_1d, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_broadcast_nd, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_reduce_scatter_contiguous, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_reduce_scatter_uneven, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_scatter_1d, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_scatter_nd, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_scatter_uneven 2024-08-06T20:36:39.8196866Z 2024-08-06T20:36:39.8197152Z Running distributed/pipelining/test_stage 1/1 ... [2024-08-06 20:36:39.817198] 2024-08-06T20:36:39.8198551Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_stage.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:36:39.817673] 2024-08-06T20:36:43.1437695Z 2024-08-06T20:36:43.1438787Z distributed/pipelining/test_stage 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_stage_1.1_d0fe65a7f29f1056_.log 2024-08-06T20:36:43.1439609Z 2024-08-06T20:36:43.1443977Z Running distributed/checkpoint/test_planner 1/1 ... [2024-08-06 20:36:43.144230] 2024-08-06T20:36:43.1448839Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_planner.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:36:43.144667] 2024-08-06T20:36:46.1694840Z 2024-08-06T20:36:46.1695997Z distributed/checkpoint/test_planner 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_planner_1.1_027d276b129ce4b2_.log 2024-08-06T20:36:46.1699462Z Running 7 items in this shard: test/distributed/checkpoint/test_planner.py::TestSavePlan::test_global_plan, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_load_with_resharding, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_load_with_world_size_diff_by_one, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_local_load_plan, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_local_plan, test/distributed/checkpoint/test_planner.py::TestPlannerHelpers::test_create_read_item_from_chunks, test/distributed/checkpoint/test_planner.py::TestLoadPlanner::test_strict 2024-08-06T20:36:46.1702286Z 2024-08-06T20:36:46.1702553Z Running distributed/checkpoint/test_fsspec 1/1 ... [2024-08-06 20:36:46.169979] 2024-08-06T20:36:46.1706874Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_fsspec.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:36:46.170453] 2024-08-06T20:36:48.8445726Z 2024-08-06T20:36:48.8446965Z distributed/checkpoint/test_fsspec 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_fsspec_1.1_65d945fcf18a1286_.log 2024-08-06T20:36:48.8448484Z Running 2 items in this shard: test/distributed/checkpoint/test_fsspec.py::TestFSSpec::test_fsspec, test/distributed/checkpoint/test_fsspec.py::TestFSSpec::test_overwrite 2024-08-06T20:36:48.8449293Z 2024-08-06T20:36:48.8452130Z Running distributed/_tensor/test_op_strategy 1/1 ... [2024-08-06 20:36:48.845033] 2024-08-06T20:36:48.8456880Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_op_strategy.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:36:48.845464] 2024-08-06T20:36:51.9703460Z 2024-08-06T20:36:51.9704667Z distributed/_tensor/test_op_strategy 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_op_strategy_1.1_3f4b828ed3e28389_.log 2024-08-06T20:36:51.9711286Z Running 15 items in this shard: test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_batch_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_bmm_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_free_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_mm_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_bmm_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_bmm_2d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_linearity_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_mm_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_mm_2d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_pointwise_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_bmm_strategies, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_mm_strategies, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_redistribute_cost_latency, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_redistribute_cost_mesh_1d, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_redistribute_cost_mesh_2d 2024-08-06T20:36:51.9717111Z 2024-08-06T20:36:51.9717454Z Running distributed/tensor/parallel/test_ddp_2d_parallel 1/1 ... [2024-08-06 20:36:51.970765] 2024-08-06T20:36:51.9718831Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_ddp_2d_parallel.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:36:51.971220] 2024-08-06T20:36:58.0526121Z 2024-08-06T20:36:58.0528123Z distributed/tensor/parallel/test_ddp_2d_parallel 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_ddp_2d_parallel_1.1_f901284cb41853ad_.log 2024-08-06T20:36:58.0532949Z Running 1 items in this shard: test/distributed/tensor/parallel/test_ddp_2d_parallel.py::Test2dParallelIntegration::test_2d_ddp_integration_functionality 2024-08-06T20:36:58.0534298Z 2024-08-06T20:36:58.0534807Z Running distributed/test_inductor_collectives 1/1 ... [2024-08-06 20:36:58.053236] 2024-08-06T20:36:58.0541540Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_inductor_collectives.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:36:58.053796] 2024-08-06T20:37:02.0304516Z 2024-08-06T20:37:02.0306217Z distributed/test_inductor_collectives 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_inductor_collectives_1.1_97d7ac9c2dbea9ac_.log 2024-08-06T20:37:02.0345419Z Running 47 items in this shard: test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_all_to_all_single_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_all_to_all_single_inductor_split_sizes_none, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allgather_contiguous_input, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allgather_into_tensor_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allgather_output_buffer_reuse, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allreduce_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allreduce_inductor_cudagraph_trees, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allreduce_input_buffer_reuse, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_broadcast_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_c10d_functional_tagged_pt2_compliant, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_eager_allreduce_inductor_wait, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_inductor_allreduce_eager_wait, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_permute_tensor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_reduce_scatter_tensor_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_backwards, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_get_world_group_source_GroupMember_WORLD, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_get_world_group_source__get_default_group, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_get_world_group_source_group_WORLD, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_graphbreaks_unsupported_async_op, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_pg_var, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_gather, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_gather_args_match, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_gather_list, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_to_all_single, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_kwargs, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_kwargs_none, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_positional, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_positional_none, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_unspecified, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op0, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op1, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op2, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op3, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op4, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_reduce_scatter, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_support_collective_op_with_async_op_False, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_all_gather_tensor, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_all_gather_tensor_pg, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_allgather_coalesced, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_allreduce, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_reduce_scatter_tensor, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_all_gather_coalesced, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_doesnt_mutate_shared, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_reduce_scatter_coalesced, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_single_op, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_steal_buffer, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_meta 2024-08-06T20:37:02.0375069Z 2024-08-06T20:37:02.0375412Z Running distributed/checkpoint/test_dtensor_checkpoint 1/1 ... [2024-08-06 20:37:02.031229] 2024-08-06T20:37:02.0376825Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_dtensor_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:37:02.031801] 2024-08-06T20:37:07.9645005Z 2024-08-06T20:37:07.9647354Z distributed/checkpoint/test_dtensor_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_dtensor_checkpoint_1.1_958ee3feb4cebbfa_.log 2024-08-06T20:37:07.9650092Z Running 1 items in this shard: test/distributed/checkpoint/test_dtensor_checkpoint.py::DTensorPlanner::test_distributed_tensor_planner 2024-08-06T20:37:07.9651212Z 2024-08-06T20:37:07.9653763Z Running distributed/launcher/test_run 1/1 ... [2024-08-06 20:37:07.965164] 2024-08-06T20:37:07.9660249Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/launcher/test_run.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:37:07.965689] 2024-08-06T20:37:46.8981803Z 2024-08-06T20:37:46.8983797Z distributed/launcher/test_run 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.launcher.test_run_1.1_e85904df2c4b6f37_.log 2024-08-06T20:37:46.8997850Z Running 24 items in this shard: test/distributed/launcher/test_run.py::ElasticLaunchTest::test_capture_logs_using_default_logs_specs, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_init_method_env_with_torchelastic, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_init_method_tcp_with_torchelastic, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_is_not_torchelastic_launched, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_is_torchelastic_launched, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_is_torchelastic_launched_with_logs_spec_defined, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic_agent_raise_exception, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic_multiple_agents, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic_worker_raise_exception, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_run_path, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_shutdown, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_standalone, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_bash, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_default_nproc, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_python, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_python_caffe2_bc, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_with_env_vars, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_logs_logs_spec_entrypoint_must_be_defined, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_min_max_nodes_parse, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_gpu_launch_configurations, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_launch_auto_configurations, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_launch_number_configurations, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_launch_unknown_configurations 2024-08-06T20:37:46.9007739Z 2024-08-06T20:37:46.9008042Z Running distributed/checkpoint/e2e/test_fine_tuning 1/1 ... [2024-08-06 20:37:46.899176] 2024-08-06T20:37:46.9009502Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/e2e/test_fine_tuning.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:37:46.899636] 2024-08-06T20:37:49.8745239Z 2024-08-06T20:37:49.8746752Z distributed/checkpoint/e2e/test_fine_tuning 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.e2e.test_fine_tuning_1.1_522f91c935e84cba_.log 2024-08-06T20:37:49.8748217Z Running 1 items in this shard: test/distributed/checkpoint/e2e/test_fine_tuning.py::TestFineTuning::test_fine_tuning 2024-08-06T20:37:49.8748805Z 2024-08-06T20:37:49.8751358Z Running distributed/checkpoint/test_compatibility 1/1 ... [2024-08-06 20:37:49.874959] 2024-08-06T20:37:49.8756030Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_compatibility.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:37:49.875393] 2024-08-06T20:37:52.8498263Z 2024-08-06T20:37:52.8499668Z distributed/checkpoint/test_compatibility 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_compatibility_1.1_2c922256b4c1246f_.log 2024-08-06T20:37:52.8501986Z Running 3 items in this shard: test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_metadata, test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_sharded_tensor_dependency, test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_storage_meta 2024-08-06T20:37:52.8503443Z 2024-08-06T20:37:52.8504756Z Running distributed/checkpoint/test_hsdp_checkpoint 1/1 ... [2024-08-06 20:37:52.850278] 2024-08-06T20:37:52.8509395Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_hsdp_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:37:52.850719] 2024-08-06T20:38:07.2500678Z 2024-08-06T20:38:07.2501935Z distributed/checkpoint/test_hsdp_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_hsdp_checkpoint_1.1_b87181bd1a77dabe_.log 2024-08-06T20:38:07.2505139Z Running 4 items in this shard: test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_checkpoint_is_even_sharded_model_False, test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_checkpoint_is_even_sharded_model_True, test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_fsdp_checkpoint_conversion_is_even_sharded_model_False, test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_fsdp_checkpoint_conversion_is_even_sharded_model_True 2024-08-06T20:38:07.2507771Z 2024-08-06T20:38:07.2508162Z Running distributed/_composable/test_composability/test_2d_composability 1/1 ... [2024-08-06 20:38:07.250479] 2024-08-06T20:38:07.2511712Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_composability/test_2d_composability.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:38:07.250936] 2024-08-06T20:38:10.3260461Z 2024-08-06T20:38:10.3261892Z distributed/_composable/test_composability/test_2d_composability 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_composability.test_2d_composability_1.1_916b9014eb50b5b4_.log 2024-08-06T20:38:10.3264744Z Running 3 items in this shard: test/distributed/_composable/test_composability/test_2d_composability.py::TestFullyShard2DTraining::test_raise_not_implemented_state_dict_if_2d, test/distributed/_composable/test_composability/test_2d_composability.py::TestFullyShard2DTraining::test_tp_with_fsdp_offloading, test/distributed/_composable/test_composability/test_2d_composability.py::TestFullyShard2DTraining::test_train_parity_2d_mlp 2024-08-06T20:38:10.3266855Z 2024-08-06T20:38:10.3267114Z Running distributed/test_collective_utils 1/1 ... [2024-08-06 20:38:10.326461] 2024-08-06T20:38:10.3270960Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_collective_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:38:10.326856] 2024-08-06T20:38:12.9603826Z 2024-08-06T20:38:12.9605021Z distributed/test_collective_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_collective_utils_1.1_de922fbafdc9415c_.log 2024-08-06T20:38:12.9605827Z 2024-08-06T20:38:12.9609131Z Running distributed/_composable/fsdp/test_fully_shard_memory 1/1 ... [2024-08-06 20:38:12.960737] 2024-08-06T20:38:12.9614466Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_memory.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:38:12.961156] 2024-08-06T20:38:16.0359687Z 2024-08-06T20:38:16.0360956Z distributed/_composable/fsdp/test_fully_shard_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_memory_1.1_2141ed70355b05d6_.log 2024-08-06T20:38:16.0362486Z Running 1 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_memory.py::TestFullyShardMemory::test_fully_shard_training_memory 2024-08-06T20:38:16.0363152Z 2024-08-06T20:38:16.0365663Z Running distributed/_composable/fsdp/test_fully_shard_init 1/1 ... [2024-08-06 20:38:16.036397] 2024-08-06T20:38:16.0370240Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_init.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:38:16.036803] 2024-08-06T20:38:19.1118343Z 2024-08-06T20:38:19.1119670Z distributed/_composable/fsdp/test_fully_shard_init 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_init_1.1_b1ec25b9f1c1a132_.log 2024-08-06T20:38:19.1137813Z Running 28 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardDeviceTensor::test_move_states_to_device_tensor, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardDeviceDTensor::test_move_states_to_device_dtensor_invalid, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardDeviceDTensor::test_move_states_to_device_dtensor_valid, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMeshArg::test_invalid_mesh_ndim, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_duplicate, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_list_of_mlps, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_nested, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_nested_fully_shard_and_replicate, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_single, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_states_list_of_mlps, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_states_nested_fully_shard, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_states_shared_params_and_buffers, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardParamModuleInfos::test_get_param_module_infos_duplicates, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardParamModuleInfos::test_get_param_module_infos_list_of_mlps, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardParamModuleInfos::test_get_param_module_infos_shared_params, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardedParameterTensor::test_shard_tensor_parameters, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardedParameterDTensor::test_shard_dtensor_parameters, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_double_lazy_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_is_root, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_module_and_param_fqns, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_multi_module_root, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_invalid_meta_device_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_meta_device_1d_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_meta_device_2d_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_rank0_broadcast_meta_device_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardProcessGroupInit::test_1d_process_group_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardProcessGroupInit::test_2d_process_group_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardHSDPBroadcast::test_hsdp_broadcast_across_replicas 2024-08-06T20:38:19.1154668Z 2024-08-06T20:38:19.1154971Z Running distributed/fsdp/test_fsdp_optim_state 1/1 ... [2024-08-06 20:38:19.112312] 2024-08-06T20:38:19.1156393Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_optim_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:38:19.112744] 2024-08-06T20:41:27.0419321Z 2024-08-06T20:41:27.0422249Z distributed/fsdp/test_fsdp_optim_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_optim_state_1.1_e4729837e3bdd046_.log 2024-08-06T20:41:27.0463680Z Running 60 items in this shard: test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_compatible_with_trec, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_flatten_sharded_optim_state_dict_nested, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_flatten_sharded_optim_state_dict_transformer, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_full_optim_state_dict_keys, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_full_optim_state_dict_nested_invalid, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_interface_arguments, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_no_grad, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_input_warning, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_without_param_groups, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type0_use_multiple_param_groups_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type0_use_multiple_param_groups_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type1_use_multiple_param_groups_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type1_use_multiple_param_groups_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_names, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_save_load_without_0th_param_state_state_dict_type0, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_save_load_without_0th_param_state_state_dict_type1, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_halve_world_size, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_transformer, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_halve_world_size, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_transformer, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type0_add_to_fsdp_module_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type0_add_to_fsdp_module_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type1_add_to_fsdp_module_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type1_add_to_fsdp_module_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_state_dict_with_none_tensor_state, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_use_orig_params, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_with_empty_optimizer_state, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_with_no_shard 2024-08-06T20:41:27.0501046Z 2024-08-06T20:41:27.0501330Z Running distributed/checkpoint/test_utils 1/1 ... [2024-08-06 20:41:27.042632] 2024-08-06T20:41:27.0502612Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:41:27.043082] 2024-08-06T20:41:30.0680776Z 2024-08-06T20:41:30.0682178Z distributed/checkpoint/test_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_utils_1.1_67f1bec2409a5b38_.log 2024-08-06T20:41:30.0685591Z Running 6 items in this shard: test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_dcp_logger, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_flat_data, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_index_hint_ignored_on_equals, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_index_hint_ignored_on_hash, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_init_convert_offset, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_sharded_tensor_lookup 2024-08-06T20:41:30.0688328Z 2024-08-06T20:41:30.0688645Z Running distributed/checkpoint/e2e/test_pipeline 1/1 ... [2024-08-06 20:41:30.068503] 2024-08-06T20:41:30.0691312Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/e2e/test_pipeline.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:41:30.068912] 2024-08-06T20:41:33.0937685Z 2024-08-06T20:41:33.0938858Z distributed/checkpoint/e2e/test_pipeline 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.e2e.test_pipeline_1.1_b1e2aa70e117e8b0_.log 2024-08-06T20:41:33.0940178Z Running 1 items in this shard: test/distributed/checkpoint/e2e/test_pipeline.py::TestPipeline::test_pipeline 2024-08-06T20:41:33.0940741Z 2024-08-06T20:41:33.0943982Z Running distributed/elastic/test_control_plane 1/1 ... [2024-08-06 20:41:33.094213] 2024-08-06T20:41:33.0948894Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/test_control_plane.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:41:33.094645] 2024-08-06T20:41:39.2740265Z 2024-08-06T20:41:39.2741766Z distributed/elastic/test_control_plane 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.test_control_plane_1.1_bceb6f15e5e85eca_.log 2024-08-06T20:41:39.2746279Z Running 9 items in this shard: test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_nccl_trace_pickle, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_nccl_trace_pickle_with_json, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_nccl_trace_pickle_with_params, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_traceback, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_get_handler_names, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_get_handler_nonexistant, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_run_handler, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_tcp, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_worker_server 2024-08-06T20:41:39.2750275Z 2024-08-06T20:41:39.2750507Z Running distributed/test_c10d_pypg 1/1 ... [2024-08-06 20:41:39.274510] 2024-08-06T20:41:39.2751820Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_pypg.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:41:39.274965] 2024-08-06T20:43:35.0221155Z 2024-08-06T20:43:35.0222205Z distributed/test_c10d_pypg 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_pypg_1.1_e81c73f9d648e24d_.log 2024-08-06T20:43:35.0241824Z Running 42 items in this shard: test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_dataclass_output, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_dataclass_output_unused_param, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_dynamic_module, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_dynamic_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_once_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_once_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_static_graph_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_static_graph_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_unused_params_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_unused_params_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_weight_sharing_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_weight_sharing_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_invoke_work_object, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_with_pypg, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_with_pypg_with_grad_views, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_invalid_powerSGD_state, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_sync_batch_norm_empty_input, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_sync_batch_norm_only_empty_input, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_dataclass_output, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_dataclass_output_unused_param, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_dynamic_module, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_dynamic_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_once_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_once_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_static_graph_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_static_graph_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_unused_params_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_unused_params_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_weight_sharing_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_weight_sharing_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_invoke_work_object, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_with_pypg, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_with_pypg_with_grad_views, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_invalid_powerSGD_state, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_sync_batch_norm_empty_input, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_sync_batch_norm_only_empty_input 2024-08-06T20:43:35.0260309Z 2024-08-06T20:43:35.0260744Z Running distributed/_composable/fsdp/test_fully_shard_mixed_precision 1/1 ... [2024-08-06 20:43:35.022646] 2024-08-06T20:43:35.0262244Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_mixed_precision.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:43:35.023115] 2024-08-06T20:43:38.0982359Z 2024-08-06T20:43:38.0985445Z distributed/_composable/fsdp/test_fully_shard_mixed_precision 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_mixed_precision_1.1_0454f70353938677_.log 2024-08-06T20:43:38.0991118Z Running 7 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionTraining::test_compute_dtype, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionTraining::test_grad_acc_with_reduce_dtype, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionTraining::test_reduce_dtype, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_float16_on_one_submodule, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_norm_modules_bf16, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_norm_modules_fp16, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_submodules_with_external_inputs 2024-08-06T20:43:38.0995658Z 2024-08-06T20:43:38.0995977Z Running distributed/fsdp/test_fsdp_ignored_modules 1/1 ... [2024-08-06 20:43:38.098753] 2024-08-06T20:43:38.0997393Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_ignored_modules.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:43:38.099204] 2024-08-06T20:43:41.1743613Z 2024-08-06T20:43:41.1745451Z distributed/fsdp/test_fsdp_ignored_modules 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_ignored_modules_1.1_539d43280f3d45ac_.log 2024-08-06T20:43:41.1760080Z Running 13 items in this shard: test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_diff_ignored_modules_across_ranks, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_invalid_composable_False, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_invalid_composable_True, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_nested, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_nested_composable, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_False_composable_False, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_False_composable_True, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_True_composable_False, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_True_composable_True, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_transformer, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_transformer_composable, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_states_auto_wrap, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_states_check 2024-08-06T20:43:41.1771294Z 2024-08-06T20:43:41.1771616Z Running distributed/test_nccl 1/1 ... [2024-08-06 20:43:41.175129] 2024-08-06T20:43:41.1773364Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_nccl.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:43:41.175725] 2024-08-06T20:43:44.2016777Z 2024-08-06T20:43:44.2017735Z distributed/test_nccl 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_nccl_1.1_5374ddf54be974bd_.log 2024-08-06T20:43:44.2018575Z Running 0 items in this shard: 2024-08-06T20:43:44.2018769Z 2024-08-06T20:43:44.2023732Z Running distributed/_composable/test_compose 1/1 ... [2024-08-06 20:43:44.202177] 2024-08-06T20:43:44.2028752Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_compose.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:43:44.202644] 2024-08-06T20:44:11.8164906Z 2024-08-06T20:44:11.8165976Z distributed/_composable/test_compose 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_compose_1.1_f00f0326495618e8_.log 2024-08-06T20:44:11.8171298Z Running 9 items in this shard: test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fsdp_submodules_non_reentrant, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fsdp_submodules_with_param, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fsdp_submodules_with_param_no_shard, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fully_shard_cast_forward_inputs, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_composable_fsdp_replicate, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_fully_shard_replicate_composability, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_fully_shard_replicate_correct_replicate_params, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_state_dict_fsdp_submodules, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_wrap_same_submodule 2024-08-06T20:44:11.8175718Z 2024-08-06T20:44:11.8176075Z Running distributed/_composable/fully_shard/test_fully_shard_util 1/1 ... [2024-08-06 20:44:11.816881] 2024-08-06T20:44:11.8177531Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_util.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:44:11.817332] 2024-08-06T20:44:17.6470288Z 2024-08-06T20:44:17.6472346Z distributed/_composable/fully_shard/test_fully_shard_util 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_util_1.1_9b8050b29216a8a2_.log 2024-08-06T20:44:17.6475374Z Running 1 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_util.py::TestUtils::test_get_sharded_module_tree_with_module_name_to_fqns 2024-08-06T20:44:17.6476760Z 2024-08-06T20:44:17.6480108Z Running distributed/_composable/fsdp/test_fully_shard_training 1/1 ... [2024-08-06 20:44:17.647765] 2024-08-06T20:44:17.6484755Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_training.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:44:17.648215] 2024-08-06T20:44:20.7234862Z 2024-08-06T20:44:20.7236184Z distributed/_composable/fsdp/test_fully_shard_training 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_training_1.1_88e859eefba0b143_.log 2024-08-06T20:44:20.7248747Z Running 19 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_training.py::test_compiled_fsdp, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardForwardInputs::test_root_move_forward_input_to_device, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardRegisteredParams::test_param_registration_after_backward, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardRegisteredParams::test_param_registration_after_forward, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardCastAfterInit::test_to_float64_after_init, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_explicit_prefetching, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_multi_forward_module, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_non_root_forward_backward, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_post_optim_event, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_multi_group, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_multi_group_cpu_offload_eager, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_single_group, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCompose::test_train_parity_with_activation_checkpointing, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardSharedParams::test_train_parity_with_shared_params, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardGradientAccumulation::test_1f1b_microbatching, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardGradientAccumulation::test_gradient_accumulation, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardNDTraining::test_2d_mlp_with_nd_mesh, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardHSDPTraining::test_train_parity_hsdp, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardCustomForwardMethod::test_register_fsdp_forward_method 2024-08-06T20:44:20.7259606Z 2024-08-06T20:44:20.7259908Z Running distributed/fsdp/test_fsdp_tp_integration 1/1 ... [2024-08-06 20:44:20.724003] 2024-08-06T20:44:20.7261285Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_tp_integration.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:44:20.724462] 2024-08-06T20:44:32.2177016Z 2024-08-06T20:44:32.2178126Z distributed/fsdp/test_fsdp_tp_integration 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_tp_integration_1.1_4bf00e8ecb928c34_.log 2024-08-06T20:44:32.2180386Z Running 3 items in this shard: test/distributed/fsdp/test_fsdp_tp_integration.py::TestTPFSDPIntegration::test_fsdp_tp_extension_grad, test/distributed/fsdp/test_fsdp_tp_integration.py::TestTPFSDPIntegration::test_fsdp_tp_integration, test/distributed/fsdp/test_fsdp_tp_integration.py::TestTPFSDPIntegration::test_fsdp_tp_sync_module_state 2024-08-06T20:44:32.2182042Z 2024-08-06T20:44:32.2183750Z Running distributed/fsdp/test_fsdp_dtensor_state_dict 1/1 ... [2024-08-06 20:44:32.218125] 2024-08-06T20:44:32.2188459Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_dtensor_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:44:32.218598] 2024-08-06T20:45:18.4370010Z 2024-08-06T20:45:18.4373037Z distributed/fsdp/test_fsdp_dtensor_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_dtensor_state_dict_1.1_0c922aa3aba65cbe_.log 2024-08-06T20:45:18.4385673Z Running 15 items in this shard: test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_False_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_False_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_True_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_True_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_False_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_False_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_True_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_True_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_False_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_False_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_True_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_True_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_fsdp_init_with_device_mesh_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_fsdp_init_with_device_mesh_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_raises_warning_or_errors 2024-08-06T20:45:18.4396920Z 2024-08-06T20:45:18.4397273Z Running distributed/_composable/fsdp/test_fully_shard_state_dict 1/1 ... [2024-08-06 20:45:18.437519] 2024-08-06T20:45:18.4398712Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:45:18.438019] 2024-08-06T20:45:21.4730354Z 2024-08-06T20:45:21.4732341Z distributed/_composable/fsdp/test_fully_shard_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_state_dict_1.1_d04693461791bc3a_.log 2024-08-06T20:45:21.4738055Z Running 3 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_dp_state_dict_cpu_offload, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_dp_state_dict_save_load, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiThread::test_rank0_offload_full_state_dict 2024-08-06T20:45:21.4741541Z 2024-08-06T20:45:21.4742036Z Running distributed/fsdp/test_fsdp_pure_fp16 1/1 ... [2024-08-06 20:45:21.473783] 2024-08-06T20:45:21.4747215Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_pure_fp16.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:45:21.474374] 2024-08-06T20:45:30.3117472Z 2024-08-06T20:45:30.3118557Z distributed/fsdp/test_fsdp_pure_fp16 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_pure_fp16_1.1_c86f78f13a6ba134_.log 2024-08-06T20:45:30.3120131Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_pure_fp16.py::TestPureFP16::test_fp16_dtypes, test/distributed/fsdp/test_fsdp_pure_fp16.py::TestPureFP16::test_pure_fp16_training 2024-08-06T20:45:30.3120965Z 2024-08-06T20:45:30.3124152Z Running distributed/checkpoint/test_checkpoint 1/1 ... [2024-08-06 20:45:30.312215] 2024-08-06T20:45:30.3128833Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:45:30.312646] 2024-08-06T20:45:38.8489396Z 2024-08-06T20:45:38.8497178Z distributed/checkpoint/test_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_checkpoint_1.1_4d2e5dc01c9865a7_.log 2024-08-06T20:45:38.8502316Z Running 8 items in this shard: test/distributed/checkpoint/test_checkpoint.py::TestDistributedCheckpointing::test_default_metadata, test/distributed/checkpoint/test_checkpoint.py::TestDistributedCheckpointing::test_tensor_metadata_with_missing_rank_spec, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_dummy_reader_works, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_dummy_writer_works, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_load_error_handling, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_load_error_handling_no_dist, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_save_error_handling, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_save_error_handling_no_dist 2024-08-06T20:45:38.8506181Z 2024-08-06T20:45:38.8506525Z Running distributed/_composable/fsdp/test_fully_shard_frozen 1/1 ... [2024-08-06 20:45:38.849413] 2024-08-06T20:45:38.8507913Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_frozen.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:45:38.849850] 2024-08-06T20:45:41.8745234Z 2024-08-06T20:45:41.8746781Z distributed/_composable/fsdp/test_fully_shard_frozen 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_frozen_1.1_87b903bc43d4fe07_.log 2024-08-06T20:45:41.8749665Z Running 3 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_frozen.py::TestFullyShardFrozen::test_multi_forward_mixed_requires_grad, test/distributed/_composable/fsdp/test_fully_shard_frozen.py::TestFullyShardFrozen::test_train_mixed_requires_grad_across_groups, test/distributed/_composable/fsdp/test_fully_shard_frozen.py::TestFullyShardFrozen::test_train_mixed_requires_grad_per_group 2024-08-06T20:45:41.8751906Z 2024-08-06T20:45:41.8752213Z Running distributed/fsdp/test_distributed_checkpoint 1/1 ... [2024-08-06 20:45:41.874951] 2024-08-06T20:45:41.8756734Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_distributed_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:45:41.875386] 2024-08-06T20:45:44.9001276Z 2024-08-06T20:45:44.9002463Z distributed/fsdp/test_distributed_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_distributed_checkpoint_1.1_6b60b17b2838d854_.log 2024-08-06T20:45:44.9004864Z Running 2 items in this shard: test/distributed/fsdp/test_distributed_checkpoint.py::TestDistributedCheckpoint::test_distributed_checkpoint_state_dict_type0, test/distributed/fsdp/test_distributed_checkpoint.py::TestDistributedCheckpoint::test_distributed_checkpoint_state_dict_type1 2024-08-06T20:45:44.9006312Z 2024-08-06T20:45:44.9007945Z Running distributed/_composable/fsdp/test_fully_shard_autograd 1/1 ... [2024-08-06 20:45:44.900547] 2024-08-06T20:45:44.9011862Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_autograd.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:45:44.900975] 2024-08-06T20:45:47.9256590Z 2024-08-06T20:45:47.9257839Z distributed/_composable/fsdp/test_fully_shard_autograd 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_autograd_1.1_ae2256a3cc7ac89e_.log 2024-08-06T20:45:47.9261698Z Running 5 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_autograd.py::TestFullyShardAutograd::test_nontensor_activations, test/distributed/_composable/fsdp/test_fully_shard_autograd.py::TestFullyShardAutograd::test_unused_forward_module, test/distributed/_composable/fsdp/test_fully_shard_autograd.py::TestFullyShardAutograd::test_unused_forward_output, test/distributed/_composable/fsdp/test_fully_shard_autograd.py::TestFullyShardPostAccGradHookMultiThread::test_post_acc_grad_hook_runs, test/distributed/_composable/fsdp/test_fully_shard_autograd.py::TestFullyShardPostAccGradHookMultiProcess::test_post_acc_grad_hook_optim_parity 2024-08-06T20:45:47.9264693Z 2024-08-06T20:45:47.9264925Z Running distributed/test_fake_pg 1/1 ... [2024-08-06 20:45:47.926082] 2024-08-06T20:45:47.9267261Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_fake_pg.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:45:47.926514] 2024-08-06T20:45:50.9512868Z 2024-08-06T20:45:50.9514533Z distributed/test_fake_pg 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_fake_pg_1.1_10098d87632f8245_.log 2024-08-06T20:45:50.9522342Z Running 13 items in this shard: test/distributed/test_fake_pg.py::TestFakePG::test_all_reduce, test/distributed/test_fake_pg.py::TestFakePG::test_allgather, test/distributed/test_fake_pg.py::TestFakePG::test_alltoall, test/distributed/test_fake_pg.py::TestFakePG::test_alltoall_base, test/distributed/test_fake_pg.py::TestFakePG::test_broadcast, test/distributed/test_fake_pg.py::TestFakePG::test_construct_fsdp, test/distributed/test_fake_pg.py::TestFakePG::test_fake_pg_tracing, test/distributed/test_fake_pg.py::TestFakePG::test_fsdp_fake_e2e, test/distributed/test_fake_pg.py::TestFakePG::test_fsdp_tp_fake_e2e, test/distributed/test_fake_pg.py::TestFakePG::test_recv, test/distributed/test_fake_pg.py::TestFakePG::test_reduce_scatter, test/distributed/test_fake_pg.py::TestFakePG::test_scatter, test/distributed/test_fake_pg.py::TestFakePG::test_send 2024-08-06T20:45:50.9528236Z 2024-08-06T20:45:50.9528750Z Running distributed/rpc/test_share_memory 1/1 ... [2024-08-06 20:45:50.952021] 2024-08-06T20:45:50.9531066Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_share_memory.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:45:50.952585] 2024-08-06T20:46:00.1097239Z 2024-08-06T20:46:00.1098890Z distributed/rpc/test_share_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_share_memory_1.1_1021956ed30bb8bc_.log 2024-08-06T20:46:00.1101448Z Running 1 items in this shard: test/distributed/rpc/test_share_memory.py::TestRPCPickler::test_case 2024-08-06T20:46:00.1104757Z Running 1 items in this shard: test/distributed/rpc/test_share_memory.py::TestRPCPickler::test_case 2024-08-06T20:46:00.1105614Z 2024-08-06T20:46:00.1106132Z Running distributed/fsdp/test_fsdp_state_dict 1/1 ... [2024-08-06 20:46:00.110365] 2024-08-06T20:46:00.1113344Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:46:00.110983] 2024-08-06T20:46:03.6370778Z 2024-08-06T20:46:03.6372453Z distributed/fsdp/test_fsdp_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_state_dict_1.1_bd1a52c3254747d0_.log 2024-08-06T20:46:03.6513773Z Running 179 items in this shard: test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_keys_state_dict_type_local_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_keys_state_dict_type_sharded_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_keys_state_dict_type_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_full_state_dict_missing_unexpected_keys_cleaned, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_local_state_dict_with_empty_ranks, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_False_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_False_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_True_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_True_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_False_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_False_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_True_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_True_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_False_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_False_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_True_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_True_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_sharded_load_multi_backend_pg, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_shared_module_and_shared_parameter, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_False_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_False_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_True_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_True_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_False_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_False_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_True_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_True_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_rank0_offload_save_load_flow_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_rank0_offload_save_load_flow_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_save_load_flow_state_dict_type_local_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_save_load_flow_state_dict_type_sharded_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_save_load_flow_state_dict_type_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_skip_module_state_dict_type_local_state_dict_double_nest_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_skip_module_state_dict_type_sharded_state_dict_double_nest_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_skip_module_state_dict_type_state_dict_double_nest_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_type, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_sharded_state_dict_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_sharded_state_dict_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_state_dict_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_state_dict_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_shared_parameters_state_dict_type_local_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_shared_parameters_state_dict_type_sharded_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_shared_parameters_state_dict_type_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_torch_save_load, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_world_size_one, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_wrong_state_dict_config, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict4GPUs::test_local_state_dict_reshard 2024-08-06T20:46:03.6651777Z 2024-08-06T20:46:03.6652062Z Running distributed/test_distributed_spawn 1/3 ... [2024-08-06 20:46:03.638186] 2024-08-06T20:46:03.6652604Z MPI not available -- MPI backend tests will be skipped 2024-08-06T20:46:03.6653096Z Running distributed tests for the test backend with env init_method 2024-08-06T20:46:03.6654600Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=1', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:46:03.646932] 2024-08-06T20:46:06.5518570Z 2024-08-06T20:46:06.5519673Z distributed/test_distributed_spawn 1/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_1.3_142bb915fb450761_.log 2024-08-06T20:46:06.5520595Z Running 0 items in this shard: 2024-08-06T20:46:06.5520791Z 2024-08-06T20:46:06.5528057Z Running distributed tests for the test backend with file init_method 2024-08-06T20:46:06.5534189Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=1', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:46:06.553181] 2024-08-06T20:46:09.4750024Z 2024-08-06T20:46:09.4751204Z distributed/test_distributed_spawn 1/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_1.3_c5881492f08eb59d_.log 2024-08-06T20:46:09.4752148Z Running 0 items in this shard: 2024-08-06T20:46:09.4752349Z 2024-08-06T20:46:09.4759201Z Running distributed tests for the gloo backend with env init_method 2024-08-06T20:46:09.4765376Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=1', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:46:09.476295] 2024-08-06T20:55:44.9365485Z 2024-08-06T20:55:44.9366599Z distributed/test_distributed_spawn 1/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_1.3_486eb39e438c3b6d_.log 2024-08-06T20:55:44.9429190Z Running 102 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_1_level_hierarchical_model_averager_equivalent_to_periodic_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_3_level_hierarchical_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_Backend_enum_class, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_2D_Input, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Channels_Last, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_Running_Value, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_No_Affine, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_requires_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_cat_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_default_pg, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max_complex_unsupported, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo_tags, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_mixed_backend_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_list_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_object_list, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_with_logger, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping_requires_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_compile_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_create_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_forward_backward_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_pickling_powerSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_ignore_params_arg, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_cpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_ignored_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_execution_trace, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_python_error_logged, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_shared_grad_acc_unused_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_module_states, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_join_disable, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs_stop_iteration_sync_bn, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_default_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_no_find_unused, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_data_parallel_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_future, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_invalid_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_irecv, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_failure_order, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_init_abort, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_world_size_not_divisible_by_group_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_tuple_module, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_api_cpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_without_logger 2024-08-06T20:55:44.9475653Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_1_level_hierarchical_model_averager_equivalent_to_periodic_model_averager 2024-08-06T20:55:44.9476969Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_3_level_hierarchical_model_averager 2024-08-06T20:55:44.9478042Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_Backend_enum_class 2024-08-06T20:55:44.9479079Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU 2024-08-06T20:55:44.9480245Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_2D_Input 2024-08-06T20:55:44.9481523Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Channels_Last 2024-08-06T20:55:44.9482894Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_Running_Value 2024-08-06T20:55:44.9484233Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_No_Affine 2024-08-06T20:55:44.9485456Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_requires_grad 2024-08-06T20:55:44.9486597Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync 2024-08-06T20:55:44.9487749Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_hook 2024-08-06T20:55:44.9488938Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_grad_is_view 2024-08-06T20:55:44.9490032Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda 2024-08-06T20:55:44.9491074Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_cat_tensor_cuda 2024-08-06T20:55:44.9492182Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_default_pg 2024-08-06T20:55:44.9493244Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_subgroup 2024-08-06T20:55:44.9494457Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_max 2024-08-06T20:55:44.9495603Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_min 2024-08-06T20:55:44.9496751Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_product 2024-08-06T20:55:44.9497913Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_sum 2024-08-06T20:55:44.9499137Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_max 2024-08-06T20:55:44.9500253Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_min 2024-08-06T20:55:44.9501383Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max 2024-08-06T20:55:44.9502531Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max_complex_unsupported 2024-08-06T20:55:44.9514883Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_min 2024-08-06T20:55:44.9516372Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_max 2024-08-06T20:55:44.9517761Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_min 2024-08-06T20:55:44.9518830Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_max 2024-08-06T20:55:44.9519828Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_max 2024-08-06T20:55:44.9520798Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_product 2024-08-06T20:55:44.9521782Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum 2024-08-06T20:55:44.9522765Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_async 2024-08-06T20:55:44.9523803Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_async 2024-08-06T20:55:44.9524829Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group 2024-08-06T20:55:44.9525870Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group_cuda 2024-08-06T20:55:44.9526893Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group 2024-08-06T20:55:44.9527927Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split 2024-08-06T20:55:44.9529099Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda_complex 2024-08-06T20:55:44.9530495Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group 2024-08-06T20:55:44.9531720Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group_cuda 2024-08-06T20:55:44.9533111Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group 2024-08-06T20:55:44.9534308Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group_cuda 2024-08-06T20:55:44.9535408Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_full_group 2024-08-06T20:55:44.9536365Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier 2024-08-06T20:55:44.9537277Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_cuda 2024-08-06T20:55:44.9538267Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group_cuda 2024-08-06T20:55:44.9539385Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_group 2024-08-06T20:55:44.9540522Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo_tags 2024-08-06T20:55:44.9541624Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_mixed_backend_err 2024-08-06T20:55:44.9542753Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_list_err 2024-08-06T20:55:44.9543769Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast 2024-08-06T20:55:44.9544732Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_object_list 2024-08-06T20:55:44.9545746Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager 2024-08-06T20:55:44.9546771Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager_async 2024-08-06T20:55:44.9547958Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_with_logger 2024-08-06T20:55:44.9549196Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping 2024-08-06T20:55:44.9550411Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping_requires_grad 2024-08-06T20:55:44.9551576Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_compile_static_graph 2024-08-06T20:55:44.9552594Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_create_graph 2024-08-06T20:55:44.9553604Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_forward_backward_hook 2024-08-06T20:55:44.9554669Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_pickling_powerSGD 2024-08-06T20:55:44.9555714Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_ignore_params_arg 2024-08-06T20:55:44.9556728Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_cpu 2024-08-06T20:55:44.9557826Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_ignored_params 2024-08-06T20:55:44.9558986Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_execution_trace 2024-08-06T20:55:44.9560056Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_python_error_logged 2024-08-06T20:55:44.9561198Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_shared_grad_acc_unused_params 2024-08-06T20:55:44.9562290Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_module_states 2024-08-06T20:55:44.9563346Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_join_disable 2024-08-06T20:55:44.9564386Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs 2024-08-06T20:55:44.9565466Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs_stop_iteration_sync_bn 2024-08-06T20:55:44.9566714Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_default_group 2024-08-06T20:55:44.9567895Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_no_find_unused 2024-08-06T20:55:44.9568982Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_full_group 2024-08-06T20:55:44.9569929Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_group 2024-08-06T20:55:44.9570932Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_data_parallel_params 2024-08-06T20:55:44.9571919Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_future 2024-08-06T20:55:44.9572828Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank 2024-08-06T20:55:44.9573789Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_invalid_static_graph 2024-08-06T20:55:44.9574747Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_irecv 2024-08-06T20:55:44.9575711Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_autograd_profiler 2024-08-06T20:55:44.9576775Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_failure_order 2024-08-06T20:55:44.9577849Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo 2024-08-06T20:55:44.9578914Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_subgroup 2024-08-06T20:55:44.9580012Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allreduce 2024-08-06T20:55:44.9581111Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_init_abort 2024-08-06T20:55:44.9582074Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups 2024-08-06T20:55:44.9583195Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_world_size_not_divisible_by_group_size 2024-08-06T20:55:44.9584393Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_tuple_module 2024-08-06T20:55:44.9585498Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_product 2024-08-06T20:55:44.9586524Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_min 2024-08-06T20:55:44.9587554Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_product 2024-08-06T20:55:44.9588487Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum 2024-08-06T20:55:44.9589456Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda 2024-08-06T20:55:44.9590403Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda 2024-08-06T20:55:44.9591352Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl 2024-08-06T20:55:44.9592393Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_autograd_profiler 2024-08-06T20:55:44.9593548Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_torch_profiler 2024-08-06T20:55:44.9594631Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_torch_profiler 2024-08-06T20:55:44.9595726Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_autograd_profiler 2024-08-06T20:55:44.9596804Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_api_cpu 2024-08-06T20:55:44.9597885Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_without_logger 2024-08-06T20:55:44.9598542Z 2024-08-06T20:55:44.9598763Z Running distributed tests for the gloo backend with file init_method 2024-08-06T20:55:44.9600044Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=1', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 20:55:44.941480] 2024-08-06T21:05:20.6199850Z 2024-08-06T21:05:20.6201041Z distributed/test_distributed_spawn 1/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_1.3_1d4aefa22ac59310_.log 2024-08-06T21:05:20.6265410Z Running 102 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_1_level_hierarchical_model_averager_equivalent_to_periodic_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_3_level_hierarchical_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_Backend_enum_class, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_2D_Input, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Channels_Last, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_Running_Value, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_No_Affine, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_requires_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_cat_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_default_pg, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max_complex_unsupported, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo_tags, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_mixed_backend_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_list_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_object_list, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_with_logger, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping_requires_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_compile_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_create_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_forward_backward_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_pickling_powerSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_ignore_params_arg, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_cpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_ignored_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_execution_trace, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_python_error_logged, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_shared_grad_acc_unused_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_module_states, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_join_disable, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs_stop_iteration_sync_bn, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_default_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_no_find_unused, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_data_parallel_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_future, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_invalid_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_irecv, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_failure_order, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_init_abort, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_world_size_not_divisible_by_group_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_tuple_module, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_api_cpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_without_logger 2024-08-06T21:05:20.6311522Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_1_level_hierarchical_model_averager_equivalent_to_periodic_model_averager 2024-08-06T21:05:20.6312903Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_3_level_hierarchical_model_averager 2024-08-06T21:05:20.6313987Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_Backend_enum_class 2024-08-06T21:05:20.6315018Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU 2024-08-06T21:05:20.6316192Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_2D_Input 2024-08-06T21:05:20.6317520Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Channels_Last 2024-08-06T21:05:20.6319223Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_Running_Value 2024-08-06T21:05:20.6320888Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_No_Affine 2024-08-06T21:05:20.6322181Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_requires_grad 2024-08-06T21:05:20.6323323Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync 2024-08-06T21:05:20.6324474Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_hook 2024-08-06T21:05:20.6325673Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_grad_is_view 2024-08-06T21:05:20.6326764Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda 2024-08-06T21:05:20.6328092Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_cat_tensor_cuda 2024-08-06T21:05:20.6329259Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_default_pg 2024-08-06T21:05:20.6330489Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_subgroup 2024-08-06T21:05:20.6331605Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_max 2024-08-06T21:05:20.6332765Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_min 2024-08-06T21:05:20.6333919Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_product 2024-08-06T21:05:20.6335243Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_sum 2024-08-06T21:05:20.6336375Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_max 2024-08-06T21:05:20.6337492Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_min 2024-08-06T21:05:20.6338557Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max 2024-08-06T21:05:20.6339698Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max_complex_unsupported 2024-08-06T21:05:20.6341014Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_min 2024-08-06T21:05:20.6342077Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_max 2024-08-06T21:05:20.6343185Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_min 2024-08-06T21:05:20.6344212Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_max 2024-08-06T21:05:20.6345195Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_max 2024-08-06T21:05:20.6346169Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_product 2024-08-06T21:05:20.6347132Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum 2024-08-06T21:05:20.6348111Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_async 2024-08-06T21:05:20.6349144Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_async 2024-08-06T21:05:20.6350166Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group 2024-08-06T21:05:20.6351196Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group_cuda 2024-08-06T21:05:20.6352212Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group 2024-08-06T21:05:20.6353260Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split 2024-08-06T21:05:20.6354397Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda_complex 2024-08-06T21:05:20.6355602Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group 2024-08-06T21:05:20.6356804Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group_cuda 2024-08-06T21:05:20.6358024Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group 2024-08-06T21:05:20.6359217Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group_cuda 2024-08-06T21:05:20.6360324Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_full_group 2024-08-06T21:05:20.6361275Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier 2024-08-06T21:05:20.6362258Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_cuda 2024-08-06T21:05:20.6363242Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group_cuda 2024-08-06T21:05:20.6364280Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_group 2024-08-06T21:05:20.6365327Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo_tags 2024-08-06T21:05:20.6366425Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_mixed_backend_err 2024-08-06T21:05:20.6367604Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_list_err 2024-08-06T21:05:20.6368618Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast 2024-08-06T21:05:20.6369588Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_object_list 2024-08-06T21:05:20.6370580Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager 2024-08-06T21:05:20.6371597Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager_async 2024-08-06T21:05:20.6372785Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_with_logger 2024-08-06T21:05:20.6374028Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping 2024-08-06T21:05:20.6375255Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping_requires_grad 2024-08-06T21:05:20.6376423Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_compile_static_graph 2024-08-06T21:05:20.6377442Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_create_graph 2024-08-06T21:05:20.6378449Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_forward_backward_hook 2024-08-06T21:05:20.6379510Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_pickling_powerSGD 2024-08-06T21:05:20.6380553Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_ignore_params_arg 2024-08-06T21:05:20.6381662Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_cpu 2024-08-06T21:05:20.6382760Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_ignored_params 2024-08-06T21:05:20.6383918Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_execution_trace 2024-08-06T21:05:20.6385090Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_python_error_logged 2024-08-06T21:05:20.6386178Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_shared_grad_acc_unused_params 2024-08-06T21:05:20.6387249Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_module_states 2024-08-06T21:05:20.6388314Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_join_disable 2024-08-06T21:05:20.6389429Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs 2024-08-06T21:05:20.6390505Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs_stop_iteration_sync_bn 2024-08-06T21:05:20.6391688Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_default_group 2024-08-06T21:05:20.6392864Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_no_find_unused 2024-08-06T21:05:20.6393954Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_full_group 2024-08-06T21:05:20.6394963Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_group 2024-08-06T21:05:20.6395958Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_data_parallel_params 2024-08-06T21:05:20.6396941Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_future 2024-08-06T21:05:20.6397863Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank 2024-08-06T21:05:20.6398813Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_invalid_static_graph 2024-08-06T21:05:20.6399765Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_irecv 2024-08-06T21:05:20.6400729Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_autograd_profiler 2024-08-06T21:05:20.6401795Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_failure_order 2024-08-06T21:05:20.6402866Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo 2024-08-06T21:05:20.6403937Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_subgroup 2024-08-06T21:05:20.6405034Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allreduce 2024-08-06T21:05:20.6406045Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_init_abort 2024-08-06T21:05:20.6407012Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups 2024-08-06T21:05:20.6408134Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_world_size_not_divisible_by_group_size 2024-08-06T21:05:20.6409349Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_tuple_module 2024-08-06T21:05:20.6410438Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_product 2024-08-06T21:05:20.6411454Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_min 2024-08-06T21:05:20.6412416Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_product 2024-08-06T21:05:20.6413341Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum 2024-08-06T21:05:20.6414288Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda 2024-08-06T21:05:20.6415238Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda 2024-08-06T21:05:20.6416257Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl 2024-08-06T21:05:20.6417284Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_autograd_profiler 2024-08-06T21:05:20.6418389Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_torch_profiler 2024-08-06T21:05:20.6419456Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_torch_profiler 2024-08-06T21:05:20.6420544Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_autograd_profiler 2024-08-06T21:05:20.6421762Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_api_cpu 2024-08-06T21:05:20.6422888Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_without_logger 2024-08-06T21:05:20.6423536Z 2024-08-06T21:05:20.6423802Z Running distributed/test_distributed_spawn 3/3 ... [2024-08-06 21:05:20.624896] 2024-08-06T21:05:20.6424376Z MPI not available -- MPI backend tests will be skipped 2024-08-06T21:05:20.6424872Z Running distributed tests for the test backend with env init_method 2024-08-06T21:05:20.6426148Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=3', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:05:20.632671] 2024-08-06T21:05:23.5623066Z 2024-08-06T21:05:23.5624489Z distributed/test_distributed_spawn 3/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_3.3_32afe8d42291c555_.log 2024-08-06T21:05:23.5625471Z Running 0 items in this shard: 2024-08-06T21:05:23.5625666Z 2024-08-06T21:05:23.5632491Z Running distributed tests for the test backend with file init_method 2024-08-06T21:05:23.5638181Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=3', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:05:23.563595] 2024-08-06T21:05:26.4685291Z 2024-08-06T21:05:26.4686505Z distributed/test_distributed_spawn 3/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_3.3_ee25328a19dc1ca0_.log 2024-08-06T21:05:26.4687438Z Running 0 items in this shard: 2024-08-06T21:05:26.4687645Z 2024-08-06T21:05:26.4693890Z Running distributed tests for the gloo backend with env init_method 2024-08-06T21:05:26.4700450Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=3', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:05:26.469815] 2024-08-06T21:14:27.4785348Z 2024-08-06T21:14:27.4787083Z distributed/test_distributed_spawn 3/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_3.3_69f3bbcd9b4f3377_.log 2024-08-06T21:14:27.4846520Z Running 91 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_gradient, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_non_default_stream, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_with_amp_and_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_stack_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_average_parameters, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_no_rank_zero_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_ring_exchange_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_self_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_without_logger, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_grad_as_bucket_view_false, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer_via_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_different_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_same_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce_process_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_post_localSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_gpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_num_params_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_shape_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_error, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_namedtuple, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_set_grad_to_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_no_set_grad_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_remove_autograd_hooks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_returns_tensor_with_no_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_static_graph_nested_types, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_grad_undefined, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_new_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_different_graph_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_checks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_wait_all_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allgather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_reduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_high_priority_stream, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_negative_input_rank, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_group_size_exceeds_world_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager_param_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda_twice, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_checks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_object_list, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_multi_forward, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_undefined_grad_parity_unused_parameters, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_with_logger 2024-08-06T21:14:27.4888046Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU_grad_is_view 2024-08-06T21:14:27.4889366Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_gradient 2024-08-06T21:14:27.4890743Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_non_default_stream 2024-08-06T21:14:27.4891987Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_with_amp_and_grad_is_view 2024-08-06T21:14:27.4893105Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather 2024-08-06T21:14:27.4894128Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_complex 2024-08-06T21:14:27.4895212Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_group 2024-08-06T21:14:27.4896244Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_full_group 2024-08-06T21:14:27.4897261Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_group 2024-08-06T21:14:27.4898304Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_stack_tensor_cuda 2024-08-06T21:14:27.4899450Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_product 2024-08-06T21:14:27.4900646Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_sum 2024-08-06T21:14:27.4901730Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_sum 2024-08-06T21:14:27.4902791Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_product 2024-08-06T21:14:27.4903864Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_product 2024-08-06T21:14:27.4904898Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_sum 2024-08-06T21:14:27.4905875Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_min 2024-08-06T21:14:27.4906899Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_complex 2024-08-06T21:14:27.4907949Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda_complex 2024-08-06T21:14:27.4909100Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group_cuda 2024-08-06T21:14:27.4910180Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group 2024-08-06T21:14:27.4911356Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group_cuda 2024-08-06T21:14:27.4912533Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda 2024-08-06T21:14:27.4913609Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_average_parameters 2024-08-06T21:14:27.4915510Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group 2024-08-06T21:14:27.4916522Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group_cuda 2024-08-06T21:14:27.4917563Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_full_group 2024-08-06T21:14:27.4918601Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo 2024-08-06T21:14:27.4919688Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_no_rank_zero_nccl 2024-08-06T21:14:27.4920840Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_ring_exchange_nccl 2024-08-06T21:14:27.4921963Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_self_nccl 2024-08-06T21:14:27.4923003Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_full_group 2024-08-06T21:14:27.4924183Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_without_logger 2024-08-06T21:14:27.4925512Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_grad_as_bucket_view_false 2024-08-06T21:14:27.4926700Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer_via_hook 2024-08-06T21:14:27.4927769Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce 2024-08-06T21:14:27.4928891Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_different_across_ranks 2024-08-06T21:14:27.4930048Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_same_across_ranks 2024-08-06T21:14:27.4931346Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device 2024-08-06T21:14:27.4932392Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce_process_group 2024-08-06T21:14:27.4933532Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_post_localSGD 2024-08-06T21:14:27.4934586Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_gpu 2024-08-06T21:14:27.4935673Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_num_params_across_ranks 2024-08-06T21:14:27.4936986Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_shape_across_ranks 2024-08-06T21:14:27.4938148Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_error 2024-08-06T21:14:27.4939225Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_namedtuple 2024-08-06T21:14:27.4940382Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_set_grad_to_none 2024-08-06T21:14:27.4941820Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_no_set_grad_none 2024-08-06T21:14:27.4943126Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd 2024-08-06T21:14:27.4944220Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd_static_graph 2024-08-06T21:14:27.4945306Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_remove_autograd_hooks 2024-08-06T21:14:27.4946398Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_returns_tensor_with_no_grad 2024-08-06T21:14:27.4947503Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_static_graph_nested_types 2024-08-06T21:14:27.4948641Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_grad_undefined 2024-08-06T21:14:27.4949804Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_new_group 2024-08-06T21:14:27.4950921Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_different_graph_across_ranks 2024-08-06T21:14:27.4951917Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather 2024-08-06T21:14:27.4952830Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_checks 2024-08-06T21:14:27.4953784Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object 2024-08-06T21:14:27.4954764Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_group 2024-08-06T21:14:27.4955828Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang 2024-08-06T21:14:27.4956938Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_wait_all_ranks 2024-08-06T21:14:27.4958034Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allgather 2024-08-06T21:14:27.4959087Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_reduce 2024-08-06T21:14:27.4960141Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_high_priority_stream 2024-08-06T21:14:27.4961289Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_negative_input_rank 2024-08-06T21:14:27.4962521Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_group_size_exceeds_world_size 2024-08-06T21:14:27.4963711Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager_param_group 2024-08-06T21:14:27.4964903Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity 2024-08-06T21:14:27.4966058Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_grad_is_view 2024-08-06T21:14:27.4967360Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd_grad_is_view 2024-08-06T21:14:27.4968578Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_min 2024-08-06T21:14:27.4969581Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_sum 2024-08-06T21:14:27.4970642Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_max 2024-08-06T21:14:27.4971596Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_min 2024-08-06T21:14:27.4972593Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_tensor_cuda 2024-08-06T21:14:27.4973620Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda_twice 2024-08-06T21:14:27.4974587Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter 2024-08-06T21:14:27.4975521Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_checks 2024-08-06T21:14:27.4976489Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_complex 2024-08-06T21:14:27.4977461Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda_complex 2024-08-06T21:14:27.4978461Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_full_group 2024-08-06T21:14:27.4979456Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_object_list 2024-08-06T21:14:27.4980408Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv 2024-08-06T21:14:27.4981544Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_autograd_profiler 2024-08-06T21:14:27.4982628Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag 2024-08-06T21:14:27.4983695Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_torch_profiler 2024-08-06T21:14:27.4984783Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum_cuda 2024-08-06T21:14:27.4985849Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_multi_forward 2024-08-06T21:14:27.4986975Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_undefined_grad_parity_unused_parameters 2024-08-06T21:14:27.4988148Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_with_logger 2024-08-06T21:14:27.4988785Z 2024-08-06T21:14:27.4989006Z Running distributed tests for the gloo backend with file init_method 2024-08-06T21:14:27.4990293Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=3', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:14:27.483221] 2024-08-06T21:23:32.1177080Z 2024-08-06T21:23:32.1178213Z distributed/test_distributed_spawn 3/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_3.3_f79194f40653068b_.log 2024-08-06T21:23:32.1234259Z Running 91 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_gradient, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_non_default_stream, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_with_amp_and_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_stack_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_average_parameters, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_no_rank_zero_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_ring_exchange_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_self_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_without_logger, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_grad_as_bucket_view_false, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer_via_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_different_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_same_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce_process_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_post_localSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_gpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_num_params_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_shape_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_error, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_namedtuple, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_set_grad_to_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_no_set_grad_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_remove_autograd_hooks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_returns_tensor_with_no_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_static_graph_nested_types, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_grad_undefined, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_new_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_different_graph_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_checks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_wait_all_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allgather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_reduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_high_priority_stream, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_negative_input_rank, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_group_size_exceeds_world_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager_param_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda_twice, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_checks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_object_list, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_multi_forward, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_undefined_grad_parity_unused_parameters, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_with_logger 2024-08-06T21:23:32.1276065Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU_grad_is_view 2024-08-06T21:23:32.1277391Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_gradient 2024-08-06T21:23:32.1278752Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_non_default_stream 2024-08-06T21:23:32.1280009Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_with_amp_and_grad_is_view 2024-08-06T21:23:32.1281105Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather 2024-08-06T21:23:32.1282112Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_complex 2024-08-06T21:23:32.1283198Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_group 2024-08-06T21:23:32.1284236Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_full_group 2024-08-06T21:23:32.1285229Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_group 2024-08-06T21:23:32.1286281Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_stack_tensor_cuda 2024-08-06T21:23:32.1287413Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_product 2024-08-06T21:23:32.1288526Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_sum 2024-08-06T21:23:32.1289605Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_sum 2024-08-06T21:23:32.1290672Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_product 2024-08-06T21:23:32.1291816Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_product 2024-08-06T21:23:32.1292832Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_sum 2024-08-06T21:23:32.1293811Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_min 2024-08-06T21:23:32.1294824Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_complex 2024-08-06T21:23:32.1295860Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda_complex 2024-08-06T21:23:32.1296967Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group_cuda 2024-08-06T21:23:32.1298056Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group 2024-08-06T21:23:32.1299223Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group_cuda 2024-08-06T21:23:32.1300442Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda 2024-08-06T21:23:32.1301519Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_average_parameters 2024-08-06T21:23:32.1302514Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group 2024-08-06T21:23:32.1303511Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group_cuda 2024-08-06T21:23:32.1304534Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_full_group 2024-08-06T21:23:32.1305580Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo 2024-08-06T21:23:32.1306673Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_no_rank_zero_nccl 2024-08-06T21:23:32.1307813Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_ring_exchange_nccl 2024-08-06T21:23:32.1308929Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_self_nccl 2024-08-06T21:23:32.1309977Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_full_group 2024-08-06T21:23:32.1311156Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_without_logger 2024-08-06T21:23:32.1312463Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_grad_as_bucket_view_false 2024-08-06T21:23:32.1313654Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer_via_hook 2024-08-06T21:23:32.1314732Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce 2024-08-06T21:23:32.1315853Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_different_across_ranks 2024-08-06T21:23:32.1317004Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_same_across_ranks 2024-08-06T21:23:32.1318116Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device 2024-08-06T21:23:32.1319264Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce_process_group 2024-08-06T21:23:32.1320407Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_post_localSGD 2024-08-06T21:23:32.1321454Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_gpu 2024-08-06T21:23:32.1322549Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_num_params_across_ranks 2024-08-06T21:23:32.1323776Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_shape_across_ranks 2024-08-06T21:23:32.1324924Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_error 2024-08-06T21:23:32.1326004Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_namedtuple 2024-08-06T21:23:32.1327170Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_set_grad_to_none 2024-08-06T21:23:32.1328549Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_no_set_grad_none 2024-08-06T21:23:32.1329752Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd 2024-08-06T21:23:32.1331010Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd_static_graph 2024-08-06T21:23:32.1332116Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_remove_autograd_hooks 2024-08-06T21:23:32.1333206Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_returns_tensor_with_no_grad 2024-08-06T21:23:32.1343900Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_static_graph_nested_types 2024-08-06T21:23:32.1345072Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_grad_undefined 2024-08-06T21:23:32.1346251Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_new_group 2024-08-06T21:23:32.1347398Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_different_graph_across_ranks 2024-08-06T21:23:32.1348382Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather 2024-08-06T21:23:32.1349320Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_checks 2024-08-06T21:23:32.1350270Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object 2024-08-06T21:23:32.1351235Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_group 2024-08-06T21:23:32.1352310Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang 2024-08-06T21:23:32.1353433Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_wait_all_ranks 2024-08-06T21:23:32.1354531Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allgather 2024-08-06T21:23:32.1355759Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_reduce 2024-08-06T21:23:32.1356822Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_high_priority_stream 2024-08-06T21:23:32.1357984Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_negative_input_rank 2024-08-06T21:23:32.1359231Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_group_size_exceeds_world_size 2024-08-06T21:23:32.1360414Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager_param_group 2024-08-06T21:23:32.1361682Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity 2024-08-06T21:23:32.1362849Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_grad_is_view 2024-08-06T21:23:32.1364158Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd_grad_is_view 2024-08-06T21:23:32.1365363Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_min 2024-08-06T21:23:32.1366391Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_sum 2024-08-06T21:23:32.1367393Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_max 2024-08-06T21:23:32.1368338Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_min 2024-08-06T21:23:32.1369336Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_tensor_cuda 2024-08-06T21:23:32.1370379Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda_twice 2024-08-06T21:23:32.1371341Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter 2024-08-06T21:23:32.1372262Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_checks 2024-08-06T21:23:32.1373224Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_complex 2024-08-06T21:23:32.1374213Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda_complex 2024-08-06T21:23:32.1375215Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_full_group 2024-08-06T21:23:32.1376202Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_object_list 2024-08-06T21:23:32.1377165Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv 2024-08-06T21:23:32.1378210Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_autograd_profiler 2024-08-06T21:23:32.1379279Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag 2024-08-06T21:23:32.1380341Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_torch_profiler 2024-08-06T21:23:32.1381532Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum_cuda 2024-08-06T21:23:32.1382667Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_multi_forward 2024-08-06T21:23:32.1383782Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_undefined_grad_parity_unused_parameters 2024-08-06T21:23:32.1384954Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_with_logger 2024-08-06T21:23:32.1385587Z 2024-08-06T21:23:32.1385848Z Running distributed/fsdp/test_fsdp_memory 1/1 ... [2024-08-06 21:23:32.122103] 2024-08-06T21:23:32.1387109Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_memory.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:23:32.122513] 2024-08-06T21:23:40.7068331Z 2024-08-06T21:23:40.7070188Z distributed/fsdp/test_fsdp_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_memory_1.1_27ed7f45266bb41b_.log 2024-08-06T21:23:40.7073200Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_memory.py::TestFSDPMemory::test_fsdp_memory_ckpt_ckpt, test/distributed/fsdp/test_fsdp_memory.py::TestFSDPMemory::test_fsdp_memory_ckpt_no_ckpt 2024-08-06T21:23:40.7074834Z 2024-08-06T21:23:40.7077595Z Running distributed/_composable/fully_shard/test_fully_shard_runtime 1/1 ... [2024-08-06 21:23:40.707494] 2024-08-06T21:23:40.7083145Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_runtime.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:23:40.708046] 2024-08-06T21:23:43.8337001Z 2024-08-06T21:23:43.8338320Z distributed/_composable/fully_shard/test_fully_shard_runtime 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_runtime_1.1_d03f5ce8749c662e_.log 2024-08-06T21:23:43.8340298Z Running 2 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_runtime.py::TestRuntime::test_training, test/distributed/_composable/fully_shard/test_fully_shard_runtime.py::TestRuntime::test_unshard_reshard_order 2024-08-06T21:23:43.8341387Z 2024-08-06T21:23:43.8343233Z Running distributed/checkpoint/test_file_system_checkpoint_cpu 1/1 ... [2024-08-06 21:23:43.834132] 2024-08-06T21:23:43.8348293Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_file_system_checkpoint_cpu.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:23:43.834591] 2024-08-06T21:24:15.5106656Z 2024-08-06T21:24:15.5108058Z distributed/checkpoint/test_file_system_checkpoint_cpu 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_file_system_checkpoint_cpu_1.1_0f6157fd6de45efd_.log 2024-08-06T21:24:15.5117969Z Running 12 items in this shard: test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoad::test_read_write_only_tensor_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoad::test_read_write_only_tensor_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoadWithSharedTensor::test_read_write_shard_tensor_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoadWithSharedTensor::test_read_write_shard_tensor_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_rowwise_to_colwise_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_rowwise_to_colwise_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_with_different_shard_plan_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_with_different_shard_plan_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_save_load_bytes_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_save_load_bytes_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_switch_between_sharded_tensor_to_tensor_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_switch_between_sharded_tensor_to_tensor_thread_count_2 2024-08-06T21:24:15.5125869Z 2024-08-06T21:24:15.5126106Z Running distributed/test_data_parallel 1/1 ... [2024-08-06 21:24:15.511196] 2024-08-06T21:24:15.5127342Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_data_parallel.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:24:15.511641] 2024-08-06T21:24:18.7370156Z 2024-08-06T21:24:18.7371195Z distributed/test_data_parallel 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_data_parallel_1.1_5daacc3045b7707b_.log 2024-08-06T21:24:18.7391292Z Running 46 items in this shard: test/distributed/test_data_parallel.py::TestDataParallel::test_autocast, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_buffers_requiring_grad, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_complex, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_device_args, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_function_deletion, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_lazy_linear, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_model_device, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_model_no_refcycles, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_module_zero_inputs, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_multiple_input, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_nested_input, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_nested_output, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_no_grad, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_rnn, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_small_back, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_sparse, test/distributed/test_data_parallel.py::TestDataParallel::test_gather_cpu, test/distributed/test_data_parallel.py::TestDataParallel::test_gather_different_len_dicts, test/distributed/test_data_parallel.py::TestDataParallel::test_gather_gpu, test/distributed/test_data_parallel.py::TestDataParallel::test_parallel_apply, test/distributed/test_data_parallel.py::TestDataParallel::test_parallel_apply_autocast, test/distributed/test_data_parallel.py::TestDataParallel::test_parallel_apply_passes_exception, test/distributed/test_data_parallel.py::TestDataParallel::test_parameter_list_dict_replica, test/distributed/test_data_parallel.py::TestDataParallel::test_replicate, test/distributed/test_data_parallel.py::TestDataParallel::test_replicate_buffers, test/distributed/test_data_parallel.py::TestDataParallel::test_save_replica_module, test/distributed/test_data_parallel.py::TestDataParallel::test_scatter_cpu, test/distributed/test_data_parallel.py::TestDataParallel::test_scatter_gpu, test/distributed/test_data_parallel.py::TestDataParallel::test_strided_grad_layout, test/distributed/test_data_parallel.py::TestDataParallel::test_zero_grad, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_cpu_float16, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_cpu_float32, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_cpu_float64, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_cpu_float16, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_cpu_float32, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_cpu_float64, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_dict_cpu_float16, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_dict_cpu_float32, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_dict_cpu_float64, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_list_cpu_float16, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_list_cpu_float32, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_list_cpu_float64, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_tuple_cpu_float16, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_tuple_cpu_float32, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_tuple_cpu_float64 2024-08-06T21:24:18.7410640Z 2024-08-06T21:24:18.7410980Z Running distributed/algorithms/ddp_comm_hooks/test_ddp_hooks 1/1 ... [2024-08-06 21:24:18.737480] 2024-08-06T21:24:18.7412391Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:24:18.737919] 2024-08-06T21:24:21.4626606Z 2024-08-06T21:24:21.4627821Z distributed/algorithms/ddp_comm_hooks/test_ddp_hooks 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.algorithms.ddp_comm_hooks.test_ddp_hooks_1.1_31e38c4cd1dcf708_.log 2024-08-06T21:24:21.4632614Z Running 6 items in this shard: test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_ddp_comm_hook_allreduce_hook, test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_ddp_comm_hook_fp16compress_hook, test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_ddp_comm_hook_noop_hook, test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_ddp_comm_hook_quantize_per_channel_hook, test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_ddp_comm_hook_quantize_per_tensor_hook, test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_is_last_hook 2024-08-06T21:24:21.4636235Z 2024-08-06T21:24:21.4636549Z Running distributed/fsdp/test_fsdp_comm_hooks 1/1 ... [2024-08-06 21:24:21.463135] 2024-08-06T21:24:21.4638749Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_comm_hooks.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:24:21.463590] 2024-08-06T21:25:10.7029914Z 2024-08-06T21:25:10.7031517Z distributed/fsdp/test_fsdp_comm_hooks 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_comm_hooks_1.1_70879fededc8b580_.log 2024-08-06T21:25:10.7049268Z Running 28 items in this shard: test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_False_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_True_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_behavior_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_behavior_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_behavior_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_False_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_True_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_False_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_True_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_hybrid_strategy, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_non_root_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_non_root_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_non_root_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_submodules_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_submodules_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_submodules_sharding_strategy2 2024-08-06T21:25:10.7065661Z 2024-08-06T21:25:10.7066091Z Running distributed/test_multi_threaded_pg 1/1 ... [2024-08-06 21:25:10.703530] 2024-08-06T21:25:10.7067347Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_multi_threaded_pg.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:25:10.704000] 2024-08-06T21:25:13.9795312Z 2024-08-06T21:25:13.9796634Z distributed/test_multi_threaded_pg 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_multi_threaded_pg_1.1_c516d5277bb6b18d_.log 2024-08-06T21:25:13.9807708Z Running 22 items in this shard: test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_all_to_all_single_list, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_all_to_all_single_none, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_all_to_all_single_tensor, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_broadcast_object_list, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_collective_error_on_rank_non_zero, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_collective_error_on_rank_non_zero_all, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_collective_error_on_rank_zero, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_skip, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_reduce, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_reduce_coalesced, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_reduce_ops, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_to_all, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_allgather, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_assert_equal_on_rank, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_broadcast, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_broadcast_object_list, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_bwd_sees_fwd_pg, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_gather, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_reduce_scatter, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_scatter, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_subpg, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_using_pg_from_another_thread 2024-08-06T21:25:13.9817214Z 2024-08-06T21:25:13.9817477Z Running distributed/fsdp/test_fsdp_apply 1/1 ... [2024-08-06 21:25:13.979963] 2024-08-06T21:25:13.9818733Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_apply.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:25:13.980432] 2024-08-06T21:25:25.5699222Z 2024-08-06T21:25:25.5700320Z distributed/fsdp/test_fsdp_apply 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_apply_1.1_64dd6ed9de9e4b6d_.log 2024-08-06T21:25:25.5702254Z Running 3 items in this shard: test/distributed/fsdp/test_fsdp_apply.py::TestApply::test_apply_in_summon_raises_error, test/distributed/fsdp/test_fsdp_apply.py::TestApply::test_nested_module_apply, test/distributed/fsdp/test_fsdp_apply.py::TestApply::test_transformer_module_apply 2024-08-06T21:25:25.5703493Z 2024-08-06T21:25:25.5707608Z Running distributed/fsdp/test_fsdp_mixed_precision 1/1 ... [2024-08-06 21:25:25.570376] 2024-08-06T21:25:25.5711739Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_mixed_precision.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:25:25.570851] 2024-08-06T21:28:04.3453618Z 2024-08-06T21:28:04.3454896Z distributed/fsdp/test_fsdp_mixed_precision 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_mixed_precision_1.1_2e84eaf8cdc62084_.log 2024-08-06T21:28:04.3491553Z Running 57 items in this shard: test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_eval_root_cast_inputs, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_full_precision_in_eval, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_full_precision_in_eval_buffers, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_full_precision_in_eval_comm, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_grads_reduced_precision, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_input_grads_with_param_mixed_precision, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_false_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_false_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_false_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_false_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_true_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_true_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_true_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_true_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_false_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_false_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_false_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_false_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_true_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_true_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_true_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_true_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_false_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_false_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_false_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_false_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_true_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_true_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_true_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_true_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_false_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_false_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_false_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_false_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_true_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_true_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_true_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_true_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_no_reshard_after_forward, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_resnet, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_batchnorm_convert_sync_bn_False, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_batchnorm_convert_sync_bn_True, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_embedding_default, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_embedding_only_params_and_bufs, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_embedding_params_and_reduce_diff, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_embedding_reduce, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionUnsharded::test_grads_reduced_precision, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionUnsharded::test_mixed_precision_e2e_full_shard, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionUnsharded::test_mixed_precision_no_reshard_after_forward, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionIgnoredModules::test_mixed_precision_with_ignored_module, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_float16_on_one_submodule, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_float16_on_one_submodule_skip_inputs, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_float16_on_one_submodule_skip_inputs_error, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_submodules_with_different_precisions, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_submodules_with_different_precisions_error, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_submodules_with_external_inputs, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPTrainEval::test_train_ema_eval_flow 2024-08-06T21:28:04.3526754Z 2024-08-06T21:28:04.3527013Z Running distributed/fsdp/test_fsdp_input 1/1 ... [2024-08-06 21:28:04.346013] 2024-08-06T21:28:04.3528267Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_input.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:28:04.346468] 2024-08-06T21:28:12.9317012Z 2024-08-06T21:28:12.9318272Z distributed/fsdp/test_fsdp_input 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_input_1.1_c7bb88a6f3c33b26_.log 2024-08-06T21:28:12.9320453Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_input.py::TestInput::test_input_type_dict, test/distributed/fsdp/test_fsdp_input.py::TestInput::test_input_type_list 2024-08-06T21:28:12.9321227Z 2024-08-06T21:28:12.9324598Z Running distributed/fsdp/test_fsdp_multiple_wrapping 1/1 ... [2024-08-06 21:28:12.932213] 2024-08-06T21:28:12.9329593Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_multiple_wrapping.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:28:12.932681] 2024-08-06T21:28:19.0163850Z 2024-08-06T21:28:19.0165006Z distributed/fsdp/test_fsdp_multiple_wrapping 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_multiple_wrapping_1.1_7880ba3467299369_.log 2024-08-06T21:28:19.0166401Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_multiple_wrapping.py::TestMultipleWrapping::test_multiple_wrapping 2024-08-06T21:28:19.0166991Z 2024-08-06T21:28:19.0169703Z Running distributed/fsdp/test_fsdp_uneven 1/1 ... [2024-08-06 21:28:19.016789] 2024-08-06T21:28:19.0174492Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_uneven.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:28:19.017230] 2024-08-06T21:28:25.2007053Z 2024-08-06T21:28:25.2008131Z distributed/fsdp/test_fsdp_uneven 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_uneven_1.1_d171228d9ffc6bc6_.log 2024-08-06T21:28:25.2009373Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_uneven.py::TestUnevenParamShard::test_one_iteration 2024-08-06T21:28:25.2009884Z 2024-08-06T21:28:25.2013189Z Running distributed/fsdp/test_fsdp_traversal 1/1 ... [2024-08-06 21:28:25.201148] 2024-08-06T21:28:25.2018173Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_traversal.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:28:25.201608] 2024-08-06T21:28:31.1822038Z 2024-08-06T21:28:31.1823200Z distributed/fsdp/test_fsdp_traversal 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_traversal_1.1_6a677045b368e573_.log 2024-08-06T21:28:31.1824732Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_traversal.py::TestTraversal::test_fsdp_modules 2024-08-06T21:28:31.1825242Z 2024-08-06T21:28:31.1829150Z Running distributed/fsdp/test_fsdp_misc 1/1 ... [2024-08-06 21:28:31.182651] 2024-08-06T21:28:31.1834085Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_misc.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:28:31.183171] 2024-08-06T21:29:13.6706264Z 2024-08-06T21:29:13.6709498Z distributed/fsdp/test_fsdp_misc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_misc_1.1_a2511ddb98bf2c79_.log 2024-08-06T21:29:13.6724138Z Running 28 items in this shard: test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_cpu_init_with_sync_module_states, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_cpu_init_stays_on_cpu, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_cpu_training, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_device_id_use_index_False, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_device_id_use_index_True, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_not_all_outputs_used_in_loss, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_optim_overlap_no_use_orig_params_error, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_optimizer_overlap, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_zero2_eval_with_prefetch, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_cpu_gpu_module, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_device_id_auto_wrap, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_device_id_cpu_offload, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_device_id_no_move_ignored_params_and_bufs, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_ignored_module_meta, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_namedtuple, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_same_model_across_ranks, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_unsupported_module_cls, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_homogeneous_attributes, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_module_device_mismatches_device_id, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_multigpu_module, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_no_params, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscWorldSize1::test_training_device_mismatch_errors, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscWorldSize1::test_unsafe_setattr, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscWorldSize1::test_world_size_1_sharding_strategy_warning 2024-08-06T21:29:13.6736980Z 2024-08-06T21:29:13.6737231Z Running distributed/test_pg_wrapper 1/1 ... [2024-08-06 21:29:13.671179] 2024-08-06T21:29:13.6738671Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_pg_wrapper.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:29:13.671636] 2024-08-06T21:30:35.1321588Z 2024-08-06T21:30:35.1323701Z distributed/test_pg_wrapper 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_pg_wrapper_1.1_24f436cf63cc7737_.log 2024-08-06T21:30:35.1334346Z Running 17 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_coalescing_manager_debug_mode_detail, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_hang, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_detail, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_off, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_debug_level_detail_no_gloo, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_new_group_no_gloo, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_hang, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode_off, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_debug_mode 2024-08-06T21:30:35.1343829Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_coalescing_manager_debug_mode_detail 2024-08-06T21:30:35.1344880Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_hang 2024-08-06T21:30:35.1345942Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_detail 2024-08-06T21:30:35.1347115Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_off 2024-08-06T21:30:35.1348204Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch 2024-08-06T21:30:35.1349275Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch_debug_mode 2024-08-06T21:30:35.1350349Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_debug_level_detail_no_gloo 2024-08-06T21:30:35.1351328Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_new_group_no_gloo 2024-08-06T21:30:35.1352268Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_hang 2024-08-06T21:30:35.1353274Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda 2024-08-06T21:30:35.1354400Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda_debug_mode 2024-08-06T21:30:35.1355690Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode 2024-08-06T21:30:35.1356840Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode_off 2024-08-06T21:30:35.1357935Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch 2024-08-06T21:30:35.1358955Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda 2024-08-06T21:30:35.1360062Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda_debug_mode 2024-08-06T21:30:35.1361388Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_debug_mode 2024-08-06T21:30:35.1361998Z 2024-08-06T21:30:35.1362279Z Running distributed/rpc/test_tensorpipe_agent 2/3 ... [2024-08-06 21:30:35.133107] 2024-08-06T21:30:35.1363612Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_tensorpipe_agent.py', '--shard-id=2', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:30:35.133605] 2024-08-06T21:45:06.8291179Z 2024-08-06T21:45:06.8292922Z distributed/rpc/test_tensorpipe_agent 2/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_tensorpipe_agent_2.3_deda1e86219ddb65_.log 2024-08-06T21:45:06.8373816Z Running 116 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_chained_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_build_rpc_profiling_key, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_multi, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_none, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_dist_init_decorator, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_duplicate_name_2, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_function_not_on_callee, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_in_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_wait_twice, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_get_worker_infos, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_rpc_twice, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_int_callee, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_builtin_remote_ret, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_non_garbage_collected_user_rref_due_to_local_circular_dependency, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_owner_rref_backward, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_async_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_remote_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_instance_method, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_multi_async_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_nested_pickle, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rpc_rref_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_reinit, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_context_debug_info, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_forward_chain, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_get_future, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_slow_init, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_scalar_add, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_add, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_rpc_arg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_send_to_rank, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_shutdown_followed_by_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rref_after_shutdown, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_raise_in_user_func, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_with_partial_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_dense, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_twice_dense, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_worker_info_pickle, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_world_size_one, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_autograd_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_tensor_dims, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_round_trips, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_node_failure, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_nested, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_script_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_self, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_unused_send_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_tensor_with_grad, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_dist_autograd_profiling, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_only_on_return_value_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_python_remote_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_nested_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_rpc_complex_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_none_grads, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_add_done_callback, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_args_and_kwargs_contain_different_types, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote_multi, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_fork_in_jit_with_profiling, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_script_function_that_not_exists_remotely_from_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_chain, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_local_rref_local_value, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_record_function_jit_end_callbacks_with_fork, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_module, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_throw, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_as_arg_and_return, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_functions_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_unexepected_kwarg_is_specified, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_dist_backward, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_get_gradients, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_jit_fork_within_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_restore_context_after_swtich_to_jit_thread, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_sync_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_send_remote_module_with_a_new_attribute_not_pickled_over_the_wire, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_train_eval, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_create_remote_module_from_module_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire_script_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_inside, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_no_ddp, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_builtin_remote_ret_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc_without_rank, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_py_udf_remote_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_my_parameter_server_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_remote_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rref_stress_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rpc_rref_args_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_get_type_timeout_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_proxy_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_py_udf_remote_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_self_remote_arg_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_set_and_get_num_worker_threads, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_no_grad_on_tensor_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_multi_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_self_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_no_grad_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_call_itself_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_multiple_backward_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_nested_backward_accumulate_grads_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_no_graph_with_tensors_not_require_grad_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_trainer_ps_sparse 2024-08-06T21:45:06.8427166Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_chained_remote 2024-08-06T21:45:06.8428226Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise_async 2024-08-06T21:45:06.8429313Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_remote 2024-08-06T21:45:06.8430564Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_build_rpc_profiling_key 2024-08-06T21:45:06.8431544Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_multi 2024-08-06T21:45:06.8432472Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_none 2024-08-06T21:45:06.8433465Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_simple 2024-08-06T21:45:06.8434399Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_dist_init_decorator 2024-08-06T21:45:06.8435353Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_duplicate_name_2 2024-08-06T21:45:06.8436328Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_function_not_on_callee 2024-08-06T21:45:06.8437657Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done 2024-08-06T21:45:06.8438606Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done_exception 2024-08-06T21:45:06.8439564Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_in_rpc 2024-08-06T21:45:06.8440493Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_wait_twice 2024-08-06T21:45:06.8441422Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_get_worker_infos 2024-08-06T21:45:06.8442363Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_rpc_twice 2024-08-06T21:45:06.8443448Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_int_callee 2024-08-06T21:45:06.8444407Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_builtin_remote_ret 2024-08-06T21:45:06.8445371Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_remote 2024-08-06T21:45:06.8446492Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_non_garbage_collected_user_rref_due_to_local_circular_dependency 2024-08-06T21:45:06.8447644Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_owner_rref_backward 2024-08-06T21:45:06.8448843Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin_single_threaded 2024-08-06T21:45:06.8449942Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_udf 2024-08-06T21:45:06.8450985Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_async_rpc 2024-08-06T21:45:06.8452053Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_remote_rpc 2024-08-06T21:45:06.8453122Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin 2024-08-06T21:45:06.8454237Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin_single_threaded 2024-08-06T21:45:06.8455345Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_instance_method 2024-08-06T21:45:06.8456340Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_multi_async_call 2024-08-06T21:45:06.8457306Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_nested_pickle 2024-08-06T21:45:06.8458232Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rpc_rref_args 2024-08-06T21:45:06.8459133Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_reinit 2024-08-06T21:45:06.8460071Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_context_debug_info 2024-08-06T21:45:06.8461104Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_forward_chain 2024-08-06T21:45:06.8462052Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_get_future 2024-08-06T21:45:06.8463002Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_slow_init 2024-08-06T21:45:06.8463937Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_scalar_add 2024-08-06T21:45:06.8464809Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_add 2024-08-06T21:45:06.8465760Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_rpc_arg 2024-08-06T21:45:06.8466727Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_send_to_rank 2024-08-06T21:45:06.8467669Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_shutdown_followed_by_rpc 2024-08-06T21:45:06.8468669Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rref_after_shutdown 2024-08-06T21:45:06.8469738Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_raise_in_user_func 2024-08-06T21:45:06.8470781Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_with_partial_exception 2024-08-06T21:45:06.8471794Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_dense 2024-08-06T21:45:06.8472804Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_twice_dense 2024-08-06T21:45:06.8473796Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_worker_info_pickle 2024-08-06T21:45:06.8474731Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_world_size_one 2024-08-06T21:45:06.8475751Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_autograd_context 2024-08-06T21:45:06.8476853Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_tensor_dims 2024-08-06T21:45:06.8478010Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_round_trips 2024-08-06T21:45:06.8479100Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_node_failure 2024-08-06T21:45:06.8480167Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_nested 2024-08-06T21:45:06.8481264Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_script_call 2024-08-06T21:45:06.8482363Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_self 2024-08-06T21:45:06.8483461Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_unused_send_function 2024-08-06T21:45:06.8484622Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_tensor_with_grad 2024-08-06T21:45:06.8485750Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_dist_autograd_profiling 2024-08-06T21:45:06.8486878Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_only_on_return_value_remote 2024-08-06T21:45:06.8488006Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_call 2024-08-06T21:45:06.8489130Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_python_remote_call 2024-08-06T21:45:06.8490210Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_nested_context 2024-08-06T21:45:06.8491220Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_rpc_complex_args 2024-08-06T21:45:06.8492274Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_none_grads 2024-08-06T21:45:06.8493305Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_add_done_callback 2024-08-06T21:45:06.8494383Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_args_and_kwargs_contain_different_types 2024-08-06T21:45:06.8495475Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote 2024-08-06T21:45:06.8496508Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote_multi 2024-08-06T21:45:06.8497647Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_fork_in_jit_with_profiling 2024-08-06T21:45:06.8498850Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_script_function_that_not_exists_remotely_from_script 2024-08-06T21:45:06.8499972Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_chain 2024-08-06T21:45:06.8501025Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_local_rref_local_value 2024-08-06T21:45:06.8502141Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_record_function_jit_end_callbacks_with_fork 2024-08-06T21:45:06.8503313Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_module 2024-08-06T21:45:06.8504303Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_throw 2024-08-06T21:45:06.8505309Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_as_arg_and_return 2024-08-06T21:45:06.8506391Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_functions_not_supported 2024-08-06T21:45:06.8507499Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_unexepected_kwarg_is_specified 2024-08-06T21:45:06.8508561Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_dist_backward 2024-08-06T21:45:06.8509591Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_get_gradients 2024-08-06T21:45:06.8510665Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_jit_fork_within_context 2024-08-06T21:45:06.8511855Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_restore_context_after_swtich_to_jit_thread 2024-08-06T21:45:06.8513024Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_sync_script 2024-08-06T21:45:06.8514261Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_send_remote_module_with_a_new_attribute_not_pickled_over_the_wire 2024-08-06T21:45:06.8515466Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_train_eval 2024-08-06T21:45:06.8516613Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_create_remote_module_from_module_rref 2024-08-06T21:45:06.8517950Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire 2024-08-06T21:45:06.8519345Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire_script_not_supported 2024-08-06T21:45:06.8520654Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_inside 2024-08-06T21:45:06.8521773Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_no_ddp 2024-08-06T21:45:06.8522928Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote 2024-08-06T21:45:06.8524121Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_builtin_remote_ret_sparse 2024-08-06T21:45:06.8525322Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc 2024-08-06T21:45:06.8526518Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc_without_rank 2024-08-06T21:45:06.8527738Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_py_udf_remote_sparse 2024-08-06T21:45:06.8528929Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_my_parameter_server_sparse 2024-08-06T21:45:06.8530250Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_remote_sparse 2024-08-06T21:45:06.8531587Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rref_stress_sparse 2024-08-06T21:45:06.8532759Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rpc_rref_args_sparse 2024-08-06T21:45:06.8533960Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_sparse 2024-08-06T21:45:06.8535205Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_get_type_timeout_blocking 2024-08-06T21:45:06.8536382Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_proxy_timeout 2024-08-06T21:45:06.8537524Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_py_udf_remote_sparse 2024-08-06T21:45:06.8538783Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_self_remote_arg_sparse 2024-08-06T21:45:06.8540052Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_set_and_get_num_worker_threads 2024-08-06T21:45:06.8541400Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_no_grad_on_tensor_sparse 2024-08-06T21:45:06.8542722Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_multi_sparse 2024-08-06T21:45:06.8544022Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_self_sparse 2024-08-06T21:45:06.8545299Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_sparse 2024-08-06T21:45:06.8546629Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_no_grad_sparse 2024-08-06T21:45:06.8547990Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_call_sparse 2024-08-06T21:45:06.8549354Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_call_itself_sparse 2024-08-06T21:45:06.8550683Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_multiple_backward_sparse 2024-08-06T21:45:06.8552032Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_nested_backward_accumulate_grads_sparse 2024-08-06T21:45:06.8553466Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_no_graph_with_tensors_not_require_grad_sparse 2024-08-06T21:45:06.8554924Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_trainer_ps_sparse 2024-08-06T21:45:06.8555594Z 2024-08-06T21:45:06.8555986Z Running distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint 1/1 ... [2024-08-06 21:45:06.836323] 2024-08-06T21:45:06.8557534Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:45:06.836802] 2024-08-06T21:45:15.3198209Z 2024-08-06T21:45:15.3200335Z distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint_1.1_8fdfcb558d429c9a_.log 2024-08-06T21:45:15.3204715Z Running 2 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint.py::TestOptimStateCheckpointing::test_optim_state_dict_save_load, test/distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint.py::TestOptimStateCheckpointing::test_optim_state_dict_submodule_fully_shard 2024-08-06T21:45:15.3207127Z 2024-08-06T21:45:15.3207675Z Running distributed/rpc/cuda/test_tensorpipe_agent 1/1 ... [2024-08-06 21:45:15.320238] 2024-08-06T21:45:15.3210334Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/cuda/test_tensorpipe_agent.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:45:15.320693] 2024-08-06T21:56:09.2066200Z 2024-08-06T21:56:09.2069266Z distributed/rpc/cuda/test_tensorpipe_agent 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.cuda.test_tensorpipe_agent_1.1_967132d7008e78e2_.log 2024-08-06T21:56:09.2141459Z Running 99 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRpcTest::test_profiler_remote_cuda, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_simple, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation_gpu_root, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device_script, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_invalid_devices, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_valid_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_nested_with_cuda_future, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_with_cuda_future, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_callback_changes_devices, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_sparse_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_sparse_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_sparse_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_int, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_str, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_not_cuda, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_modify_tensor_inplace, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_replace_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_value_on_bad_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_multi, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested_multi, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default_to_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_5, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_6, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_7, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_8, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_5, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_6, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_7, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_8, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default_to_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_in_options, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_local_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_remote_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_min_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_many_to_one, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_loop, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_not_timeout, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote_response, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response_loop, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu_self, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_one_to_many, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_remote, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu_self, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_wrong_worker_name, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_mismatch, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch_reverse, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization5, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_with_unpickleable_attributes, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_tensor_view_as_return_value, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_device_maps_backward_pass, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_dist_autograd_sync_streams, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_gradients_synchronizations 2024-08-06T21:56:09.2207282Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRpcTest::test_profiler_remote_cuda 2024-08-06T21:56:09.2208387Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_simple 2024-08-06T21:56:09.2209524Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation 2024-08-06T21:56:09.2210785Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation_gpu_root 2024-08-06T21:56:09.2212054Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device 2024-08-06T21:56:09.2213303Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device_script 2024-08-06T21:56:09.2214500Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_invalid_devices 2024-08-06T21:56:09.2215596Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_valid_device 2024-08-06T21:56:09.2216798Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote_gpu 2024-08-06T21:56:09.2218176Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_nested_with_cuda_future 2024-08-06T21:56:09.2219741Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_with_cuda_future 2024-08-06T21:56:09.2221146Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_callback_changes_devices 2024-08-06T21:56:09.2222548Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_sparse_tensor 2024-08-06T21:56:09.2223941Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_tensor 2024-08-06T21:56:09.2225492Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_sparse_tensor 2024-08-06T21:56:09.2227020Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_tensor 2024-08-06T21:56:09.2228532Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_sparse_tensor 2024-08-06T21:56:09.2230011Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_tensor 2024-08-06T21:56:09.2231560Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_device 2024-08-06T21:56:09.2232843Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_int 2024-08-06T21:56:09.2234119Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_str 2024-08-06T21:56:09.2235411Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_not_cuda 2024-08-06T21:56:09.2236737Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_modify_tensor_inplace 2024-08-06T21:56:09.2238043Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_replace_tensor 2024-08-06T21:56:09.2239351Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_value_on_bad_device 2024-08-06T21:56:09.2240606Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream 2024-08-06T21:56:09.2241809Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_multi 2024-08-06T21:56:09.2243019Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested 2024-08-06T21:56:09.2244278Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested_multi 2024-08-06T21:56:09.2245518Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu 2024-08-06T21:56:09.2246759Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_default 2024-08-06T21:56:09.2248091Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_non_default 2024-08-06T21:56:09.2249559Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default 2024-08-06T21:56:09.2250889Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default_to_non_default 2024-08-06T21:56:09.2252208Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_1 2024-08-06T21:56:09.2253447Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_2 2024-08-06T21:56:09.2254786Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_3 2024-08-06T21:56:09.2256040Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_4 2024-08-06T21:56:09.2257288Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_5 2024-08-06T21:56:09.2258544Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_6 2024-08-06T21:56:09.2259797Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_7 2024-08-06T21:56:09.2261112Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_8 2024-08-06T21:56:09.2262384Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_1 2024-08-06T21:56:09.2263689Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_2 2024-08-06T21:56:09.2264985Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_3 2024-08-06T21:56:09.2266295Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_4 2024-08-06T21:56:09.2267581Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_5 2024-08-06T21:56:09.2268884Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_6 2024-08-06T21:56:09.2270182Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_7 2024-08-06T21:56:09.2271485Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_8 2024-08-06T21:56:09.2272822Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default 2024-08-06T21:56:09.2274159Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default_to_default 2024-08-06T21:56:09.2275513Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_default 2024-08-06T21:56:09.2276860Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_non_default 2024-08-06T21:56:09.2278180Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_gpu 2024-08-06T21:56:09.2279401Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_in_options 2024-08-06T21:56:09.2280720Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_local_device 2024-08-06T21:56:09.2282110Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_remote_device 2024-08-06T21:56:09.2283450Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_min_device 2024-08-06T21:56:09.2284803Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_many_to_one 2024-08-06T21:56:09.2286087Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config 2024-08-06T21:56:09.2287404Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_loop 2024-08-06T21:56:09.2288768Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_not_timeout 2024-08-06T21:56:09.2290147Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote 2024-08-06T21:56:09.2291805Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote_response 2024-08-06T21:56:09.2293213Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response 2024-08-06T21:56:09.2294594Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response_loop 2024-08-06T21:56:09.2295935Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu 2024-08-06T21:56:09.2297194Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu_self 2024-08-06T21:56:09.2298468Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_one_to_many 2024-08-06T21:56:09.2299716Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_remote 2024-08-06T21:56:09.2301056Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu 2024-08-06T21:56:09.2302340Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu_self 2024-08-06T21:56:09.2303662Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_wrong_worker_name 2024-08-06T21:56:09.2304914Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_mismatch 2024-08-06T21:56:09.2306140Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch 2024-08-06T21:56:09.2307434Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch_reverse 2024-08-06T21:56:09.2308857Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization1 2024-08-06T21:56:09.2310231Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization2 2024-08-06T21:56:09.2311605Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization3 2024-08-06T21:56:09.2312966Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization4 2024-08-06T21:56:09.2314366Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization1 2024-08-06T21:56:09.2315678Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization2 2024-08-06T21:56:09.2316987Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization3 2024-08-06T21:56:09.2318277Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization4 2024-08-06T21:56:09.2319585Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization5 2024-08-06T21:56:09.2320896Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization1 2024-08-06T21:56:09.2322218Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization2 2024-08-06T21:56:09.2323527Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization3 2024-08-06T21:56:09.2324841Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization4 2024-08-06T21:56:09.2326158Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization1 2024-08-06T21:56:09.2327480Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization2 2024-08-06T21:56:09.2328783Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization3 2024-08-06T21:56:09.2330268Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization4 2024-08-06T21:56:09.2331602Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_with_unpickleable_attributes 2024-08-06T21:56:09.2332925Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_tensor_view_as_return_value 2024-08-06T21:56:09.2334223Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_device_maps_backward_pass 2024-08-06T21:56:09.2335556Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_dist_autograd_sync_streams 2024-08-06T21:56:09.2336904Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_gradients_synchronizations 2024-08-06T21:56:09.2337761Z 2024-08-06T21:56:09.2338094Z Running distributed/_shard/sharded_optim/test_sharded_optim 1/1 ... [2024-08-06 21:56:09.210639] 2024-08-06T21:56:09.2339475Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_optim/test_sharded_optim.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:56:09.211096] 2024-08-06T21:56:11.8854028Z 2024-08-06T21:56:11.8855228Z distributed/_shard/sharded_optim/test_sharded_optim 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_optim.test_sharded_optim_1.1_598463a687355753_.log 2024-08-06T21:56:11.8857463Z Running 2 items in this shard: test/distributed/_shard/sharded_optim/test_sharded_optim.py::TestShardedOptimizer::test_named_params_with_sharded_tensor, test/distributed/_shard/sharded_optim/test_sharded_optim.py::TestShardedOptimizer::test_sharded_optim 2024-08-06T21:56:11.8858625Z 2024-08-06T21:56:11.8861039Z Running distributed/fsdp/test_fsdp_multiple_forward 1/1 ... [2024-08-06 21:56:11.885910] 2024-08-06T21:56:11.8866020Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_multiple_forward.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:56:11.886389] 2024-08-06T21:56:18.0177366Z 2024-08-06T21:56:18.0178509Z distributed/fsdp/test_fsdp_multiple_forward 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_multiple_forward_1.1_f1ba9b661bb80707_.log 2024-08-06T21:56:18.0179874Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_multiple_forward.py::TestMultiForward::test_multi_forward 2024-08-06T21:56:18.0180430Z 2024-08-06T21:56:18.0183472Z Running distributed/fsdp/test_fsdp_comm 1/1 ... [2024-08-06 21:56:18.018169] 2024-08-06T21:56:18.0188328Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_comm.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:56:18.018620] 2024-08-06T21:56:44.5428852Z 2024-08-06T21:56:44.5429903Z distributed/fsdp/test_fsdp_comm 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_comm_1.1_d625cb5f5f65bae8_.log 2024-08-06T21:56:44.5436390Z Running 10 items in this shard: test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestExplicitUnshard::test_unshard_async_use_orig_params_False, test/distributed/fsdp/test_fsdp_comm.py::TestExplicitUnshard::test_unshard_async_use_orig_params_True 2024-08-06T21:56:44.5442342Z 2024-08-06T21:56:44.5442579Z Running distributed/test_c10d_common 1/1 ... [2024-08-06 21:56:44.543394] 2024-08-06T21:56:44.5444100Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_common.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:56:44.543858] 2024-08-06T21:59:10.3749555Z 2024-08-06T21:59:10.3750951Z distributed/test_c10d_common 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_common_1.1_eff37e3028e47349_.log 2024-08-06T21:59:10.3761957Z Running 23 items in this shard: test/distributed/test_c10d_common.py::TimeoutTest::test_store_based_barrier, test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_multi_limit_multi_dtype, test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_multi_limit_single_dtype, test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_single_limit_multi_dtype, test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_single_limit_single_dtype, test/distributed/test_c10d_common.py::CommTest::test_debug_level, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_backend_class_attr, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_backend_config, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_collectives, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_get_backend_name, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_init_process_group_with_multiple_backends, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_is_backend_available, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_send_recv, test/distributed/test_c10d_common.py::ProcessGroupWithDispatchedCollectivesTests::test_init_process_group_for_all_backends, test/distributed/test_c10d_common.py::ProcessGroupWithDispatchedCollectivesTests::test_init_process_group_optional_backend, test/distributed/test_c10d_common.py::ReduceOpTest::test_op_isinstance_of_reduceop, test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_copyable, test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_equal, test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_pickle, test/distributed/test_c10d_common.py::LocalRankTest::testNodeLocalRank, test/distributed/test_c10d_common.py::LocalRankTest::testNodeLocalRankOverridesFallback, test/distributed/test_c10d_common.py::LocalRankTest::testWithoutEnv, test/distributed/test_c10d_common.py::LocalRankTest::testWithoutEnvWithFallback 2024-08-06T21:59:10.3771977Z Running 1 items in this shard: test/distributed/test_c10d_common.py::TimeoutTest::test_store_based_barrier 2024-08-06T21:59:10.3772904Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_multi_limit_multi_dtype 2024-08-06T21:59:10.3773921Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_multi_limit_single_dtype 2024-08-06T21:59:10.3774940Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_single_limit_multi_dtype 2024-08-06T21:59:10.3775946Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_single_limit_single_dtype 2024-08-06T21:59:10.3776831Z Running 1 items in this shard: test/distributed/test_c10d_common.py::CommTest::test_debug_level 2024-08-06T21:59:10.3777703Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_backend_class_attr 2024-08-06T21:59:10.3778681Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_backend_config 2024-08-06T21:59:10.3779642Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_collectives 2024-08-06T21:59:10.3780843Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_get_backend_name 2024-08-06T21:59:10.3781981Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_init_process_group_with_multiple_backends 2024-08-06T21:59:10.3783095Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_is_backend_available 2024-08-06T21:59:10.3784073Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_send_recv 2024-08-06T21:59:10.3785189Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ProcessGroupWithDispatchedCollectivesTests::test_init_process_group_for_all_backends 2024-08-06T21:59:10.3786540Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ProcessGroupWithDispatchedCollectivesTests::test_init_process_group_optional_backend 2024-08-06T21:59:10.3787617Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ReduceOpTest::test_op_isinstance_of_reduceop 2024-08-06T21:59:10.3788473Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_copyable 2024-08-06T21:59:10.3789289Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_equal 2024-08-06T21:59:10.3790082Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_pickle 2024-08-06T21:59:10.3790882Z Running 1 items in this shard: test/distributed/test_c10d_common.py::LocalRankTest::testNodeLocalRank 2024-08-06T21:59:10.3791750Z Running 1 items in this shard: test/distributed/test_c10d_common.py::LocalRankTest::testNodeLocalRankOverridesFallback 2024-08-06T21:59:10.3792607Z Running 1 items in this shard: test/distributed/test_c10d_common.py::LocalRankTest::testWithoutEnv 2024-08-06T21:59:10.3793407Z Running 1 items in this shard: test/distributed/test_c10d_common.py::LocalRankTest::testWithoutEnvWithFallback 2024-08-06T21:59:10.3793907Z 2024-08-06T21:59:10.3794209Z Running distributed/_shard/sharded_tensor/ops/test_init 1/1 ... [2024-08-06 21:59:10.375968] 2024-08-06T21:59:10.3795561Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_init.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:59:10.376418] 2024-08-06T21:59:13.1007156Z 2024-08-06T21:59:13.1008351Z distributed/_shard/sharded_tensor/ops/test_init 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_init_1.1_43df3a106417c42b_.log 2024-08-06T21:59:13.1010907Z Running 3 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_init.py::TestShardedTensorNNInit::test_init_sharded_tensor_with_kaiming_uniform, test/distributed/_shard/sharded_tensor/ops/test_init.py::TestShardedTensorNNInit::test_init_sharded_tensor_with_normal, test/distributed/_shard/sharded_tensor/ops/test_init.py::TestShardedTensorNNInit::test_init_sharded_tensor_with_uniform 2024-08-06T21:59:13.1012661Z 2024-08-06T21:59:13.1013345Z Running distributed/rpc/test_faulty_agent 1/1 ... [2024-08-06 21:59:13.101179] 2024-08-06T21:59:13.1018677Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_faulty_agent.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 21:59:13.101633] 2024-08-06T22:04:59.6805216Z 2024-08-06T22:04:59.6806951Z distributed/rpc/test_faulty_agent 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_faulty_agent_1.1_2484546d66319725_.log 2024-08-06T22:04:59.6825245Z Running 28 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_check_failed_messages, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_faulty_messages, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_messages_to_delay, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_no_faulty_messages, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_builtin_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_script_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rref_to_here_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_verify_backend_options, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_context_cleanup_tensor_with_grad, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_verify_backend_options, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_remote_timeout_to_here_in_jit, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_in_jit, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_script_func, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_to_here_timeout_in_jit, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_python, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_torchscript_function 2024-08-06T22:04:59.6839507Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout 2024-08-06T22:04:59.6840728Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout_to_self 2024-08-06T22:04:59.6841846Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_check_failed_messages 2024-08-06T22:04:59.6842861Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_faulty_messages 2024-08-06T22:04:59.6844079Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_messages_to_delay 2024-08-06T22:04:59.6845088Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_no_faulty_messages 2024-08-06T22:04:59.6846164Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout 2024-08-06T22:04:59.6847463Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout_to_self 2024-08-06T22:04:59.6848623Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle 2024-08-06T22:04:59.6849745Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle_to_self 2024-08-06T22:04:59.6850899Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout 2024-08-06T22:04:59.6852072Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout_to_self 2024-08-06T22:04:59.6853294Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_builtin_timeout 2024-08-06T22:04:59.6854279Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_script_timeout 2024-08-06T22:04:59.6855273Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rref_to_here_timeout 2024-08-06T22:04:59.6856325Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout 2024-08-06T22:04:59.6857456Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout_to_self 2024-08-06T22:04:59.6858614Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout 2024-08-06T22:04:59.6859780Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout_to_self 2024-08-06T22:04:59.6860977Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_verify_backend_options 2024-08-06T22:04:59.6862084Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_context_cleanup_tensor_with_grad 2024-08-06T22:04:59.6863251Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_verify_backend_options 2024-08-06T22:04:59.6864377Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_remote_timeout_to_here_in_jit 2024-08-06T22:04:59.6865477Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_in_jit 2024-08-06T22:04:59.6866577Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_script_func 2024-08-06T22:04:59.6867700Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_to_here_timeout_in_jit 2024-08-06T22:04:59.6868746Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_python 2024-08-06T22:04:59.6869814Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_torchscript_function 2024-08-06T22:04:59.6870444Z 2024-08-06T22:04:59.6870799Z Running distributed/_shard/sharded_tensor/test_sharded_tensor_reshard 1/1 ... [2024-08-06 22:04:59.682262] 2024-08-06T22:04:59.6872272Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/test_sharded_tensor_reshard.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:04:59.682730] 2024-08-06T22:05:02.4069476Z 2024-08-06T22:05:02.4071048Z distributed/_shard/sharded_tensor/test_sharded_tensor_reshard 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_reshard_1.1_c21fb6496563c026_.log 2024-08-06T22:05:02.4073128Z Running 2 items in this shard: test/distributed/_shard/sharded_tensor/test_sharded_tensor_reshard.py::TestReshard::test_sharded_tensor_reshard, test/distributed/_shard/sharded_tensor/test_sharded_tensor_reshard.py::TestReshard::test_sharded_tensor_reshard_errors 2024-08-06T22:05:02.4074236Z 2024-08-06T22:05:02.4075702Z Running distributed/_shard/sharded_tensor/test_sharded_tensor 1/1 ... [2024-08-06 22:05:02.407394] 2024-08-06T22:05:02.4080731Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/test_sharded_tensor.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:05:02.407853] 2024-08-06T22:05:11.6455138Z 2024-08-06T22:05:11.6457390Z distributed/_shard/sharded_tensor/test_sharded_tensor 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_1.1_79cd4017e892f7b6_.log 2024-08-06T22:05:11.6517079Z Running 69 items in this shard: test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorMetadata::test_serialize_and_deserialize, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestCreateTensorFromParams::test_empty, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardParameter::test_shard_parameter, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardParameter::test_shard_parameter_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardTensor::test_shard_tensor, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardTensor::test_shard_tensor_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardTensor::test_shard_tensor_with_empty_shard, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestModuleHookApi::test_collect_local_shard, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestModuleHookApi::test_reshard_output, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestLocalTensor::test_local_tensor, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestLocalTensor::test_local_tensor_error, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_cleanup, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_complete_world_size, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_like, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_full, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_ones, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_rand, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_zeros, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_gather_even, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_gather_uneven, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_insufficient_sharding_dims, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_invalid_pg_rpc_ranks, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_invalid_sharding, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_load_state_dict_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_multiple_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_partial_world_size, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_sharded_tensor_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_sharded_tensor_sizes, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_sharding_columns, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_state_dict, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_state_dict_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_state_dict_no_sharded_tensors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_create_sharded_tensor_with_ones, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_gather_even, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_gather_uneven, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_grid_sharding, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_multiple_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_partial_world_size, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_device, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_to_cpu, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_to_cuda, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_to_test, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_uneven_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_with_rpc_names, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalTensor::test_init_from_local_tensor, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalTensor::test_init_from_local_tensor_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_and_global_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_and_global_metadata_invalid_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_pin_memory, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_property_cross_ranks, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_shards_gaps, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_shards_overlap, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_st_base_init_from_local_shards_and_global_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorCustomOps::test_custom_op, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorCustomOps::test_custom_op_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorCustomOps::test_custom_op_override, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardMetadata::test_create_shard_with_no_placement, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardMetadata::test_shard_metadata_init, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorSubGroupInit::test_sub_process_group_placement_validation, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorSubGroupInit::test_sub_process_group_sharded_tensor_init, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestCreateTensorNoProcessGroupMode::test_init_from_local_shards_and_global_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestCreateTensorNoProcessGroupMode::test_non_contiguous_local_shards 2024-08-06T22:05:11.6553266Z 2024-08-06T22:05:11.6553629Z Running distributed/_shard/sharded_tensor/ops/test_embedding 1/1 ... [2024-08-06 22:05:11.646495] 2024-08-06T22:05:11.6555057Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_embedding.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:05:11.647093] 2024-08-06T22:05:14.3730574Z 2024-08-06T22:05:14.3731873Z distributed/_shard/sharded_tensor/ops/test_embedding 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_1.1_38ec777470955cac_.log 2024-08-06T22:05:14.3733899Z Running 2 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_embedding.py::TestShardedEmbedding::test_sharded_embedding_colwise, test/distributed/_shard/sharded_tensor/ops/test_embedding.py::TestShardedEmbedding::test_sharded_embedding_rowwise 2024-08-06T22:05:14.3735046Z 2024-08-06T22:05:14.3737086Z Running distributed/_shard/sharded_tensor/ops/test_embedding_bag 1/1 ... [2024-08-06 22:05:14.373523] 2024-08-06T22:05:14.3742516Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_embedding_bag.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:05:14.374018] 2024-08-06T22:05:17.1487295Z 2024-08-06T22:05:17.1488615Z distributed/_shard/sharded_tensor/ops/test_embedding_bag 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_bag_1.1_57989b22ea5ba46a_.log 2024-08-06T22:05:17.1490903Z Running 2 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_embedding_bag.py::TestShardedEmbeddingBag::test_sharded_embedding_bag_colwise, test/distributed/_shard/sharded_tensor/ops/test_embedding_bag.py::TestShardedEmbeddingBag::test_sharded_embedding_bag_rowwise 2024-08-06T22:05:17.1492416Z 2024-08-06T22:05:17.1493211Z Running distributed/test_c10d_nccl 1/1 ... [2024-08-06 22:05:17.149145] 2024-08-06T22:05:17.1498109Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_nccl.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:05:17.149562] 2024-08-06T22:05:18.6893133Z 2024-08-06T22:05:18.6895040Z distributed/test_c10d_nccl 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_nccl_1.1_27e66ce799dcffef_.log 2024-08-06T22:05:18.6895880Z 2024-08-06T22:05:18.6898575Z Running distributed/test_c10d_spawn_gloo 1/1 ... [2024-08-06 22:05:18.689681] 2024-08-06T22:05:18.6903553Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_spawn_gloo.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:05:18.690138] 2024-08-06T22:06:31.9167919Z 2024-08-06T22:06:31.9170557Z distributed/test_c10d_spawn_gloo 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_spawn_gloo_1.1_e51cb837bb0c611a_.log 2024-08-06T22:06:31.9178130Z Running 15 items in this shard: test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allgather_chunk_gloo, test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allgather_gloo, test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allreduce_gloo, test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_broadcast_gloo, test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cpu, test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cuda, test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_rnn, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_gather, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all_single, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_allreduce, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_broadcast, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_gather, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_reduce, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_scatter 2024-08-06T22:06:31.9185237Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allgather_chunk_gloo 2024-08-06T22:06:31.9186294Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allgather_gloo 2024-08-06T22:06:31.9187301Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allreduce_gloo 2024-08-06T22:06:31.9188319Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_broadcast_gloo 2024-08-06T22:06:31.9189337Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cpu 2024-08-06T22:06:31.9190345Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cuda 2024-08-06T22:06:31.9191334Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_rnn 2024-08-06T22:06:31.9192306Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_gather 2024-08-06T22:06:31.9193252Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all 2024-08-06T22:06:31.9194231Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all_single 2024-08-06T22:06:31.9195201Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_allreduce 2024-08-06T22:06:31.9196306Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_broadcast 2024-08-06T22:06:31.9197236Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_gather 2024-08-06T22:06:31.9198139Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_reduce 2024-08-06T22:06:31.9199058Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_scatter 2024-08-06T22:06:31.9199588Z 2024-08-06T22:06:31.9199833Z Running distributed/test_c10d_spawn_ucc 1/1 ... [2024-08-06 22:06:31.917307] 2024-08-06T22:06:31.9201109Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_spawn_ucc.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:06:31.917764] 2024-08-06T22:07:02.0084701Z 2024-08-06T22:07:02.0086025Z distributed/test_c10d_spawn_ucc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_spawn_ucc_1.1_05abf7da03334ecf_.log 2024-08-06T22:07:02.0090617Z Running 9 items in this shard: test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_allgather_ucc, test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_allreduce_ucc, test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_broadcast_ucc, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_gather, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_to_all, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_to_all_single, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_allreduce, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_broadcast, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_reduce 2024-08-06T22:07:02.0095157Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_allgather_ucc 2024-08-06T22:07:02.0096153Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_allreduce_ucc 2024-08-06T22:07:02.0097156Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_broadcast_ucc 2024-08-06T22:07:02.0098417Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_gather 2024-08-06T22:07:02.0099364Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_to_all 2024-08-06T22:07:02.0100315Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_to_all_single 2024-08-06T22:07:02.0101351Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_allreduce 2024-08-06T22:07:02.0102275Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_broadcast 2024-08-06T22:07:02.0103172Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_reduce 2024-08-06T22:07:02.0103689Z 2024-08-06T22:07:02.0103939Z Running distributed/test_c10d_spawn_nccl 1/1 ... [2024-08-06 22:07:02.009211] 2024-08-06T22:07:02.0105223Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_spawn_nccl.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:07:02.009705] 2024-08-06T22:07:44.4105106Z 2024-08-06T22:07:44.4107074Z distributed/test_c10d_spawn_nccl 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_spawn_nccl_1.1_08b73a242f5e8db4_.log 2024-08-06T22:07:44.4115069Z Running 13 items in this shard: test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_allgather_nccl, test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_allreduce_nccl, test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_broadcast_nccl, test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_reduce_nccl, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather_base, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all_single, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_allreduce, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_broadcast, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter_non_contiguous 2024-08-06T22:07:44.4121725Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_allgather_nccl 2024-08-06T22:07:44.4122746Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_allreduce_nccl 2024-08-06T22:07:44.4123746Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_broadcast_nccl 2024-08-06T22:07:44.4124754Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_reduce_nccl 2024-08-06T22:07:44.4125726Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather 2024-08-06T22:07:44.4126692Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather_base 2024-08-06T22:07:44.4127643Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all 2024-08-06T22:07:44.4128616Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all_single 2024-08-06T22:07:44.4129592Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_allreduce 2024-08-06T22:07:44.4130696Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_broadcast 2024-08-06T22:07:44.4131624Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce 2024-08-06T22:07:44.4132576Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter 2024-08-06T22:07:44.4133629Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter_non_contiguous 2024-08-06T22:07:44.4134245Z 2024-08-06T22:07:44.4134468Z Running distributed/test_launcher 1/1 ... [2024-08-06 22:07:44.411266] 2024-08-06T22:07:44.4135646Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_launcher.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:07:44.411717] 2024-08-06T22:07:48.5883547Z 2024-08-06T22:07:48.5885215Z distributed/test_launcher 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_launcher_1.1_cc90eb11fce887b0_.log 2024-08-06T22:07:48.5887694Z Running 1 items in this shard: test/distributed/test_launcher.py::TestDistributedLaunch::test_launch_user_script 2024-08-06T22:07:48.5888658Z 2024-08-06T22:07:48.5892985Z Running distributed/test_store 1/1 ... [2024-08-06 22:07:48.589083] 2024-08-06T22:07:48.5899844Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_store.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:07:48.589644] 2024-08-06T22:12:34.8217990Z 2024-08-06T22:12:34.8219130Z distributed/test_store 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_store_1.1_3c313cf16bee93be_.log 2024-08-06T22:12:34.8255269Z Running 86 items in this shard: test/distributed/test_store.py::FileStoreTest::test_append, test/distributed/test_store.py::FileStoreTest::test_compare_set, test/distributed/test_store.py::FileStoreTest::test_init_pg_and_rpc_with_same_file, test/distributed/test_store.py::FileStoreTest::test_multi_get, test/distributed/test_store.py::FileStoreTest::test_multi_set, test/distributed/test_store.py::FileStoreTest::test_refcount, test/distributed/test_store.py::FileStoreTest::test_set_get_check, test/distributed/test_store.py::FileStoreTest::test_simple_wait, test/distributed/test_store.py::HashStoreTest::test_append, test/distributed/test_store.py::HashStoreTest::test_compare_set, test/distributed/test_store.py::HashStoreTest::test_multi_get, test/distributed/test_store.py::HashStoreTest::test_multi_set, test/distributed/test_store.py::HashStoreTest::test_set_get_check, test/distributed/test_store.py::HashStoreTest::test_simple_wait, test/distributed/test_store.py::PrefixStoreTest::test_get_underlying_store, test/distributed/test_store.py::PrefixFileStoreTest::test_append, test/distributed/test_store.py::PrefixFileStoreTest::test_compare_set, test/distributed/test_store.py::PrefixFileStoreTest::test_multi_get, test/distributed/test_store.py::PrefixFileStoreTest::test_multi_set, test/distributed/test_store.py::PrefixFileStoreTest::test_set_get_check, test/distributed/test_store.py::PrefixFileStoreTest::test_simple_wait, test/distributed/test_store.py::TCPStoreTest::test_address_already_in_use, test/distributed/test_store.py::TCPStoreTest::test_append, test/distributed/test_store.py::TCPStoreTest::test_compare_set, test/distributed/test_store.py::TCPStoreTest::test_init_pg_and_rpc_with_same_socket, test/distributed/test_store.py::TCPStoreTest::test_multi_get, test/distributed/test_store.py::TCPStoreTest::test_multi_set, test/distributed/test_store.py::TCPStoreTest::test_multi_worker_with_fixed_world_size, test/distributed/test_store.py::TCPStoreTest::test_multi_worker_with_nonfixed_world_size, test/distributed/test_store.py::TCPStoreTest::test_multitenancy, test/distributed/test_store.py::TCPStoreTest::test_numkeys_delkeys, test/distributed/test_store.py::TCPStoreTest::test_repr, test/distributed/test_store.py::TCPStoreTest::test_set_get_check, test/distributed/test_store.py::TCPStoreTest::test_simple_wait, test/distributed/test_store.py::TCPStoreTest::test_store_timeout_on_missing_clients, test/distributed/test_store.py::TCPStoreTest::test_take_over_listen_socket, test/distributed/test_store.py::LibUvTCPStoreTest::test_address_already_in_use, test/distributed/test_store.py::LibUvTCPStoreTest::test_append, test/distributed/test_store.py::LibUvTCPStoreTest::test_compare_set, test/distributed/test_store.py::LibUvTCPStoreTest::test_init_pg_and_rpc_with_same_socket, test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_get, test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_set, test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_worker_with_fixed_world_size, test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_worker_with_nonfixed_world_size, test/distributed/test_store.py::LibUvTCPStoreTest::test_multitenancy, test/distributed/test_store.py::LibUvTCPStoreTest::test_numkeys_delkeys, test/distributed/test_store.py::LibUvTCPStoreTest::test_repr, test/distributed/test_store.py::LibUvTCPStoreTest::test_set_get_check, test/distributed/test_store.py::LibUvTCPStoreTest::test_simple_wait, test/distributed/test_store.py::LibUvTCPStoreTest::test_store_timeout_on_missing_clients, test/distributed/test_store.py::LibUvTCPStoreTest::test_take_over_listen_socket, test/distributed/test_store.py::PrefixTCPStoreTest::test_append, test/distributed/test_store.py::PrefixTCPStoreTest::test_compare_set, test/distributed/test_store.py::PrefixTCPStoreTest::test_multi_get, test/distributed/test_store.py::PrefixTCPStoreTest::test_multi_set, test/distributed/test_store.py::PrefixTCPStoreTest::test_set_get_check, test/distributed/test_store.py::PrefixTCPStoreTest::test_simple_wait, test/distributed/test_store.py::PrefixTCPStoreTest::test_underlying_non_prefix_store, test/distributed/test_store.py::PythonStoreTest::test_set_get, test/distributed/test_store.py::RendezvousTest::test_unknown_handler, test/distributed/test_store.py::RendezvousTest::test_url_with_node_params, test/distributed/test_store.py::RendezvousEnvTest::test_nominal, test/distributed/test_store.py::RendezvousFileTest::test_common_errors, test/distributed/test_store.py::RendezvousFileTest::test_nominal, test/distributed/test_store.py::RendezvousTCPTest::test_common_errors, test/distributed/test_store.py::RendezvousTCPTest::test_dns_timeout, test/distributed/test_store.py::RendezvousTCPTest::test_nominal, test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_timeout_doest_break_client, test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_timeout_set, test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_url_with_libuv, test/distributed/test_store.py::TestPythonStore::test_append_roundtrip, test/distributed/test_store.py::TestPythonStore::test_extended_methods_fallbacks, test/distributed/test_store.py::TestPythonStore::test_has_extended_api_passthrough, test/distributed/test_store.py::TestPythonStore::test_has_extended_api_roundtrip, test/distributed/test_store.py::TestPythonStore::test_multi_get_roundtrip, test/distributed/test_store.py::TestPythonStore::test_multi_set_roundtrip, test/distributed/test_store.py::TestPythonStore::test_optional_methods_fail, test/distributed/test_store.py::TestMultiThreadedWait::test_wait_file_store, test/distributed/test_store.py::TestMultiThreadedWait::test_wait_hash_store, test/distributed/test_store.py::TestMultiThreadedWait::test_wait_prefix_file_store, test/distributed/test_store.py::TestMultiThreadedWait::test_wait_tcp_store, test/distributed/test_store.py::TestMultiThreadedWait::test_wait_tcp_store_uv, test/distributed/test_store.py::TimeoutTest::test_interrupt_doesnt_break_wait, test/distributed/test_store.py::InitPgWithNonUvStore::test_with_env_var, test/distributed/test_store.py::InitPgWithNonUvStore::test_with_url_param, test/distributed/test_store.py::TestClientProtocol::test_client_connect 2024-08-06T22:12:34.8281399Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_append 2024-08-06T22:12:34.8282139Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_compare_set 2024-08-06T22:12:34.8282993Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_init_pg_and_rpc_with_same_file 2024-08-06T22:12:34.8283786Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_multi_get 2024-08-06T22:12:34.8284514Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_multi_set 2024-08-06T22:12:34.8285235Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_refcount 2024-08-06T22:12:34.8285960Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_set_get_check 2024-08-06T22:12:34.8286718Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_simple_wait 2024-08-06T22:12:34.8287432Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_append 2024-08-06T22:12:34.8288242Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_compare_set 2024-08-06T22:12:34.8288959Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_multi_get 2024-08-06T22:12:34.8289681Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_multi_set 2024-08-06T22:12:34.8290423Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_set_get_check 2024-08-06T22:12:34.8291152Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_simple_wait 2024-08-06T22:12:34.8291939Z Running 1 items in this shard: test/distributed/test_store.py::PrefixStoreTest::test_get_underlying_store 2024-08-06T22:12:34.8292792Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_append 2024-08-06T22:12:34.8293553Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_compare_set 2024-08-06T22:12:34.8294347Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_multi_get 2024-08-06T22:12:34.8295125Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_multi_set 2024-08-06T22:12:34.8295916Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_set_get_check 2024-08-06T22:12:34.8296703Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_simple_wait 2024-08-06T22:12:34.8297503Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_address_already_in_use 2024-08-06T22:12:34.8298249Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_append 2024-08-06T22:12:34.8298952Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_compare_set 2024-08-06T22:12:34.8299758Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_init_pg_and_rpc_with_same_socket 2024-08-06T22:12:34.8300624Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_multi_get 2024-08-06T22:12:34.8301321Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_multi_set 2024-08-06T22:12:34.8302126Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_multi_worker_with_fixed_world_size 2024-08-06T22:12:34.8303035Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_multi_worker_with_nonfixed_world_size 2024-08-06T22:12:34.8303872Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_multitenancy 2024-08-06T22:12:34.8304609Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_numkeys_delkeys 2024-08-06T22:12:34.8305325Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_repr 2024-08-06T22:12:34.8306027Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_set_get_check 2024-08-06T22:12:34.8306761Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_simple_wait 2024-08-06T22:12:34.8307566Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_store_timeout_on_missing_clients 2024-08-06T22:12:34.8308420Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_take_over_listen_socket 2024-08-06T22:12:34.8309257Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_address_already_in_use 2024-08-06T22:12:34.8310040Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_append 2024-08-06T22:12:34.8310795Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_compare_set 2024-08-06T22:12:34.8311655Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_init_pg_and_rpc_with_same_socket 2024-08-06T22:12:34.8312491Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_get 2024-08-06T22:12:34.8313315Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_set 2024-08-06T22:12:34.8314172Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_worker_with_fixed_world_size 2024-08-06T22:12:34.8315127Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_worker_with_nonfixed_world_size 2024-08-06T22:12:34.8316016Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_multitenancy 2024-08-06T22:12:34.8316821Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_numkeys_delkeys 2024-08-06T22:12:34.8317644Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_repr 2024-08-06T22:12:34.8318383Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_set_get_check 2024-08-06T22:12:34.8319175Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_simple_wait 2024-08-06T22:12:34.8320033Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_store_timeout_on_missing_clients 2024-08-06T22:12:34.8320922Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_take_over_listen_socket 2024-08-06T22:12:34.8321734Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_append 2024-08-06T22:12:34.8322510Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_compare_set 2024-08-06T22:12:34.8323289Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_multi_get 2024-08-06T22:12:34.8324044Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_multi_set 2024-08-06T22:12:34.8324826Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_set_get_check 2024-08-06T22:12:34.8325620Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_simple_wait 2024-08-06T22:12:34.8326455Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_underlying_non_prefix_store 2024-08-06T22:12:34.8327268Z Running 1 items in this shard: test/distributed/test_store.py::PythonStoreTest::test_set_get 2024-08-06T22:12:34.8328013Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTest::test_unknown_handler 2024-08-06T22:12:34.8328806Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTest::test_url_with_node_params 2024-08-06T22:12:34.8329571Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousEnvTest::test_nominal 2024-08-06T22:12:34.8330540Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousFileTest::test_common_errors 2024-08-06T22:12:34.8331321Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousFileTest::test_nominal 2024-08-06T22:12:34.8332085Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_common_errors 2024-08-06T22:12:34.8332868Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_dns_timeout 2024-08-06T22:12:34.8333624Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_nominal 2024-08-06T22:12:34.8334475Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_timeout_doest_break_client 2024-08-06T22:12:34.8335390Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_timeout_set 2024-08-06T22:12:34.8336244Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_url_with_libuv 2024-08-06T22:12:34.8337081Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_append_roundtrip 2024-08-06T22:12:34.8338049Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_extended_methods_fallbacks 2024-08-06T22:12:34.8338926Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_has_extended_api_passthrough 2024-08-06T22:12:34.8339801Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_has_extended_api_roundtrip 2024-08-06T22:12:34.8340689Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_multi_get_roundtrip 2024-08-06T22:12:34.8341496Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_multi_set_roundtrip 2024-08-06T22:12:34.8342318Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_optional_methods_fail 2024-08-06T22:12:34.8343239Z Running 1 items in this shard: test/distributed/test_store.py::TestMultiThreadedWait::test_wait_file_store 2024-08-06T22:12:34.8344056Z Running 1 items in this shard: test/distributed/test_store.py::TestMultiThreadedWait::test_wait_hash_store 2024-08-06T22:12:34.8344917Z Running 1 items in this shard: test/distributed/test_store.py::TestMultiThreadedWait::test_wait_prefix_file_store 2024-08-06T22:12:34.8345777Z Running 1 items in this shard: test/distributed/test_store.py::TestMultiThreadedWait::test_wait_tcp_store 2024-08-06T22:12:34.8346604Z Running 1 items in this shard: test/distributed/test_store.py::TestMultiThreadedWait::test_wait_tcp_store_uv 2024-08-06T22:12:34.8347451Z Running 1 items in this shard: test/distributed/test_store.py::TimeoutTest::test_interrupt_doesnt_break_wait 2024-08-06T22:12:34.8348278Z Running 1 items in this shard: test/distributed/test_store.py::InitPgWithNonUvStore::test_with_env_var 2024-08-06T22:12:34.8349089Z Running 1 items in this shard: test/distributed/test_store.py::InitPgWithNonUvStore::test_with_url_param 2024-08-06T22:12:34.8349896Z Running 1 items in this shard: test/distributed/test_store.py::TestClientProtocol::test_client_connect 2024-08-06T22:12:34.8350367Z 2024-08-06T22:12:34.8350634Z Running distributed/elastic/events/lib_test 1/1 ... [2024-08-06 22:12:34.824651] 2024-08-06T22:12:34.8351910Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/events/lib_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:12:34.825119] 2024-08-06T22:12:37.9000023Z 2024-08-06T22:12:37.9001398Z distributed/elastic/events/lib_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.events.lib_test_1.1_659e399622ceb358_.log 2024-08-06T22:12:37.9005402Z Running 8 items in this shard: test/distributed/elastic/events/lib_test.py::EventLibTest::test_event_created, test/distributed/elastic/events/lib_test.py::EventLibTest::test_event_deser, test/distributed/elastic/events/lib_test.py::EventLibTest::test_get_or_create_logger, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_construct_and_record_rdzv_event, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_construct_and_record_rdzv_event_does_not_run_if_invalid_dest, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_rdzv_event_created, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_rdzv_event_deserialize, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_rdzv_event_str 2024-08-06T22:12:37.9009087Z 2024-08-06T22:12:37.9009363Z Running distributed/elastic/metrics/api_test 1/1 ... [2024-08-06 22:12:37.900422] 2024-08-06T22:12:37.9011211Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/metrics/api_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:12:37.900875] 2024-08-06T22:12:40.9258525Z 2024-08-06T22:12:40.9260735Z distributed/elastic/metrics/api_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.metrics.api_test_1.1_de47256bd473c07e_.log 2024-08-06T22:12:40.9264458Z Running 3 items in this shard: test/distributed/elastic/metrics/api_test.py::MetricsApiTest::test_get_metric_name, test/distributed/elastic/metrics/api_test.py::MetricsApiTest::test_inheritance, test/distributed/elastic/metrics/api_test.py::MetricsApiTest::test_profile 2024-08-06T22:12:40.9266748Z 2024-08-06T22:12:40.9269845Z Running distributed/elastic/multiprocessing/api_test 1/1 ... [2024-08-06 22:12:40.926744] 2024-08-06T22:12:40.9276107Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/multiprocessing/api_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:12:40.927277] 2024-08-06T22:13:13.6041889Z 2024-08-06T22:13:13.6043598Z distributed/elastic/multiprocessing/api_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.multiprocessing.api_test_1.1_cad35b52be207fac_.log 2024-08-06T22:13:13.6064578Z Running 40 items in this shard: test/distributed/elastic/multiprocessing/api_test.py::RunProcResultsTest::test_get_failures, test/distributed/elastic/multiprocessing/api_test.py::RunProcResultsTest::test_is_failed, test/distributed/elastic/multiprocessing/api_test.py::StdTest::test_from_str_bad_input, test/distributed/elastic/multiprocessing/api_test.py::StdTest::test_from_value, test/distributed/elastic/multiprocessing/api_test.py::StdTest::test_from_value_map, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_args_env_len_mismatch, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_binary_exit, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_binary_incorrect_entrypoint, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_binary_raises, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_function_large_ret_val, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_function_raise, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_function_with_tensor, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_invalid_log_dir, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_multiprocess_context_close, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_multiprocessing_context_poll_raises_exception, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_pcontext_wait, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_subprocess_context_close, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_to_map, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_validate_full_rank, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_void_function, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_wait_for_all_child_procs_to_exit, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_args_env_len_mismatch, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_binary, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_binary_exit, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_binary_incorrect_entrypoint, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_binary_raises, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_binary_redirect_and_tee, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_function, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_function_large_ret_val, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_function_raise, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_function_with_tensor, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_invalid_log_dir, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_multiprocess_context_close, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_multiprocessing_context_poll_raises_exception, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_pcontext_wait, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_subprocess_context_close, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_to_map, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_validate_full_rank, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_void_function, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_wait_for_all_child_procs_to_exit 2024-08-06T22:13:13.6082934Z 2024-08-06T22:13:13.6083218Z Running distributed/elastic/timer/api_test 1/1 ... [2024-08-06 22:13:13.605101] 2024-08-06T22:13:13.6084502Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/timer/api_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:13:13.605587] 2024-08-06T22:13:15.1207895Z 2024-08-06T22:13:15.1209243Z distributed/elastic/timer/api_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.timer.api_test_1.1_daa97a9fe495140d_.log 2024-08-06T22:13:15.1210096Z 2024-08-06T22:13:15.1213836Z Running distributed/elastic/timer/local_timer_example 1/1 ... [2024-08-06 22:13:15.121206] 2024-08-06T22:13:15.1218957Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/timer/local_timer_example.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:13:15.121663] 2024-08-06T22:13:32.4257936Z 2024-08-06T22:13:32.4259101Z distributed/elastic/timer/local_timer_example 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.timer.local_timer_example_1.1_a2a26c6288d157ac_.log 2024-08-06T22:13:32.4261031Z Running 2 items in this shard: test/distributed/elastic/timer/local_timer_example.py::LocalTimerExample::test_example_start_method_spawn, test/distributed/elastic/timer/local_timer_example.py::LocalTimerExample::test_torch_mp_example 2024-08-06T22:13:32.4262120Z 2024-08-06T22:13:32.4264402Z Running distributed/elastic/timer/local_timer_test 1/1 ... [2024-08-06 22:13:32.426256] 2024-08-06T22:13:32.4269341Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/timer/local_timer_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:13:32.426709] 2024-08-06T22:13:40.0097035Z 2024-08-06T22:13:40.0098157Z distributed/elastic/timer/local_timer_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.timer.local_timer_test_1.1_7361dba96913f6a2_.log 2024-08-06T22:13:40.0105613Z Running 14 items in this shard: test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_client_interaction, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_exception_propagation, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_get_timer_recursive, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_happy_path, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_no_client, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_timer, test/distributed/elastic/timer/local_timer_test.py::MultiprocessingRequestQueueTest::test_get, test/distributed/elastic/timer/local_timer_test.py::MultiprocessingRequestQueueTest::test_get_less_than_size, test/distributed/elastic/timer/local_timer_test.py::MultiprocessingRequestQueueTest::test_get_size, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_acquire_release, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_expired_timers, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_valid_timers, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_watchdog_call_count, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_watchdog_empty_queue 2024-08-06T22:13:40.0112219Z 2024-08-06T22:13:40.0112535Z Running distributed/elastic/utils/distributed_test 1/1 ... [2024-08-06 22:13:40.010154] 2024-08-06T22:13:40.0113910Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/utils/distributed_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:13:40.010613] 2024-08-06T22:13:47.7434542Z 2024-08-06T22:13:47.7436067Z distributed/elastic/utils/distributed_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.utils.distributed_test_1.1_bf972dbdd5cab056_.log 2024-08-06T22:13:47.7440852Z Running 8 items in this shard: test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_multi, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_no_port_multi, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_single_server, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_timeout_on_server, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_timeout_on_worker, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_with_libuv_support, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_port_already_in_use_on_server, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_port_already_in_use_on_worker 2024-08-06T22:13:47.7445179Z 2024-08-06T22:13:47.7445461Z Running distributed/elastic/utils/logging_test 1/1 ... [2024-08-06 22:13:47.744113] 2024-08-06T22:13:47.7447903Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/utils/logging_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:13:47.744555] 2024-08-06T22:13:50.8700022Z 2024-08-06T22:13:50.8701418Z distributed/elastic/utils/logging_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.utils.logging_test_1.1_80fe99122b7a4c66_.log 2024-08-06T22:13:50.8703419Z Running 2 items in this shard: test/distributed/elastic/utils/logging_test.py::LoggingTest::test_derive_module_name, test/distributed/elastic/utils/logging_test.py::LoggingTest::test_logger_name 2024-08-06T22:13:50.8704287Z 2024-08-06T22:13:50.8706346Z Running distributed/elastic/utils/util_test 1/1 ... [2024-08-06 22:13:50.870448] 2024-08-06T22:13:50.8710975Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/utils/util_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-06 22:13:50.870885] 2024-08-06T22:13:53.9963404Z 2024-08-06T22:13:53.9965039Z distributed/elastic/utils/util_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.utils.util_test_1.1_d00d19c45784c2e6_.log 2024-08-06T22:13:53.9969731Z Running 10 items in this shard: test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_barrier, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_barrier_hash_store, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_get_all_rank_0, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_get_all_rank_n, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_synchronize, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_synchronize_hash_store, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger_custom_name, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger_different, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger_none 2024-08-06T22:13:53.9975652Z 2024-08-06T22:13:54.9821478Z 2024-08-06T22:13:54.9822045Z real 105m27.873s 2024-08-06T22:13:54.9822327Z user 210m2.125s 2024-08-06T22:13:54.9822568Z sys 41m11.785s 2024-08-06T22:13:54.9822840Z + assert_git_not_dirty 2024-08-06T22:13:54.9823128Z + [[ linux-jammy-py3.8-gcc11 != *rocm* ]] 2024-08-06T22:13:54.9823482Z + [[ linux-jammy-py3.8-gcc11 != *xla* ]] 2024-08-06T22:13:54.9828226Z ++ git status --porcelain 2024-08-06T22:13:54.9829061Z ++ grep -v '?? third_party' 2024-08-06T22:14:18.3983319Z ++ true 2024-08-06T22:14:18.3984446Z + git_status= 2024-08-06T22:14:18.3984836Z + [[ -n '' ]] 2024-08-06T22:14:18.3985239Z + [[ linux-jammy-py3.8-gcc11 == *cuda* ]] 2024-08-06T22:14:18.3985819Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-06T22:14:18.3986298Z + [[ 2 == 1 ]] 2024-08-06T22:14:18.3986644Z + cleanup_workspace 2024-08-06T22:14:18.3987428Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-06T22:14:18.3988729Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-06T22:14:18.3989788Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-06T22:14:18.3990372Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-06T22:14:18.3990914Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-06T22:14:18.3991519Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-06T22:14:18.3992014Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-06T22:14:21.2761564Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-06T22:14:21.2761965Z cat test/**/*_toprint.log || true 2024-08-06T22:14:21.2798035Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T22:14:21.2798437Z env: 2024-08-06T22:14:21.2798655Z GIT_DEFAULT_BRANCH: main 2024-08-06T22:14:21.2799147Z DOCKER_CONTAINER_ID: 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:21.2799664Z ##[endgroup] 2024-08-06T22:14:21.2885803Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-06T22:14:21.2922836Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-06T22:14:21.2923211Z kill "$MONITOR_SCRIPT_PID" 2024-08-06T22:14:21.2929502Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T22:14:21.2929899Z env: 2024-08-06T22:14:21.2930389Z GIT_DEFAULT_BRANCH: main 2024-08-06T22:14:21.2930850Z DOCKER_CONTAINER_ID: 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:21.2931374Z MONITOR_SCRIPT_PID: 343968 2024-08-06T22:14:21.2931656Z ##[endgroup] 2024-08-06T22:14:21.3087129Z Prepare all required actions 2024-08-06T22:14:21.3087554Z Getting action download info 2024-08-06T22:14:21.4902354Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-06T22:14:21.7556889Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-06T22:14:21.7557262Z with: 2024-08-06T22:14:21.7557604Z file-suffix: test-distributed-2-2-amz2023.linux.2xlarge_28427732833 2024-08-06T22:14:21.7558057Z s3-bucket: gha-artifacts 2024-08-06T22:14:21.7558322Z env: 2024-08-06T22:14:21.7558527Z GIT_DEFAULT_BRANCH: main 2024-08-06T22:14:21.7558994Z DOCKER_CONTAINER_ID: 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:21.7559494Z ##[endgroup] 2024-08-06T22:14:21.7589196Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-06T22:14:21.7589686Z # Remove any previous test jsons if they exist 2024-08-06T22:14:21.7590072Z rm -f test-jsons-*.zip 2024-08-06T22:14:21.7590523Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-06T22:14:21.7596967Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T22:14:21.7597369Z env: 2024-08-06T22:14:21.7597600Z GIT_DEFAULT_BRANCH: main 2024-08-06T22:14:21.7598235Z DOCKER_CONTAINER_ID: 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:21.7598881Z FILE_SUFFIX: test-distributed-2-2-amz2023.linux.2xlarge_28427732833 2024-08-06T22:14:21.7599317Z ##[endgroup] 2024-08-06T22:14:21.7868291Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-06T22:14:21.7895412Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-06T22:14:21.7895964Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-06T22:14:21.7902050Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-06T22:14:21.7908476Z adding: test/test-reports/td_exclusions-1f0c234448687f8bfcb9.json (deflated 86%) 2024-08-06T22:14:21.7909064Z adding: test/.pytorch-slow-tests.json (deflated 66%) 2024-08-06T22:14:21.7921741Z adding: test/.pytorch-disabled-tests.json (deflated 89%) 2024-08-06T22:14:21.7961371Z ##[group]Run # Remove any previous test reports if they exist 2024-08-06T22:14:21.7961884Z # Remove any previous test reports if they exist 2024-08-06T22:14:21.7962268Z rm -f test-reports-*.zip 2024-08-06T22:14:21.7962719Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-06T22:14:21.7968533Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T22:14:21.7968906Z env: 2024-08-06T22:14:21.7969133Z GIT_DEFAULT_BRANCH: main 2024-08-06T22:14:21.7969606Z DOCKER_CONTAINER_ID: 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:21.7970227Z FILE_SUFFIX: test-distributed-2-2-amz2023.linux.2xlarge_28427732833 2024-08-06T22:14:21.7970660Z ##[endgroup] 2024-08-06T22:14:21.8270060Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_nested_dict/distributed.checkpoint.test_nested_dict-0ea5953c9f241154.xml (deflated 51%) 2024-08-06T22:14:21.8271552Z adding: test/test-reports/python-pytest/distributed._tensor.debug.test_op_coverage/distributed._tensor.debug.test_op_coverage-8fb8bb79786e2b5e.xml (deflated 34%) 2024-08-06T22:14:21.8272953Z adding: test/test-reports/python-pytest/distributed._tools.test_mod_tracker/distributed._tools.test_mod_tracker-c0b48c5410f89085.xml (deflated 64%) 2024-08-06T22:14:21.8274310Z adding: test/test-reports/python-pytest/distributed.pipelining.test_backward/distributed.pipelining.test_backward-6cbf269c9491a6ed.xml (deflated 37%) 2024-08-06T22:14:21.8275863Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_micro_pipeline_tp/distributed.tensor.parallel.test_micro_pipeline_tp-1b5d72b07e41a350.xml (deflated 95%) 2024-08-06T22:14:21.8277771Z adding: test/test-reports/python-pytest/distributed.pipelining.test_transformer/distributed.pipelining.test_transformer-32d989e31cf7df34.xml (deflated 39%) 2024-08-06T22:14:21.8279306Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.test_logger/distributed._shard.sharded_tensor.test_logger-ff8727c0fae436bd.xml (deflated 37%) 2024-08-06T22:14:21.8280712Z adding: test/test-reports/python-pytest/distributed.test_symmetric_memory/distributed.test_symmetric_memory-8c52cb242fa6d930.xml (deflated 93%) 2024-08-06T22:14:21.8282082Z adding: test/test-reports/python-pytest/distributed._tensor.test_dtensor_compile/distributed._tensor.test_dtensor_compile-c285e75612c86bfa.xml (deflated 84%) 2024-08-06T22:14:21.8283414Z adding: test/test-reports/python-pytest/distributed._tensor.test_attention/distributed._tensor.test_attention-a7dc4ff63b242abc.xml (deflated 85%) 2024-08-06T22:14:21.8284729Z adding: test/test-reports/python-pytest/distributed._composable.test_contract/distributed._composable.test_contract-2900133e42751ddd.xml (deflated 69%) 2024-08-06T22:14:21.8286116Z adding: test/test-reports/python-pytest/distributed._tensor.test_dtensor/distributed._tensor.test_dtensor-ccfa383d90080c02.xml (deflated 82%) 2024-08-06T22:14:21.8287538Z adding: test/test-reports/python-pytest/distributed.checkpoint.fsdp.test_fsdp_dsd/distributed.checkpoint.fsdp.test_fsdp_dsd-be7e6c85511014ee.xml (deflated 62%) 2024-08-06T22:14:21.8289088Z adding: test/test-reports/python-pytest/distributed._tensor.test_xla_integration/distributed._tensor.test_xla_integration-646f600211a12947.xml (deflated 74%) 2024-08-06T22:14:21.8290513Z adding: test/test-reports/python-pytest/distributed._tensor.experimental.test_tp_transform/distributed._tensor.experimental.test_tp_transform-e0924bf1005c0529.xml (deflated 62%) 2024-08-06T22:14:21.8292144Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_dedup_tensors/distributed.checkpoint.test_dedup_tensors-6e6ac9f1ca319ea6.xml (deflated 38%) 2024-08-06T22:14:21.8293590Z adding: test/test-reports/python-pytest/distributed.pipelining.test_unflatten/distributed.pipelining.test_unflatten-bf0563cbad56f363.xml (deflated 41%) 2024-08-06T22:14:21.8295040Z adding: test/test-reports/python-pytest/distributed._tensor.test_convolution_ops/distributed._tensor.test_convolution_ops-f7d59540cbe7761a.xml (deflated 54%) 2024-08-06T22:14:21.8296446Z adding: test/test-reports/python-pytest/distributed._tensor.test_tensor_ops/distributed._tensor.test_tensor_ops-86ba49bd99cd6516.xml (deflated 85%) 2024-08-06T22:14:21.8297825Z adding: test/test-reports/python-pytest/distributed.pipelining.test_microbatch/distributed.pipelining.test_microbatch-40b06925db34b74d.xml (deflated 54%) 2024-08-06T22:14:21.8299383Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_tp_examples/distributed.tensor.parallel.test_tp_examples-1e5549274b039460.xml (deflated 83%) 2024-08-06T22:14:21.8300920Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_traverse/distributed.checkpoint.test_traverse-c434911cdeec9c09.xml (deflated 75%) 2024-08-06T22:14:21.8302188Z adding: test/test-reports/python-pytest/distributed.test_device_mesh/distributed.test_device_mesh-63417847250a2180.xml (deflated 85%) 2024-08-06T22:14:21.8303391Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_planner/distributed.checkpoint.test_planner-3161147a94594f6a.xml (deflated 67%) 2024-08-06T22:14:21.8304762Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_fsspec/distributed.checkpoint.test_fsspec-7a31543fe5b2d2e3.xml (deflated 65%) 2024-08-06T22:14:21.8306097Z adding: test/test-reports/python-pytest/distributed._tensor.test_op_strategy/distributed._tensor.test_op_strategy-94c69e731de5149c.xml (deflated 84%) 2024-08-06T22:14:21.8307559Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_ddp_2d_parallel/distributed.tensor.parallel.test_ddp_2d_parallel-92a156ed43d2241c.xml (deflated 46%) 2024-08-06T22:14:21.8309126Z adding: test/test-reports/python-pytest/distributed.test_inductor_collectives/distributed.test_inductor_collectives-fda760a1d1f2e4f7.xml (deflated 94%) 2024-08-06T22:14:21.8310594Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_dtensor_checkpoint/distributed.checkpoint.test_dtensor_checkpoint-20763763ae5fd483.xml (deflated 47%) 2024-08-06T22:14:21.8312048Z adding: test/test-reports/python-pytest/distributed.launcher.test_run/distributed.launcher.test_run-34bb15877b33ac4f.xml (deflated 84%) 2024-08-06T22:14:21.8313470Z adding: test/test-reports/python-pytest/distributed.checkpoint.e2e.test_fine_tuning/distributed.checkpoint.e2e.test_fine_tuning-b62b3bcf3c4bd2fc.xml (deflated 38%) 2024-08-06T22:14:21.8315070Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_compatibility/distributed.checkpoint.test_compatibility-9f183f828b22553b.xml (deflated 66%) 2024-08-06T22:14:21.8316656Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_hsdp_checkpoint/distributed.checkpoint.test_hsdp_checkpoint-f03490f8a3708b03.xml (deflated 79%) 2024-08-06T22:14:21.8318488Z adding: test/test-reports/python-pytest/distributed._composable.test_composability.test_2d_composability/distributed._composable.test_composability.test_2d_composability-f394b0058e962002.xml (deflated 62%) 2024-08-06T22:14:21.8320341Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_memory/distributed._composable.fsdp.test_fully_shard_memory-77f366ab4b93a1ff.xml (deflated 38%) 2024-08-06T22:14:21.8322194Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_init/distributed._composable.fsdp.test_fully_shard_init-62ccbf73e86eb5f0.xml (deflated 92%) 2024-08-06T22:14:21.8323800Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_optim_state/distributed.fsdp.test_fsdp_optim_state-96aaea0e5f160ae2.xml (deflated 96%) 2024-08-06T22:14:21.8325273Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_utils/distributed.checkpoint.test_utils-bf669298420c69c5.xml (deflated 71%) 2024-08-06T22:14:21.8326796Z adding: test/test-reports/python-pytest/distributed.checkpoint.e2e.test_pipeline/distributed.checkpoint.e2e.test_pipeline-45b01abe88cb7786.xml (deflated 38%) 2024-08-06T22:14:21.8328353Z adding: test/test-reports/python-pytest/distributed.elastic.test_control_plane/distributed.elastic.test_control_plane-436fb2b8a0b9d8dc.xml (deflated 78%) 2024-08-06T22:14:21.8329738Z adding: test/test-reports/python-pytest/distributed.test_c10d_pypg/distributed.test_c10d_pypg-a687652918d89b5b.xml (deflated 94%) 2024-08-06T22:14:21.8331469Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_mixed_precision/distributed._composable.fsdp.test_fully_shard_mixed_precision-df116d7f4edd3789.xml (deflated 83%) 2024-08-06T22:14:21.8333258Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_ignored_modules/distributed.fsdp.test_fsdp_ignored_modules-04c3806841acbc74.xml (deflated 86%) 2024-08-06T22:14:21.8334619Z adding: test/test-reports/python-pytest/distributed.test_nccl/distributed.test_nccl-a86a558dce9380de.xml (deflated 28%) 2024-08-06T22:14:21.8335943Z adding: test/test-reports/python-pytest/distributed._composable.test_compose/distributed._composable.test_compose-a40dce98ef5e2370.xml (deflated 86%) 2024-08-06T22:14:21.8337596Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_util/distributed._composable.fully_shard.test_fully_shard_util-9064ef1e73b22dd5.xml (deflated 47%) 2024-08-06T22:14:21.8339433Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_training/distributed._composable.fsdp.test_fully_shard_training-9f63b6420c32f55b.xml (deflated 84%) 2024-08-06T22:14:21.8341182Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_tp_integration/distributed.fsdp.test_fsdp_tp_integration-84679484b81040b0.xml (deflated 72%) 2024-08-06T22:14:21.8342930Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_dtensor_state_dict/distributed.fsdp.test_fsdp_dtensor_state_dict-e1fc546c48b1a054.xml (deflated 92%) 2024-08-06T22:14:21.8344650Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_state_dict/distributed._composable.fsdp.test_fully_shard_state_dict-5635695b52760fd4.xml (deflated 62%) 2024-08-06T22:14:21.8346265Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_pure_fp16/distributed.fsdp.test_fsdp_pure_fp16-4199524a857826fc.xml (deflated 62%) 2024-08-06T22:14:21.8347769Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_checkpoint/distributed.checkpoint.test_checkpoint-3740c4af32d2679d.xml (deflated 84%) 2024-08-06T22:14:21.8349413Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_frozen/distributed._composable.fsdp.test_fully_shard_frozen-292258ec98d97d91.xml (deflated 62%) 2024-08-06T22:14:21.8351096Z adding: test/test-reports/python-pytest/distributed.fsdp.test_distributed_checkpoint/distributed.fsdp.test_distributed_checkpoint-9180d57017bfea0c.xml (deflated 77%) 2024-08-06T22:14:21.8352790Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_autograd/distributed._composable.fsdp.test_fully_shard_autograd-577fa070403a06ae.xml (deflated 68%) 2024-08-06T22:14:21.8354315Z adding: test/test-reports/python-pytest/distributed.test_fake_pg/distributed.test_fake_pg-11aa3edd1b6badcd.xml (deflated 81%) 2024-08-06T22:14:21.8355730Z adding: test/test-reports/python-pytest/distributed.rpc.test_share_memory/distributed.rpc.test_share_memory-f4fa38ae6832342b.xml (deflated 46%) 2024-08-06T22:14:21.8357169Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_state_dict/distributed.fsdp.test_fsdp_state_dict-a587f10b1c68d9aa.xml (deflated 97%) 2024-08-06T22:14:21.8358538Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_memory/distributed.fsdp.test_fsdp_memory-c21f259f5a533636.xml (deflated 64%) 2024-08-06T22:14:21.8360195Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_runtime/distributed._composable.fully_shard.test_fully_shard_runtime-ce21ac70701cf0e9.xml (deflated 53%) 2024-08-06T22:14:21.8362005Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_file_system_checkpoint_cpu/distributed.checkpoint.test_file_system_checkpoint_cpu-2a1dcd0cce111199.xml (deflated 78%) 2024-08-06T22:14:21.8363575Z adding: test/test-reports/python-pytest/distributed.test_data_parallel/distributed.test_data_parallel-ec571d1570962311.xml (deflated 94%) 2024-08-06T22:14:21.8365094Z adding: test/test-reports/python-pytest/distributed.algorithms.ddp_comm_hooks.test_ddp_hooks/distributed.algorithms.ddp_comm_hooks.test_ddp_hooks-4edb39c74d33cba0.xml (deflated 85%) 2024-08-06T22:14:21.8366675Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_comm_hooks/distributed.fsdp.test_fsdp_comm_hooks-fb407a616fa69ea1.xml (deflated 94%) 2024-08-06T22:14:21.8368098Z adding: test/test-reports/python-pytest/distributed.test_multi_threaded_pg/distributed.test_multi_threaded_pg-6f08a3be606824b6.xml (deflated 81%) 2024-08-06T22:14:21.8369461Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_apply/distributed.fsdp.test_fsdp_apply-4b6e3c53a55bedf4.xml (deflated 71%) 2024-08-06T22:14:21.8370902Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_mixed_precision/distributed.fsdp.test_fsdp_mixed_precision-c092fcae71a7c3cf.xml (deflated 95%) 2024-08-06T22:14:21.8372364Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_input/distributed.fsdp.test_fsdp_input-fd2de36cdfdb98f9.xml (deflated 64%) 2024-08-06T22:14:21.8373853Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_multiple_wrapping/distributed.fsdp.test_fsdp_multiple_wrapping-ca121efb12d95ccb.xml (deflated 47%) 2024-08-06T22:14:21.8375341Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_uneven/distributed.fsdp.test_fsdp_uneven-f8d81c66a74007cc.xml (deflated 44%) 2024-08-06T22:14:21.8376862Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_traversal/distributed.fsdp.test_fsdp_traversal-a339adb1f441af69.xml (deflated 45%) 2024-08-06T22:14:21.8378331Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_misc/distributed.fsdp.test_fsdp_misc-1d3e4fc0cecd7c06.xml (deflated 89%) 2024-08-06T22:14:21.8379642Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-20b0e9054f6ea92a.xml (deflated 44%) 2024-08-06T22:14:21.8380952Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-afe004983991d007.xml (deflated 44%) 2024-08-06T22:14:21.8382233Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-914d102c71773d68.xml (deflated 44%) 2024-08-06T22:14:21.8383454Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-2e0587a8c0504e67.xml (deflated 43%) 2024-08-06T22:14:21.8384716Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-e6a652bd5424085d.xml (deflated 44%) 2024-08-06T22:14:21.8385967Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-b789f2ac6fc835fc.xml (deflated 44%) 2024-08-06T22:14:21.8387332Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-d5370cd0f5ba90c2.xml (deflated 44%) 2024-08-06T22:14:21.8388554Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-3157c90d737ca03f.xml (deflated 44%) 2024-08-06T22:14:21.8389819Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-99f513d9ee83589f.xml (deflated 34%) 2024-08-06T22:14:21.8391082Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-72786816ba67db52.xml (deflated 42%) 2024-08-06T22:14:21.8392327Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-495917c77d82c02d.xml (deflated 42%) 2024-08-06T22:14:21.8393587Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-c668e74d32ee59b3.xml (deflated 34%) 2024-08-06T22:14:21.8394839Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-eb9bed0876f828a9.xml (deflated 34%) 2024-08-06T22:14:21.8396088Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-d578fac664876cae.xml (deflated 35%) 2024-08-06T22:14:21.8397372Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-0fab5d137746ccbf.xml (deflated 42%) 2024-08-06T22:14:21.8398638Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-06f9fef512fb0250.xml (deflated 42%) 2024-08-06T22:14:21.8399905Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-49942b1799f37a6a.xml (deflated 35%) 2024-08-06T22:14:21.8401275Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e1452309aef01778.xml (deflated 36%) 2024-08-06T22:14:21.8402747Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d806587f3857b1b8.xml (deflated 36%) 2024-08-06T22:14:21.8404209Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8bf9993eec1e0c9b.xml (deflated 35%) 2024-08-06T22:14:21.8405683Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-76ddd1c9024dff7e.xml (deflated 35%) 2024-08-06T22:14:21.8407185Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c58eb422faef6d4d.xml (deflated 36%) 2024-08-06T22:14:21.8408774Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f75d1cc1d3993a17.xml (deflated 35%) 2024-08-06T22:14:21.8410285Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-eb195b2b690380b3.xml (deflated 36%) 2024-08-06T22:14:21.8411788Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0da3a7f7d867eb2d.xml (deflated 37%) 2024-08-06T22:14:21.8413165Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d48da146b586d7f5.xml (deflated 36%) 2024-08-06T22:14:21.8414448Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f58c733b7607fee8.xml (deflated 36%) 2024-08-06T22:14:21.8415899Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-14dff2a4a1c31677.xml (deflated 35%) 2024-08-06T22:14:21.8417347Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-70b4899315e4dd84.xml (deflated 35%) 2024-08-06T22:14:21.8418815Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0ee6744739397405.xml (deflated 36%) 2024-08-06T22:14:21.8420359Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fac5912b5450955e.xml (deflated 35%) 2024-08-06T22:14:21.8421891Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-03d9304d25ec6408.xml (deflated 36%) 2024-08-06T22:14:21.8423345Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-17b49b957953f9e1.xml (deflated 36%) 2024-08-06T22:14:21.8424801Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5a317b540f119efa.xml (deflated 35%) 2024-08-06T22:14:21.8426254Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e47e6e3c2f17fa0d.xml (deflated 36%) 2024-08-06T22:14:21.8427738Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ea86fefcd0715be1.xml (deflated 36%) 2024-08-06T22:14:21.8429177Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-83f63c079377bd33.xml (deflated 36%) 2024-08-06T22:14:21.8430831Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0bf95319f8f64e15.xml (deflated 35%) 2024-08-06T22:14:21.8433221Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a90fb094419f6d47.xml (deflated 35%) 2024-08-06T22:14:21.8435773Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-78e6552cdfdf6c94.xml (deflated 35%) 2024-08-06T22:14:21.8438315Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a963d33bca3cc6ff.xml (deflated 35%) 2024-08-06T22:14:21.8440851Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-eb6338bf0fa2b88d.xml (deflated 36%) 2024-08-06T22:14:21.8443376Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9da1f0d75ae5f0fc.xml (deflated 35%) 2024-08-06T22:14:21.8445823Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1925343bef534f74.xml (deflated 35%) 2024-08-06T22:14:21.8447638Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6258c5c969047021.xml (deflated 36%) 2024-08-06T22:14:21.8450240Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-401346f029da12ba.xml (deflated 35%) 2024-08-06T22:14:21.8452607Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f55db3776f3d8399.xml (deflated 36%) 2024-08-06T22:14:21.8454759Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d0f4acc37f421a71.xml (deflated 36%) 2024-08-06T22:14:21.8456708Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9ef505506e4b56fd.xml (deflated 36%) 2024-08-06T22:14:21.8459070Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-714554959fa69225.xml (deflated 35%) 2024-08-06T22:14:21.8461656Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1eee2719421a47f8.xml (deflated 36%) 2024-08-06T22:14:21.8464143Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f144efdd20129adc.xml (deflated 36%) 2024-08-06T22:14:21.8466871Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3d09690f75536911.xml (deflated 36%) 2024-08-06T22:14:21.8469313Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-703028fc16f5cce2.xml (deflated 35%) 2024-08-06T22:14:21.8471762Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4e62369081f95ea7.xml (deflated 35%) 2024-08-06T22:14:21.8474232Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-958c0b70aa209f66.xml (deflated 36%) 2024-08-06T22:14:21.8476725Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e171c2d54485a19c.xml (deflated 37%) 2024-08-06T22:14:21.8479182Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e23bf2aa00880a3e.xml (deflated 35%) 2024-08-06T22:14:21.8481675Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-31080ad224e4a851.xml (deflated 35%) 2024-08-06T22:14:21.8484134Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b8b19cd60ee689b6.xml (deflated 35%) 2024-08-06T22:14:21.8486622Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f552fc1033d00c1a.xml (deflated 35%) 2024-08-06T22:14:21.8489097Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1bacfe86616d2565.xml (deflated 36%) 2024-08-06T22:14:21.8491555Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ee417472adf1ded2.xml (deflated 35%) 2024-08-06T22:14:21.8494010Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0489c9edb3c8c14d.xml (deflated 35%) 2024-08-06T22:14:21.8496492Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6f4c59a20ea33242.xml (deflated 35%) 2024-08-06T22:14:21.8498980Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-342dc528ffaaeb3b.xml (deflated 36%) 2024-08-06T22:14:21.8501538Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6f43bd38ef7a8879.xml (deflated 36%) 2024-08-06T22:14:21.8504140Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5a31a1cc34e6050a.xml (deflated 34%) 2024-08-06T22:14:21.8506590Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b47f439794433d34.xml (deflated 35%) 2024-08-06T22:14:21.8509055Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4d9a9523c109e90e.xml (deflated 35%) 2024-08-06T22:14:21.8511566Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2d6f7896ccfa9857.xml (deflated 35%) 2024-08-06T22:14:21.8514032Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b5f2d5bf066e58f9.xml (deflated 35%) 2024-08-06T22:14:21.8516509Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7582c16eeb758a93.xml (deflated 35%) 2024-08-06T22:14:21.8519001Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c025de45a8dd5ebe.xml (deflated 35%) 2024-08-06T22:14:21.8521507Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1be819b235ebd1f3.xml (deflated 36%) 2024-08-06T22:14:21.8524108Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fcdf6d17a5c90ed8.xml (deflated 35%) 2024-08-06T22:14:21.8526603Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-bc507b554bd24b44.xml (deflated 35%) 2024-08-06T22:14:21.8528689Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c1ae402c35048c0b.xml (deflated 35%) 2024-08-06T22:14:21.8531065Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-362092a34d2d525c.xml (deflated 35%) 2024-08-06T22:14:21.8532982Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8448a0f5e76bb14e.xml (deflated 35%) 2024-08-06T22:14:21.8535059Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3872bde762452fc7.xml (deflated 36%) 2024-08-06T22:14:21.8537362Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c7c286cfc33dfbb1.xml (deflated 35%) 2024-08-06T22:14:21.8539727Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-26136a84a71c2685.xml (deflated 35%) 2024-08-06T22:14:21.8541694Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-70dc76693f59e8ab.xml (deflated 35%) 2024-08-06T22:14:21.8543622Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f23566bd301bac9c.xml (deflated 35%) 2024-08-06T22:14:21.8545193Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e56c820029baa1b4.xml (deflated 34%) 2024-08-06T22:14:21.8546506Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cd0d33f0572ecd85.xml (deflated 36%) 2024-08-06T22:14:21.8547792Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-bda5a5610adcb639.xml (deflated 35%) 2024-08-06T22:14:21.8549086Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2ef5335c4e8cdc03.xml (deflated 36%) 2024-08-06T22:14:21.8550607Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-267467285a78d8ea.xml (deflated 34%) 2024-08-06T22:14:21.8551900Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-74c3cc3fd4ded38d.xml (deflated 35%) 2024-08-06T22:14:21.8553177Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-08ee1d4936c03a58.xml (deflated 35%) 2024-08-06T22:14:21.8554483Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5f6b1164ce5bf0ed.xml (deflated 35%) 2024-08-06T22:14:21.8555774Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1aaf86f4d30051bc.xml (deflated 36%) 2024-08-06T22:14:21.8557067Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-77df4675687b5e6a.xml (deflated 35%) 2024-08-06T22:14:21.8558346Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b17fdbd8dacd8730.xml (deflated 34%) 2024-08-06T22:14:21.8559643Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-23335c2f965c4ec4.xml (deflated 35%) 2024-08-06T22:14:21.8561036Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e1a4f06599435b4b.xml (deflated 34%) 2024-08-06T22:14:21.8562332Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f7568ebb4cb3688a.xml (deflated 34%) 2024-08-06T22:14:21.8563607Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ac3e3f8f2602a7a6.xml (deflated 35%) 2024-08-06T22:14:21.8564905Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-148d3d51081f03c7.xml (deflated 36%) 2024-08-06T22:14:21.8566201Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b50a71be88be3093.xml (deflated 35%) 2024-08-06T22:14:21.8567485Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c95849e88a9903c7.xml (deflated 36%) 2024-08-06T22:14:21.8568767Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8851f4347d3f260f.xml (deflated 35%) 2024-08-06T22:14:21.8570061Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-06b3dd8c39cdb787.xml (deflated 35%) 2024-08-06T22:14:21.8571355Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-43f2aa8697c039fe.xml (deflated 55%) 2024-08-06T22:14:21.8572632Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-590d81881d4d58ed.xml (deflated 34%) 2024-08-06T22:14:21.8573931Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f1eed9e1b3dc975f.xml (deflated 35%) 2024-08-06T22:14:21.8575228Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6f81483553349ff2.xml (deflated 37%) 2024-08-06T22:14:21.8576519Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7260825df8bad92d.xml (deflated 37%) 2024-08-06T22:14:21.8577792Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8141080ae730f3a9.xml (deflated 37%) 2024-08-06T22:14:21.8579088Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-303e3d7cba41e7bd.xml (deflated 36%) 2024-08-06T22:14:21.8580452Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c95658ecc987a642.xml (deflated 38%) 2024-08-06T22:14:21.8581838Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4dc36d0e6c28e576.xml (deflated 37%) 2024-08-06T22:14:21.8583126Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-dba779b48fdd22b1.xml (deflated 37%) 2024-08-06T22:14:21.8584424Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-50e0a6033b895ef8.xml (deflated 37%) 2024-08-06T22:14:21.8585707Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-100fc59161ba9143.xml (deflated 37%) 2024-08-06T22:14:21.8586997Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-63cc625e624f331a.xml (deflated 37%) 2024-08-06T22:14:21.8588276Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f58ed1467a30b616.xml (deflated 36%) 2024-08-06T22:14:21.8589552Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c97867fb0f4d087e.xml (deflated 36%) 2024-08-06T22:14:21.8590912Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-58258886d26890d4.xml (deflated 38%) 2024-08-06T22:14:21.8592188Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-997c0f5524494c10.xml (deflated 37%) 2024-08-06T22:14:21.8593461Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1ccfac07d5983270.xml (deflated 37%) 2024-08-06T22:14:21.8594759Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9c637867f66c4ea5.xml (deflated 37%) 2024-08-06T22:14:21.8596045Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3b2af3888f54d45e.xml (deflated 36%) 2024-08-06T22:14:21.8597339Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e4e477ddfcd5fbfa.xml (deflated 36%) 2024-08-06T22:14:21.8598614Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-95e1bd0888914589.xml (deflated 38%) 2024-08-06T22:14:21.8599910Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9f750eeacfef2395.xml (deflated 37%) 2024-08-06T22:14:21.8601205Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3f87270d38fb142c.xml (deflated 37%) 2024-08-06T22:14:21.8602494Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b0ced638f5f7cd5a.xml (deflated 36%) 2024-08-06T22:14:21.8603770Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4ef7b96c97965f33.xml (deflated 37%) 2024-08-06T22:14:21.8605070Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0a8112248e37ae8b.xml (deflated 37%) 2024-08-06T22:14:21.8606360Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3148facfc7f3fbac.xml (deflated 37%) 2024-08-06T22:14:21.8607923Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint-91439ca023bc0f5f.xml (deflated 67%) 2024-08-06T22:14:21.8609577Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-912e9942cb408a5a.xml (deflated 41%) 2024-08-06T22:14:21.8610949Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-c2cc855839c03923.xml (deflated 41%) 2024-08-06T22:14:21.8612635Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-56ef60f74eb39f45.xml (deflated 41%) 2024-08-06T22:14:21.8614017Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-f20e300fb7f01fb2.xml (deflated 41%) 2024-08-06T22:14:21.8615377Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-454ddcdf476b645d.xml (deflated 41%) 2024-08-06T22:14:21.8616762Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d1455af2c988b463.xml (deflated 41%) 2024-08-06T22:14:21.8618141Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d6781254a4ebc7f8.xml (deflated 41%) 2024-08-06T22:14:21.8619592Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-bc5bdd74ea060b29.xml (deflated 41%) 2024-08-06T22:14:21.8621020Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-50470efa0b15bd0a.xml (deflated 43%) 2024-08-06T22:14:21.8622402Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-e740a34a3b09b7af.xml (deflated 41%) 2024-08-06T22:14:21.8623786Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d61279ef767f8960.xml (deflated 41%) 2024-08-06T22:14:21.8625166Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-1b7142799729af8d.xml (deflated 42%) 2024-08-06T22:14:21.8626529Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-21391ea4389fa2b1.xml (deflated 42%) 2024-08-06T22:14:21.8627903Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-2d5aaff2783bc243.xml (deflated 42%) 2024-08-06T22:14:21.8629279Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-191590d6758b2fdc.xml (deflated 42%) 2024-08-06T22:14:21.8631190Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-36913e15f4354593.xml (deflated 42%) 2024-08-06T22:14:21.8632642Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-cb98e338ecad4429.xml (deflated 42%) 2024-08-06T22:14:21.8634020Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-09c73a0a6a900142.xml (deflated 42%) 2024-08-06T22:14:21.8635394Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-fec3b3edcd7a2453.xml (deflated 42%) 2024-08-06T22:14:21.8636771Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-a306fd4472a20fe2.xml (deflated 42%) 2024-08-06T22:14:21.8638131Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b74f1b0ce96a18d1.xml (deflated 42%) 2024-08-06T22:14:21.8639632Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-99b762586c5dc9fa.xml (deflated 42%) 2024-08-06T22:14:21.8641014Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-4965d631ac4e29ba.xml (deflated 42%) 2024-08-06T22:14:21.8642404Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d99156afb4aac6b9.xml (deflated 42%) 2024-08-06T22:14:21.8643765Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-a740ae824e774035.xml (deflated 42%) 2024-08-06T22:14:21.8645129Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-78b451ea98844e23.xml (deflated 42%) 2024-08-06T22:14:21.8646492Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-9941e8cad3870f03.xml (deflated 42%) 2024-08-06T22:14:21.8647866Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-44fa9341251dcf0a.xml (deflated 42%) 2024-08-06T22:14:21.8649353Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b9c48b4e1920b247.xml (deflated 42%) 2024-08-06T22:14:21.8650719Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-84b052c14f541530.xml (deflated 36%) 2024-08-06T22:14:21.8652088Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d288a96c63c3f888.xml (deflated 42%) 2024-08-06T22:14:21.8653470Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-448dfada2b8554fb.xml (deflated 42%) 2024-08-06T22:14:21.8654837Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-40f1a1e4f8482d71.xml (deflated 42%) 2024-08-06T22:14:21.8656209Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-cf8e1b71ceeaa5c0.xml (deflated 42%) 2024-08-06T22:14:21.8657589Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-063960cef2c578b6.xml (deflated 42%) 2024-08-06T22:14:21.8658956Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-f8171840f00c8546.xml (deflated 41%) 2024-08-06T22:14:21.8660315Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-864f1827260cf572.xml (deflated 42%) 2024-08-06T22:14:21.8661765Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-86791bd0000afe78.xml (deflated 42%) 2024-08-06T22:14:21.8663140Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-3b13a0d1d9b5ee9e.xml (deflated 42%) 2024-08-06T22:14:21.8664520Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-373c006c5aed5c68.xml (deflated 41%) 2024-08-06T22:14:21.8665883Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-22540d907ad0f319.xml (deflated 42%) 2024-08-06T22:14:21.8667258Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-76ae7db413ad7894.xml (deflated 41%) 2024-08-06T22:14:21.8668700Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-70b90be4bc871e92.xml (deflated 42%) 2024-08-06T22:14:21.8670075Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-c54c4542e7f66f35.xml (deflated 41%) 2024-08-06T22:14:21.8671446Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-9eefce3e9ed995af.xml (deflated 42%) 2024-08-06T22:14:21.8672812Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-a81e91f681d9f42d.xml (deflated 42%) 2024-08-06T22:14:21.8674194Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-3532cf6f4828fce6.xml (deflated 42%) 2024-08-06T22:14:21.8675612Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-a3f666e81aa3a9ed.xml (deflated 41%) 2024-08-06T22:14:21.8676980Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d748117f300b0d14.xml (deflated 42%) 2024-08-06T22:14:21.8678351Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-9dfe7a42bea1fc09.xml (deflated 41%) 2024-08-06T22:14:21.8679793Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-90e4706b271e60b0.xml (deflated 42%) 2024-08-06T22:14:21.8681166Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-018e2bb55adfd17b.xml (deflated 42%) 2024-08-06T22:14:21.8682534Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-77a1a766926a70c3.xml (deflated 42%) 2024-08-06T22:14:21.8683921Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-3e3799efb79d4e59.xml (deflated 42%) 2024-08-06T22:14:21.8685314Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-24c52acf21a3404e.xml (deflated 42%) 2024-08-06T22:14:21.8686897Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-4fe731d4dab82dfb.xml (deflated 42%) 2024-08-06T22:14:21.8688280Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-f2b9aaac5f2f0bef.xml (deflated 42%) 2024-08-06T22:14:21.8689679Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-7e308e550f30e4ce.xml (deflated 42%) 2024-08-06T22:14:21.8691064Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-a5bc433d39340c72.xml (deflated 42%) 2024-08-06T22:14:21.8692448Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-3260ca62ac7078ea.xml (deflated 42%) 2024-08-06T22:14:21.8693817Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-88b529a002df9c83.xml (deflated 42%) 2024-08-06T22:14:21.8695193Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-74a933dd0da10a40.xml (deflated 42%) 2024-08-06T22:14:21.8696574Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-cd83f7e58aaf7184.xml (deflated 42%) 2024-08-06T22:14:21.8697947Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-8809c3bbe0da434a.xml (deflated 42%) 2024-08-06T22:14:21.8699435Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0fcff2250b4809b4.xml (deflated 42%) 2024-08-06T22:14:21.8700904Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b28e078932b6d290.xml (deflated 42%) 2024-08-06T22:14:21.8702285Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-2ced1398149f341c.xml (deflated 42%) 2024-08-06T22:14:21.8703667Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-2ee7835e326c19d5.xml (deflated 42%) 2024-08-06T22:14:21.8705049Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-9ff999edce0b133e.xml (deflated 42%) 2024-08-06T22:14:21.8706425Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-541607c59c2160b3.xml (deflated 42%) 2024-08-06T22:14:21.8707810Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-00e34b8c71eadbd7.xml (deflated 42%) 2024-08-06T22:14:21.8709298Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b766217727bab588.xml (deflated 42%) 2024-08-06T22:14:21.8710673Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-f58b8460ed63bf13.xml (deflated 42%) 2024-08-06T22:14:21.8712062Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-a1872d3153695b01.xml (deflated 42%) 2024-08-06T22:14:21.8713439Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-7a8f1f89f45ab78f.xml (deflated 42%) 2024-08-06T22:14:21.8714839Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-bc248ebef7cee34c.xml (deflated 42%) 2024-08-06T22:14:21.8716226Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-6c1db9a675d572ee.xml (deflated 42%) 2024-08-06T22:14:21.8717609Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-2e6ba66109e2325d.xml (deflated 41%) 2024-08-06T22:14:21.8718965Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-5621768967ea0199.xml (deflated 41%) 2024-08-06T22:14:21.8720353Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-52ef09e5bc7c70fc.xml (deflated 41%) 2024-08-06T22:14:21.8721742Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0b518baa919eb877.xml (deflated 41%) 2024-08-06T22:14:21.8723124Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-43d58f59ba6a91e1.xml (deflated 41%) 2024-08-06T22:14:21.8724509Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-99635a2fdab77baf.xml (deflated 41%) 2024-08-06T22:14:21.8725899Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-5a8c3eb8914e9368.xml (deflated 41%) 2024-08-06T22:14:21.8727281Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-aaf4f36d5f6477dd.xml (deflated 42%) 2024-08-06T22:14:21.8728771Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-ac45589c051ac98f.xml (deflated 41%) 2024-08-06T22:14:21.8730720Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-2e2c81969d9542a4.xml (deflated 41%) 2024-08-06T22:14:21.8732140Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-6c7e1ad196a99e4d.xml (deflated 41%) 2024-08-06T22:14:21.8733531Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-6e6f9cf853d77deb.xml (deflated 41%) 2024-08-06T22:14:21.8734916Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-6d483a987c999a10.xml (deflated 41%) 2024-08-06T22:14:21.8736284Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-acc3e177141459df.xml (deflated 41%) 2024-08-06T22:14:21.8737656Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-72f0d3128e393977.xml (deflated 41%) 2024-08-06T22:14:21.8739196Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-076998718f31621a.xml (deflated 41%) 2024-08-06T22:14:21.8740573Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-3d500897b47a083b.xml (deflated 41%) 2024-08-06T22:14:21.8741997Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-18296e1cea58842a.xml (deflated 42%) 2024-08-06T22:14:21.8743381Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-21b01a924f1e16c7.xml (deflated 42%) 2024-08-06T22:14:21.8744762Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-ad6abda009445c33.xml (deflated 42%) 2024-08-06T22:14:21.8746142Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-86a2199592e256ed.xml (deflated 43%) 2024-08-06T22:14:21.8747501Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-a1a62715f815e97d.xml (deflated 42%) 2024-08-06T22:14:21.8748938Z adding: test/test-reports/python-pytest/distributed._shard.sharded_optim.test_sharded_optim/distributed._shard.sharded_optim.test_sharded_optim-330523f5c32ae137.xml (deflated 67%) 2024-08-06T22:14:21.8750378Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_multiple_forward/distributed.fsdp.test_fsdp_multiple_forward-1c37cf9b77028f59.xml (deflated 46%) 2024-08-06T22:14:21.8751677Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_comm/distributed.fsdp.test_fsdp_comm-dcbfe933ca9ab453.xml (deflated 87%) 2024-08-06T22:14:21.8752831Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-793ccf1528636d1e.xml (deflated 37%) 2024-08-06T22:14:21.8753967Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-3092bd6cbd2aed4b.xml (deflated 35%) 2024-08-06T22:14:21.8755097Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-953413635df12d51.xml (deflated 35%) 2024-08-06T22:14:21.8756225Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-8d1121b008ab5795.xml (deflated 34%) 2024-08-06T22:14:21.8757340Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-dafe6e9d5bc11ea4.xml (deflated 36%) 2024-08-06T22:14:21.8758597Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-316e5d0097bedf3c.xml (deflated 34%) 2024-08-06T22:14:21.8759736Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-3f8d33eca07e1414.xml (deflated 34%) 2024-08-06T22:14:21.8760861Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-4cd8d87bdf6f45e9.xml (deflated 34%) 2024-08-06T22:14:21.8762001Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-c3a9e6b7a2ed9caa.xml (deflated 34%) 2024-08-06T22:14:21.8763134Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-1bc20f40f926e3c7.xml (deflated 34%) 2024-08-06T22:14:21.8764249Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-10244e19e346d793.xml (deflated 35%) 2024-08-06T22:14:21.8765359Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-d131f82583cb1968.xml (deflated 34%) 2024-08-06T22:14:21.8766480Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-d7a9eeb31e970ff4.xml (deflated 34%) 2024-08-06T22:14:21.8767715Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-59ac2b711ed5a695.xml (deflated 34%) 2024-08-06T22:14:21.8768845Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-7a88686d64dc4cb4.xml (deflated 34%) 2024-08-06T22:14:21.8769951Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-173b0b4e5e98cf9b.xml (deflated 36%) 2024-08-06T22:14:21.8771075Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-09616f1cf297017c.xml (deflated 36%) 2024-08-06T22:14:21.8772213Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-63ba8c73a439aedb.xml (deflated 36%) 2024-08-06T22:14:21.8773333Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-826f5a5f4d99fb56.xml (deflated 36%) 2024-08-06T22:14:21.8774465Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-bf3b642ddec4d2b6.xml (deflated 36%) 2024-08-06T22:14:21.8775599Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-85c412ba54518259.xml (deflated 34%) 2024-08-06T22:14:21.8776720Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-2c84038b14e1749e.xml (deflated 33%) 2024-08-06T22:14:21.8777830Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-6dcf01b09ff27d93.xml (deflated 33%) 2024-08-06T22:14:21.8779127Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_init/distributed._shard.sharded_tensor.ops.test_init-f49b624d0b7d0eab.xml (deflated 76%) 2024-08-06T22:14:21.8780462Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-521f63ccfeab27e0.xml (deflated 36%) 2024-08-06T22:14:21.8781780Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-8064d0e0148427b9.xml (deflated 36%) 2024-08-06T22:14:21.8782975Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-c0a965d1784aa23f.xml (deflated 36%) 2024-08-06T22:14:21.8784188Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-fe42a2e989a88ddb.xml (deflated 37%) 2024-08-06T22:14:21.8785403Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-17e41bba7a7c8d26.xml (deflated 36%) 2024-08-06T22:14:21.8786697Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-c925ef7f8c283d56.xml (deflated 37%) 2024-08-06T22:14:21.8787892Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-0eca5ec1af9f0a32.xml (deflated 36%) 2024-08-06T22:14:21.8789107Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-7f4a5724ae31cff7.xml (deflated 36%) 2024-08-06T22:14:21.8790310Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-38705126f33ad6c0.xml (deflated 36%) 2024-08-06T22:14:21.8791516Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-010d935aef5b7d1d.xml (deflated 36%) 2024-08-06T22:14:21.8792712Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-6b62b00208e7e511.xml (deflated 36%) 2024-08-06T22:14:21.8793924Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-4b1fe46c07839837.xml (deflated 36%) 2024-08-06T22:14:21.8795131Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-a4195d7677856ddf.xml (deflated 36%) 2024-08-06T22:14:21.8796391Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-22457e22aebb3793.xml (deflated 36%) 2024-08-06T22:14:21.8797596Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-35f4aa8458ab5183.xml (deflated 36%) 2024-08-06T22:14:21.8798804Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-a43c17b7ccea468f.xml (deflated 36%) 2024-08-06T22:14:21.8800013Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-7918a81954a60947.xml (deflated 36%) 2024-08-06T22:14:21.8801204Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-27c219106702cd5a.xml (deflated 36%) 2024-08-06T22:14:21.8802410Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-2d0244c9af3a489a.xml (deflated 36%) 2024-08-06T22:14:21.8803624Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-a993e6a5a0f76d6c.xml (deflated 35%) 2024-08-06T22:14:21.8804826Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-30d91d953d8f3333.xml (deflated 36%) 2024-08-06T22:14:21.8806022Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-a4660db849d92459.xml (deflated 35%) 2024-08-06T22:14:21.8807230Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-538c937a6d716849.xml (deflated 35%) 2024-08-06T22:14:21.8808446Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-d0c3212c0c0a9505.xml (deflated 35%) 2024-08-06T22:14:21.8809648Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-1c4142d5dc639f15.xml (deflated 35%) 2024-08-06T22:14:21.8810843Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-1a52df8015db9ceb.xml (deflated 35%) 2024-08-06T22:14:21.8812055Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-97804f48edbaf4c6.xml (deflated 35%) 2024-08-06T22:14:21.8813261Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-ead92b897c5d6712.xml (deflated 36%) 2024-08-06T22:14:21.8814812Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.test_sharded_tensor_reshard/distributed._shard.sharded_tensor.test_sharded_tensor_reshard-e224801b0be87455.xml (deflated 70%) 2024-08-06T22:14:21.8816419Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.test_sharded_tensor/distributed._shard.sharded_tensor.test_sharded_tensor-cdbc7f7664346047.xml (deflated 95%) 2024-08-06T22:14:21.8817967Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_embedding/distributed._shard.sharded_tensor.ops.test_embedding-975d3ee976da894c.xml (deflated 68%) 2024-08-06T22:14:21.8819531Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_embedding_bag/distributed._shard.sharded_tensor.ops.test_embedding_bag-df6cc5975644231d.xml (deflated 68%) 2024-08-06T22:14:21.8820988Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-778762f3d6fc56f2.xml (deflated 43%) 2024-08-06T22:14:21.8822168Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-4a63a993bf3b1c62.xml (deflated 43%) 2024-08-06T22:14:21.8823357Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-50d0a50f558ed205.xml (deflated 43%) 2024-08-06T22:14:21.8824717Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-fc5aab87c6656961.xml (deflated 43%) 2024-08-06T22:14:21.8825897Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-ca10379adf9865d8.xml (deflated 36%) 2024-08-06T22:14:21.8827063Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-b8e79354ee9e4cc3.xml (deflated 43%) 2024-08-06T22:14:21.8828242Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-9ad7f75f3e6f5bf0.xml (deflated 43%) 2024-08-06T22:14:21.8829424Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-6f679b562e389b03.xml (deflated 43%) 2024-08-06T22:14:21.8830833Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-32abe605cf3f432c.xml (deflated 44%) 2024-08-06T22:14:21.8832010Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-876a5cfc00744d96.xml (deflated 44%) 2024-08-06T22:14:21.8833191Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-c402b53dc0e152cd.xml (deflated 44%) 2024-08-06T22:14:21.8834379Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-8e59408c1c9dcb8e.xml (deflated 44%) 2024-08-06T22:14:21.8835548Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-cf2a264fd8e78472.xml (deflated 44%) 2024-08-06T22:14:21.8836733Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-5910c84d1c6b58d2.xml (deflated 44%) 2024-08-06T22:14:21.8837914Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-75dc0393e3950130.xml (deflated 44%) 2024-08-06T22:14:21.8839081Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-48a96876bd4549cb.xml (deflated 43%) 2024-08-06T22:14:21.8840228Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-9a22ae14b40a716b.xml (deflated 43%) 2024-08-06T22:14:21.8841385Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-201c9533695438e6.xml (deflated 43%) 2024-08-06T22:14:21.8842538Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-a5c5e4b292a59818.xml (deflated 45%) 2024-08-06T22:14:21.8843840Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-31e8afd58299fcc8.xml (deflated 45%) 2024-08-06T22:14:21.8844985Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-28df2d8ed589a860.xml (deflated 46%) 2024-08-06T22:14:21.8846147Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-3edd7b12e7db81f8.xml (deflated 45%) 2024-08-06T22:14:21.8847303Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-1a1bf81140f84218.xml (deflated 46%) 2024-08-06T22:14:21.8848462Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-2567d908677574a5.xml (deflated 46%) 2024-08-06T22:14:21.8849614Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-0df7018b7ec5597f.xml (deflated 43%) 2024-08-06T22:14:21.8850799Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-36185cb1400dbb11.xml (deflated 44%) 2024-08-06T22:14:21.8851978Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-2cfece78e9ad0c27.xml (deflated 43%) 2024-08-06T22:14:21.8853235Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-221f3c7fe543e4bf.xml (deflated 43%) 2024-08-06T22:14:21.8854400Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-2038cdcc2ecc0a49.xml (deflated 45%) 2024-08-06T22:14:21.8855574Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-3f4537acc464e71e.xml (deflated 46%) 2024-08-06T22:14:21.8856748Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-1762cba57e84b97a.xml (deflated 46%) 2024-08-06T22:14:21.8857920Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-f351cc98ceefc713.xml (deflated 46%) 2024-08-06T22:14:21.8859098Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-6a0c5de436e96ae0.xml (deflated 46%) 2024-08-06T22:14:21.8860275Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-21cc834504aa4420.xml (deflated 46%) 2024-08-06T22:14:21.8861517Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-9e5edf123f0197ea.xml (deflated 46%) 2024-08-06T22:14:21.8862683Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-adf747878858adf9.xml (deflated 45%) 2024-08-06T22:14:21.8863856Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-b66f51848902f1b9.xml (deflated 45%) 2024-08-06T22:14:21.8864998Z adding: test/test-reports/python-pytest/distributed.test_launcher/distributed.test_launcher-d694d3bc01ce809d.xml (deflated 40%) 2024-08-06T22:14:21.8866066Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-122522bcfdd0be70.xml (deflated 44%) 2024-08-06T22:14:21.8867087Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-645155477aa39ed0.xml (deflated 37%) 2024-08-06T22:14:21.8868119Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-3eeadbc5cbca7898.xml (deflated 39%) 2024-08-06T22:14:21.8869151Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-50c72c3fb1ea96e1.xml (deflated 43%) 2024-08-06T22:14:21.8870188Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-e74922dbc44436d8.xml (deflated 44%) 2024-08-06T22:14:21.8871208Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-f7d6e88caddc2921.xml (deflated 36%) 2024-08-06T22:14:21.8872344Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-206feb72c0fe8198.xml (deflated 36%) 2024-08-06T22:14:21.8873388Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-db2e768dda87ef14.xml (deflated 36%) 2024-08-06T22:14:21.8874415Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-4b03bd552d88992a.xml (deflated 37%) 2024-08-06T22:14:21.8875449Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-49cdfd5f83c091ad.xml (deflated 37%) 2024-08-06T22:14:21.8876483Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-f7ef3ec20f37f828.xml (deflated 37%) 2024-08-06T22:14:21.8877514Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-ed028c9cb5865079.xml (deflated 37%) 2024-08-06T22:14:21.8878537Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-07fc81ca1bb36c26.xml (deflated 37%) 2024-08-06T22:14:21.8879563Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-88164032c1e1d952.xml (deflated 37%) 2024-08-06T22:14:21.8880593Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-ee5e6d3d230a02cc.xml (deflated 36%) 2024-08-06T22:14:21.8881671Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-5d3407d396b2e719.xml (deflated 43%) 2024-08-06T22:14:21.8882699Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-8f4f42d7d65bee73.xml (deflated 36%) 2024-08-06T22:14:21.8883732Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-9384cbbaa0ad66ce.xml (deflated 43%) 2024-08-06T22:14:21.8884771Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-6ea000d6b9062f5a.xml (deflated 43%) 2024-08-06T22:14:21.8885793Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-8e9511a63d1964bb.xml (deflated 35%) 2024-08-06T22:14:21.8886829Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-137cb1d326e4a6ff.xml (deflated 36%) 2024-08-06T22:14:21.8887863Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-d35563e801e9e82c.xml (deflated 35%) 2024-08-06T22:14:21.8888902Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-d2574ad4eadb1a00.xml (deflated 36%) 2024-08-06T22:14:21.8889926Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-02aadfc78e7b924e.xml (deflated 36%) 2024-08-06T22:14:21.8890965Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-6db20f25504d9bea.xml (deflated 38%) 2024-08-06T22:14:21.8891997Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-6127c8cefe86b0b5.xml (deflated 36%) 2024-08-06T22:14:21.8893025Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-bc47bb259dee0587.xml (deflated 36%) 2024-08-06T22:14:21.8894059Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-4af7c60ec1220d21.xml (deflated 34%) 2024-08-06T22:14:21.8895096Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-de39c6717ebff68c.xml (deflated 35%) 2024-08-06T22:14:21.8896126Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-2988d3175420548a.xml (deflated 36%) 2024-08-06T22:14:21.8897142Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-8d7dfe92de723ff1.xml (deflated 36%) 2024-08-06T22:14:21.8898174Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-d08c8414ea59bc4c.xml (deflated 37%) 2024-08-06T22:14:21.8899270Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-d41992e0dc42036b.xml (deflated 36%) 2024-08-06T22:14:21.8900290Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-6b23fed145815e2f.xml (deflated 36%) 2024-08-06T22:14:21.8901395Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-102e6d91506cf401.xml (deflated 37%) 2024-08-06T22:14:21.8902441Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-3b0feb609c8b604a.xml (deflated 36%) 2024-08-06T22:14:21.8903478Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-adacade249b650d1.xml (deflated 34%) 2024-08-06T22:14:21.8904501Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-d0d41c71a7483f11.xml (deflated 35%) 2024-08-06T22:14:21.8905530Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-346725fbcae4b519.xml (deflated 35%) 2024-08-06T22:14:21.8906572Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-e8fd0d6dfa5c4e61.xml (deflated 38%) 2024-08-06T22:14:21.8907602Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-847998516c83f743.xml (deflated 34%) 2024-08-06T22:14:21.8908684Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-27db076491218d60.xml (deflated 35%) 2024-08-06T22:14:21.8909716Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-ae6e9ef12cf5f1c9.xml (deflated 34%) 2024-08-06T22:14:21.8910748Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-342b7f1048bef921.xml (deflated 34%) 2024-08-06T22:14:21.8911760Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-2b2612823f4d43c2.xml (deflated 34%) 2024-08-06T22:14:21.8912793Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-cd5d4dcfb86df140.xml (deflated 35%) 2024-08-06T22:14:21.8913833Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-72ede2549b640136.xml (deflated 35%) 2024-08-06T22:14:21.8914862Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-759ff73329aa86fa.xml (deflated 35%) 2024-08-06T22:14:21.8915878Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-5c9e1c2d4d50784d.xml (deflated 35%) 2024-08-06T22:14:21.8916914Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-fe4df1773a4fae6a.xml (deflated 35%) 2024-08-06T22:14:21.8917949Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-60f1b38945793dc3.xml (deflated 34%) 2024-08-06T22:14:21.8918964Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-e16d4ded5c4e0dcf.xml (deflated 36%) 2024-08-06T22:14:21.8920006Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-69c688fb74ca9211.xml (deflated 35%) 2024-08-06T22:14:21.8921040Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-2c7597597e606ab9.xml (deflated 34%) 2024-08-06T22:14:21.8922073Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-4ba92ee0a2553576.xml (deflated 36%) 2024-08-06T22:14:21.8923100Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-c0b75897d76d27c2.xml (deflated 35%) 2024-08-06T22:14:21.8924134Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-0b711cbb4c120583.xml (deflated 36%) 2024-08-06T22:14:21.8925164Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-0b398709106eaf68.xml (deflated 35%) 2024-08-06T22:14:21.8926176Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-e9a8f7252e03a050.xml (deflated 37%) 2024-08-06T22:14:21.8927278Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-f6d287bcb994ccac.xml (deflated 36%) 2024-08-06T22:14:21.8928314Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-ad342192eadd5e40.xml (deflated 35%) 2024-08-06T22:14:21.8929350Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-f4758898d21bd410.xml (deflated 36%) 2024-08-06T22:14:21.8930559Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-73e18946dd430cdd.xml (deflated 36%) 2024-08-06T22:14:21.8931597Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-872fe13ac8fa31e0.xml (deflated 35%) 2024-08-06T22:14:21.8932648Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-0fdec97db47db845.xml (deflated 36%) 2024-08-06T22:14:21.8933685Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-85255d6af3e75e90.xml (deflated 35%) 2024-08-06T22:14:21.8934716Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-e88938caf97e1d9b.xml (deflated 35%) 2024-08-06T22:14:21.8935758Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-2256b4aec24c38cc.xml (deflated 35%) 2024-08-06T22:14:21.8936922Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-f51cc6a627b1735e.xml (deflated 35%) 2024-08-06T22:14:21.8937944Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-0529ba7d482e6b69.xml (deflated 35%) 2024-08-06T22:14:21.8938994Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-0fd9baf8ba72eec6.xml (deflated 35%) 2024-08-06T22:14:21.8940039Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-2098af0dd5f6fc51.xml (deflated 35%) 2024-08-06T22:14:21.8941142Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-69f4adda1cc0e9bc.xml (deflated 35%) 2024-08-06T22:14:21.8942165Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-b915673c4601a5ac.xml (deflated 35%) 2024-08-06T22:14:21.8943198Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-0881ef21e08bdb28.xml (deflated 36%) 2024-08-06T22:14:21.8944243Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-493fe65a806a7971.xml (deflated 35%) 2024-08-06T22:14:21.8945352Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-9eeac53a6dfb96d1.xml (deflated 36%) 2024-08-06T22:14:21.8946381Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-683d7e1622c12377.xml (deflated 36%) 2024-08-06T22:14:21.8947412Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-b41628602a9169ca.xml (deflated 35%) 2024-08-06T22:14:21.8948450Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-185f8cc468d581e6.xml (deflated 35%) 2024-08-06T22:14:21.8949474Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-5e42cda234252d6b.xml (deflated 35%) 2024-08-06T22:14:21.8950511Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-cc0171671b275e3d.xml (deflated 35%) 2024-08-06T22:14:21.8951546Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-0aadc812bf4a30c2.xml (deflated 37%) 2024-08-06T22:14:21.8952577Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-4b0b56b98612b189.xml (deflated 33%) 2024-08-06T22:14:21.8953589Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-6d566662cb139d65.xml (deflated 33%) 2024-08-06T22:14:21.8954613Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-8a10da758e167b79.xml (deflated 35%) 2024-08-06T22:14:21.8955873Z adding: test/test-reports/python-pytest/distributed.elastic.events.lib_test/distributed.elastic.events.lib_test-80a78837d6225fc2.xml (deflated 76%) 2024-08-06T22:14:21.8957156Z adding: test/test-reports/python-pytest/distributed.elastic.metrics.api_test/distributed.elastic.metrics.api_test-3eb8d85461e28144.xml (deflated 60%) 2024-08-06T22:14:21.8958556Z adding: test/test-reports/python-pytest/distributed.elastic.multiprocessing.api_test/distributed.elastic.multiprocessing.api_test-2a2672814f19122e.xml (deflated 88%) 2024-08-06T22:14:21.8960024Z adding: test/test-reports/python-pytest/distributed.elastic.timer.local_timer_example/distributed.elastic.timer.local_timer_example-8815c792bcf41c5d.xml (deflated 51%) 2024-08-06T22:14:21.8961512Z adding: test/test-reports/python-pytest/distributed.elastic.timer.local_timer_test/distributed.elastic.timer.local_timer_test-72855696e2467689.xml (deflated 80%) 2024-08-06T22:14:21.8963005Z adding: test/test-reports/python-pytest/distributed.elastic.utils.distributed_test/distributed.elastic.utils.distributed_test-7b69c3e0ed4ae11f.xml (deflated 78%) 2024-08-06T22:14:21.8964445Z adding: test/test-reports/python-pytest/distributed.elastic.utils.logging_test/distributed.elastic.utils.logging_test-a8b0cfb13e1f45b2.xml (deflated 51%) 2024-08-06T22:14:21.8965848Z adding: test/test-reports/python-pytest/distributed.elastic.utils.util_test/distributed.elastic.utils.util_test-678d4008ba630790.xml (deflated 80%) 2024-08-06T22:14:21.8967102Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8a21fb1126612f6b.xml (deflated 43%) 2024-08-06T22:14:21.8968304Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ccf381ee87e4ae8f.xml (deflated 43%) 2024-08-06T22:14:21.8969515Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-948d2feccbe3ddcb.xml (deflated 37%) 2024-08-06T22:14:21.8970745Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f38f733326e4f244.xml (deflated 37%) 2024-08-06T22:14:21.8971961Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-47825b00cd13eab7.xml (deflated 41%) 2024-08-06T22:14:21.8973169Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-10b463e616cb1174.xml (deflated 41%) 2024-08-06T22:14:21.8974364Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c8125b48171f330f.xml (deflated 41%) 2024-08-06T22:14:21.8975578Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9b3e6db9f85ac477.xml (deflated 42%) 2024-08-06T22:14:21.8976786Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-82828ccce0881cc4.xml (deflated 36%) 2024-08-06T22:14:21.8977979Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0045086cbaf8c90a.xml (deflated 36%) 2024-08-06T22:14:21.8979175Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7c0bd3f1a1ad1524.xml (deflated 35%) 2024-08-06T22:14:21.8980382Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-037d374e83a71cf5.xml (deflated 35%) 2024-08-06T22:14:21.8981652Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b3ce215312131928.xml (deflated 43%) 2024-08-06T22:14:21.8982847Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d415ff125a39387c.xml (deflated 45%) 2024-08-06T22:14:21.8984052Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0b6f0b4106074b02.xml (deflated 35%) 2024-08-06T22:14:21.8985386Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-531e8ad6181bc788.xml (deflated 35%) 2024-08-06T22:14:21.8986599Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-98d3bcd22d38177b.xml (deflated 35%) 2024-08-06T22:14:21.8987795Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e255f1e2ddda60bd.xml (deflated 35%) 2024-08-06T22:14:21.8989002Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6ccc355a94f18ae1.xml (deflated 36%) 2024-08-06T22:14:21.8990209Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c0b36d00ea277c78.xml (deflated 36%) 2024-08-06T22:14:21.8991401Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f43a05a81132de26.xml (deflated 35%) 2024-08-06T22:14:21.8992591Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-079fa0cacc9443a0.xml (deflated 36%) 2024-08-06T22:14:21.8993795Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ca55aaa7d3f19043.xml (deflated 36%) 2024-08-06T22:14:21.8995084Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d507312cef5056b8.xml (deflated 35%) 2024-08-06T22:14:21.8996283Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c9435976a4ade9c5.xml (deflated 35%) 2024-08-06T22:14:21.8997470Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6fba6c2610b3107c.xml (deflated 35%) 2024-08-06T22:14:21.8998670Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d74361e35a38a367.xml (deflated 35%) 2024-08-06T22:14:21.8999871Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c0dc2d37c6750c28.xml (deflated 35%) 2024-08-06T22:14:21.9001049Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c54d6cd0245c2b5d.xml (deflated 36%) 2024-08-06T22:14:21.9002249Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-542b5093cd33c4e3.xml (deflated 35%) 2024-08-06T22:14:21.9003448Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5c5e6cccc73121f9.xml (deflated 36%) 2024-08-06T22:14:21.9004647Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c8a82d0436226d13.xml (deflated 35%) 2024-08-06T22:14:21.9005833Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-093be0f63bb32a30.xml (deflated 42%) 2024-08-06T22:14:21.9007046Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-031abdb252876f63.xml (deflated 43%) 2024-08-06T22:14:21.9008248Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ecfff6f9b90e3409.xml (deflated 43%) 2024-08-06T22:14:21.9009447Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-200b716c428564c2.xml (deflated 43%) 2024-08-06T22:14:21.9010629Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3c3ec0970f1c4212.xml (deflated 44%) 2024-08-06T22:14:21.9011829Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4e81009e2bf2d612.xml (deflated 44%) 2024-08-06T22:14:21.9013113Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bdc41c8e54eb4638.xml (deflated 44%) 2024-08-06T22:14:21.9014303Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0db5e80917e24b66.xml (deflated 45%) 2024-08-06T22:14:21.9015486Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-46375ab3ec5a7f83.xml (deflated 44%) 2024-08-06T22:14:21.9016694Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-35451ed52cb96b29.xml (deflated 44%) 2024-08-06T22:14:21.9017884Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-aa51e3b397e17aae.xml (deflated 43%) 2024-08-06T22:14:21.9019077Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-844cf1d280c79c78.xml (deflated 36%) 2024-08-06T22:14:21.9020263Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-38df6afcb918e3aa.xml (deflated 42%) 2024-08-06T22:14:21.9021539Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4bbf6ae6b758aa61.xml (deflated 42%) 2024-08-06T22:14:21.9022736Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e2de0c10412a131f.xml (deflated 36%) 2024-08-06T22:14:21.9023996Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1c8f9d4a674603f8.xml (deflated 35%) 2024-08-06T22:14:21.9025203Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7ea4716bf7f902fa.xml (deflated 42%) 2024-08-06T22:14:21.9026401Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-aa311e0721fed8d2.xml (deflated 42%) 2024-08-06T22:14:21.9027604Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6082246fbb1a0db3.xml (deflated 36%) 2024-08-06T22:14:21.9028790Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ba4cbd42f1b50f6e.xml (deflated 47%) 2024-08-06T22:14:21.9029985Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ec7050bd18286dba.xml (deflated 47%) 2024-08-06T22:14:21.9031394Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c0facc9ce2355e28.xml (deflated 47%) 2024-08-06T22:14:21.9032588Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-03a6578ea5698498.xml (deflated 42%) 2024-08-06T22:14:21.9033777Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6ba6ad1cbae542e8.xml (deflated 42%) 2024-08-06T22:14:21.9034980Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ff834fb6890b41e8.xml (deflated 42%) 2024-08-06T22:14:21.9036181Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-52c60897c4d3e7db.xml (deflated 42%) 2024-08-06T22:14:21.9037378Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5d29842f67a0c747.xml (deflated 37%) 2024-08-06T22:14:21.9038560Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-938b83f390f201f0.xml (deflated 42%) 2024-08-06T22:14:21.9039754Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4b28a8fc312d1cc0.xml (deflated 43%) 2024-08-06T22:14:21.9040948Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cbb3b09e0731fb66.xml (deflated 42%) 2024-08-06T22:14:21.9042300Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-db4422aca6e941fb.xml (deflated 36%) 2024-08-06T22:14:21.9043495Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bebe132a00caca65.xml (deflated 42%) 2024-08-06T22:14:21.9044699Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9222bc5e372e04fc.xml (deflated 43%) 2024-08-06T22:14:21.9045911Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a0d3f7a3e50abcc3.xml (deflated 43%) 2024-08-06T22:14:21.9047096Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-96b3b6e7e77b69bc.xml (deflated 42%) 2024-08-06T22:14:21.9048298Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9eb78b0f148812a6.xml (deflated 42%) 2024-08-06T22:14:21.9049505Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-db29b9b6c027a1d4.xml (deflated 42%) 2024-08-06T22:14:21.9050696Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9f243931814b3de1.xml (deflated 42%) 2024-08-06T22:14:21.9051959Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2627cb5dccba0244.xml (deflated 42%) 2024-08-06T22:14:21.9053158Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2d6a7c3a9efbefa6.xml (deflated 43%) 2024-08-06T22:14:21.9054360Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-aaffda32369e3641.xml (deflated 43%) 2024-08-06T22:14:21.9055561Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0caffff420e528f2.xml (deflated 36%) 2024-08-06T22:14:21.9056747Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3391945231924940.xml (deflated 35%) 2024-08-06T22:14:21.9057935Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-21be761c0338e582.xml (deflated 42%) 2024-08-06T22:14:21.9059130Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f25fc0382277a79d.xml (deflated 36%) 2024-08-06T22:14:21.9060380Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8d31603dc24f0182.xml (deflated 36%) 2024-08-06T22:14:21.9061660Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2d8850ff2d7f1ed3.xml (deflated 42%) 2024-08-06T22:14:21.9062875Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ac376b9c664f37bd.xml (deflated 35%) 2024-08-06T22:14:21.9064090Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c8edce9871a4e7a7.xml (deflated 35%) 2024-08-06T22:14:21.9065286Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0dfe288a0b6871ea.xml (deflated 36%) 2024-08-06T22:14:21.9066492Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-51dee3c54d041461.xml (deflated 35%) 2024-08-06T22:14:21.9067752Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d56247b156838fdd.xml (deflated 35%) 2024-08-06T22:14:21.9068930Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-804ab91a74b74dcb.xml (deflated 44%) 2024-08-06T22:14:21.9070118Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4d12a588ce002b76.xml (deflated 43%) 2024-08-06T22:14:21.9071380Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d8d97f08e80e1826.xml (deflated 43%) 2024-08-06T22:14:21.9072578Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-af22b56d57c8d944.xml (deflated 43%) 2024-08-06T22:14:21.9073769Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f83d77256e51a6ff.xml (deflated 42%) 2024-08-06T22:14:21.9074979Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6467964358c044a4.xml (deflated 35%) 2024-08-06T22:14:21.9076178Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-91d013c887c1307c.xml (deflated 35%) 2024-08-06T22:14:21.9077371Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1a7ed071bbf0cf59.xml (deflated 36%) 2024-08-06T22:14:21.9078562Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-509381557b33b4f5.xml (deflated 36%) 2024-08-06T22:14:21.9079759Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3b83b2dad8390813.xml (deflated 43%) 2024-08-06T22:14:21.9081033Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2f55632f349d99a2.xml (deflated 42%) 2024-08-06T22:14:21.9082230Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a88ce5592be5fafc.xml (deflated 41%) 2024-08-06T22:14:21.9083425Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-468baaf8b79df19d.xml (deflated 41%) 2024-08-06T22:14:21.9084630Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e5879ca4faf2c950.xml (deflated 42%) 2024-08-06T22:14:21.9085834Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-96c436592fc3c84e.xml (deflated 35%) 2024-08-06T22:14:21.9087028Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-287c16b0b4e59043.xml (deflated 36%) 2024-08-06T22:14:21.9088222Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7b2d75968da69d10.xml (deflated 36%) 2024-08-06T22:14:21.9089428Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8a9f6b78a9586cd3.xml (deflated 42%) 2024-08-06T22:14:21.9090649Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a72ba097b090021e.xml (deflated 42%) 2024-08-06T22:14:21.9091839Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-eba2054ad7373b70.xml (deflated 43%) 2024-08-06T22:14:21.9093058Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b991b4937f77ded7.xml (deflated 36%) 2024-08-06T22:14:21.9094272Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bae37af149ad11e9.xml (deflated 36%) 2024-08-06T22:14:21.9095496Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-24cb5c6c5246aa01.xml (deflated 41%) 2024-08-06T22:14:21.9096689Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-729df5328ad935a7.xml (deflated 41%) 2024-08-06T22:14:21.9097895Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-af91114d932fa748.xml (deflated 41%) 2024-08-06T22:14:21.9099179Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cba246f630c7a89f.xml (deflated 41%) 2024-08-06T22:14:21.9100398Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-be2de44c4c7312ef.xml (deflated 36%) 2024-08-06T22:14:21.9101651Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7a7a23507c796fbc.xml (deflated 35%) 2024-08-06T22:14:21.9102859Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e53d7d0449ab1163.xml (deflated 35%) 2024-08-06T22:14:21.9104075Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3142aec2bdf3ac96.xml (deflated 36%) 2024-08-06T22:14:21.9105284Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-dba1977e1fd40135.xml (deflated 43%) 2024-08-06T22:14:21.9106474Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6aa0aef1be8ce151.xml (deflated 45%) 2024-08-06T22:14:21.9107681Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-41c97ffdf0909367.xml (deflated 35%) 2024-08-06T22:14:21.9108883Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-20d9f36e2cf9f606.xml (deflated 35%) 2024-08-06T22:14:21.9110155Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-11c00b88930a1b92.xml (deflated 36%) 2024-08-06T22:14:21.9111339Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-139d99eaa37043e5.xml (deflated 35%) 2024-08-06T22:14:21.9112552Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c21ee8aba38431a9.xml (deflated 35%) 2024-08-06T22:14:21.9113757Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-de86ee2da4323e38.xml (deflated 35%) 2024-08-06T22:14:21.9114935Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d43834a11248894d.xml (deflated 35%) 2024-08-06T22:14:21.9116132Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5dbd3af37db2a183.xml (deflated 36%) 2024-08-06T22:14:21.9117339Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e106a328d51e05d7.xml (deflated 36%) 2024-08-06T22:14:21.9118532Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-934a2a2676b56e19.xml (deflated 35%) 2024-08-06T22:14:21.9119712Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7efe703997a1e999.xml (deflated 36%) 2024-08-06T22:14:21.9120916Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5fcec299af370c75.xml (deflated 36%) 2024-08-06T22:14:21.9122120Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6b4cbdfbdf40b8e5.xml (deflated 36%) 2024-08-06T22:14:21.9123323Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ce6a4b42032a30c9.xml (deflated 35%) 2024-08-06T22:14:21.9124502Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c619c3012f13fc43.xml (deflated 36%) 2024-08-06T22:14:21.9125694Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-dcedfd439921edc0.xml (deflated 36%) 2024-08-06T22:14:21.9126895Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-eb8ddb7538ba36ab.xml (deflated 36%) 2024-08-06T22:14:21.9128164Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5f38efaf89ce400e.xml (deflated 35%) 2024-08-06T22:14:21.9129342Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d622567e70994b4e.xml (deflated 42%) 2024-08-06T22:14:21.9130729Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3ebc21da8dac7405.xml (deflated 43%) 2024-08-06T22:14:21.9131927Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d42dd63d5b4416d5.xml (deflated 43%) 2024-08-06T22:14:21.9133117Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9236d18721e26241.xml (deflated 43%) 2024-08-06T22:14:21.9134292Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-06a48c129a076c59.xml (deflated 44%) 2024-08-06T22:14:21.9135489Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5abe59423ad1936d.xml (deflated 44%) 2024-08-06T22:14:21.9136683Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1f2d595868d2aea9.xml (deflated 44%) 2024-08-06T22:14:21.9137991Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-350f36923bf2f2f3.xml (deflated 44%) 2024-08-06T22:14:21.9139193Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a8b3fb10fcacd971.xml (deflated 44%) 2024-08-06T22:14:21.9140382Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-368361940164521a.xml (deflated 44%) 2024-08-06T22:14:21.9141630Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0920ae778c7c93c8.xml (deflated 43%) 2024-08-06T22:14:21.9142819Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-dceb0501e80a54b5.xml (deflated 36%) 2024-08-06T22:14:21.9144007Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-45616ffb793eed22.xml (deflated 42%) 2024-08-06T22:14:21.9145218Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-551150bc050b33fc.xml (deflated 42%) 2024-08-06T22:14:21.9146412Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-030d4a55738dcb2d.xml (deflated 36%) 2024-08-06T22:14:21.9147596Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5e4bdb18e9ee4691.xml (deflated 35%) 2024-08-06T22:14:21.9148800Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2ed6d9d772bb17b1.xml (deflated 42%) 2024-08-06T22:14:21.9149996Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a011f8b42ef75fdc.xml (deflated 42%) 2024-08-06T22:14:21.9151196Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-53cd63bdaf05f552.xml (deflated 36%) 2024-08-06T22:14:21.9152381Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0e7b9774a17d97c1.xml (deflated 47%) 2024-08-06T22:14:21.9153580Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-665e3d7e86f0ead3.xml (deflated 47%) 2024-08-06T22:14:21.9154777Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9c4723a8c124bd03.xml (deflated 47%) 2024-08-06T22:14:21.9155979Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-24132b1823c54f44.xml (deflated 42%) 2024-08-06T22:14:21.9157272Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1b3cb48c24cfb50b.xml (deflated 42%) 2024-08-06T22:14:21.9158469Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c8df3f355feb1206.xml (deflated 42%) 2024-08-06T22:14:21.9159677Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-76a18bce50f9fa31.xml (deflated 42%) 2024-08-06T22:14:21.9160871Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a247eb7118f6ed38.xml (deflated 35%) 2024-08-06T22:14:21.9162068Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3ab030efd8477d63.xml (deflated 42%) 2024-08-06T22:14:21.9163267Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-665b90116b81871d.xml (deflated 43%) 2024-08-06T22:14:21.9164470Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cd55c5c64fcffebf.xml (deflated 42%) 2024-08-06T22:14:21.9165659Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d006f69d1422d57e.xml (deflated 35%) 2024-08-06T22:14:21.9166915Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a3d46a0021c60e45.xml (deflated 42%) 2024-08-06T22:14:21.9168106Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-36efa4d271586f1a.xml (deflated 43%) 2024-08-06T22:14:21.9169297Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fce177e4f85a367e.xml (deflated 43%) 2024-08-06T22:14:21.9170476Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-61876ebfc56e969a.xml (deflated 42%) 2024-08-06T22:14:21.9171676Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-34ea9a72b88a89ab.xml (deflated 42%) 2024-08-06T22:14:21.9172869Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e0c6e707276e27a2.xml (deflated 42%) 2024-08-06T22:14:21.9174072Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-490da30146fc87ed.xml (deflated 42%) 2024-08-06T22:14:21.9175254Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3082c4d7a5ad2078.xml (deflated 42%) 2024-08-06T22:14:21.9176450Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-66a48bc88dfe26d0.xml (deflated 43%) 2024-08-06T22:14:21.9177653Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ba5206a8297576e4.xml (deflated 43%) 2024-08-06T22:14:21.9178892Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-00f31915e716bf4d.xml (deflated 35%) 2024-08-06T22:14:21.9180072Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-db244f985f05ecb6.xml (deflated 36%) 2024-08-06T22:14:21.9181342Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-88280f357d17dcde.xml (deflated 43%) 2024-08-06T22:14:21.9182547Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-03ee8d4ad440e46c.xml (deflated 36%) 2024-08-06T22:14:21.9183752Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9ac5c9f433a5012c.xml (deflated 36%) 2024-08-06T22:14:21.9184965Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-743b10db0f38de8c.xml (deflated 42%) 2024-08-06T22:14:21.9186246Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-15505c9c273d2459.xml (deflated 36%) 2024-08-06T22:14:21.9187464Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-17560a1e1b42e5d9.xml (deflated 35%) 2024-08-06T22:14:21.9188685Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c80fa10b6358e8aa.xml (deflated 36%) 2024-08-06T22:14:21.9189907Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-83759746ae601bce.xml (deflated 36%) 2024-08-06T22:14:21.9191110Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cb069235d9e52cc7.xml (deflated 35%) 2024-08-06T22:14:21.9192321Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-285a5156a5be9f8e.xml (deflated 44%) 2024-08-06T22:14:21.9193522Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9ea270df6bceb0a8.xml (deflated 42%) 2024-08-06T22:14:21.9194746Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b520b1ab0d635707.xml (deflated 43%) 2024-08-06T22:14:21.9196073Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ee31c5c147ece2c6.xml (deflated 43%) 2024-08-06T22:14:21.9197291Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-86352b7fb399ff05.xml (deflated 42%) 2024-08-06T22:14:21.9198493Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-99a4e75da6a7a4cc.xml (deflated 35%) 2024-08-06T22:14:21.9199732Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1915889648ef5ea2.xml (deflated 35%) 2024-08-06T22:14:21.9200933Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5bd235725382777b.xml (deflated 36%) 2024-08-06T22:14:21.9202134Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c116ab1a72c7b83f.xml (deflated 36%) 2024-08-06T22:14:21.9203345Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bfe92ce8dea80a3a.xml (deflated 43%) 2024-08-06T22:14:21.9204558Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3f2a336d448498e3.xml (deflated 42%) 2024-08-06T22:14:21.9205770Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c1fe0d76f3ad9744.xml (deflated 41%) 2024-08-06T22:14:21.9206973Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ca543cb9c14611b3.xml (deflated 41%) 2024-08-06T22:14:21.9208185Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2fcaed9d68904a22.xml (deflated 42%) 2024-08-06T22:14:21.9209392Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-531d19d8c3849199.xml (deflated 36%) 2024-08-06T22:14:21.9210619Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f88e9d144744c2b1.xml (deflated 36%) 2024-08-06T22:14:21.9211894Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7d2f54c75d9b195d.xml (deflated 36%) 2024-08-06T22:14:21.9213097Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2966449e343e9667.xml (deflated 42%) 2024-08-06T22:14:21.9214379Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c5d387e4ddb58106.xml (deflated 36%) 2024-08-06T22:14:21.9215573Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d1eb2b912a340fcb.xml (deflated 41%) 2024-08-06T22:14:21.9216770Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4e0149d31ce7ea1d.xml (deflated 43%) 2024-08-06T22:14:21.9217965Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6629a63297652707.xml (deflated 42%) 2024-08-06T22:14:21.9219165Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c57aff4df0e4e3af.xml (deflated 35%) 2024-08-06T22:14:21.9220369Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-28f8e98f650466c7.xml (deflated 36%) 2024-08-06T22:14:21.9221630Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f9d2d136bab528d7.xml (deflated 36%) 2024-08-06T22:14:21.9222842Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c5b4aecb8cdebd5d.xml (deflated 35%) 2024-08-06T22:14:21.9224122Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a57bc9a51fb3a32a.xml (deflated 36%) 2024-08-06T22:14:21.9225315Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f78a30e4999a7fa6.xml (deflated 45%) 2024-08-06T22:14:21.9226516Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ffa781731723926f.xml (deflated 36%) 2024-08-06T22:14:21.9227714Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-47b64e5dd48c76bc.xml (deflated 36%) 2024-08-06T22:14:21.9228910Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-353b65d7c00ad644.xml (deflated 35%) 2024-08-06T22:14:21.9230258Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9d877444a03651fe.xml (deflated 35%) 2024-08-06T22:14:21.9231469Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c94b52bc66d7fada.xml (deflated 35%) 2024-08-06T22:14:21.9232672Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4a855e8adb1e4c8a.xml (deflated 35%) 2024-08-06T22:14:21.9233873Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0b906bc05260f2a0.xml (deflated 36%) 2024-08-06T22:14:21.9235050Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-488119fee7078501.xml (deflated 42%) 2024-08-06T22:14:21.9236246Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4668785d8ca3164f.xml (deflated 43%) 2024-08-06T22:14:21.9237435Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-279e09853d0e051b.xml (deflated 44%) 2024-08-06T22:14:21.9238639Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1dbacae6f3b5a0d0.xml (deflated 44%) 2024-08-06T22:14:21.9239823Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ab1814f81628499b.xml (deflated 44%) 2024-08-06T22:14:21.9241021Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c8fa6caacea9c3bb.xml (deflated 44%) 2024-08-06T22:14:21.9242227Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-987ba1bd44524697.xml (deflated 42%) 2024-08-06T22:14:21.9243603Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-343e9536ded6e1b2.xml (deflated 35%) 2024-08-06T22:14:21.9244791Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-56eaa12b42c5c54f.xml (deflated 42%) 2024-08-06T22:14:21.9245992Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-77f986abef367a4d.xml (deflated 35%) 2024-08-06T22:14:21.9247198Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-daa5e7dc9fe2a2eb.xml (deflated 35%) 2024-08-06T22:14:21.9248391Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bcaa2bf2b2463331.xml (deflated 41%) 2024-08-06T22:14:21.9249596Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-79477b40947cb6db.xml (deflated 42%) 2024-08-06T22:14:21.9250791Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f97345e13703b35a.xml (deflated 42%) 2024-08-06T22:14:21.9251978Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8602799fef5f62c9.xml (deflated 35%) 2024-08-06T22:14:21.9253248Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fcec33d6ed65344f.xml (deflated 42%) 2024-08-06T22:14:21.9254450Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-be2d2d55ec7a8f2e.xml (deflated 41%) 2024-08-06T22:14:21.9255646Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-65116e504dfecb49.xml (deflated 42%) 2024-08-06T22:14:21.9256840Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9a8b0801c0be52c8.xml (deflated 42%) 2024-08-06T22:14:21.9258025Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0c8d643cbe1215da.xml (deflated 42%) 2024-08-06T22:14:21.9259219Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7e85e20c782258ac.xml (deflated 42%) 2024-08-06T22:14:21.9260418Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-276600f0b4be7f91.xml (deflated 43%) 2024-08-06T22:14:21.9261674Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-76cbd7a9e606db17.xml (deflated 42%) 2024-08-06T22:14:21.9262866Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-323c6ff4fae16252.xml (deflated 42%) 2024-08-06T22:14:21.9264071Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-de0cd99aee120d7b.xml (deflated 42%) 2024-08-06T22:14:21.9265286Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f4aacc9834a97d2f.xml (deflated 42%) 2024-08-06T22:14:21.9266485Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-50e65569bde45418.xml (deflated 42%) 2024-08-06T22:14:21.9267667Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0f09265cbeac2615.xml (deflated 42%) 2024-08-06T22:14:21.9268859Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-25c0037533240644.xml (deflated 43%) 2024-08-06T22:14:21.9270055Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-89c564bac633c644.xml (deflated 41%) 2024-08-06T22:14:21.9271235Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-55847c73e7f9e238.xml (deflated 42%) 2024-08-06T22:14:21.9272608Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-acfb56fe9cfbbae4.xml (deflated 42%) 2024-08-06T22:14:21.9273812Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6038c44ad6449b96.xml (deflated 42%) 2024-08-06T22:14:21.9275010Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f2717e705ecaad32.xml (deflated 42%) 2024-08-06T22:14:21.9276191Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4ff99d7846c962bf.xml (deflated 42%) 2024-08-06T22:14:21.9277389Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-377371fae73883af.xml (deflated 43%) 2024-08-06T22:14:21.9278584Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a260c204782c74b0.xml (deflated 43%) 2024-08-06T22:14:21.9279773Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e7d96f076310f5ea.xml (deflated 43%) 2024-08-06T22:14:21.9280959Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3ff77a0e1d4eab05.xml (deflated 42%) 2024-08-06T22:14:21.9282219Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-dbbb55c73de4103e.xml (deflated 36%) 2024-08-06T22:14:21.9283414Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8b58cb5c2e379422.xml (deflated 35%) 2024-08-06T22:14:21.9284610Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c88d0b551c442abf.xml (deflated 36%) 2024-08-06T22:14:21.9285796Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cc7fdc0b99465465.xml (deflated 35%) 2024-08-06T22:14:21.9286982Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-016bb427c22996d6.xml (deflated 42%) 2024-08-06T22:14:21.9288172Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-26b2872c1e638f15.xml (deflated 36%) 2024-08-06T22:14:21.9289364Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-52669cf217639a83.xml (deflated 44%) 2024-08-06T22:14:21.9290542Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d1ca8bfe990e7a2a.xml (deflated 45%) 2024-08-06T22:14:21.9291739Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-70361cb4605ce0c9.xml (deflated 45%) 2024-08-06T22:14:21.9292935Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-80c47c448c5c6a4b.xml (deflated 41%) 2024-08-06T22:14:21.9294110Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-14a951f0d70a7f24.xml (deflated 42%) 2024-08-06T22:14:21.9295298Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e7c9b7436a5365a4.xml (deflated 42%) 2024-08-06T22:14:21.9296490Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-79f1d62e921062a0.xml (deflated 41%) 2024-08-06T22:14:21.9297687Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6c5b9a89f99a9811.xml (deflated 41%) 2024-08-06T22:14:21.9298873Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d068676bab456efb.xml (deflated 41%) 2024-08-06T22:14:21.9300146Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ec7e98777ea9c80d.xml (deflated 35%) 2024-08-06T22:14:21.9301424Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-067263b0136e0fd9.xml (deflated 35%) 2024-08-06T22:14:21.9302626Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-94a633d7798e81c8.xml (deflated 35%) 2024-08-06T22:14:21.9303809Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0d913a22e80ffa60.xml (deflated 35%) 2024-08-06T22:14:21.9304993Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-55ad02f5c965af70.xml (deflated 45%) 2024-08-06T22:14:21.9326281Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-86048de609df39f5.xml (deflated 42%) 2024-08-06T22:14:21.9327556Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7a233600d834b678.xml (deflated 36%) 2024-08-06T22:14:21.9328743Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c172810da06df27f.xml (deflated 36%) 2024-08-06T22:14:21.9330361Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8ae490e4a899206a.xml (deflated 36%) 2024-08-06T22:14:21.9331569Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-08e6823e20224ee6.xml (deflated 42%) 2024-08-06T22:14:21.9332764Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bc78490664348910.xml (deflated 36%) 2024-08-06T22:14:21.9333941Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9e13db6e112f024f.xml (deflated 35%) 2024-08-06T22:14:21.9335140Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5554a72a72d88607.xml (deflated 36%) 2024-08-06T22:14:21.9336339Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-adaeb34f28b9e2ca.xml (deflated 35%) 2024-08-06T22:14:21.9337532Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-34aba007de1790c0.xml (deflated 36%) 2024-08-06T22:14:21.9338732Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-032aa9a7ccc0e337.xml (deflated 36%) 2024-08-06T22:14:21.9339929Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-48e66202ad6af2be.xml (deflated 42%) 2024-08-06T22:14:21.9341184Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-454fe20f23845812.xml (deflated 42%) 2024-08-06T22:14:21.9342368Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4b920f746e554058.xml (deflated 43%) 2024-08-06T22:14:21.9343567Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ca506f540b372e21.xml (deflated 42%) 2024-08-06T22:14:21.9344767Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e9d17642f320ef64.xml (deflated 36%) 2024-08-06T22:14:21.9345964Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-664e5b1c3f1bb628.xml (deflated 41%) 2024-08-06T22:14:21.9347143Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-89784e92e1e141d7.xml (deflated 43%) 2024-08-06T22:14:21.9348336Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a5e92aae21dd4613.xml (deflated 42%) 2024-08-06T22:14:21.9349699Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-885c20e4bbd18395.xml (deflated 35%) 2024-08-06T22:14:21.9350893Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2818edeed50cc284.xml (deflated 36%) 2024-08-06T22:14:21.9352089Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a0e8f53ab8d90633.xml (deflated 35%) 2024-08-06T22:14:21.9353291Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9edda8d54ca165f6.xml (deflated 36%) 2024-08-06T22:14:21.9354493Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1a355ff4abd6fe0f.xml (deflated 36%) 2024-08-06T22:14:21.9355687Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c622ff3211b5421d.xml (deflated 45%) 2024-08-06T22:14:21.9356864Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e388ef6f7c9b55ba.xml (deflated 36%) 2024-08-06T22:14:21.9358061Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a0b7f2082bf3a1fd.xml (deflated 36%) 2024-08-06T22:14:21.9359350Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f0d0cab225d3cbc8.xml (deflated 36%) 2024-08-06T22:14:21.9360528Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9557ab95668e9cf7.xml (deflated 36%) 2024-08-06T22:14:21.9361711Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a3d25d254812af4d.xml (deflated 36%) 2024-08-06T22:14:21.9362898Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c38f8767f5412754.xml (deflated 35%) 2024-08-06T22:14:21.9364089Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7e0c81feee5848c0.xml (deflated 35%) 2024-08-06T22:14:21.9365275Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5bf1ba24bd495548.xml (deflated 42%) 2024-08-06T22:14:21.9366474Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6e64673ce43f1db3.xml (deflated 43%) 2024-08-06T22:14:21.9367667Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4e90d4b74510bd95.xml (deflated 44%) 2024-08-06T22:14:21.9368860Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8f140c4fb90e1c50.xml (deflated 44%) 2024-08-06T22:14:21.9370077Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-746bbf86ac2bcb32.xml (deflated 44%) 2024-08-06T22:14:21.9371359Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cd9158b20e693eb6.xml (deflated 45%) 2024-08-06T22:14:21.9372587Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-33251f19e75d8eff.xml (deflated 42%) 2024-08-06T22:14:21.9373799Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1d9d69bbafd085d6.xml (deflated 35%) 2024-08-06T22:14:21.9374978Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2a7cf8a965f722c3.xml (deflated 42%) 2024-08-06T22:14:21.9376164Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-65d07049a665f62d.xml (deflated 36%) 2024-08-06T22:14:21.9377358Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1c170147a5169b21.xml (deflated 35%) 2024-08-06T22:14:21.9378608Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-912f1f6159019de0.xml (deflated 41%) 2024-08-06T22:14:21.9379788Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-62838e640dbc278d.xml (deflated 42%) 2024-08-06T22:14:21.9381084Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-643e67ea08217d7d.xml (deflated 42%) 2024-08-06T22:14:21.9382286Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-dd7aa3aa197e2348.xml (deflated 35%) 2024-08-06T22:14:21.9383470Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a0269863ad60475e.xml (deflated 42%) 2024-08-06T22:14:21.9384678Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-be6326a2d6625481.xml (deflated 42%) 2024-08-06T22:14:21.9385878Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5fe034a9dcdaa57a.xml (deflated 42%) 2024-08-06T22:14:21.9387075Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9cd14d78ba6f288e.xml (deflated 42%) 2024-08-06T22:14:21.9388329Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f27da85a779d6a6a.xml (deflated 42%) 2024-08-06T22:14:21.9389524Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4e2f1bcfe0391a8b.xml (deflated 42%) 2024-08-06T22:14:21.9390722Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ff84d4152a3a8a9b.xml (deflated 43%) 2024-08-06T22:14:21.9391925Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ce645c9f959914b8.xml (deflated 42%) 2024-08-06T22:14:21.9393196Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e70ece320b278a63.xml (deflated 41%) 2024-08-06T22:14:21.9394389Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1b3cb5e23165b7a7.xml (deflated 42%) 2024-08-06T22:14:21.9395586Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-938b7ecb7c9f3156.xml (deflated 42%) 2024-08-06T22:14:21.9396777Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0c3f5815be18861d.xml (deflated 42%) 2024-08-06T22:14:21.9397962Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f0c31ec0bb96651e.xml (deflated 42%) 2024-08-06T22:14:21.9399158Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fcd4c73b672f1980.xml (deflated 43%) 2024-08-06T22:14:21.9400343Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e398d643937032f8.xml (deflated 41%) 2024-08-06T22:14:21.9401535Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d7044d5ce95b8c5c.xml (deflated 42%) 2024-08-06T22:14:21.9402720Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9d72d30fe7e20029.xml (deflated 42%) 2024-08-06T22:14:21.9403912Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d8e1b446c240f24c.xml (deflated 42%) 2024-08-06T22:14:21.9405104Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-979948081c46f892.xml (deflated 42%) 2024-08-06T22:14:21.9406362Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0247e5fa42e4f70d.xml (deflated 42%) 2024-08-06T22:14:21.9407555Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2b26c68b130ba005.xml (deflated 42%) 2024-08-06T22:14:21.9408750Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-32b2747f6acfa25c.xml (deflated 42%) 2024-08-06T22:14:21.9409943Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e02a8e8e477126a1.xml (deflated 43%) 2024-08-06T22:14:21.9411118Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-467e93b452d6c3ea.xml (deflated 42%) 2024-08-06T22:14:21.9412308Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7d71215700e53089.xml (deflated 37%) 2024-08-06T22:14:21.9413500Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5dba74715f1f9e05.xml (deflated 35%) 2024-08-06T22:14:21.9414698Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-24b77374509da1a0.xml (deflated 36%) 2024-08-06T22:14:21.9415941Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b006da4bb03d7c6d.xml (deflated 35%) 2024-08-06T22:14:21.9417140Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cd99d0ef8f46322b.xml (deflated 42%) 2024-08-06T22:14:21.9418337Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f54dc78ce61580f4.xml (deflated 36%) 2024-08-06T22:14:21.9419527Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3774e5d0238bfdc4.xml (deflated 45%) 2024-08-06T22:14:21.9420770Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-682718813431d7be.xml (deflated 45%) 2024-08-06T22:14:21.9421964Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4c4d258a86e6a65e.xml (deflated 45%) 2024-08-06T22:14:21.9423162Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cf48f30d446d3f1c.xml (deflated 41%) 2024-08-06T22:14:21.9424358Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c00458fa9308b327.xml (deflated 42%) 2024-08-06T22:14:21.9425539Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3a8b96dd7ff8d43b.xml (deflated 42%) 2024-08-06T22:14:21.9426746Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1dc742dfd3c3458d.xml (deflated 42%) 2024-08-06T22:14:21.9427945Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-db16b35c70547301.xml (deflated 41%) 2024-08-06T22:14:21.9429117Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-57b1fea97068b685.xml (deflated 41%) 2024-08-06T22:14:21.9430575Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7af26f3e049b162b.xml (deflated 35%) 2024-08-06T22:14:21.9431778Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c2637f2ff6d8440d.xml (deflated 36%) 2024-08-06T22:14:21.9432972Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-43a45a61b281acca.xml (deflated 35%) 2024-08-06T22:14:21.9434158Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bb6e10471ec85c77.xml (deflated 36%) 2024-08-06T22:14:21.9435534Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0ddcad51e25975a0.xml (deflated 45%) 2024-08-06T22:14:21.9436741Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9d5d3ed6bdd512bf.xml (deflated 43%) 2024-08-06T22:14:21.9437943Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6b643f7eeac33682.xml (deflated 36%) 2024-08-06T22:14:21.9439126Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-06f600a4ab9335fb.xml (deflated 36%) 2024-08-06T22:14:21.9440330Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-dc8bd1ebd83e8824.xml (deflated 35%) 2024-08-06T22:14:21.9441533Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ffe355869b487e83.xml (deflated 42%) 2024-08-06T22:14:21.9442730Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-17ee4fc4e8248ec3.xml (deflated 35%) 2024-08-06T22:14:21.9443910Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2f7871bfd61ff390.xml (deflated 35%) 2024-08-06T22:14:21.9445200Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-679f42b8ec48cc1f.xml (deflated 36%) 2024-08-06T22:14:21.9446386Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5c33839eb6349051.xml (deflated 35%) 2024-08-06T22:14:21.9447581Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0369a9eec12cbd23.xml (deflated 36%) 2024-08-06T22:14:21.9448767Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-08bc1ee0030b918a.xml (deflated 35%) 2024-08-06T22:14:21.9449969Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ad784c6f93bfb53a.xml (deflated 42%) 2024-08-06T22:14:21.9451168Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e38d7e92b0d8608f.xml (deflated 42%) 2024-08-06T22:14:21.9452358Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a3d371abc18d1c0b.xml (deflated 42%) 2024-08-06T22:14:21.9453560Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-db953ef70eaf3ebb.xml (deflated 42%) 2024-08-06T22:14:21.9503425Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-06T22:14:21.9503915Z # Remove any previous usage logs if they exist 2024-08-06T22:14:21.9504294Z rm -f logs-*.zip 2024-08-06T22:14:21.9504755Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-06T22:14:21.9505325Z # so check to see if the file exists first 2024-08-06T22:14:21.9505692Z if [ -f 'usage_log.txt' ]; then 2024-08-06T22:14:21.9506057Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-06T22:14:21.9506427Z fi 2024-08-06T22:14:21.9506730Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-06T22:14:21.9507154Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-06T22:14:21.9507519Z fi 2024-08-06T22:14:21.9513134Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T22:14:21.9513511Z env: 2024-08-06T22:14:21.9513739Z GIT_DEFAULT_BRANCH: main 2024-08-06T22:14:21.9514203Z DOCKER_CONTAINER_ID: 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:21.9514818Z FILE_SUFFIX: test-distributed-2-2-amz2023.linux.2xlarge_28427732833 2024-08-06T22:14:21.9515245Z ##[endgroup] 2024-08-06T22:14:21.9614333Z adding: usage_log.txt (deflated 92%) 2024-08-06T22:14:21.9880624Z adding: test/test-reports/distributed.checkpoint.test_nested_dict_1.1_3982453ef09520ae_.log (deflated 59%) 2024-08-06T22:14:21.9882195Z adding: test/test-reports/distributed._tensor.debug.test_op_coverage_1.1_35b1de57236a0d7a_.log (deflated 54%) 2024-08-06T22:14:21.9883407Z adding: test/test-reports/distributed._tools.test_mod_tracker_1.1_abf692734e99f766_.log (deflated 64%) 2024-08-06T22:14:21.9885063Z adding: test/test-reports/distributed.pipelining.test_backward_1.1_706e065e140c5a02_.log (deflated 54%) 2024-08-06T22:14:21.9886321Z adding: test/test-reports/distributed.elastic.timer.api_test_1.1_daa97a9fe495140d_.log (stored 0%) 2024-08-06T22:14:21.9887576Z adding: test/test-reports/distributed.tensor.parallel.test_micro_pipeline_tp_1.1_d4d4bf780fa701ad_.log (deflated 90%) 2024-08-06T22:14:21.9889170Z adding: test/test-reports/distributed.pipelining.test_transformer_1.1_c9dca16cc8b1d03e_.log (deflated 54%) 2024-08-06T22:14:21.9890098Z adding: test/test-reports/distributed.elastic.timer.local_timer_example_1.1_a2a26c6288d157ac_.log (deflated 67%) 2024-08-06T22:14:21.9891036Z adding: test/test-reports/distributed._shard.sharded_tensor.test_logger_1.1_7a62f46a4ac1757a_.log (deflated 55%) 2024-08-06T22:14:21.9892716Z adding: test/test-reports/distributed.test_symmetric_memory_1.1_538ea45d724dfe9e_.log (deflated 86%) 2024-08-06T22:14:21.9894591Z adding: test/test-reports/distributed._tensor.test_dtensor_compile_1.1_7d26bc3deb116800_.log (deflated 84%) 2024-08-06T22:14:21.9896220Z adding: test/test-reports/distributed._tensor.test_attention_1.1_6c7430925d044677_.log (deflated 76%) 2024-08-06T22:14:21.9897123Z adding: test/test-reports/distributed._composable.test_contract_1.1_bcc7cf669987594b_.log (deflated 67%) 2024-08-06T22:14:21.9897991Z adding: test/test-reports/distributed._tensor.test_dtensor_1.1_554229b9a54ce713_.log (deflated 90%) 2024-08-06T22:14:21.9898850Z adding: test/test-reports/distributed.checkpoint.fsdp.test_fsdp_dsd_1.1_85b4ecdbcd97bf0e_.log (deflated 67%) 2024-08-06T22:14:21.9899773Z adding: test/test-reports/distributed._tensor.test_xla_integration_1.1_fcb656dce4b7076e_.log (deflated 66%) 2024-08-06T22:14:21.9900716Z adding: test/test-reports/distributed.elastic.timer.local_timer_test_1.1_7361dba96913f6a2_.log (deflated 78%) 2024-08-06T22:14:21.9901665Z adding: test/test-reports/distributed._tensor.experimental.test_tp_transform_1.1_427c20fdd6b0a402_.log (deflated 65%) 2024-08-06T22:14:21.9902606Z adding: test/test-reports/distributed.checkpoint.test_dedup_tensors_1.1_19c6bb14af1a40b5_.log (deflated 52%) 2024-08-06T22:14:21.9904221Z adding: test/test-reports/distributed.pipelining.test_unflatten_1.1_59ecf857e03f6768_.log (deflated 54%) 2024-08-06T22:14:21.9905103Z adding: test/test-reports/distributed._tensor.test_convolution_ops_1.1_57bf8d653013d9cd_.log (deflated 62%) 2024-08-06T22:14:21.9905976Z adding: test/test-reports/distributed._tensor.test_tensor_ops_1.1_2e51d5de20662031_.log (deflated 95%) 2024-08-06T22:14:21.9907470Z adding: test/test-reports/distributed.pipelining.test_composability_1.1_62f32717b5b569f3_.log (deflated 3%) 2024-08-06T22:14:21.9908786Z adding: test/test-reports/distributed.pipelining.test_microbatch_1.1_e8e184ca756f82c6_.log (deflated 58%) 2024-08-06T22:14:21.9909860Z adding: test/test-reports/distributed.elastic.utils.distributed_test_1.1_bf972dbdd5cab056_.log (deflated 93%) 2024-08-06T22:14:21.9911616Z adding: test/test-reports/distributed.tensor.parallel.test_tp_examples_1.1_327e0e6708f6aa5d_.log (deflated 87%) 2024-08-06T22:14:21.9913319Z adding: test/test-reports/distributed.checkpoint.test_traverse_1.1_6be42eacf814c4fd_.log (deflated 71%) 2024-08-06T22:14:21.9914549Z adding: test/test-reports/distributed.test_device_mesh_1.1_6902e9689b8f6306_.log (deflated 83%) 2024-08-06T22:14:21.9915566Z adding: test/test-reports/distributed.pipelining.test_stage_1.1_d0fe65a7f29f1056_.log (deflated 2%) 2024-08-06T22:14:21.9917264Z adding: test/test-reports/distributed.checkpoint.test_planner_1.1_027d276b129ce4b2_.log (deflated 67%) 2024-08-06T22:14:21.9918105Z adding: test/test-reports/distributed.checkpoint.test_fsspec_1.1_65d945fcf18a1286_.log (deflated 58%) 2024-08-06T22:14:21.9918947Z adding: test/test-reports/distributed._tensor.test_op_strategy_1.1_3f4b828ed3e28389_.log (deflated 79%) 2024-08-06T22:14:21.9920585Z adding: test/test-reports/distributed.elastic.utils.logging_test_1.1_80fe99122b7a4c66_.log (deflated 58%) 2024-08-06T22:14:21.9922322Z adding: test/test-reports/distributed.tensor.parallel.test_ddp_2d_parallel_1.1_f901284cb41853ad_.log (deflated 56%) 2024-08-06T22:14:21.9923582Z adding: test/test-reports/distributed.test_inductor_collectives_1.1_97d7ac9c2dbea9ac_.log (deflated 90%) 2024-08-06T22:14:21.9925004Z adding: test/test-reports/distributed.elastic.utils.util_test_1.1_d00d19c45784c2e6_.log (deflated 75%) 2024-08-06T22:14:21.9926304Z adding: test/test-reports/distributed.checkpoint.test_dtensor_checkpoint_1.1_958ee3feb4cebbfa_.log (deflated 55%) 2024-08-06T22:14:21.9927167Z adding: test/test-reports/distributed.launcher.test_run_1.1_e85904df2c4b6f37_.log (deflated 89%) 2024-08-06T22:14:21.9928018Z adding: test/test-reports/distributed.checkpoint.e2e.test_fine_tuning_1.1_522f91c935e84cba_.log (deflated 53%) 2024-08-06T22:14:21.9929022Z adding: test/test-reports/distributed.checkpoint.test_compatibility_1.1_2c922256b4c1246f_.log (deflated 63%) 2024-08-06T22:14:21.9929920Z adding: test/test-reports/distributed.checkpoint.test_hsdp_checkpoint_1.1_b87181bd1a77dabe_.log (deflated 71%) 2024-08-06T22:14:21.9931111Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_1.1_38ec777470955cac_.log (deflated 62%) 2024-08-06T22:14:21.9932160Z adding: test/test-reports/distributed._composable.test_composability.test_2d_composability_1.1_916b9014eb50b5b4_.log (deflated 68%) 2024-08-06T22:14:21.9933974Z adding: test/test-reports/distributed.test_collective_utils_1.1_de922fbafdc9415c_.log (stored 0%) 2024-08-06T22:14:21.9935286Z adding: test/test-reports/distributed.test_c10d_spawn_ucc_1.1_05abf7da03334ecf_.log (deflated 92%) 2024-08-06T22:14:21.9936575Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_memory_1.1_2141ed70355b05d6_.log (deflated 55%) 2024-08-06T22:14:21.9938244Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_init_1.1_b1ec25b9f1c1a132_.log (deflated 87%) 2024-08-06T22:14:21.9939974Z adding: test/test-reports/distributed.fsdp.test_fsdp_optim_state_1.1_e4729837e3bdd046_.log (deflated 97%) 2024-08-06T22:14:21.9940958Z adding: test/test-reports/distributed.checkpoint.test_utils_1.1_67f1bec2409a5b38_.log (deflated 66%) 2024-08-06T22:14:21.9942539Z adding: test/test-reports/distributed.checkpoint.e2e.test_pipeline_1.1_b1e2aa70e117e8b0_.log (deflated 53%) 2024-08-06T22:14:21.9943809Z adding: test/test-reports/distributed.elastic.test_control_plane_1.1_bceb6f15e5e85eca_.log (deflated 73%) 2024-08-06T22:14:21.9945106Z adding: test/test-reports/distributed.test_c10d_pypg_1.1_e81c73f9d648e24d_.log (deflated 90%) 2024-08-06T22:14:21.9946394Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_bag_1.1_57989b22ea5ba46a_.log (deflated 63%) 2024-08-06T22:14:21.9947417Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_mixed_precision_1.1_0454f70353938677_.log (deflated 74%) 2024-08-06T22:14:21.9948387Z adding: test/test-reports/distributed.fsdp.test_fsdp_ignored_modules_1.1_539d43280f3d45ac_.log (deflated 85%) 2024-08-06T22:14:21.9949184Z adding: test/test-reports/distributed.test_nccl_1.1_5374ddf54be974bd_.log (deflated 49%) 2024-08-06T22:14:21.9949969Z adding: test/test-reports/distributed._composable.test_compose_1.1_f00f0326495618e8_.log (deflated 78%) 2024-08-06T22:14:21.9950787Z adding: test/test-reports/distributed.test_c10d_spawn_nccl_1.1_08b73a242f5e8db4_.log (deflated 93%) 2024-08-06T22:14:21.9951678Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_util_1.1_9b8050b29216a8a2_.log (deflated 57%) 2024-08-06T22:14:21.9952749Z adding: test/test-reports/distributed.test_launcher_1.1_cc90eb11fce887b0_.log (deflated 58%) 2024-08-06T22:14:21.9953627Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_training_1.1_88e859eefba0b143_.log (deflated 84%) 2024-08-06T22:14:21.9954578Z adding: test/test-reports/distributed.fsdp.test_fsdp_tp_integration_1.1_4bf00e8ecb928c34_.log (deflated 67%) 2024-08-06T22:14:21.9955531Z adding: test/test-reports/distributed.fsdp.test_fsdp_dtensor_state_dict_1.1_0c922aa3aba65cbe_.log (deflated 86%) 2024-08-06T22:14:21.9957129Z adding: test/test-reports/distributed.test_store_1.1_3c313cf16bee93be_.log (deflated 95%) 2024-08-06T22:14:21.9958593Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_state_dict_1.1_d04693461791bc3a_.log (deflated 66%) 2024-08-06T22:14:21.9960379Z adding: test/test-reports/distributed.fsdp.test_fsdp_pure_fp16_1.1_c86f78f13a6ba134_.log (deflated 61%) 2024-08-06T22:14:21.9961969Z adding: test/test-reports/distributed.checkpoint.test_checkpoint_1.1_4d2e5dc01c9865a7_.log (deflated 81%) 2024-08-06T22:14:21.9963453Z adding: test/test-reports/distributed.elastic.events.lib_test_1.1_659e399622ceb358_.log (deflated 73%) 2024-08-06T22:14:21.9964368Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_frozen_1.1_87b903bc43d4fe07_.log (deflated 68%) 2024-08-06T22:14:21.9965430Z adding: test/test-reports/distributed.fsdp.test_distributed_checkpoint_1.1_6b60b17b2838d854_.log (deflated 65%) 2024-08-06T22:14:21.9966329Z adding: test/test-reports/distributed.elastic.metrics.api_test_1.1_de47256bd473c07e_.log (deflated 62%) 2024-08-06T22:14:21.9967257Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_autograd_1.1_ae2256a3cc7ac89e_.log (deflated 72%) 2024-08-06T22:14:21.9968123Z adding: test/test-reports/distributed.test_fake_pg_1.1_10098d87632f8245_.log (deflated 74%) 2024-08-06T22:14:21.9968886Z adding: test/test-reports/distributed.rpc.test_share_memory_1.1_1021956ed30bb8bc_.log (deflated 56%) 2024-08-06T22:14:21.9971462Z adding: test/test-reports/distributed.fsdp.test_fsdp_state_dict_1.1_bd1a52c3254747d0_.log (deflated 96%) 2024-08-06T22:14:21.9972618Z adding: test/test-reports/distributed.test_distributed_spawn_1.3_142bb915fb450761_.log (deflated 37%) 2024-08-06T22:14:21.9973444Z adding: test/test-reports/distributed.test_distributed_spawn_1.3_c5881492f08eb59d_.log (deflated 37%) 2024-08-06T22:14:21.9991621Z adding: test/test-reports/distributed.test_distributed_spawn_1.3_486eb39e438c3b6d_.log (deflated 97%) 2024-08-06T22:14:22.0010856Z adding: test/test-reports/distributed.test_distributed_spawn_1.3_1d4aefa22ac59310_.log (deflated 97%) 2024-08-06T22:14:22.0012043Z adding: test/test-reports/distributed.test_distributed_spawn_3.3_32afe8d42291c555_.log (deflated 37%) 2024-08-06T22:14:22.0012862Z adding: test/test-reports/distributed.test_distributed_spawn_3.3_ee25328a19dc1ca0_.log (deflated 37%) 2024-08-06T22:14:22.0028323Z adding: test/test-reports/distributed.test_distributed_spawn_3.3_69f3bbcd9b4f3377_.log (deflated 97%) 2024-08-06T22:14:22.0045110Z adding: test/test-reports/distributed.test_distributed_spawn_3.3_f79194f40653068b_.log (deflated 97%) 2024-08-06T22:14:22.0046637Z adding: test/test-reports/distributed.fsdp.test_fsdp_memory_1.1_27ed7f45266bb41b_.log (deflated 60%) 2024-08-06T22:14:22.0047412Z adding: test/test-reports/distributed.test_c10d_nccl_1.1_27e66ce799dcffef_.log (stored 0%) 2024-08-06T22:14:22.0048304Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_runtime_1.1_d03f5ce8749c662e_.log (deflated 62%) 2024-08-06T22:14:22.0049746Z adding: test/test-reports/distributed.checkpoint.test_file_system_checkpoint_cpu_1.1_0f6157fd6de45efd_.log (deflated 94%) 2024-08-06T22:14:22.0051024Z adding: test/test-reports/distributed.test_data_parallel_1.1_5daacc3045b7707b_.log (deflated 89%) 2024-08-06T22:14:22.0053178Z adding: test/test-reports/distributed.elastic.multiprocessing.api_test_1.1_cad35b52be207fac_.log (deflated 88%) 2024-08-06T22:14:22.0055239Z adding: test/test-reports/distributed.algorithms.ddp_comm_hooks.test_ddp_hooks_1.1_31e38c4cd1dcf708_.log (deflated 76%) 2024-08-06T22:14:22.0055968Z adding: test/test-reports/distributed.fsdp.test_fsdp_comm_hooks_1.1_70879fededc8b580_.log (deflated 89%) 2024-08-06T22:14:22.0057154Z adding: test/test-reports/distributed.test_multi_threaded_pg_1.1_c516d5277bb6b18d_.log (deflated 85%) 2024-08-06T22:14:22.0058081Z adding: test/test-reports/distributed.fsdp.test_fsdp_apply_1.1_64dd6ed9de9e4b6d_.log (deflated 63%) 2024-08-06T22:14:22.0060895Z adding: test/test-reports/distributed.fsdp.test_fsdp_mixed_precision_1.1_2e84eaf8cdc62084_.log (deflated 92%) 2024-08-06T22:14:22.0061578Z adding: test/test-reports/distributed.fsdp.test_fsdp_input_1.1_c7bb88a6f3c33b26_.log (deflated 58%) 2024-08-06T22:14:22.0062348Z adding: test/test-reports/distributed.fsdp.test_fsdp_multiple_wrapping_1.1_7880ba3467299369_.log (deflated 56%) 2024-08-06T22:14:22.0063056Z adding: test/test-reports/distributed.fsdp.test_fsdp_uneven_1.1_d171228d9ffc6bc6_.log (deflated 54%) 2024-08-06T22:14:22.0063451Z adding: test/test-reports/distributed.fsdp.test_fsdp_traversal_1.1_6a677045b368e573_.log (deflated 53%) 2024-08-06T22:14:22.0065045Z adding: test/test-reports/distributed.fsdp.test_fsdp_misc_1.1_a2511ddb98bf2c79_.log (deflated 90%) 2024-08-06T22:14:22.0067188Z adding: test/test-reports/distributed.test_pg_wrapper_1.1_24f436cf63cc7737_.log (deflated 93%) 2024-08-06T22:14:22.0103480Z adding: test/test-reports/distributed.rpc.test_tensorpipe_agent_2.3_deda1e86219ddb65_.log (deflated 97%) 2024-08-06T22:14:22.0105892Z adding: test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_1.1_79cd4017e892f7b6_.log (deflated 91%) 2024-08-06T22:14:22.0106636Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint_1.1_8fdfcb558d429c9a_.log (deflated 65%) 2024-08-06T22:14:22.0126142Z adding: test/test-reports/distributed.rpc.cuda.test_tensorpipe_agent_1.1_967132d7008e78e2_.log (deflated 98%) 2024-08-06T22:14:22.0126988Z adding: test/test-reports/distributed._shard.sharded_optim.test_sharded_optim_1.1_598463a687355753_.log (deflated 62%) 2024-08-06T22:14:22.0127508Z adding: test/test-reports/distributed.fsdp.test_fsdp_multiple_forward_1.1_f1ba9b661bb80707_.log (deflated 55%) 2024-08-06T22:14:22.0128207Z adding: test/test-reports/distributed.fsdp.test_fsdp_comm_1.1_d625cb5f5f65bae8_.log (deflated 81%) 2024-08-06T22:14:22.0131617Z adding: test/test-reports/distributed.test_c10d_common_1.1_eff37e3028e47349_.log (deflated 94%) 2024-08-06T22:14:22.0132353Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_init_1.1_43df3a106417c42b_.log (deflated 67%) 2024-08-06T22:14:22.0139840Z adding: test/test-reports/distributed.rpc.test_faulty_agent_1.1_2484546d66319725_.log (deflated 96%) 2024-08-06T22:14:22.0141698Z adding: test/test-reports/distributed.test_c10d_spawn_gloo_1.1_e51cb837bb0c611a_.log (deflated 94%) 2024-08-06T22:14:22.0142497Z adding: test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_reshard_1.1_c21fb6496563c026_.log (deflated 64%) 2024-08-06T22:14:22.0180700Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-06T22:14:22.0181045Z # Remove any previous debugging artifacts if they exist 2024-08-06T22:14:22.0181208Z rm -f debug-*.zip 2024-08-06T22:14:22.0181369Z if [ -d 'test/debug' ]; then 2024-08-06T22:14:22.0181597Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-06T22:14:22.0181731Z fi 2024-08-06T22:14:22.0189670Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T22:14:22.0189812Z env: 2024-08-06T22:14:22.0189984Z GIT_DEFAULT_BRANCH: main 2024-08-06T22:14:22.0190493Z DOCKER_CONTAINER_ID: 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:22.0190892Z FILE_SUFFIX: test-distributed-2-2-amz2023.linux.2xlarge_28427732833 2024-08-06T22:14:22.0191033Z ##[endgroup] 2024-08-06T22:14:22.0302105Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-06T22:14:22.0302205Z with: 2024-08-06T22:14:22.0302314Z s3-bucket: gha-artifacts 2024-08-06T22:14:22.0302476Z s3-prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T22:14:22.0302588Z retention-days: 14 2024-08-06T22:14:22.0302690Z if-no-files-found: warn 2024-08-06T22:14:22.0302807Z path: test-jsons-*.zip 2024-08-06T22:14:22.0302895Z name: artifact 2024-08-06T22:14:22.0302986Z region: us-east-1 2024-08-06T22:14:22.0303089Z env: 2024-08-06T22:14:22.0303189Z GIT_DEFAULT_BRANCH: main 2024-08-06T22:14:22.0303489Z DOCKER_CONTAINER_ID: 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:22.0303579Z ##[endgroup] 2024-08-06T22:14:22.4616150Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-06T22:14:22.4616951Z With the provided path, there will be 1 file uploaded 2024-08-06T22:14:22.4617612Z Uploading to s3 prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T22:14:22.4773793Z Starting upload of test-jsons-test-distributed-2-2-amz2023.linux.2xlarge_28427732833.zip 2024-08-06T22:14:22.6745197Z Finished upload of test-jsons-test-distributed-2-2-amz2023.linux.2xlarge_28427732833.zip 2024-08-06T22:14:22.6953055Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-06T22:14:22.6953553Z with: 2024-08-06T22:14:22.6953769Z s3-bucket: gha-artifacts 2024-08-06T22:14:22.6954101Z s3-prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T22:14:22.6954831Z retention-days: 14 2024-08-06T22:14:22.6955088Z if-no-files-found: error 2024-08-06T22:14:22.6955379Z path: test-reports-*.zip 2024-08-06T22:14:22.6955650Z name: artifact 2024-08-06T22:14:22.6955875Z region: us-east-1 2024-08-06T22:14:22.6956114Z env: 2024-08-06T22:14:22.6956328Z GIT_DEFAULT_BRANCH: main 2024-08-06T22:14:22.6956778Z DOCKER_CONTAINER_ID: 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:22.6957276Z ##[endgroup] 2024-08-06T22:14:23.0493784Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-06T22:14:23.0494414Z With the provided path, there will be 1 file uploaded 2024-08-06T22:14:23.0494873Z Uploading to s3 prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T22:14:23.0535091Z Starting upload of test-reports-test-distributed-2-2-amz2023.linux.2xlarge_28427732833.zip 2024-08-06T22:14:23.2251493Z Finished upload of test-reports-test-distributed-2-2-amz2023.linux.2xlarge_28427732833.zip 2024-08-06T22:14:23.2448612Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-06T22:14:23.2448973Z with: 2024-08-06T22:14:23.2449195Z s3-bucket: gha-artifacts 2024-08-06T22:14:23.2449528Z s3-prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T22:14:23.2449892Z retention-days: 14 2024-08-06T22:14:23.2450153Z if-no-files-found: ignore 2024-08-06T22:14:23.2450435Z path: logs-*.zip 2024-08-06T22:14:23.2450678Z name: artifact 2024-08-06T22:14:23.2450900Z region: us-east-1 2024-08-06T22:14:23.2451135Z env: 2024-08-06T22:14:23.2451356Z GIT_DEFAULT_BRANCH: main 2024-08-06T22:14:23.2451827Z DOCKER_CONTAINER_ID: 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:23.2452324Z ##[endgroup] 2024-08-06T22:14:23.5986673Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-06T22:14:23.5987504Z With the provided path, there will be 1 file uploaded 2024-08-06T22:14:23.5988197Z Uploading to s3 prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T22:14:23.6029020Z Starting upload of logs-test-distributed-2-2-amz2023.linux.2xlarge_28427732833.zip 2024-08-06T22:14:23.7974232Z Finished upload of logs-test-distributed-2-2-amz2023.linux.2xlarge_28427732833.zip 2024-08-06T22:14:23.8179435Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-06T22:14:23.8179780Z with: 2024-08-06T22:14:23.8180018Z s3-bucket: gha-artifacts 2024-08-06T22:14:23.8180348Z s3-prefix: pytorch/pytorch/10273124344/1/artifact 2024-08-06T22:14:23.8180773Z retention-days: 14 2024-08-06T22:14:23.8181042Z if-no-files-found: ignore 2024-08-06T22:14:23.8181498Z path: debug-*.zip 2024-08-06T22:14:23.8181735Z name: artifact 2024-08-06T22:14:23.8181980Z region: us-east-1 2024-08-06T22:14:23.8182211Z env: 2024-08-06T22:14:23.8182435Z GIT_DEFAULT_BRANCH: main 2024-08-06T22:14:23.8182906Z DOCKER_CONTAINER_ID: 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:23.8183410Z ##[endgroup] 2024-08-06T22:14:24.1668444Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-06T22:14:24.1869757Z ##[group]Run # shellcheck disable=SC2156 2024-08-06T22:14:24.1870147Z # shellcheck disable=SC2156 2024-08-06T22:14:24.1870750Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-06T22:14:24.1876669Z shell: /usr/bin/bash -e {0} 2024-08-06T22:14:24.1876950Z env: 2024-08-06T22:14:24.1877185Z GIT_DEFAULT_BRANCH: main 2024-08-06T22:14:24.1877659Z DOCKER_CONTAINER_ID: 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:24.1878151Z ##[endgroup] 2024-08-06T22:14:24.4110423Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-06T22:14:24.4110885Z with: 2024-08-06T22:14:24.4111094Z env: 2024-08-06T22:14:24.4111490Z GIT_DEFAULT_BRANCH: main 2024-08-06T22:14:24.4111972Z DOCKER_CONTAINER_ID: 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:24.4112470Z ##[endgroup] 2024-08-06T22:14:24.4137637Z ##[group]Run set -eou pipefail 2024-08-06T22:14:24.4137982Z set -eou pipefail 2024-08-06T22:14:24.4138247Z  2024-08-06T22:14:24.4138623Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-06T22:14:24.4139095Z for _ in $(seq 1440); do 2024-08-06T22:14:24.4139423Z  # Break if no ssh session exists anymore 2024-08-06T22:14:24.4139784Z  if [ "$(who)" = "" ]; then 2024-08-06T22:14:24.4140090Z  break 2024-08-06T22:14:24.4140322Z  fi 2024-08-06T22:14:24.4140718Z  echo "." 2024-08-06T22:14:24.4140963Z  sleep 5 2024-08-06T22:14:24.4141216Z done 2024-08-06T22:14:24.4150465Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T22:14:24.4150864Z env: 2024-08-06T22:14:24.4151092Z GIT_DEFAULT_BRANCH: main 2024-08-06T22:14:24.4151559Z DOCKER_CONTAINER_ID: 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:24.4152050Z ##[endgroup] 2024-08-06T22:14:24.4175661Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-06T22:14:24.4270844Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-06T22:14:24.4271421Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-06T22:14:24.4271848Z # shellcheck disable=SC2046 2024-08-06T22:14:24.4272196Z docker stop $(docker ps -q) || true 2024-08-06T22:14:24.4272554Z # Prune all of the docker images 2024-08-06T22:14:24.4272880Z docker system prune -af 2024-08-06T22:14:24.4278997Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T22:14:24.4279395Z env: 2024-08-06T22:14:24.4279612Z GIT_DEFAULT_BRANCH: main 2024-08-06T22:14:24.4280093Z DOCKER_CONTAINER_ID: 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:24.4280599Z ##[endgroup] 2024-08-06T22:14:35.1284151Z 3a8f9a9b0555 2024-08-06T22:14:35.8017263Z Deleted Containers: 2024-08-06T22:14:35.8017732Z 3a8f9a9b05552f46b744a5e05d70c37f4a00bda3d804913e8cd2ce0272a279b9 2024-08-06T22:14:35.8018073Z 2024-08-06T22:14:44.0342195Z Deleted Images: 2024-08-06T22:14:44.0343170Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-06T22:14:44.0344870Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11@sha256:c75221497aa22766802555fccb76014708414eebd47578b213b43b3f8f4cddb9 2024-08-06T22:14:44.0345897Z deleted: sha256:eb2e8636b63e804e38c5e56190d8b860c4bf8a9e0794f88825eff958ed6b1e76 2024-08-06T22:14:44.0346548Z deleted: sha256:3a75aa289a82638e31ac4b969360c49c8013f3546a449692cb458fcde3935f22 2024-08-06T22:14:44.0347200Z deleted: sha256:bdfa3e02d65d776f3bb70b6bf71d852ddca4c1dfbdca2356544996f07fef2ed8 2024-08-06T22:14:44.0347854Z deleted: sha256:6a85078ea3791ab81c962cdfc07c05a8b7eb4c5576721c3b56b45f4adf3fd0d1 2024-08-06T22:14:44.0348556Z deleted: sha256:935ccaf7b02ef4b0aabe934f8ed13eaf6bbc24da0954bdc3eea688aafcf5aea6 2024-08-06T22:14:44.0349219Z deleted: sha256:eef572047e96af9ee6bb247ba9acc9c558d7468ff89e2297aa4ad37d1f6043bb 2024-08-06T22:14:44.0349876Z deleted: sha256:4cde019d0b2c20c0af3f1631adfd033c260533dfeefc25dad09f2900dd488b69 2024-08-06T22:14:44.0350517Z deleted: sha256:d2e3c7156b77f135b6bfcbd686f30b96a9fdd1cf7c08b62815572e2d2f7c68be 2024-08-06T22:14:44.0351172Z deleted: sha256:9cd733cd617c46bdc2cb513ead975cec6d449f22a08e36d803cbcef61fa1ef01 2024-08-06T22:14:44.0352101Z deleted: sha256:64744261caf2f507a0f6160868b1d4f0a4a435fb546701053735b889023799eb 2024-08-06T22:14:44.0352737Z deleted: sha256:a2047a4af9745168d52ba90b39a780bc22dc15918425324774e959c0755645a5 2024-08-06T22:14:44.0353365Z deleted: sha256:7d8b4203c0f94a0eccdf0a3ee5819d3be23e8acb9412c3330f66669f03d23e4a 2024-08-06T22:14:44.0354259Z deleted: sha256:7c66ee01d3a6002bf7c53c998d9320f1fb14268af96da395dc28488d5e28001e 2024-08-06T22:14:44.0354902Z deleted: sha256:c1821ba8bdf74b922560615bc28d42e36b63152f7f8e7b07fead450cf78c03b3 2024-08-06T22:14:44.0355543Z deleted: sha256:738da9c7930e061cb698fdaff5ab4d9019fd00adeeacbc24c07eeef3b9a1e8f3 2024-08-06T22:14:44.0356199Z deleted: sha256:b035793979cb8d15291d75f7ceb86be4c7e50b0b55c3ad2de59633310f8eb2ad 2024-08-06T22:14:44.0356835Z deleted: sha256:8caa23c592f4ec71096f15d741c7f74b3674d14f201069b87058d486ee469d80 2024-08-06T22:14:44.0357451Z deleted: sha256:5ec471cacf0cf9317535522156613c2b647715813f96552a7756767caa05b5bc 2024-08-06T22:14:44.0358081Z deleted: sha256:3ddc946b392989b8e7e8086bc23fdab9391ec27042559856bb40fd155a0626b0 2024-08-06T22:14:44.0358724Z deleted: sha256:8bb9b390febb36e87a303534a0672e799f14263107dccecb8fb816711b19b197 2024-08-06T22:14:44.0359359Z deleted: sha256:7636da8a45668f13fc8bdd44f4bcbdfe08506e1222f2e6bf0b6173ff1ebc9d73 2024-08-06T22:14:44.0360019Z deleted: sha256:db811dce25acbc1220f0b64f6b6378af1a8715aadf36172b548ccf95a8a10f5c 2024-08-06T22:14:44.0360669Z deleted: sha256:ea9f1876d9bef2b9737a8084aa575ae2ccf54320762796b1a25b2fd7a1f9a842 2024-08-06T22:14:44.0361301Z deleted: sha256:ee9308661047f85a0cb1e065673e3e34a223a0fabc0d758ca482d1caf72bc156 2024-08-06T22:14:44.0361948Z deleted: sha256:83af056623fdf4bdd8eb6689dd830c618fc706f07f5745771bd23dee1a9df53c 2024-08-06T22:14:44.0362591Z deleted: sha256:1697cd38e32034c99390abaf80e0ef262175ceb38304c2a33a2391ea089625ce 2024-08-06T22:14:44.0363231Z deleted: sha256:2c941f6435258652fe85b0aa3fe7dfd6446f8ad4b90ff57b75cff5173ca901d1 2024-08-06T22:14:44.0363854Z deleted: sha256:de31781f10299152d97a453b662468e0fe812bf2c80e40d7e87d21f1aeebe0de 2024-08-06T22:14:44.0364500Z deleted: sha256:df51025351b2d7b823ae2da119ef42f7a8bcb65067852d58b58793b87a4020cd 2024-08-06T22:14:44.0365146Z deleted: sha256:dea53edb79ae100c832ff7d8e52aa46514a05f85b18820e9798e14cfca8d2b0a 2024-08-06T22:14:44.0365789Z deleted: sha256:30a0b14b371bbf3393cc59e86ebddb2c2bf5ec6e14a2623357e0e8522004373a 2024-08-06T22:14:44.0366449Z deleted: sha256:86275d2a7f4ab08c4db777a2d58601c8e9a33f085073ee419cecc0f80706ba0a 2024-08-06T22:14:44.0367088Z deleted: sha256:be527c207999e42b06dd2b66329fee5905a775d5a617576a8822937689e181b2 2024-08-06T22:14:44.0367712Z deleted: sha256:b726bfe9f806f36a474aff38d139841ad1f3062ad40b2644b38a070efa891fc4 2024-08-06T22:14:44.0368351Z deleted: sha256:79621bf70363b2e574b56de08889a123c0aafdf331b6d403b1227d601196878f 2024-08-06T22:14:44.0368985Z deleted: sha256:23da9a238173789a5e64303a1d5b580bdc2ff4ee4adb1726a937e74fccdb3304 2024-08-06T22:14:44.0369611Z deleted: sha256:6ae72960efb5326803e5d131f2b788bea7d945094d6b4154fbb0d23a1724bc9d 2024-08-06T22:14:44.0370254Z deleted: sha256:4b522ca122bee088c81b99d32935d0c7f798acd919e4787f71f60860f47039c8 2024-08-06T22:14:44.0370884Z deleted: sha256:8099c7b0342e10a71c3957854138df7a86715582a05dfeb507862407dda38fc3 2024-08-06T22:14:44.0371520Z deleted: sha256:9a3099fdec71a88416fee7bf46f4a7692b473fb9684dd93aaa0370840bb1ec2a 2024-08-06T22:14:44.0372156Z deleted: sha256:c7ffa0248132fad86167206fd29b1b8f49bf20a42406063d47df97a13d8f198a 2024-08-06T22:14:44.0372796Z deleted: sha256:496001e36192545049eca8ecc830251c2fa1eb6daf21421c8dafe0acb1f81538 2024-08-06T22:14:44.0373436Z deleted: sha256:9813a6d93cac15977f4540b3e5b3fe849269f35658ed780bdda563ee74b15b3f 2024-08-06T22:14:44.0374056Z deleted: sha256:e29ab5e9037a9838843863a372ef1748406bc95406369a70955134a7cae06d63 2024-08-06T22:14:44.0374691Z deleted: sha256:27d3f1ba6f4aaf25f597a8dcf5d90168631cbbaa2283d5d82528bb267dbc9aa4 2024-08-06T22:14:44.0375336Z deleted: sha256:e7b742f0c6815a00a368e84b648d3d3707bb9f50fc1111d2c99506f5e446552f 2024-08-06T22:14:44.0375961Z deleted: sha256:fd8caf0a273887946a30471b6d8565942faac80f5294f4b3f60779ba4fb1589c 2024-08-06T22:14:44.0376708Z deleted: sha256:6b4ae34804843731029058254b2a78ec9bf655c143179356562fb6792a6d387e 2024-08-06T22:14:44.0377360Z deleted: sha256:5eb13a2c08fccfe8a68cb2eedd5374ca8533971837acdecf1a6241b970be84d4 2024-08-06T22:14:44.0378083Z deleted: sha256:d782a8ac99055b7a7b36960eadce2aa26f0702a2e9de910d9a282b4ad4cf0409 2024-08-06T22:14:44.0378738Z deleted: sha256:32b90a12e0e9cbec20f720d2dd8dc1fe94d05f62a3106ccbe443c7674bcc4ce0 2024-08-06T22:14:44.0379398Z deleted: sha256:ec714b229d3a0dff80ffbf136428ab5a5271e5df7b61198f3fefcda6287ccb6c 2024-08-06T22:14:44.0380025Z deleted: sha256:2f5754399f29badb5803e0b433f6704280c4b79475b44de6728af1dfbcbb2323 2024-08-06T22:14:44.0380766Z deleted: sha256:7bf42df28ece50b780ec31bab2ecea6382f1fe26a88816cc6b9718265cf3465e 2024-08-06T22:14:44.0381423Z deleted: sha256:9a6aa7079f47ded223c38219eed1824b2ffb668af29fa9bcb485254f7e83ab21 2024-08-06T22:14:44.0382080Z deleted: sha256:e9724bcc7ec56c5acb95e352940c94b1c2a9ee52109ad8aab536930bc9728176 2024-08-06T22:14:44.0382718Z deleted: sha256:8f8ef3805ce4283573b6bf58f161a9b3331f50a60ce8d81d4ac659f1fff055c0 2024-08-06T22:14:44.0383358Z deleted: sha256:0729580f4cbe15e55af6aad9352c76c40a739f10bb9b43550680d486bd5c076b 2024-08-06T22:14:44.0384011Z deleted: sha256:4a0b4eb049da56b51a7fdfa4d8206378bdbb1b016dfc6e0bca982791a12fd5eb 2024-08-06T22:14:44.0384671Z deleted: sha256:cf4eeeecbbf6221e1ef64a38d6ba85c73e0bbf10869125ebf081e18f5ef4e9e0 2024-08-06T22:14:44.0385320Z deleted: sha256:f3f2f0324ca26243445f66f17828f234c7a77640dc5d9c44b7a90746c31a2c69 2024-08-06T22:14:44.0385960Z deleted: sha256:014362212fc6cec487fa443cf40098dfadb1562fbd6c17eaa1d987bcac2005d5 2024-08-06T22:14:44.0386604Z deleted: sha256:9d8bd07fc065e2f3faa82c6be619acdbe2a0ac5c84fa4d34564dd9ca50c1abb4 2024-08-06T22:14:44.0387261Z deleted: sha256:be515201f04cb68521fbdaf149a170e1cfee851eb7cf0d0e21cf44788d2d2738 2024-08-06T22:14:44.0387893Z deleted: sha256:a80708e23e0af5c08681444d907936033a760a3707ff65145db0d08946092e6c 2024-08-06T22:14:44.0388515Z deleted: sha256:a12ed841d341f49b467e30c6f384e0cba615d2dc38e7c23f7a3e926f8218fee0 2024-08-06T22:14:44.0389166Z deleted: sha256:508edf28b5543e1a9d07f1caeb9dddf6e77d34ef0d416dc734399f3f49335923 2024-08-06T22:14:44.0389813Z deleted: sha256:0e101de6946d43b7137acf4e038ab8d10e32dbf0c2480641c24be8ed6086abc1 2024-08-06T22:14:44.0390467Z deleted: sha256:c1c19d0e8afe1ecbbde109ac71c1d3f17a00bfcf3d50834770b8d5464ee9c8b2 2024-08-06T22:14:44.0391101Z deleted: sha256:9dc9d6055307261f20c9b3b25bff6eba44e06a05a5b285d733599015160c7dbc 2024-08-06T22:14:44.0391746Z deleted: sha256:7b94babd6975875afe9642a2b9bcf59e402dc13a48cc8be6fd65c2a5fef3ca7c 2024-08-06T22:14:44.0392396Z deleted: sha256:3873bca3dc5064ab463ae26e0a2ab4e9372bea50dd21aa9d60f740676dad7486 2024-08-06T22:14:44.0393039Z deleted: sha256:cbf23c83ce2d413d3afd8790f223a461b296a546765135e62cd690dad638d411 2024-08-06T22:14:44.0393693Z deleted: sha256:bf83fffdacd881f54312039756eae0f29ee91313edb99240a21b46a7a8a52112 2024-08-06T22:14:44.0394337Z deleted: sha256:f8a51c6f429112858d9fa585e8865b12aef6f984426a6b249a9c2405bd6bc617 2024-08-06T22:14:44.0394982Z deleted: sha256:a5f3bdddd25f3ff04aeee3425b589034eea5abb8f3356a23da937bdfae388f96 2024-08-06T22:14:44.0395644Z deleted: sha256:a63fb4f23633e92cf452b4d42c11fabe08bc5941a7b3ccfdb514d9f5df78ace9 2024-08-06T22:14:44.0396306Z deleted: sha256:55935042e44f44818cd67bfb9dbce2a9b57e383adb3caee10c52a60bb4ca2907 2024-08-06T22:14:44.0396999Z deleted: sha256:4e77f30e7a210eadab33be7d5a8e30b207fe0595cc887f660034793669ad4882 2024-08-06T22:14:44.0397657Z deleted: sha256:ff95c9e3cc791feac6467bb1a86bd910894d5aa807dbc7ef8d0656f4c313fb55 2024-08-06T22:14:44.0398312Z deleted: sha256:2d609ec5d6f0f513dc06ab2d069d1f3705dbb888c1c33b153d4d7b402a3a8445 2024-08-06T22:14:44.0398941Z deleted: sha256:6871dd9601fad505a94c55343d42d89a15ba2f3a15352e4264e4aef36be2d3b2 2024-08-06T22:14:44.0399580Z deleted: sha256:e7aab1aad704a9457fcc4018383183080c494fdd601a6d38dc2a06489e3819a6 2024-08-06T22:14:44.0400227Z deleted: sha256:77b1d18f5457391c44c5bf9bfbbdac615becb4cf2149d82856165b77ad19132e 2024-08-06T22:14:44.0400968Z deleted: sha256:5ad805ac6b7297997e85c12ead27e5b34775f55d66c96af2c9c643752c231060 2024-08-06T22:14:44.0401591Z deleted: sha256:f169358745a88fef9997e00330b991a45ceb3553651bbc391a938a896ddeaebf 2024-08-06T22:14:44.0402304Z deleted: sha256:51b3d62ea293d971aac7f5c6e7a8cac80e68e53419b75c0f225b3e14c1b78b1b 2024-08-06T22:14:44.0402952Z deleted: sha256:380d79122c2fdff1088471ef238b65b189372fe3d073caf4500c53935a53a6f5 2024-08-06T22:14:44.0403576Z deleted: sha256:cf4619277942bc0846de485114cca961fa146acbd85711f3a6ee5160681d0a8a 2024-08-06T22:14:44.0404226Z deleted: sha256:331c0a0477e8aa63e33aba884616ecd6cf5dafa7fe512374b8e8d01fb50bdf6f 2024-08-06T22:14:44.0404882Z deleted: sha256:052665ffd1620d190087d835e6fae9b128aaea33ff929767ce2a7d6b5c9c77f2 2024-08-06T22:14:44.0405511Z deleted: sha256:4534c8036770494ac61772b0ffe2379aaa62dbeb0d1307fff0164be698430f2c 2024-08-06T22:14:44.0406203Z deleted: sha256:931b7ff0cb6f494b27d31a4cbec3efe62ac54676add9c7469560302f1541ecaf 2024-08-06T22:14:44.0406585Z 2024-08-06T22:14:44.0434075Z Total reclaimed space: 15.25GB 2024-08-06T22:14:44.0517802Z Post job cleanup. 2024-08-06T22:14:44.0574119Z Post job cleanup. 2024-08-06T22:14:44.1485077Z [command]/usr/bin/git version 2024-08-06T22:14:44.1735065Z git version 2.40.1 2024-08-06T22:14:44.1774356Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/50ae9293-a5e4-44d3-b34e-3025c8dad1da' before making global git config changes 2024-08-06T22:14:44.1775323Z Adding repository directory to the temporary git global config as a safe directory 2024-08-06T22:14:44.1779691Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-06T22:14:44.1830458Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-06T22:14:44.1864533Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-06T22:14:44.2200775Z Entering 'android/libs/fbjni' 2024-08-06T22:14:44.2254398Z Entering 'third_party/FP16' 2024-08-06T22:14:44.2304461Z Entering 'third_party/FXdiv' 2024-08-06T22:14:44.2354814Z Entering 'third_party/NNPACK' 2024-08-06T22:14:44.2403744Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T22:14:44.2454742Z Entering 'third_party/XNNPACK' 2024-08-06T22:14:44.2522968Z Entering 'third_party/benchmark' 2024-08-06T22:14:44.2574890Z Entering 'third_party/cpp-httplib' 2024-08-06T22:14:44.2624810Z Entering 'third_party/cpuinfo' 2024-08-06T22:14:44.2676625Z Entering 'third_party/cudnn_frontend' 2024-08-06T22:14:44.2726973Z Entering 'third_party/cutlass' 2024-08-06T22:14:44.2786385Z Entering 'third_party/eigen' 2024-08-06T22:14:44.2839387Z Entering 'third_party/fbgemm' 2024-08-06T22:14:44.2890925Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T22:14:44.2940323Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T22:14:44.2990194Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T22:14:44.3046546Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T22:14:44.3097535Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T22:14:44.3148865Z Entering 'third_party/flatbuffers' 2024-08-06T22:14:44.3203797Z Entering 'third_party/fmt' 2024-08-06T22:14:44.3255268Z Entering 'third_party/foxi' 2024-08-06T22:14:44.3306973Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T22:14:44.3360182Z Entering 'third_party/gloo' 2024-08-06T22:14:44.3411457Z Entering 'third_party/googletest' 2024-08-06T22:14:44.3462957Z Entering 'third_party/ideep' 2024-08-06T22:14:44.3512344Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T22:14:44.3571663Z Entering 'third_party/ittapi' 2024-08-06T22:14:44.3623475Z Entering 'third_party/kineto' 2024-08-06T22:14:44.3676964Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T22:14:44.3727785Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T22:14:44.3780343Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T22:14:44.3832307Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T22:14:44.3884991Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T22:14:44.3935624Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T22:14:44.3988413Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T22:14:44.4038523Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T22:14:44.4089298Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T22:14:44.4141343Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T22:14:44.4195230Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T22:14:44.4247387Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T22:14:44.4301201Z Entering 'third_party/mimalloc' 2024-08-06T22:14:44.4353759Z Entering 'third_party/nccl/nccl' 2024-08-06T22:14:44.4403948Z Entering 'third_party/nlohmann' 2024-08-06T22:14:44.4456220Z Entering 'third_party/onnx' 2024-08-06T22:14:44.4523132Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T22:14:44.4575024Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T22:14:44.4628711Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T22:14:44.4681649Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T22:14:44.4732177Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T22:14:44.4782141Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T22:14:44.4832758Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T22:14:44.4882817Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T22:14:44.4932681Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T22:14:44.4982350Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T22:14:44.5031212Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T22:14:44.5084178Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T22:14:44.5136312Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T22:14:44.5209240Z Entering 'third_party/pocketfft' 2024-08-06T22:14:44.5262906Z Entering 'third_party/protobuf' 2024-08-06T22:14:44.5316999Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T22:14:44.5367449Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T22:14:44.5419737Z Entering 'third_party/psimd' 2024-08-06T22:14:44.5471593Z Entering 'third_party/pthreadpool' 2024-08-06T22:14:44.5523114Z Entering 'third_party/pybind11' 2024-08-06T22:14:44.5575207Z Entering 'third_party/python-peachpy' 2024-08-06T22:14:44.5626208Z Entering 'third_party/sleef' 2024-08-06T22:14:44.5677891Z Entering 'third_party/tensorpipe' 2024-08-06T22:14:44.5728341Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T22:14:44.5780886Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T22:14:44.5831518Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T22:14:44.5882627Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T22:14:44.5932373Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T22:14:44.6002949Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-06T22:14:44.6032223Z http.https://github.com/.extraheader 2024-08-06T22:14:44.6042235Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-06T22:14:44.6079722Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-06T22:14:44.6366249Z Entering 'android/libs/fbjni' 2024-08-06T22:14:44.6401328Z http.https://github.com/.extraheader 2024-08-06T22:14:44.6433912Z Entering 'third_party/FP16' 2024-08-06T22:14:44.6468682Z http.https://github.com/.extraheader 2024-08-06T22:14:44.6499474Z Entering 'third_party/FXdiv' 2024-08-06T22:14:44.6534145Z http.https://github.com/.extraheader 2024-08-06T22:14:44.6565658Z Entering 'third_party/NNPACK' 2024-08-06T22:14:44.6600133Z http.https://github.com/.extraheader 2024-08-06T22:14:44.6631261Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-06T22:14:44.6666004Z http.https://github.com/.extraheader 2024-08-06T22:14:44.6697274Z Entering 'third_party/XNNPACK' 2024-08-06T22:14:44.6733463Z http.https://github.com/.extraheader 2024-08-06T22:14:44.6782801Z Entering 'third_party/benchmark' 2024-08-06T22:14:44.6817607Z http.https://github.com/.extraheader 2024-08-06T22:14:44.6850453Z Entering 'third_party/cpp-httplib' 2024-08-06T22:14:44.6884991Z http.https://github.com/.extraheader 2024-08-06T22:14:44.6917461Z Entering 'third_party/cpuinfo' 2024-08-06T22:14:44.6963408Z http.https://github.com/.extraheader 2024-08-06T22:14:44.6982631Z Entering 'third_party/cudnn_frontend' 2024-08-06T22:14:44.7017030Z http.https://github.com/.extraheader 2024-08-06T22:14:44.7049787Z Entering 'third_party/cutlass' 2024-08-06T22:14:44.7083247Z http.https://github.com/.extraheader 2024-08-06T22:14:44.7123018Z Entering 'third_party/eigen' 2024-08-06T22:14:44.7157693Z http.https://github.com/.extraheader 2024-08-06T22:14:44.7190862Z Entering 'third_party/fbgemm' 2024-08-06T22:14:44.7224365Z http.https://github.com/.extraheader 2024-08-06T22:14:44.7256862Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-06T22:14:44.7290745Z http.https://github.com/.extraheader 2024-08-06T22:14:44.7322521Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-06T22:14:44.7355754Z http.https://github.com/.extraheader 2024-08-06T22:14:44.7387707Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-06T22:14:44.7419905Z http.https://github.com/.extraheader 2024-08-06T22:14:44.7459241Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-06T22:14:44.7491562Z http.https://github.com/.extraheader 2024-08-06T22:14:44.7523035Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-06T22:14:44.7555779Z http.https://github.com/.extraheader 2024-08-06T22:14:44.7588156Z Entering 'third_party/flatbuffers' 2024-08-06T22:14:44.7622837Z http.https://github.com/.extraheader 2024-08-06T22:14:44.7656134Z Entering 'third_party/fmt' 2024-08-06T22:14:44.7691107Z http.https://github.com/.extraheader 2024-08-06T22:14:44.7721588Z Entering 'third_party/foxi' 2024-08-06T22:14:44.7755104Z http.https://github.com/.extraheader 2024-08-06T22:14:44.7786029Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-06T22:14:44.7819916Z http.https://github.com/.extraheader 2024-08-06T22:14:44.7851565Z Entering 'third_party/gloo' 2024-08-06T22:14:44.7885545Z http.https://github.com/.extraheader 2024-08-06T22:14:44.7916328Z Entering 'third_party/googletest' 2024-08-06T22:14:44.7950158Z http.https://github.com/.extraheader 2024-08-06T22:14:44.7981522Z Entering 'third_party/ideep' 2024-08-06T22:14:44.8015372Z http.https://github.com/.extraheader 2024-08-06T22:14:44.8047135Z Entering 'third_party/ideep/mkl-dnn' 2024-08-06T22:14:44.8080525Z http.https://github.com/.extraheader 2024-08-06T22:14:44.8118996Z Entering 'third_party/ittapi' 2024-08-06T22:14:44.8154641Z http.https://github.com/.extraheader 2024-08-06T22:14:44.8185612Z Entering 'third_party/kineto' 2024-08-06T22:14:44.8221189Z http.https://github.com/.extraheader 2024-08-06T22:14:44.8254339Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-06T22:14:44.8289160Z http.https://github.com/.extraheader 2024-08-06T22:14:44.8321576Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-06T22:14:44.8359141Z http.https://github.com/.extraheader 2024-08-06T22:14:44.8392178Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-06T22:14:44.8426040Z http.https://github.com/.extraheader 2024-08-06T22:14:44.8458805Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-06T22:14:44.8492202Z http.https://github.com/.extraheader 2024-08-06T22:14:44.8523083Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-06T22:14:44.8558695Z http.https://github.com/.extraheader 2024-08-06T22:14:44.8588756Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-06T22:14:44.8623021Z http.https://github.com/.extraheader 2024-08-06T22:14:44.8657132Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-06T22:14:44.8690542Z http.https://github.com/.extraheader 2024-08-06T22:14:44.8721793Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-06T22:14:44.8756322Z http.https://github.com/.extraheader 2024-08-06T22:14:44.8788187Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-06T22:14:44.8821394Z http.https://github.com/.extraheader 2024-08-06T22:14:44.8854268Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-06T22:14:44.8887960Z http.https://github.com/.extraheader 2024-08-06T22:14:44.8921140Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-06T22:14:44.8955496Z http.https://github.com/.extraheader 2024-08-06T22:14:44.8986176Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-06T22:14:44.9018706Z http.https://github.com/.extraheader 2024-08-06T22:14:44.9051916Z Entering 'third_party/mimalloc' 2024-08-06T22:14:44.9087134Z http.https://github.com/.extraheader 2024-08-06T22:14:44.9119215Z Entering 'third_party/nccl/nccl' 2024-08-06T22:14:44.9152858Z http.https://github.com/.extraheader 2024-08-06T22:14:44.9183372Z Entering 'third_party/nlohmann' 2024-08-06T22:14:44.9217424Z http.https://github.com/.extraheader 2024-08-06T22:14:44.9250270Z Entering 'third_party/onnx' 2024-08-06T22:14:44.9284522Z http.https://github.com/.extraheader 2024-08-06T22:14:44.9332449Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-06T22:14:44.9366713Z http.https://github.com/.extraheader 2024-08-06T22:14:44.9398516Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-06T22:14:44.9432024Z http.https://github.com/.extraheader 2024-08-06T22:14:44.9467701Z Entering 'third_party/opentelemetry-cpp' 2024-08-06T22:14:44.9500359Z http.https://github.com/.extraheader 2024-08-06T22:14:44.9533750Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-06T22:14:44.9566318Z http.https://github.com/.extraheader 2024-08-06T22:14:44.9596673Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-06T22:14:44.9628927Z http.https://github.com/.extraheader 2024-08-06T22:14:44.9661134Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-06T22:14:44.9693532Z http.https://github.com/.extraheader 2024-08-06T22:14:44.9724068Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-06T22:14:44.9756570Z http.https://github.com/.extraheader 2024-08-06T22:14:44.9788468Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-06T22:14:44.9820465Z http.https://github.com/.extraheader 2024-08-06T22:14:44.9851314Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-06T22:14:44.9883876Z http.https://github.com/.extraheader 2024-08-06T22:14:44.9914429Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-06T22:14:44.9946225Z http.https://github.com/.extraheader 2024-08-06T22:14:44.9976494Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-06T22:14:45.0009643Z http.https://github.com/.extraheader 2024-08-06T22:14:45.0044365Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-06T22:14:45.0077214Z http.https://github.com/.extraheader 2024-08-06T22:14:45.0111082Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-06T22:14:45.0145515Z http.https://github.com/.extraheader 2024-08-06T22:14:45.0197463Z Entering 'third_party/pocketfft' 2024-08-06T22:14:45.0232498Z http.https://github.com/.extraheader 2024-08-06T22:14:45.0263738Z Entering 'third_party/protobuf' 2024-08-06T22:14:45.0297656Z http.https://github.com/.extraheader 2024-08-06T22:14:45.0332978Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-06T22:14:45.0366569Z http.https://github.com/.extraheader 2024-08-06T22:14:45.0397548Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-06T22:14:45.0432820Z http.https://github.com/.extraheader 2024-08-06T22:14:45.0465743Z Entering 'third_party/psimd' 2024-08-06T22:14:45.0500079Z http.https://github.com/.extraheader 2024-08-06T22:14:45.0532790Z Entering 'third_party/pthreadpool' 2024-08-06T22:14:45.0566672Z http.https://github.com/.extraheader 2024-08-06T22:14:45.0598240Z Entering 'third_party/pybind11' 2024-08-06T22:14:45.0631686Z http.https://github.com/.extraheader 2024-08-06T22:14:45.0663420Z Entering 'third_party/python-peachpy' 2024-08-06T22:14:45.0696444Z http.https://github.com/.extraheader 2024-08-06T22:14:45.0726986Z Entering 'third_party/sleef' 2024-08-06T22:14:45.0760772Z http.https://github.com/.extraheader 2024-08-06T22:14:45.0792027Z Entering 'third_party/tensorpipe' 2024-08-06T22:14:45.0825613Z http.https://github.com/.extraheader 2024-08-06T22:14:45.0858113Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-06T22:14:45.0892268Z http.https://github.com/.extraheader 2024-08-06T22:14:45.0923134Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-06T22:14:45.0957450Z http.https://github.com/.extraheader 2024-08-06T22:14:45.0988907Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-06T22:14:45.1021440Z http.https://github.com/.extraheader 2024-08-06T22:14:45.1053327Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-06T22:14:45.1086879Z http.https://github.com/.extraheader 2024-08-06T22:14:45.1119278Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-06T22:14:45.1155363Z http.https://github.com/.extraheader 2024-08-06T22:14:45.1270144Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-06T22:14:45.1314191Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-06T22:14:45.1319208Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-06T22:14:45.1319597Z ##[endgroup] 2024-08-06T22:14:53.4140019Z Cleaning up orphan processes