2024-08-20T21:35:51.7838545Z Current runner version: '2.319.1' 2024-08-20T21:35:51.7845288Z Runner name: 'i-0e2ef67d50e8ceb09' 2024-08-20T21:35:51.7846013Z Runner group name: 'Default' 2024-08-20T21:35:51.7846816Z Machine name: 'ip-10-0-74-218' 2024-08-20T21:35:51.7864143Z Testing runner upgrade compatibility 2024-08-20T21:35:52.0542876Z ##[group]GITHUB_TOKEN Permissions 2024-08-20T21:35:52.0545169Z Actions: read 2024-08-20T21:35:52.0545628Z Attestations: read 2024-08-20T21:35:52.0546250Z Checks: read 2024-08-20T21:35:52.0546698Z Contents: read 2024-08-20T21:35:52.0547112Z Deployments: read 2024-08-20T21:35:52.0547626Z Discussions: read 2024-08-20T21:35:52.0548129Z Issues: read 2024-08-20T21:35:52.0548514Z Metadata: read 2024-08-20T21:35:52.0548995Z Packages: read 2024-08-20T21:35:52.0549438Z Pages: read 2024-08-20T21:35:52.0549824Z PullRequests: read 2024-08-20T21:35:52.0550405Z RepositoryProjects: read 2024-08-20T21:35:52.0550937Z SecurityEvents: read 2024-08-20T21:35:52.0551375Z Statuses: read 2024-08-20T21:35:52.0552133Z ##[endgroup] 2024-08-20T21:35:52.0555614Z Secret source: Actions 2024-08-20T21:35:52.0556417Z Prepare workflow directory 2024-08-20T21:35:52.4123962Z Prepare all required actions 2024-08-20T21:35:52.4290397Z Getting action download info 2024-08-20T21:35:52.6825372Z Download action repository 'pytorch/test-infra@main' (SHA:0c3a2634aaa2f638c8f640e743f03d696ce1191f) 2024-08-20T21:35:53.0458002Z Download action repository 'pytorch/pytorch@main' (SHA:15b5a0b67fc3f34fb0bf1afa6f91e0c4c2b7fd8d) 2024-08-20T21:35:56.1095716Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-20T21:35:56.2468309Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-20T21:35:56.5311299Z Getting action download info 2024-08-20T21:35:56.6229401Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-20T21:35:56.8029304Z Getting action download info 2024-08-20T21:35:57.0314908Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-20T21:35:57.1913978Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/pull/133712/merge (f2fb9405c2fa9f9502a76363091cce6fd8179736) 2024-08-20T21:35:57.1916635Z ##[group] Inputs 2024-08-20T21:35:57.1917056Z build-environment: linux-jammy-py3.8-gcc11 2024-08-20T21:35:57.1920817Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]} 2024-08-20T21:35:57.1925039Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:35:57.1926030Z sync-tag: 2024-08-20T21:35:57.1926835Z timeout-minutes: 240 2024-08-20T21:35:57.1927158Z use-gha: 2024-08-20T21:35:57.1927433Z dashboard-tag: 2024-08-20T21:35:57.1927719Z s3-bucket: gha-artifacts 2024-08-20T21:35:57.1928057Z aws-role-to-assume: 2024-08-20T21:35:57.1928374Z ##[endgroup] 2024-08-20T21:35:57.1929146Z Complete job name: linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-20T21:35:57.2520568Z A job started hook has been configured by the self-hosted runner administrator 2024-08-20T21:35:57.2666530Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-20T21:35:57.2676855Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:35:57.2677410Z ##[endgroup] 2024-08-20T21:35:58.8862680Z Runner Type: amz2023.linux.2xlarge 2024-08-20T21:35:58.8863223Z Instance Type: c5.2xlarge 2024-08-20T21:35:58.8864301Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-20T21:35:58.8864889Z AMI ID: ami-06c68f701d8090592 2024-08-20T21:36:04.8672114Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-20T21:36:04.8672711Z with: 2024-08-20T21:36:04.8673427Z github-secret: *** 2024-08-20T21:36:04.8674370Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T21:36:04.8675403Z activate-with-label: false 2024-08-20T21:36:04.8675744Z label: with-ssh 2024-08-20T21:36:04.8676034Z remove-existing-keys: true 2024-08-20T21:36:04.8676378Z fail-silently: true 2024-08-20T21:36:04.8676684Z env: 2024-08-20T21:36:04.8676927Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:36:04.8677253Z ##[endgroup] 2024-08-20T21:36:04.9548961Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-20T21:36:05.2605741Z Grabbing public ssh keys from https://github.com/pytorchmergebot.keys 2024-08-20T21:36:05.3277749Z No SSH keys found for user pytorchmergebot 2024-08-20T21:36:05.3278495Z Grabbing public ssh keys from https://github.com/XuehaiPan.keys 2024-08-20T21:36:05.4022862Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-20T21:36:05.4036915Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-20T21:36:05.4062216Z Login using: ssh ec2-user@ec2-3-239-129-189.compute-1.amazonaws.com 2024-08-20T21:36:05.4063410Z All testing is done inside the container, to start an interactive session run: 2024-08-20T21:36:05.4064223Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T21:36:05.4179361Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-20T21:36:05.4179905Z with: 2024-08-20T21:36:05.4180152Z submodules: recursive 2024-08-20T21:36:05.4180468Z fetch-depth: 0 2024-08-20T21:36:05.4180736Z env: 2024-08-20T21:36:05.4180975Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:36:05.4181300Z ##[endgroup] 2024-08-20T21:36:05.4403873Z ##[group]Run retry () { 2024-08-20T21:36:05.4404229Z retry () { 2024-08-20T21:36:05.4404848Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-20T21:36:05.4405405Z } 2024-08-20T21:36:05.4405696Z echo "${GITHUB_WORKSPACE}" 2024-08-20T21:36:05.4406086Z if [ -z "${NO_SUDO}" ]; then 2024-08-20T21:36:05.4406547Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T21:36:05.4406989Z else 2024-08-20T21:36:05.4407300Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T21:36:05.4407722Z fi 2024-08-20T21:36:05.4408047Z mkdir "${GITHUB_WORKSPACE}" 2024-08-20T21:36:05.4416880Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:36:05.4417369Z env: 2024-08-20T21:36:05.4417632Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:36:05.4417964Z NO_SUDO: 2024-08-20T21:36:05.4418209Z ##[endgroup] 2024-08-20T21:36:05.4444823Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:36:07.4676963Z ##[group]Run malfet/checkout@silent-checkout 2024-08-20T21:36:07.4677397Z with: 2024-08-20T21:36:07.4677703Z ref: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:36:07.4678143Z fetch-depth: 0 2024-08-20T21:36:07.4678443Z submodules: recursive 2024-08-20T21:36:07.4678759Z quiet-checkout: true 2024-08-20T21:36:07.4679093Z repository: pytorch/pytorch 2024-08-20T21:36:07.4679584Z token: *** 2024-08-20T21:36:07.4679847Z ssh-strict: true 2024-08-20T21:36:07.4680162Z persist-credentials: true 2024-08-20T21:36:07.4680507Z clean: true 2024-08-20T21:36:07.4681015Z sparse-checkout-cone-mode: true 2024-08-20T21:36:07.4681390Z lfs: false 2024-08-20T21:36:07.4681675Z set-safe-directory: true 2024-08-20T21:36:07.4681983Z env: 2024-08-20T21:36:07.4682240Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:36:07.4682569Z ##[endgroup] 2024-08-20T21:36:07.5632327Z Syncing repository: pytorch/pytorch 2024-08-20T21:36:07.5634277Z ##[group]Getting Git version info 2024-08-20T21:36:07.5635265Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T21:36:07.5636124Z [command]/usr/bin/git version 2024-08-20T21:36:07.5636467Z git version 2.40.1 2024-08-20T21:36:07.5637775Z ##[endgroup] 2024-08-20T21:36:07.5650461Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/c04f8309-3b3a-496e-a608-44f85234be2d' before making global git config changes 2024-08-20T21:36:07.5651746Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T21:36:07.5652900Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:36:07.5691318Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T21:36:07.5695449Z ##[group]Initializing the repository 2024-08-20T21:36:07.5698340Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:36:07.5796081Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-20T21:36:07.5797207Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-20T21:36:07.5798031Z hint: of your new repositories, which will suppress this warning, call: 2024-08-20T21:36:07.5798574Z hint: 2024-08-20T21:36:07.5799197Z hint: git config --global init.defaultBranch 2024-08-20T21:36:07.5799660Z hint: 2024-08-20T21:36:07.5800179Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-20T21:36:07.5801013Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-20T21:36:07.5801874Z hint: 2024-08-20T21:36:07.5802187Z hint: git branch -m 2024-08-20T21:36:07.5802928Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-20T21:36:07.5806902Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-20T21:36:07.5843045Z ##[endgroup] 2024-08-20T21:36:07.5843592Z ##[group]Disabling automatic garbage collection 2024-08-20T21:36:07.5846375Z [command]/usr/bin/git config --local gc.auto 0 2024-08-20T21:36:07.5875062Z ##[endgroup] 2024-08-20T21:36:07.5875549Z ##[group]Setting up auth 2024-08-20T21:36:07.5881276Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T21:36:07.5911117Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T21:36:07.6166553Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T21:36:07.6198390Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T21:36:07.6452816Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T21:36:07.6499344Z ##[endgroup] 2024-08-20T21:36:07.6500282Z ##[group]Fetching the repository 2024-08-20T21:36:07.6507238Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-20T21:36:10.3669471Z remote: Enumerating objects: 1008521 2024-08-20T21:36:10.3670053Z remote: Enumerating objects: 1010108, done. 2024-08-20T21:36:10.3670799Z remote: Counting objects: 0% (1/1587) 2024-08-20T21:36:10.3673095Z remote: Counting objects: 1% (16/1587) 2024-08-20T21:36:10.3674601Z remote: Counting objects: 2% (32/1587) 2024-08-20T21:36:10.3675352Z remote: Counting objects: 3% (48/1587) 2024-08-20T21:36:10.3675829Z remote: Counting objects: 4% (64/1587) 2024-08-20T21:36:10.3676419Z remote: Counting objects: 5% (80/1587) 2024-08-20T21:36:10.3677503Z remote: Counting objects: 6% (96/1587) 2024-08-20T21:36:10.3677991Z remote: Counting objects: 7% (112/1587) 2024-08-20T21:36:10.3678664Z remote: Counting objects: 8% (127/1587) 2024-08-20T21:36:10.3679536Z remote: Counting objects: 9% (143/1587) 2024-08-20T21:36:10.3680012Z remote: Counting objects: 10% (159/1587) 2024-08-20T21:36:10.3680638Z remote: Counting objects: 11% (175/1587) 2024-08-20T21:36:10.3681551Z remote: Counting objects: 12% (191/1587) 2024-08-20T21:36:10.3682348Z remote: Counting objects: 13% (207/1587) 2024-08-20T21:36:10.3682877Z remote: Counting objects: 14% (223/1587) 2024-08-20T21:36:10.3683772Z remote: Counting objects: 15% (239/1587) 2024-08-20T21:36:10.3684767Z remote: Counting objects: 16% (254/1587) 2024-08-20T21:36:10.3685327Z remote: Counting objects: 17% (270/1587) 2024-08-20T21:36:10.3686069Z remote: Counting objects: 18% (286/1587) 2024-08-20T21:36:10.3687106Z remote: Counting objects: 19% (302/1587) 2024-08-20T21:36:10.3687588Z remote: Counting objects: 20% (318/1587) 2024-08-20T21:36:10.3688081Z remote: Counting objects: 21% (334/1587) 2024-08-20T21:36:10.3688700Z remote: Counting objects: 22% (350/1587) 2024-08-20T21:36:10.3689731Z remote: Counting objects: 23% (366/1587) 2024-08-20T21:36:10.3690775Z remote: Counting objects: 24% (381/1587) 2024-08-20T21:36:10.3691494Z remote: Counting objects: 25% (397/1587) 2024-08-20T21:36:10.3691980Z remote: Counting objects: 26% (413/1587) 2024-08-20T21:36:10.3692452Z remote: Counting objects: 27% (429/1587) 2024-08-20T21:36:10.3693566Z remote: Counting objects: 28% (445/1587) 2024-08-20T21:36:10.3694435Z remote: Counting objects: 29% (461/1587) 2024-08-20T21:36:10.3695171Z remote: Counting objects: 30% (477/1587) 2024-08-20T21:36:10.3696040Z remote: Counting objects: 31% (492/1587) 2024-08-20T21:36:10.3696681Z remote: Counting objects: 32% (508/1587) 2024-08-20T21:36:10.3697549Z remote: Counting objects: 33% (524/1587) 2024-08-20T21:36:10.3698074Z remote: Counting objects: 34% (540/1587) 2024-08-20T21:36:10.3698559Z remote: Counting objects: 35% (556/1587) 2024-08-20T21:36:10.3699032Z remote: Counting objects: 36% (572/1587) 2024-08-20T21:36:10.3699516Z remote: Counting objects: 37% (588/1587) 2024-08-20T21:36:10.3700001Z remote: Counting objects: 38% (604/1587) 2024-08-20T21:36:10.3700472Z remote: Counting objects: 39% (619/1587) 2024-08-20T21:36:10.3700959Z remote: Counting objects: 40% (635/1587) 2024-08-20T21:36:10.3701464Z remote: Counting objects: 41% (651/1587) 2024-08-20T21:36:10.3701935Z remote: Counting objects: 42% (667/1587) 2024-08-20T21:36:10.3702418Z remote: Counting objects: 43% (683/1587) 2024-08-20T21:36:10.3702906Z remote: Counting objects: 44% (699/1587) 2024-08-20T21:36:10.3703378Z remote: Counting objects: 45% (715/1587) 2024-08-20T21:36:10.3703859Z remote: Counting objects: 46% (731/1587) 2024-08-20T21:36:10.3704343Z remote: Counting objects: 47% (746/1587) 2024-08-20T21:36:10.3704825Z remote: Counting objects: 48% (762/1587) 2024-08-20T21:36:10.3705293Z remote: Counting objects: 49% (778/1587) 2024-08-20T21:36:10.3705779Z remote: Counting objects: 50% (794/1587) 2024-08-20T21:36:10.3706260Z remote: Counting objects: 51% (810/1587) 2024-08-20T21:36:10.3706755Z remote: Counting objects: 52% (826/1587) 2024-08-20T21:36:10.3707239Z remote: Counting objects: 53% (842/1587) 2024-08-20T21:36:10.3707905Z remote: Counting objects: 54% (857/1587) 2024-08-20T21:36:10.3708373Z remote: Counting objects: 55% (873/1587) 2024-08-20T21:36:10.3708857Z remote: Counting objects: 56% (889/1587) 2024-08-20T21:36:10.3709339Z remote: Counting objects: 57% (905/1587) 2024-08-20T21:36:10.3709806Z remote: Counting objects: 58% (921/1587) 2024-08-20T21:36:10.3710294Z remote: Counting objects: 59% (937/1587) 2024-08-20T21:36:10.3710776Z remote: Counting objects: 60% (953/1587) 2024-08-20T21:36:10.3711246Z remote: Counting objects: 61% (969/1587) 2024-08-20T21:36:10.3711730Z remote: Counting objects: 62% (984/1587) 2024-08-20T21:36:10.3712235Z remote: Counting objects: 63% (1000/1587) 2024-08-20T21:36:10.3712743Z remote: Counting objects: 64% (1016/1587) 2024-08-20T21:36:10.3713237Z remote: Counting objects: 65% (1032/1587) 2024-08-20T21:36:10.3713748Z remote: Counting objects: 66% (1048/1587) 2024-08-20T21:36:10.3714261Z remote: Counting objects: 67% (1064/1587) 2024-08-20T21:36:10.3714747Z remote: Counting objects: 68% (1080/1587) 2024-08-20T21:36:10.3715247Z remote: Counting objects: 69% (1096/1587) 2024-08-20T21:36:10.3715749Z remote: Counting objects: 70% (1111/1587) 2024-08-20T21:36:10.3716253Z remote: Counting objects: 71% (1127/1587) 2024-08-20T21:36:10.3716759Z remote: Counting objects: 72% (1143/1587) 2024-08-20T21:36:10.3717259Z remote: Counting objects: 73% (1159/1587) 2024-08-20T21:36:10.3717754Z remote: Counting objects: 74% (1175/1587) 2024-08-20T21:36:10.3718246Z remote: Counting objects: 75% (1191/1587) 2024-08-20T21:36:10.3718748Z remote: Counting objects: 76% (1207/1587) 2024-08-20T21:36:10.3719245Z remote: Counting objects: 77% (1222/1587) 2024-08-20T21:36:10.3719730Z remote: Counting objects: 78% (1238/1587) 2024-08-20T21:36:10.3720334Z remote: Counting objects: 79% (1254/1587) 2024-08-20T21:36:10.3720835Z remote: Counting objects: 80% (1270/1587) 2024-08-20T21:36:10.3721320Z remote: Counting objects: 81% (1286/1587) 2024-08-20T21:36:10.3721822Z remote: Counting objects: 82% (1302/1587) 2024-08-20T21:36:10.3722323Z remote: Counting objects: 83% (1318/1587) 2024-08-20T21:36:10.3722823Z remote: Counting objects: 84% (1334/1587) 2024-08-20T21:36:10.3723310Z remote: Counting objects: 85% (1349/1587) 2024-08-20T21:36:10.3723808Z remote: Counting objects: 86% (1365/1587) 2024-08-20T21:36:10.3724307Z remote: Counting objects: 87% (1381/1587) 2024-08-20T21:36:10.3724795Z remote: Counting objects: 88% (1397/1587) 2024-08-20T21:36:10.3725294Z remote: Counting objects: 89% (1413/1587) 2024-08-20T21:36:10.3725789Z remote: Counting objects: 90% (1429/1587) 2024-08-20T21:36:10.3726276Z remote: Counting objects: 91% (1445/1587) 2024-08-20T21:36:10.3726783Z remote: Counting objects: 92% (1461/1587) 2024-08-20T21:36:10.3727282Z remote: Counting objects: 93% (1476/1587) 2024-08-20T21:36:10.3727768Z remote: Counting objects: 94% (1492/1587) 2024-08-20T21:36:10.3728264Z remote: Counting objects: 95% (1508/1587) 2024-08-20T21:36:10.3728760Z remote: Counting objects: 96% (1524/1587) 2024-08-20T21:36:10.3729247Z remote: Counting objects: 97% (1540/1587) 2024-08-20T21:36:10.3729862Z remote: Counting objects: 98% (1556/1587) 2024-08-20T21:36:10.3730367Z remote: Counting objects: 99% (1572/1587) 2024-08-20T21:36:10.3730852Z remote: Counting objects: 100% (1587/1587) 2024-08-20T21:36:10.3731392Z remote: Counting objects: 100% (1587/1587), done. 2024-08-20T21:36:10.3808670Z remote: Compressing objects: 0% (1/815) 2024-08-20T21:36:10.3973097Z remote: Compressing objects: 1% (9/815) 2024-08-20T21:36:10.4443600Z remote: Compressing objects: 2% (17/815) 2024-08-20T21:36:10.5087210Z remote: Compressing objects: 3% (25/815) 2024-08-20T21:36:10.5688464Z remote: Compressing objects: 4% (33/815) 2024-08-20T21:36:10.6158136Z remote: Compressing objects: 5% (41/815) 2024-08-20T21:36:10.6535780Z remote: Compressing objects: 6% (49/815) 2024-08-20T21:36:10.6901077Z remote: Compressing objects: 7% (58/815) 2024-08-20T21:36:10.7173111Z remote: Compressing objects: 8% (66/815) 2024-08-20T21:36:10.7445630Z remote: Compressing objects: 9% (74/815) 2024-08-20T21:36:10.7697908Z remote: Compressing objects: 10% (82/815) 2024-08-20T21:36:10.7909536Z remote: Compressing objects: 11% (90/815) 2024-08-20T21:36:10.8136575Z remote: Compressing objects: 12% (98/815) 2024-08-20T21:36:10.8295653Z remote: Compressing objects: 13% (106/815) 2024-08-20T21:36:10.8421547Z remote: Compressing objects: 14% (115/815) 2024-08-20T21:36:10.8513550Z remote: Compressing objects: 15% (123/815) 2024-08-20T21:36:10.8598305Z remote: Compressing objects: 16% (131/815) 2024-08-20T21:36:10.8692240Z remote: Compressing objects: 17% (139/815) 2024-08-20T21:36:10.8755945Z remote: Compressing objects: 18% (147/815) 2024-08-20T21:36:10.8797214Z remote: Compressing objects: 19% (155/815) 2024-08-20T21:36:10.8841081Z remote: Compressing objects: 20% (163/815) 2024-08-20T21:36:10.8868430Z remote: Compressing objects: 21% (172/815) 2024-08-20T21:36:10.8883077Z remote: Compressing objects: 22% (180/815) 2024-08-20T21:36:10.8886749Z remote: Compressing objects: 23% (188/815) 2024-08-20T21:36:10.8894102Z remote: Compressing objects: 24% (196/815) 2024-08-20T21:36:10.8902663Z remote: Compressing objects: 25% (204/815) 2024-08-20T21:36:10.8911488Z remote: Compressing objects: 26% (212/815) 2024-08-20T21:36:10.8936579Z remote: Compressing objects: 27% (221/815) 2024-08-20T21:36:10.8951019Z remote: Compressing objects: 28% (229/815) 2024-08-20T21:36:10.8966571Z remote: Compressing objects: 29% (237/815) 2024-08-20T21:36:10.8973676Z remote: Compressing objects: 30% (245/815) 2024-08-20T21:36:10.8980931Z remote: Compressing objects: 31% (253/815) 2024-08-20T21:36:10.8989023Z remote: Compressing objects: 32% (261/815) 2024-08-20T21:36:10.8994104Z remote: Compressing objects: 33% (269/815) 2024-08-20T21:36:10.9002365Z remote: Compressing objects: 34% (278/815) 2024-08-20T21:36:10.9006813Z remote: Compressing objects: 35% (286/815) 2024-08-20T21:36:10.9017479Z remote: Compressing objects: 36% (294/815) 2024-08-20T21:36:10.9023069Z remote: Compressing objects: 37% (302/815) 2024-08-20T21:36:10.9029226Z remote: Compressing objects: 38% (310/815) 2024-08-20T21:36:10.9034089Z remote: Compressing objects: 39% (318/815) 2024-08-20T21:36:10.9043873Z remote: Compressing objects: 40% (326/815) 2024-08-20T21:36:10.9046830Z remote: Compressing objects: 41% (335/815) 2024-08-20T21:36:10.9052369Z remote: Compressing objects: 42% (343/815) 2024-08-20T21:36:10.9058080Z remote: Compressing objects: 43% (351/815) 2024-08-20T21:36:10.9063586Z remote: Compressing objects: 44% (359/815) 2024-08-20T21:36:10.9067651Z remote: Compressing objects: 45% (367/815) 2024-08-20T21:36:10.9077214Z remote: Compressing objects: 46% (375/815) 2024-08-20T21:36:10.9082346Z remote: Compressing objects: 47% (384/815) 2024-08-20T21:36:10.9089210Z remote: Compressing objects: 48% (392/815) 2024-08-20T21:36:10.9089835Z remote: Compressing objects: 49% (400/815) 2024-08-20T21:36:10.9094864Z remote: Compressing objects: 50% (408/815) 2024-08-20T21:36:10.9095683Z remote: Compressing objects: 51% (416/815) 2024-08-20T21:36:10.9096204Z remote: Compressing objects: 52% (424/815) 2024-08-20T21:36:10.9096740Z remote: Compressing objects: 53% (432/815) 2024-08-20T21:36:10.9101339Z remote: Compressing objects: 54% (441/815) 2024-08-20T21:36:10.9101845Z remote: Compressing objects: 55% (449/815) 2024-08-20T21:36:10.9106764Z remote: Compressing objects: 56% (457/815) 2024-08-20T21:36:10.9107642Z remote: Compressing objects: 57% (465/815) 2024-08-20T21:36:10.9108153Z remote: Compressing objects: 58% (473/815) 2024-08-20T21:36:10.9108670Z remote: Compressing objects: 59% (481/815) 2024-08-20T21:36:10.9109187Z remote: Compressing objects: 60% (489/815) 2024-08-20T21:36:10.9109681Z remote: Compressing objects: 61% (498/815) 2024-08-20T21:36:10.9110200Z remote: Compressing objects: 62% (506/815) 2024-08-20T21:36:10.9110714Z remote: Compressing objects: 63% (514/815) 2024-08-20T21:36:10.9111224Z remote: Compressing objects: 64% (522/815) 2024-08-20T21:36:10.9112119Z remote: Compressing objects: 65% (530/815) 2024-08-20T21:36:10.9118296Z remote: Compressing objects: 66% (538/815) 2024-08-20T21:36:10.9124922Z remote: Compressing objects: 67% (547/815) 2024-08-20T21:36:10.9130339Z remote: Compressing objects: 68% (555/815) 2024-08-20T21:36:10.9135943Z remote: Compressing objects: 69% (563/815) 2024-08-20T21:36:10.9136617Z remote: Compressing objects: 70% (571/815) 2024-08-20T21:36:10.9137533Z remote: Compressing objects: 71% (579/815) 2024-08-20T21:36:10.9138038Z remote: Compressing objects: 72% (587/815) 2024-08-20T21:36:10.9140213Z remote: Compressing objects: 73% (595/815) 2024-08-20T21:36:10.9143522Z remote: Compressing objects: 74% (604/815) 2024-08-20T21:36:10.9145587Z remote: Compressing objects: 75% (612/815) 2024-08-20T21:36:10.9148444Z remote: Compressing objects: 76% (620/815) 2024-08-20T21:36:10.9150598Z remote: Compressing objects: 77% (628/815) 2024-08-20T21:36:10.9152581Z remote: Compressing objects: 78% (636/815) 2024-08-20T21:36:10.9154390Z remote: Compressing objects: 79% (644/815) 2024-08-20T21:36:10.9155930Z remote: Compressing objects: 80% (652/815) 2024-08-20T21:36:10.9158093Z remote: Compressing objects: 81% (661/815) 2024-08-20T21:36:10.9160234Z remote: Compressing objects: 82% (669/815) 2024-08-20T21:36:10.9161734Z remote: Compressing objects: 83% (677/815) 2024-08-20T21:36:10.9163223Z remote: Compressing objects: 84% (685/815) 2024-08-20T21:36:10.9164937Z remote: Compressing objects: 85% (693/815) 2024-08-20T21:36:10.9166460Z remote: Compressing objects: 86% (701/815) 2024-08-20T21:36:10.9168496Z remote: Compressing objects: 87% (710/815) 2024-08-20T21:36:10.9170295Z remote: Compressing objects: 88% (718/815) 2024-08-20T21:36:10.9172244Z remote: Compressing objects: 89% (726/815) 2024-08-20T21:36:10.9173747Z remote: Compressing objects: 90% (734/815) 2024-08-20T21:36:10.9175297Z remote: Compressing objects: 91% (742/815) 2024-08-20T21:36:10.9175947Z remote: Compressing objects: 92% (750/815) 2024-08-20T21:36:10.9176452Z remote: Compressing objects: 93% (758/815) 2024-08-20T21:36:10.9177037Z remote: Compressing objects: 94% (767/815) 2024-08-20T21:36:10.9178399Z remote: Compressing objects: 95% (775/815) 2024-08-20T21:36:10.9179725Z remote: Compressing objects: 96% (783/815) 2024-08-20T21:36:10.9181163Z remote: Compressing objects: 97% (791/815) 2024-08-20T21:36:10.9182229Z remote: Compressing objects: 98% (799/815) 2024-08-20T21:36:10.9183584Z remote: Compressing objects: 99% (807/815) 2024-08-20T21:36:10.9184908Z remote: Compressing objects: 100% (815/815) 2024-08-20T21:36:10.9185466Z remote: Compressing objects: 100% (815/815), done. 2024-08-20T21:36:31.1631121Z remote: Total 1010108 (delta 1021), reused 1239 (delta 769), pack-reused 1008521 (from 1) 2024-08-20T21:36:58.1975512Z [command]/usr/bin/git rev-parse --verify --quiet 40ec5f6ddd9787aca0449b24128343ff4c4a88b3^{object} 2024-08-20T21:36:58.2004715Z 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:36:58.2009814Z ##[endgroup] 2024-08-20T21:36:58.2012341Z ##[group]Determining the checkout info 2024-08-20T21:36:58.2013468Z ##[endgroup] 2024-08-20T21:36:58.2014390Z ##[group]Checking out the ref 2024-08-20T21:36:58.2016487Z [command]/usr/bin/git checkout --quiet --force 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:36:59.5790072Z ##[endgroup] 2024-08-20T21:36:59.5790686Z ##[group]Setting up auth for fetching submodules 2024-08-20T21:36:59.5794021Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T21:36:59.5846751Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-20T21:36:59.5879418Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-20T21:36:59.5910563Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-20T21:36:59.5938302Z ##[endgroup] 2024-08-20T21:36:59.5938798Z ##[group]Fetching submodules 2024-08-20T21:36:59.5942447Z [command]/usr/bin/git submodule sync --recursive 2024-08-20T21:36:59.6225622Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-20T21:36:59.6509529Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-20T21:36:59.6511312Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-20T21:36:59.6512881Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-20T21:36:59.6514451Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-20T21:36:59.6516998Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-20T21:36:59.6519053Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-20T21:36:59.6521830Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-20T21:36:59.6524676Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-20T21:36:59.6527413Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-20T21:36:59.6530493Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-20T21:36:59.6533441Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-20T21:36:59.6536429Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-20T21:36:59.6539711Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-20T21:36:59.6543066Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-20T21:36:59.6546427Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-20T21:36:59.6550093Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:36:59.6555145Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-20T21:36:59.6558884Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-20T21:36:59.6562660Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-20T21:36:59.6566546Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-20T21:36:59.6570549Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-20T21:36:59.6575046Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-20T21:36:59.6579196Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-20T21:36:59.6583296Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-20T21:36:59.6587613Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-20T21:36:59.6592372Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-20T21:36:59.6597020Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-20T21:36:59.6601631Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-20T21:36:59.6606309Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-20T21:36:59.6611360Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-20T21:36:59.6616066Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-20T21:36:59.6621239Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-20T21:36:59.6626235Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-20T21:36:59.6632800Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-20T21:36:59.6661070Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-20T21:37:00.0231040Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-20T21:37:00.2129944Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-20T21:37:00.3848064Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-20T21:37:00.6507341Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-20T21:37:02.8812531Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-20T21:37:16.5885238Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-20T21:37:17.0080553Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-20T21:37:17.4678832Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-20T21:37:18.0521515Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-20T21:37:19.2089030Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-20T21:37:21.1905689Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-20T21:37:26.7112340Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-20T21:37:28.2357487Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-20T21:37:30.0642142Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-20T21:37:31.5062380Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-20T21:37:31.9327683Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-20T21:37:32.2504657Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-20T21:37:33.4334141Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-20T21:37:33.7886702Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-20T21:37:34.0368227Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-20T21:37:35.5293453Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-20T21:37:36.2471263Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-20T21:37:36.9178180Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-20T21:37:43.3013144Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-20T21:37:45.5843902Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-20T21:37:51.7956633Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-20T21:37:52.0513311Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-20T21:38:02.1374950Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-20T21:38:02.3230788Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-20T21:38:02.5229518Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-20T21:38:03.5112087Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-20T21:38:03.8123501Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-20T21:38:04.5354687Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-20T21:38:04.9473903Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-20T21:38:04.9588595Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-20T21:38:04.9675151Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-20T21:38:04.9906694Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-20T21:38:05.0277445Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-20T21:38:05.9541915Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-20T21:38:05.9778613Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-20T21:38:06.0216874Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-20T21:38:06.1168615Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-20T21:38:06.1506122Z Submodule path 'third_party/cudnn_frontend': checked out '23511ba176243f27b3b275da1fb3814ea805a171' 2024-08-20T21:38:06.6370767Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-20T21:38:06.8838803Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-20T21:38:06.9640217Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-20T21:38:06.9659781Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:38:06.9662069Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:38:06.9663772Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:38:06.9665469Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:38:06.9667469Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:38:06.9693567Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-20T21:38:08.0615639Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-20T21:38:08.6581357Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-20T21:38:10.6844324Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-20T21:38:11.8533228Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-20T21:38:12.1778285Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-20T21:38:12.2718623Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-20T21:38:12.6629970Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-20T21:38:12.7252569Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-20T21:38:12.7379262Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-20T21:38:12.8541698Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-20T21:38:12.8951343Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-20T21:38:12.9356134Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-20T21:38:12.9608408Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-20T21:38:13.0060591Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T21:38:13.0189429Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-20T21:38:13.0204185Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-20T21:38:13.0229109Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-20T21:38:27.0370655Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-20T21:38:27.0546552Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-20T21:38:27.1400368Z Submodule path 'third_party/kineto': checked out 'd9753139d181b9ff42872465aac0e5d3018be415' 2024-08-20T21:38:27.1417961Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:38:27.1421361Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:38:27.1423282Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:38:27.1447567Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-20T21:38:27.7786995Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-20T21:38:29.1260726Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-20T21:38:30.3804051Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-20T21:38:30.3820527Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:38:30.3822397Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:38:30.3825039Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:38:30.3827758Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:38:30.3830509Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:38:30.3834584Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:38:30.3837389Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:38:30.3839967Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:38:30.3866287Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-20T21:38:31.1839818Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-20T21:38:31.5672665Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-20T21:38:32.9499781Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-20T21:38:33.2374304Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-20T21:38:33.7689330Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-20T21:38:34.8955920Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-20T21:38:42.2017039Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-20T21:38:42.5713496Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-20T21:38:42.5895990Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-20T21:38:42.6255290Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-20T21:38:42.6387627Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-20T21:38:42.6402169Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:38:42.6428446Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-20T21:38:42.9392950Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-20T21:38:42.9577520Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-20T21:38:42.9979310Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-20T21:38:43.0977046Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-20T21:38:43.1138172Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-20T21:38:43.1537038Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-20T21:38:43.2107140Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-20T21:38:43.2477457Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-20T21:38:43.2717038Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-20T21:38:43.3731182Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-20T21:38:43.7153628Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-20T21:38:43.7187994Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-20T21:38:43.7190143Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-20T21:38:43.7219064Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-20T21:38:44.1817752Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-20T21:38:45.1204830Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-20T21:38:45.1534477Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-20T21:38:45.2195775Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-20T21:38:45.2214637Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:38:45.2216799Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:38:45.2219209Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:38:45.2222155Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:38:45.2225461Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:38:45.2228375Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:38:45.2231463Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:38:45.2234397Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:38:45.2262135Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-20T21:38:45.6420266Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-20T21:38:46.8573646Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-20T21:38:47.1466213Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-20T21:38:53.3809790Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-20T21:38:53.6565072Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-20T21:38:53.9092172Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-20T21:38:54.2152417Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-20T21:39:00.7021815Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-20T21:39:00.7482491Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-20T21:39:00.7635588Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-20T21:39:00.8594919Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-20T21:39:00.8723885Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-20T21:39:00.8866705Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-20T21:39:00.9018116Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-20T21:39:00.9032093Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:39:00.9034292Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:39:00.9058388Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-20T21:39:03.1058769Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-20T21:39:04.5329046Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-20T21:39:04.5783461Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T21:39:05.0342805Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-20T21:39:05.0456465Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-20T21:39:05.3051397Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-20T21:39:05.3071901Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:39:05.3073925Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-20T21:39:05.3100847Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-20T21:39:05.7783259Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-20T21:39:06.9815468Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-20T21:39:07.0517739Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-20T21:39:07.0612581Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-20T21:39:07.0732215Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-20T21:39:07.1097874Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-20T21:39:07.1380638Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-20T21:39:07.1792850Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-20T21:39:07.2056439Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-20T21:39:07.2074570Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:39:07.2077612Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:39:07.2079507Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:39:07.2081204Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:39:07.2108750Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-20T21:39:08.4345492Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-20T21:39:08.6611921Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-20T21:39:09.8713108Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-20T21:39:10.8723891Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-20T21:39:10.8876665Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-20T21:39:10.9542103Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-20T21:39:10.9825082Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-20T21:39:10.9841271Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:39:10.9865421Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-20T21:39:11.1767146Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-20T21:39:11.1802126Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-20T21:39:11.2081498Z Entering 'android/libs/fbjni' 2024-08-20T21:39:11.2122658Z Entering 'third_party/FP16' 2024-08-20T21:39:11.2163465Z Entering 'third_party/FXdiv' 2024-08-20T21:39:11.2204291Z Entering 'third_party/NNPACK' 2024-08-20T21:39:11.2244648Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:39:11.2283406Z Entering 'third_party/XNNPACK' 2024-08-20T21:39:11.2340419Z Entering 'third_party/benchmark' 2024-08-20T21:39:11.2378579Z Entering 'third_party/cpp-httplib' 2024-08-20T21:39:11.2419619Z Entering 'third_party/cpuinfo' 2024-08-20T21:39:11.2460295Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:39:11.2499259Z Entering 'third_party/cutlass' 2024-08-20T21:39:11.2545745Z Entering 'third_party/eigen' 2024-08-20T21:39:11.2585289Z Entering 'third_party/fbgemm' 2024-08-20T21:39:11.2625067Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:39:11.2662810Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:39:11.2700476Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:39:11.2746615Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:39:11.2784004Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:39:11.2825117Z Entering 'third_party/flatbuffers' 2024-08-20T21:39:11.2868930Z Entering 'third_party/fmt' 2024-08-20T21:39:11.2907579Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:39:11.2945577Z Entering 'third_party/gloo' 2024-08-20T21:39:11.2984496Z Entering 'third_party/googletest' 2024-08-20T21:39:11.3023594Z Entering 'third_party/ideep' 2024-08-20T21:39:11.3062599Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:39:11.3108360Z Entering 'third_party/ittapi' 2024-08-20T21:39:11.3147795Z Entering 'third_party/kineto' 2024-08-20T21:39:11.3186236Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:39:11.3225593Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:39:11.3265403Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:39:11.3304516Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:39:11.3343030Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:39:11.3381972Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:39:11.3423689Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:39:11.3461227Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:39:11.3499879Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:39:11.3538119Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:39:11.3577273Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:39:11.3615994Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:39:11.3654787Z Entering 'third_party/mimalloc' 2024-08-20T21:39:11.3694459Z Entering 'third_party/nccl/nccl' 2024-08-20T21:39:11.3732158Z Entering 'third_party/nlohmann' 2024-08-20T21:39:11.3770872Z Entering 'third_party/onnx' 2024-08-20T21:39:11.3827595Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:39:11.3867466Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:39:11.3908187Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:39:11.3949058Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:39:11.3986536Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:39:11.4023492Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:39:11.4060942Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:39:11.4100490Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:39:11.4138354Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:39:11.4176361Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:39:11.4214631Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:39:11.4254166Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:39:11.4294274Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:39:11.4355056Z Entering 'third_party/pocketfft' 2024-08-20T21:39:11.4396173Z Entering 'third_party/protobuf' 2024-08-20T21:39:11.4438798Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:39:11.4477873Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:39:11.4518603Z Entering 'third_party/psimd' 2024-08-20T21:39:11.4558812Z Entering 'third_party/pthreadpool' 2024-08-20T21:39:11.4597420Z Entering 'third_party/pybind11' 2024-08-20T21:39:11.4636579Z Entering 'third_party/python-peachpy' 2024-08-20T21:39:11.4674971Z Entering 'third_party/sleef' 2024-08-20T21:39:11.4714833Z Entering 'third_party/tensorpipe' 2024-08-20T21:39:11.4754482Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:39:11.4792479Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:39:11.4831888Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:39:11.4869800Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:39:11.4907084Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:39:11.4962460Z ##[endgroup] 2024-08-20T21:39:11.4964152Z ##[group]Persisting credentials for submodules 2024-08-20T21:39:11.4966329Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-20T21:39:11.5240688Z Entering 'android/libs/fbjni' 2024-08-20T21:39:11.5289890Z Entering 'third_party/FP16' 2024-08-20T21:39:11.5338709Z Entering 'third_party/FXdiv' 2024-08-20T21:39:11.5386503Z Entering 'third_party/NNPACK' 2024-08-20T21:39:11.5435641Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:39:11.5483892Z Entering 'third_party/XNNPACK' 2024-08-20T21:39:11.5551294Z Entering 'third_party/benchmark' 2024-08-20T21:39:11.5601862Z Entering 'third_party/cpp-httplib' 2024-08-20T21:39:11.5652942Z Entering 'third_party/cpuinfo' 2024-08-20T21:39:11.5703800Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:39:11.5754698Z Entering 'third_party/cutlass' 2024-08-20T21:39:11.5815020Z Entering 'third_party/eigen' 2024-08-20T21:39:11.5868364Z Entering 'third_party/fbgemm' 2024-08-20T21:39:11.5921125Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:39:11.5971357Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:39:11.6021857Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:39:11.6078802Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:39:11.6129173Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:39:11.6180290Z Entering 'third_party/flatbuffers' 2024-08-20T21:39:11.6235369Z Entering 'third_party/fmt' 2024-08-20T21:39:11.6285167Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:39:11.6335221Z Entering 'third_party/gloo' 2024-08-20T21:39:11.6384764Z Entering 'third_party/googletest' 2024-08-20T21:39:11.6435097Z Entering 'third_party/ideep' 2024-08-20T21:39:11.6482641Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:39:11.6538766Z Entering 'third_party/ittapi' 2024-08-20T21:39:11.6587785Z Entering 'third_party/kineto' 2024-08-20T21:39:11.6637169Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:39:11.6686304Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:39:11.6737403Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:39:11.6786272Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:39:11.6835241Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:39:11.6883180Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:39:11.6936169Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:39:11.6986256Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:39:11.7036970Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:39:11.7087293Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:39:11.7142277Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:39:11.7192506Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:39:11.7245060Z Entering 'third_party/mimalloc' 2024-08-20T21:39:11.7297723Z Entering 'third_party/nccl/nccl' 2024-08-20T21:39:11.7348570Z Entering 'third_party/nlohmann' 2024-08-20T21:39:11.7400048Z Entering 'third_party/onnx' 2024-08-20T21:39:11.7465439Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:39:11.7516066Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:39:11.7567442Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:39:11.7620336Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:39:11.7668952Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:39:11.7718045Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:39:11.7765931Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:39:11.7815666Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:39:11.7863209Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:39:11.7911300Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:39:11.7959102Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:39:11.8008901Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:39:11.8059786Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:39:11.8128911Z Entering 'third_party/pocketfft' 2024-08-20T21:39:11.8179944Z Entering 'third_party/protobuf' 2024-08-20T21:39:11.8234388Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:39:11.8283417Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:39:11.8339259Z Entering 'third_party/psimd' 2024-08-20T21:39:11.8389391Z Entering 'third_party/pthreadpool' 2024-08-20T21:39:11.8440626Z Entering 'third_party/pybind11' 2024-08-20T21:39:11.8491001Z Entering 'third_party/python-peachpy' 2024-08-20T21:39:11.8540584Z Entering 'third_party/sleef' 2024-08-20T21:39:11.8590363Z Entering 'third_party/tensorpipe' 2024-08-20T21:39:11.8640100Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:39:11.8688346Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:39:11.8738630Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:39:11.8788625Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:39:11.8838518Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:39:11.8905751Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-20T21:39:11.9183959Z Entering 'android/libs/fbjni' 2024-08-20T21:39:11.9233068Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-20T21:39:11.9247909Z Entering 'third_party/FP16' 2024-08-20T21:39:11.9294149Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-20T21:39:11.9310750Z Entering 'third_party/FXdiv' 2024-08-20T21:39:11.9356696Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-20T21:39:11.9370268Z Entering 'third_party/NNPACK' 2024-08-20T21:39:11.9416347Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-20T21:39:11.9430319Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:39:11.9475537Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-20T21:39:11.9489151Z Entering 'third_party/XNNPACK' 2024-08-20T21:39:11.9535767Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-20T21:39:11.9568408Z Entering 'third_party/benchmark' 2024-08-20T21:39:11.9615467Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:39:11.9630471Z Entering 'third_party/cpp-httplib' 2024-08-20T21:39:11.9677516Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-20T21:39:11.9692478Z Entering 'third_party/cpuinfo' 2024-08-20T21:39:11.9740605Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T21:39:11.9755831Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:39:11.9803406Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-20T21:39:11.9818293Z Entering 'third_party/cutlass' 2024-08-20T21:39:11.9864587Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-20T21:39:11.9888090Z Entering 'third_party/eigen' 2024-08-20T21:39:11.9934818Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-20T21:39:11.9952085Z Entering 'third_party/fbgemm' 2024-08-20T21:39:12.0000526Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-20T21:39:12.0015336Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:39:12.0061503Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-20T21:39:12.0075390Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:39:12.0122549Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T21:39:12.0136998Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:39:12.0183904Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-20T21:39:12.0205074Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:39:12.0251362Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:39:12.0265350Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:39:12.0312822Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-20T21:39:12.0328060Z Entering 'third_party/flatbuffers' 2024-08-20T21:39:12.0376565Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-20T21:39:12.0394165Z Entering 'third_party/fmt' 2024-08-20T21:39:12.0440291Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-20T21:39:12.0454988Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:39:12.0502609Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-20T21:39:12.0517778Z Entering 'third_party/gloo' 2024-08-20T21:39:12.0563847Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-20T21:39:12.0578863Z Entering 'third_party/googletest' 2024-08-20T21:39:12.0626315Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:39:12.0641232Z Entering 'third_party/ideep' 2024-08-20T21:39:12.0687804Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-20T21:39:12.0701631Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:39:12.0747635Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-20T21:39:12.0770305Z Entering 'third_party/ittapi' 2024-08-20T21:39:12.0820161Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-20T21:39:12.0834875Z Entering 'third_party/kineto' 2024-08-20T21:39:12.0881506Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-20T21:39:12.0896177Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:39:12.0942512Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-20T21:39:12.0956549Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:39:12.1004706Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-20T21:39:12.1020675Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:39:12.1069308Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-20T21:39:12.1081534Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:39:12.1127818Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-20T21:39:12.1142664Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:39:12.1190069Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-20T21:39:12.1204012Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:39:12.1251867Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-20T21:39:12.1268511Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:39:12.1316957Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-20T21:39:12.1331599Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:39:12.1378612Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:39:12.1394026Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:39:12.1441145Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-20T21:39:12.1456672Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:39:12.1506188Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-20T21:39:12.1523357Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:39:12.1570589Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-20T21:39:12.1584904Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:39:12.1631576Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-20T21:39:12.1648027Z Entering 'third_party/mimalloc' 2024-08-20T21:39:12.1696281Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-20T21:39:12.1712245Z Entering 'third_party/nccl/nccl' 2024-08-20T21:39:12.1758947Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-20T21:39:12.1774608Z Entering 'third_party/nlohmann' 2024-08-20T21:39:12.1820840Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-20T21:39:12.1837215Z Entering 'third_party/onnx' 2024-08-20T21:39:12.1883289Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-20T21:39:12.1913947Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:39:12.1961150Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:39:12.1976158Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:39:12.2022578Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-20T21:39:12.2039933Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:39:12.2087111Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-20T21:39:12.2104132Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:39:12.2150856Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:39:12.2165066Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:39:12.2212356Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:39:12.2227161Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:39:12.2273022Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-20T21:39:12.2287401Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:39:12.2334126Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-20T21:39:12.2349878Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:39:12.2398080Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-20T21:39:12.2412971Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:39:12.2459257Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-20T21:39:12.2473870Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:39:12.2522977Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-20T21:39:12.2536804Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:39:12.2584921Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-20T21:39:12.2602280Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:39:12.2650157Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-20T21:39:12.2666783Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:39:12.2713656Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-20T21:39:12.2748667Z Entering 'third_party/pocketfft' 2024-08-20T21:39:12.2796977Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-20T21:39:12.2812346Z Entering 'third_party/protobuf' 2024-08-20T21:39:12.2860663Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-20T21:39:12.2878867Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:39:12.2927537Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:39:12.2942241Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:39:12.2989591Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:39:12.3007005Z Entering 'third_party/psimd' 2024-08-20T21:39:12.3055919Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-20T21:39:12.3071282Z Entering 'third_party/pthreadpool' 2024-08-20T21:39:12.3120150Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-20T21:39:12.3135665Z Entering 'third_party/pybind11' 2024-08-20T21:39:12.3185056Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-20T21:39:12.3201197Z Entering 'third_party/python-peachpy' 2024-08-20T21:39:12.3249229Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-20T21:39:12.3264375Z Entering 'third_party/sleef' 2024-08-20T21:39:12.3311327Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-20T21:39:12.3326198Z Entering 'third_party/tensorpipe' 2024-08-20T21:39:12.3373946Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-20T21:39:12.3388704Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:39:12.3437022Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:39:12.3451499Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:39:12.3498532Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-20T21:39:12.3511969Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:39:12.3556814Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-20T21:39:12.3571191Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:39:12.3617126Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-20T21:39:12.3630156Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:39:12.3676278Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-20T21:39:12.4303809Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-20T21:39:12.4596031Z Entering 'android/libs/fbjni' 2024-08-20T21:39:12.4635724Z Entering 'third_party/FP16' 2024-08-20T21:39:12.4674545Z Entering 'third_party/FXdiv' 2024-08-20T21:39:12.4715182Z Entering 'third_party/NNPACK' 2024-08-20T21:39:12.4753377Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:39:12.4792510Z Entering 'third_party/XNNPACK' 2024-08-20T21:39:12.4849868Z Entering 'third_party/benchmark' 2024-08-20T21:39:12.4888863Z Entering 'third_party/cpp-httplib' 2024-08-20T21:39:12.4928946Z Entering 'third_party/cpuinfo' 2024-08-20T21:39:12.4968938Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:39:12.5007662Z Entering 'third_party/cutlass' 2024-08-20T21:39:12.5054963Z Entering 'third_party/eigen' 2024-08-20T21:39:12.5096313Z Entering 'third_party/fbgemm' 2024-08-20T21:39:12.5135743Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:39:12.5173088Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:39:12.5211060Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:39:12.5255509Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:39:12.5292230Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:39:12.5331390Z Entering 'third_party/flatbuffers' 2024-08-20T21:39:12.5372360Z Entering 'third_party/fmt' 2024-08-20T21:39:12.5413847Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:39:12.5452000Z Entering 'third_party/gloo' 2024-08-20T21:39:12.5489969Z Entering 'third_party/googletest' 2024-08-20T21:39:12.5528370Z Entering 'third_party/ideep' 2024-08-20T21:39:12.5565518Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:39:12.5610673Z Entering 'third_party/ittapi' 2024-08-20T21:39:12.5649471Z Entering 'third_party/kineto' 2024-08-20T21:39:12.5688513Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:39:12.5727267Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:39:12.5766671Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:39:12.5806434Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:39:12.5844093Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:39:12.5881918Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:39:12.5922618Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:39:12.5960472Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:39:12.6000223Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:39:12.6041235Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:39:12.6081581Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:39:12.6120154Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:39:12.6159482Z Entering 'third_party/mimalloc' 2024-08-20T21:39:12.6198831Z Entering 'third_party/nccl/nccl' 2024-08-20T21:39:12.6237700Z Entering 'third_party/nlohmann' 2024-08-20T21:39:12.6276393Z Entering 'third_party/onnx' 2024-08-20T21:39:12.6332016Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:39:12.6370580Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:39:12.6409782Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:39:12.6450374Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:39:12.6488012Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:39:12.6529042Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:39:12.6566782Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:39:12.6606007Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:39:12.6643967Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:39:12.6681874Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:39:12.6720137Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:39:12.6759921Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:39:12.6801587Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:39:12.6860801Z Entering 'third_party/pocketfft' 2024-08-20T21:39:12.6901468Z Entering 'third_party/protobuf' 2024-08-20T21:39:12.6944628Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:39:12.6983668Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:39:12.7025482Z Entering 'third_party/psimd' 2024-08-20T21:39:12.7065761Z Entering 'third_party/pthreadpool' 2024-08-20T21:39:12.7106516Z Entering 'third_party/pybind11' 2024-08-20T21:39:12.7147005Z Entering 'third_party/python-peachpy' 2024-08-20T21:39:12.7185785Z Entering 'third_party/sleef' 2024-08-20T21:39:12.7225797Z Entering 'third_party/tensorpipe' 2024-08-20T21:39:12.7265386Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:39:12.7304100Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:39:12.7342400Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:39:12.7380425Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:39:12.7419330Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:39:12.7472993Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-20T21:39:12.7761535Z Entering 'android/libs/fbjni' 2024-08-20T21:39:12.7801888Z Entering 'third_party/FP16' 2024-08-20T21:39:12.7841649Z Entering 'third_party/FXdiv' 2024-08-20T21:39:12.7881906Z Entering 'third_party/NNPACK' 2024-08-20T21:39:12.7921554Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:39:12.7960351Z Entering 'third_party/XNNPACK' 2024-08-20T21:39:12.8020613Z Entering 'third_party/benchmark' 2024-08-20T21:39:12.8060521Z Entering 'third_party/cpp-httplib' 2024-08-20T21:39:12.8099668Z Entering 'third_party/cpuinfo' 2024-08-20T21:39:12.8139304Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:39:12.8178414Z Entering 'third_party/cutlass' 2024-08-20T21:39:12.8226274Z Entering 'third_party/eigen' 2024-08-20T21:39:12.8267459Z Entering 'third_party/fbgemm' 2024-08-20T21:39:12.8307709Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:39:12.8346557Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:39:12.8385648Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:39:12.8430425Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:39:12.8468412Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:39:12.8507724Z Entering 'third_party/flatbuffers' 2024-08-20T21:39:12.8550819Z Entering 'third_party/fmt' 2024-08-20T21:39:12.8589881Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:39:12.8630091Z Entering 'third_party/gloo' 2024-08-20T21:39:12.8669413Z Entering 'third_party/googletest' 2024-08-20T21:39:12.8710214Z Entering 'third_party/ideep' 2024-08-20T21:39:12.8749413Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:39:12.8797158Z Entering 'third_party/ittapi' 2024-08-20T21:39:12.8838197Z Entering 'third_party/kineto' 2024-08-20T21:39:12.8878171Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:39:12.8918537Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:39:12.8958170Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:39:12.8997002Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:39:12.9035272Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:39:12.9072636Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:39:12.9114007Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:39:12.9151882Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:39:12.9190710Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:39:12.9230950Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:39:12.9272601Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:39:12.9312962Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:39:12.9353175Z Entering 'third_party/mimalloc' 2024-08-20T21:39:12.9393509Z Entering 'third_party/nccl/nccl' 2024-08-20T21:39:12.9433655Z Entering 'third_party/nlohmann' 2024-08-20T21:39:12.9474136Z Entering 'third_party/onnx' 2024-08-20T21:39:12.9529655Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:39:12.9569749Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:39:12.9612461Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:39:12.9654470Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:39:12.9692752Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:39:12.9731338Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:39:12.9769094Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:39:12.9809112Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:39:12.9847047Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:39:12.9884381Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:39:12.9922562Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:39:12.9963249Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:39:13.0003849Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:39:13.0063130Z Entering 'third_party/pocketfft' 2024-08-20T21:39:13.0102934Z Entering 'third_party/protobuf' 2024-08-20T21:39:13.0145806Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:39:13.0184474Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:39:13.0224759Z Entering 'third_party/psimd' 2024-08-20T21:39:13.0264459Z Entering 'third_party/pthreadpool' 2024-08-20T21:39:13.0306165Z Entering 'third_party/pybind11' 2024-08-20T21:39:13.0345712Z Entering 'third_party/python-peachpy' 2024-08-20T21:39:13.0384726Z Entering 'third_party/sleef' 2024-08-20T21:39:13.0426018Z Entering 'third_party/tensorpipe' 2024-08-20T21:39:13.0466142Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:39:13.0505448Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:39:13.0546875Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:39:13.0585148Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:39:13.0623503Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:39:13.0676094Z ##[endgroup] 2024-08-20T21:39:13.0714341Z [command]/usr/bin/git log -1 --format='%H' 2024-08-20T21:39:13.0741118Z '40ec5f6ddd9787aca0449b24128343ff4c4a88b3' 2024-08-20T21:39:13.0944264Z Prepare all required actions 2024-08-20T21:39:13.0944741Z Getting action download info 2024-08-20T21:39:13.2534002Z ##[group]Run ./.github/actions/setup-linux 2024-08-20T21:39:13.2534414Z env: 2024-08-20T21:39:13.2534669Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:13.2535131Z ##[endgroup] 2024-08-20T21:39:13.2598956Z ##[group]Run set -euo pipefail 2024-08-20T21:39:13.2599391Z set -euo pipefail 2024-08-20T21:39:13.2599749Z function get_ec2_metadata() { 2024-08-20T21:39:13.2600264Z  # Pulled from instance metadata endpoint for EC2 2024-08-20T21:39:13.2601163Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-20T21:39:13.2601920Z  category=$1 2024-08-20T21:39:13.2602429Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-20T21:39:13.2603048Z  runner_name_str=i-0e2ef67d50e8ceb09 2024-08-20T21:39:13.2603531Z  if [[ -f /.inarc ]]; then 2024-08-20T21:39:13.2604002Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-20T21:39:13.2604534Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-20T21:39:13.2605208Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-20T21:39:13.2605814Z  else 2024-08-20T21:39:13.2606269Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-20T21:39:13.2606883Z  fi 2024-08-20T21:39:13.2607141Z } 2024-08-20T21:39:13.2607463Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-20T21:39:13.2608028Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-20T21:39:13.2608668Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-20T21:39:13.2609204Z echo "system info $(uname -a)" 2024-08-20T21:39:13.2617468Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:13.2617960Z env: 2024-08-20T21:39:13.2618214Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:13.2618544Z ##[endgroup] 2024-08-20T21:39:13.2772628Z ami-id: ami-06c68f701d8090592 2024-08-20T21:39:13.2830026Z instance-id: i-0e2ef67d50e8ceb09 2024-08-20T21:39:13.2890435Z instance-type: c5.2xlarge 2024-08-20T21:39:13.2901567Z system info Linux ip-10-0-74-218.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-20T21:39:13.2926885Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T21:39:13.2927807Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T21:39:13.2934981Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:13.2935467Z env: 2024-08-20T21:39:13.2935713Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:13.2936037Z ##[endgroup] 2024-08-20T21:39:13.3007200Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-20T21:39:13.3007808Z if systemctl is-active --quiet docker; then 2024-08-20T21:39:13.3008336Z  echo "Docker daemon is running..."; 2024-08-20T21:39:13.3008776Z else 2024-08-20T21:39:13.3009241Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-20T21:39:13.3009930Z fi 2024-08-20T21:39:13.3015592Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:13.3016085Z env: 2024-08-20T21:39:13.3016348Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:13.3016663Z ##[endgroup] 2024-08-20T21:39:13.3089801Z Docker daemon is running... 2024-08-20T21:39:13.3138667Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T21:39:13.3139223Z with: 2024-08-20T21:39:13.3139467Z shell: bash 2024-08-20T21:39:13.3139730Z timeout_minutes: 5 2024-08-20T21:39:13.3140028Z max_attempts: 3 2024-08-20T21:39:13.3140319Z retry_wait_seconds: 30 2024-08-20T21:39:13.3143640Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2024-08-20T21:39:13.3146932Z polling_interval_seconds: 1 2024-08-20T21:39:13.3147294Z warning_on_retry: true 2024-08-20T21:39:13.3147613Z continue_on_error: false 2024-08-20T21:39:13.3147930Z env: 2024-08-20T21:39:13.3148181Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:13.3148503Z AWS_RETRY_MODE: standard 2024-08-20T21:39:13.3148831Z AWS_MAX_ATTEMPTS: 5 2024-08-20T21:39:13.3149149Z AWS_DEFAULT_REGION: us-east-1 2024-08-20T21:39:13.3149603Z ##[endgroup] 2024-08-20T21:39:14.5388965Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T21:39:14.5390524Z Configure a credential helper to remove this warning. See 2024-08-20T21:39:14.5391909Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T21:39:14.5392457Z 2024-08-20T21:39:14.5392561Z Login Succeeded 2024-08-20T21:39:15.3712312Z Command completed after 1 attempt(s). 2024-08-20T21:39:15.3763437Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T21:39:15.3764165Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T21:39:15.3764817Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T21:39:15.3770806Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:15.3771284Z env: 2024-08-20T21:39:15.3771546Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:15.3771873Z ##[endgroup] 2024-08-20T21:39:15.3850573Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T21:39:15.3851346Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T21:39:15.3851926Z # shellcheck disable=SC2046 2024-08-20T21:39:15.3852377Z docker stop $(docker ps -q) || true 2024-08-20T21:39:15.3852841Z # Prune all of the docker images 2024-08-20T21:39:15.3853272Z docker system prune -af 2024-08-20T21:39:15.3858808Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:15.3859285Z env: 2024-08-20T21:39:15.3859551Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:15.3859877Z ##[endgroup] 2024-08-20T21:39:15.4131364Z "docker stop" requires at least 1 argument. 2024-08-20T21:39:15.4132445Z See 'docker stop --help'. 2024-08-20T21:39:15.4132830Z 2024-08-20T21:39:15.4133217Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-20T21:39:15.4133934Z 2024-08-20T21:39:15.4134193Z Stop one or more running containers 2024-08-20T21:39:15.4292104Z Total reclaimed space: 0B 2024-08-20T21:39:15.4327608Z ##[group]Run set +e 2024-08-20T21:39:15.4327961Z set +e 2024-08-20T21:39:15.4328228Z set -x 2024-08-20T21:39:15.4328501Z  2024-08-20T21:39:15.4328803Z PT_DOMAIN=download.pytorch.org 2024-08-20T21:39:15.4329592Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-20T21:39:15.4330799Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-20T21:39:15.4331560Z # one is returned at random 2024-08-20T21:39:15.4332073Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-20T21:39:15.4332577Z  2024-08-20T21:39:15.4332870Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T21:39:15.4333469Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-20T21:39:15.4334401Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-20T21:39:15.4334943Z  2024-08-20T21:39:15.4335245Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T21:39:15.4335764Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-20T21:39:15.4336255Z  exit 1 2024-08-20T21:39:15.4336643Z  fi 2024-08-20T21:39:15.4336897Z fi 2024-08-20T21:39:15.4337149Z  2024-08-20T21:39:15.4337483Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-20T21:39:15.4337970Z  # Clean up any old records first 2024-08-20T21:39:15.4338459Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-20T21:39:15.4338893Z fi 2024-08-20T21:39:15.4339137Z  2024-08-20T21:39:15.4339548Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-20T21:39:15.4340078Z cat /etc/hosts 2024-08-20T21:39:15.4345884Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:15.4346374Z env: 2024-08-20T21:39:15.4346646Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:15.4346967Z ##[endgroup] 2024-08-20T21:39:15.4371267Z + PT_DOMAIN=download.pytorch.org 2024-08-20T21:39:15.4377287Z ++ dig -4 +short download.pytorch.org 2024-08-20T21:39:15.4378124Z ++ tail -n1 2024-08-20T21:39:15.4958840Z + RESOLVED_IP=108.138.64.116 2024-08-20T21:39:15.4959715Z + '[' -z 108.138.64.116 ']' 2024-08-20T21:39:15.4960466Z + grep -r download.pytorch.org /etc/hosts 2024-08-20T21:39:15.4970987Z 18.165.98.69 download.pytorch.org 2024-08-20T21:39:15.4972704Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-08-20T21:39:15.7751029Z + echo '108.138.64.116 download.pytorch.org' 2024-08-20T21:39:15.7751600Z + sudo tee -a /etc/hosts 2024-08-20T21:39:15.8162605Z 108.138.64.116 download.pytorch.org 2024-08-20T21:39:15.8178072Z + cat /etc/hosts 2024-08-20T21:39:15.8186685Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-20T21:39:15.8201454Z ::1 localhost6 localhost6.localdomain6 2024-08-20T21:39:15.8202008Z 108.138.64.116 download.pytorch.org 2024-08-20T21:39:15.8364258Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-20T21:39:15.8364860Z with: 2024-08-20T21:39:15.8365745Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:15.8366806Z docker-build-dir: .ci/docker 2024-08-20T21:39:15.8367174Z working-directory: . 2024-08-20T21:39:15.8367616Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:15.8368136Z force-push: false 2024-08-20T21:39:15.8368427Z env: 2024-08-20T21:39:15.8368667Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:15.8368991Z ##[endgroup] 2024-08-20T21:39:15.8389373Z ##[group]Run set -ex 2024-08-20T21:39:15.8389720Z set -ex 2024-08-20T21:39:15.8390042Z  2024-08-20T21:39:15.8390576Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-20T21:39:15.8391590Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-20T21:39:15.8392390Z # job could then download the pre-built image as usual 2024-08-20T21:39:15.8393136Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-20T21:39:15.8394034Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:15.8394671Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:15.8395215Z  2024-08-20T21:39:15.8395715Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-20T21:39:15.8396340Z  exit 0 2024-08-20T21:39:15.8396619Z else 2024-08-20T21:39:15.8396967Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:15.8397408Z fi 2024-08-20T21:39:15.8397649Z  2024-08-20T21:39:15.8398099Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-20T21:39:15.8398960Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-20T21:39:15.8399722Z  # use it as it is, but first let's extract the tag 2024-08-20T21:39:15.8400422Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-20T21:39:15.8401289Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:15.8401980Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:15.8402519Z else 2024-08-20T21:39:15.8402935Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-20T21:39:15.8403588Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:15.8404474Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:15.8405237Z fi 2024-08-20T21:39:15.8413186Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:15.8413664Z env: 2024-08-20T21:39:15.8413918Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:15.8414259Z REPO_NAME: pytorch 2024-08-20T21:39:15.8415169Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:15.8416193Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T21:39:15.8416677Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:15.8417188Z ##[endgroup] 2024-08-20T21:39:15.8442973Z + [[ ! -d .ci/docker ]] 2024-08-20T21:39:15.8443411Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-20T21:39:15.8443818Z + echo skip=false 2024-08-20T21:39:15.8445460Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-20T21:39:15.8451388Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:15.8452464Z ++ awk -F '[:,]' '{print $2}' 2024-08-20T21:39:15.8613314Z + DOCKER_TAG=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:15.8614156Z + echo docker-tag=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:15.8615718Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:15.8663059Z ##[group]Run set +e 2024-08-20T21:39:15.8663410Z set +e 2024-08-20T21:39:15.8663688Z set -x 2024-08-20T21:39:15.8663951Z  2024-08-20T21:39:15.8664191Z login() { 2024-08-20T21:39:15.8664839Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T21:39:15.8665551Z } 2024-08-20T21:39:15.8665788Z  2024-08-20T21:39:15.8666038Z retry () { 2024-08-20T21:39:15.8666414Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T21:39:15.8666832Z } 2024-08-20T21:39:15.8667079Z  2024-08-20T21:39:15.8667368Z retry login "${DOCKER_REGISTRY}" 2024-08-20T21:39:15.8667762Z  2024-08-20T21:39:15.8668208Z # Check if image already exists, if it does then skip building it 2024-08-20T21:39:15.8668903Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-20T21:39:15.8669383Z  exit 0 2024-08-20T21:39:15.8669656Z fi 2024-08-20T21:39:15.8669905Z  2024-08-20T21:39:15.8670374Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-20T21:39:15.8671194Z # be empty. The default action would be to continue rebuild the image 2024-08-20T21:39:15.8671917Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-20T21:39:15.8672576Z  # if we're on the base branch then use the parent commit 2024-08-20T21:39:15.8673141Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-20T21:39:15.8673551Z else 2024-08-20T21:39:15.8674001Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-20T21:39:15.8674800Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-20T21:39:15.8675288Z fi 2024-08-20T21:39:15.8675545Z  2024-08-20T21:39:15.8675823Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-20T21:39:15.8676308Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:15.8676751Z  2024-08-20T21:39:15.8677381Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-20T21:39:15.8678138Z  exit 0 2024-08-20T21:39:15.8678418Z fi 2024-08-20T21:39:15.8678663Z  2024-08-20T21:39:15.8679079Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-20T21:39:15.8680066Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-20T21:39:15.8680875Z  exit 1 2024-08-20T21:39:15.8681143Z fi 2024-08-20T21:39:15.8681403Z  2024-08-20T21:39:15.8681886Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-20T21:39:15.8682834Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-20T21:39:15.8683713Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-20T21:39:15.8684689Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-20T21:39:15.8685786Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-20T21:39:15.8686433Z fi 2024-08-20T21:39:15.8686737Z  2024-08-20T21:39:15.8687072Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:15.8693345Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:15.8694158Z env: 2024-08-20T21:39:15.8694430Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:15.8694771Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T21:39:15.8695237Z BASE_REVISION: 91f3d614142df02f619e44a68e3d9e0dfeba49ec 2024-08-20T21:39:15.8696317Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:15.8697375Z DOCKER_TAG: f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:15.8697971Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:15.8698478Z ##[endgroup] 2024-08-20T21:39:15.8723412Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:15.8724049Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:15.8725793Z + aws ecr get-login-password --region us-east-1 2024-08-20T21:39:15.8727464Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:16.4573492Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T21:39:16.4574302Z Configure a credential helper to remove this warning. See 2024-08-20T21:39:16.4575865Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T21:39:16.4576416Z 2024-08-20T21:39:16.4576528Z Login Succeeded 2024-08-20T21:39:16.4588911Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:16.7640629Z { 2024-08-20T21:39:16.7641180Z "schemaVersion": 2, 2024-08-20T21:39:16.7641989Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-20T21:39:16.7642568Z "config": { 2024-08-20T21:39:16.7643208Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-20T21:39:16.7643993Z "size": 43814, 2024-08-20T21:39:16.7644705Z "digest": "sha256:0d70ec8288d42bb7b6bd827783d06ac33bce3462f66d9a810a9a9ddd77c887e9" 2024-08-20T21:39:16.7645593Z }, 2024-08-20T21:39:16.7645969Z "layers": [ 2024-08-20T21:39:16.7646432Z { 2024-08-20T21:39:16.7647029Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7648288Z "size": 30440714, 2024-08-20T21:39:16.7649037Z "digest": "sha256:762bedf4b1b784c3de6c5022c5307d63123d3b7cdd59211317e37e9d477deaa0" 2024-08-20T21:39:16.7650191Z }, 2024-08-20T21:39:16.7650627Z { 2024-08-20T21:39:16.7651637Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7652933Z "size": 1823, 2024-08-20T21:39:16.7654179Z "digest": "sha256:8c0ee87aea3c3abdeb56755560432f327745225b47799d3832637903a5ec7dee" 2024-08-20T21:39:16.7655648Z }, 2024-08-20T21:39:16.7656159Z { 2024-08-20T21:39:16.7657149Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7658465Z "size": 313164825, 2024-08-20T21:39:16.7659815Z "digest": "sha256:c28adb4ed8869daa4ae6102cbbcf69b74f140920975e4751b0dfea9b1abfd15c" 2024-08-20T21:39:16.7661333Z }, 2024-08-20T21:39:16.7661862Z { 2024-08-20T21:39:16.7662727Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7663803Z "size": 804, 2024-08-20T21:39:16.7664848Z "digest": "sha256:531dcd0b8280a361aa8bbe329afe35405c64d13abc418090cce1c5a62537b888" 2024-08-20T21:39:16.7666320Z }, 2024-08-20T21:39:16.7666905Z { 2024-08-20T21:39:16.7667907Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7669021Z "size": 106, 2024-08-20T21:39:16.7669874Z "digest": "sha256:c81ff9d20d36fbd48a567918c08f020d23bd83515aec7d23fda14cd56786be81" 2024-08-20T21:39:16.7670871Z }, 2024-08-20T21:39:16.7671227Z { 2024-08-20T21:39:16.7672019Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7673068Z "size": 544, 2024-08-20T21:39:16.7674238Z "digest": "sha256:3a2d8ecdebc5053e5cb8589cd71f956e88cc3d16f21742993bfd69b85e6d0916" 2024-08-20T21:39:16.7675306Z }, 2024-08-20T21:39:16.7675896Z + exit 0 2024-08-20T21:39:16.7676261Z { 2024-08-20T21:39:16.7676997Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7678053Z "size": 1214, 2024-08-20T21:39:16.7679069Z "digest": "sha256:8538dcd89f0cac92de23cde0e22b0e7cfde631fbef3062f6387bbc78c4b8c051" 2024-08-20T21:39:16.7680255Z }, 2024-08-20T21:39:16.7680679Z { 2024-08-20T21:39:16.7681444Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7682397Z "size": 484, 2024-08-20T21:39:16.7683284Z "digest": "sha256:4a7b11d08282dad5413dab307cc0027d2c9da8629b5c69f9139ac2c5b32884b1" 2024-08-20T21:39:16.7684445Z }, 2024-08-20T21:39:16.7684870Z { 2024-08-20T21:39:16.7685661Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7686697Z "size": 110321271, 2024-08-20T21:39:16.7687367Z "digest": "sha256:ccda2116464a17da478e5a702df1aaff473b9569444e7f8a78cf1ef549212b72" 2024-08-20T21:39:16.7687988Z }, 2024-08-20T21:39:16.7688199Z { 2024-08-20T21:39:16.7688622Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7689173Z "size": 3840, 2024-08-20T21:39:16.7689805Z "digest": "sha256:78022b868ce3065df97adce1cca995409a90a7e0321a0c0a8919ede32d1d2a04" 2024-08-20T21:39:16.7690423Z }, 2024-08-20T21:39:16.7690652Z { 2024-08-20T21:39:16.7691060Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7691610Z "size": 1909, 2024-08-20T21:39:16.7692152Z "digest": "sha256:5f75f9c1c9a3e6c8b67494920cea0a5775c08b3cbd49bd01b09b02c94261db04" 2024-08-20T21:39:16.7692788Z }, 2024-08-20T21:39:16.7693047Z { 2024-08-20T21:39:16.7693537Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7694319Z "size": 701, 2024-08-20T21:39:16.7694854Z "digest": "sha256:aaf08982c0021a1af298f0fb571c20346c1e9df09c43e44acf0e1f3588dd1039" 2024-08-20T21:39:16.7695477Z }, 2024-08-20T21:39:16.7695691Z { 2024-08-20T21:39:16.7696107Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7696656Z "size": 2663383105, 2024-08-20T21:39:16.7697363Z "digest": "sha256:906cc3fc25c79e8d91b2ff6cd961c131dafa908907006971311498636c618f4e" 2024-08-20T21:39:16.7697962Z }, 2024-08-20T21:39:16.7698193Z { 2024-08-20T21:39:16.7698612Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7699145Z "size": 32, 2024-08-20T21:39:16.7699676Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:16.7700300Z }, 2024-08-20T21:39:16.7700508Z { 2024-08-20T21:39:16.7700923Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7701471Z "size": 379, 2024-08-20T21:39:16.7701993Z "digest": "sha256:88e873d7feffb728766b8b1afd9335fba2d10983015bf5c7235d16c6e0721d83" 2024-08-20T21:39:16.7702605Z }, 2024-08-20T21:39:16.7702831Z { 2024-08-20T21:39:16.7703234Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7703784Z "size": 222878, 2024-08-20T21:39:16.7704326Z "digest": "sha256:5a604d52e1bbf5414010aece200b94d9c29be535b0c445f1c780eff46c94e660" 2024-08-20T21:39:16.7704940Z }, 2024-08-20T21:39:16.7705175Z { 2024-08-20T21:39:16.7705598Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7706131Z "size": 231, 2024-08-20T21:39:16.7706659Z "digest": "sha256:63e67358db63853afb7d84d335e022e828783ba8615f70dfa3539906b82b13ea" 2024-08-20T21:39:16.7707266Z }, 2024-08-20T21:39:16.7707482Z { 2024-08-20T21:39:16.7707906Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7708458Z "size": 2913865, 2024-08-20T21:39:16.7709002Z "digest": "sha256:3a626210afeb83b0d9b1f0dadad249a35b778b7cc7e18f5949cc1a77148ebf95" 2024-08-20T21:39:16.7709630Z }, 2024-08-20T21:39:16.7709858Z { 2024-08-20T21:39:16.7710372Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7710929Z "size": 1805, 2024-08-20T21:39:16.7711463Z "digest": "sha256:a0eb49634d885aa915f63f923ebf39fd202300a3b24f0b07394d04dd57c3c683" 2024-08-20T21:39:16.7712066Z }, 2024-08-20T21:39:16.7712298Z { 2024-08-20T21:39:16.7712719Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7713256Z "size": 105, 2024-08-20T21:39:16.7713793Z "digest": "sha256:fc983e33a069ae2891a2161f9d24540f9e2728dddc9fd0e63c5b82454ab51f61" 2024-08-20T21:39:16.7714405Z }, 2024-08-20T21:39:16.7714619Z { 2024-08-20T21:39:16.7715039Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7715580Z "size": 564, 2024-08-20T21:39:16.7716099Z "digest": "sha256:d6ba8b647425b28f2cf40f0d3e2aaf52ca28231eb984ca3bc3fdb3267d74f8d6" 2024-08-20T21:39:16.7716720Z }, 2024-08-20T21:39:16.7716941Z { 2024-08-20T21:39:16.7717350Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7717893Z "size": 32, 2024-08-20T21:39:16.7718422Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:16.7719024Z }, 2024-08-20T21:39:16.7719256Z { 2024-08-20T21:39:16.7719668Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7720200Z "size": 104, 2024-08-20T21:39:16.7720727Z "digest": "sha256:800ce5e504c0c5ef446d189679b014e4d99547deec19ea367df57f38df73ea8d" 2024-08-20T21:39:16.7721332Z }, 2024-08-20T21:39:16.7721542Z { 2024-08-20T21:39:16.7721958Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7722497Z "size": 504, 2024-08-20T21:39:16.7722988Z "digest": "sha256:8335a3887870883e9ab33c78d7922178763a5738e0581ac7d3a9d1b730e9905e" 2024-08-20T21:39:16.7723575Z }, 2024-08-20T21:39:16.7723798Z { 2024-08-20T21:39:16.7724201Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7724755Z "size": 101109848, 2024-08-20T21:39:16.7725309Z "digest": "sha256:3a7c6ce8a2c5f019bdef0c2fef6f78b1a710c9e93726696167df535316c122ca" 2024-08-20T21:39:16.7725919Z }, 2024-08-20T21:39:16.7726142Z { 2024-08-20T21:39:16.7726632Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7727167Z "size": 109, 2024-08-20T21:39:16.7727690Z "digest": "sha256:a7f2a22883c001b9e0c7f51668300802f939700131ca215ebb600e3c76f3aa95" 2024-08-20T21:39:16.7728291Z }, 2024-08-20T21:39:16.7728507Z { 2024-08-20T21:39:16.7728935Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7729485Z "size": 489, 2024-08-20T21:39:16.7730102Z "digest": "sha256:43b5054ea31a137a93ab1677c0c030c426fd54e534255dc881c08d982a5cf360" 2024-08-20T21:39:16.7730714Z }, 2024-08-20T21:39:16.7730943Z { 2024-08-20T21:39:16.7731349Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7731899Z "size": 336, 2024-08-20T21:39:16.7732430Z "digest": "sha256:8da2931100c6700d3d495b362dc3c52df3b3ef7fa3e6d1145bb9083a2c96bb0a" 2024-08-20T21:39:16.7733031Z }, 2024-08-20T21:39:16.7733259Z { 2024-08-20T21:39:16.7733676Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7734211Z "size": 103, 2024-08-20T21:39:16.7734752Z "digest": "sha256:11a81df37a9c3cafd304fb86e5cbc18b2f1223705a3afdda291f6712e2b86905" 2024-08-20T21:39:16.7735371Z }, 2024-08-20T21:39:16.7735583Z { 2024-08-20T21:39:16.7735999Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7736547Z "size": 1473, 2024-08-20T21:39:16.7737076Z "digest": "sha256:076c3c501fa9f3809ba5a9aadfb65cd50da4f6015e3a7b71b60fc9381341ab39" 2024-08-20T21:39:16.7737694Z }, 2024-08-20T21:39:16.7737917Z { 2024-08-20T21:39:16.7738319Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7738866Z "size": 611525567, 2024-08-20T21:39:16.7739502Z "digest": "sha256:1c49c3d4f7fcd12cef9c59b7a881976957d419c18ca4c544e3088ae75ba4d1be" 2024-08-20T21:39:16.7740107Z }, 2024-08-20T21:39:16.7740339Z { 2024-08-20T21:39:16.7740756Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7741297Z "size": 163, 2024-08-20T21:39:16.7741832Z "digest": "sha256:4868a90a66f1b883ee705b02862ebfe0edbfc8f6eb4ffa849e5729c41a14cfeb" 2024-08-20T21:39:16.7742451Z }, 2024-08-20T21:39:16.7742664Z { 2024-08-20T21:39:16.7743096Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7743635Z "size": 2211, 2024-08-20T21:39:16.7744149Z "digest": "sha256:4e5011c5c63b068abb700845d59ed21ac55841263d1cd90cda63570f21e69818" 2024-08-20T21:39:16.7744752Z }, 2024-08-20T21:39:16.7744970Z { 2024-08-20T21:39:16.7745372Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7745921Z "size": 152, 2024-08-20T21:39:16.7746459Z "digest": "sha256:d4e444cefea48760ec5af0ccb68377eae0087817e8827ad51a2e11ee70d4ff5f" 2024-08-20T21:39:16.7747064Z }, 2024-08-20T21:39:16.7747288Z { 2024-08-20T21:39:16.7747704Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7748232Z "size": 646, 2024-08-20T21:39:16.7748770Z "digest": "sha256:81ac92e10710a795e9b9b5c1d704febcf6b405f86d96fb9f3b119c30f5d2c7f3" 2024-08-20T21:39:16.7749392Z }, 2024-08-20T21:39:16.7749609Z { 2024-08-20T21:39:16.7750034Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7750579Z "size": 32, 2024-08-20T21:39:16.7751101Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:16.7751726Z }, 2024-08-20T21:39:16.7751954Z { 2024-08-20T21:39:16.7752360Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7752911Z "size": 165, 2024-08-20T21:39:16.7753435Z "digest": "sha256:ca1818242f05ea8f9a9668e56046c7122f1ff0523e72365f3cb5f253d38667a3" 2024-08-20T21:39:16.7754032Z }, 2024-08-20T21:39:16.7754261Z { 2024-08-20T21:39:16.7754682Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7755219Z "size": 109, 2024-08-20T21:39:16.7755746Z "digest": "sha256:025b0cee3641783843deac4a1c228846cd3332053da3f8fb2d9124744d0825c2" 2024-08-20T21:39:16.7756477Z }, 2024-08-20T21:39:16.7756693Z { 2024-08-20T21:39:16.7757118Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7757667Z "size": 102, 2024-08-20T21:39:16.7758178Z "digest": "sha256:1f0b12a5c004b373780c76332f606861cec88202fb4220c4faf2d236aa63042a" 2024-08-20T21:39:16.7758781Z }, 2024-08-20T21:39:16.7759007Z { 2024-08-20T21:39:16.7759415Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7759961Z "size": 422, 2024-08-20T21:39:16.7760478Z "digest": "sha256:d63ee2a6050488c4ff7d9569705f3bd716e1e8f305825db9f7463924859b0d84" 2024-08-20T21:39:16.7761071Z }, 2024-08-20T21:39:16.7761298Z { 2024-08-20T21:39:16.7761714Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7762245Z "size": 32, 2024-08-20T21:39:16.7762772Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:16.7763392Z }, 2024-08-20T21:39:16.7763603Z { 2024-08-20T21:39:16.7764013Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7764557Z "size": 111, 2024-08-20T21:39:16.7765072Z "digest": "sha256:d040b74545d921b180e6cbfaafbd320965a8536b804b7941ab0347b9d7b834f5" 2024-08-20T21:39:16.7765677Z }, 2024-08-20T21:39:16.7765900Z { 2024-08-20T21:39:16.7766302Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7766848Z "size": 474, 2024-08-20T21:39:16.7767364Z "digest": "sha256:13bc165424849bb5c2cd7de3f72d1a4099505ff976f11a079ba62299f4761b09" 2024-08-20T21:39:16.7767962Z }, 2024-08-20T21:39:16.7768187Z { 2024-08-20T21:39:16.7768693Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7769231Z "size": 32, 2024-08-20T21:39:16.7769857Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:16.7770487Z }, 2024-08-20T21:39:16.7770702Z { 2024-08-20T21:39:16.7771120Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7771668Z "size": 112, 2024-08-20T21:39:16.7772222Z "digest": "sha256:dfb6cbc954b5bde01bb88cb42dabfa484b63eefdae9ed7c0ddf7f56ef77c66b9" 2024-08-20T21:39:16.7772868Z }, 2024-08-20T21:39:16.7773089Z { 2024-08-20T21:39:16.7773489Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7774032Z "size": 566, 2024-08-20T21:39:16.7774563Z "digest": "sha256:5b1463b1c261bc0d99a5d8dd48cc7452fcadeece1d43010599a0fb2f8dfee9b7" 2024-08-20T21:39:16.7775179Z }, 2024-08-20T21:39:16.7775399Z { 2024-08-20T21:39:16.7775812Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7776353Z "size": 43163684, 2024-08-20T21:39:16.7776878Z "digest": "sha256:172d385449d5e4a862d9b54229c96916775d9ea683b716de04d592529bba93c5" 2024-08-20T21:39:16.7777469Z }, 2024-08-20T21:39:16.7777680Z { 2024-08-20T21:39:16.7778101Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7778640Z "size": 106, 2024-08-20T21:39:16.7779146Z "digest": "sha256:cc93353090ff2554f6d9e804b424b5f18d4d2b65c4d1d56e39fb2394e8f2ce32" 2024-08-20T21:39:16.7779753Z }, 2024-08-20T21:39:16.7779975Z { 2024-08-20T21:39:16.7780376Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7780945Z "size": 295, 2024-08-20T21:39:16.7781472Z "digest": "sha256:cf364f5f62440ba1d90661380e6f98200d23dfc3e70bab35428f5c7afbd9b912" 2024-08-20T21:39:16.7782071Z }, 2024-08-20T21:39:16.7782294Z { 2024-08-20T21:39:16.7782710Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7783241Z "size": 32, 2024-08-20T21:39:16.7783774Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:16.7784389Z }, 2024-08-20T21:39:16.7784604Z { 2024-08-20T21:39:16.7785017Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7785643Z "size": 106, 2024-08-20T21:39:16.7786169Z "digest": "sha256:8d5cbba0c8c6214d0ffb769778159ff9383c2e69b65f5f765ae11ae53875a387" 2024-08-20T21:39:16.7786791Z }, 2024-08-20T21:39:16.7787020Z { 2024-08-20T21:39:16.7787428Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7787976Z "size": 425, 2024-08-20T21:39:16.7788511Z "digest": "sha256:2e373a30f750fb90cbb430e883cce1aa23d1292246f2ea10ede17832c26f6d53" 2024-08-20T21:39:16.7789122Z }, 2024-08-20T21:39:16.7789331Z { 2024-08-20T21:39:16.7789744Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7790281Z "size": 19308720, 2024-08-20T21:39:16.7790824Z "digest": "sha256:529c734f1880e18d646a45fc8dfcc22f05e608acfe618798e202c6d96de66085" 2024-08-20T21:39:16.7791422Z }, 2024-08-20T21:39:16.7791634Z { 2024-08-20T21:39:16.7792046Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7792589Z "size": 108, 2024-08-20T21:39:16.7793095Z "digest": "sha256:9c1479550b9d4b66feec866b30965a3908c809f58cba06f362598253ba819c50" 2024-08-20T21:39:16.7793966Z }, 2024-08-20T21:39:16.7794198Z { 2024-08-20T21:39:16.7794608Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7795157Z "size": 438, 2024-08-20T21:39:16.7795674Z "digest": "sha256:292d6ba03740d88893589c73591368207eba9e59f8244794ffd9cfebca7a92b1" 2024-08-20T21:39:16.7796279Z }, 2024-08-20T21:39:16.7796502Z { 2024-08-20T21:39:16.7796924Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7797471Z "size": 701, 2024-08-20T21:39:16.7798124Z "digest": "sha256:aaf08982c0021a1af298f0fb571c20346c1e9df09c43e44acf0e1f3588dd1039" 2024-08-20T21:39:16.7798744Z }, 2024-08-20T21:39:16.7798970Z { 2024-08-20T21:39:16.7799379Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7799926Z "size": 141, 2024-08-20T21:39:16.7800457Z "digest": "sha256:81b70407002c1d04c80e5d35c577fa9acfb07012a8e71912df7c33aa99562ce8" 2024-08-20T21:39:16.7801052Z }, 2024-08-20T21:39:16.7801278Z { 2024-08-20T21:39:16.7801694Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7802230Z "size": 135, 2024-08-20T21:39:16.7802772Z "digest": "sha256:1fc092fa7a5da990144d1006de1a9f4d219f9f2b85aec30d8cf82f60db1509aa" 2024-08-20T21:39:16.7803396Z }, 2024-08-20T21:39:16.7803607Z { 2024-08-20T21:39:16.7804020Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7804557Z "size": 32, 2024-08-20T21:39:16.7805081Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:16.7805698Z }, 2024-08-20T21:39:16.7805918Z { 2024-08-20T21:39:16.7806319Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7806862Z "size": 194, 2024-08-20T21:39:16.7807389Z "digest": "sha256:95d6f4de1332b0dc8f020f1b15a84ad4b8d61d6666a876b0b040534e88a0edcc" 2024-08-20T21:39:16.7807985Z }, 2024-08-20T21:39:16.7808203Z { 2024-08-20T21:39:16.7808617Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7809148Z "size": 1379, 2024-08-20T21:39:16.7809732Z "digest": "sha256:84b6af65e56859603f392b9cd3ffa2295934861b02eb0adc10001b6947ad4c13" 2024-08-20T21:39:16.7810351Z }, 2024-08-20T21:39:16.7810564Z { 2024-08-20T21:39:16.7810985Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7811534Z "size": 701, 2024-08-20T21:39:16.7812051Z "digest": "sha256:aaf08982c0021a1af298f0fb571c20346c1e9df09c43e44acf0e1f3588dd1039" 2024-08-20T21:39:16.7812661Z }, 2024-08-20T21:39:16.7812888Z { 2024-08-20T21:39:16.7813290Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7813834Z "size": 139, 2024-08-20T21:39:16.7814363Z "digest": "sha256:f89bdeb677945f4ff3d44f53f98f7d97d06a6e68f80ad569c95ba7a6225e2f38" 2024-08-20T21:39:16.7815087Z }, 2024-08-20T21:39:16.7815310Z { 2024-08-20T21:39:16.7815729Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7816269Z "size": 3735818322, 2024-08-20T21:39:16.7816823Z "digest": "sha256:350a7e72a699716ec997ab67d62a66f8cb1b173ce9383e5f543455cdf36777da" 2024-08-20T21:39:16.7817430Z }, 2024-08-20T21:39:16.7817640Z { 2024-08-20T21:39:16.7818054Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7818599Z "size": 156, 2024-08-20T21:39:16.7819119Z "digest": "sha256:b4e22c165860fcb12b0b96f9ffbd4f42d6928f44909a6ff246af514622ab3006" 2024-08-20T21:39:16.7819730Z }, 2024-08-20T21:39:16.7819951Z { 2024-08-20T21:39:16.7820354Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7820963Z "size": 849, 2024-08-20T21:39:16.7821509Z "digest": "sha256:b7a95f68be138948dfef44cb0d4131b38e0f46ef6248bfb83a0d9eedd4b99db7" 2024-08-20T21:39:16.7822135Z }, 2024-08-20T21:39:16.7822347Z { 2024-08-20T21:39:16.7822766Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7823310Z "size": 701, 2024-08-20T21:39:16.7823825Z "digest": "sha256:aaf08982c0021a1af298f0fb571c20346c1e9df09c43e44acf0e1f3588dd1039" 2024-08-20T21:39:16.7824433Z }, 2024-08-20T21:39:16.7824655Z { 2024-08-20T21:39:16.7825057Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7825602Z "size": 139, 2024-08-20T21:39:16.7826138Z "digest": "sha256:b7ae7e6f572a74ed87cfc6763f7db7e8ab70b00cc37e04f151b4e81b9ae00f18" 2024-08-20T21:39:16.7826755Z }, 2024-08-20T21:39:16.7826976Z { 2024-08-20T21:39:16.7827464Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7827999Z "size": 32, 2024-08-20T21:39:16.7828533Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:16.7829149Z }, 2024-08-20T21:39:16.7829365Z { 2024-08-20T21:39:16.7829784Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7830331Z "size": 159, 2024-08-20T21:39:16.7830868Z "digest": "sha256:b83dd8df6637d14b28f8fd5ecda7ccfad76d4c1a8c1d871c843ce9328193ac6c" 2024-08-20T21:39:16.7831496Z }, 2024-08-20T21:39:16.7831717Z { 2024-08-20T21:39:16.7832120Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7832663Z "size": 907, 2024-08-20T21:39:16.7833190Z "digest": "sha256:bb36ec6ca619d54831d85fe0039ab5fb58d803893da4f2d98768262e15e55984" 2024-08-20T21:39:16.7833808Z }, 2024-08-20T21:39:16.7834017Z { 2024-08-20T21:39:16.7834432Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7834966Z "size": 701, 2024-08-20T21:39:16.7835494Z "digest": "sha256:aaf08982c0021a1af298f0fb571c20346c1e9df09c43e44acf0e1f3588dd1039" 2024-08-20T21:39:16.7836105Z }, 2024-08-20T21:39:16.7836315Z { 2024-08-20T21:39:16.7836728Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7837273Z "size": 135, 2024-08-20T21:39:16.7837790Z "digest": "sha256:c52074bda1207120b6407464ecc855fe7f04e36a52d8b60efbb424a794a32beb" 2024-08-20T21:39:16.7838382Z }, 2024-08-20T21:39:16.7838608Z { 2024-08-20T21:39:16.7839012Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7839559Z "size": 32, 2024-08-20T21:39:16.7840091Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:16.7840713Z }, 2024-08-20T21:39:16.7840927Z { 2024-08-20T21:39:16.7841385Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7841929Z "size": 158, 2024-08-20T21:39:16.7842464Z "digest": "sha256:8f914a53bad07f5fd9d29cbb462aa752d67806de4b91e97ad4b5441ed805cf14" 2024-08-20T21:39:16.7843088Z }, 2024-08-20T21:39:16.7843315Z { 2024-08-20T21:39:16.7843720Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7844335Z "size": 1578, 2024-08-20T21:39:16.7844860Z "digest": "sha256:3d207949ee5a24d15c09664559300ce68d99f4e09d3d7abf927fb1e918f6c7ab" 2024-08-20T21:39:16.7845455Z }, 2024-08-20T21:39:16.7845685Z { 2024-08-20T21:39:16.7846100Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7846636Z "size": 32, 2024-08-20T21:39:16.7847174Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:16.7847797Z }, 2024-08-20T21:39:16.7848007Z { 2024-08-20T21:39:16.7848418Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7848960Z "size": 137, 2024-08-20T21:39:16.7849475Z "digest": "sha256:638224ef337df8b465596c5afc104d8c3139dceb0eb7048e29dc1cc6ae497573" 2024-08-20T21:39:16.7850176Z }, 2024-08-20T21:39:16.7850403Z { 2024-08-20T21:39:16.7850807Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7851360Z "size": 378, 2024-08-20T21:39:16.7851899Z "digest": "sha256:cf9d145104f43db76219176cf41bc07e1ca19c6bfafd89ba819a7ca1e76d1a6b" 2024-08-20T21:39:16.7852500Z }, 2024-08-20T21:39:16.7852721Z { 2024-08-20T21:39:16.7853136Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7853663Z "size": 32, 2024-08-20T21:39:16.7854191Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:16.7854804Z }, 2024-08-20T21:39:16.7855016Z { 2024-08-20T21:39:16.7855427Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7855965Z "size": 104, 2024-08-20T21:39:16.7856501Z "digest": "sha256:c4d2ce3c205c90ba7fdae94e14ecafdb42ab30440efacb631f29541e95859ca0" 2024-08-20T21:39:16.7857243Z }, 2024-08-20T21:39:16.7857475Z { 2024-08-20T21:39:16.7857879Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7858427Z "size": 1843, 2024-08-20T21:39:16.7858949Z "digest": "sha256:d417897013fc53f6de8d7b0b4214713595073871542e20c42071ba22d158052a" 2024-08-20T21:39:16.7859529Z }, 2024-08-20T21:39:16.7859755Z { 2024-08-20T21:39:16.7860179Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7860710Z "size": 7529593, 2024-08-20T21:39:16.7861242Z "digest": "sha256:1ba13f47f442f3a411f6ec25c6412d51709261fd97e71049b92964f50cc9be8f" 2024-08-20T21:39:16.7861839Z }, 2024-08-20T21:39:16.7862048Z { 2024-08-20T21:39:16.7862460Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7863001Z "size": 106, 2024-08-20T21:39:16.7863512Z "digest": "sha256:e6f28cfa301012334e1e91c6ad2d133801d9d50284153ebd9da6c9900aed1f7b" 2024-08-20T21:39:16.7864127Z }, 2024-08-20T21:39:16.7864354Z { 2024-08-20T21:39:16.7864760Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7865304Z "size": 163, 2024-08-20T21:39:16.7865832Z "digest": "sha256:305e45e19ff762cb7ee4c719fade35f7de552d2642f3116a9249586bbdad3043" 2024-08-20T21:39:16.7866437Z }, 2024-08-20T21:39:16.7866660Z { 2024-08-20T21:39:16.7867071Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7867599Z "size": 7943, 2024-08-20T21:39:16.7868125Z "digest": "sha256:78839cbb08a623ce9a097776428b4fd20a31a5726da6fedf8682f67f4d5ecb3c" 2024-08-20T21:39:16.7868735Z }, 2024-08-20T21:39:16.7868946Z { 2024-08-20T21:39:16.7869361Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7869909Z "size": 8067, 2024-08-20T21:39:16.7870422Z "digest": "sha256:97e6589defb285605e5c361be7f0ca344b9b3853b464100f6c6d9471f7984635" 2024-08-20T21:39:16.7871030Z }, 2024-08-20T21:39:16.7871256Z { 2024-08-20T21:39:16.7871667Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7872210Z "size": 303, 2024-08-20T21:39:16.7872745Z "digest": "sha256:00151a7db07be68fce69e7041b8ece2b33570eaa8ed7df74a309be4627d919cd" 2024-08-20T21:39:16.7873452Z }, 2024-08-20T21:39:16.7873685Z { 2024-08-20T21:39:16.7874104Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7874638Z "size": 32, 2024-08-20T21:39:16.7875174Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:16.7875796Z }, 2024-08-20T21:39:16.7876009Z { 2024-08-20T21:39:16.7876423Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7876966Z "size": 108, 2024-08-20T21:39:16.7877468Z "digest": "sha256:2853b72b213b163a214d97724c39b2824ba456e42b3e100cf849ebd5e8086bb0" 2024-08-20T21:39:16.7878066Z }, 2024-08-20T21:39:16.7878289Z { 2024-08-20T21:39:16.7878701Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7879247Z "size": 54145664, 2024-08-20T21:39:16.7879811Z "digest": "sha256:84ce0a95a6cef3dcd43f4e81ca3d17d58b3cc673471804a8909f1d2cc87d0c6d" 2024-08-20T21:39:16.7880423Z }, 2024-08-20T21:39:16.7880653Z { 2024-08-20T21:39:16.7881065Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:16.7881597Z "size": 32, 2024-08-20T21:39:16.7882128Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:16.7882746Z } 2024-08-20T21:39:16.7882955Z ] 2024-08-20T21:39:16.7883179Z } 2024-08-20T21:39:16.7992515Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T21:39:16.7992992Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T21:39:16.7993620Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-20T21:39:16.8000156Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:16.8000644Z env: 2024-08-20T21:39:16.8000899Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:16.8001840Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:16.8002825Z ##[endgroup] 2024-08-20T21:39:16.8029181Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-jammy-py3.8-gcc11-f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:16.8088186Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-20T21:39:16.8088758Z with: 2024-08-20T21:39:16.8089710Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:16.8090840Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:16.8091331Z env: 2024-08-20T21:39:16.8091588Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:16.8091912Z ##[endgroup] 2024-08-20T21:39:16.8130384Z ##[group]Run set -x 2024-08-20T21:39:16.8130705Z set -x 2024-08-20T21:39:16.8130968Z set +e 2024-08-20T21:39:16.8131236Z  2024-08-20T21:39:16.8131476Z login() { 2024-08-20T21:39:16.8132121Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T21:39:16.8132840Z } 2024-08-20T21:39:16.8133089Z  2024-08-20T21:39:16.8133380Z retry () { 2024-08-20T21:39:16.8133733Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T21:39:16.8134163Z } 2024-08-20T21:39:16.8134417Z  2024-08-20T21:39:16.8134692Z retry login "${DOCKER_REGISTRY}" 2024-08-20T21:39:16.8135092Z  2024-08-20T21:39:16.8135334Z set -e 2024-08-20T21:39:16.8135773Z # ignore output since only exit code is used for conditional 2024-08-20T21:39:16.8136459Z # only pull docker image if it's not available locally 2024-08-20T21:39:16.8137216Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-20T21:39:16.8137893Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-20T21:39:16.8138301Z fi 2024-08-20T21:39:16.8143798Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:16.8144289Z env: 2024-08-20T21:39:16.8144674Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:16.8145619Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:16.8146726Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:16.8147217Z ##[endgroup] 2024-08-20T21:39:16.8170365Z + set +e 2024-08-20T21:39:16.8171049Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:16.8171690Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:16.8173799Z + aws ecr get-login-password --region us-east-1 2024-08-20T21:39:16.8175108Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:17.4172025Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T21:39:17.4173079Z Configure a credential helper to remove this warning. See 2024-08-20T21:39:17.4173938Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T21:39:17.4174497Z 2024-08-20T21:39:17.4174603Z Login Succeeded 2024-08-20T21:39:17.4185405Z + set -e 2024-08-20T21:39:17.4187261Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:17.4332638Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:17.4334419Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:17.6818115Z f6d216893d65c7b8ae43df4daaf247db808378e9: Pulling from pytorch/pytorch-linux-jammy-py3.8-gcc11 2024-08-20T21:39:17.6832404Z 762bedf4b1b7: Pulling fs layer 2024-08-20T21:39:17.6833063Z 8c0ee87aea3c: Pulling fs layer 2024-08-20T21:39:17.6833604Z c28adb4ed886: Pulling fs layer 2024-08-20T21:39:17.6834117Z 531dcd0b8280: Pulling fs layer 2024-08-20T21:39:17.6834760Z c81ff9d20d36: Pulling fs layer 2024-08-20T21:39:17.6835665Z 3a2d8ecdebc5: Pulling fs layer 2024-08-20T21:39:17.6836251Z 8538dcd89f0c: Pulling fs layer 2024-08-20T21:39:17.6836705Z 4a7b11d08282: Pulling fs layer 2024-08-20T21:39:17.6837220Z ccda2116464a: Pulling fs layer 2024-08-20T21:39:17.6837744Z 78022b868ce3: Pulling fs layer 2024-08-20T21:39:17.6838330Z 5f75f9c1c9a3: Pulling fs layer 2024-08-20T21:39:17.6838821Z aaf08982c002: Pulling fs layer 2024-08-20T21:39:17.6839387Z 531dcd0b8280: Waiting 2024-08-20T21:39:17.6839730Z 906cc3fc25c7: Pulling fs layer 2024-08-20T21:39:17.6840129Z 3a2d8ecdebc5: Waiting 2024-08-20T21:39:17.6840438Z c81ff9d20d36: Waiting 2024-08-20T21:39:17.6840805Z 4f4fb700ef54: Pulling fs layer 2024-08-20T21:39:17.6841359Z 8538dcd89f0c: Waiting 2024-08-20T21:39:17.6841867Z ccda2116464a: Waiting 2024-08-20T21:39:17.6842187Z 88e873d7feff: Pulling fs layer 2024-08-20T21:39:17.6842521Z 4a7b11d08282: Waiting 2024-08-20T21:39:17.6842827Z 5a604d52e1bb: Pulling fs layer 2024-08-20T21:39:17.6843164Z 78022b868ce3: Waiting 2024-08-20T21:39:17.6843473Z aaf08982c002: Waiting 2024-08-20T21:39:17.6843779Z 63e67358db63: Pulling fs layer 2024-08-20T21:39:17.6844167Z 5f75f9c1c9a3: Waiting 2024-08-20T21:39:17.6844626Z 906cc3fc25c7: Waiting 2024-08-20T21:39:17.6845166Z 3a626210afeb: Pulling fs layer 2024-08-20T21:39:17.6845796Z a0eb49634d88: Pulling fs layer 2024-08-20T21:39:17.6846322Z fc983e33a069: Pulling fs layer 2024-08-20T21:39:17.6846752Z d6ba8b647425: Pulling fs layer 2024-08-20T21:39:17.6847088Z 4f4fb700ef54: Waiting 2024-08-20T21:39:17.6847443Z 3a626210afeb: Waiting 2024-08-20T21:39:17.6847757Z 800ce5e504c0: Pulling fs layer 2024-08-20T21:39:17.6848256Z 5a604d52e1bb: Waiting 2024-08-20T21:39:17.6848540Z 63e67358db63: Waiting 2024-08-20T21:39:17.6848847Z 8335a3887870: Pulling fs layer 2024-08-20T21:39:17.6849205Z 3a7c6ce8a2c5: Pulling fs layer 2024-08-20T21:39:17.6849538Z a0eb49634d88: Waiting 2024-08-20T21:39:17.6849925Z 88e873d7feff: Waiting 2024-08-20T21:39:17.6850442Z a7f2a22883c0: Pulling fs layer 2024-08-20T21:39:17.6850775Z d6ba8b647425: Waiting 2024-08-20T21:39:17.6851082Z fc983e33a069: Waiting 2024-08-20T21:39:17.6851393Z 43b5054ea31a: Pulling fs layer 2024-08-20T21:39:17.6851727Z 800ce5e504c0: Waiting 2024-08-20T21:39:17.6852023Z 3a7c6ce8a2c5: Waiting 2024-08-20T21:39:17.6852320Z a7f2a22883c0: Waiting 2024-08-20T21:39:17.6852624Z 8da2931100c6: Pulling fs layer 2024-08-20T21:39:17.6852964Z 8335a3887870: Waiting 2024-08-20T21:39:17.6853257Z 43b5054ea31a: Waiting 2024-08-20T21:39:17.6853557Z 11a81df37a9c: Pulling fs layer 2024-08-20T21:39:17.6853916Z 076c3c501fa9: Pulling fs layer 2024-08-20T21:39:17.6854256Z 8da2931100c6: Waiting 2024-08-20T21:39:17.6854605Z 1c49c3d4f7fc: Pulling fs layer 2024-08-20T21:39:17.6854946Z 11a81df37a9c: Waiting 2024-08-20T21:39:17.6855280Z 076c3c501fa9: Waiting 2024-08-20T21:39:17.6855583Z 4868a90a66f1: Pulling fs layer 2024-08-20T21:39:17.6856018Z 4e5011c5c63b: Pulling fs layer 2024-08-20T21:39:17.6856381Z d4e444cefea4: Pulling fs layer 2024-08-20T21:39:17.6856750Z 81ac92e10710: Pulling fs layer 2024-08-20T21:39:17.6857104Z ca1818242f05: Pulling fs layer 2024-08-20T21:39:17.6857457Z 025b0cee3641: Pulling fs layer 2024-08-20T21:39:17.6857799Z 1f0b12a5c004: Pulling fs layer 2024-08-20T21:39:17.6858194Z d63ee2a60504: Pulling fs layer 2024-08-20T21:39:17.6858539Z 4e5011c5c63b: Waiting 2024-08-20T21:39:17.6858836Z d040b74545d9: Pulling fs layer 2024-08-20T21:39:17.6859213Z 025b0cee3641: Waiting 2024-08-20T21:39:17.6859555Z ca1818242f05: Waiting 2024-08-20T21:39:17.6859842Z 1f0b12a5c004: Waiting 2024-08-20T21:39:17.6860131Z 81ac92e10710: Waiting 2024-08-20T21:39:17.6860439Z 13bc16542484: Pulling fs layer 2024-08-20T21:39:17.6860811Z d63ee2a60504: Waiting 2024-08-20T21:39:17.6861111Z 13bc16542484: Waiting 2024-08-20T21:39:17.6861523Z dfb6cbc954b5: Pulling fs layer 2024-08-20T21:39:17.6861862Z d4e444cefea4: Waiting 2024-08-20T21:39:17.6862176Z 5b1463b1c261: Pulling fs layer 2024-08-20T21:39:17.6862528Z 172d385449d5: Pulling fs layer 2024-08-20T21:39:17.6862877Z cc93353090ff: Pulling fs layer 2024-08-20T21:39:17.6863229Z cf364f5f6244: Pulling fs layer 2024-08-20T21:39:17.6863699Z 172d385449d5: Waiting 2024-08-20T21:39:17.6863985Z cc93353090ff: Waiting 2024-08-20T21:39:17.6864276Z cf364f5f6244: Waiting 2024-08-20T21:39:17.6864577Z 8d5cbba0c8c6: Pulling fs layer 2024-08-20T21:39:17.6864921Z 5b1463b1c261: Waiting 2024-08-20T21:39:17.6865232Z 2e373a30f750: Pulling fs layer 2024-08-20T21:39:17.6865572Z 529c734f1880: Pulling fs layer 2024-08-20T21:39:17.6865920Z 9c1479550b9d: Pulling fs layer 2024-08-20T21:39:17.6866268Z 292d6ba03740: Pulling fs layer 2024-08-20T21:39:17.6866605Z 81b70407002c: Pulling fs layer 2024-08-20T21:39:17.6866939Z 529c734f1880: Waiting 2024-08-20T21:39:17.6867245Z 1fc092fa7a5d: Pulling fs layer 2024-08-20T21:39:17.6867571Z 292d6ba03740: Waiting 2024-08-20T21:39:17.6867882Z 95d6f4de1332: Pulling fs layer 2024-08-20T21:39:17.6868270Z 1fc092fa7a5d: Waiting 2024-08-20T21:39:17.6868604Z 84b6af65e568: Pulling fs layer 2024-08-20T21:39:17.6868947Z 81b70407002c: Waiting 2024-08-20T21:39:17.6869254Z f89bdeb67794: Pulling fs layer 2024-08-20T21:39:17.6869589Z 95d6f4de1332: Waiting 2024-08-20T21:39:17.6869900Z 350a7e72a699: Pulling fs layer 2024-08-20T21:39:17.6870244Z 84b6af65e568: Waiting 2024-08-20T21:39:17.6870541Z b4e22c165860: Pulling fs layer 2024-08-20T21:39:17.6870906Z b7a95f68be13: Pulling fs layer 2024-08-20T21:39:17.6871268Z b7ae7e6f572a: Pulling fs layer 2024-08-20T21:39:17.6871598Z 9c1479550b9d: Waiting 2024-08-20T21:39:17.6871913Z b83dd8df6637: Pulling fs layer 2024-08-20T21:39:17.6872256Z 350a7e72a699: Waiting 2024-08-20T21:39:17.6872558Z bb36ec6ca619: Pulling fs layer 2024-08-20T21:39:17.6872907Z b7a95f68be13: Waiting 2024-08-20T21:39:17.6873207Z b4e22c165860: Waiting 2024-08-20T21:39:17.6873505Z c52074bda120: Pulling fs layer 2024-08-20T21:39:17.6873865Z 8f914a53bad0: Pulling fs layer 2024-08-20T21:39:17.6874225Z 3d207949ee5a: Pulling fs layer 2024-08-20T21:39:17.6874565Z 638224ef337d: Pulling fs layer 2024-08-20T21:39:17.6875001Z bb36ec6ca619: Waiting 2024-08-20T21:39:17.6875314Z cf9d145104f4: Pulling fs layer 2024-08-20T21:39:17.6875648Z b83dd8df6637: Waiting 2024-08-20T21:39:17.6875947Z 3d207949ee5a: Waiting 2024-08-20T21:39:17.6876242Z 8f914a53bad0: Waiting 2024-08-20T21:39:17.6876539Z c4d2ce3c205c: Pulling fs layer 2024-08-20T21:39:17.6876890Z c52074bda120: Waiting 2024-08-20T21:39:17.6877194Z d417897013fc: Pulling fs layer 2024-08-20T21:39:17.6877522Z cf9d145104f4: Waiting 2024-08-20T21:39:17.6877835Z 1ba13f47f442: Pulling fs layer 2024-08-20T21:39:17.6878189Z e6f28cfa3010: Pulling fs layer 2024-08-20T21:39:17.6878530Z 305e45e19ff7: Pulling fs layer 2024-08-20T21:39:17.6878888Z 78839cbb08a6: Pulling fs layer 2024-08-20T21:39:17.6879223Z d417897013fc: Waiting 2024-08-20T21:39:17.6879522Z 97e6589defb2: Pulling fs layer 2024-08-20T21:39:17.6879865Z 1ba13f47f442: Waiting 2024-08-20T21:39:17.6880162Z e6f28cfa3010: Waiting 2024-08-20T21:39:17.6880615Z 00151a7db07b: Pulling fs layer 2024-08-20T21:39:17.6881267Z 2853b72b213b: Pulling fs layer 2024-08-20T21:39:17.6881913Z 84ce0a95a6ce: Pulling fs layer 2024-08-20T21:39:17.6882483Z 78839cbb08a6: Waiting 2024-08-20T21:39:17.6882984Z 2853b72b213b: Waiting 2024-08-20T21:39:17.6883489Z 84ce0a95a6ce: Waiting 2024-08-20T21:39:17.6884012Z 97e6589defb2: Waiting 2024-08-20T21:39:17.6884539Z 638224ef337d: Waiting 2024-08-20T21:39:17.6885072Z f89bdeb67794: Waiting 2024-08-20T21:39:17.7524528Z 8c0ee87aea3c: Verifying Checksum 2024-08-20T21:39:17.7525142Z 8c0ee87aea3c: Download complete 2024-08-20T21:39:17.8238182Z 531dcd0b8280: Verifying Checksum 2024-08-20T21:39:17.8238695Z 531dcd0b8280: Download complete 2024-08-20T21:39:17.8916244Z c81ff9d20d36: Verifying Checksum 2024-08-20T21:39:17.8916680Z c81ff9d20d36: Download complete 2024-08-20T21:39:17.9662972Z 3a2d8ecdebc5: Verifying Checksum 2024-08-20T21:39:17.9663624Z 3a2d8ecdebc5: Download complete 2024-08-20T21:39:18.0346552Z 8538dcd89f0c: Download complete 2024-08-20T21:39:18.0354908Z 762bedf4b1b7: Download complete 2024-08-20T21:39:18.1077932Z 4a7b11d08282: Download complete 2024-08-20T21:39:18.1933553Z 78022b868ce3: Download complete 2024-08-20T21:39:18.2623275Z 5f75f9c1c9a3: Verifying Checksum 2024-08-20T21:39:18.2624058Z 5f75f9c1c9a3: Download complete 2024-08-20T21:39:18.3311742Z aaf08982c002: Download complete 2024-08-20T21:39:19.0135324Z 762bedf4b1b7: Pull complete 2024-08-20T21:39:19.0279057Z 8c0ee87aea3c: Pull complete 2024-08-20T21:39:19.1847203Z ccda2116464a: Verifying Checksum 2024-08-20T21:39:19.1847705Z ccda2116464a: Download complete 2024-08-20T21:39:19.1956937Z 4f4fb700ef54: Verifying Checksum 2024-08-20T21:39:19.1957436Z 4f4fb700ef54: Download complete 2024-08-20T21:39:19.2758977Z 88e873d7feff: Verifying Checksum 2024-08-20T21:39:19.2759734Z 88e873d7feff: Download complete 2024-08-20T21:39:19.3517144Z 5a604d52e1bb: Verifying Checksum 2024-08-20T21:39:19.3517719Z 5a604d52e1bb: Download complete 2024-08-20T21:39:19.4295971Z 63e67358db63: Verifying Checksum 2024-08-20T21:39:19.4296800Z 63e67358db63: Download complete 2024-08-20T21:39:19.5287513Z 3a626210afeb: Verifying Checksum 2024-08-20T21:39:19.5287990Z 3a626210afeb: Download complete 2024-08-20T21:39:19.6081375Z a0eb49634d88: Verifying Checksum 2024-08-20T21:39:19.6081843Z a0eb49634d88: Download complete 2024-08-20T21:39:19.7122924Z fc983e33a069: Download complete 2024-08-20T21:39:19.8035696Z d6ba8b647425: Verifying Checksum 2024-08-20T21:39:19.8036460Z d6ba8b647425: Download complete 2024-08-20T21:39:19.8769802Z 800ce5e504c0: Download complete 2024-08-20T21:39:19.9803439Z 8335a3887870: Verifying Checksum 2024-08-20T21:39:19.9804096Z 8335a3887870: Download complete 2024-08-20T21:39:20.8842774Z c28adb4ed886: Verifying Checksum 2024-08-20T21:39:20.8843478Z c28adb4ed886: Download complete 2024-08-20T21:39:20.9574315Z a7f2a22883c0: Download complete 2024-08-20T21:39:21.0339722Z 3a7c6ce8a2c5: Verifying Checksum 2024-08-20T21:39:21.0340278Z 3a7c6ce8a2c5: Download complete 2024-08-20T21:39:21.0354252Z 43b5054ea31a: Verifying Checksum 2024-08-20T21:39:21.0354778Z 43b5054ea31a: Download complete 2024-08-20T21:39:21.0960471Z 8da2931100c6: Download complete 2024-08-20T21:39:21.1116096Z 11a81df37a9c: Verifying Checksum 2024-08-20T21:39:21.1116730Z 11a81df37a9c: Download complete 2024-08-20T21:39:21.1662318Z 076c3c501fa9: Verifying Checksum 2024-08-20T21:39:21.1663047Z 076c3c501fa9: Download complete 2024-08-20T21:39:21.2250184Z 4868a90a66f1: Verifying Checksum 2024-08-20T21:39:21.2250674Z 4868a90a66f1: Download complete 2024-08-20T21:39:21.2980756Z 4e5011c5c63b: Download complete 2024-08-20T21:39:21.3732196Z d4e444cefea4: Download complete 2024-08-20T21:39:21.4446209Z 81ac92e10710: Verifying Checksum 2024-08-20T21:39:21.4447019Z 81ac92e10710: Download complete 2024-08-20T21:39:21.5141021Z ca1818242f05: Download complete 2024-08-20T21:39:21.5812693Z 025b0cee3641: Download complete 2024-08-20T21:39:21.6694372Z 1f0b12a5c004: Download complete 2024-08-20T21:39:21.7703960Z d63ee2a60504: Verifying Checksum 2024-08-20T21:39:21.7704742Z d63ee2a60504: Download complete 2024-08-20T21:39:21.8507536Z d040b74545d9: Download complete 2024-08-20T21:39:21.9312306Z 13bc16542484: Verifying Checksum 2024-08-20T21:39:21.9312847Z 13bc16542484: Download complete 2024-08-20T21:39:22.0103172Z dfb6cbc954b5: Verifying Checksum 2024-08-20T21:39:22.0103907Z dfb6cbc954b5: Download complete 2024-08-20T21:39:22.0982150Z 5b1463b1c261: Verifying Checksum 2024-08-20T21:39:22.0982818Z 5b1463b1c261: Download complete 2024-08-20T21:39:22.5890073Z 172d385449d5: Verifying Checksum 2024-08-20T21:39:22.5890861Z 172d385449d5: Download complete 2024-08-20T21:39:22.6663932Z cc93353090ff: Download complete 2024-08-20T21:39:22.7492433Z cf364f5f6244: Verifying Checksum 2024-08-20T21:39:22.7493357Z cf364f5f6244: Download complete 2024-08-20T21:39:22.8361538Z 8d5cbba0c8c6: Verifying Checksum 2024-08-20T21:39:22.8362212Z 8d5cbba0c8c6: Download complete 2024-08-20T21:39:22.8994438Z 2e373a30f750: Download complete 2024-08-20T21:39:23.1424543Z 529c734f1880: Verifying Checksum 2024-08-20T21:39:23.1425131Z 529c734f1880: Download complete 2024-08-20T21:39:23.2052274Z 9c1479550b9d: Verifying Checksum 2024-08-20T21:39:23.2052880Z 9c1479550b9d: Download complete 2024-08-20T21:39:23.2716312Z 292d6ba03740: Verifying Checksum 2024-08-20T21:39:23.2716949Z 292d6ba03740: Download complete 2024-08-20T21:39:23.3475082Z 81b70407002c: Download complete 2024-08-20T21:39:23.4116013Z 1fc092fa7a5d: Verifying Checksum 2024-08-20T21:39:23.4116787Z 1fc092fa7a5d: Download complete 2024-08-20T21:39:23.4837321Z 95d6f4de1332: Verifying Checksum 2024-08-20T21:39:23.4838051Z 95d6f4de1332: Download complete 2024-08-20T21:39:23.5560207Z 84b6af65e568: Verifying Checksum 2024-08-20T21:39:23.5560729Z 84b6af65e568: Download complete 2024-08-20T21:39:23.6341800Z f89bdeb67794: Download complete 2024-08-20T21:39:27.2799533Z 1c49c3d4f7fc: Verifying Checksum 2024-08-20T21:39:27.2800276Z 1c49c3d4f7fc: Download complete 2024-08-20T21:39:27.3561057Z b4e22c165860: Verifying Checksum 2024-08-20T21:39:27.3561871Z b4e22c165860: Download complete 2024-08-20T21:39:27.4255867Z b7a95f68be13: Verifying Checksum 2024-08-20T21:39:27.4256527Z b7a95f68be13: Download complete 2024-08-20T21:39:27.5285802Z b7ae7e6f572a: Verifying Checksum 2024-08-20T21:39:27.5286310Z b7ae7e6f572a: Download complete 2024-08-20T21:39:27.5986644Z b83dd8df6637: Verifying Checksum 2024-08-20T21:39:27.5987844Z b83dd8df6637: Download complete 2024-08-20T21:39:27.6758076Z bb36ec6ca619: Verifying Checksum 2024-08-20T21:39:27.6758572Z bb36ec6ca619: Download complete 2024-08-20T21:39:27.7488932Z c52074bda120: Verifying Checksum 2024-08-20T21:39:27.7489762Z c52074bda120: Download complete 2024-08-20T21:39:27.8206308Z 8f914a53bad0: Verifying Checksum 2024-08-20T21:39:27.8206912Z 8f914a53bad0: Download complete 2024-08-20T21:39:27.8850607Z 3d207949ee5a: Download complete 2024-08-20T21:39:28.0069656Z 638224ef337d: Verifying Checksum 2024-08-20T21:39:28.0070352Z 638224ef337d: Download complete 2024-08-20T21:39:28.0874832Z cf9d145104f4: Verifying Checksum 2024-08-20T21:39:28.0875337Z cf9d145104f4: Download complete 2024-08-20T21:39:28.1752422Z c4d2ce3c205c: Verifying Checksum 2024-08-20T21:39:28.1753473Z c4d2ce3c205c: Download complete 2024-08-20T21:39:28.2552370Z d417897013fc: Download complete 2024-08-20T21:39:28.4313019Z 1ba13f47f442: Verifying Checksum 2024-08-20T21:39:28.4314162Z 1ba13f47f442: Download complete 2024-08-20T21:39:28.5061975Z e6f28cfa3010: Verifying Checksum 2024-08-20T21:39:28.5062585Z e6f28cfa3010: Download complete 2024-08-20T21:39:28.5768748Z 305e45e19ff7: Verifying Checksum 2024-08-20T21:39:28.5769542Z 305e45e19ff7: Download complete 2024-08-20T21:39:28.6492197Z 78839cbb08a6: Download complete 2024-08-20T21:39:28.7475082Z 97e6589defb2: Download complete 2024-08-20T21:39:28.8291025Z 00151a7db07b: Verifying Checksum 2024-08-20T21:39:28.8291726Z 00151a7db07b: Download complete 2024-08-20T21:39:28.9035771Z 2853b72b213b: Download complete 2024-08-20T21:39:29.4889925Z c28adb4ed886: Pull complete 2024-08-20T21:39:29.5028686Z 84ce0a95a6ce: Verifying Checksum 2024-08-20T21:39:29.5029380Z 84ce0a95a6ce: Download complete 2024-08-20T21:39:29.6214224Z 531dcd0b8280: Pull complete 2024-08-20T21:39:29.7276810Z c81ff9d20d36: Pull complete 2024-08-20T21:39:29.8308221Z 3a2d8ecdebc5: Pull complete 2024-08-20T21:39:30.0005527Z 8538dcd89f0c: Pull complete 2024-08-20T21:39:30.1217239Z 4a7b11d08282: Pull complete 2024-08-20T21:39:51.8001363Z ccda2116464a: Pull complete 2024-08-20T21:39:52.0361436Z 78022b868ce3: Pull complete 2024-08-20T21:39:52.2570277Z 5f75f9c1c9a3: Pull complete 2024-08-20T21:39:52.4702822Z aaf08982c002: Pull complete 2024-08-20T21:39:59.3468372Z 906cc3fc25c7: Download complete 2024-08-20T21:40:13.3237052Z 350a7e72a699: Verifying Checksum 2024-08-20T21:40:13.3237531Z 350a7e72a699: Download complete 2024-08-20T21:41:00.8070571Z 906cc3fc25c7: Pull complete 2024-08-20T21:41:00.9569397Z 4f4fb700ef54: Pull complete 2024-08-20T21:41:01.0632038Z 88e873d7feff: Pull complete 2024-08-20T21:41:01.2908349Z 5a604d52e1bb: Pull complete 2024-08-20T21:41:01.4736442Z 63e67358db63: Pull complete 2024-08-20T21:41:01.6591977Z 3a626210afeb: Pull complete 2024-08-20T21:41:01.7938729Z a0eb49634d88: Pull complete 2024-08-20T21:41:01.8902507Z fc983e33a069: Pull complete 2024-08-20T21:41:02.0143679Z d6ba8b647425: Pull complete 2024-08-20T21:41:02.1888126Z 800ce5e504c0: Pull complete 2024-08-20T21:41:02.2862663Z 8335a3887870: Pull complete 2024-08-20T21:41:05.5235048Z 3a7c6ce8a2c5: Pull complete 2024-08-20T21:41:05.6335118Z a7f2a22883c0: Pull complete 2024-08-20T21:41:05.6899670Z 43b5054ea31a: Pull complete 2024-08-20T21:41:05.8006058Z 8da2931100c6: Pull complete 2024-08-20T21:41:05.9263806Z 11a81df37a9c: Pull complete 2024-08-20T21:41:06.0391731Z 076c3c501fa9: Pull complete 2024-08-20T21:41:29.7803336Z 1c49c3d4f7fc: Pull complete 2024-08-20T21:41:29.9990187Z 4868a90a66f1: Pull complete 2024-08-20T21:41:30.1248735Z 4e5011c5c63b: Pull complete 2024-08-20T21:41:30.3755202Z d4e444cefea4: Pull complete 2024-08-20T21:41:30.6154561Z 81ac92e10710: Pull complete 2024-08-20T21:41:30.9452377Z ca1818242f05: Pull complete 2024-08-20T21:41:31.1106594Z 025b0cee3641: Pull complete 2024-08-20T21:41:31.3999424Z 1f0b12a5c004: Pull complete 2024-08-20T21:41:31.6253162Z d63ee2a60504: Pull complete 2024-08-20T21:41:32.0090301Z d040b74545d9: Pull complete 2024-08-20T21:41:32.1069838Z 13bc16542484: Pull complete 2024-08-20T21:41:32.2717079Z dfb6cbc954b5: Pull complete 2024-08-20T21:41:32.3213411Z 5b1463b1c261: Pull complete 2024-08-20T21:41:34.9314559Z 172d385449d5: Pull complete 2024-08-20T21:41:35.1143134Z cc93353090ff: Pull complete 2024-08-20T21:41:35.1308077Z cf364f5f6244: Pull complete 2024-08-20T21:41:35.1677336Z 8d5cbba0c8c6: Pull complete 2024-08-20T21:41:35.1834965Z 2e373a30f750: Pull complete 2024-08-20T21:41:35.4800047Z 529c734f1880: Pull complete 2024-08-20T21:41:35.4965915Z 9c1479550b9d: Pull complete 2024-08-20T21:41:35.5127291Z 292d6ba03740: Pull complete 2024-08-20T21:41:35.5453406Z 81b70407002c: Pull complete 2024-08-20T21:41:35.5618380Z 1fc092fa7a5d: Pull complete 2024-08-20T21:41:35.5913138Z 95d6f4de1332: Pull complete 2024-08-20T21:41:35.6086982Z 84b6af65e568: Pull complete 2024-08-20T21:41:35.6381526Z f89bdeb67794: Pull complete 2024-08-20T21:43:24.0320072Z 350a7e72a699: Pull complete 2024-08-20T21:43:24.1524944Z b4e22c165860: Pull complete 2024-08-20T21:43:24.3471277Z b7a95f68be13: Pull complete 2024-08-20T21:43:24.7262677Z b7ae7e6f572a: Pull complete 2024-08-20T21:43:25.0987698Z b83dd8df6637: Pull complete 2024-08-20T21:43:25.1872755Z bb36ec6ca619: Pull complete 2024-08-20T21:43:25.5648443Z c52074bda120: Pull complete 2024-08-20T21:43:25.9394464Z 8f914a53bad0: Pull complete 2024-08-20T21:43:26.1366735Z 3d207949ee5a: Pull complete 2024-08-20T21:43:26.5152577Z 638224ef337d: Pull complete 2024-08-20T21:43:26.7121521Z cf9d145104f4: Pull complete 2024-08-20T21:43:27.0172289Z c4d2ce3c205c: Pull complete 2024-08-20T21:43:27.1303722Z d417897013fc: Pull complete 2024-08-20T21:43:27.6132196Z 1ba13f47f442: Pull complete 2024-08-20T21:43:27.6961486Z e6f28cfa3010: Pull complete 2024-08-20T21:43:27.7288357Z 305e45e19ff7: Pull complete 2024-08-20T21:43:27.7726609Z 78839cbb08a6: Pull complete 2024-08-20T21:43:27.8647824Z 97e6589defb2: Pull complete 2024-08-20T21:43:28.0477150Z 00151a7db07b: Pull complete 2024-08-20T21:43:28.4496837Z 2853b72b213b: Pull complete 2024-08-20T21:43:30.7553796Z 84ce0a95a6ce: Pull complete 2024-08-20T21:43:30.8574261Z Digest: sha256:9c16298ecf1ba0ec71e25268bd4244acb2b2c8c60846e6c8a5e8eb35b9eff836 2024-08-20T21:43:30.8663173Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:43:30.8774508Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:43:30.8848418Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T21:43:30.8849353Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T21:43:30.8858199Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:30.8858710Z env: 2024-08-20T21:43:30.8858972Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:30.8859288Z ##[endgroup] 2024-08-20T21:43:30.8973886Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T21:43:30.8974657Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T21:43:30.8975339Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-20T21:43:30.8975971Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:43:30.8982006Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:30.8982495Z env: 2024-08-20T21:43:30.8982754Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:30.8983066Z ##[endgroup] 2024-08-20T21:43:31.3679945Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T21:43:31.3925216Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.9/site-packages (5.9.1) 2024-08-20T21:43:31.3932827Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.9/site-packages (11.525.84) 2024-08-20T21:43:31.6171021Z Prepare all required actions 2024-08-20T21:43:31.6171921Z Getting action download info 2024-08-20T21:43:31.7185792Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-20T21:43:31.9214196Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-20T21:43:32.0356887Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-20T21:43:32.0357373Z with: 2024-08-20T21:43:32.0357652Z name: linux-jammy-py3.8-gcc11 2024-08-20T21:43:32.0358016Z s3-bucket: gha-artifacts 2024-08-20T21:43:32.0358338Z env: 2024-08-20T21:43:32.0358595Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:32.0358910Z ##[endgroup] 2024-08-20T21:43:32.0487007Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T21:43:32.0487444Z with: 2024-08-20T21:43:32.0487724Z name: linux-jammy-py3.8-gcc11 2024-08-20T21:43:32.0488326Z s3-bucket: gha-artifacts 2024-08-20T21:43:32.0488707Z region: us-east-1 2024-08-20T21:43:32.0488977Z env: 2024-08-20T21:43:32.0489240Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:32.0489660Z ##[endgroup] 2024-08-20T21:43:32.5752502Z (node:46625) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T21:43:32.5753314Z 2024-08-20T21:43:32.5753577Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T21:43:32.5754294Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T21:43:32.5755219Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T21:43:32.6466033Z Found 1 objects with prefix pytorch/pytorch/10479309237/linux-jammy-py3.8-gcc11/ 2024-08-20T21:43:32.6467124Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T21:43:36.3846793Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T21:43:36.3854379Z Artifact download has finished successfully 2024-08-20T21:43:36.4030296Z ##[group]Run unzip -o artifacts.zip 2024-08-20T21:43:36.4030708Z unzip -o artifacts.zip 2024-08-20T21:43:36.4037698Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:36.4038201Z env: 2024-08-20T21:43:36.4038450Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:36.4038781Z ##[endgroup] 2024-08-20T21:43:36.4096618Z Archive: artifacts.zip 2024-08-20T21:43:36.4097482Z creating: dist/ 2024-08-20T21:43:37.3451467Z inflating: dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl 2024-08-20T21:43:37.3452147Z creating: build/custom_test_artifacts/ 2024-08-20T21:43:37.3452825Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-20T21:43:37.3453553Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-20T21:43:37.3455015Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-20T21:43:37.3456475Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T21:43:37.3457509Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-20T21:43:37.3458992Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T21:43:37.3460637Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T21:43:37.3461701Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T21:43:37.3463531Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T21:43:37.3465344Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T21:43:37.3466826Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T21:43:37.3468542Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T21:43:37.3471127Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T21:43:37.3473516Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T21:43:37.3474985Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T21:43:37.3476177Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T21:43:37.3477396Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T21:43:37.3478621Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T21:43:37.3479631Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-20T21:43:37.3480764Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-20T21:43:37.3481726Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-20T21:43:37.3482785Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-20T21:43:37.3483996Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-20T21:43:37.3485149Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-20T21:43:37.3486230Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-20T21:43:37.3487333Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-20T21:43:37.3488455Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-20T21:43:37.3489684Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-20T21:43:37.3490819Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-20T21:43:37.3492206Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-20T21:43:37.3493507Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-20T21:43:37.3621708Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-20T21:43:37.3623689Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-20T21:43:37.3625856Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-20T21:43:37.3627371Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-20T21:43:37.3628591Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-20T21:43:37.3629741Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-20T21:43:37.3630942Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-20T21:43:37.3632144Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-20T21:43:37.3633324Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-20T21:43:37.3634515Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-20T21:43:37.3635690Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-20T21:43:37.3645109Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-20T21:43:37.3720777Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-20T21:43:37.3723170Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:43:37.3724872Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-20T21:43:37.3726355Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-20T21:43:37.3727286Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-20T21:43:37.3728227Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-20T21:43:37.3729120Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-20T21:43:37.3729985Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-20T21:43:37.3730814Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-20T21:43:37.3837460Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-20T21:43:37.3896044Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-20T21:43:37.3897212Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-20T21:43:37.3897911Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-20T21:43:37.3898756Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-20T21:43:37.3900356Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T21:43:37.3901685Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-20T21:43:37.3902802Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T21:43:37.3904313Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T21:43:37.3905701Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T21:43:37.3907239Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T21:43:37.3908804Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T21:43:37.3910465Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T21:43:37.3911908Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T21:43:37.3913996Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T21:43:37.3916250Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T21:43:37.3918453Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T21:43:37.3919629Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T21:43:37.3920848Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T21:43:37.3922058Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T21:43:37.3923068Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-20T21:43:37.3923998Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-20T21:43:37.3924983Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-20T21:43:37.3926085Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-20T21:43:37.3927316Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-20T21:43:37.3928509Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-20T21:43:37.3929925Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-20T21:43:37.3931100Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-20T21:43:37.3932244Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-20T21:43:37.3933411Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-20T21:43:37.3934567Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-20T21:43:37.3935712Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-20T21:43:37.3937369Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-20T21:43:37.3996273Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-20T21:43:37.3998788Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:43:37.4000504Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-20T21:43:37.4002121Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-20T21:43:37.4003052Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-20T21:43:37.4003970Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-20T21:43:37.4004839Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-20T21:43:37.4005631Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-20T21:43:37.4006453Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-20T21:43:37.4048776Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-20T21:43:37.4049973Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-20T21:43:37.4050753Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-20T21:43:37.4051665Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-20T21:43:37.4053604Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T21:43:37.4054754Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-20T21:43:37.4056203Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T21:43:37.4057958Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T21:43:37.4059019Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T21:43:37.4060810Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T21:43:37.4062771Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T21:43:37.4064399Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T21:43:37.4066268Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T21:43:37.4068731Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T21:43:37.4071171Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T21:43:37.4072435Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T21:43:37.4073701Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T21:43:37.4075588Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T21:43:37.4076876Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T21:43:37.4077965Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-20T21:43:37.4078984Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-20T21:43:37.4080035Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-20T21:43:37.4081216Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-20T21:43:37.4082550Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-20T21:43:37.4083828Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-20T21:43:37.4085119Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-20T21:43:37.4086345Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-20T21:43:37.4087592Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-20T21:43:37.4088846Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-20T21:43:37.4090170Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-20T21:43:37.4091410Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-20T21:43:37.4092736Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-20T21:43:37.4190819Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-20T21:43:37.4193124Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-20T21:43:37.4195561Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-20T21:43:37.4196960Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-20T21:43:37.4198307Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-20T21:43:37.4199565Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-20T21:43:37.4200874Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-20T21:43:37.4202203Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-20T21:43:37.4203520Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-20T21:43:37.4204822Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-20T21:43:37.4206117Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-20T21:43:37.4214223Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-20T21:43:37.4264747Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-20T21:43:37.4267322Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:43:37.4269188Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-20T21:43:37.4270939Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-20T21:43:37.4271934Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-20T21:43:37.4272942Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-20T21:43:37.4273894Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-20T21:43:37.4274760Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-20T21:43:37.4275636Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-20T21:43:37.4368345Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-20T21:43:37.4409267Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-20T21:43:37.4410359Z creating: build/lib/ 2024-08-20T21:43:37.4410960Z inflating: build/lib/libclog.a 2024-08-20T21:43:37.4419134Z inflating: build/lib/libpthreadpool.a 2024-08-20T21:43:37.4426510Z inflating: build/lib/libcpuinfo_internals.a 2024-08-20T21:43:37.4434141Z inflating: build/lib/libcpuinfo.a 2024-08-20T21:43:37.4497147Z inflating: build/lib/libgtest.a 2024-08-20T21:43:37.4504238Z inflating: build/lib/libittnotify.a 2024-08-20T21:43:37.4582661Z inflating: build/lib/libprotobuf-lite.a 2024-08-20T21:43:37.4651495Z inflating: build/lib/libbenchmark.a 2024-08-20T21:43:37.4677601Z inflating: build/lib/libtensorpipe_uv.a 2024-08-20T21:43:37.4738202Z inflating: build/lib/libasmjit.a 2024-08-20T21:43:37.4845656Z inflating: build/lib/libgloo.a 2024-08-20T21:43:37.5267246Z inflating: build/lib/libprotobuf.a 2024-08-20T21:43:37.5286450Z inflating: build/lib/libfmt.a 2024-08-20T21:43:37.5287633Z inflating: build/lib/libtorch_global_deps.so 2024-08-20T21:43:37.5378688Z inflating: build/lib/libc10.so 2024-08-20T21:43:37.5380939Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-20T21:43:37.5399235Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-20T21:43:37.5870129Z inflating: build/lib/libprotoc.a 2024-08-20T21:43:37.5886880Z inflating: build/lib/libgmock.a 2024-08-20T21:43:37.5887689Z inflating: build/lib/libgtest_main.a 2024-08-20T21:43:37.5888546Z inflating: build/lib/libbenchmark_main.a 2024-08-20T21:43:37.5905602Z inflating: build/lib/libnnpack.a 2024-08-20T21:43:37.6426135Z inflating: build/lib/libtensorpipe.a 2024-08-20T21:43:37.7403936Z inflating: build/lib/libfbgemm.a 2024-08-20T21:43:37.7404730Z inflating: build/lib/libgmock_main.a 2024-08-20T21:43:38.6467440Z inflating: build/lib/libdnnl.a 2024-08-20T21:43:38.6738222Z inflating: build/lib/libkineto.a 2024-08-20T21:43:38.6776582Z inflating: build/lib/libonnx_proto.a 2024-08-20T21:43:38.6949886Z inflating: build/lib/libXNNPACK.a 2024-08-20T21:43:38.7600691Z inflating: build/lib/libonnx.a 2024-08-20T21:43:40.9820741Z inflating: build/lib/libtorch_cpu.so 2024-08-20T21:43:40.9821451Z inflating: build/lib/libtorch.so 2024-08-20T21:43:40.9825996Z inflating: build/lib/libunbox_lib.a 2024-08-20T21:43:40.9829551Z inflating: build/lib/libshm.so 2024-08-20T21:43:40.9897087Z inflating: build/lib/libtorchbind_test.so 2024-08-20T21:43:40.9915816Z inflating: build/lib/libjitbackend_test.so 2024-08-20T21:43:40.9939671Z inflating: build/lib/libaoti_custom_ops.so 2024-08-20T21:43:40.9963737Z inflating: build/lib/libbackend_with_compiler.so 2024-08-20T21:43:41.1765249Z inflating: build/lib/libtorch_python.so 2024-08-20T21:43:41.1798351Z inflating: build/lib/libnnapi_backend.so 2024-08-20T21:43:41.1798799Z creating: build/bin/ 2024-08-20T21:43:41.1799153Z creating: build/bin/CMakeFiles/ 2024-08-20T21:43:41.1799735Z inflating: build/bin/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:43:41.1800411Z extracting: build/bin/CMakeFiles/progress.marks 2024-08-20T21:43:41.1801143Z inflating: build/bin/Makefile 2024-08-20T21:43:41.1801595Z inflating: build/bin/cmake_install.cmake 2024-08-20T21:43:41.1802064Z inflating: build/bin/CTestTestfile.cmake 2024-08-20T21:43:41.1851690Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-20T21:43:41.1903389Z inflating: build/bin/c10_DeviceGuard_test 2024-08-20T21:43:41.1955044Z inflating: build/bin/c10_Device_test 2024-08-20T21:43:41.2014403Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-20T21:43:41.2068154Z inflating: build/bin/c10_Scalar_test 2024-08-20T21:43:41.2117314Z inflating: build/bin/c10_StreamGuard_test 2024-08-20T21:43:41.2168691Z inflating: build/bin/c10_SymInt_test 2024-08-20T21:43:41.2223791Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-20T21:43:41.2278844Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-20T21:43:41.2334594Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-20T21:43:41.2404725Z inflating: build/bin/c10_cow_test 2024-08-20T21:43:41.2457808Z inflating: build/bin/c10_Bitset_test 2024-08-20T21:43:41.2506700Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-20T21:43:41.2556477Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-20T21:43:41.2607375Z inflating: build/bin/c10_Half_test 2024-08-20T21:43:41.2663048Z inflating: build/bin/c10_LeftRight_test 2024-08-20T21:43:41.2717983Z inflating: build/bin/c10_Metaprogramming_test 2024-08-20T21:43:41.2768087Z inflating: build/bin/c10_Synchronized_test 2024-08-20T21:43:41.2823243Z inflating: build/bin/c10_ThreadLocal_test 2024-08-20T21:43:41.2875158Z inflating: build/bin/c10_TypeIndex_test 2024-08-20T21:43:41.2926502Z inflating: build/bin/c10_TypeList_test 2024-08-20T21:43:41.2975490Z inflating: build/bin/c10_TypeTraits_test 2024-08-20T21:43:41.3027379Z inflating: build/bin/c10_accumulate_test 2024-08-20T21:43:41.3082305Z inflating: build/bin/c10_bfloat16_test 2024-08-20T21:43:41.3132763Z inflating: build/bin/c10_bit_cast_test 2024-08-20T21:43:41.3188972Z inflating: build/bin/c10_complex_math_test 2024-08-20T21:43:41.3244283Z inflating: build/bin/c10_complex_test 2024-08-20T21:43:41.3296936Z inflating: build/bin/c10_exception_test 2024-08-20T21:43:41.3347035Z inflating: build/bin/c10_flags_test 2024-08-20T21:43:41.3397414Z inflating: build/bin/c10_generic_math_test 2024-08-20T21:43:41.3448212Z inflating: build/bin/c10_irange_test 2024-08-20T21:43:41.3606766Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-20T21:43:41.3660577Z inflating: build/bin/c10_lazy_test 2024-08-20T21:43:41.3718127Z inflating: build/bin/c10_logging_test 2024-08-20T21:43:41.3792338Z inflating: build/bin/c10_optional_test 2024-08-20T21:43:41.3854263Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-20T21:43:41.3907678Z inflating: build/bin/c10_registry_test 2024-08-20T21:43:41.4055048Z inflating: build/bin/c10_small_vector_test 2024-08-20T21:43:41.4106490Z inflating: build/bin/c10_ssize_test 2024-08-20T21:43:41.4158557Z inflating: build/bin/c10_string_util_test 2024-08-20T21:43:41.4217416Z inflating: build/bin/c10_string_view_test 2024-08-20T21:43:41.4267935Z inflating: build/bin/c10_tempfile_test 2024-08-20T21:43:41.4324055Z inflating: build/bin/c10_typeid_test 2024-08-20T21:43:41.4370929Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-20T21:43:41.4809574Z inflating: build/bin/protoc-3.13.0.0 2024-08-20T21:43:41.5247873Z inflating: build/bin/protoc 2024-08-20T21:43:41.5568851Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-20T21:43:41.5907469Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-20T21:43:41.6257272Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-20T21:43:41.6309982Z inflating: build/bin/BackoffTest 2024-08-20T21:43:41.6365541Z inflating: build/bin/TCPStoreTest 2024-08-20T21:43:41.6418798Z inflating: build/bin/FileStoreTest 2024-08-20T21:43:41.6471982Z inflating: build/bin/HashStoreTest 2024-08-20T21:43:41.6474892Z inflating: build/bin/example_allreduce 2024-08-20T21:43:41.6539746Z inflating: build/bin/ProcessGroupGlooTest 2024-08-20T21:43:41.6593585Z inflating: build/bin/test_dist_autograd 2024-08-20T21:43:41.6596149Z inflating: build/bin/parallel_benchmark 2024-08-20T21:43:41.6663314Z inflating: build/bin/test_cpp_rpc 2024-08-20T21:43:41.6756032Z inflating: build/bin/test_mobile_nnc 2024-08-20T21:43:41.6756762Z inflating: build/bin/aot_model_compiler_test 2024-08-20T21:43:41.6796357Z inflating: build/bin/static_runtime_bench 2024-08-20T21:43:41.7124059Z inflating: build/bin/test_lazy 2024-08-20T21:43:41.7196987Z inflating: build/bin/Dict_test 2024-08-20T21:43:41.7249454Z inflating: build/bin/Dimname_test 2024-08-20T21:43:41.7313858Z inflating: build/bin/MaybeOwned_test 2024-08-20T21:43:41.7370555Z inflating: build/bin/NamedTensor_test 2024-08-20T21:43:41.7612730Z inflating: build/bin/static_runtime_test 2024-08-20T21:43:41.7671221Z inflating: build/bin/apply_utils_test 2024-08-20T21:43:41.7729205Z inflating: build/bin/atest 2024-08-20T21:43:41.7792017Z inflating: build/bin/basic 2024-08-20T21:43:41.7846819Z inflating: build/bin/broadcast_test 2024-08-20T21:43:41.7897931Z inflating: build/bin/cpu_allocator_test 2024-08-20T21:43:41.7956086Z inflating: build/bin/cpu_generator_test 2024-08-20T21:43:41.8008830Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-20T21:43:41.8099734Z inflating: build/bin/cpu_rng_test 2024-08-20T21:43:41.8149765Z inflating: build/bin/dispatch_key_set_test 2024-08-20T21:43:41.8200176Z inflating: build/bin/dlconvertor_test 2024-08-20T21:43:41.8257786Z inflating: build/bin/extension_backend_test 2024-08-20T21:43:41.8312777Z inflating: build/bin/half_test 2024-08-20T21:43:41.8405610Z inflating: build/bin/ivalue_test 2024-08-20T21:43:41.8455272Z inflating: build/bin/lazy_tensor_test 2024-08-20T21:43:41.8508510Z inflating: build/bin/math_kernel_test 2024-08-20T21:43:41.8561706Z inflating: build/bin/memory_format_test 2024-08-20T21:43:41.8615120Z inflating: build/bin/memory_overlapping_test 2024-08-20T21:43:41.8667766Z inflating: build/bin/mobile_memory_cleanup 2024-08-20T21:43:41.9754563Z inflating: build/bin/test_api 2024-08-20T21:43:41.9810175Z inflating: build/bin/native_test 2024-08-20T21:43:41.9861004Z inflating: build/bin/operator_name_test 2024-08-20T21:43:41.9911788Z inflating: build/bin/operators_test 2024-08-20T21:43:41.9964159Z inflating: build/bin/packedtensoraccessor_test 2024-08-20T21:43:42.0030197Z inflating: build/bin/pow_test 2024-08-20T21:43:42.0087194Z inflating: build/bin/quantized_test 2024-08-20T21:43:42.0136991Z inflating: build/bin/reduce_ops_test 2024-08-20T21:43:42.0188611Z inflating: build/bin/reportMemoryUsage_test 2024-08-20T21:43:42.0244903Z inflating: build/bin/scalar_tensor_test 2024-08-20T21:43:42.0296295Z inflating: build/bin/StorageUtils_test 2024-08-20T21:43:42.0354124Z inflating: build/bin/scalar_test 2024-08-20T21:43:42.0407021Z inflating: build/bin/stride_properties_test 2024-08-20T21:43:42.0461750Z inflating: build/bin/type_ptr_test 2024-08-20T21:43:42.0464418Z inflating: build/bin/thread_init_test 2024-08-20T21:43:42.0541452Z inflating: build/bin/tensor_iterator_test 2024-08-20T21:43:42.0595033Z inflating: build/bin/test_parallel 2024-08-20T21:43:42.0647079Z inflating: build/bin/undefined_tensor_test 2024-08-20T21:43:42.0648528Z inflating: build/bin/verify_api_visibility 2024-08-20T21:43:42.0707786Z inflating: build/bin/type_test 2024-08-20T21:43:42.0776527Z inflating: build/bin/legacy_vmap_test 2024-08-20T21:43:42.0828020Z inflating: build/bin/weakref_test 2024-08-20T21:43:42.0931736Z inflating: build/bin/List_test 2024-08-20T21:43:42.0983310Z inflating: build/bin/xla_tensor_test 2024-08-20T21:43:42.1042571Z inflating: build/bin/IListRef_test 2024-08-20T21:43:42.1094228Z inflating: build/bin/wrapdim_test 2024-08-20T21:43:42.1159407Z inflating: build/bin/KernelFunction_test 2024-08-20T21:43:42.1277173Z inflating: build/bin/kernel_function_legacy_test 2024-08-20T21:43:42.1371159Z inflating: build/bin/kernel_function_test 2024-08-20T21:43:42.1494166Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-20T21:43:42.1592805Z inflating: build/bin/kernel_lambda_test 2024-08-20T21:43:42.1652830Z inflating: build/bin/kernel_stackbased_test 2024-08-20T21:43:42.1703941Z inflating: build/bin/CppSignature_test 2024-08-20T21:43:42.1797062Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-20T21:43:42.1845690Z inflating: build/bin/op_allowlist_test 2024-08-20T21:43:42.1908945Z inflating: build/bin/inline_container_test 2024-08-20T21:43:42.1963819Z inflating: build/bin/backend_fallback_test 2024-08-20T21:43:42.2263232Z inflating: build/bin/op_registration_test 2024-08-20T21:43:42.2317109Z inflating: build/bin/test_edge_op_registration 2024-08-20T21:43:42.2321056Z inflating: build/bin/torch_shm_manager 2024-08-20T21:43:42.2333551Z inflating: build/bin/tutorial_tensorexpr 2024-08-20T21:43:42.3093371Z inflating: build/bin/test_tensorexpr 2024-08-20T21:43:42.3650448Z inflating: build/bin/test_jit 2024-08-20T21:43:42.3650868Z creating: .additional_ci_files/ 2024-08-20T21:43:42.3707759Z inflating: .additional_ci_files/test-times.json 2024-08-20T21:43:42.3935916Z inflating: .additional_ci_files/test-class-times.json 2024-08-20T21:43:42.3966154Z ##[group]Run rm artifacts.zip 2024-08-20T21:43:42.3966541Z rm artifacts.zip 2024-08-20T21:43:42.3972728Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:42.3973223Z env: 2024-08-20T21:43:42.3973489Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:42.3973811Z ##[endgroup] 2024-08-20T21:43:42.4241847Z ##[group]Run df -H 2024-08-20T21:43:42.4242169Z df -H 2024-08-20T21:43:42.4248090Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:42.4248570Z env: 2024-08-20T21:43:42.4248835Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:42.4249168Z ##[endgroup] 2024-08-20T21:43:42.4292550Z Filesystem Size Used Avail Use% Mounted on 2024-08-20T21:43:42.4293148Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-20T21:43:42.4294112Z tmpfs 8.2G 21k 8.2G 1% /dev/shm 2024-08-20T21:43:42.4294856Z tmpfs 3.3G 488k 3.3G 1% /run 2024-08-20T21:43:42.4295547Z /dev/nvme0n1p1 161G 32G 130G 20% / 2024-08-20T21:43:42.4296229Z tmpfs 8.2G 8.2k 8.2G 1% /tmp 2024-08-20T21:43:42.4296709Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-20T21:43:42.4341791Z Prepare all required actions 2024-08-20T21:43:42.4342234Z Getting action download info 2024-08-20T21:43:42.5540639Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-20T21:43:42.5541090Z with: 2024-08-20T21:43:42.5541339Z env: 2024-08-20T21:43:42.5541597Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:42.5541906Z ##[endgroup] 2024-08-20T21:43:42.5577419Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T21:43:42.5577856Z with: 2024-08-20T21:43:42.5578104Z name: td_results 2024-08-20T21:43:42.5578392Z s3-bucket: gha-artifacts 2024-08-20T21:43:42.5578724Z region: us-east-1 2024-08-20T21:43:42.5579003Z env: 2024-08-20T21:43:42.5579241Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:42.5579560Z ##[endgroup] 2024-08-20T21:43:43.0354089Z (node:46644) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T21:43:43.0354805Z 2024-08-20T21:43:43.0355182Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T21:43:43.0356187Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T21:43:43.0357490Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T21:43:43.1036041Z Found 1 objects with prefix pytorch/pytorch/10479309237/td_results/ 2024-08-20T21:43:43.1037234Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-20T21:43:43.1531746Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-20T21:43:43.1537868Z Artifact download has finished successfully 2024-08-20T21:43:43.1683267Z ##[group]Run mkdir -p .additional_ci_files 2024-08-20T21:43:43.1683744Z mkdir -p .additional_ci_files 2024-08-20T21:43:43.1684303Z mv td_results.json .additional_ci_files/td_results.json 2024-08-20T21:43:43.1690495Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:43.1690972Z env: 2024-08-20T21:43:43.1691249Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:43.1691578Z ##[endgroup] 2024-08-20T21:43:43.1789895Z ##[group]Run .github/scripts/parse_ref.py 2024-08-20T21:43:43.1790371Z .github/scripts/parse_ref.py 2024-08-20T21:43:43.1796716Z shell: /usr/bin/bash -e {0} 2024-08-20T21:43:43.1797185Z env: 2024-08-20T21:43:43.1797441Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:43.1797757Z ##[endgroup] 2024-08-20T21:43:43.2083036Z Prepare all required actions 2024-08-20T21:43:43.2125276Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-20T21:43:43.2125725Z with: 2024-08-20T21:43:43.2126295Z github-token: *** 2024-08-20T21:43:43.2126568Z env: 2024-08-20T21:43:43.2126825Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:43.2127153Z ##[endgroup] 2024-08-20T21:43:43.2145807Z ##[group]Run set -eux 2024-08-20T21:43:43.2146124Z set -eux 2024-08-20T21:43:43.2146703Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-20T21:43:43.2152681Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:43.2153153Z env: 2024-08-20T21:43:43.2153413Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:43.2153916Z GITHUB_TOKEN: *** 2024-08-20T21:43:43.2154198Z ##[endgroup] 2024-08-20T21:43:43.2178491Z + python3 .github/scripts/get_workflow_job_id.py 10479309237 i-0e2ef67d50e8ceb09 2024-08-20T21:43:45.0546969Z setting job-id=29025446952 2024-08-20T21:43:45.0547870Z setting job-name=linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-20T21:43:45.0785522Z Prepare all required actions 2024-08-20T21:43:45.0785953Z Getting action download info 2024-08-20T21:43:45.2028285Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-20T21:43:45.2028727Z with: 2024-08-20T21:43:45.2029176Z github-token: *** 2024-08-20T21:43:45.2032744Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]} 2024-08-20T21:43:45.2036728Z job-name: linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-20T21:43:45.2037364Z env: 2024-08-20T21:43:45.2037610Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:45.2037940Z ##[endgroup] 2024-08-20T21:43:45.2082185Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T21:43:45.2082742Z with: 2024-08-20T21:43:45.2082977Z shell: bash 2024-08-20T21:43:45.2083254Z timeout_minutes: 10 2024-08-20T21:43:45.2083546Z max_attempts: 5 2024-08-20T21:43:45.2083839Z retry_wait_seconds: 30 2024-08-20T21:43:45.2084973Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T21:43:45.2086191Z polling_interval_seconds: 1 2024-08-20T21:43:45.2086540Z warning_on_retry: true 2024-08-20T21:43:45.2086871Z continue_on_error: false 2024-08-20T21:43:45.2087188Z env: 2024-08-20T21:43:45.2087420Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:45.2087922Z GITHUB_TOKEN: *** 2024-08-20T21:43:45.2088211Z ##[endgroup] 2024-08-20T21:43:45.2851475Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T21:43:45.5286534Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T21:43:45.5448247Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.9/site-packages (2.27.1) 2024-08-20T21:43:45.5452283Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.9/site-packages (6.0.1) 2024-08-20T21:43:45.5567242Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-08-20T21:43:45.5577864Z Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2.0.12) 2024-08-20T21:43:45.5586458Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-08-20T21:43:45.5592466Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2024.7.4) 2024-08-20T21:43:46.2653414Z Command completed after 1 attempt(s). 2024-08-20T21:43:46.2700420Z ##[group]Run set -x 2024-08-20T21:43:46.2700718Z set -x 2024-08-20T21:43:46.2700997Z  2024-08-20T21:43:46.2701525Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T21:43:46.2702185Z # in runner workspace 2024-08-20T21:43:46.2702702Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-20T21:43:46.2708627Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:46.2709108Z env: 2024-08-20T21:43:46.2709370Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:46.2709884Z ##[endgroup] 2024-08-20T21:43:46.2734819Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-20T21:43:46.2971108Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T21:43:46.2971968Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T21:43:46.2972424Z echo "Job name: ${JOB_NAME}" 2024-08-20T21:43:46.2972793Z  2024-08-20T21:43:46.2973324Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T21:43:46.2973998Z # in runner workspace 2024-08-20T21:43:46.2974550Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-20T21:43:46.2975192Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-20T21:43:46.2975649Z  --job-name "${JOB_NAME}" \ 2024-08-20T21:43:46.2979354Z  --test-matrix "{"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]}" \ 2024-08-20T21:43:46.2983048Z  --selected-test-configs "" \ 2024-08-20T21:43:46.2983500Z  --pr-number "${PR_NUMBER}" \ 2024-08-20T21:43:46.2983888Z  --tag "${TAG}" \ 2024-08-20T21:43:46.2984264Z  --event-name "${EVENT_NAME}" \ 2024-08-20T21:43:46.2984696Z  --schedule "${SCHEDULE}" \ 2024-08-20T21:43:46.2985094Z  --branch "${HEAD_BRANCH}" 2024-08-20T21:43:46.2990775Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:46.2991265Z env: 2024-08-20T21:43:46.2991519Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:46.2992047Z GITHUB_TOKEN: *** 2024-08-20T21:43:46.2992604Z JOB_NAME: linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-20T21:43:46.2993238Z PR_NUMBER: 133712 2024-08-20T21:43:46.2993528Z TAG: 2024-08-20T21:43:46.2994297Z EVENT_NAME: pull_request 2024-08-20T21:43:46.2994623Z SCHEDULE: 2024-08-20T21:43:46.2994891Z HEAD_BRANCH: 2024-08-20T21:43:46.2995168Z ##[endgroup] 2024-08-20T21:43:46.3018117Z Workflow: pull 2024-08-20T21:43:46.3019043Z Job name: linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-20T21:43:46.5661298Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-20T21:43:46.7332906Z ##[group]Run echo "Filtered matrix:" 2024-08-20T21:43:46.7333367Z echo "Filtered matrix:" 2024-08-20T21:43:46.7336987Z echo "{"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]}" 2024-08-20T21:43:46.7340584Z  2024-08-20T21:43:46.7340974Z echo 2024-08-20T21:43:46.7341334Z echo "Is the current job unstable? False" 2024-08-20T21:43:46.7341785Z  2024-08-20T21:43:46.7342020Z echo 2024-08-20T21:43:46.7342359Z echo "Is keep-going label set? False" 2024-08-20T21:43:46.7342784Z  2024-08-20T21:43:46.7343017Z echo 2024-08-20T21:43:46.7343311Z echo "Renabled issues? " 2024-08-20T21:43:46.7349042Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:46.7349534Z env: 2024-08-20T21:43:46.7349793Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:46.7350109Z ##[endgroup] 2024-08-20T21:43:46.7373625Z Filtered matrix: 2024-08-20T21:43:46.7378463Z {include: [{config: default, shard: 1, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: default, shard: 2, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: default, shard: 3, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: default, shard: 4, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: docs_test, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: jit_legacy, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: backwards_compat, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: distributed, shard: 1, num_shards: 2, runner: amz2023.linux.2xlarge}, {config: distributed, shard: 2, num_shards: 2, runner: amz2023.linux.2xlarge}]} 2024-08-20T21:43:46.7381845Z 2024-08-20T21:43:46.7382003Z Is the current job unstable? False 2024-08-20T21:43:46.7382270Z 2024-08-20T21:43:46.7382607Z Is keep-going label set? False 2024-08-20T21:43:46.7382866Z 2024-08-20T21:43:46.7382974Z Renabled issues? 2024-08-20T21:43:46.7435649Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T21:43:46.7436368Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T21:43:46.7441987Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:46.7442460Z env: 2024-08-20T21:43:46.7442722Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:46.7443060Z JOB_TIMEOUT: 240 2024-08-20T21:43:46.7443332Z ##[endgroup] 2024-08-20T21:43:46.7524300Z ##[group]Run set -x 2024-08-20T21:43:46.7524693Z set -x 2024-08-20T21:43:46.7524967Z  2024-08-20T21:43:46.7525339Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-20T21:43:46.7525993Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-20T21:43:46.7526543Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-20T21:43:46.7527039Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-20T21:43:46.7527425Z else 2024-08-20T21:43:46.7527741Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T21:43:46.7528143Z fi 2024-08-20T21:43:46.7528386Z  2024-08-20T21:43:46.7528975Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-20T21:43:46.7529837Z # TODO: Stop building test binaries as part of the build phase 2024-08-20T21:43:46.7530497Z # Used for GPU_FLAG since that doesn't play nice 2024-08-20T21:43:46.7531077Z # shellcheck disable=SC2086,SC2090 2024-08-20T21:43:46.7531511Z container_name=$(docker run \ 2024-08-20T21:43:46.7531910Z  ${GPU_FLAG:-} \ 2024-08-20T21:43:46.7532259Z  -e BUILD_ENVIRONMENT \ 2024-08-20T21:43:46.7532627Z  -e PR_NUMBER \ 2024-08-20T21:43:46.7532974Z  -e GITHUB_ACTIONS \ 2024-08-20T21:43:46.7533345Z  -e GITHUB_REPOSITORY \ 2024-08-20T21:43:46.7533720Z  -e GITHUB_WORKFLOW \ 2024-08-20T21:43:46.7534085Z  -e GITHUB_JOB \ 2024-08-20T21:43:46.7534427Z  -e GITHUB_RUN_ID \ 2024-08-20T21:43:46.7534793Z  -e GITHUB_RUN_NUMBER \ 2024-08-20T21:43:46.7535167Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-20T21:43:46.7535553Z  -e JOB_ID \ 2024-08-20T21:43:46.7535874Z  -e JOB_NAME \ 2024-08-20T21:43:46.7536185Z  -e BASE_SHA \ 2024-08-20T21:43:46.7536511Z  -e BRANCH \ 2024-08-20T21:43:46.7536823Z  -e SHA1 \ 2024-08-20T21:43:46.7537132Z  -e AWS_DEFAULT_REGION \ 2024-08-20T21:43:46.7537517Z  -e IN_WHEEL_TEST \ 2024-08-20T21:43:46.7537876Z  -e SHARD_NUMBER \ 2024-08-20T21:43:46.7538213Z  -e TEST_CONFIG \ 2024-08-20T21:43:46.7538563Z  -e NUM_TEST_SHARDS \ 2024-08-20T21:43:46.7538937Z  -e REENABLED_ISSUES \ 2024-08-20T21:43:46.7539319Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-20T21:43:46.7539727Z  -e VERBOSE_TEST_LOGS \ 2024-08-20T21:43:46.7553893Z  -e TEST_SHOWLOCALS \ 2024-08-20T21:43:46.7554364Z  -e NO_TEST_TIMEOUT \ 2024-08-20T21:43:46.7554732Z  -e NO_TD \ 2024-08-20T21:43:46.7555036Z  -e TD_DISTRIBUTED \ 2024-08-20T21:43:46.7555411Z  -e PR_LABELS \ 2024-08-20T21:43:46.7555799Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-20T21:43:46.7556233Z  -e SCCACHE_BUCKET \ 2024-08-20T21:43:46.7556605Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-20T21:43:46.7556996Z  -e XLA_CUDA \ 2024-08-20T21:43:46.7557369Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-20T21:43:46.7557861Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-20T21:43:46.7558369Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-20T21:43:46.7558865Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-20T21:43:46.7559306Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-20T21:43:46.7559737Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2024-08-20T21:43:46.7560160Z  -e DASHBOARD_TAG \ 2024-08-20T21:43:46.7560606Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-20T21:43:46.7561146Z  --security-opt seccomp=unconfined \ 2024-08-20T21:43:46.7561598Z  --cap-add=SYS_PTRACE \ 2024-08-20T21:43:46.7561952Z  --ipc=host \ 2024-08-20T21:43:46.7562286Z  --shm-size="${SHM_SIZE}" \ 2024-08-20T21:43:46.7562661Z  --tty \ 2024-08-20T21:43:46.7562938Z  --detach \ 2024-08-20T21:43:46.7563433Z  --name="${container_name}" \ 2024-08-20T21:43:46.7563839Z  --user jenkins \ 2024-08-20T21:43:46.7564292Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-20T21:43:46.7564841Z  -w /var/lib/jenkins/workspace \ 2024-08-20T21:43:46.7565264Z  "${DOCKER_IMAGE}" 2024-08-20T21:43:46.7565581Z ) 2024-08-20T21:43:46.7565969Z # Propagate download.pytorch.org IP to container 2024-08-20T21:43:46.7566900Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-20T21:43:46.7567874Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-20T21:43:46.7568856Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-20T21:43:46.7574842Z shell: /usr/bin/bash -e {0} 2024-08-20T21:43:46.7575183Z env: 2024-08-20T21:43:46.7575424Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:46.7575823Z BUILD_ENVIRONMENT: linux-jammy-py3.8-gcc11 2024-08-20T21:43:46.7576258Z PR_NUMBER: 133712 2024-08-20T21:43:46.7576566Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-20T21:43:46.7576964Z GITHUB_WORKFLOW: pull 2024-08-20T21:43:46.7577280Z GITHUB_JOB: test 2024-08-20T21:43:46.7577560Z GITHUB_RUN_ID: 10479309237 2024-08-20T21:43:46.7577899Z GITHUB_RUN_NUMBER: 238988 2024-08-20T21:43:46.7578236Z GITHUB_RUN_ATTEMPT: 1 2024-08-20T21:43:46.7578533Z JOB_ID: 29025446952 2024-08-20T21:43:46.7579080Z JOB_NAME: linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-20T21:43:46.7579726Z BRANCH: pull/133712 2024-08-20T21:43:46.7580071Z SHA1: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:43:46.7580590Z BASE_SHA: 91f3d614142df02f619e44a68e3d9e0dfeba49ec 2024-08-20T21:43:46.7581043Z TEST_CONFIG: distributed 2024-08-20T21:43:46.7581356Z SHARD_NUMBER: 1 2024-08-20T21:43:46.7581637Z NUM_TEST_SHARDS: 2 2024-08-20T21:43:46.7581940Z REENABLED_ISSUES: 2024-08-20T21:43:46.7582251Z CONTINUE_THROUGH_ERROR: False 2024-08-20T21:43:46.7582624Z VERBOSE_TEST_LOGS: False 2024-08-20T21:43:46.7582961Z TEST_SHOWLOCALS: False 2024-08-20T21:43:46.7583273Z NO_TEST_TIMEOUT: False 2024-08-20T21:43:46.7583587Z NO_TD: False 2024-08-20T21:43:46.7583871Z TD_DISTRIBUTED: False 2024-08-20T21:43:46.7584262Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-20T21:43:46.7584748Z SCCACHE_S3_KEY_PREFIX: pull 2024-08-20T21:43:46.7585096Z SHM_SIZE: 1g 2024-08-20T21:43:46.7585961Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:43:46.7586937Z XLA_CUDA: 2024-08-20T21:43:46.7587412Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-20T21:43:46.7588025Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-20T21:43:46.7588436Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-20T21:43:46.7588836Z DASHBOARD_TAG: 2024-08-20T21:43:46.7589143Z HUGGING_FACE_HUB_TOKEN: 2024-08-20T21:43:46.7589484Z SCRIBE_GRAPHQL_ACCESS_TOKEN: 2024-08-20T21:43:46.7589846Z ##[endgroup] 2024-08-20T21:43:46.7614456Z + [[ distributed == \m\u\l\t\i\g\p\u ]] 2024-08-20T21:43:46.7615144Z + [[ linux-jammy-py3.8-gcc11 == *onnx* ]] 2024-08-20T21:43:46.7615584Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T21:43:46.7623082Z +++ nproc --ignore=2 2024-08-20T21:43:46.7656431Z ++ docker run -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=6 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10479309237 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=1g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:43:56.6330396Z + container_name=a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T21:43:56.6332447Z + grep download.pytorch.org /etc/hosts 2024-08-20T21:43:56.6334330Z + docker exec -i a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-20T21:43:56.7774588Z + echo DOCKER_CONTAINER_ID=a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T21:43:56.7778983Z ++ echo dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl 2024-08-20T21:43:56.7781372Z + docker exec -t a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 sh -c 'pip install dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-20T21:43:57.6580353Z Processing ./dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl (from torch==2.5.0a0+git40ec5f6) 2024-08-20T21:43:58.1038414Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.13.1) 2024-08-20T21:43:58.1041870Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (4.12.2) 2024-08-20T21:43:58.1045893Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.8.8) 2024-08-20T21:43:58.1048619Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.1.4) 2024-08-20T21:43:58.1052470Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2024.6.1) 2024-08-20T21:43:58.1058976Z Requirement already satisfied: sympy==1.12.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.12.1) 2024-08-20T21:43:58.1078841Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy==1.12.1->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.3.0) 2024-08-20T21:43:58.1093222Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.3.0) 2024-08-20T21:43:58.1116582Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.21.2) 2024-08-20T21:43:58.1514029Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.1.5) 2024-08-20T21:43:58.6786573Z Installing collected packages: torch 2024-08-20T21:44:06.6019729Z Successfully installed torch-2.5.0a0+git40ec5f6 2024-08-20T21:44:06.6827348Z + export TERM=vt100 2024-08-20T21:44:06.6827815Z + TERM=vt100 2024-08-20T21:44:06.6828532Z ++ dirname .ci/pytorch/test.sh 2024-08-20T21:44:06.6843616Z + source .ci/pytorch/common.sh 2024-08-20T21:44:06.6849672Z +++ dirname .ci/pytorch/common.sh 2024-08-20T21:44:06.6856166Z ++ source .ci/pytorch/common_utils.sh 2024-08-20T21:44:06.6860831Z +++ declare -f -t trap_add 2024-08-20T21:44:06.6865014Z ++ set -ex 2024-08-20T21:44:06.6865634Z ++ [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-20T21:44:06.6866854Z ++ BUILD_TEST_LIBTORCH=0 2024-08-20T21:44:06.6867594Z + [[ linux-jammy-py3.8-gcc11 != *rocm* ]] 2024-08-20T21:44:06.6868738Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-20T21:44:06.6893974Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-20T21:44:06.6894424Z + trap_add cleanup_workspace EXIT 2024-08-20T21:44:06.6894828Z + trap_add_cmd=cleanup_workspace 2024-08-20T21:44:06.6895173Z + shift 2024-08-20T21:44:06.6895432Z + for trap_add_name in "$@" 2024-08-20T21:44:06.6899782Z +++ trap -p EXIT 2024-08-20T21:44:06.6902845Z ++ eval 'extract_trap_cmd ' 2024-08-20T21:44:06.6903393Z +++ extract_trap_cmd 2024-08-20T21:44:06.6903760Z +++ printf '%s\n' '' 2024-08-20T21:44:06.6905090Z ++ printf '%s\n' cleanup_workspace 2024-08-20T21:44:06.6905724Z + trap -- ' 2024-08-20T21:44:06.6906105Z cleanup_workspace' EXIT 2024-08-20T21:44:06.6906554Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-20T21:44:07.1405475Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-20T21:44:07.1604557Z + echo 'Environment variables:' 2024-08-20T21:44:07.1605015Z Environment variables: 2024-08-20T21:44:07.1605325Z + env 2024-08-20T21:44:07.1622573Z INSTALLED_DB=yes 2024-08-20T21:44:07.1623630Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:44:07.1624575Z CONTINUE_THROUGH_ERROR=False 2024-08-20T21:44:07.1625174Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11 2024-08-20T21:44:07.1625851Z HOSTNAME=a5702959bcb2 2024-08-20T21:44:07.1627299Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_971f906e-00ca-41f0-8553-385d2c8ff1b7 2024-08-20T21:44:07.1628671Z GITHUB_ACTION=__self 2024-08-20T21:44:07.1629208Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T21:44:07.1629881Z GITHUB_RUN_NUMBER=238988 2024-08-20T21:44:07.1630497Z TEST_CONFIG=distributed 2024-08-20T21:44:07.1631143Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T21:44:07.1631843Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T21:44:07.1632520Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T21:44:07.1633233Z GITHUB_TRIGGERING_ACTOR=pytorchmergebot 2024-08-20T21:44:07.1633981Z GITHUB_REF_TYPE=branch 2024-08-20T21:44:07.1634561Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T21:44:07.1635045Z BASE_SHA=91f3d614142df02f619e44a68e3d9e0dfeba49ec 2024-08-20T21:44:07.1635468Z XLA_CUDA= 2024-08-20T21:44:07.1635738Z HUGGING_FACE_HUB_TOKEN= 2024-08-20T21:44:07.1638973Z *** 2024-08-20T21:44:07.1639445Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T21:44:07.1639836Z GITHUB_ACTIONS=true 2024-08-20T21:44:07.1640221Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:44:07.1640713Z GITHUB_SHA=f2fb9405c2fa9f9502a76363091cce6fd8179736 2024-08-20T21:44:07.1641457Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/133712/merge 2024-08-20T21:44:07.1642113Z UCC_HOME=/usr 2024-08-20T21:44:07.1642433Z VERBOSE_TEST_LOGS=False 2024-08-20T21:44:07.1642754Z GITHUB_REF=refs/pull/133712/merge 2024-08-20T21:44:07.1643118Z SHARD_NUMBER=1 2024-08-20T21:44:07.1643417Z GITHUB_REF_PROTECTED=false 2024-08-20T21:44:07.1643751Z HOME=/var/lib/jenkins 2024-08-20T21:44:07.1644102Z GITHUB_API_URL=https://api.github.com 2024-08-20T21:44:07.1644531Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T21:44:07.1644901Z UCX_COMMIT= 2024-08-20T21:44:07.1645182Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-20T21:44:07.1645514Z NUM_TEST_SHARDS=2 2024-08-20T21:44:07.1645822Z UCX_HOME=/usr 2024-08-20T21:44:07.1646745Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_971f906e-00ca-41f0-8553-385d2c8ff1b7 2024-08-20T21:44:07.1647922Z JOB_NAME=linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-20T21:44:07.1649081Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_971f906e-00ca-41f0-8553-385d2c8ff1b7 2024-08-20T21:44:07.1650348Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T21:44:07.1651018Z GITHUB_EVENT_NAME=pull_request 2024-08-20T21:44:07.1651364Z DASHBOARD_TAG= 2024-08-20T21:44:07.1651857Z GITHUB_RUN_ID=10479309237 2024-08-20T21:44:07.1652807Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_971f906e-00ca-41f0-8553-385d2c8ff1b7 2024-08-20T21:44:07.1653699Z GITHUB_ACTOR=pytorchmergebot 2024-08-20T21:44:07.1654030Z PR_NUMBER=133712 2024-08-20T21:44:07.1654312Z DESIRED_CUDA= 2024-08-20T21:44:07.1654586Z GITHUB_RUN_ATTEMPT=1 2024-08-20T21:44:07.1654886Z ANACONDA_PYTHON_VERSION=3.8 2024-08-20T21:44:07.1655310Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T21:44:07.1655753Z TERM=vt100 2024-08-20T21:44:07.1656000Z INSTALLED_VISION=yes 2024-08-20T21:44:07.1656302Z BRANCH=pull/133712 2024-08-20T21:44:07.1656708Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T21:44:07.1657048Z CUDA_PATH=/usr/local/cuda 2024-08-20T21:44:07.1657814Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T21:44:07.1658570Z GITHUB_SERVER_URL=https://github.com 2024-08-20T21:44:07.1658941Z UCC_COMMIT= 2024-08-20T21:44:07.1659232Z REENABLED_ISSUES= 2024-08-20T21:44:07.1659506Z DOCS=yes 2024-08-20T21:44:07.1659744Z INSTALLED_ANDROID= 2024-08-20T21:44:07.1660025Z SHLVL=1 2024-08-20T21:44:07.1660259Z MAX_JOBS=6 2024-08-20T21:44:07.1660509Z GITHUB_ACTOR_ID=97764156 2024-08-20T21:44:07.1660953Z GITHUB_WORKFLOW_SHA=f2fb9405c2fa9f9502a76363091cce6fd8179736 2024-08-20T21:44:07.1661452Z GITHUB_REF_NAME=133712/merge 2024-08-20T21:44:07.1662041Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T21:44:07.1662609Z GITHUB_JOB=test 2024-08-20T21:44:07.1662887Z NO_TEST_TIMEOUT=False 2024-08-20T21:44:07.1663180Z TD_DISTRIBUTED=False 2024-08-20T21:44:07.1663502Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T21:44:07.1663873Z GITHUB_RETENTION_DAYS=90 2024-08-20T21:44:07.1664197Z OPENSSL_DIR=/opt/openssl 2024-08-20T21:44:07.1664525Z GITHUB_ACTION_REPOSITORY= 2024-08-20T21:44:07.1665579Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:44:07.1666696Z GITHUB_BASE_REF=gh/XuehaiPan/146/base 2024-08-20T21:44:07.1667083Z INSTALLED_ACL= 2024-08-20T21:44:07.1667346Z CI=true 2024-08-20T21:44:07.1667600Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T21:44:07.1667955Z JOB_ID=29025446952 2024-08-20T21:44:07.1668242Z INSTALLED_PROTOBUF=yes 2024-08-20T21:44:07.1668567Z GITHUB_HEAD_REF=gh/XuehaiPan/146/head 2024-08-20T21:44:07.1668950Z GITHUB_ACTION_REF= 2024-08-20T21:44:07.1669373Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T21:44:07.1669815Z TEST_SHOWLOCALS=False 2024-08-20T21:44:07.1670124Z GITHUB_WORKFLOW=pull 2024-08-20T21:44:07.1670443Z DEBIAN_FRONTEND=noninteractive 2024-08-20T21:44:07.1671357Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_971f906e-00ca-41f0-8553-385d2c8ff1b7 2024-08-20T21:44:07.1672174Z NO_TD=False 2024-08-20T21:44:07.1672459Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T21:44:07.1672787Z _=/usr/bin/env 2024-08-20T21:44:07.1673261Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-20T21:44:07.1872406Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch 2024-08-20T21:44:07.1874118Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-08-20T21:44:07.1875380Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib 2024-08-20T21:44:07.1876285Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/test 2024-08-20T21:44:07.1876920Z + BUILD_DIR=build 2024-08-20T21:44:07.1877232Z + BUILD_RENAMED_DIR=build_renamed 2024-08-20T21:44:07.1877792Z + BUILD_BIN_DIR=build/bin 2024-08-20T21:44:07.1878344Z + SHARD_NUMBER=1 2024-08-20T21:44:07.1878631Z + NUM_TEST_SHARDS=2 2024-08-20T21:44:07.1878930Z + export VALGRIND=ON 2024-08-20T21:44:07.1879218Z + VALGRIND=ON 2024-08-20T21:44:07.1879602Z + [[ linux-jammy-py3.8-gcc11 == *clang9* ]] 2024-08-20T21:44:07.1880013Z + [[ 0 == \1 ]] 2024-08-20T21:44:07.1880310Z + [[ False == \1 ]] 2024-08-20T21:44:07.1880953Z + [[ linux-jammy-py3.8-gcc11 != *bazel* ]] 2024-08-20T21:44:07.1881410Z ++ realpath build/custom_test_artifacts 2024-08-20T21:44:07.1900981Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-20T21:44:07.1902008Z + [[ -n '' ]] 2024-08-20T21:44:07.1902343Z + echo 'Environment variables' 2024-08-20T21:44:07.1902706Z Environment variables 2024-08-20T21:44:07.1902990Z + env 2024-08-20T21:44:07.1909529Z INSTALLED_DB=yes 2024-08-20T21:44:07.1910606Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:44:07.1911615Z CONTINUE_THROUGH_ERROR=False 2024-08-20T21:44:07.1912157Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11 2024-08-20T21:44:07.1913221Z HOSTNAME=a5702959bcb2 2024-08-20T21:44:07.1914834Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_971f906e-00ca-41f0-8553-385d2c8ff1b7 2024-08-20T21:44:07.1916366Z GITHUB_ACTION=__self 2024-08-20T21:44:07.1916857Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T21:44:07.1917539Z GITHUB_RUN_NUMBER=238988 2024-08-20T21:44:07.1918050Z TEST_CONFIG=distributed 2024-08-20T21:44:07.1918552Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T21:44:07.1919444Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T21:44:07.1920162Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T21:44:07.1920798Z GITHUB_TRIGGERING_ACTOR=pytorchmergebot 2024-08-20T21:44:07.1921220Z GITHUB_REF_TYPE=branch 2024-08-20T21:44:07.1921533Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T21:44:07.1921946Z BASE_SHA=91f3d614142df02f619e44a68e3d9e0dfeba49ec 2024-08-20T21:44:07.1922385Z XLA_CUDA= 2024-08-20T21:44:07.1922637Z HUGGING_FACE_HUB_TOKEN= 2024-08-20T21:44:07.1923074Z *** 2024-08-20T21:44:07.1923337Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T21:44:07.1923676Z GITHUB_ACTIONS=true 2024-08-20T21:44:07.1924030Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:44:07.1924535Z GITHUB_SHA=f2fb9405c2fa9f9502a76363091cce6fd8179736 2024-08-20T21:44:07.1925240Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/133712/merge 2024-08-20T21:44:07.1925877Z UCC_HOME=/usr 2024-08-20T21:44:07.1926160Z VERBOSE_TEST_LOGS=False 2024-08-20T21:44:07.1926478Z GITHUB_REF=refs/pull/133712/merge 2024-08-20T21:44:07.1926836Z SHARD_NUMBER=1 2024-08-20T21:44:07.1927126Z GITHUB_REF_PROTECTED=false 2024-08-20T21:44:07.1927441Z HOME=/var/lib/jenkins 2024-08-20T21:44:07.1927785Z GITHUB_API_URL=https://api.github.com 2024-08-20T21:44:07.1928209Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T21:44:07.1928568Z UCX_COMMIT= 2024-08-20T21:44:07.1928838Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-20T21:44:07.1929168Z NUM_TEST_SHARDS=2 2024-08-20T21:44:07.1929436Z UCX_HOME=/usr 2024-08-20T21:44:07.1930408Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_971f906e-00ca-41f0-8553-385d2c8ff1b7 2024-08-20T21:44:07.1931575Z JOB_NAME=linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-20T21:44:07.1932738Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_971f906e-00ca-41f0-8553-385d2c8ff1b7 2024-08-20T21:44:07.1933888Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T21:44:07.1934553Z GITHUB_EVENT_NAME=pull_request 2024-08-20T21:44:07.1934884Z DASHBOARD_TAG= 2024-08-20T21:44:07.1935165Z GITHUB_RUN_ID=10479309237 2024-08-20T21:44:07.1936109Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_971f906e-00ca-41f0-8553-385d2c8ff1b7 2024-08-20T21:44:07.1937432Z GITHUB_ACTOR=pytorchmergebot 2024-08-20T21:44:07.1937786Z PR_NUMBER=133712 2024-08-20T21:44:07.1938071Z DESIRED_CUDA= 2024-08-20T21:44:07.1938336Z GITHUB_RUN_ATTEMPT=1 2024-08-20T21:44:07.1938642Z VALGRIND=ON 2024-08-20T21:44:07.1938918Z ANACONDA_PYTHON_VERSION=3.8 2024-08-20T21:44:07.1939327Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T21:44:07.1939775Z TERM=vt100 2024-08-20T21:44:07.1940039Z INSTALLED_VISION=yes 2024-08-20T21:44:07.1940327Z BRANCH=pull/133712 2024-08-20T21:44:07.1940832Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T21:44:07.1941191Z CUDA_PATH=/usr/local/cuda 2024-08-20T21:44:07.1941958Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T21:44:07.1942718Z GITHUB_SERVER_URL=https://github.com 2024-08-20T21:44:07.1943101Z UCC_COMMIT= 2024-08-20T21:44:07.1943348Z REENABLED_ISSUES= 2024-08-20T21:44:07.1943622Z DOCS=yes 2024-08-20T21:44:07.1943878Z INSTALLED_ANDROID= 2024-08-20T21:44:07.1944144Z SHLVL=1 2024-08-20T21:44:07.1944381Z MAX_JOBS=6 2024-08-20T21:44:07.1944650Z GITHUB_ACTOR_ID=97764156 2024-08-20T21:44:07.1945084Z GITHUB_WORKFLOW_SHA=f2fb9405c2fa9f9502a76363091cce6fd8179736 2024-08-20T21:44:07.1945669Z GITHUB_REF_NAME=133712/merge 2024-08-20T21:44:07.1946277Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T21:44:07.1946836Z GITHUB_JOB=test 2024-08-20T21:44:07.1947123Z NO_TEST_TIMEOUT=False 2024-08-20T21:44:07.1947434Z TD_DISTRIBUTED=False 2024-08-20T21:44:07.1947753Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T21:44:07.1948131Z GITHUB_RETENTION_DAYS=90 2024-08-20T21:44:07.1948458Z OPENSSL_DIR=/opt/openssl 2024-08-20T21:44:07.1948770Z GITHUB_ACTION_REPOSITORY= 2024-08-20T21:44:07.1949830Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:44:07.1951027Z GITHUB_BASE_REF=gh/XuehaiPan/146/base 2024-08-20T21:44:07.1951409Z INSTALLED_ACL= 2024-08-20T21:44:07.1951671Z CI=true 2024-08-20T21:44:07.1951940Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T21:44:07.1952282Z JOB_ID=29025446952 2024-08-20T21:44:07.1952599Z INSTALLED_PROTOBUF=yes 2024-08-20T21:44:07.1952955Z GITHUB_HEAD_REF=gh/XuehaiPan/146/head 2024-08-20T21:44:07.1953328Z GITHUB_ACTION_REF= 2024-08-20T21:44:07.1953889Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T21:44:07.1954576Z TEST_SHOWLOCALS=False 2024-08-20T21:44:07.1955145Z GITHUB_WORKFLOW=pull 2024-08-20T21:44:07.1955655Z DEBIAN_FRONTEND=noninteractive 2024-08-20T21:44:07.1957114Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_971f906e-00ca-41f0-8553-385d2c8ff1b7 2024-08-20T21:44:07.1958007Z NO_TD=False 2024-08-20T21:44:07.1958279Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T21:44:07.1958627Z _=/usr/bin/env 2024-08-20T21:44:07.1958941Z + echo 'Testing pytorch' 2024-08-20T21:44:07.1959270Z Testing pytorch 2024-08-20T21:44:07.1959568Z + export LANG=C.UTF-8 2024-08-20T21:44:07.1959868Z + LANG=C.UTF-8 2024-08-20T21:44:07.1960139Z + PR_NUMBER=133712 2024-08-20T21:44:07.1960456Z + [[ distributed == \d\e\f\a\u\l\t ]] 2024-08-20T21:44:07.1960880Z + [[ distributed == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-20T21:44:07.1961480Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-20T21:44:07.1961909Z + [[ distributed == \s\l\o\w ]] 2024-08-20T21:44:07.1962394Z + [[ linux-jammy-py3.8-gcc11 == *slow-gradcheck* ]] 2024-08-20T21:44:07.1962939Z + [[ linux-jammy-py3.8-gcc11 == *cuda* ]] 2024-08-20T21:44:07.1963441Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-20T21:44:07.1963914Z + [[ linux-jammy-py3.8-gcc11 == *xpu* ]] 2024-08-20T21:44:07.1964344Z + [[ distributed == *crossref* ]] 2024-08-20T21:44:07.1964801Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-20T21:44:07.1965304Z + [[ linux-jammy-py3.8-gcc11 == *xpu* ]] 2024-08-20T21:44:07.1965791Z + [[ linux-jammy-py3.8-gcc11 != *-bazel-* ]] 2024-08-20T21:44:07.1966280Z + pip_install --user ninja==1.10.2 2024-08-20T21:44:07.1966810Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-20T21:44:07.6240642Z Collecting ninja==1.10.2 2024-08-20T21:44:07.6457301Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-20T21:44:07.6553333Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-20T21:44:08.0784984Z Installing collected packages: ninja 2024-08-20T21:44:08.0876153Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-20T21:44:08.0877531Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-20T21:44:08.1120947Z Successfully installed ninja-1.10.2 2024-08-20T21:44:08.1800190Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:44:08.1802432Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:44:08.1804241Z + [[ linux-jammy-py3.8-gcc11 == *aarch64* ]] 2024-08-20T21:44:08.1804662Z + install_tlparse 2024-08-20T21:44:08.1805032Z + pip_install --user tlparse==0.3.25 2024-08-20T21:44:08.1805593Z + pip install --progress-bar off --user tlparse==0.3.25 2024-08-20T21:44:08.6198961Z Collecting tlparse==0.3.25 2024-08-20T21:44:08.6311970Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB) 2024-08-20T21:44:08.6757800Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-20T21:44:09.1430956Z Installing collected packages: tlparse 2024-08-20T21:44:09.2319529Z Successfully installed tlparse-0.3.25 2024-08-20T21:44:09.3032790Z ++ python -m site --user-base 2024-08-20T21:44:09.3394609Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:44:09.3396541Z + [[ linux-jammy-py3.8-gcc11 == *asan* ]] 2024-08-20T21:44:09.3397346Z + [[ linux-jammy-py3.8-gcc11 == *-debug* ]] 2024-08-20T21:44:09.3398179Z + [[ linux-jammy-py3.8-gcc11 != *-bazel-* ]] 2024-08-20T21:44:09.3398978Z + echo 'We are not in debug mode: linux-jammy-py3.8-gcc11. Expect the assertion to pass' 2024-08-20T21:44:09.3399976Z We are not in debug mode: linux-jammy-py3.8-gcc11. Expect the assertion to pass 2024-08-20T21:44:09.3400600Z + cd test 2024-08-20T21:44:09.3401101Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-20T21:44:12.2710697Z + [[ distributed == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-20T21:44:12.2711261Z + [[ distributed == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-20T21:44:12.2713170Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-20T21:44:12.2713830Z + [[ distributed == *pr_time_benchmarks* ]] 2024-08-20T21:44:12.2714674Z + [[ distributed == *dynamo_eager* ]] 2024-08-20T21:44:12.2715074Z + [[ distributed == *aot_eager* ]] 2024-08-20T21:44:12.2715504Z + [[ distributed == *aot_inductor* ]] 2024-08-20T21:44:12.2715911Z + [[ distributed == *inductor* ]] 2024-08-20T21:44:12.2716282Z + [[ distributed == *dynamic* ]] 2024-08-20T21:44:12.2716660Z + [[ distributed == *cpu* ]] 2024-08-20T21:44:12.2717242Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-20T21:44:12.2735297Z + [[ linux-jammy-py3.8-gcc11 == *libtorch* ]] 2024-08-20T21:44:12.2735923Z + [[ linux-jammy-py3.8-gcc11 == *-bazel-* ]] 2024-08-20T21:44:12.2738563Z + cd test 2024-08-20T21:44:12.2739216Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-20T21:44:13.5066216Z PyTorch built with: 2024-08-20T21:44:13.5067136Z - GCC 11.4 2024-08-20T21:44:13.5067713Z - C++ Version: 201703 2024-08-20T21:44:13.5068954Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T21:44:13.5070067Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T21:44:13.5070712Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T21:44:13.5071233Z - LAPACK is enabled (usually provided by MKL) 2024-08-20T21:44:13.5071700Z - NNPACK is enabled 2024-08-20T21:44:13.5072057Z - CPU capability usage: AVX512 2024-08-20T21:44:13.5079959Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-20T21:44:13.5087019Z 2024-08-20T21:44:13.7476682Z + cd test 2024-08-20T21:44:13.7477468Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-20T21:44:14.9727801Z ATen/Parallel: 2024-08-20T21:44:14.9728537Z at::get_num_threads() : 4 2024-08-20T21:44:14.9728934Z at::get_num_interop_threads() : 4 2024-08-20T21:44:14.9729319Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T21:44:14.9729775Z omp_get_max_threads() : 4 2024-08-20T21:44:14.9730812Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T21:44:14.9731639Z mkl_get_max_threads() : 4 2024-08-20T21:44:14.9732257Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T21:44:14.9732878Z std::thread::hardware_concurrency() : 8 2024-08-20T21:44:14.9733266Z Environment variables: 2024-08-20T21:44:14.9733606Z OMP_NUM_THREADS : [not set] 2024-08-20T21:44:14.9733959Z MKL_NUM_THREADS : [not set] 2024-08-20T21:44:14.9734307Z ATen parallel backend: OpenMP 2024-08-20T21:44:14.9734563Z 2024-08-20T21:44:15.2179665Z + [[ linux-jammy-py3.8-gcc11 == *aarch64* ]] 2024-08-20T21:44:15.2180309Z + [[ distributed == *backward* ]] 2024-08-20T21:44:15.2180687Z + [[ distributed == *xla* ]] 2024-08-20T21:44:15.2181073Z + [[ distributed == *executorch* ]] 2024-08-20T21:44:15.2181498Z + [[ distributed == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-20T21:44:15.2182037Z + [[ linux-jammy-py3.8-gcc11 == *libtorch* ]] 2024-08-20T21:44:15.2182476Z + [[ distributed == distributed ]] 2024-08-20T21:44:15.2182873Z + test_distributed 2024-08-20T21:44:15.2183262Z + echo 'Testing distributed python tests' 2024-08-20T21:44:15.2183686Z Testing distributed python tests 2024-08-20T21:44:15.2184292Z + python test/run_test.py --distributed-tests --shard 1 2 --verbose 2024-08-20T21:44:15.3410515Z test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-20T21:44:15.3411493Z import pkg_resources 2024-08-20T21:44:19.2138355Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-20T21:44:19.2629928Z Ignoring disabled issues: [''] 2024-08-20T21:44:19.2746817Z Found test times from artifacts 2024-08-20T21:44:19.3186707Z Found test times from artifacts 2024-08-20T21:44:19.3205506Z Running all tests 2024-08-20T21:44:19.3335271Z Running parallel tests on 3 processes 2024-08-20T21:44:19.3340569Z Name: tests to run (est. time: 60.19min) 2024-08-20T21:44:19.3341271Z Serial tests (101): 2024-08-20T21:44:19.3342044Z distributed/test_inductor_collectives 1/1 2024-08-20T21:44:19.3342566Z distributed/_tensor/test_dtensor_compile 1/1 2024-08-20T21:44:19.3343036Z distributed/test_data_parallel 1/1 2024-08-20T21:44:19.3343768Z distributed/_composable/test_replicate_with_compiler 1/1 2024-08-20T21:44:19.3344338Z distributed/_tensor/debug/test_op_coverage 1/1 2024-08-20T21:44:19.3344839Z distributed/test_c10d_functional_native 1/1 2024-08-20T21:44:19.3345317Z distributed/pipelining/test_transformer 1/1 2024-08-20T21:44:19.3345800Z distributed/_tensor/test_dtensor_ops 1/1 2024-08-20T21:44:19.3346298Z distributed/flight_recorder/test_fr_analysis 1/1 2024-08-20T21:44:19.3346779Z distributed/pipelining/test_pipe 1/1 2024-08-20T21:44:19.3347296Z distributed/tensor/parallel/test_micro_pipeline_tp 1/1 2024-08-20T21:44:19.3347863Z distributed/fsdp/test_fsdp_sharded_grad_scaler 1/1 2024-08-20T21:44:19.3348351Z distributed/launcher/test_run 1/1 2024-08-20T21:44:19.3348928Z distributed/pipelining/test_microbatch 1/1 2024-08-20T21:44:19.3349413Z distributed/_composable/test_contract 1/1 2024-08-20T21:44:19.3349884Z distributed/_tensor/test_common_rules 1/1 2024-08-20T21:44:19.3350334Z distributed/_tensor/test_random_ops 1/1 2024-08-20T21:44:19.3350889Z distributed/_tensor/experimental/test_register_sharding 1/1 2024-08-20T21:44:19.3351453Z distributed/optim/test_named_optimizer 1/1 2024-08-20T21:44:19.3351891Z distributed/_tensor/test_api 1/1 2024-08-20T21:44:19.3352340Z distributed/_tensor/test_experimental_ops 1/1 2024-08-20T21:44:19.3352804Z distributed/fsdp/test_fsdp_fx 1/1 2024-08-20T21:44:19.3353213Z distributed/_tensor/test_tensor_ops 1/1 2024-08-20T21:44:19.3353644Z distributed/test_device_mesh 1/1 2024-08-20T21:44:19.3354145Z distributed/_composable/fsdp/test_fully_shard_overlap 1/1 2024-08-20T21:44:19.3354733Z distributed/_tensor/experimental/test_tp_transform 1/1 2024-08-20T21:44:19.3355285Z distributed/tensor/parallel/test_tp_style 1/1 2024-08-20T21:44:19.3355778Z distributed/_tensor/test_redistribute 1/1 2024-08-20T21:44:19.3356224Z distributed/_tensor/test_matrix_ops 1/1 2024-08-20T21:44:19.3356665Z distributed/_tensor/test_math_ops 1/1 2024-08-20T21:44:19.3357152Z distributed/tensor/parallel/test_tp_examples 1/1 2024-08-20T21:44:19.3357705Z distributed/optim/test_apply_optimizer_in_backward 1/1 2024-08-20T21:44:19.3358247Z distributed/checkpoint/e2e/test_pipeline 1/1 2024-08-20T21:44:19.3358790Z distributed/_composable/fsdp/test_fully_shard_state 1/1 2024-08-20T21:44:19.3359299Z distributed/test_collective_utils 1/1 2024-08-20T21:44:19.3359852Z distributed/_composable/fully_shard/test_fully_shard_compile 1/1 2024-08-20T21:44:19.3360466Z distributed/checkpoint/test_state_dict_utils 1/1 2024-08-20T21:44:19.3360957Z distributed/pipelining/test_schedule 1/1 2024-08-20T21:44:19.3361436Z distributed/_tensor/test_embedding_ops 1/1 2024-08-20T21:44:19.3361899Z distributed/checkpoint/test_utils 1/1 2024-08-20T21:44:19.3362345Z distributed/_tensor/debug/test_comm_mode 1/1 2024-08-20T21:44:19.3362873Z distributed/optim/test_zero_redundancy_optimizer 1/1 2024-08-20T21:44:19.3363374Z distributed/fsdp/test_shard_utils 1/1 2024-08-20T21:44:19.3363862Z distributed/checkpoint/test_fsdp_model_state 1/1 2024-08-20T21:44:19.3364339Z distributed/rpc/test_share_memory 1/1 2024-08-20T21:44:19.3364792Z distributed/checkpoint/test_checkpoint 1/1 2024-08-20T21:44:19.3365233Z distributed/test_nccl 1/1 2024-08-20T21:44:19.3365755Z distributed/_composable/fully_shard/test_fully_shard_mixed_precision 1/1 2024-08-20T21:44:19.3366461Z distributed/_composable/fully_shard/test_fully_shard_util 1/1 2024-08-20T21:44:19.3367031Z distributed/_composable/test_replicate 1/1 2024-08-20T21:44:19.3367498Z distributed/checkpoint/test_state_dict 1/1 2024-08-20T21:44:19.3368009Z distributed/fsdp/test_fsdp_dtensor_state_dict 1/1 2024-08-20T21:44:19.3368493Z distributed/test_c10d_logger 1/1 2024-08-20T21:44:19.3368911Z distributed/fsdp/test_fsdp_state_dict 1/1 2024-08-20T21:44:19.3369546Z distributed/_composable/fsdp/test_fully_shard_autograd 1/1 2024-08-20T21:44:19.3370167Z distributed/checkpoint/test_file_system_checkpoint 1/1 2024-08-20T21:44:19.3370916Z distributed/_composable/test_composability/test_2d_composability 1/1 2024-08-20T21:44:19.3371594Z distributed/_composable/fsdp/test_fully_shard_frozen 1/1 2024-08-20T21:44:19.3372165Z distributed/fsdp/test_distributed_checkpoint 1/1 2024-08-20T21:44:19.3372780Z distributed/_composable/fsdp/test_fully_shard_mixed_precision 1/1 2024-08-20T21:44:19.3373351Z distributed/_tensor/test_utils 1/1 2024-08-20T21:44:19.3373781Z distributed/test_distributed_spawn 2/3 2024-08-20T21:44:19.3374381Z distributed/_composable/fully_shard/test_fully_shard_model_checkpoint 1/1 2024-08-20T21:44:19.3375018Z distributed/test_c10d_object_collectives 1/1 2024-08-20T21:44:19.3375654Z distributed/_composable/fully_shard/test_fully_shard_init 1/1 2024-08-20T21:44:19.3376233Z distributed/fsdp/test_fsdp_tp_integration 1/1 2024-08-20T21:44:19.3376699Z distributed/fsdp/test_fsdp_grad_acc 1/1 2024-08-20T21:44:19.3377198Z distributed/fsdp/test_fsdp_freezing_weights 1/1 2024-08-20T21:44:19.3377696Z distributed/test_multi_threaded_pg 1/1 2024-08-20T21:44:19.3378216Z distributed/algorithms/ddp_comm_hooks/test_ddp_hooks 1/1 2024-08-20T21:44:19.3378777Z distributed/fsdp/test_fsdp_flatten_params 1/1 2024-08-20T21:44:19.3379349Z distributed/checkpoint/test_file_system_checkpoint_cpu 1/1 2024-08-20T21:44:19.3379867Z distributed/test_c10d_pypg 1/1 2024-08-20T21:44:19.3380375Z distributed/_composable/fsdp/test_fully_shard_extensions 1/1 2024-08-20T21:44:19.3380926Z distributed/fsdp/test_fsdp_memory 1/1 2024-08-20T21:44:19.3381405Z distributed/fsdp/test_hsdp_dtensor_state_dict 1/1 2024-08-20T21:44:19.3382022Z distributed/_composable/fully_shard/test_fully_shard_runtime 1/1 2024-08-20T21:44:19.3382695Z distributed/algorithms/quantization/test_quantization 1/1 2024-08-20T21:44:19.3383254Z distributed/fsdp/test_fsdp_hybrid_shard 1/1 2024-08-20T21:44:19.3383723Z distributed/algorithms/test_join 1/1 2024-08-20T21:44:19.3384152Z distributed/fsdp/test_utils 1/1 2024-08-20T21:44:19.3384551Z distributed/fsdp/test_fsdp_core 1/1 2024-08-20T21:44:19.3385011Z distributed/fsdp/test_fsdp_checkpoint 1/1 2024-08-20T21:44:19.3385503Z distributed/fsdp/test_fsdp_use_orig_params 1/1 2024-08-20T21:44:19.3386051Z distributed/_shard/sharding_spec/test_sharding_spec 1/1 2024-08-20T21:44:19.3386575Z distributed/fsdp/test_fsdp_apply 1/1 2024-08-20T21:44:19.3387081Z distributed/_shard/sharding_plan/test_sharding_plan 1/1 2024-08-20T21:44:19.3387573Z distributed/test_pg_wrapper 1/1 2024-08-20T21:44:19.3387981Z distributed/fsdp/test_fsdp_input 1/1 2024-08-20T21:44:19.3388462Z distributed/fsdp/test_fsdp_multiple_wrapping 1/1 2024-08-20T21:44:19.3388964Z distributed/fsdp/test_fsdp_exec_order 1/1 2024-08-20T21:44:19.3389456Z distributed/fsdp/test_fsdp_mixed_precision 1/1 2024-08-20T21:44:19.3389950Z distributed/fsdp/test_fsdp_fine_tune 1/1 2024-08-20T21:44:19.3390402Z distributed/fsdp/test_fsdp_traversal 1/1 2024-08-20T21:44:19.3390945Z distributed/_shard/sharded_tensor/ops/test_tensor_ops 1/1 2024-08-20T21:44:19.3391639Z distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint 1/1 2024-08-20T21:44:19.3392327Z distributed/_shard/sharded_optim/test_sharded_optim 1/1 2024-08-20T21:44:19.3392850Z distributed/rpc/test_tensorpipe_agent 1/3 2024-08-20T21:44:19.3393326Z distributed/rpc/test_tensorpipe_agent 3/3 2024-08-20T21:44:19.3394012Z distributed/test_c10d_ucc 1/1 2024-08-20T21:44:19.3394421Z distributed/elastic/timer/api_test 1/1 2024-08-20T21:44:19.3394844Z Parallel tests (0): 2024-08-20T21:44:19.3395181Z Name: excluded (est. time: 0.0min) 2024-08-20T21:44:19.3395547Z Serial tests (0): 2024-08-20T21:44:19.3395850Z Parallel tests (0): 2024-08-20T21:44:19.3597295Z Running distributed/test_inductor_collectives 1/1 ... [2024-08-20 21:44:19.359154] 2024-08-20T21:44:19.3598510Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:44:19.3603921Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_inductor_collectives.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:44:19.359847] 2024-08-20T21:44:23.5399743Z 2024-08-20T21:44:23.5402528Z distributed/test_inductor_collectives 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_inductor_collectives_1.1_7e758b11cb23cba1_.log 2024-08-20T21:44:23.5460783Z Running 47 items in this shard: test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_all_to_all_single_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_all_to_all_single_inductor_split_sizes_none, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allgather_contiguous_input, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allgather_into_tensor_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allgather_output_buffer_reuse, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allreduce_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allreduce_inductor_cudagraph_trees, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allreduce_input_buffer_reuse, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_broadcast_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_c10d_functional_tagged_pt2_compliant, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_eager_allreduce_inductor_wait, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_inductor_allreduce_eager_wait, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_permute_tensor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_reduce_scatter_tensor_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_backwards, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_get_world_group_source_GroupMember_WORLD, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_get_world_group_source__get_default_group, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_get_world_group_source_group_WORLD, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_graphbreaks_unsupported_async_op, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_pg_var, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_gather, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_gather_args_match, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_gather_list, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_to_all_single, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_kwargs, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_kwargs_none, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_positional, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_positional_none, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_unspecified, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op0, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op1, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op2, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op3, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op4, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_reduce_scatter, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_support_collective_op_with_async_op_False, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_all_gather_tensor, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_all_gather_tensor_pg, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_allgather_coalesced, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_allreduce, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_reduce_scatter_tensor, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_all_gather_coalesced, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_doesnt_mutate_shared, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_reduce_scatter_coalesced, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_single_op, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_steal_buffer, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_meta 2024-08-20T21:44:23.5517273Z 2024-08-20T21:44:23.5518325Z Running distributed/_tensor/test_dtensor_compile 1/1 ... [2024-08-20 21:44:23.540665] 2024-08-20T21:44:23.5519600Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:44:23.5523067Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_dtensor_compile.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:44:23.541145] 2024-08-20T21:44:52.1801478Z 2024-08-20T21:44:52.1887535Z distributed/_tensor/test_dtensor_compile 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_dtensor_compile_1.1_cb0420759f8eaa45_.log 2024-08-20T21:44:52.1918144Z Running 26 items in this shard: test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_device_mesh_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_attribute_access_on_intermediate, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_constructor_w_dynamo_disable, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_constructor_w_graph_break, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_contiguous_dtensor_noncontiguous_local_as_tangent, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_different_gradient_placement, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_dont_recompile_on_same_placement_devicemesh, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_dynamo_device_mesh_attrs, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_noncontiguous_output, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_partial_placement_graph_output, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_partial_placement_redistribute_unbalanced_correct_strides, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor_from_local, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor_from_local_redistribute, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor_recompile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_to_local_kwargs, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_to_local_kwargs_forward_hook, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_fakify_dtensor, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_graph_input_is_async, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_placement_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_tp_compile_comm_reordering, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_2d_fsdp_tp_ac_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_2d_fsdp_tp_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_compile_dtensor_redistribute_backward, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_tp_compile_fullgraph_is_seq_parallel_False, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_tp_compile_fullgraph_is_seq_parallel_True 2024-08-20T21:44:52.1945549Z 2024-08-20T21:44:52.1946404Z Running distributed/test_data_parallel 1/1 ... [2024-08-20 21:44:52.181375] 2024-08-20T21:44:52.1947540Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:44:52.1950741Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_data_parallel.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:44:52.182240] 2024-08-20T21:44:55.4616622Z 2024-08-20T21:44:55.4618459Z distributed/test_data_parallel 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_data_parallel_1.1_f019788f800e2435_.log 2024-08-20T21:44:55.4644171Z Running 46 items in this shard: test/distributed/test_data_parallel.py::TestDataParallel::test_autocast, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_buffers_requiring_grad, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_complex, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_device_args, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_function_deletion, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_lazy_linear, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_model_device, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_model_no_refcycles, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_module_zero_inputs, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_multiple_input, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_nested_input, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_nested_output, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_no_grad, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_rnn, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_small_back, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_sparse, test/distributed/test_data_parallel.py::TestDataParallel::test_gather_cpu, test/distributed/test_data_parallel.py::TestDataParallel::test_gather_different_len_dicts, test/distributed/test_data_parallel.py::TestDataParallel::test_gather_gpu, test/distributed/test_data_parallel.py::TestDataParallel::test_parallel_apply, test/distributed/test_data_parallel.py::TestDataParallel::test_parallel_apply_autocast, test/distributed/test_data_parallel.py::TestDataParallel::test_parallel_apply_passes_exception, test/distributed/test_data_parallel.py::TestDataParallel::test_parameter_list_dict_replica, test/distributed/test_data_parallel.py::TestDataParallel::test_replicate, test/distributed/test_data_parallel.py::TestDataParallel::test_replicate_buffers, test/distributed/test_data_parallel.py::TestDataParallel::test_save_replica_module, test/distributed/test_data_parallel.py::TestDataParallel::test_scatter_cpu, test/distributed/test_data_parallel.py::TestDataParallel::test_scatter_gpu, test/distributed/test_data_parallel.py::TestDataParallel::test_strided_grad_layout, test/distributed/test_data_parallel.py::TestDataParallel::test_zero_grad, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_cpu_float16, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_cpu_float32, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_cpu_float64, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_cpu_float16, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_cpu_float32, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_cpu_float64, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_dict_cpu_float16, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_dict_cpu_float32, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_dict_cpu_float64, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_list_cpu_float16, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_list_cpu_float32, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_list_cpu_float64, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_tuple_cpu_float16, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_tuple_cpu_float32, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_tuple_cpu_float64 2024-08-20T21:44:55.4668429Z 2024-08-20T21:44:55.4669019Z Running distributed/_composable/test_replicate_with_compiler 1/1 ... [2024-08-20 21:44:55.462154] 2024-08-20T21:44:55.4669747Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:44:55.4671641Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_replicate_with_compiler.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:44:55.462586] 2024-08-20T21:44:58.5438133Z 2024-08-20T21:44:58.5441101Z distributed/_composable/test_replicate_with_compiler 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_replicate_with_compiler_1.1_d99c76b00dfc5226_.log 2024-08-20T21:44:58.5452641Z Running 10 items in this shard: test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_bucketing_coalesced_op, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_bucketing_concat_op, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_backward_only, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_bf16, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_cpu, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_cpu_no_sync, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_fp16, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_gpu, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_gpu_ac, test/distributed/_composable/test_replicate_with_compiler.py::DDP_TP_Test::test_ddp_tp 2024-08-20T21:44:58.5461161Z 2024-08-20T21:44:58.5461997Z Running distributed/_tensor/debug/test_op_coverage 1/1 ... [2024-08-20 21:44:58.544497] 2024-08-20T21:44:58.5462941Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:44:58.5465441Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/debug/test_op_coverage.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:44:58.545073] 2024-08-20T21:45:01.7343275Z 2024-08-20T21:45:01.7345385Z distributed/_tensor/debug/test_op_coverage 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.debug.test_op_coverage_1.1_b10a038b6d1b625f_.log 2024-08-20T21:45:01.7347188Z Running 1 items in this shard: test/distributed/_tensor/debug/test_op_coverage.py::TestOpCoverage::test_trace_with_inductor_decomp 2024-08-20T21:45:01.7347987Z 2024-08-20T21:45:01.7349799Z Running distributed/test_c10d_functional_native 1/1 ... [2024-08-20 21:45:01.734764] 2024-08-20T21:45:01.7350525Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:45:01.7355209Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_functional_native.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:45:01.735211] 2024-08-20T21:45:04.6104345Z 2024-08-20T21:45:04.6106423Z distributed/test_c10d_functional_native 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_functional_native_1.1_4be27fce5d78c9e7_.log 2024-08-20T21:45:04.6121305Z Running 25 items in this shard: test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_gather_into_tensor_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_gather_into_tensor_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_coalesced_, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_single_, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_to_all_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_broadcast, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_inductor_dtypeview_memory_leak, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_py_work, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_reduce_scatter_tensor_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_threading, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_unwaited, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_gather_into_tensor_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_gather_into_tensor_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_reduce_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_reduce_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_to_all_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_broadcast, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_inplace_op_on_view, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reduce_scatter_tensor_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reuse_buffer_after_inplace_collective, test/distributed/test_c10d_functional_native.py::CompileTest::test_ranks_and_tag 2024-08-20T21:45:04.6133918Z 2024-08-20T21:45:04.6134437Z Running distributed/pipelining/test_transformer 1/1 ... [2024-08-20 21:45:04.610912] 2024-08-20T21:45:04.6135102Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:45:04.6137048Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_transformer.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:45:04.611403] 2024-08-20T21:45:08.3382866Z 2024-08-20T21:45:08.3384852Z distributed/pipelining/test_transformer 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_transformer_1.1_83c2e6f61728cb36_.log 2024-08-20T21:45:08.3386529Z Running 1 items in this shard: test/distributed/pipelining/test_transformer.py::TransformerTests::test_ir 2024-08-20T21:45:08.3387192Z 2024-08-20T21:45:08.3389296Z Running distributed/_tensor/test_dtensor_ops 1/1 ... [2024-08-20 21:45:08.338727] 2024-08-20T21:45:08.3389916Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:45:08.3394808Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_dtensor_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:45:08.339167] 2024-08-20T21:46:29.1262287Z 2024-08-20T21:46:29.1264314Z distributed/_tensor/test_dtensor_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_dtensor_ops_1.1_bd362836885712c6_.log 2024-08-20T21:46:29.1974846Z Running 667 items in this shard: test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_H_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_T_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___getitem___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___radd___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rdiv___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rmatmul___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rmod___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rmul___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rpow___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rsub___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__batch_norm_with_update_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__chunk_cat_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__native_batch_norm_legit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__segment_reduce_lengths_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__segment_reduce_offsets_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__softmax_backward_data_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__unsafe_masked_index_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__unsafe_masked_index_put_accumulate_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__upsample_bilinear2d_aa_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_abs_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_acos_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_acosh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_add_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addbmm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addcdiv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addcmul_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addmm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addmm_decomposed_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addmv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_alias_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_all_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_allclose_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_amax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_amin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_aminmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_angle_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_any_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_arange_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argmin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argsort_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argwhere_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_partial_views_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_asin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_asinh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atan2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atan_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atanh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atleast_1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atleast_2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atleast_3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_baddbmm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bernoulli_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bfloat16_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_block_diag_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bmm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bool_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_broadcast_shapes_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_broadcast_tensors_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_broadcast_to_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bucketize_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_byte_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cartesian_prod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cat_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cauchy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cdist_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cdouble_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ceil_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cfloat_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_chalf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_char_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cholesky_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cholesky_inverse_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cholesky_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_chunk_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clamp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clamp_max_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clamp_min_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clone_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_column_stack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_combinations_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_complex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_conj_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_conj_physical_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_constant_pad_nd_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_contiguous_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_copysign_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_corrcoef_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cos_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cosh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_count_nonzero_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cov_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cross_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cummax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cummin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cumprod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cumsum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cumulative_trapezoid_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_deg2rad_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diag_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diag_embed_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagflat_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagonal_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagonal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagonal_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diff_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_digamma_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dist_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_div_floor_rounding_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_div_no_rounding_mode_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_div_trunc_rounding_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_double_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dsplit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dstack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_einsum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_permuted_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_strided_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_eq_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_equal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_erf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_erfc_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_erfinv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_exp2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_exp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_expand_as_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_expand_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_expand_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_expm1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_exponential_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_eye_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fftshift_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_hfft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_hfft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_hfftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifftshift_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ihfft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ihfft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ihfftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_irfft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_irfft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_irfftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_rfft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_rfft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_rfftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fill_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_flatten_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_flip_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fliplr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_flipud_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_float_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_float_power_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_floor_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_floor_divide_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fmin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fmod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_frac_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_frexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_full_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_full_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_gather_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ge_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_geometric_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_geqrf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_gradient_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_grid_sampler_2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_gt_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_half_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_heaviside_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_histc_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_histogram_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_histogramdd_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_hsplit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_hstack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_hypot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_i0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_igamma_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_igammac_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_add_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_fill_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_put_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_amax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_amin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_prod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_select_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_inner_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_int_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isclose_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isfinite_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isinf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isnan_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isneginf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isposinf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isreal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_item_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_2inputs_2outputs_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_4inputs_with_extra_args_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_binary_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_binary_return_by_ref_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_unary_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_kron_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_kthvalue_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ldexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_le_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lerp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lgamma_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cholesky_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cholesky_ex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cond_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cross_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_det_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_det_singular_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_diagonal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eig_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eigh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eigvals_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eigvalsh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_householder_product_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_inv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_inv_ex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_ldl_factor_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_ldl_factor_ex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_ldl_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lstsq_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lstsq_grad_oriented_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_factor_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_factor_ex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_power_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_rank_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_rank_hermitian_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_multi_dot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_norm_subgradients_at_zero_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_pinv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_pinv_hermitian_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_pinv_singular_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_qr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_slogdet_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_solve_ex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_solve_triangular_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_svd_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_svdvals_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_tensorinv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_tensorsolve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_vander_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_vecdot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_vector_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linspace_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linspace_tensor_overload_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log10_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log1p_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_normal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_softmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_softmax_with_dtype_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logaddexp2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logaddexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logcumsumexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logdet_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_and_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_not_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_or_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_xor_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logspace_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logspace_tensor_overload_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logsumexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_long_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lt_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lu_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lu_unpack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mH_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mT_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_amax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_amin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_argmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_argmin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_cumprod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_cumsum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_fill_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_log_softmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_logaddexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_logsumexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_median_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_normalize_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_prod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_select_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_softmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_softmin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_std_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_sum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_var_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_matmul_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_matrix_exp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_binary_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_pool2d_with_indices_backward_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_reduction_no_dim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_reduction_with_dim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_maximum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_median_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_meshgrid_list_of_tensors_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_meshgrid_variadic_tensors_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_min_binary_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_min_reduction_no_dim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_min_reduction_with_dim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_minimum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mode_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_movedim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_msort_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mul_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_multinomial_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mvlgamma_mvlgamma_p_1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mvlgamma_mvlgamma_p_3_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mvlgamma_mvlgamma_p_5_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nan_to_num_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nanmean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nanmedian_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nanquantile_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nansum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_narrow_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_narrow_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_native_batch_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_native_dropout_backward_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_native_layer_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ne_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_neg_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_empty_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_empty_strided_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_full_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_ones_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_zeros_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nextafter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_avg_pool1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_avg_pool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_avg_pool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_max_pool1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_max_pool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_max_pool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_alpha_dropout_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_avg_pool1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_avg_pool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_avg_pool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_batch_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_bilinear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_binary_cross_entropy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_binary_cross_entropy_with_logits_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_celu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_channel_shuffle_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv_transpose1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv_transpose2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv_transpose3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_cosine_embedding_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_cosine_similarity_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_cross_entropy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_ctc_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_dropout2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_dropout3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_dropout_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_elu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_embedding_bag_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_embedding_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_feature_alpha_dropout_with_train_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_feature_alpha_dropout_without_train_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_fractional_max_pool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_fractional_max_pool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_gaussian_nll_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_gelu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_glu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_grid_sample_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_group_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardshrink_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardsigmoid_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardswish_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardtanh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hinge_embedding_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_huber_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_instance_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_area_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_bicubic_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_bilinear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_linear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_nearest-exact_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_nearest_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_trilinear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_kl_div_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_l1_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_layer_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_leaky_relu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_linear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_local_response_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_logsigmoid_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_margin_ranking_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_pool1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_pool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_pool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool1d_grad_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool2d_grad_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool3d_grad_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_mish_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_mse_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multi_head_attention_forward_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multi_margin_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multilabel_margin_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multilabel_soft_margin_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_nll_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_normalize_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_circular_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_constant_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_reflect_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_replicate_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_replicate_negative_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pairwise_distance_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pdist_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pixel_shuffle_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pixel_unshuffle_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_poisson_nll_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_prelu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_relu6_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_relu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_rms_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_rrelu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_scaled_dot_product_attention_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_selu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_silu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_smooth_l1_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_soft_margin_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softmin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softmin_with_dtype_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softplus_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softshrink_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softsign_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_tanhshrink_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_threshold_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_triplet_margin_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_triplet_margin_with_distance_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_unfold_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_upsample_bilinear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_upsample_nearest_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nonzero_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nonzero_static_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_fro_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_inf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_nuc_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_normal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_normal_in_place_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_normal_number_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ones_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ones_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ormqr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_outer_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_pca_lowrank_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_permute_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_pinverse_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polar_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_3_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_4_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_positive_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_pow_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_prod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_put_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_qr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_quantile_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rad2deg_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rand_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randint_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randint_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randn_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ravel_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_real_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_reciprocal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_remainder_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_renorm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_repeat_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_repeat_interleave_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_reshape_as_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_reshape_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resize__cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resize_as__cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resolve_conj_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resolve_neg_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_roll_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rot90_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_decimals_0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_decimals_3_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_decimals_neg_3_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rsqrt_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rsub_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scalar_tensor_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_add_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_amax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_amin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_prod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_sum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_searchsorted_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_select_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_select_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sgn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_short_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sigmoid_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sign_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_bartlett_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_blackman_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_cosine_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_exponential_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_gaussian_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_general_cosine_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_general_hamming_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_hamming_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_hann_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_kaiser_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_nuttall_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signbit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sinc_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sinh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_slice_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_slice_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_softmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_softmax_with_dtype_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sort_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sparse_mm_reduce_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sparse_sampled_addmm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_airy_ai_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_j0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_j1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_y0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_y1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_t_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_u_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_v_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_w_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_entr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_erfcx_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_hermite_polynomial_h_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_hermite_polynomial_he_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_i0e_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_i1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_i1e_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_laguerre_polynomial_l_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_legendre_polynomial_p_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_log_ndtr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_i0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_i1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_k0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_k1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_ndtr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_ndtri_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_polygamma_special_polygamma_n_0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_scaled_modified_bessel_k0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_scaled_modified_bessel_k1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_t_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_u_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_v_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_w_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_spherical_bessel_j0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_xlog1py_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_zeta_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_list_args_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_with_sizes_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_with_sizes_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sqrt_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_square_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_squeeze_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_squeeze_multiple_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_stack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_mean_unbiased_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_unbiased_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_stft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sub_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sum_to_size_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_svd_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_svd_lowrank_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_t_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_t_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_take_along_dim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_take_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tan_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tanh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tensor_split_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tensordot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tile_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_to_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_to_sparse_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_topk_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_torch_ops_aten__safe_softmax_default_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trace_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_transpose_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trapezoid_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trapz_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_triangular_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tril_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_triu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_true_divide_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trunc_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unbind_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unflatten_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unfold_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unfold_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_uniform_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unique_consecutive_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unique_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unsafe_chunk_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unsafe_split_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unsqueeze_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unsqueeze_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_mean_unbiased_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_unbiased_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_vdot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_as_complex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_as_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_vsplit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_vstack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_where_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_xlogy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_zero__cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_zeros_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_zeros_like_cpu_float32 2024-08-20T21:46:29.2350463Z 2024-08-20T21:46:29.2351149Z Running distributed/flight_recorder/test_fr_analysis 1/1 ... [2024-08-20 21:46:29.217949] 2024-08-20T21:46:29.2351855Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:46:29.2353699Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/flight_recorder/test_fr_analysis.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:46:29.218469] 2024-08-20T21:46:32.2189251Z 2024-08-20T21:46:32.2191628Z distributed/flight_recorder/test_fr_analysis 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.flight_recorder.test_fr_analysis_1.1_aec96f27df022b66_.log 2024-08-20T21:46:32.2194109Z Running 1 items in this shard: test/distributed/flight_recorder/test_fr_analysis.py::FlightRecorderEventTest::test_match_one_event 2024-08-20T21:46:32.2195261Z 2024-08-20T21:46:32.2195747Z Running distributed/pipelining/test_pipe 1/1 ... [2024-08-20 21:46:32.219333] 2024-08-20T21:46:32.2196389Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:46:32.2200940Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_pipe.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:46:32.219805] 2024-08-20T21:46:36.0465157Z 2024-08-20T21:46:36.0467423Z distributed/pipelining/test_pipe 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_pipe_1.1_6fa8bdeb647c917c_.log 2024-08-20T21:46:36.0470352Z Running 3 items in this shard: test/distributed/pipelining/test_pipe.py::PipeTests::test_model_split_ModelClass0, test/distributed/pipelining/test_pipe.py::PipeTests::test_model_split_ModelClass1, test/distributed/pipelining/test_pipe.py::PipeTests::test_model_split_ModelClass2 2024-08-20T21:46:36.0472259Z 2024-08-20T21:46:36.0472841Z Running distributed/tensor/parallel/test_micro_pipeline_tp 1/1 ... [2024-08-20 21:46:36.046976] 2024-08-20T21:46:36.0473643Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:46:36.0477176Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_micro_pipeline_tp.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:46:36.047413] 2024-08-20T21:46:39.9241501Z 2024-08-20T21:46:39.9243724Z distributed/tensor/parallel/test_micro_pipeline_tp 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_micro_pipeline_tp_1.1_603298d07a481f63_.log 2024-08-20T21:46:39.9267302Z Running 29 items in this shard: test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_dtensor_seq_par_shard_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_dtensor_seq_par_shard_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_find_all_gather_patterns, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_find_reduce_scatter_patterns, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_2_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_2_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_2_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_3_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_3_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_3_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_2_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_2_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_2_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_3_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_3_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_3_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_2_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_2_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_2_scatter_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_3_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_3_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_3_scatter_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_2_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_2_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_2_scatter_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_3_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_3_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_3_scatter_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_get_unexposed_collectives 2024-08-20T21:46:39.9288370Z 2024-08-20T21:46:39.9288937Z Running distributed/fsdp/test_fsdp_sharded_grad_scaler 1/1 ... [2024-08-20 21:46:39.924583] 2024-08-20T21:46:39.9289644Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:46:39.9291572Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_sharded_grad_scaler.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:46:39.925067] 2024-08-20T21:47:33.5085818Z 2024-08-20T21:47:33.5087697Z distributed/fsdp/test_fsdp_sharded_grad_scaler 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_sharded_grad_scaler_1.1_e257fd07db47c557_.log 2024-08-20T21:47:33.5107860Z Running 20 items in this shard: test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardGradScaler::test_grad_scaling, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardGradScaler::test_inf_gradients_skip_optim_step, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardGradScaler::test_scaling_unscaling_sparse, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_none_mixed_precision_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_none_mixed_precision_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_none_none_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_none_none_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_shard_grad_op_mixed_precision_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_shard_grad_op_mixed_precision_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_shard_grad_op_none_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_shard_grad_op_none_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_none_mixed_precision_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_none_mixed_precision_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_none_none_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_none_none_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_shard_grad_op_mixed_precision_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_shard_grad_op_mixed_precision_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_shard_grad_op_none_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_shard_grad_op_none_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_sharded_grad_scaler_found_inf 2024-08-20T21:47:33.5125200Z 2024-08-20T21:47:33.5125746Z Running distributed/launcher/test_run 1/1 ... [2024-08-20 21:47:33.508942] 2024-08-20T21:47:33.5126355Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:47:33.5128104Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/launcher/test_run.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:47:33.509484] 2024-08-20T21:48:12.4525366Z 2024-08-20T21:48:12.4528042Z distributed/launcher/test_run 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.launcher.test_run_1.1_533cd6f2282d992b_.log 2024-08-20T21:48:12.4544971Z Running 24 items in this shard: test/distributed/launcher/test_run.py::ElasticLaunchTest::test_capture_logs_using_default_logs_specs, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_init_method_env_with_torchelastic, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_init_method_tcp_with_torchelastic, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_is_not_torchelastic_launched, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_is_torchelastic_launched, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_is_torchelastic_launched_with_logs_spec_defined, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic_agent_raise_exception, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic_multiple_agents, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic_worker_raise_exception, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_run_path, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_shutdown, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_standalone, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_bash, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_default_nproc, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_python, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_python_caffe2_bc, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_with_env_vars, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_logs_logs_spec_entrypoint_must_be_defined, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_min_max_nodes_parse, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_gpu_launch_configurations, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_launch_auto_configurations, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_launch_number_configurations, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_launch_unknown_configurations 2024-08-20T21:48:12.4573021Z 2024-08-20T21:48:12.4573640Z Running distributed/pipelining/test_microbatch 1/1 ... [2024-08-20 21:48:12.453463] 2024-08-20T21:48:12.4574326Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:48:12.4576163Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_microbatch.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:48:12.453935] 2024-08-20T21:48:15.9304412Z 2024-08-20T21:48:15.9306392Z distributed/pipelining/test_microbatch 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_microbatch_1.1_43db94dfe9ada2c1_.log 2024-08-20T21:48:15.9309171Z Running 2 items in this shard: test/distributed/pipelining/test_microbatch.py::MicrobatchTests::test_chunk_spec, test/distributed/pipelining/test_microbatch.py::MicrobatchTests::test_split_and_merge 2024-08-20T21:48:15.9310419Z 2024-08-20T21:48:15.9311617Z Running distributed/_composable/test_contract 1/1 ... [2024-08-20 21:48:15.930942] 2024-08-20T21:48:15.9312267Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:48:15.9316998Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_contract.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:48:15.931406] 2024-08-20T21:48:18.9569447Z 2024-08-20T21:48:18.9571718Z distributed/_composable/test_contract 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_contract_1.1_6765637ff7c081a3_.log 2024-08-20T21:48:18.9575145Z Running 5 items in this shard: test/distributed/_composable/test_contract.py::TestContract::test_add_hooks, test/distributed/_composable/test_contract.py::TestContract::test_modify_fqn, test/distributed/_composable/test_contract.py::TestContract::test_multi_module_api, test/distributed/_composable/test_contract.py::TestContract::test_registry, test/distributed/_composable/test_contract.py::TestContract::test_state 2024-08-20T21:48:18.9578035Z 2024-08-20T21:48:18.9578503Z Running distributed/_tensor/test_common_rules 1/1 ... [2024-08-20 21:48:18.957476] 2024-08-20T21:48:18.9579161Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:48:18.9582728Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_common_rules.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:48:18.957972] 2024-08-20T21:48:51.7469514Z 2024-08-20T21:48:51.7471547Z distributed/_tensor/test_common_rules 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_common_rules_1.1_e62fb4d92b26a577_.log 2024-08-20T21:48:51.7478713Z Running 10 items in this shard: test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_basic_propagation, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_errors, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_linearity, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_merge_sharding, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_multi_sharding_on_mesh_dim, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_pointwise_propagation, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_enforce_sharding_multi_sharding_on_mesh_dim, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_multi_sharding_on_mesh_dim, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_rules_broadcasting, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_rules_suggestion 2024-08-20T21:48:51.7484248Z 2024-08-20T21:48:51.7484869Z Running distributed/_tensor/test_random_ops 1/1 ... [2024-08-20 21:48:51.747442] 2024-08-20T21:48:51.7485510Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:48:51.7487294Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_random_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:48:51.747944] 2024-08-20T21:49:15.4235359Z 2024-08-20T21:49:15.4238053Z distributed/_tensor/test_random_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_random_ops_1.1_40fdcbc4dc7ed87f_.log 2024-08-20T21:49:15.4247037Z Running 7 items in this shard: test/distributed/_tensor/test_random_ops.py::DistTensorRandomInitTest::test_init_ops, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_dropout_1d, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_rand_1d, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_uniform_2d, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_manual_seed, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_meta_tensor_init, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_rng_tracker_init 2024-08-20T21:49:15.4254276Z 2024-08-20T21:49:15.4255416Z Running distributed/_tensor/experimental/test_register_sharding 1/1 ... [2024-08-20 21:49:15.424160] 2024-08-20T21:49:15.4256857Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:49:15.4260465Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/experimental/test_register_sharding.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:49:15.424649] 2024-08-20T21:49:24.8140572Z 2024-08-20T21:49:24.8142808Z distributed/_tensor/experimental/test_register_sharding 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.experimental.test_register_sharding_1.1_7e7f2678996c73b2_.log 2024-08-20T21:49:24.8145413Z Running 2 items in this shard: test/distributed/_tensor/experimental/test_register_sharding.py::TestRegisterSharding::test_argmax, test/distributed/_tensor/experimental/test_register_sharding.py::TestRegisterSharding::test_softmax_fwd 2024-08-20T21:49:24.8147270Z 2024-08-20T21:49:24.8147798Z Running distributed/optim/test_named_optimizer 1/1 ... [2024-08-20 21:49:24.814481] 2024-08-20T21:49:24.8148469Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:49:24.8152529Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/optim/test_named_optimizer.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:49:24.814922] 2024-08-20T21:49:27.3796708Z 2024-08-20T21:49:27.3799402Z distributed/optim/test_named_optimizer 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.optim.test_named_optimizer_1.1_8b27d463d086bcfa_.log 2024-08-20T21:49:27.3801497Z 2024-08-20T21:49:27.3805855Z Running distributed/_tensor/test_api 1/1 ... [2024-08-20 21:49:27.380324] 2024-08-20T21:49:27.3806906Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:49:27.3813161Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_api.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:49:27.380870] 2024-08-20T21:49:54.2577787Z 2024-08-20T21:49:54.2579860Z distributed/_tensor/test_api 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_api_1.1_7085bda8a23e0f6d_.log 2024-08-20T21:49:54.2585328Z Running 8 items in this shard: test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_module, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_module_casting, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_module_input_fn_output_fn, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_module_input_fn_output_fn_warning, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_module_meta, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_tensor, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_tensor_errors, test/distributed/_tensor/test_api.py::DTensorAPITest::test_distribute_tensor_uneven_sharding 2024-08-20T21:49:54.2589684Z 2024-08-20T21:49:54.2590200Z Running distributed/_tensor/test_experimental_ops 1/1 ... [2024-08-20 21:49:54.258345] 2024-08-20T21:49:54.2590877Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:49:54.2592708Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_experimental_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:49:54.258797] 2024-08-20T21:50:05.8505522Z 2024-08-20T21:50:05.8507644Z distributed/_tensor/test_experimental_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_experimental_ops_1.1_a94ba2b7a383c63e_.log 2024-08-20T21:50:05.8510368Z Running 3 items in this shard: test/distributed/_tensor/test_experimental_ops.py::DistOtherOpsTest::test_bernoulli, test/distributed/_tensor/test_experimental_ops.py::DistOtherOpsTest::test_nll, test/distributed/_tensor/test_experimental_ops.py::DistOtherOpsTest::test_slice 2024-08-20T21:50:05.8512334Z 2024-08-20T21:50:05.8512755Z Running distributed/fsdp/test_fsdp_fx 1/1 ... [2024-08-20 21:50:05.850919] 2024-08-20T21:50:05.8513363Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:50:05.8516774Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_fx.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:50:05.851358] 2024-08-20T21:50:09.0772923Z 2024-08-20T21:50:09.0774579Z distributed/fsdp/test_fsdp_fx 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_fx_1.1_f9b99d5f60b76fc9_.log 2024-08-20T21:50:09.0776989Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_fx.py::TestSymbolicTracing::test_symbolic_tracing_outputs 2024-08-20T21:50:09.0777885Z 2024-08-20T21:50:09.0779341Z Running distributed/_tensor/test_tensor_ops 1/1 ... [2024-08-20 21:50:09.077696] 2024-08-20T21:50:09.0779965Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:50:09.0784975Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_tensor_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:50:09.078127] 2024-08-20T21:51:33.2261615Z 2024-08-20T21:51:33.2264774Z distributed/_tensor/test_tensor_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_tensor_ops_1.1_4017135920ceadd3_.log 2024-08-20T21:51:33.2285356Z Running 25 items in this shard: test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_aten_contiguous, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_clone, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_contiguous, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_detach, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_dtensor_dtype_conversion, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_empty_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_equal, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_fill_inplace, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_fill_inplace_partial_sum, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_full_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_gather, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_index, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_inplace_op, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_new_empty_strided, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_new_full, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_ones_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_ones_like_partial_sum, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_op_out_variant, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_scatter, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_slice, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_stack, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_where_type_promotion, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_zero_inplace, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_zeros_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_zeros_like_partial_sum 2024-08-20T21:51:33.2302680Z 2024-08-20T21:51:33.2303378Z Running distributed/test_device_mesh 1/1 ... [2024-08-20 21:51:33.227166] 2024-08-20T21:51:33.2304689Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:51:33.2307185Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_device_mesh.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:51:33.227729] 2024-08-20T21:54:11.4449269Z 2024-08-20T21:54:11.4451362Z distributed/test_device_mesh 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_device_mesh_1.1_cfa564fa7fd9e116_.log 2024-08-20T21:54:11.4473135Z Running 41 items in this shard: test/distributed/test_device_mesh.py::DeviceMeshTestGlooBackend::test_device_mesh_reuse_default_group, test/distributed/test_device_mesh.py::DeviceMeshTest::test_assert_invalid_mesh_tensor, test/distributed/test_device_mesh.py::DeviceMeshTest::test_device_mesh_2d, test/distributed/test_device_mesh.py::DeviceMeshTest::test_device_mesh_init_backend, test/distributed/test_device_mesh.py::DeviceMeshTest::test_fake_pg_device_mesh, test/distributed/test_device_mesh.py::DeviceMeshTest::test_from_group_with_global_pg, test/distributed/test_device_mesh.py::DeviceMeshTest::test_from_group_with_invalid_mesh, test/distributed/test_device_mesh.py::DeviceMeshTest::test_get_group_and_get_all_groups, test/distributed/test_device_mesh.py::DeviceMeshTest::test_get_local_rank, test/distributed/test_device_mesh.py::DeviceMeshTest::test_get_local_rank_raises_exception, test/distributed/test_device_mesh.py::DeviceMeshTest::test_init_process_group, test/distributed/test_device_mesh.py::DeviceMeshTest::test_raises_invalid_device_type, test/distributed/test_device_mesh.py::DeviceMeshTest::test_set_mesh_dim_group_options, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_device_mesh_hash, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_device_mesh_nd, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_device_mesh_parent_child_hash, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_from_group_with_mesh_shape, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_get_local_rank_3d, test/distributed/test_device_mesh.py::InitDeviceMeshTest::test_init_device_mesh, test/distributed/test_device_mesh.py::InitDeviceMeshTest::test_raises_duplicate_mesh_dim_names, test/distributed/test_device_mesh.py::InitDeviceMeshTest::test_raises_mesh_shape_mesh_dim_names_mismatch, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_cache_and_reuse_submesh_slice_result, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_flatten_mesh, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_1d, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_2d, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_3d, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_3d_noncontiguous_slicing, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_raises_invalid_mesh_dim_name, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_raises_no_mesh_dim_found, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_mesh_dim_by_name, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_root_mesh, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_root_mesh_dim_exist, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_root_mesh_dim_not_exist, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_all_gather_uneven, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_broadcast_1d, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_broadcast_nd, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_reduce_scatter_contiguous, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_reduce_scatter_uneven, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_scatter_1d, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_scatter_nd, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_scatter_uneven 2024-08-20T21:54:11.4493196Z 2024-08-20T21:54:11.4494037Z Running distributed/_composable/fsdp/test_fully_shard_overlap 1/1 ... [2024-08-20 21:54:11.445251] 2024-08-20T21:54:11.4494783Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:54:11.4496683Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_overlap.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:54:11.445749] 2024-08-20T21:54:14.4712372Z 2024-08-20T21:54:14.4714335Z distributed/_composable/fsdp/test_fully_shard_overlap 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_overlap_1.1_bb6c792a84bc09e9_.log 2024-08-20T21:54:14.4717090Z Running 2 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_overlap.py::TestFullyShardOverlap::test_fully_shard_post_optim_event_overlap, test/distributed/_composable/fsdp/test_fully_shard_overlap.py::TestFullyShardOverlap::test_fully_shard_training_overlap 2024-08-20T21:54:14.4718709Z 2024-08-20T21:54:14.4719728Z Running distributed/_tensor/experimental/test_tp_transform 1/1 ... [2024-08-20 21:54:14.471743] 2024-08-20T21:54:14.4720448Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:54:14.4727329Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/experimental/test_tp_transform.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:54:14.472193] 2024-08-20T21:54:28.5712318Z 2024-08-20T21:54:28.5714282Z distributed/_tensor/experimental/test_tp_transform 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.experimental.test_tp_transform_1.1_b362bbe64109f46a_.log 2024-08-20T21:54:28.5717528Z Running 3 items in this shard: test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_e2e, test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_no_bias, test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_with_uncovered_op 2024-08-20T21:54:28.5719892Z 2024-08-20T21:54:28.5720396Z Running distributed/tensor/parallel/test_tp_style 1/1 ... [2024-08-20 21:54:28.571645] 2024-08-20T21:54:28.5721105Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:54:28.5723631Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_tp_style.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:54:28.572085] 2024-08-20T21:54:58.9050825Z 2024-08-20T21:54:58.9052920Z distributed/tensor/parallel/test_tp_style 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_tp_style_1.1_f8620952d2a90112_.log 2024-08-20T21:54:58.9059766Z Running 9 items in this shard: test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_colwise_parallel_embedding, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_colwise_parallel_style, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_prepare_module_input, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_prepare_module_input_multiple_inputs, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_prepare_module_kwargs_input, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_prepare_module_output, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_rowwise_parallel_embedding, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_rowwise_parallel_style, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_sequence_parallel_style 2024-08-20T21:54:58.9065506Z 2024-08-20T21:54:58.9065964Z Running distributed/_tensor/test_redistribute 1/1 ... [2024-08-20 21:54:58.905571] 2024-08-20T21:54:58.9066615Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:54:58.9068415Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_redistribute.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:54:58.906002] 2024-08-20T21:55:45.3400711Z 2024-08-20T21:55:45.3403025Z distributed/_tensor/test_redistribute 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_redistribute_1.1_d6251b308089ddb2_.log 2024-08-20T21:55:45.3412281Z Running 13 items in this shard: test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_partial_to_replicate_forward_backward, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_partial_to_shard, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_redistribute_negative_shard_dim, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_redistribute_shard_dim_change, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_redistribute_uneven_sharding, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_replicate_to_local_partial_grad, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_replicate_to_partial, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_replicate_to_replicate_forward_backward, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_replicate_to_shard_forward_backward, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_shard_dim_alltoall, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_shard_to_replicate_forward_backward, test/distributed/_tensor/test_redistribute.py::MultiDimRedistributeTest::test_multi_dim_mesh, test/distributed/_tensor/test_redistribute.py::MultiDimRedistributeTest::test_redistribute_shard_dim_multi_dim_mesh 2024-08-20T21:55:45.3420844Z 2024-08-20T21:55:45.3421797Z Running distributed/_tensor/test_matrix_ops 1/1 ... [2024-08-20 21:55:45.341803] 2024-08-20T21:55:45.3422719Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:55:45.3425076Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_matrix_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:55:45.342231] 2024-08-20T21:56:14.7738113Z 2024-08-20T21:56:14.7739869Z distributed/_tensor/test_matrix_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_matrix_ops_1.1_849579c2d450512d_.log 2024-08-20T21:56:14.7745323Z Running 9 items in this shard: test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_addmm, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_addmm_auto_redistribute, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_addmm_empty_operand, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_baddbmm, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_bmm, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_mm, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_scaled_dot_product_attention, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_t, test/distributed/_tensor/test_matrix_ops.py::DistMatrixOpsTest::test_t_partial 2024-08-20T21:56:14.7749729Z 2024-08-20T21:56:14.7750253Z Running distributed/_tensor/test_math_ops 1/1 ... [2024-08-20 21:56:14.774274] 2024-08-20T21:56:14.7750877Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:56:14.7752815Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_math_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:56:14.774714] 2024-08-20T21:56:52.2742622Z 2024-08-20T21:56:52.2744578Z distributed/_tensor/test_math_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_math_ops_1.1_9478f74093b3b373_.log 2024-08-20T21:56:52.2750619Z Running 11 items in this shard: test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_foreach_norm, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_layer_norm_bwd, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_layer_norm_fwd, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_linear_op_reductions, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_mean, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_nll_loss_and_cross_entropy, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_shard0_svd, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_shard_math_ops, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_softmax_fwd, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_softmax_with_bwd, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_topk 2024-08-20T21:56:52.2755569Z 2024-08-20T21:56:52.2756106Z Running distributed/tensor/parallel/test_tp_examples 1/1 ... [2024-08-20 21:56:52.274716] 2024-08-20T21:56:52.2756816Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:56:52.2758919Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_tp_examples.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:56:52.275183] 2024-08-20T21:57:23.0592353Z 2024-08-20T21:57:23.0594587Z distributed/tensor/parallel/test_tp_examples 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_tp_examples_1.1_a45cb9773efe350f_.log 2024-08-20T21:57:23.0604063Z Running 9 items in this shard: test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_loss_parallel, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_mlp_inference, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_mlp_training_is_seq_parallel_False_recompute_activation_False, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_mlp_training_is_seq_parallel_True_recompute_activation_False, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_False_float32, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_False_float64, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_True_float32, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_True_float64, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_weight_tying 2024-08-20T21:57:23.0611407Z 2024-08-20T21:57:23.0611987Z Running distributed/optim/test_apply_optimizer_in_backward 1/1 ... [2024-08-20 21:57:23.059734] 2024-08-20T21:57:23.0612717Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:57:23.0614601Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/optim/test_apply_optimizer_in_backward.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:57:23.060174] 2024-08-20T21:57:25.6371784Z 2024-08-20T21:57:25.6373951Z distributed/optim/test_apply_optimizer_in_backward 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.optim.test_apply_optimizer_in_backward_1.1_151ad3fa4d99e303_.log 2024-08-20T21:57:25.6375217Z 2024-08-20T21:57:25.6378299Z Running distributed/checkpoint/e2e/test_pipeline 1/1 ... [2024-08-20 21:57:25.637584] 2024-08-20T21:57:25.6379011Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:57:25.6383405Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/e2e/test_pipeline.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:57:25.638035] 2024-08-20T21:57:28.7137138Z 2024-08-20T21:57:28.7139151Z distributed/checkpoint/e2e/test_pipeline 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.e2e.test_pipeline_1.1_1bd62d1493d3fadc_.log 2024-08-20T21:57:28.7140891Z Running 1 items in this shard: test/distributed/checkpoint/e2e/test_pipeline.py::TestPipeline::test_pipeline 2024-08-20T21:57:28.7141573Z 2024-08-20T21:57:28.7143730Z Running distributed/_composable/fsdp/test_fully_shard_state 1/1 ... [2024-08-20 21:57:28.714158] 2024-08-20T21:57:28.7144469Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:57:28.7149139Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:57:28.714594] 2024-08-20T21:57:31.7438887Z 2024-08-20T21:57:31.7441910Z distributed/_composable/fsdp/test_fully_shard_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_state_1.1_2f3abab9f8e38169_.log 2024-08-20T21:57:31.7449385Z Running 5 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_state.py::TestFullyShardState::test_fully_shard_cls, test/distributed/_composable/fsdp/test_fully_shard_state.py::TestFullyShardState::test_fully_shard_deepcopy, test/distributed/_composable/fsdp/test_fully_shard_state.py::TestFullyShardState::test_fully_shard_reapply, test/distributed/_composable/fsdp/test_fully_shard_state.py::TestFullyShardState::test_fully_shard_state, test/distributed/_composable/fsdp/test_fully_shard_state.py::TestFullyShardState::test_fully_shard_unsupported_module_cls 2024-08-20T21:57:31.7455229Z 2024-08-20T21:57:31.7456042Z Running distributed/test_collective_utils 1/1 ... [2024-08-20 21:57:31.744580] 2024-08-20T21:57:31.7456916Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:57:31.7459558Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_collective_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:57:31.745175] 2024-08-20T21:57:34.2281869Z 2024-08-20T21:57:34.2283585Z distributed/test_collective_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_collective_utils_1.1_000784736f4f6574_.log 2024-08-20T21:57:34.2284737Z 2024-08-20T21:57:34.2288034Z Running distributed/_composable/fully_shard/test_fully_shard_compile 1/1 ... [2024-08-20 21:57:34.228598] 2024-08-20T21:57:34.2288805Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:57:34.2293475Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_compile.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:57:34.229053] 2024-08-20T21:57:36.9038882Z 2024-08-20T21:57:36.9041185Z distributed/_composable/fully_shard/test_fully_shard_compile 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_compile_1.1_88971626bfb36fbf_.log 2024-08-20T21:57:36.9043138Z Running 1 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_compile.py::TestCompile::test_compile 2024-08-20T21:57:36.9044086Z 2024-08-20T21:57:36.9045136Z Running distributed/checkpoint/test_state_dict_utils 1/1 ... [2024-08-20 21:57:36.904288] 2024-08-20T21:57:36.9045837Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:57:36.9050255Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_state_dict_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:57:36.904737] 2024-08-20T21:57:39.8801025Z 2024-08-20T21:57:39.8802809Z distributed/checkpoint/test_state_dict_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_state_dict_utils_1.1_1801ef69048d924b_.log 2024-08-20T21:57:39.8807178Z Running 5 items in this shard: test/distributed/checkpoint/test_state_dict_utils.py::TestStateDictUtils::test_complicated_dict, test/distributed/checkpoint/test_state_dict_utils.py::TestStateDictUtils::test_cpu_and_ranks_only, test/distributed/checkpoint/test_state_dict_utils.py::TestStateDictUtils::test_create_cpu_state_dict, test/distributed/checkpoint/test_state_dict_utils.py::TestStateDictUtils::test_gather_state_dict_dtensor, test/distributed/checkpoint/test_state_dict_utils.py::TestStateDictUtils::test_gather_with_cpu_and_ranks_only 2024-08-20T21:57:39.8811159Z 2024-08-20T21:57:39.8811996Z Running distributed/pipelining/test_schedule 1/1 ... [2024-08-20 21:57:39.880550] 2024-08-20T21:57:39.8812663Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:57:39.8814470Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_schedule.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:57:39.880993] 2024-08-20T21:58:48.4002651Z 2024-08-20T21:58:48.4004629Z distributed/pipelining/test_schedule 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_schedule_1.1_e852c240c45bbc4f_.log 2024-08-20T21:58:48.4014433Z Running 13 items in this shard: test/distributed/pipelining/test_schedule.py::ScheduleTest::test_get_schedule_class, test/distributed/pipelining/test_schedule.py::TestSchedulePlan::test_pipeline_order_ScheduleClass0, test/distributed/pipelining/test_schedule.py::TestSchedulePlan::test_pipeline_order_ScheduleClass1, test/distributed/pipelining/test_schedule.py::TestSchedulePlan::test_pipeline_order_flex_and_zero_bubble_ScheduleClass0, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_action_parse_action_str_and_ref0, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_action_parse_action_str_and_ref1, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_action_parse_action_str_and_ref2, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_action_parse_action_str_and_ref3, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_action_parse_action_str_and_ref4, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_action_parse_action_str_and_ref5, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_action_parse_action_str_and_ref6, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_send_recv_test_info0, test/distributed/pipelining/test_schedule.py::TestScheduleLowering::test_unshard_reshard_test_info0 2024-08-20T21:58:48.4022238Z 2024-08-20T21:58:48.4022722Z Running distributed/_tensor/test_embedding_ops 1/1 ... [2024-08-20 21:58:48.400645] 2024-08-20T21:58:48.4023381Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:58:48.4025181Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_embedding_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:58:48.401119] 2024-08-20T21:59:04.0535787Z 2024-08-20T21:59:04.0537763Z distributed/_tensor/test_embedding_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_embedding_ops_1.1_b1e7ff9be3d32f48_.log 2024-08-20T21:59:04.0541362Z Running 4 items in this shard: test/distributed/_tensor/test_embedding_ops.py::TestEmbeddingOp::test_multiple_embeddings_rowwise, test/distributed/_tensor/test_embedding_ops.py::TestEmbeddingOp::test_sharded_embedding_colwise, test/distributed/_tensor/test_embedding_ops.py::TestEmbeddingOp::test_sharded_embedding_colwise_max_norm_errors, test/distributed/_tensor/test_embedding_ops.py::TestEmbeddingOp::test_sharded_embedding_rowwise 2024-08-20T21:59:04.0543985Z 2024-08-20T21:59:04.0544917Z Running distributed/checkpoint/test_utils 1/1 ... [2024-08-20 21:59:04.054233] 2024-08-20T21:59:04.0545589Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:59:04.0551429Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:59:04.054838] 2024-08-20T21:59:07.0806704Z 2024-08-20T21:59:07.0808972Z distributed/checkpoint/test_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_utils_1.1_b48de287102e45e0_.log 2024-08-20T21:59:07.0813360Z Running 6 items in this shard: test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_dcp_logger, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_flat_data, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_index_hint_ignored_on_equals, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_index_hint_ignored_on_hash, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_init_convert_offset, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_sharded_tensor_lookup 2024-08-20T21:59:07.0816619Z 2024-08-20T21:59:07.0817173Z Running distributed/_tensor/debug/test_comm_mode 1/1 ... [2024-08-20 21:59:07.081181] 2024-08-20T21:59:07.0817952Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:59:07.0819785Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/debug/test_comm_mode.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:59:07.081650] 2024-08-20T21:59:10.1071258Z 2024-08-20T21:59:10.1073725Z distributed/_tensor/debug/test_comm_mode 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.debug.test_comm_mode_1.1_c82556452f2dfdf9_.log 2024-08-20T21:59:10.1076805Z Running 4 items in this shard: test/distributed/_tensor/debug/test_comm_mode.py::TestCommMode::test_comm_mode, test/distributed/_tensor/debug/test_comm_mode.py::TestCommMode::test_comm_mode_coalesced, test/distributed/_tensor/debug/test_comm_mode.py::TestCommMode::test_comm_mode_with_c10d, test/distributed/_tensor/debug/test_comm_mode.py::TestCommMode::test_comm_mode_with_dtensor 2024-08-20T21:59:10.1079022Z 2024-08-20T21:59:10.1079559Z Running distributed/optim/test_zero_redundancy_optimizer 1/1 ... [2024-08-20 21:59:10.107679] 2024-08-20T21:59:10.1080286Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:59:10.1084923Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/optim/test_zero_redundancy_optimizer.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:59:10.108178] 2024-08-20T22:00:25.6992514Z 2024-08-20T22:00:25.6995673Z distributed/optim/test_zero_redundancy_optimizer 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.optim.test_zero_redundancy_optimizer_1.1_28eca574f53a18a6_.log 2024-08-20T22:00:25.7059573Z Running 42 items in this shard: test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_constructor, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_lr_scheduler, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_same_dense_param_type, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_state_dict, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_step_with_extra_inner_key, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_step_with_kwargs, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_step_without_closure, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_zero_grad, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_add_param_group, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_collect_shards, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_False_static_graph_False_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_False_static_graph_False_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_False_static_graph_True_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_False_static_graph_True_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_True_static_graph_False_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_True_static_graph_False_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_True_static_graph_True_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_True_static_graph_True_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_False_static_graph_False_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_False_static_graph_False_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_False_static_graph_True_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_False_static_graph_True_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_True_static_graph_False_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_True_static_graph_False_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_True_static_graph_True_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_True_static_graph_True_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_AdamW_maximize_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_AdamW_maximize_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_Adam_maximize_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_Adam_maximize_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_SGD_maximize_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_SGD_maximize_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_lr_scheduler, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_multiple_param_groups, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_nondefault_process_group, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_sharding, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_step, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_step_with_closure, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_zero_join_cpu, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_zero_join_gpu, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_zero_model_parallel_parameters_as_bucket_view_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_zero_model_parallel_parameters_as_bucket_view_True 2024-08-20T22:00:25.7098757Z 2024-08-20T22:00:25.7099389Z Running distributed/fsdp/test_shard_utils 1/1 ... [2024-08-20 22:00:25.700247] 2024-08-20T22:00:25.7100030Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:00:25.7101804Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_shard_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:00:25.700813] 2024-08-20T22:00:34.4877904Z 2024-08-20T22:00:34.4879875Z distributed/fsdp/test_shard_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_shard_utils_1.1_f1d68bcca60f03e3_.log 2024-08-20T22:00:34.4882210Z Running 2 items in this shard: test/distributed/fsdp/test_shard_utils.py::TestShardUtilsDistributed::test_create_chunk_sharded_tensor, test/distributed/fsdp/test_shard_utils.py::TestShardUtilsDistributedDTensor::test_create_chunk_dtensor 2024-08-20T22:00:34.4883848Z 2024-08-20T22:00:34.4884460Z Running distributed/checkpoint/test_fsdp_model_state 1/1 ... [2024-08-20 22:00:34.488217] 2024-08-20T22:00:34.4885153Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:00:34.4890308Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_fsdp_model_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:00:34.488685] 2024-08-20T22:00:43.5264668Z 2024-08-20T22:00:43.5266940Z distributed/checkpoint/test_fsdp_model_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_fsdp_model_state_1.1_32130ab8f1d2e433_.log 2024-08-20T22:00:43.5269803Z Running 2 items in this shard: test/distributed/checkpoint/test_fsdp_model_state.py::FsdpModelStateCheckpoint::test_fsdp_model_state_no_resharding, test/distributed/checkpoint/test_fsdp_model_state.py::FsdpModelStateCheckpoint::test_fsdp_model_state_with_resharding 2024-08-20T22:00:43.5271711Z 2024-08-20T22:00:43.5273215Z Running distributed/rpc/test_share_memory 1/1 ... [2024-08-20 22:00:43.527057] 2024-08-20T22:00:43.5273965Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:00:43.5280043Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_share_memory.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:00:43.527575] 2024-08-20T22:00:52.4880117Z 2024-08-20T22:00:52.4882250Z distributed/rpc/test_share_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_share_memory_1.1_53aba8f94c312608_.log 2024-08-20T22:00:52.4884152Z Running 1 items in this shard: test/distributed/rpc/test_share_memory.py::TestRPCPickler::test_case 2024-08-20T22:00:52.4885391Z Running 1 items in this shard: test/distributed/rpc/test_share_memory.py::TestRPCPickler::test_case 2024-08-20T22:00:52.4886499Z 2024-08-20T22:00:52.4887205Z Running distributed/checkpoint/test_checkpoint 1/1 ... [2024-08-20 22:00:52.488441] 2024-08-20T22:00:52.4887921Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:00:52.4892661Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:00:52.488928] 2024-08-20T22:01:01.7771564Z 2024-08-20T22:01:01.7773341Z distributed/checkpoint/test_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_checkpoint_1.1_d9340295738cda5b_.log 2024-08-20T22:01:01.7779966Z Running 8 items in this shard: test/distributed/checkpoint/test_checkpoint.py::TestDistributedCheckpointing::test_default_metadata, test/distributed/checkpoint/test_checkpoint.py::TestDistributedCheckpointing::test_tensor_metadata_with_missing_rank_spec, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_dummy_reader_works, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_dummy_writer_works, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_load_error_handling, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_load_error_handling_no_dist, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_save_error_handling, test/distributed/checkpoint/test_checkpoint.py::TestDistributedFailure::test_save_error_handling_no_dist 2024-08-20T22:01:01.7785154Z 2024-08-20T22:01:01.7785514Z Running distributed/test_nccl 1/1 ... [2024-08-20 22:01:01.777649] 2024-08-20T22:01:01.7786074Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:01:01.7787777Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_nccl.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:01:01.778107] 2024-08-20T22:01:04.7917518Z 2024-08-20T22:01:04.7919399Z distributed/test_nccl 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_nccl_1.1_4d076f6693dbb7ce_.log 2024-08-20T22:01:04.7920460Z Running 0 items in this shard: 2024-08-20T22:01:04.7920712Z 2024-08-20T22:01:04.7924346Z Running distributed/_composable/fully_shard/test_fully_shard_mixed_precision 1/1 ... [2024-08-20 22:01:04.792158] 2024-08-20T22:01:04.7925158Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:01:04.7929080Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_mixed_precision.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:01:04.792579] 2024-08-20T22:01:10.5732218Z 2024-08-20T22:01:10.5735498Z distributed/_composable/fully_shard/test_fully_shard_mixed_precision 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_mixed_precision_1.1_c1ded751ac3d4eaa_.log 2024-08-20T22:01:10.5739614Z Running 1 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_mixed_precision.py::TestMixedPrecision::test_float16_cast_forward 2024-08-20T22:01:10.5741117Z 2024-08-20T22:01:10.5742246Z Running distributed/_composable/fully_shard/test_fully_shard_util 1/1 ... [2024-08-20 22:01:10.573406] 2024-08-20T22:01:10.5743619Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:01:10.5746329Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_util.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:01:10.573994] 2024-08-20T22:01:16.4059077Z 2024-08-20T22:01:16.4061159Z distributed/_composable/fully_shard/test_fully_shard_util 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_util_1.1_50a149cff676e4a7_.log 2024-08-20T22:01:16.4063308Z Running 1 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_util.py::TestUtils::test_get_sharded_module_tree_with_module_name_to_fqns 2024-08-20T22:01:16.4064526Z 2024-08-20T22:01:16.4065459Z Running distributed/_composable/test_replicate 1/1 ... [2024-08-20 22:01:16.406293] 2024-08-20T22:01:16.4066149Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:01:16.4070311Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_replicate.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:01:16.406727] 2024-08-20T22:02:07.4111182Z 2024-08-20T22:02:07.4113864Z distributed/_composable/test_replicate 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_replicate_1.1_0e3eb8cfe148a166_.log 2024-08-20T22:02:07.4131195Z Running 17 items in this shard: test/distributed/_composable/test_replicate.py::ReplicateStateDictTest::test_replicate_non_root_multiple_save_load, test/distributed/_composable/test_replicate.py::ReplicateStateDictTest::test_replicate_single_module_save_load, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_device_id, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_ignore_module, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_move_args_kwargs_to_device, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_multi_module, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_single_module, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_with_kwargs, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_wrong_device_id_type, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_device_id, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_fully_shard_init, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_ignore_module, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_move_args_kwargs_to_device, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_multi_module, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_single_module, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_with_kwargs, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_wrong_device_id_type 2024-08-20T22:02:07.4145201Z 2024-08-20T22:02:07.4146028Z Running distributed/checkpoint/test_state_dict 1/1 ... [2024-08-20 22:02:07.411818] 2024-08-20T22:02:07.4147030Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:02:07.4150152Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:02:07.412400] 2024-08-20T22:02:13.9463874Z 2024-08-20T22:02:13.9465676Z distributed/checkpoint/test_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_state_dict_1.1_f5c68d98949b276c_.log 2024-08-20T22:02:13.9479384Z Running 25 items in this shard: test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_activation_ckpt_fqns_ddp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_activation_ckpt_fqns_fsdp1, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_broadcast_from_rank0, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_broadcast_from_rank0_hsdp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_compiled_fsdp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_cpu_offload_full_state_dict, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_ddp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_deprecate_fsdp_api, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_deprecate_partial, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_extra_state, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_flattened_osd, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_frozen_parameters, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_fsdp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_fsdp2, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_fsdp_ddp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_fsdp_root_not_initialized, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_non_persistent_buffers, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_optim_state_dict_param_matching, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_optim_state_dict_tensor_matching, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_setting_meta_device_model, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_setting_meta_device_model_broadcasting, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_shared_weight, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_single_gpu, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_strict, test/distributed/checkpoint/test_state_dict.py::TestNoComm::test_no_dist 2024-08-20T22:02:13.9492036Z 2024-08-20T22:02:13.9492579Z Running distributed/fsdp/test_fsdp_dtensor_state_dict 1/1 ... [2024-08-20 22:02:13.946886] 2024-08-20T22:02:13.9493255Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:02:13.9495340Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_dtensor_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:02:13.947341] 2024-08-20T22:03:00.8669207Z 2024-08-20T22:03:00.8671792Z distributed/fsdp/test_fsdp_dtensor_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_dtensor_state_dict_1.1_c3616971e6df82dd_.log 2024-08-20T22:03:00.8690795Z Running 15 items in this shard: test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_False_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_False_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_True_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_True_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_False_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_False_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_True_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_True_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_False_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_False_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_True_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_True_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_fsdp_init_with_device_mesh_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_fsdp_init_with_device_mesh_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_raises_warning_or_errors 2024-08-20T22:03:00.8705037Z 2024-08-20T22:03:00.8705597Z Running distributed/test_c10d_logger 1/1 ... [2024-08-20 22:03:00.867667] 2024-08-20T22:03:00.8706190Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:03:00.8707940Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_logger.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:03:00.868208] 2024-08-20T22:03:12.1560474Z 2024-08-20T22:03:12.1564869Z distributed/test_c10d_logger 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_logger_1.1_9bd5c5ef8a508001_.log 2024-08-20T22:03:12.1567413Z Running 3 items in this shard: test/distributed/test_c10d_logger.py::C10dErrorLoggerTest::test_exception_logger, test/distributed/test_c10d_logger.py::C10dErrorLoggerTest::test_get_or_create_logger, test/distributed/test_c10d_logger.py::C10dErrorLoggerTest::test_time_logger 2024-08-20T22:03:12.1568985Z 2024-08-20T22:03:12.1569433Z Running distributed/fsdp/test_fsdp_state_dict 1/1 ... [2024-08-20 22:03:12.156507] 2024-08-20T22:03:12.1570155Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:03:12.1572693Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:03:12.156985] 2024-08-20T22:03:15.6835444Z 2024-08-20T22:03:15.6837263Z distributed/fsdp/test_fsdp_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_state_dict_1.1_fd27c415e0b06a5c_.log 2024-08-20T22:03:15.7017180Z Running 179 items in this shard: test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_keys_state_dict_type_local_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_keys_state_dict_type_sharded_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_keys_state_dict_type_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_full_state_dict_missing_unexpected_keys_cleaned, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_local_state_dict_with_empty_ranks, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_False_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_False_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_True_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_True_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_False_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_False_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_True_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_True_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_False_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_False_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_True_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_True_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_sharded_load_multi_backend_pg, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_shared_module_and_shared_parameter, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_False_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_False_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_True_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_True_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_False_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_False_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_True_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_True_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_rank0_offload_save_load_flow_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_rank0_offload_save_load_flow_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_save_load_flow_state_dict_type_local_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_save_load_flow_state_dict_type_sharded_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_save_load_flow_state_dict_type_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_skip_module_state_dict_type_local_state_dict_double_nest_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_skip_module_state_dict_type_sharded_state_dict_double_nest_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_skip_module_state_dict_type_state_dict_double_nest_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_type, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_sharded_state_dict_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_sharded_state_dict_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_state_dict_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_state_dict_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_shared_parameters_state_dict_type_local_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_shared_parameters_state_dict_type_sharded_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_shared_parameters_state_dict_type_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_torch_save_load, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_world_size_one, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_wrong_state_dict_config, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict4GPUs::test_local_state_dict_reshard 2024-08-20T22:03:15.7192669Z 2024-08-20T22:03:15.7193383Z Running distributed/_composable/fsdp/test_fully_shard_autograd 1/1 ... [2024-08-20 22:03:15.684593] 2024-08-20T22:03:15.7194316Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:03:15.7196241Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_autograd.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:03:15.685052] 2024-08-20T22:03:18.7608622Z 2024-08-20T22:03:18.7610773Z distributed/_composable/fsdp/test_fully_shard_autograd 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_autograd_1.1_556302e8ec7a2cfa_.log 2024-08-20T22:03:18.7616233Z Running 5 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_autograd.py::TestFullyShardAutograd::test_nontensor_activations, test/distributed/_composable/fsdp/test_fully_shard_autograd.py::TestFullyShardAutograd::test_unused_forward_module, test/distributed/_composable/fsdp/test_fully_shard_autograd.py::TestFullyShardAutograd::test_unused_forward_output, test/distributed/_composable/fsdp/test_fully_shard_autograd.py::TestFullyShardPostAccGradHookMultiThread::test_post_acc_grad_hook_runs, test/distributed/_composable/fsdp/test_fully_shard_autograd.py::TestFullyShardPostAccGradHookMultiProcess::test_post_acc_grad_hook_optim_parity 2024-08-20T22:03:18.7620234Z 2024-08-20T22:03:18.7620854Z Running distributed/checkpoint/test_file_system_checkpoint 1/1 ... [2024-08-20 22:03:18.761300] 2024-08-20T22:03:18.7621645Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:03:18.7623696Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_file_system_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:03:18.761733] 2024-08-20T22:03:21.8375989Z 2024-08-20T22:03:21.8377826Z distributed/checkpoint/test_file_system_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_file_system_checkpoint_1.1_89dfd5305deeb66c_.log 2024-08-20T22:03:21.8383892Z Running 6 items in this shard: test/distributed/checkpoint/test_file_system_checkpoint.py::TestDistributedStateDictSaveLoad::test_read_write_only_tensor, test/distributed/checkpoint/test_file_system_checkpoint.py::TestDistributedStateDictSaveLoadWithSharedTensor::test_read_write_shard_tensor, test/distributed/checkpoint/test_file_system_checkpoint.py::TestDistributedReshardOnLoad::test_load_rowwise_to_colwise, test/distributed/checkpoint/test_file_system_checkpoint.py::TestDistributedReshardOnLoad::test_load_with_different_shard_plan, test/distributed/checkpoint/test_file_system_checkpoint.py::TestDistributedReshardOnLoad::test_save_load_bytes, test/distributed/checkpoint/test_file_system_checkpoint.py::TestDistributedReshardOnLoad::test_switch_between_sharded_tensor_to_tensor 2024-08-20T22:03:21.8388447Z 2024-08-20T22:03:21.8389086Z Running distributed/_composable/test_composability/test_2d_composability 1/1 ... [2024-08-20 22:03:21.838037] 2024-08-20T22:03:21.8389880Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:03:21.8391851Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_composability/test_2d_composability.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:03:21.838465] 2024-08-20T22:04:03.7468768Z 2024-08-20T22:04:03.7471206Z distributed/_composable/test_composability/test_2d_composability 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_composability.test_2d_composability_1.1_5fd2014a4785eea2_.log 2024-08-20T22:04:03.7485933Z Running 17 items in this shard: test/distributed/_composable/test_composability/test_2d_composability.py::TestFullyShard2DTraining::test_2d_uneven_shard_raise_error, test/distributed/_composable/test_composability/test_2d_composability.py::TestFullyShard2DTraining::test_tp_with_fsdp_offloading, test/distributed/_composable/test_composability/test_2d_composability.py::TestFullyShard2DTraining::test_train_parity_2d_mlp, test/distributed/_composable/test_composability/test_2d_composability.py::TestFullyShard2DTraining::test_train_parity_2d_transformer_checkpoint_resume, test/distributed/_composable/test_composability/test_2d_composability.py::Test2dFSDP1ParallelIntegration::test_2d_ddp_integration_functionality, test/distributed/_composable/test_composability/test_2d_composability.py::TestNew2dParallelTraining::test_2d_e2e_training_default, test/distributed/_composable/test_composability/test_2d_composability.py::TestNew2dParallelTraining::test_2d_e2e_training_not_use_orig_params, test/distributed/_composable/test_composability/test_2d_composability.py::TestNew2dParallelTraining::test_2d_e2e_training_use_orig_params, test/distributed/_composable/test_composability/test_2d_composability.py::TestNew2dParallelTraining::test_2d_fsdp_state_enable_extension, test/distributed/_composable/test_composability/test_2d_composability.py::TestNew2dParallelTraining::test_raise_invalid_tp_composition, test/distributed/_composable/test_composability/test_2d_composability.py::TestNew2dParallelStateDict::test_2d_load_state_dict_is_even_sharded_model_False, test/distributed/_composable/test_composability/test_2d_composability.py::TestNew2dParallelStateDict::test_2d_load_state_dict_is_even_sharded_model_True, test/distributed/_composable/test_composability/test_2d_composability.py::TestNew2dParallelStateDict::test_2d_optim_state_dict_is_even_sharded_model_False, test/distributed/_composable/test_composability/test_2d_composability.py::TestNew2dParallelStateDict::test_2d_optim_state_dict_is_even_sharded_model_True, test/distributed/_composable/test_composability/test_2d_composability.py::TestNew2dParallelStateDict::test_2d_state_dict_is_even_sharded_model_False, test/distributed/_composable/test_composability/test_2d_composability.py::TestNew2dParallelStateDict::test_2d_state_dict_is_even_sharded_model_True, test/distributed/_composable/test_composability/test_2d_composability.py::TestNew2dParallelStateDict::test_fsdp_2d_extension 2024-08-20T22:04:03.7499391Z 2024-08-20T22:04:03.7499998Z Running distributed/_composable/fsdp/test_fully_shard_frozen 1/1 ... [2024-08-20 22:04:03.747356] 2024-08-20T22:04:03.7500714Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:04:03.7502604Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_frozen.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:04:03.747850] 2024-08-20T22:04:06.7736129Z 2024-08-20T22:04:06.7738386Z distributed/_composable/fsdp/test_fully_shard_frozen 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_frozen_1.1_23549e75305dbde1_.log 2024-08-20T22:04:06.7742381Z Running 3 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_frozen.py::TestFullyShardFrozen::test_multi_forward_mixed_requires_grad, test/distributed/_composable/fsdp/test_fully_shard_frozen.py::TestFullyShardFrozen::test_train_mixed_requires_grad_across_groups, test/distributed/_composable/fsdp/test_fully_shard_frozen.py::TestFullyShardFrozen::test_train_mixed_requires_grad_per_group 2024-08-20T22:04:06.7744797Z 2024-08-20T22:04:06.7745368Z Running distributed/fsdp/test_distributed_checkpoint 1/1 ... [2024-08-20 22:04:06.773999] 2024-08-20T22:04:06.7746408Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:04:06.7748349Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_distributed_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:04:06.774439] 2024-08-20T22:04:09.7503560Z 2024-08-20T22:04:09.7505596Z distributed/fsdp/test_distributed_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_distributed_checkpoint_1.1_86e29cfd31fefa77_.log 2024-08-20T22:04:09.7508328Z Running 2 items in this shard: test/distributed/fsdp/test_distributed_checkpoint.py::TestDistributedCheckpoint::test_distributed_checkpoint_state_dict_type0, test/distributed/fsdp/test_distributed_checkpoint.py::TestDistributedCheckpoint::test_distributed_checkpoint_state_dict_type1 2024-08-20T22:04:09.7510286Z 2024-08-20T22:04:09.7510885Z Running distributed/_composable/fsdp/test_fully_shard_mixed_precision 1/1 ... [2024-08-20 22:04:09.750805] 2024-08-20T22:04:09.7511660Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:04:09.7515629Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_mixed_precision.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:04:09.751246] 2024-08-20T22:04:12.7769973Z 2024-08-20T22:04:12.7772520Z distributed/_composable/fsdp/test_fully_shard_mixed_precision 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_mixed_precision_1.1_10cac1d7361e2625_.log 2024-08-20T22:04:12.7779468Z Running 7 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionTraining::test_compute_dtype, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionTraining::test_grad_acc_with_reduce_dtype, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionTraining::test_reduce_dtype, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_float16_on_one_submodule, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_norm_modules_bf16, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_norm_modules_fp16, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_submodules_with_external_inputs 2024-08-20T22:04:12.7785237Z 2024-08-20T22:04:12.7785727Z Running distributed/_tensor/test_utils 1/1 ... [2024-08-20 22:04:12.777477] 2024-08-20T22:04:12.7786401Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:04:12.7788262Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:04:12.777941] 2024-08-20T22:05:00.7557269Z 2024-08-20T22:05:00.7559139Z distributed/_tensor/test_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_utils_1.1_ec131e458f85b19d_.log 2024-08-20T22:05:00.7567030Z Running 11 items in this shard: test/distributed/_tensor/test_utils.py::UtilTest::test_compute_local_shape_2d_uneven, test/distributed/_tensor/test_utils.py::UtilTest::test_compute_local_shape_and_global_offset_1D, test/distributed/_tensor/test_utils.py::UtilTest::test_compute_local_shape_and_global_offset_2D, test/distributed/_tensor/test_utils.py::UtilTest::test_fsdp_tp_meta_compute, test/distributed/_tensor/test_utils.py::UtilTest::test_hsdp_tp_meta_compute, test/distributed/_tensor/test_utils.py::UtilTest::test_strided_sharding_assumption_in_meta_compute, test/distributed/_tensor/test_utils.py::TestStridedSharding::test_1d_mesh_strided_sharding, test/distributed/_tensor/test_utils.py::TestStridedSharding::test_2d_mesh_2d_tensor_strided_sharding, test/distributed/_tensor/test_utils.py::TestStridedSharding::test_2d_mesh_strided_sharding, test/distributed/_tensor/test_utils.py::Test2DStridedLocalShard::test_fsdp1_tp_2d_dtensor_local_shards_and_offsets, test/distributed/_tensor/test_utils.py::Test2DStridedLocalShard::test_fsdp2_tp_2d_dtensor_local_shards_and_offsets 2024-08-20T22:05:00.7572933Z 2024-08-20T22:05:00.7573405Z Running distributed/test_distributed_spawn 2/3 ... [2024-08-20 22:05:00.756275] 2024-08-20T22:05:00.7650772Z MPI not available -- MPI backend tests will be skipped 2024-08-20T22:05:00.7651900Z Running distributed tests for the test backend with env init_method 2024-08-20T22:05:00.7654661Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:05:00.7660752Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=2', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:05:00.765778] 2024-08-20T22:05:03.8225628Z 2024-08-20T22:05:03.8227801Z distributed/test_distributed_spawn 2/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_2.3_a93cc3a4eadbf990_.log 2024-08-20T22:05:03.8229137Z Running 0 items in this shard: 2024-08-20T22:05:03.8229387Z 2024-08-20T22:05:03.8239112Z Running distributed tests for the test backend with file init_method 2024-08-20T22:05:03.8240255Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:05:03.8245501Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=2', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:05:03.824244] 2024-08-20T22:05:06.7157986Z 2024-08-20T22:05:06.7159924Z distributed/test_distributed_spawn 2/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_2.3_3bc95e0ae5c78bc6_.log 2024-08-20T22:05:06.7161473Z Running 0 items in this shard: 2024-08-20T22:05:06.7161733Z 2024-08-20T22:05:06.7167381Z Running distributed tests for the gloo backend with env init_method 2024-08-20T22:05:06.7168334Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:05:06.7173498Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=2', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:05:06.717056] 2024-08-20T22:13:41.0770297Z 2024-08-20T22:13:41.0771968Z distributed/test_distributed_spawn 2/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_2.3_f9b55a3a7cbb0a65_.log 2024-08-20T22:13:41.0862608Z Running 92 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Single_Input_Per_Process, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_half, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedSampler_padding, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_SyncBatchNorm_process_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_with_then_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_simple, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_with_empty, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_v_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_complex_unsupported_ops, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_global, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_ignored_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce_return_future, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_comm_hook_logging, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device_mesh_initialization, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_grad_div_uneven_inputs, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_has_finalized, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_powerSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_inference, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_join_model_equivalence, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_err_ignore_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_no_set_grad_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_set_grad_to_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sink_noclone, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_bn_training_vs_eval, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_exception, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_unused_params_rebuild_buckets_exception, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_zero_output_features, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_detect_ddp_is_actually_static, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_dump_DDP_relevant_env_vars, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_backend, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_grads_same_across_ranks_with_no_sync, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang_wait_all_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_rank_0_timeout, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_broadcast, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_input_rank_exceeds_world_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_overlap_not_allowed, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_dict_module, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_step_reload, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_v_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_twice, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_stateless_api_with_ddp, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sync_bn_logged 2024-08-20T22:13:41.0921066Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel 2024-08-20T22:13:41.0922598Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm 2024-08-20T22:13:41.0924352Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Single_Input_Per_Process 2024-08-20T22:13:41.0926064Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_half 2024-08-20T22:13:41.0927580Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedSampler_padding 2024-08-20T22:13:41.0929026Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_SyncBatchNorm_process_group 2024-08-20T22:13:41.0930881Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_with_then_hook 2024-08-20T22:13:41.0932449Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_full_group 2024-08-20T22:13:41.0933905Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_simple 2024-08-20T22:13:41.0935347Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_with_empty 2024-08-20T22:13:41.0936740Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_complex 2024-08-20T22:13:41.0938187Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda_complex 2024-08-20T22:13:41.0939563Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_v_cuda 2024-08-20T22:13:41.0940944Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_product 2024-08-20T22:13:41.0942409Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_complex_unsupported_ops 2024-08-20T22:13:41.0960370Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_sum 2024-08-20T22:13:41.0962039Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_min 2024-08-20T22:13:41.0963485Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_complex 2024-08-20T22:13:41.0964848Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda 2024-08-20T22:13:41.0966174Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all 2024-08-20T22:13:41.0967474Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_complex 2024-08-20T22:13:41.0968767Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda 2024-08-20T22:13:41.0970284Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_complex 2024-08-20T22:13:41.0971807Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda 2024-08-20T22:13:41.0973296Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split 2024-08-20T22:13:41.0974804Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_complex 2024-08-20T22:13:41.0976378Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda_complex 2024-08-20T22:13:41.0977989Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group_cuda 2024-08-20T22:13:41.0979556Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group 2024-08-20T22:13:41.0980947Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_group 2024-08-20T22:13:41.0982215Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group 2024-08-20T22:13:41.0983704Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_global 2024-08-20T22:13:41.0985079Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_nccl 2024-08-20T22:13:41.0986448Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_err 2024-08-20T22:13:41.0987785Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_cuda 2024-08-20T22:13:41.0989081Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_group 2024-08-20T22:13:41.0990560Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward 2024-08-20T22:13:41.0992060Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_ignored_params 2024-08-20T22:13:41.0993528Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer 2024-08-20T22:13:41.0995338Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce_return_future 2024-08-20T22:13:41.0996801Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_comm_hook_logging 2024-08-20T22:13:41.0998215Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device_mesh_initialization 2024-08-20T22:13:41.0999677Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_grad_div_uneven_inputs 2024-08-20T22:13:41.1001048Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_has_finalized 2024-08-20T22:13:41.1002399Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce 2024-08-20T22:13:41.1004140Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_powerSGD 2024-08-20T22:13:41.1005485Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_inference 2024-08-20T22:13:41.1006842Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_join_model_equivalence 2024-08-20T22:13:41.1008373Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_err_ignore_params 2024-08-20T22:13:41.1010174Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_no_set_grad_none 2024-08-20T22:13:41.1011966Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_set_grad_to_none 2024-08-20T22:13:41.1013603Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_autograd_profiler 2024-08-20T22:13:41.1015042Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_torch_profiler 2024-08-20T22:13:41.1016412Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sink_noclone 2024-08-20T22:13:41.1017784Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_bn_training_vs_eval 2024-08-20T22:13:41.1019212Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_exception 2024-08-20T22:13:41.1020922Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_unused_params_rebuild_buckets_exception 2024-08-20T22:13:41.1022420Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_zero_output_features 2024-08-20T22:13:41.1023797Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_full_group 2024-08-20T22:13:41.1025116Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_group 2024-08-20T22:13:41.1026470Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_detect_ddp_is_actually_static 2024-08-20T22:13:41.1028010Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_dump_DDP_relevant_env_vars 2024-08-20T22:13:41.1029360Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_cuda 2024-08-20T22:13:41.1030688Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object_subgroup 2024-08-20T22:13:41.1031990Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_backend 2024-08-20T22:13:41.1033334Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_full_group 2024-08-20T22:13:41.1034787Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_grads_same_across_ranks_with_no_sync 2024-08-20T22:13:41.1036154Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend 2024-08-20T22:13:41.1037427Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_torch_profiler 2024-08-20T22:13:41.1038921Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang_wait_all_ranks 2024-08-20T22:13:41.1040495Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_rank_0_timeout 2024-08-20T22:13:41.1041956Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_broadcast 2024-08-20T22:13:41.1043389Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration 2024-08-20T22:13:41.1044977Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_input_rank_exceeds_world_size 2024-08-20T22:13:41.1046592Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_overlap_not_allowed 2024-08-20T22:13:41.1048075Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_dict_module 2024-08-20T22:13:41.1049520Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager 2024-08-20T22:13:41.1051129Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd 2024-08-20T22:13:41.1052725Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_step_reload 2024-08-20T22:13:41.1054161Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_max 2024-08-20T22:13:41.1055528Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_product 2024-08-20T22:13:41.1056935Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_sum 2024-08-20T22:13:41.1058213Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_max 2024-08-20T22:13:41.1059511Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_v_cuda 2024-08-20T22:13:41.1060853Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_twice 2024-08-20T22:13:41.1062155Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_group 2024-08-20T22:13:41.1063573Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source 2024-08-20T22:13:41.1064985Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_torch_profiler 2024-08-20T22:13:41.1066457Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_autograd_profiler 2024-08-20T22:13:41.1067855Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum 2024-08-20T22:13:41.1069233Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_stateless_api_with_ddp 2024-08-20T22:13:41.1070557Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sync_bn_logged 2024-08-20T22:13:41.1071295Z 2024-08-20T22:13:41.1071586Z Running distributed tests for the gloo backend with file init_method 2024-08-20T22:13:41.1072180Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:13:41.1074200Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=2', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:13:41.082714] 2024-08-20T22:22:15.7171612Z 2024-08-20T22:22:15.7174355Z distributed/test_distributed_spawn 2/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_2.3_9a430e8595b1e6a5_.log 2024-08-20T22:22:15.7243410Z Running 92 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Single_Input_Per_Process, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_half, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedSampler_padding, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_SyncBatchNorm_process_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_with_then_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_simple, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_with_empty, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_v_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_complex_unsupported_ops, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_global, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_ignored_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce_return_future, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_comm_hook_logging, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device_mesh_initialization, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_grad_div_uneven_inputs, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_has_finalized, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_powerSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_inference, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_join_model_equivalence, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_err_ignore_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_no_set_grad_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_set_grad_to_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sink_noclone, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_bn_training_vs_eval, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_exception, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_unused_params_rebuild_buckets_exception, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_zero_output_features, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_detect_ddp_is_actually_static, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_dump_DDP_relevant_env_vars, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_backend, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_grads_same_across_ranks_with_no_sync, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang_wait_all_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_rank_0_timeout, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_broadcast, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_input_rank_exceeds_world_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_overlap_not_allowed, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_dict_module, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_step_reload, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_v_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_twice, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_stateless_api_with_ddp, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sync_bn_logged 2024-08-20T22:22:15.7295441Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel 2024-08-20T22:22:15.7297063Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm 2024-08-20T22:22:15.7298759Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Single_Input_Per_Process 2024-08-20T22:22:15.7300463Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_half 2024-08-20T22:22:15.7301989Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedSampler_padding 2024-08-20T22:22:15.7303502Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_SyncBatchNorm_process_group 2024-08-20T22:22:15.7305080Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_with_then_hook 2024-08-20T22:22:15.7306650Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_full_group 2024-08-20T22:22:15.7308096Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_simple 2024-08-20T22:22:15.7309547Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_with_empty 2024-08-20T22:22:15.7310952Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_complex 2024-08-20T22:22:15.7312307Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda_complex 2024-08-20T22:22:15.7313667Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_v_cuda 2024-08-20T22:22:15.7315045Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_product 2024-08-20T22:22:15.7316572Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_complex_unsupported_ops 2024-08-20T22:22:15.7318013Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_sum 2024-08-20T22:22:15.7319393Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_min 2024-08-20T22:22:15.7320763Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_complex 2024-08-20T22:22:15.7322126Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda 2024-08-20T22:22:15.7323412Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all 2024-08-20T22:22:15.7324719Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_complex 2024-08-20T22:22:15.7326031Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda 2024-08-20T22:22:15.7327442Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_complex 2024-08-20T22:22:15.7328945Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda 2024-08-20T22:22:15.7330502Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split 2024-08-20T22:22:15.7332014Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_complex 2024-08-20T22:22:15.7333640Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda_complex 2024-08-20T22:22:15.7335236Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group_cuda 2024-08-20T22:22:15.7336811Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group 2024-08-20T22:22:15.7338214Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_group 2024-08-20T22:22:15.7339552Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group 2024-08-20T22:22:15.7340875Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_global 2024-08-20T22:22:15.7342248Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_nccl 2024-08-20T22:22:15.7343632Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_err 2024-08-20T22:22:15.7344959Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_cuda 2024-08-20T22:22:15.7346254Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_group 2024-08-20T22:22:15.7347610Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward 2024-08-20T22:22:15.7349117Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_ignored_params 2024-08-20T22:22:15.7350579Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer 2024-08-20T22:22:15.7352038Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce_return_future 2024-08-20T22:22:15.7353493Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_comm_hook_logging 2024-08-20T22:22:15.7354905Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device_mesh_initialization 2024-08-20T22:22:15.7356341Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_grad_div_uneven_inputs 2024-08-20T22:22:15.7357711Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_has_finalized 2024-08-20T22:22:15.7359095Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce 2024-08-20T22:22:15.7360492Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_powerSGD 2024-08-20T22:22:15.7361815Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_inference 2024-08-20T22:22:15.7363171Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_join_model_equivalence 2024-08-20T22:22:15.7364719Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_err_ignore_params 2024-08-20T22:22:15.7366453Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_no_set_grad_none 2024-08-20T22:22:15.7368294Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_set_grad_to_none 2024-08-20T22:22:15.7369995Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_autograd_profiler 2024-08-20T22:22:15.7371467Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_torch_profiler 2024-08-20T22:22:15.7372844Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sink_noclone 2024-08-20T22:22:15.7374212Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_bn_training_vs_eval 2024-08-20T22:22:15.7375701Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_exception 2024-08-20T22:22:15.7377211Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_unused_params_rebuild_buckets_exception 2024-08-20T22:22:15.7378695Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_zero_output_features 2024-08-20T22:22:15.7380062Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_full_group 2024-08-20T22:22:15.7381370Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_group 2024-08-20T22:22:15.7382733Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_detect_ddp_is_actually_static 2024-08-20T22:22:15.7384147Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_dump_DDP_relevant_env_vars 2024-08-20T22:22:15.7385496Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_cuda 2024-08-20T22:22:15.7386818Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object_subgroup 2024-08-20T22:22:15.7388135Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_backend 2024-08-20T22:22:15.7389449Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_full_group 2024-08-20T22:22:15.7390901Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_grads_same_across_ranks_with_no_sync 2024-08-20T22:22:15.7392258Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend 2024-08-20T22:22:15.7393543Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_torch_profiler 2024-08-20T22:22:15.7395204Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang_wait_all_ranks 2024-08-20T22:22:15.7396789Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_rank_0_timeout 2024-08-20T22:22:15.7398267Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_broadcast 2024-08-20T22:22:15.7399712Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration 2024-08-20T22:22:15.7401287Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_input_rank_exceeds_world_size 2024-08-20T22:22:15.7402908Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_overlap_not_allowed 2024-08-20T22:22:15.7404515Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_dict_module 2024-08-20T22:22:15.7405963Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager 2024-08-20T22:22:15.7407491Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd 2024-08-20T22:22:15.7409093Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_step_reload 2024-08-20T22:22:15.7410612Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_max 2024-08-20T22:22:15.7412120Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_product 2024-08-20T22:22:15.7413446Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_sum 2024-08-20T22:22:15.7414738Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_max 2024-08-20T22:22:15.7416045Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_v_cuda 2024-08-20T22:22:15.7417380Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_twice 2024-08-20T22:22:15.7418677Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_group 2024-08-20T22:22:15.7419998Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source 2024-08-20T22:22:15.7421432Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_torch_profiler 2024-08-20T22:22:15.7422884Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_autograd_profiler 2024-08-20T22:22:15.7424278Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum 2024-08-20T22:22:15.7425651Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_stateless_api_with_ddp 2024-08-20T22:22:15.7426988Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sync_bn_logged 2024-08-20T22:22:15.7427705Z 2024-08-20T22:22:15.7428480Z Running distributed/_composable/fully_shard/test_fully_shard_model_checkpoint 1/1 ... [2024-08-20 22:22:15.721954] 2024-08-20T22:22:15.7429293Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:22:15.7431278Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_model_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:22:15.722341] 2024-08-20T22:22:29.8676389Z 2024-08-20T22:22:29.8678985Z distributed/_composable/fully_shard/test_fully_shard_model_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_model_checkpoint_1.1_60404ed95c02037e_.log 2024-08-20T22:22:29.8684938Z Running 4 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_model_checkpoint.py::TestModelCheckpointing::test_full_state_dict_save_load_mixed_sharding, test/distributed/_composable/fully_shard/test_fully_shard_model_checkpoint.py::TestModelCheckpointing::test_state_dict_save_load_flow, test/distributed/_composable/fully_shard/test_fully_shard_model_checkpoint.py::TestModelCheckpointing::test_state_dict_save_load_root_fully_shard, test/distributed/_composable/fully_shard/test_fully_shard_model_checkpoint.py::TestModelCheckpointing::test_state_dict_save_load_submodule_fully_shard 2024-08-20T22:22:29.8688187Z 2024-08-20T22:22:29.8688673Z Running distributed/test_c10d_object_collectives 1/1 ... [2024-08-20 22:22:29.867747] 2024-08-20T22:22:29.8689335Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:22:29.8691263Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_object_collectives.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:22:29.868245] 2024-08-20T22:22:58.2350328Z 2024-08-20T22:22:58.2352257Z distributed/test_c10d_object_collectives 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_object_collectives_1.1_1b9b51bf717fe7dd_.log 2024-08-20T22:22:58.2358999Z Running 9 items in this shard: test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_all_gather_object, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_broadcast_object_list, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_gather_object, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_scatter_object_list, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_send_recv_object_list, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_subpg_all_gather_object, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_subpg_broadcast_object, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_subpg_gather_object, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_subpg_scatter_object 2024-08-20T22:22:58.2364073Z 2024-08-20T22:22:58.2364663Z Running distributed/_composable/fully_shard/test_fully_shard_init 1/1 ... [2024-08-20 22:22:58.235471] 2024-08-20T22:22:58.2365416Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:22:58.2367324Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_init.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:22:58.235943] 2024-08-20T22:23:21.0419140Z 2024-08-20T22:23:21.0422262Z distributed/_composable/fully_shard/test_fully_shard_init 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_init_1.1_77ee4b05cf72dc6a_.log 2024-08-20T22:23:21.0431699Z Running 7 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_device_id, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_manual_fully_shard, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_materialize_meta_module, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_nested_fully_shard_shared_state, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_policy, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_raise_scalar_parameter, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_sync_module_states 2024-08-20T22:23:21.0438598Z 2024-08-20T22:23:21.0439509Z Running distributed/fsdp/test_fsdp_tp_integration 1/1 ... [2024-08-20 22:23:21.042513] 2024-08-20T22:23:21.0440635Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:23:21.0443310Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_tp_integration.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:23:21.043110] 2024-08-20T22:23:33.1388691Z 2024-08-20T22:23:33.1391913Z distributed/fsdp/test_fsdp_tp_integration 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_tp_integration_1.1_6c62be84c47ce187_.log 2024-08-20T22:23:33.1397247Z Running 3 items in this shard: test/distributed/fsdp/test_fsdp_tp_integration.py::TestTPFSDPIntegration::test_fsdp_tp_extension_grad, test/distributed/fsdp/test_fsdp_tp_integration.py::TestTPFSDPIntegration::test_fsdp_tp_integration, test/distributed/fsdp/test_fsdp_tp_integration.py::TestTPFSDPIntegration::test_fsdp_tp_sync_module_state 2024-08-20T22:23:33.1400362Z 2024-08-20T22:23:33.1401196Z Running distributed/fsdp/test_fsdp_grad_acc 1/1 ... [2024-08-20 22:23:33.139499] 2024-08-20T22:23:33.1402487Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:23:33.1405552Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_grad_acc.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:23:33.140101] 2024-08-20T22:23:52.7435472Z 2024-08-20T22:23:52.7438154Z distributed/fsdp/test_fsdp_grad_acc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_grad_acc_1.1_0edaddb08a08094e_.log 2024-08-20T22:23:52.7442584Z Running 6 items in this shard: test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_configs0_use_orig_params_False, test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_configs0_use_orig_params_True, test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_configs1_use_orig_params_False, test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_configs1_use_orig_params_True, test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_cpu_offload_use_orig_params_False, test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_cpu_offload_use_orig_params_True 2024-08-20T22:23:52.7446903Z 2024-08-20T22:23:52.7447415Z Running distributed/fsdp/test_fsdp_freezing_weights 1/1 ... [2024-08-20 22:23:52.744270] 2024-08-20T22:23:52.7448103Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:23:52.7453162Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_freezing_weights.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:23:52.744948] 2024-08-20T22:25:31.3741317Z 2024-08-20T22:25:31.3744125Z distributed/fsdp/test_fsdp_freezing_weights 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_freezing_weights_1.1_0c05d99add36a5cd_.log 2024-08-20T22:25:31.3806326Z Running 32 items in this shard: test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_True 2024-08-20T22:25:31.3846522Z 2024-08-20T22:25:31.3847118Z Running distributed/test_multi_threaded_pg 1/1 ... [2024-08-20 22:25:31.375065] 2024-08-20T22:25:31.3847772Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:25:31.3849533Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_multi_threaded_pg.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:25:31.375659] 2024-08-20T22:25:34.5031012Z 2024-08-20T22:25:34.5032785Z distributed/test_multi_threaded_pg 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_multi_threaded_pg_1.1_7bc2092b22dc7d99_.log 2024-08-20T22:25:34.5046065Z Running 22 items in this shard: test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_all_to_all_single_list, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_all_to_all_single_none, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_all_to_all_single_tensor, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_broadcast_object_list, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_collective_error_on_rank_non_zero, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_collective_error_on_rank_non_zero_all, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_collective_error_on_rank_zero, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_skip, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_reduce, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_reduce_coalesced, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_reduce_ops, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_to_all, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_allgather, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_assert_equal_on_rank, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_broadcast, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_broadcast_object_list, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_bwd_sees_fwd_pg, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_gather, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_reduce_scatter, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_scatter, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_subpg, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_using_pg_from_another_thread 2024-08-20T22:25:34.5058117Z 2024-08-20T22:25:34.5058722Z Running distributed/algorithms/ddp_comm_hooks/test_ddp_hooks 1/1 ... [2024-08-20 22:25:34.503624] 2024-08-20T22:25:34.5059447Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:25:34.5061351Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:25:34.504072] 2024-08-20T22:25:37.2794317Z 2024-08-20T22:25:37.2796819Z distributed/algorithms/ddp_comm_hooks/test_ddp_hooks 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.algorithms.ddp_comm_hooks.test_ddp_hooks_1.1_0ec5ab8740d987a7_.log 2024-08-20T22:25:37.2803105Z Running 6 items in this shard: test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_ddp_comm_hook_allreduce_hook, test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_ddp_comm_hook_fp16compress_hook, test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_ddp_comm_hook_noop_hook, test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_ddp_comm_hook_quantize_per_channel_hook, test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_ddp_comm_hook_quantize_per_tensor_hook, test/distributed/algorithms/ddp_comm_hooks/test_ddp_hooks.py::DistributedDataParallelCommHookTest::test_is_last_hook 2024-08-20T22:25:37.2808276Z 2024-08-20T22:25:37.2808771Z Running distributed/fsdp/test_fsdp_flatten_params 1/1 ... [2024-08-20 22:25:37.279891] 2024-08-20T22:25:37.2809435Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:25:37.2811701Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_flatten_params.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:25:37.280354] 2024-08-20T22:26:09.4003132Z 2024-08-20T22:26:09.4005923Z distributed/fsdp/test_fsdp_flatten_params 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_flatten_params_1.1_ab393392fff544d2_.log 2024-08-20T22:26:09.4019059Z Running 11 items in this shard: test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_empty_module, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_flat_param_shard_metadata_aligned_full_precision, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_flat_param_shard_metadata_aligned_mixed_precision, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_flat_param_shard_metadata_unaligned, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_flatten_nothing, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_numel_with_shared_params, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_numel_without_shared_params, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_output_with_shared_params, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_output_without_shared_params, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_partial_flattening, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_pnorm_after_step_with_shared_params 2024-08-20T22:26:09.4029430Z 2024-08-20T22:26:09.4030335Z Running distributed/checkpoint/test_file_system_checkpoint_cpu 1/1 ... [2024-08-20 22:26:09.401064] 2024-08-20T22:26:09.4031385Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:26:09.4034025Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_file_system_checkpoint_cpu.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:26:09.401672] 2024-08-20T22:26:41.1227299Z 2024-08-20T22:26:41.1230326Z distributed/checkpoint/test_file_system_checkpoint_cpu 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_file_system_checkpoint_cpu_1.1_efa8529bd5dcfd21_.log 2024-08-20T22:26:41.1248843Z Running 12 items in this shard: test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoad::test_read_write_only_tensor_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoad::test_read_write_only_tensor_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoadWithSharedTensor::test_read_write_shard_tensor_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoadWithSharedTensor::test_read_write_shard_tensor_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_rowwise_to_colwise_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_rowwise_to_colwise_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_with_different_shard_plan_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_with_different_shard_plan_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_save_load_bytes_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_save_load_bytes_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_switch_between_sharded_tensor_to_tensor_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_switch_between_sharded_tensor_to_tensor_thread_count_2 2024-08-20T22:26:41.1263383Z 2024-08-20T22:26:41.1264067Z Running distributed/test_c10d_pypg 1/1 ... [2024-08-20 22:26:41.123713] 2024-08-20T22:26:41.1264901Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:26:41.1267476Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_pypg.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:26:41.124296] 2024-08-20T22:28:35.9181805Z 2024-08-20T22:28:35.9184542Z distributed/test_c10d_pypg 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_pypg_1.1_70e5a95c9be17b9a_.log 2024-08-20T22:28:35.9212251Z Running 42 items in this shard: test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_dataclass_output, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_dataclass_output_unused_param, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_dynamic_module, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_dynamic_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_once_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_once_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_static_graph_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_static_graph_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_unused_params_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_unused_params_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_weight_sharing_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_weight_sharing_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_invoke_work_object, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_with_pypg, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_with_pypg_with_grad_views, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_invalid_powerSGD_state, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_sync_batch_norm_empty_input, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_sync_batch_norm_only_empty_input, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_dataclass_output, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_dataclass_output_unused_param, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_dynamic_module, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_dynamic_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_once_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_once_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_static_graph_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_static_graph_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_unused_params_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_unused_params_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_weight_sharing_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_weight_sharing_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_invoke_work_object, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_with_pypg, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_with_pypg_with_grad_views, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_invalid_powerSGD_state, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_sync_batch_norm_empty_input, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_sync_batch_norm_only_empty_input 2024-08-20T22:28:35.9236370Z 2024-08-20T22:28:35.9237037Z Running distributed/_composable/fsdp/test_fully_shard_extensions 1/1 ... [2024-08-20 22:28:35.918260] 2024-08-20T22:28:35.9237791Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:28:35.9239692Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_extensions.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:28:35.918836] 2024-08-20T22:28:42.2000963Z 2024-08-20T22:28:42.2003227Z distributed/_composable/fsdp/test_fully_shard_extensions 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_extensions_1.1_8de963b98b3ca90d_.log 2024-08-20T22:28:42.2007308Z Running 3 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_extensions.py::TestFullyShardAllGatherExtensionsMultiProcess::test_all_gather_extensions_train_parity, test/distributed/_composable/fsdp/test_fully_shard_extensions.py::TestFullyShardAllGatherExtensionsMultiThread::test_all_gather_extensions_end_to_end, test/distributed/_composable/fsdp/test_fully_shard_extensions.py::TestFullyShardAllGatherExtensionsMultiThread::test_all_gather_extensions_monkey_patch 2024-08-20T22:28:42.2010551Z 2024-08-20T22:28:42.2011003Z Running distributed/fsdp/test_fsdp_memory 1/1 ... [2024-08-20 22:28:42.200566] 2024-08-20T22:28:42.2011841Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:28:42.2015333Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_memory.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:28:42.201084] 2024-08-20T22:28:50.9360129Z 2024-08-20T22:28:50.9361866Z distributed/fsdp/test_fsdp_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_memory_1.1_089ba420ec3005d5_.log 2024-08-20T22:28:50.9364026Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_memory.py::TestFSDPMemory::test_fsdp_memory_ckpt_ckpt, test/distributed/fsdp/test_fsdp_memory.py::TestFSDPMemory::test_fsdp_memory_ckpt_no_ckpt 2024-08-20T22:28:50.9365379Z 2024-08-20T22:28:50.9367190Z Running distributed/fsdp/test_hsdp_dtensor_state_dict 1/1 ... [2024-08-20 22:28:50.936435] 2024-08-20T22:28:50.9367940Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:28:50.9372025Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_hsdp_dtensor_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:28:50.936866] 2024-08-20T22:29:17.1617126Z 2024-08-20T22:29:17.1619254Z distributed/fsdp/test_hsdp_dtensor_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_hsdp_dtensor_state_dict_1.1_c60b6b43126a5031_.log 2024-08-20T22:29:17.1627390Z Running 8 items in this shard: test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_False, test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_True, test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_False, test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_True, test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_False, test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_True, test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_hsdp_init_with_device_mesh, test/distributed/fsdp/test_hsdp_dtensor_state_dict.py::TestHSDPWithDeviceMeshAndDTensor::test_root_module_is_not_FSDP 2024-08-20T22:29:17.1634249Z 2024-08-20T22:29:17.1634871Z Running distributed/_composable/fully_shard/test_fully_shard_runtime 1/1 ... [2024-08-20 22:29:17.162202] 2024-08-20T22:29:17.1635639Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:29:17.1637547Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_runtime.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:29:17.162683] 2024-08-20T22:29:20.3384510Z 2024-08-20T22:29:20.3386807Z distributed/_composable/fully_shard/test_fully_shard_runtime 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_runtime_1.1_4d06f7e775a93721_.log 2024-08-20T22:29:20.3389603Z Running 2 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_runtime.py::TestRuntime::test_training, test/distributed/_composable/fully_shard/test_fully_shard_runtime.py::TestRuntime::test_unshard_reshard_order 2024-08-20T22:29:20.3391094Z 2024-08-20T22:29:20.3392013Z Running distributed/algorithms/quantization/test_quantization 1/1 ... [2024-08-20 22:29:20.338975] 2024-08-20T22:29:20.3465496Z MPI not available -- MPI backend tests will be skipped 2024-08-20T22:29:20.3467666Z Running distributed tests for the test backend with env init_method 2024-08-20T22:29:20.3470293Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:29:20.3476876Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/algorithms/quantization/test_quantization.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:29:20.347361] 2024-08-20T22:29:22.9777210Z 2024-08-20T22:29:22.9779228Z distributed/algorithms/quantization/test_quantization 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_d2ea3bb9e2c49b30_.log 2024-08-20T22:29:22.9780727Z Running 0 items in this shard: 2024-08-20T22:29:22.9780987Z 2024-08-20T22:29:22.9786358Z Running distributed tests for the test backend with file init_method 2024-08-20T22:29:22.9788179Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:29:22.9793618Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/algorithms/quantization/test_quantization.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:29:22.979035] 2024-08-20T22:29:25.6210434Z 2024-08-20T22:29:25.6212677Z distributed/algorithms/quantization/test_quantization 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_3727a4fc912118c6_.log 2024-08-20T22:29:25.6214161Z Running 0 items in this shard: 2024-08-20T22:29:25.6214414Z 2024-08-20T22:29:25.6219892Z Running distributed tests for the gloo backend with env init_method 2024-08-20T22:29:25.6220963Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:29:25.6226806Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/algorithms/quantization/test_quantization.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:29:25.622297] 2024-08-20T22:29:50.8992877Z 2024-08-20T22:29:50.8995096Z distributed/algorithms/quantization/test_quantization 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_53152eb7f3adb516_.log 2024-08-20T22:29:50.9000473Z Running 6 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_bfp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_fp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_bfp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_fp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_bfp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_fp16 2024-08-20T22:29:50.9005637Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_bfp16 2024-08-20T22:29:50.9007192Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_fp16 2024-08-20T22:29:50.9008886Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_bfp16 2024-08-20T22:29:50.9010504Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_fp16 2024-08-20T22:29:50.9012101Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_bfp16 2024-08-20T22:29:50.9013722Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_fp16 2024-08-20T22:29:50.9014593Z 2024-08-20T22:29:50.9014898Z Running distributed tests for the gloo backend with file init_method 2024-08-20T22:29:50.9015472Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:29:50.9017529Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/algorithms/quantization/test_quantization.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:29:50.900782] 2024-08-20T22:30:16.4874651Z 2024-08-20T22:30:16.4877356Z distributed/algorithms/quantization/test_quantization 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_875781efd6fee1e1_.log 2024-08-20T22:30:16.4882500Z Running 6 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_bfp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_fp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_bfp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_fp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_bfp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_fp16 2024-08-20T22:30:16.4887144Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_bfp16 2024-08-20T22:30:16.4889139Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_fp16 2024-08-20T22:30:16.4890830Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_bfp16 2024-08-20T22:30:16.4892468Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_fp16 2024-08-20T22:30:16.4894298Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_bfp16 2024-08-20T22:30:16.4895919Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_fp16 2024-08-20T22:30:16.4896972Z 2024-08-20T22:30:16.4897536Z Running distributed/fsdp/test_fsdp_hybrid_shard 1/1 ... [2024-08-20 22:30:16.488711] 2024-08-20T22:30:16.4898185Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:30:16.4900013Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_hybrid_shard.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:30:16.489129] 2024-08-20T22:30:36.3938480Z 2024-08-20T22:30:36.3941280Z distributed/fsdp/test_fsdp_hybrid_shard 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_hybrid_shard_1.1_b69dfe17348ed6ac_.log 2024-08-20T22:30:36.3946157Z Running 6 items in this shard: test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_fsdp_hybrid_shard_basic_setup, test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_fsdp_hybrid_shard_parity, test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_hsdp_save_load_state_dict, test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_hsdp_sync_module_state, test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_invalid_pg_specification_raises, test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_raises_manual_wrap_hybrid_shard_when_none_policy 2024-08-20T22:30:36.3950808Z 2024-08-20T22:30:36.3951407Z Running distributed/algorithms/test_join 1/1 ... [2024-08-20 22:30:36.394504] 2024-08-20T22:30:36.3952331Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:30:36.3954407Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/algorithms/test_join.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:30:36.395053] 2024-08-20T22:31:04.7737997Z 2024-08-20T22:31:04.7740061Z distributed/algorithms/test_join 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.algorithms.test_join_1.1_502c3447c24b4005_.log 2024-08-20T22:31:04.7745819Z Running 9 items in this shard: test/distributed/algorithms/test_join.py::TestJoin::test_join_kwargs, test/distributed/algorithms/test_join.py::TestJoin::test_multiple_joinable_disable, test/distributed/algorithms/test_join.py::TestJoin::test_multiple_joinables, test/distributed/algorithms/test_join.py::TestJoin::test_multiple_joinables_throw, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_disable, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_main_hooks, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_post_hooks, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_throw 2024-08-20T22:31:04.7750338Z 2024-08-20T22:31:04.7750749Z Running distributed/fsdp/test_utils 1/1 ... [2024-08-20 22:31:04.774309] 2024-08-20T22:31:04.7751371Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:31:04.7753354Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:31:04.774783] 2024-08-20T22:31:07.8503240Z 2024-08-20T22:31:07.8505256Z distributed/fsdp/test_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_utils_1.1_e224f7f2b17ed7c0_.log 2024-08-20T22:31:07.8508897Z Running 5 items in this shard: test/distributed/fsdp/test_utils.py::TestUtils::test_apply_to_tensors_cpu_cuda, test/distributed/fsdp/test_utils.py::TestUtils::test_apply_to_tensors_devices0, test/distributed/fsdp/test_utils.py::TestUtils::test_apply_to_tensors_devices1, test/distributed/fsdp/test_utils.py::TestUtils::test_packed_sequence, test/distributed/fsdp/test_utils.py::TestUtils::test_replace_by_prefix 2024-08-20T22:31:07.8511908Z 2024-08-20T22:31:07.8512378Z Running distributed/fsdp/test_fsdp_core 1/1 ... [2024-08-20 22:31:07.850782] 2024-08-20T22:31:07.8513053Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:31:07.8515522Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_core.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:31:07.851246] 2024-08-20T22:34:08.8046297Z 2024-08-20T22:34:08.8048531Z distributed/fsdp/test_fsdp_core 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_core_1.1_950977b690e57120_.log 2024-08-20T22:34:08.8086463Z Running 60 items in this shard: test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_optim_step_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_optim_step_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_optim_step_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_optim_step_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_optim_step_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_optim_step_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_reduce_scatter_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_reduce_scatter_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_reduce_scatter_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_reduce_scatter_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_reduce_scatter_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_reduce_scatter_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_with_delay_before_free_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_with_delay_before_free_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_with_delay_before_free_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_with_delay_before_free_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_with_delay_before_free_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_with_delay_before_free_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_always_wrap_model_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_always_wrap_model_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_always_wrap_model_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_always_wrap_model_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_always_wrap_model_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_always_wrap_model_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_single_iteration_mixed_precision_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_single_iteration_mixed_precision_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_single_iteration_mixed_precision_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_single_iteration_mixed_precision_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_single_iteration_mixed_precision_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_single_iteration_mixed_precision_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_transformer_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_transformer_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_transformer_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_transformer_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_transformer_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_transformer_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParamInit::test_param_change_after_init_mixed_precision_False, test/distributed/fsdp/test_fsdp_core.py::TestParamInit::test_param_change_after_init_mixed_precision_True, test/distributed/fsdp/test_fsdp_core.py::TestHooks::test_pre_backward_hook_registration_after_state_dict, test/distributed/fsdp/test_fsdp_core.py::TestHooks::test_pre_backward_hook_registration_cuda_first_False, test/distributed/fsdp/test_fsdp_core.py::TestHooks::test_pre_backward_hook_registration_cuda_first_True, test/distributed/fsdp/test_fsdp_core.py::TestHooks::test_register_functions_called_cuda_first_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_core.py::TestHooks::test_register_functions_called_cuda_first_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_core.py::TestHooks::test_register_functions_called_cuda_first_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_core.py::TestHooks::test_register_functions_called_cuda_first_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_core.py::TestNoGrad::test_transformer_no_grad_mixed_precision_False, test/distributed/fsdp/test_fsdp_core.py::TestNoGrad::test_transformer_no_grad_mixed_precision_True, test/distributed/fsdp/test_fsdp_core.py::TestAutograd::test_unshard_params_as_tensors 2024-08-20T22:34:08.8122405Z 2024-08-20T22:34:08.8122940Z Running distributed/fsdp/test_fsdp_checkpoint 1/1 ... [2024-08-20 22:34:08.805318] 2024-08-20T22:34:08.8123603Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:34:08.8125398Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:34:08.805782] 2024-08-20T22:35:02.0454860Z 2024-08-20T22:35:02.0457731Z distributed/fsdp/test_fsdp_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_checkpoint_1.1_8249b56a329e380c_.log 2024-08-20T22:35:02.0482033Z Running 17 items in this shard: test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpointSubmodule::test_checkpoint_submodule_use_reentrant_False 2024-08-20T22:35:02.0502790Z 2024-08-20T22:35:02.0503699Z Running distributed/fsdp/test_fsdp_use_orig_params 1/1 ... [2024-08-20 22:35:02.046209] 2024-08-20T22:35:02.0504674Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:35:02.0507680Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_use_orig_params.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:35:02.046817] 2024-08-20T22:36:04.1186196Z 2024-08-20T22:36:04.1188091Z distributed/fsdp/test_fsdp_use_orig_params 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_use_orig_params_1.1_779b6079a333d90c_.log 2024-08-20T22:36:04.1208922Z Running 25 items in this shard: test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_cpu_offload_sharding_strategy_str_full_shard, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_cpu_offload_sharding_strategy_str_no_shard, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_cpu_offload_sharding_strategy_str_shard_grad_op, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_sharding_strategy_str_full_shard, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_sharding_strategy_str_no_shard, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_sharding_strategy_str_shard_grad_op, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_trainability, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_fsdp_compile, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_multiple_optimizers, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsUnshardReshard::test_multiple_forward_offload_params_False, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsUnshardReshard::test_multiple_forward_offload_params_True, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsUnshardReshard::test_summon_between_two_forwards_offload_params_False, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsUnshardReshard::test_summon_between_two_forwards_offload_params_True, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsParamAccess::test_access_params_after_forward, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsWriteback::test_grad_writeback, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsWriteback::test_no_reshard_and_mixed_precision, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsWriteback::test_param_writeback, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsWriteback::test_writeback_between_fwd_and_bwd_for_no_reshard_raises, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsWriteback::test_writeback_shape_mismatch, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsFQNs::test_named_parameters_in_forward, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsNoSync::test_no_sync_correctness, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsNoSync::test_no_sync_mixed_precision, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsInit::test_non_uniform_requires_grad, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestMultiTensorApply::test_multi_tensor_apply_size0_tensors_cpu, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestMultiTensorApply::test_multi_tensor_apply_size0_tensors_cuda 2024-08-20T22:36:04.1228138Z 2024-08-20T22:36:04.1228869Z Running distributed/_shard/sharding_spec/test_sharding_spec 1/1 ... [2024-08-20 22:36:04.119057] 2024-08-20T22:36:04.1229595Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:36:04.1231689Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharding_spec/test_sharding_spec.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:36:04.119558] 2024-08-20T22:36:10.6525823Z 2024-08-20T22:36:10.6528095Z distributed/_shard/sharding_spec/test_sharding_spec 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharding_spec.test_sharding_spec_1.1_1ea20b8dffd3c0cc_.log 2024-08-20T22:36:10.6536655Z Running 11 items in this shard: test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_check_overlapping, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_chunked_sharding_spec, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_device_placement, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_enumerable_sharding_spec, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_get_chunk_sharding_params, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_get_chunked_dim_size, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_get_split_size, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_infer_sharding_spec_from_shards_metadata, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestCustomShardingSpec::test_custom_sharding_spec, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestCustomShardingSpec::test_custom_sharding_spec_shard_tensor, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestCustomShardingSpec::test_custom_sharding_spec_tensor_ctor 2024-08-20T22:36:10.6543767Z 2024-08-20T22:36:10.6544240Z Running distributed/fsdp/test_fsdp_apply 1/1 ... [2024-08-20 22:36:10.652800] 2024-08-20T22:36:10.6544877Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:36:10.6546630Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_apply.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:36:10.653276] 2024-08-20T22:36:22.2423389Z 2024-08-20T22:36:22.2425070Z distributed/fsdp/test_fsdp_apply 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_apply_1.1_8f7cb1ff0823d6c1_.log 2024-08-20T22:36:22.2427728Z Running 3 items in this shard: test/distributed/fsdp/test_fsdp_apply.py::TestApply::test_apply_in_summon_raises_error, test/distributed/fsdp/test_fsdp_apply.py::TestApply::test_nested_module_apply, test/distributed/fsdp/test_fsdp_apply.py::TestApply::test_transformer_module_apply 2024-08-20T22:36:22.2429741Z 2024-08-20T22:36:22.2430317Z Running distributed/_shard/sharding_plan/test_sharding_plan 1/1 ... [2024-08-20 22:36:22.242656] 2024-08-20T22:36:22.2431116Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:36:22.2435061Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharding_plan/test_sharding_plan.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:36:22.243170] 2024-08-20T22:36:25.0687553Z 2024-08-20T22:36:25.0689847Z distributed/_shard/sharding_plan/test_sharding_plan 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharding_plan.test_sharding_plan_1.1_86dc68755fd3da5a_.log 2024-08-20T22:36:25.0693335Z Running 3 items in this shard: test/distributed/_shard/sharding_plan/test_sharding_plan.py::TestShardingPlan::test_custom_sharding_planner, test/distributed/_shard/sharding_plan/test_sharding_plan.py::TestShardingPlan::test_shard_module_sub_process_group, test/distributed/_shard/sharding_plan/test_sharding_plan.py::TestShardingPlan::test_sharding_plan_errors 2024-08-20T22:36:25.0696127Z 2024-08-20T22:36:25.0696558Z Running distributed/test_pg_wrapper 1/1 ... [2024-08-20 22:36:25.069209] 2024-08-20T22:36:25.0697143Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:36:25.0700781Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_pg_wrapper.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:36:25.069740] 2024-08-20T22:37:47.1422903Z 2024-08-20T22:37:47.1427921Z distributed/test_pg_wrapper 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_pg_wrapper_1.1_4f099d5c09e51039_.log 2024-08-20T22:37:47.1439830Z Running 17 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_coalescing_manager_debug_mode_detail, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_hang, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_detail, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_off, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_debug_level_detail_no_gloo, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_new_group_no_gloo, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_hang, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode_off, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_debug_mode 2024-08-20T22:37:47.1450581Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_coalescing_manager_debug_mode_detail 2024-08-20T22:37:47.1460097Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_hang 2024-08-20T22:37:47.1461648Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_detail 2024-08-20T22:37:47.1463193Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_off 2024-08-20T22:37:47.1464646Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch 2024-08-20T22:37:47.1466081Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch_debug_mode 2024-08-20T22:37:47.1467506Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_debug_level_detail_no_gloo 2024-08-20T22:37:47.1468861Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_new_group_no_gloo 2024-08-20T22:37:47.1470147Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_hang 2024-08-20T22:37:47.1471754Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda 2024-08-20T22:37:47.1473239Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda_debug_mode 2024-08-20T22:37:47.1474752Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode 2024-08-20T22:37:47.1476272Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode_off 2024-08-20T22:37:47.1477714Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch 2024-08-20T22:37:47.1479166Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda 2024-08-20T22:37:47.1480645Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda_debug_mode 2024-08-20T22:37:47.1482144Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_debug_mode 2024-08-20T22:37:47.1482951Z 2024-08-20T22:37:47.1483506Z Running distributed/fsdp/test_fsdp_input 1/1 ... [2024-08-20 22:37:47.143121] 2024-08-20T22:37:47.1484112Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:37:47.1485898Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_input.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:37:47.143617] 2024-08-20T22:37:55.7780828Z 2024-08-20T22:37:55.7783028Z distributed/fsdp/test_fsdp_input 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_input_1.1_6c520b1858866906_.log 2024-08-20T22:37:55.7785219Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_input.py::TestInput::test_input_type_dict, test/distributed/fsdp/test_fsdp_input.py::TestInput::test_input_type_list 2024-08-20T22:37:55.7786681Z 2024-08-20T22:37:55.7787550Z Running distributed/fsdp/test_fsdp_multiple_wrapping 1/1 ... [2024-08-20 22:37:55.778468] 2024-08-20T22:37:55.7788229Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:37:55.7792800Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_multiple_wrapping.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:37:55.778957] 2024-08-20T22:38:01.9113591Z 2024-08-20T22:38:01.9115709Z distributed/fsdp/test_fsdp_multiple_wrapping 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_multiple_wrapping_1.1_63daa244e5aa0b2f_.log 2024-08-20T22:38:01.9117578Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_multiple_wrapping.py::TestMultipleWrapping::test_multiple_wrapping 2024-08-20T22:38:01.9118368Z 2024-08-20T22:38:01.9120233Z Running distributed/fsdp/test_fsdp_exec_order 1/1 ... [2024-08-20 22:38:01.911782] 2024-08-20T22:38:01.9120886Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:38:01.9125708Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_exec_order.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:38:01.912285] 2024-08-20T22:38:28.8397851Z 2024-08-20T22:38:28.8400074Z distributed/fsdp/test_fsdp_exec_order 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_exec_order_1.1_926b3d7197af8cd9_.log 2024-08-20T22:38:28.8408748Z Running 8 items in this shard: test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_invalid_first_iter_order_sharding_strategy0, test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_invalid_first_iter_order_sharding_strategy1, test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_invalid_later_iter_order_sharding_strategy0_iters_before_path_change_1, test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_invalid_later_iter_order_sharding_strategy0_iters_before_path_change_3, test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_invalid_later_iter_order_sharding_strategy1_iters_before_path_change_1, test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_invalid_later_iter_order_sharding_strategy1_iters_before_path_change_3, test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_train_eval_sharding_strategy0, test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_train_eval_sharding_strategy1 2024-08-20T22:38:28.8415106Z 2024-08-20T22:38:28.8415693Z Running distributed/fsdp/test_fsdp_mixed_precision 1/1 ... [2024-08-20 22:38:28.840404] 2024-08-20T22:38:28.8416447Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:38:28.8418438Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_mixed_precision.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:38:28.840975] 2024-08-20T22:41:06.5798641Z 2024-08-20T22:41:06.5800737Z distributed/fsdp/test_fsdp_mixed_precision 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_mixed_precision_1.1_9ceb524957a75a33_.log 2024-08-20T22:41:06.5849388Z Running 57 items in this shard: test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_eval_root_cast_inputs, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_full_precision_in_eval, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_full_precision_in_eval_buffers, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_full_precision_in_eval_comm, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_grads_reduced_precision, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_input_grads_with_param_mixed_precision, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_false_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_false_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_false_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_false_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_true_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_true_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_true_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_fp16_offload_true_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_false_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_false_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_false_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_false_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_true_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_true_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_true_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_no_mp_offload_true_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_false_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_false_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_false_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_false_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_true_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_true_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_true_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_param_and_buf_offload_true_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_false_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_false_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_false_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_false_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_true_fp32_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_true_fp32_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_true_fp64_enable_sharded_grad_scaler, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_e2e_full_shard_mp_only_reduce_offload_true_fp64_none, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_no_reshard_after_forward, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mixed_precision_resnet, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_batchnorm_convert_sync_bn_False, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_batchnorm_convert_sync_bn_True, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_embedding_default, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_embedding_only_params_and_bufs, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_embedding_params_and_reduce_diff, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionSharded::test_mp_embedding_reduce, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionUnsharded::test_grads_reduced_precision, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionUnsharded::test_mixed_precision_e2e_full_shard, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionUnsharded::test_mixed_precision_no_reshard_after_forward, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPMixedPrecisionIgnoredModules::test_mixed_precision_with_ignored_module, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_float16_on_one_submodule, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_float16_on_one_submodule_skip_inputs, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_float16_on_one_submodule_skip_inputs_error, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_submodules_with_different_precisions, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_submodules_with_different_precisions_error, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPDifferentSubmodulePrecision::test_submodules_with_external_inputs, test/distributed/fsdp/test_fsdp_mixed_precision.py::TestFSDPTrainEval::test_train_ema_eval_flow 2024-08-20T22:41:06.5894467Z 2024-08-20T22:41:06.5894981Z Running distributed/fsdp/test_fsdp_fine_tune 1/1 ... [2024-08-20 22:41:06.580526] 2024-08-20T22:41:06.5895619Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:41:06.5897421Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_fine_tune.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:41:06.581032] 2024-08-20T22:41:09.6066637Z 2024-08-20T22:41:09.6068578Z distributed/fsdp/test_fsdp_fine_tune 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_fine_tune_1.1_31f573bc48a8cfe0_.log 2024-08-20T22:41:09.6072372Z Running 4 items in this shard: test/distributed/fsdp/test_fsdp_fine_tune.py::TestFSDPFineTune::test_backward_reshard_hooks, test/distributed/fsdp/test_fsdp_fine_tune.py::TestFSDPFineTune::test_hooks_multi_traversal, test/distributed/fsdp/test_fsdp_fine_tune.py::TestFSDPFineTune::test_parity_with_ddp, test/distributed/fsdp/test_fsdp_fine_tune.py::TestFSDPFineTune::test_parity_with_non_frozen_fsdp 2024-08-20T22:41:09.6075138Z 2024-08-20T22:41:09.6075596Z Running distributed/fsdp/test_fsdp_traversal 1/1 ... [2024-08-20 22:41:09.607159] 2024-08-20T22:41:09.6076250Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:41:09.6079358Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_traversal.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:41:09.607656] 2024-08-20T22:41:15.6382878Z 2024-08-20T22:41:15.6384582Z distributed/fsdp/test_fsdp_traversal 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_traversal_1.1_bce50c9f7f3ab100_.log 2024-08-20T22:41:15.6386510Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_traversal.py::TestTraversal::test_fsdp_modules 2024-08-20T22:41:15.6387193Z 2024-08-20T22:41:15.6389062Z Running distributed/_shard/sharded_tensor/ops/test_tensor_ops 1/1 ... [2024-08-20 22:41:15.638677] 2024-08-20T22:41:15.6389778Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:41:15.6394767Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_tensor_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:41:15.639153] 2024-08-20T22:41:18.4643641Z 2024-08-20T22:41:18.4645529Z distributed/_shard/sharded_tensor/ops/test_tensor_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_tensor_ops_1.1_477a0215b26c5557_.log 2024-08-20T22:41:18.4650063Z Running 5 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_clone, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_deep_copy, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_detach, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_inplace_copy, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_set_requires_grad 2024-08-20T22:41:18.4652820Z 2024-08-20T22:41:18.4653528Z Running distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint 1/1 ... [2024-08-20 22:41:18.464748] 2024-08-20T22:41:18.4654455Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:41:18.4656438Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:41:18.465246] 2024-08-20T22:41:27.1502249Z 2024-08-20T22:41:27.1504696Z distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint_1.1_e44c8d76e1478aca_.log 2024-08-20T22:41:27.1507915Z Running 2 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint.py::TestOptimStateCheckpointing::test_optim_state_dict_save_load, test/distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint.py::TestOptimStateCheckpointing::test_optim_state_dict_submodule_fully_shard 2024-08-20T22:41:27.1509750Z 2024-08-20T22:41:27.1510288Z Running distributed/_shard/sharded_optim/test_sharded_optim 1/1 ... [2024-08-20 22:41:27.150674] 2024-08-20T22:41:27.1511001Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:41:27.1514521Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_optim/test_sharded_optim.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:41:27.151133] 2024-08-20T22:41:29.8761025Z 2024-08-20T22:41:29.8762977Z distributed/_shard/sharded_optim/test_sharded_optim 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_optim.test_sharded_optim_1.1_05930829ab77c03f_.log 2024-08-20T22:41:29.8765533Z Running 2 items in this shard: test/distributed/_shard/sharded_optim/test_sharded_optim.py::TestShardedOptimizer::test_named_params_with_sharded_tensor, test/distributed/_shard/sharded_optim/test_sharded_optim.py::TestShardedOptimizer::test_sharded_optim 2024-08-20T22:41:29.8767137Z 2024-08-20T22:41:29.8767646Z Running distributed/rpc/test_tensorpipe_agent 1/3 ... [2024-08-20 22:41:29.876541] 2024-08-20T22:41:29.8768298Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:41:29.8773370Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_tensorpipe_agent.py', '--shard-id=1', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:41:29.877010] 2024-08-20T23:00:19.2711073Z 2024-08-20T23:00:19.2713065Z distributed/rpc/test_tensorpipe_agent 1/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_tensorpipe_agent_1.3_2c4c626cf6aa1639_.log 2024-08-20T23:00:19.2839114Z Running 153 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_add, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_add_with_id, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_all_gather, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_all_gather_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_method, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_method_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_chained, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_chained_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_chained_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_fanout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_fanout_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_nested, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_nested_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_with_future_ctor, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_double_end_callbacks, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_static_method, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_builtin_remote_self, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_chain, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_in_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_wrong_arg_num, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_cannot_infer_backend_from_options, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_custom_exception_throw_during_reconstruction, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_deadlock, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_debug_info, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_expected_src, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_nested_callback, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_pg_then_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_shutdown_with_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_value_not_on_owner, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_mark_future_twice, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_my_parameter_server, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_pass_local_rrefs, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_pg_init_no_rpc_init, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_rpc_key_names, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_rpc_memory, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_autograd_context_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_builtin_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_async_rpc_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_udf_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_constructor, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_method, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_function_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_no_return_result, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_raise_in_user_func, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_raise_in_user_func_escaped_str, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors_in_container, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_remote_same_worker, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_remote_throw, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_remote_with_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_local_rrefs, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_multithreaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_partial_subset, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_subset, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_async_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_non_exist, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_reuse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_tensor_self, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_py_pickle_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_non_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_owner_non_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_with_error_non_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_py_udf_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_remote_arg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_stress_heavy_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_stress_heavy_rpc_torchscript, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_stress_light_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rref_backward, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rrefs_confirmed, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rrefs_confirmed_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_exit_early_builtin, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_exit_early_script_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_multiple_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_raise_in_body, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wrong_types, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_async_dist_autograd, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_invalid_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_no_grad_on_tensor, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_python_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_without_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_without_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backwards_nested_python_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_clean_context_during_backward, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_debug_info, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_error_in_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_copy_sparse_indices_extra_ref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_only_on_return_value, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_builtin_remote_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_call_itself, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_multiple_backward, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_nested_backward_accumulate_grads, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_post_hooks, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_exception_on_constructor, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_all_kwargs_are_populated_by_defaults, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_args_kwargs_are_neither_passed, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_wrong_return_type, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_script_throw, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_script_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_with_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_kwargs_not_passed, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_less_than_needed_args_are_specified, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_more_than_needed_args_are_specified, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_no_kwargs_are_populated_by_defaults, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_return_local_script_module_rref_in_py_and_use_in_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rpc_torchscript_record_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_is_owner, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_jit_pickle_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_some_kwargs_are_populated_by_defaults, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_function_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_user_rrefs_confirmed, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_bad_module, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_sync, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_remote_module_py_pickle_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_unsupported_methods, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_outside, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_outside_uneven_inputs, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_dist_autograd_sparse_grads, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeReinforcementLearningRpcTest::test_rl_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_existing_rank_can_communicate_with_new_rank_cuda, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_new_rank_can_communicated_with_existing_rank, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_mismatched_type_for_options, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_builtin_remote_ret_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_rpc_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rref_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_user_share_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_get_type_timeout_non_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_wait_all_workers_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_world_size_one_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_nested_rpc_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_with_grad_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_remote_call_itself_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_python_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_no_graph_with_tensors_not_require_grad_remote_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_rpc_complex_args_sparse 2024-08-20T23:00:19.2923935Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_add 2024-08-20T23:00:19.2925133Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_add_with_id 2024-08-20T23:00:19.2926481Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_all_gather 2024-08-20T23:00:19.2927740Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_all_gather_timeout 2024-08-20T23:00:19.2929027Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_method 2024-08-20T23:00:19.2930432Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_method_remote 2024-08-20T23:00:19.2931777Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy 2024-08-20T23:00:19.2933194Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_chained 2024-08-20T23:00:19.2934577Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_chained_async 2024-08-20T23:00:19.2936135Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_chained_remote 2024-08-20T23:00:19.2937574Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_fanout 2024-08-20T23:00:19.2939005Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_fanout_async 2024-08-20T23:00:19.2940380Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_nested 2024-08-20T23:00:19.2941752Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_nested_remote 2024-08-20T23:00:19.2943101Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise 2024-08-20T23:00:19.2944438Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise_remote 2024-08-20T23:00:19.2945786Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_simple 2024-08-20T23:00:19.2947158Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_with_future_ctor 2024-08-20T23:00:19.2948584Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type 2024-08-20T23:00:19.2950053Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_double_end_callbacks 2024-08-20T23:00:19.2951477Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_static_method 2024-08-20T23:00:19.2952780Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_builtin_remote_self 2024-08-20T23:00:19.2954043Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_chain 2024-08-20T23:00:19.2955287Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_in_rpc 2024-08-20T23:00:19.2956587Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_wrong_arg_num 2024-08-20T23:00:19.2957969Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_cannot_infer_backend_from_options 2024-08-20T23:00:19.2959466Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_custom_exception_throw_during_reconstruction 2024-08-20T23:00:19.2960817Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_deadlock 2024-08-20T23:00:19.2962076Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_debug_info 2024-08-20T23:00:19.2963293Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_expected_src 2024-08-20T23:00:19.2964555Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_nested_callback 2024-08-20T23:00:19.2965859Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_pg_then_rpc 2024-08-20T23:00:19.2967170Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_shutdown_with_rpc 2024-08-20T23:00:19.2968565Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_value_not_on_owner 2024-08-20T23:00:19.2969918Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_mark_future_twice 2024-08-20T23:00:19.2971170Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_rpc 2024-08-20T23:00:19.2972423Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_my_parameter_server 2024-08-20T23:00:19.2973676Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rpc 2024-08-20T23:00:19.2974898Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_pass_local_rrefs 2024-08-20T23:00:19.2976177Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_pg_init_no_rpc_init 2024-08-20T23:00:19.2977494Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_rpc_key_names 2024-08-20T23:00:19.2978811Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_rpc_memory 2024-08-20T23:00:19.2980240Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_autograd_context_single_threaded 2024-08-20T23:00:19.2981801Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_builtin_single_threaded 2024-08-20T23:00:19.2983358Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_async_rpc_single_threaded 2024-08-20T23:00:19.2984828Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_udf 2024-08-20T23:00:19.2986279Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_udf_single_threaded 2024-08-20T23:00:19.2987699Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_constructor 2024-08-20T23:00:19.2988977Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_method 2024-08-20T23:00:19.2990255Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_function_exception 2024-08-20T23:00:19.2991566Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_no_return_result 2024-08-20T23:00:19.2992864Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_raise_in_user_func 2024-08-20T23:00:19.2994410Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_raise_in_user_func_escaped_str 2024-08-20T23:00:19.2995810Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors_in_container 2024-08-20T23:00:19.2997250Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_remote_same_worker 2024-08-20T23:00:19.2998511Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_remote_throw 2024-08-20T23:00:19.2999791Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_remote_with_exception 2024-08-20T23:00:19.3001088Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_local_rrefs 2024-08-20T23:00:19.3002415Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_multithreaded 2024-08-20T23:00:19.3003875Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_partial_subset 2024-08-20T23:00:19.3005211Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_subset 2024-08-20T23:00:19.3006544Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_async_function 2024-08-20T23:00:19.3007896Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_non_exist 2024-08-20T23:00:19.3009184Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_reuse 2024-08-20T23:00:19.3010533Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_tensor_self 2024-08-20T23:00:19.3011905Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_py_pickle_not_supported 2024-08-20T23:00:19.3013225Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_timeout 2024-08-20T23:00:19.3014485Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_blocking 2024-08-20T23:00:19.3015791Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_non_blocking 2024-08-20T23:00:19.3017150Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_owner_non_blocking 2024-08-20T23:00:19.3018569Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_with_error_non_blocking 2024-08-20T23:00:19.3019927Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_py_udf_remote 2024-08-20T23:00:19.3021262Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_remote_arg 2024-08-20T23:00:19.3022605Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_stress_heavy_rpc 2024-08-20T23:00:19.3023934Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_stress_heavy_rpc_torchscript 2024-08-20T23:00:19.3025258Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_stress_light_rpc 2024-08-20T23:00:19.3026516Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rref_backward 2024-08-20T23:00:19.3027810Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rrefs_confirmed 2024-08-20T23:00:19.3029161Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rrefs_confirmed_remote 2024-08-20T23:00:19.3030526Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_exit_early_builtin 2024-08-20T23:00:19.3031935Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_exit_early_script_function 2024-08-20T23:00:19.3033404Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_multiple_call 2024-08-20T23:00:19.3034739Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_raise_in_body 2024-08-20T23:00:19.3036014Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_timeout 2024-08-20T23:00:19.3037254Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wrong_types 2024-08-20T23:00:19.3038577Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_async_dist_autograd 2024-08-20T23:00:19.3040071Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_invalid_args 2024-08-20T23:00:19.3041523Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_no_grad_on_tensor 2024-08-20T23:00:19.3042947Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref 2024-08-20T23:00:19.3044313Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple 2024-08-20T23:00:19.3045750Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_python_udf 2024-08-20T23:00:19.3047222Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_without_context 2024-08-20T23:00:19.3048683Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_without_rpc 2024-08-20T23:00:19.3050211Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backwards_nested_python_udf 2024-08-20T23:00:19.3051735Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_clean_context_during_backward 2024-08-20T23:00:19.3053144Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_debug_info 2024-08-20T23:00:19.3054496Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_error_in_context 2024-08-20T23:00:19.3055976Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_copy_sparse_indices_extra_ref 2024-08-20T23:00:19.3057485Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_only_on_return_value 2024-08-20T23:00:19.3059003Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_builtin_remote_call 2024-08-20T23:00:19.3060543Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_call_itself 2024-08-20T23:00:19.3062014Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_multiple_backward 2024-08-20T23:00:19.3063488Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_nested_backward_accumulate_grads 2024-08-20T23:00:19.3064931Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_post_hooks 2024-08-20T23:00:19.3066257Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim 2024-08-20T23:00:19.3067649Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_exception 2024-08-20T23:00:19.3069212Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_exception_on_constructor 2024-08-20T23:00:19.3070762Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_all_kwargs_are_populated_by_defaults 2024-08-20T23:00:19.3072253Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_args_kwargs_are_neither_passed 2024-08-20T23:00:19.3073656Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_simple 2024-08-20T23:00:19.3075051Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_wrong_return_type 2024-08-20T23:00:19.3076504Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_script_throw 2024-08-20T23:00:19.3077815Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_script_udf 2024-08-20T23:00:19.3079112Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_simple 2024-08-20T23:00:19.3080439Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_with_exception 2024-08-20T23:00:19.3081783Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_kwargs_not_passed 2024-08-20T23:00:19.3083188Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_less_than_needed_args_are_specified 2024-08-20T23:00:19.3084693Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_more_than_needed_args_are_specified 2024-08-20T23:00:19.3086184Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_no_kwargs_are_populated_by_defaults 2024-08-20T23:00:19.3087579Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_udf 2024-08-20T23:00:19.3089089Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_return_local_script_module_rref_in_py_and_use_in_script 2024-08-20T23:00:19.3090717Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rpc_torchscript_record_function 2024-08-20T23:00:19.3092090Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_is_owner 2024-08-20T23:00:19.3093451Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_jit_pickle_not_supported 2024-08-20T23:00:19.3095122Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_some_kwargs_are_populated_by_defaults 2024-08-20T23:00:19.3096557Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_function 2024-08-20T23:00:19.3097973Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_function_exception 2024-08-20T23:00:19.3099382Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_user_rrefs_confirmed 2024-08-20T23:00:19.3100722Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_bad_module 2024-08-20T23:00:19.3102034Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_sync 2024-08-20T23:00:19.3103512Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_remote_module_py_pickle_not_supported 2024-08-20T23:00:19.3105135Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_unsupported_methods 2024-08-20T23:00:19.3106622Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_outside 2024-08-20T23:00:19.3108219Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_outside_uneven_inputs 2024-08-20T23:00:19.3109839Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_dist_autograd_sparse_grads 2024-08-20T23:00:19.3111339Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeReinforcementLearningRpcTest::test_rl_rpc 2024-08-20T23:00:19.3113100Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_existing_rank_can_communicate_with_new_rank_cuda 2024-08-20T23:00:19.3114984Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_new_rank_can_communicated_with_existing_rank 2024-08-20T23:00:19.3116712Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_mismatched_type_for_options 2024-08-20T23:00:19.3118310Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_builtin_remote_ret_sparse 2024-08-20T23:00:19.3119862Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_rpc_sparse 2024-08-20T23:00:19.3121343Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rref_sparse 2024-08-20T23:00:19.3122896Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_user_share_sparse 2024-08-20T23:00:19.3124530Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_get_type_timeout_non_blocking 2024-08-20T23:00:19.3126122Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_wait_all_workers_sparse 2024-08-20T23:00:19.3127643Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_world_size_one_sparse 2024-08-20T23:00:19.3129221Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_sparse 2024-08-20T23:00:19.3130996Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_nested_rpc_sparse 2024-08-20T23:00:19.3132803Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_with_grad_sparse 2024-08-20T23:00:19.3134654Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_remote_call_itself_sparse 2024-08-20T23:00:19.3136469Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_python_call_sparse 2024-08-20T23:00:19.3138390Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_no_graph_with_tensors_not_require_grad_remote_sparse 2024-08-20T23:00:19.3140212Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_rpc_complex_args_sparse 2024-08-20T23:00:19.3141133Z 2024-08-20T23:00:19.3141744Z Running distributed/rpc/test_tensorpipe_agent 3/3 ... [2024-08-20 23:00:19.280316] 2024-08-20T23:00:19.3142396Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:00:19.3144352Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_tensorpipe_agent.py', '--shard-id=3', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:00:19.280799] 2024-08-20T23:21:07.6447417Z 2024-08-20T23:21:07.6449117Z distributed/rpc/test_tensorpipe_agent 3/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_tensorpipe_agent_3.3_9ed920af1c9514c2_.log 2024-08-20T23:21:07.6583765Z Running 167 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_add_done_callback, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_add, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_chained, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_fanout_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_with_future_ctor_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_cbs_jit_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_legacy, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_static_method_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_builtin_remote_ret, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_call_method_on_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_with_error, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_with_ret, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_wrong_arg_type, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_default_timeout_used, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_disable_gil_profiling, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_duplicate_name, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_graceful_shutdown_with_uneven_workload, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_handle_send_exceptions, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_ignore_rref_leak, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_rpc_then_pg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_invalid_names, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_rref_no_fork, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_shutdown, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_layer_nested_async_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_py_udf_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rref_stress, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_non_cont_tensors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nonzero, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_owner_equality, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_pickle_future, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_export_trace, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_remote_events_profiled, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_remote_events_profiled_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_rpc_record_shapes, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_udf_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_autograd_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_builtin, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_udf_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_remote_rpc_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_sync_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_sync_rpc_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_built_in, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_static_method, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rref_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rref_args_user_share, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors_multi_async_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_udf_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_user_defined, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_register_rpc_backend_and_set_and_start_rpc_backend, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_all, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_async_function_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_remote_record_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_return_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_timeouts, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_leak, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_class, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_class_self, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_tensor, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_str, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_owner_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_with_error_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_self_remote_arg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_self_rpc_arg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_server_process_global_profiler, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_set_and_get_default_rpc_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_shutdown_errors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rpc_pickler, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_exit_early_python, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_with_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_worker_id, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeParameterServerTest::test_batch_updating_parameter_server, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_accumulate_grads, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_autograd_engine_error, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_complex_python_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_dtypes, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_output_tensors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_roots, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_node_failure_python_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_python_udf_error, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_multi, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_unused_tensors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_verify_hooks, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_nested_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_no_tensors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_tensor_no_grad, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_builtin_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_remote_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_remote_call_itself, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_python_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_mixed_requires_grad, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_multiple_backward_with_errors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_grad_copy, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_grad_copy_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_graph_with_tensors_not_require_grad, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_graph_with_tensors_not_require_grad_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_remote_complex_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_thread_local_context_id, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_trainer_ps, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_trainer_ps_torchscript_functions, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_worker_ids_recorded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_wrong_decorator_order, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_wrong_return_type_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_python_function_remotely_from_script_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_rpc_with_profiling, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_script_function_that_raises_remotely_from_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_create_local_script_class_rref_in_py, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_create_local_script_module_rref_in_py, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_create_script_module_on_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_future_passed_between_python_and_jit, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_future_python_annotation, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_load_script_module_with_pickled_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_my_script_module_with_rrefs, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_record_function_on_caller_rpc_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_return_local_script_class_rref_in_py_and_use_in_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rpc_async_jit_profiled, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_list_mutate, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_local_value, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_python_annotation, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_user_rrefs_confirmed_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_async_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_with_kwargs, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_get_module_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_remote_module_py_pickle_not_supported_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_remote_parameters, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_comparison, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_comparison_uneven_inputs, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_builtin_remote_self_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_and_static_init_rpc_together, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_existing_rank_can_communicate_with_new_rank, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_infer_backend_from_options, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rpc_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_op_with_invalid_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_sparse_tensors_in_container, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_remote_arg_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_rpc_arg_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_self_rpc_arg_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_send_to_rank_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_stress_heavy_rpc_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_tensorpipe_options_throw_on_timedelta_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_tensorpipe_set_default_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_wait_all_workers_twice_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_different_dtypes_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_multiple_round_trips_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_nested_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_python_udf_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_script_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backwards_nested_python_udf_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_embedding_bag_with_no_grad_tensors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_remote_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_remote_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_python_remote_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_mixed_requires_grad_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_remote_complex_args_sparse 2024-08-20T23:21:07.6679354Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_add_done_callback 2024-08-20T23:21:07.6680646Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_add 2024-08-20T23:21:07.6681964Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy_async 2024-08-20T23:21:07.6683372Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy_remote 2024-08-20T23:21:07.6684758Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_chained 2024-08-20T23:21:07.6686190Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_fanout_remote 2024-08-20T23:21:07.6687678Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_with_future_ctor_remote 2024-08-20T23:21:07.6689184Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_async 2024-08-20T23:21:07.6690721Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_cbs_jit_call 2024-08-20T23:21:07.6692147Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_legacy 2024-08-20T23:21:07.6693535Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_static_method_remote 2024-08-20T23:21:07.6695052Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_builtin_remote_ret 2024-08-20T23:21:07.6696335Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_call_method_on_rref 2024-08-20T23:21:07.6697641Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_with_error 2024-08-20T23:21:07.6698926Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_with_ret 2024-08-20T23:21:07.6700441Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_wrong_arg_type 2024-08-20T23:21:07.6701764Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_default_timeout_used 2024-08-20T23:21:07.6703083Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_disable_gil_profiling 2024-08-20T23:21:07.6704378Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_duplicate_name 2024-08-20T23:21:07.6705737Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_graceful_shutdown_with_uneven_workload 2024-08-20T23:21:07.6707258Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_handle_send_exceptions 2024-08-20T23:21:07.6708557Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_ignore_rref_leak 2024-08-20T23:21:07.6709831Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_rpc_then_pg 2024-08-20T23:21:07.6711182Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_invalid_names 2024-08-20T23:21:07.6712453Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_rref_no_fork 2024-08-20T23:21:07.6713721Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_shutdown 2024-08-20T23:21:07.6715037Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_layer_nested_async_rpc 2024-08-20T23:21:07.6716378Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_py_udf_remote 2024-08-20T23:21:07.6717635Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rref 2024-08-20T23:21:07.6718892Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rref_stress 2024-08-20T23:21:07.6720164Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_non_cont_tensors 2024-08-20T23:21:07.6721376Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nonzero 2024-08-20T23:21:07.6722587Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_owner_equality 2024-08-20T23:21:07.6723825Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_pickle_future 2024-08-20T23:21:07.6725097Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_export_trace 2024-08-20T23:21:07.6726480Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_remote_events_profiled 2024-08-20T23:21:07.6727991Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_remote_events_profiled_single_threaded 2024-08-20T23:21:07.6729478Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_rpc_record_shapes 2024-08-20T23:21:07.6730931Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin 2024-08-20T23:21:07.6732342Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_udf 2024-08-20T23:21:07.6733812Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_udf_single_threaded 2024-08-20T23:21:07.6735303Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_autograd_context 2024-08-20T23:21:07.6736776Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_builtin 2024-08-20T23:21:07.6738234Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_udf_single_threaded 2024-08-20T23:21:07.6739787Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_remote_rpc_single_threaded 2024-08-20T23:21:07.6741285Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_sync_rpc 2024-08-20T23:21:07.6742831Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_sync_rpc_single_threaded 2024-08-20T23:21:07.6744220Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_built_in 2024-08-20T23:21:07.6745502Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_static_method 2024-08-20T23:21:07.6746781Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rref_args 2024-08-20T23:21:07.6748055Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rref_args_user_share 2024-08-20T23:21:07.6749331Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors 2024-08-20T23:21:07.6750632Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors_multi_async_call 2024-08-20T23:21:07.6751931Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_udf_remote 2024-08-20T23:21:07.6753175Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_user_defined 2024-08-20T23:21:07.6754621Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_register_rpc_backend_and_set_and_start_rpc_backend 2024-08-20T23:21:07.6756047Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future 2024-08-20T23:21:07.6757301Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future_async 2024-08-20T23:21:07.6758604Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future_remote 2024-08-20T23:21:07.6759890Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_all 2024-08-20T23:21:07.6761307Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_async_function_single_threaded 2024-08-20T23:21:07.6762812Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_remote_record_function 2024-08-20T23:21:07.6764178Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_return_rref 2024-08-20T23:21:07.6765423Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_timeouts 2024-08-20T23:21:07.6766639Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_leak 2024-08-20T23:21:07.6767863Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_class 2024-08-20T23:21:07.6769161Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_class_self 2024-08-20T23:21:07.6770548Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_tensor 2024-08-20T23:21:07.6771844Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_str 2024-08-20T23:21:07.6773116Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_owner_blocking 2024-08-20T23:21:07.6774497Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_with_error_blocking 2024-08-20T23:21:07.6775931Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_self_remote_arg 2024-08-20T23:21:07.6777368Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_self_rpc_arg 2024-08-20T23:21:07.6778879Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_server_process_global_profiler 2024-08-20T23:21:07.6780308Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_set_and_get_default_rpc_timeout 2024-08-20T23:21:07.6781651Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_shutdown_errors 2024-08-20T23:21:07.6782905Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rpc_pickler 2024-08-20T23:21:07.6784129Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all 2024-08-20T23:21:07.6785416Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_exit_early_python 2024-08-20T23:21:07.6786776Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_with_exception 2024-08-20T23:21:07.6788112Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_timeout 2024-08-20T23:21:07.6789388Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_worker_id 2024-08-20T23:21:07.6790778Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeParameterServerTest::test_batch_updating_parameter_server 2024-08-20T23:21:07.6792309Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_accumulate_grads 2024-08-20T23:21:07.6794011Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_autograd_engine_error 2024-08-20T23:21:07.6795553Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_complex_python_udf 2024-08-20T23:21:07.6797055Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_dtypes 2024-08-20T23:21:07.6798566Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_output_tensors 2024-08-20T23:21:07.6800088Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_roots 2024-08-20T23:21:07.6801597Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_node_failure_python_udf 2024-08-20T23:21:07.6803113Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_python_udf_error 2024-08-20T23:21:07.6804550Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_multi 2024-08-20T23:21:07.6805995Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_unused_tensors 2024-08-20T23:21:07.6807594Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_verify_hooks 2024-08-20T23:21:07.6809074Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_nested_rpc 2024-08-20T23:21:07.6810739Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_no_tensors 2024-08-20T23:21:07.6812256Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_tensor_no_grad 2024-08-20T23:21:07.6813749Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_builtin_call 2024-08-20T23:21:07.6815354Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_remote_call 2024-08-20T23:21:07.6816922Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_remote_call_itself 2024-08-20T23:21:07.6818450Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_python_call 2024-08-20T23:21:07.6819879Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_mixed_requires_grad 2024-08-20T23:21:07.6821470Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_multiple_backward_with_errors 2024-08-20T23:21:07.6822891Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_grad_copy 2024-08-20T23:21:07.6824271Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_grad_copy_sparse 2024-08-20T23:21:07.6825783Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_graph_with_tensors_not_require_grad 2024-08-20T23:21:07.6827429Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_graph_with_tensors_not_require_grad_remote 2024-08-20T23:21:07.6828969Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_remote_complex_args 2024-08-20T23:21:07.6830480Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_thread_local_context_id 2024-08-20T23:21:07.6831869Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_trainer_ps 2024-08-20T23:21:07.6833308Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_trainer_ps_torchscript_functions 2024-08-20T23:21:07.6834785Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_worker_ids_recorded 2024-08-20T23:21:07.6836258Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_wrong_decorator_order 2024-08-20T23:21:07.6837786Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_wrong_return_type_remote 2024-08-20T23:21:07.6839402Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_python_function_remotely_from_script_not_supported 2024-08-20T23:21:07.6840950Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_rpc_with_profiling 2024-08-20T23:21:07.6842480Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_script_function_that_raises_remotely_from_script 2024-08-20T23:21:07.6844084Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_create_local_script_class_rref_in_py 2024-08-20T23:21:07.6845657Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_create_local_script_module_rref_in_py 2024-08-20T23:21:07.6847157Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_create_script_module_on_remote 2024-08-20T23:21:07.6848644Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_future_passed_between_python_and_jit 2024-08-20T23:21:07.6850161Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_future_python_annotation 2024-08-20T23:21:07.6851615Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_load_script_module_with_pickled_rref 2024-08-20T23:21:07.6853153Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_my_script_module_with_rrefs 2024-08-20T23:21:07.6854626Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_record_function_on_caller_rpc_async 2024-08-20T23:21:07.6856217Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_return_local_script_class_rref_in_py_and_use_in_script 2024-08-20T23:21:07.6857727Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rpc_async_jit_profiled 2024-08-20T23:21:07.6859070Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_list_mutate 2024-08-20T23:21:07.6860376Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_local_value 2024-08-20T23:21:07.6861714Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_python_annotation 2024-08-20T23:21:07.6863096Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_user_rrefs_confirmed_remote 2024-08-20T23:21:07.6864491Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_async 2024-08-20T23:21:07.6865877Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_async_script 2024-08-20T23:21:07.6867295Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_with_kwargs 2024-08-20T23:21:07.6868678Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_get_module_rref 2024-08-20T23:21:07.6870197Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_remote_module_py_pickle_not_supported_script 2024-08-20T23:21:07.6871736Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_remote_parameters 2024-08-20T23:21:07.6873139Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_comparison 2024-08-20T23:21:07.6874588Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_comparison_uneven_inputs 2024-08-20T23:21:07.6876148Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_builtin_remote_self_sparse 2024-08-20T23:21:07.6877776Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_and_static_init_rpc_together 2024-08-20T23:21:07.6879546Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_existing_rank_can_communicate_with_new_rank 2024-08-20T23:21:07.6881322Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_infer_backend_from_options 2024-08-20T23:21:07.6882856Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rpc_sparse 2024-08-20T23:21:07.6884357Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_op_with_invalid_args 2024-08-20T23:21:07.6885918Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_sparse_tensors_in_container 2024-08-20T23:21:07.6887554Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_remote_arg_sparse 2024-08-20T23:21:07.6889293Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_rpc_arg_sparse 2024-08-20T23:21:07.6891031Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_self_rpc_arg_sparse 2024-08-20T23:21:07.6892628Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_send_to_rank_sparse 2024-08-20T23:21:07.6894334Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_stress_heavy_rpc_sparse 2024-08-20T23:21:07.6895990Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_tensorpipe_options_throw_on_timedelta_timeout 2024-08-20T23:21:07.6897684Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_tensorpipe_set_default_timeout 2024-08-20T23:21:07.6899294Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_wait_all_workers_twice_sparse 2024-08-20T23:21:07.6900976Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_different_dtypes_sparse 2024-08-20T23:21:07.6902756Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_multiple_round_trips_sparse 2024-08-20T23:21:07.6904508Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_nested_sparse 2024-08-20T23:21:07.6906238Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_python_udf_sparse 2024-08-20T23:21:07.6908003Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_script_call_sparse 2024-08-20T23:21:07.6909785Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backwards_nested_python_udf_sparse 2024-08-20T23:21:07.6911681Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_embedding_bag_with_no_grad_tensors 2024-08-20T23:21:07.6913466Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_remote_call_sparse 2024-08-20T23:21:07.6915220Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_call_sparse 2024-08-20T23:21:07.6916998Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_remote_call_sparse 2024-08-20T23:21:07.6918815Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_python_remote_call_sparse 2024-08-20T23:21:07.6920747Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_mixed_requires_grad_sparse 2024-08-20T23:21:07.6922442Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_remote_complex_args_sparse 2024-08-20T23:21:07.6923383Z 2024-08-20T23:21:07.6923935Z Running distributed/test_c10d_ucc 1/1 ... [2024-08-20 23:21:07.654187] 2024-08-20T23:21:07.6924513Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:21:07.6926307Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_ucc.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:21:07.654671] 2024-08-20T23:21:09.2867179Z 2024-08-20T23:21:09.2868764Z distributed/test_c10d_ucc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_ucc_1.1_a4f035ad4a6de4dd_.log 2024-08-20T23:21:09.2869725Z 2024-08-20T23:21:09.2873273Z Running distributed/elastic/timer/api_test 1/1 ... [2024-08-20 23:21:09.287128] 2024-08-20T23:21:09.2873908Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:21:09.2878331Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/timer/api_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:21:09.287557] 2024-08-20T23:21:10.7622244Z 2024-08-20T23:21:10.7624182Z distributed/elastic/timer/api_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.timer.api_test_1.1_ed2909ab65472812_.log 2024-08-20T23:21:10.7625300Z 2024-08-20T23:21:11.2527280Z Running test batch 'tests to run' cost 5811.92 seconds 2024-08-20T23:21:11.7495281Z 2024-08-20T23:21:11.7495936Z real 96m56.531s 2024-08-20T23:21:11.7496449Z user 215m33.512s 2024-08-20T23:21:11.7496742Z sys 39m8.199s 2024-08-20T23:21:11.7497023Z + assert_git_not_dirty 2024-08-20T23:21:11.7497619Z + [[ linux-jammy-py3.8-gcc11 != *rocm* ]] 2024-08-20T23:21:11.7498125Z + [[ linux-jammy-py3.8-gcc11 != *xla* ]] 2024-08-20T23:21:11.7503346Z ++ git status --porcelain 2024-08-20T23:21:11.7504119Z ++ grep -v '?? third_party' 2024-08-20T23:21:29.9998644Z ++ true 2024-08-20T23:21:29.9999313Z + git_status= 2024-08-20T23:21:30.0000142Z + [[ -n '' ]] 2024-08-20T23:21:30.0000543Z + [[ linux-jammy-py3.8-gcc11 == *cuda* ]] 2024-08-20T23:21:30.0001040Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-20T23:21:30.0001443Z + [[ 1 == 1 ]] 2024-08-20T23:21:30.0001736Z + test_rpc 2024-08-20T23:21:30.0002042Z + echo 'Testing RPC C++ tests' 2024-08-20T23:21:30.0002403Z Testing RPC C++ tests 2024-08-20T23:21:30.0005618Z + ln -sf /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libtorch.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libtorch_global_deps.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libtorch_python.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libtorchbind_test.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-08-20T23:21:30.0025062Z + ln -sf /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libc10.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-08-20T23:21:30.0034688Z + CPP_TESTS_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-08-20T23:21:30.0035479Z + python test/run_test.py --cpp --verbose -i cpp/test_cpp_rpc 2024-08-20T23:21:30.1038097Z test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-20T23:21:30.1039097Z import pkg_resources 2024-08-20T23:21:33.3106104Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-20T23:21:33.3214416Z Found test times from artifacts 2024-08-20T23:21:33.3655734Z Found test times from artifacts 2024-08-20T23:21:33.3674985Z Running all tests 2024-08-20T23:21:33.3678437Z Running parallel tests on 3 processes 2024-08-20T23:21:33.3678979Z Name: tests to run (est. time: 0.0min) 2024-08-20T23:21:33.3679421Z Serial tests (1): 2024-08-20T23:21:33.3679755Z cpp/test_cpp_rpc 1/1 2024-08-20T23:21:33.3680085Z Parallel tests (0): 2024-08-20T23:21:33.3680477Z Name: excluded (est. time: 0.0min) 2024-08-20T23:21:33.3680836Z Serial tests (0): 2024-08-20T23:21:33.3681191Z Parallel tests (0): 2024-08-20T23:21:33.3908242Z Running cpp/test_cpp_rpc 1/1 ... [2024-08-20 23:21:33.390320] 2024-08-20T23:21:33.3909269Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:21:33.3920700Z Executing ['pytest', '/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin/test_cpp_rpc', '-v', '-vv', '-rfEX', '-n', '3', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-3f0478fe07d20784.xml', '-x', '--reruns=2'] ... [2024-08-20 23:21:33.391479] 2024-08-20T23:21:38.2827085Z 2024-08-20T23:21:38.2828742Z cpp/test_cpp_rpc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.test_cpp_rpc_1.1_df4a3e8372971af1_.log 2024-08-20T23:21:38.2829719Z 2024-08-20T23:21:38.8086773Z Running test batch 'tests to run' cost 5.44 seconds 2024-08-20T23:21:39.2905769Z + cleanup_workspace 2024-08-20T23:21:39.2907207Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-20T23:21:39.2908328Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-20T23:21:39.2909281Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-20T23:21:39.2909920Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-20T23:21:39.2910754Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-20T23:21:39.2911647Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-20T23:21:39.2912330Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-20T23:21:39.7639900Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-20T23:21:39.7640404Z cat test/**/*_toprint.log || true 2024-08-20T23:21:39.7649778Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:21:39.7650269Z env: 2024-08-20T23:21:39.7650533Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:21:39.7651149Z DOCKER_CONTAINER_ID: a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:39.7651791Z ##[endgroup] 2024-08-20T23:21:39.7730624Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-20T23:21:39.7763090Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:21:39.7763538Z kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:21:39.7769753Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:21:39.7770255Z env: 2024-08-20T23:21:39.7770517Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:21:39.7771114Z DOCKER_CONTAINER_ID: a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:39.7771790Z MONITOR_SCRIPT_PID: 46615 2024-08-20T23:21:39.7772127Z ##[endgroup] 2024-08-20T23:21:39.7975835Z Prepare all required actions 2024-08-20T23:21:39.7976299Z Getting action download info 2024-08-20T23:21:39.9923390Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-20T23:21:40.1532357Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-20T23:21:40.1532811Z with: 2024-08-20T23:21:40.1533267Z file-suffix: test-distributed-1-2-amz2023.linux.2xlarge_29025446952 2024-08-20T23:21:40.1533862Z s3-bucket: gha-artifacts 2024-08-20T23:21:40.1534168Z env: 2024-08-20T23:21:40.1534426Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:21:40.1535030Z DOCKER_CONTAINER_ID: a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:40.1535674Z ##[endgroup] 2024-08-20T23:21:40.1563648Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-20T23:21:40.1564278Z # Remove any previous test jsons if they exist 2024-08-20T23:21:40.1564786Z rm -f test-jsons-*.zip 2024-08-20T23:21:40.1565335Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-20T23:21:40.1571544Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:21:40.1572039Z env: 2024-08-20T23:21:40.1572294Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:21:40.1572901Z DOCKER_CONTAINER_ID: a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:40.1574012Z FILE_SUFFIX: test-distributed-1-2-amz2023.linux.2xlarge_29025446952 2024-08-20T23:21:40.1574565Z ##[endgroup] 2024-08-20T23:21:40.1847474Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-20T23:21:40.1875022Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-20T23:21:40.1875722Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-20T23:21:40.1881905Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-20T23:21:40.1887415Z adding: test/slow_tests.json (deflated 82%) 2024-08-20T23:21:40.1890166Z adding: test/test-reports/td_exclusions-88db862e97754ab99164.json (deflated 86%) 2024-08-20T23:21:40.1891313Z adding: test/test-reports/td_exclusions-f11f66eecfad7da9eba6.json (deflated 18%) 2024-08-20T23:21:40.1902384Z adding: test/.pytorch-disabled-tests.json (deflated 88%) 2024-08-20T23:21:40.1934008Z ##[group]Run # Remove any previous test reports if they exist 2024-08-20T23:21:40.1934658Z # Remove any previous test reports if they exist 2024-08-20T23:21:40.1935189Z rm -f test-reports-*.zip 2024-08-20T23:21:40.1935768Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-20T23:21:40.1942028Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:21:40.1942519Z env: 2024-08-20T23:21:40.1942770Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:21:40.1943366Z DOCKER_CONTAINER_ID: a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:40.1944208Z FILE_SUFFIX: test-distributed-1-2-amz2023.linux.2xlarge_29025446952 2024-08-20T23:21:40.1944781Z ##[endgroup] 2024-08-20T23:21:40.2204285Z adding: test/test-reports/python-pytest/distributed.test_inductor_collectives/distributed.test_inductor_collectives-f73cb9378876a201.xml (deflated 94%) 2024-08-20T23:21:40.2207258Z adding: test/test-reports/python-pytest/distributed._tensor.test_dtensor_compile/distributed._tensor.test_dtensor_compile-c2804c1f28b5c3fc.xml (deflated 85%) 2024-08-20T23:21:40.2209793Z adding: test/test-reports/python-pytest/distributed.test_data_parallel/distributed.test_data_parallel-6e3ab2e6b0de4bdc.xml (deflated 94%) 2024-08-20T23:21:40.2212560Z adding: test/test-reports/python-pytest/distributed._composable.test_replicate_with_compiler/distributed._composable.test_replicate_with_compiler-238ab81f4ad9a280.xml (deflated 86%) 2024-08-20T23:21:40.2214774Z adding: test/test-reports/python-pytest/distributed._tensor.debug.test_op_coverage/distributed._tensor.debug.test_op_coverage-5a2bd94ae991c1c6.xml (deflated 34%) 2024-08-20T23:21:40.2216938Z adding: test/test-reports/python-pytest/distributed.test_c10d_functional_native/distributed.test_c10d_functional_native-c804f6862ed82ded.xml (deflated 93%) 2024-08-20T23:21:40.2218817Z adding: test/test-reports/python-pytest/distributed.pipelining.test_transformer/distributed.pipelining.test_transformer-6ad6a5814470b6d9.xml (deflated 39%) 2024-08-20T23:21:40.2226382Z adding: test/test-reports/python-pytest/distributed._tensor.test_dtensor_ops/distributed._tensor.test_dtensor_ops-f7b8dce1d34ecf1f.xml (deflated 96%) 2024-08-20T23:21:40.2228421Z adding: test/test-reports/python-pytest/distributed.flight_recorder.test_fr_analysis/distributed.flight_recorder.test_fr_analysis-1a8a000d88507fe2.xml (deflated 36%) 2024-08-20T23:21:40.2230402Z adding: test/test-reports/python-pytest/distributed.pipelining.test_pipe/distributed.pipelining.test_pipe-68107221a7b18adf.xml (deflated 65%) 2024-08-20T23:21:40.2232540Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_micro_pipeline_tp/distributed.tensor.parallel.test_micro_pipeline_tp-e008b4e76f822dd4.xml (deflated 95%) 2024-08-20T23:21:40.2234755Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_sharded_grad_scaler/distributed.fsdp.test_fsdp_sharded_grad_scaler-05886011f95117c3.xml (deflated 93%) 2024-08-20T23:21:40.2236645Z adding: test/test-reports/python-pytest/distributed.launcher.test_run/distributed.launcher.test_run-453b0ecd29b830af.xml (deflated 84%) 2024-08-20T23:21:40.2238737Z adding: test/test-reports/python-pytest/distributed.pipelining.test_microbatch/distributed.pipelining.test_microbatch-0f441699cd4ae1b0.xml (deflated 54%) 2024-08-20T23:21:40.2240715Z adding: test/test-reports/python-pytest/distributed._composable.test_contract/distributed._composable.test_contract-51daba30e531cdde.xml (deflated 69%) 2024-08-20T23:21:40.2242625Z adding: test/test-reports/python-pytest/distributed._tensor.test_common_rules/distributed._tensor.test_common_rules-2231d6e71e9495e0.xml (deflated 80%) 2024-08-20T23:21:40.2244523Z adding: test/test-reports/python-pytest/distributed._tensor.test_random_ops/distributed._tensor.test_random_ops-ce5a3b8bfcc4c754.xml (deflated 82%) 2024-08-20T23:21:40.2246652Z adding: test/test-reports/python-pytest/distributed._tensor.experimental.test_register_sharding/distributed._tensor.experimental.test_register_sharding-92e2f5a7cd6ca00e.xml (deflated 52%) 2024-08-20T23:21:40.2248600Z adding: test/test-reports/python-pytest/distributed._tensor.test_api/distributed._tensor.test_api-eb1eb99776d0fe1c.xml (deflated 77%) 2024-08-20T23:21:40.2250765Z adding: test/test-reports/python-pytest/distributed._tensor.test_experimental_ops/distributed._tensor.test_experimental_ops-36d854eb83a3f2bf.xml (deflated 60%) 2024-08-20T23:21:40.2252689Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_fx/distributed.fsdp.test_fsdp_fx-78850ae84daa4734.xml (deflated 37%) 2024-08-20T23:21:40.2254477Z adding: test/test-reports/python-pytest/distributed._tensor.test_tensor_ops/distributed._tensor.test_tensor_ops-405595fa45032f96.xml (deflated 85%) 2024-08-20T23:21:40.2256202Z adding: test/test-reports/python-pytest/distributed.test_device_mesh/distributed.test_device_mesh-bdea37b9603274c7.xml (deflated 85%) 2024-08-20T23:21:40.2258153Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_overlap/distributed._composable.fsdp.test_fully_shard_overlap-148449185f782e8b.xml (deflated 55%) 2024-08-20T23:21:40.2260349Z adding: test/test-reports/python-pytest/distributed._tensor.experimental.test_tp_transform/distributed._tensor.experimental.test_tp_transform-f78ba639e0acc592.xml (deflated 62%) 2024-08-20T23:21:40.2262492Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_tp_style/distributed.tensor.parallel.test_tp_style-ccc94b87411594e7.xml (deflated 80%) 2024-08-20T23:21:40.2264511Z adding: test/test-reports/python-pytest/distributed._tensor.test_redistribute/distributed._tensor.test_redistribute-773ba687904a039c.xml (deflated 82%) 2024-08-20T23:21:40.2266498Z adding: test/test-reports/python-pytest/distributed._tensor.test_matrix_ops/distributed._tensor.test_matrix_ops-099feefa4d490770.xml (deflated 77%) 2024-08-20T23:21:40.2268318Z adding: test/test-reports/python-pytest/distributed._tensor.test_math_ops/distributed._tensor.test_math_ops-0b7f53d772d4ea93.xml (deflated 79%) 2024-08-20T23:21:40.2270306Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_tp_examples/distributed.tensor.parallel.test_tp_examples-0164bed64cea2e49.xml (deflated 83%) 2024-08-20T23:21:40.2272413Z adding: test/test-reports/python-pytest/distributed.checkpoint.e2e.test_pipeline/distributed.checkpoint.e2e.test_pipeline-5ac3af66c099fca9.xml (deflated 38%) 2024-08-20T23:21:40.2274601Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_state/distributed._composable.fsdp.test_fully_shard_state-dfec03c65d2dac6f.xml (deflated 81%) 2024-08-20T23:21:40.2276918Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_compile/distributed._composable.fully_shard.test_fully_shard_compile-0f467c55d5fe2cba.xml (deflated 49%) 2024-08-20T23:21:40.2279184Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_state_dict_utils/distributed.checkpoint.test_state_dict_utils-0b58cc3d2bdac487.xml (deflated 71%) 2024-08-20T23:21:40.2281215Z adding: test/test-reports/python-pytest/distributed.pipelining.test_schedule/distributed.pipelining.test_schedule-1dd2490ffe3c2344.xml (deflated 83%) 2024-08-20T23:21:40.2283490Z adding: test/test-reports/python-pytest/distributed._tensor.test_embedding_ops/distributed._tensor.test_embedding_ops-4b578534b174c488.xml (deflated 67%) 2024-08-20T23:21:40.2285332Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_utils/distributed.checkpoint.test_utils-9572b3ecf48a519f.xml (deflated 71%) 2024-08-20T23:21:40.2287280Z adding: test/test-reports/python-pytest/distributed._tensor.debug.test_comm_mode/distributed._tensor.debug.test_comm_mode-2f873bced72048ab.xml (deflated 60%) 2024-08-20T23:21:40.2289413Z adding: test/test-reports/python-pytest/distributed.optim.test_zero_redundancy_optimizer/distributed.optim.test_zero_redundancy_optimizer-df127a5cf1c91121.xml (deflated 94%) 2024-08-20T23:21:40.2292978Z adding: test/test-reports/python-pytest/distributed.fsdp.test_shard_utils/distributed.fsdp.test_shard_utils-6c88f3775b06b15e.xml (deflated 63%) 2024-08-20T23:21:40.2296928Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_fsdp_model_state/distributed.checkpoint.test_fsdp_model_state-1ad8140d432ecc89.xml (deflated 65%) 2024-08-20T23:21:40.2299520Z adding: test/test-reports/python-pytest/distributed.rpc.test_share_memory/distributed.rpc.test_share_memory-1aaa192c39ac8eae.xml (deflated 47%) 2024-08-20T23:21:40.2301435Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_checkpoint/distributed.checkpoint.test_checkpoint-f26dbc6b6017fbef.xml (deflated 84%) 2024-08-20T23:21:40.2303225Z adding: test/test-reports/python-pytest/distributed.test_nccl/distributed.test_nccl-b6b33afb28e4a5d4.xml (deflated 28%) 2024-08-20T23:21:40.2305398Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_mixed_precision/distributed._composable.fully_shard.test_fully_shard_mixed_precision-9d212f223df03f2f.xml (deflated 47%) 2024-08-20T23:21:40.2307954Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_util/distributed._composable.fully_shard.test_fully_shard_util-5c3fe656d22713b3.xml (deflated 47%) 2024-08-20T23:21:40.2310144Z adding: test/test-reports/python-pytest/distributed._composable.test_replicate/distributed._composable.test_replicate-8d7ef429d6058601.xml (deflated 87%) 2024-08-20T23:21:40.2312155Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_state_dict/distributed.checkpoint.test_state_dict-e461f01bb98dfc4b.xml (deflated 85%) 2024-08-20T23:21:40.2314447Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_dtensor_state_dict/distributed.fsdp.test_fsdp_dtensor_state_dict-d9589240c3ed51bd.xml (deflated 92%) 2024-08-20T23:21:40.2316362Z adding: test/test-reports/python-pytest/distributed.test_c10d_logger/distributed.test_c10d_logger-e88f3bb17920e4cb.xml (deflated 65%) 2024-08-20T23:21:40.2318244Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_state_dict/distributed.fsdp.test_fsdp_state_dict-6c46c2dd1184a415.xml (deflated 97%) 2024-08-20T23:21:40.2320460Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_autograd/distributed._composable.fsdp.test_fully_shard_autograd-a89c25b5b1aa347a.xml (deflated 68%) 2024-08-20T23:21:40.2322819Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_file_system_checkpoint/distributed.checkpoint.test_file_system_checkpoint-aace9d84f2e5539f.xml (deflated 79%) 2024-08-20T23:21:40.2325291Z adding: test/test-reports/python-pytest/distributed._composable.test_composability.test_2d_composability/distributed._composable.test_composability.test_2d_composability-8f9b002c9cc93702.xml (deflated 89%) 2024-08-20T23:21:40.2327767Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_frozen/distributed._composable.fsdp.test_fully_shard_frozen-233d23a458b576dc.xml (deflated 62%) 2024-08-20T23:21:40.2330531Z adding: test/test-reports/python-pytest/distributed.fsdp.test_distributed_checkpoint/distributed.fsdp.test_distributed_checkpoint-44755ddcbcf86e30.xml (deflated 77%) 2024-08-20T23:21:40.2335074Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_mixed_precision/distributed._composable.fsdp.test_fully_shard_mixed_precision-a6f550d67d1baeaf.xml (deflated 83%) 2024-08-20T23:21:40.2339098Z adding: test/test-reports/python-pytest/distributed._tensor.test_utils/distributed._tensor.test_utils-f35bef092c0f5423.xml (deflated 78%) 2024-08-20T23:21:40.2343428Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_model_checkpoint/distributed._composable.fully_shard.test_fully_shard_model_checkpoint-f6233a3e68a4e8f9.xml (deflated 80%) 2024-08-20T23:21:40.2347774Z adding: test/test-reports/python-pytest/distributed.test_c10d_object_collectives/distributed.test_c10d_object_collectives-6187bbd1ec32a92c.xml (deflated 80%) 2024-08-20T23:21:40.2351844Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_init/distributed._composable.fully_shard.test_fully_shard_init-cdfe37ea11174284.xml (deflated 85%) 2024-08-20T23:21:40.2355884Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_tp_integration/distributed.fsdp.test_fsdp_tp_integration-e732700ce7ecdf49.xml (deflated 72%) 2024-08-20T23:21:40.2359469Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_grad_acc/distributed.fsdp.test_fsdp_grad_acc-2ed41058dbeec47a.xml (deflated 84%) 2024-08-20T23:21:40.2362863Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_freezing_weights/distributed.fsdp.test_fsdp_freezing_weights-ae58a2a4507f65d7.xml (deflated 96%) 2024-08-20T23:21:40.2366419Z adding: test/test-reports/python-pytest/distributed.test_multi_threaded_pg/distributed.test_multi_threaded_pg-7ae546141684b4f3.xml (deflated 81%) 2024-08-20T23:21:40.2370247Z adding: test/test-reports/python-pytest/distributed.algorithms.ddp_comm_hooks.test_ddp_hooks/distributed.algorithms.ddp_comm_hooks.test_ddp_hooks-65f6bda55f284e39.xml (deflated 85%) 2024-08-20T23:21:40.2374208Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_flatten_params/distributed.fsdp.test_fsdp_flatten_params-3246904c26db779b.xml (deflated 84%) 2024-08-20T23:21:40.2378137Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_file_system_checkpoint_cpu/distributed.checkpoint.test_file_system_checkpoint_cpu-8259f4d1f7a9818d.xml (deflated 78%) 2024-08-20T23:21:40.2381842Z adding: test/test-reports/python-pytest/distributed.test_c10d_pypg/distributed.test_c10d_pypg-225e085b04744692.xml (deflated 94%) 2024-08-20T23:21:40.2385495Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_extensions/distributed._composable.fsdp.test_fully_shard_extensions-21ba1d8d3d56784a.xml (deflated 71%) 2024-08-20T23:21:40.2389223Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_memory/distributed.fsdp.test_fsdp_memory-4ae3fc2a91768d24.xml (deflated 64%) 2024-08-20T23:21:40.2392796Z adding: test/test-reports/python-pytest/distributed.fsdp.test_hsdp_dtensor_state_dict/distributed.fsdp.test_hsdp_dtensor_state_dict-6c27ff6f166cfcb0.xml (deflated 86%) 2024-08-20T23:21:40.2397899Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_runtime/distributed._composable.fully_shard.test_fully_shard_runtime-cb8aaebe95a8a25a.xml (deflated 54%) 2024-08-20T23:21:40.2401816Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_hybrid_shard/distributed.fsdp.test_fsdp_hybrid_shard-a597a62684f84bc0.xml (deflated 80%) 2024-08-20T23:21:40.2405244Z adding: test/test-reports/python-pytest/distributed.algorithms.test_join/distributed.algorithms.test_join-729926cc7f299af5.xml (deflated 79%) 2024-08-20T23:21:40.2408462Z adding: test/test-reports/python-pytest/distributed.fsdp.test_utils/distributed.fsdp.test_utils-9fb15e14b62fabe4.xml (deflated 70%) 2024-08-20T23:21:40.2411667Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_core/distributed.fsdp.test_fsdp_core-01110a2bd1319db1.xml (deflated 95%) 2024-08-20T23:21:40.2415324Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_checkpoint/distributed.fsdp.test_fsdp_checkpoint-0dadcb2e36dac4cc.xml (deflated 93%) 2024-08-20T23:21:40.2419150Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_use_orig_params/distributed.fsdp.test_fsdp_use_orig_params-a4462905ac3024c6.xml (deflated 90%) 2024-08-20T23:21:40.2423036Z adding: test/test-reports/python-pytest/distributed._shard.sharding_spec.test_sharding_spec/distributed._shard.sharding_spec.test_sharding_spec-db2817039ec5ca3b.xml (deflated 83%) 2024-08-20T23:21:40.2426697Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_apply/distributed.fsdp.test_fsdp_apply-81e39d744b229d39.xml (deflated 71%) 2024-08-20T23:21:40.2430380Z adding: test/test-reports/python-pytest/distributed._shard.sharding_plan.test_sharding_plan/distributed._shard.sharding_plan.test_sharding_plan-7b1f31d15feb1051.xml (deflated 75%) 2024-08-20T23:21:40.2433894Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-01148bae7a121b44.xml (deflated 44%) 2024-08-20T23:21:40.2436955Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-6363b9896f8a25d2.xml (deflated 44%) 2024-08-20T23:21:40.2440008Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-f980de54a0fa3d49.xml (deflated 44%) 2024-08-20T23:21:40.2443070Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-60a3a30aea7f174d.xml (deflated 43%) 2024-08-20T23:21:40.2446091Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-d52eb6e2b44e490f.xml (deflated 44%) 2024-08-20T23:21:40.2449090Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-f143b5739a0074e1.xml (deflated 44%) 2024-08-20T23:21:40.2452182Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-a2a69d6c5c6d9e1b.xml (deflated 44%) 2024-08-20T23:21:40.2455199Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-991f03917eeae959.xml (deflated 44%) 2024-08-20T23:21:40.2458234Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-e61df6f54d767af6.xml (deflated 34%) 2024-08-20T23:21:40.2461466Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-1256887d0bfc13df.xml (deflated 42%) 2024-08-20T23:21:40.2464498Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-c856708f1423e2ba.xml (deflated 42%) 2024-08-20T23:21:40.2467533Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-b17d79fe085026c2.xml (deflated 34%) 2024-08-20T23:21:40.2470538Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-b0e7c77aa706125f.xml (deflated 34%) 2024-08-20T23:21:40.2473573Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-ea2438d8c3f1b33a.xml (deflated 35%) 2024-08-20T23:21:40.2476596Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-c9d202c3504d9697.xml (deflated 42%) 2024-08-20T23:21:40.2479598Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-28db73acdecd1621.xml (deflated 42%) 2024-08-20T23:21:40.2482594Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-d510906ac783d9b0.xml (deflated 34%) 2024-08-20T23:21:40.2485758Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_input/distributed.fsdp.test_fsdp_input-c5b6325e03f5ea6d.xml (deflated 63%) 2024-08-20T23:21:40.2489303Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_multiple_wrapping/distributed.fsdp.test_fsdp_multiple_wrapping-4e99cec6d74c964c.xml (deflated 47%) 2024-08-20T23:21:40.2493104Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_exec_order/distributed.fsdp.test_fsdp_exec_order-f94b3a8813b42e63.xml (deflated 87%) 2024-08-20T23:21:40.2495711Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_mixed_precision/distributed.fsdp.test_fsdp_mixed_precision-c6d4e9bcef6ee166.xml (deflated 95%) 2024-08-20T23:21:40.2497583Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_fine_tune/distributed.fsdp.test_fsdp_fine_tune-05d94bdd1fb83593.xml (deflated 64%) 2024-08-20T23:21:40.2499387Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_traversal/distributed.fsdp.test_fsdp_traversal-1f5d76c2a59bb18d.xml (deflated 45%) 2024-08-20T23:21:40.2501395Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_tensor_ops/distributed._shard.sharded_tensor.ops.test_tensor_ops-1ff473562ea72c5a.xml (deflated 83%) 2024-08-20T23:21:40.2504384Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint-aadf20042000da17.xml (deflated 67%) 2024-08-20T23:21:40.2506680Z adding: test/test-reports/python-pytest/distributed._shard.sharded_optim.test_sharded_optim/distributed._shard.sharded_optim.test_sharded_optim-f1380b17a4da30b5.xml (deflated 67%) 2024-08-20T23:21:40.2508630Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4bd83937df10ef48.xml (deflated 36%) 2024-08-20T23:21:40.2510450Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c7f35b5104fc8c97.xml (deflated 35%) 2024-08-20T23:21:40.2512248Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f768142ba26011cd.xml (deflated 36%) 2024-08-20T23:21:40.2514052Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-775b24341930e9ff.xml (deflated 36%) 2024-08-20T23:21:40.2515851Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-63b7e1c73d4f1829.xml (deflated 36%) 2024-08-20T23:21:40.2517665Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f5a64c4379eaee4a.xml (deflated 36%) 2024-08-20T23:21:40.2519601Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fbc671c1f601c0b2.xml (deflated 36%) 2024-08-20T23:21:40.2521415Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1c15a33433771203.xml (deflated 36%) 2024-08-20T23:21:40.2523224Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5dddbae9af274cc3.xml (deflated 37%) 2024-08-20T23:21:40.2525043Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a1f58e3a614d9c23.xml (deflated 36%) 2024-08-20T23:21:40.2526839Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a0eaf781bc87930f.xml (deflated 35%) 2024-08-20T23:21:40.2528670Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f9ef2e3959a0475f.xml (deflated 36%) 2024-08-20T23:21:40.2530577Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-148f50b0bcb94bdc.xml (deflated 36%) 2024-08-20T23:21:40.2532390Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-88089046d0e58ed6.xml (deflated 36%) 2024-08-20T23:21:40.2534292Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ad7d8df8dface98e.xml (deflated 36%) 2024-08-20T23:21:40.2536111Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-edc3f3d4ed7c6325.xml (deflated 35%) 2024-08-20T23:21:40.2537922Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ea393b5c87054045.xml (deflated 36%) 2024-08-20T23:21:40.2539732Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4e7bb900c8698274.xml (deflated 36%) 2024-08-20T23:21:40.2541534Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e8427a12d018190d.xml (deflated 35%) 2024-08-20T23:21:40.2543330Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-dca65e0c73eaa826.xml (deflated 36%) 2024-08-20T23:21:40.2545147Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8c5ad1c1088d83f9.xml (deflated 36%) 2024-08-20T23:21:40.2546962Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a7eac73efe62f052.xml (deflated 36%) 2024-08-20T23:21:40.2548780Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ce8ebe8f5d390cd0.xml (deflated 36%) 2024-08-20T23:21:40.2550581Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fa22b46e7ff77c41.xml (deflated 36%) 2024-08-20T23:21:40.2552388Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-80246f24b42bf7b6.xml (deflated 35%) 2024-08-20T23:21:40.2554194Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e3b36a25b950c86a.xml (deflated 35%) 2024-08-20T23:21:40.2555999Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b6739fa9dcb75a74.xml (deflated 36%) 2024-08-20T23:21:40.2557796Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c8c6a4cb6473e59f.xml (deflated 36%) 2024-08-20T23:21:40.2559671Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a51d8d0ff9ee6f48.xml (deflated 35%) 2024-08-20T23:21:40.2561480Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fca6b949654d63e5.xml (deflated 36%) 2024-08-20T23:21:40.2563280Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7b81c64c8bab84e2.xml (deflated 36%) 2024-08-20T23:21:40.2565081Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-dc5e8b6e0115fce0.xml (deflated 35%) 2024-08-20T23:21:40.2566896Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1cd637d9aeb12d1e.xml (deflated 35%) 2024-08-20T23:21:40.2568702Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2245725c1d9bd78e.xml (deflated 35%) 2024-08-20T23:21:40.2570591Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b257d7a2d68d601d.xml (deflated 36%) 2024-08-20T23:21:40.2572389Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6b555b0bc5231470.xml (deflated 36%) 2024-08-20T23:21:40.2574201Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3744a5370eb39292.xml (deflated 36%) 2024-08-20T23:21:40.2576094Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6b37eef18eafc470.xml (deflated 36%) 2024-08-20T23:21:40.2577902Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ea6a0db548df041f.xml (deflated 36%) 2024-08-20T23:21:40.2579707Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d8365095ccd9598e.xml (deflated 36%) 2024-08-20T23:21:40.2581528Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-911a20b27e0bfdaa.xml (deflated 56%) 2024-08-20T23:21:40.2583332Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-207657de23e906d1.xml (deflated 36%) 2024-08-20T23:21:40.2585150Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8cf72ce953a1f314.xml (deflated 35%) 2024-08-20T23:21:40.2586963Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c324fe0c6de9c245.xml (deflated 35%) 2024-08-20T23:21:40.2588778Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-59877500e63c27ed.xml (deflated 36%) 2024-08-20T23:21:40.2590590Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3df01dd41183c58f.xml (deflated 35%) 2024-08-20T23:21:40.2592416Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-49e1f0af5c412faf.xml (deflated 35%) 2024-08-20T23:21:40.2594434Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9616fe72e80b772b.xml (deflated 37%) 2024-08-20T23:21:40.2596271Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a813a0779a645daa.xml (deflated 36%) 2024-08-20T23:21:40.2598079Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4f8086968f8b56a1.xml (deflated 36%) 2024-08-20T23:21:40.2599889Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7f96c2abaf325a2a.xml (deflated 36%) 2024-08-20T23:21:40.2601879Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-abf4ef92fab0a8cc.xml (deflated 36%) 2024-08-20T23:21:40.2603699Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e0e147317d332b4b.xml (deflated 36%) 2024-08-20T23:21:40.2605519Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-22cf0781422bd3eb.xml (deflated 37%) 2024-08-20T23:21:40.2607434Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0f7b8bf509b191d8.xml (deflated 36%) 2024-08-20T23:21:40.2609231Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1b798d7ee0c81865.xml (deflated 36%) 2024-08-20T23:21:40.2611104Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-85a76fc63508b707.xml (deflated 36%) 2024-08-20T23:21:40.2612905Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-044c7e2cef19d134.xml (deflated 36%) 2024-08-20T23:21:40.2614717Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-09b1eafdd2fa6506.xml (deflated 36%) 2024-08-20T23:21:40.2616622Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-03c76388fce7a202.xml (deflated 36%) 2024-08-20T23:21:40.2618424Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-37887191c5ef5c43.xml (deflated 36%) 2024-08-20T23:21:40.2620227Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6f4e91f47476db91.xml (deflated 36%) 2024-08-20T23:21:40.2622050Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-33819e9f6c6b7139.xml (deflated 35%) 2024-08-20T23:21:40.2623855Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3f2badc6114dd430.xml (deflated 35%) 2024-08-20T23:21:40.2625672Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3d69dc613cde6983.xml (deflated 36%) 2024-08-20T23:21:40.2627492Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-285c24cab9574e6b.xml (deflated 36%) 2024-08-20T23:21:40.2629313Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c0e6af9d7e9e202d.xml (deflated 36%) 2024-08-20T23:21:40.2631136Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2f672fc5b4c76ca8.xml (deflated 36%) 2024-08-20T23:21:40.2632939Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b3ff8056e2f5310b.xml (deflated 36%) 2024-08-20T23:21:40.2634752Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d3b73ff40f1eafd6.xml (deflated 36%) 2024-08-20T23:21:40.2636569Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b9f60443e6b6484d.xml (deflated 35%) 2024-08-20T23:21:40.2638377Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f56d8d6b8e77ea27.xml (deflated 36%) 2024-08-20T23:21:40.2640183Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8fab616d24d5d0d7.xml (deflated 37%) 2024-08-20T23:21:40.2642083Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5de47fd09784c8cb.xml (deflated 36%) 2024-08-20T23:21:40.2643896Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8c89af6a3585cecf.xml (deflated 36%) 2024-08-20T23:21:40.2645795Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6b4fb3f47bbfa251.xml (deflated 36%) 2024-08-20T23:21:40.2647706Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-755e158b0d7bea9d.xml (deflated 36%) 2024-08-20T23:21:40.2649529Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5b42d3ae6a70df58.xml (deflated 36%) 2024-08-20T23:21:40.2651419Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-532d677d028429aa.xml (deflated 35%) 2024-08-20T23:21:40.2653239Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-18dbb0e6c3775789.xml (deflated 35%) 2024-08-20T23:21:40.2655035Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6e26567e24529d31.xml (deflated 36%) 2024-08-20T23:21:40.2656848Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d22b4636f5389c21.xml (deflated 36%) 2024-08-20T23:21:40.2658747Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9d26b906aadeed65.xml (deflated 35%) 2024-08-20T23:21:40.2660565Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-caee090aa2248294.xml (deflated 35%) 2024-08-20T23:21:40.2662364Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1101d967c9089111.xml (deflated 36%) 2024-08-20T23:21:40.2664182Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0ca70ad328783a21.xml (deflated 36%) 2024-08-20T23:21:40.2665984Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0228b2e89812fdb5.xml (deflated 35%) 2024-08-20T23:21:40.2667793Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cbee1253d667f114.xml (deflated 36%) 2024-08-20T23:21:40.2669583Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d4082d8228aba58e.xml (deflated 34%) 2024-08-20T23:21:40.2671394Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a837b82b6f689b7e.xml (deflated 35%) 2024-08-20T23:21:40.2673202Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ec0f5999f2e53014.xml (deflated 34%) 2024-08-20T23:21:40.2674993Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3077e9be5770119c.xml (deflated 35%) 2024-08-20T23:21:40.2676773Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-065c5a0cb5ad412b.xml (deflated 35%) 2024-08-20T23:21:40.2678587Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0b56bc840e2fb7b3.xml (deflated 35%) 2024-08-20T23:21:40.2680399Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e25c765661ca23c5.xml (deflated 34%) 2024-08-20T23:21:40.2682295Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ee342fa1e2eeb399.xml (deflated 35%) 2024-08-20T23:21:40.2684096Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f3caaafa975c1d90.xml (deflated 35%) 2024-08-20T23:21:40.2685907Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b367d8d8579ac9de.xml (deflated 35%) 2024-08-20T23:21:40.2687731Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7ec8af09cdffcca3.xml (deflated 35%) 2024-08-20T23:21:40.2689545Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a47c014415f8d413.xml (deflated 35%) 2024-08-20T23:21:40.2691391Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c54f3688f3a25f8d.xml (deflated 36%) 2024-08-20T23:21:40.2693200Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6824180957c3b184.xml (deflated 35%) 2024-08-20T23:21:40.2695183Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b4a8e4d38f0af122.xml (deflated 35%) 2024-08-20T23:21:40.2696993Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8f891896318de4b5.xml (deflated 35%) 2024-08-20T23:21:40.2698942Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-45d4ab5d5490bd7f.xml (deflated 36%) 2024-08-20T23:21:40.2700756Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4c895a86e804af38.xml (deflated 36%) 2024-08-20T23:21:40.2702574Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-05403cfeb1cbff5d.xml (deflated 36%) 2024-08-20T23:21:40.2704391Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a7fa5c7c193765f7.xml (deflated 35%) 2024-08-20T23:21:40.2706192Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-304c19e4ffe5a2ef.xml (deflated 36%) 2024-08-20T23:21:40.2708010Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a7c495aef37bf65f.xml (deflated 35%) 2024-08-20T23:21:40.2709818Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6a580fd01071a47c.xml (deflated 35%) 2024-08-20T23:21:40.2711625Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5f1288eb08b7a173.xml (deflated 35%) 2024-08-20T23:21:40.2713425Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b4839503d13367be.xml (deflated 35%) 2024-08-20T23:21:40.2715235Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ec8c50132352cd99.xml (deflated 35%) 2024-08-20T23:21:40.2717032Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-138761c96539fd16.xml (deflated 35%) 2024-08-20T23:21:40.2718836Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ba19c9c7699e54c6.xml (deflated 35%) 2024-08-20T23:21:40.2720637Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2a3e68d070a6bbbc.xml (deflated 36%) 2024-08-20T23:21:40.2722427Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-da6dc04c9ff750df.xml (deflated 36%) 2024-08-20T23:21:40.2724346Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-bb91dc299165c989.xml (deflated 35%) 2024-08-20T23:21:40.2726173Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5b371e9dba2656d7.xml (deflated 35%) 2024-08-20T23:21:40.2727996Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1fcc8c07b6f73a59.xml (deflated 36%) 2024-08-20T23:21:40.2729869Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a9cebb5a664b073b.xml (deflated 35%) 2024-08-20T23:21:40.2731684Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e190ee7c19183ce7.xml (deflated 35%) 2024-08-20T23:21:40.2733503Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-feb559016bc71dec.xml (deflated 35%) 2024-08-20T23:21:40.2735318Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-164fa89d45fc3480.xml (deflated 35%) 2024-08-20T23:21:40.2737105Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6efb64915332cbb4.xml (deflated 35%) 2024-08-20T23:21:40.2738983Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5960b6275645337a.xml (deflated 36%) 2024-08-20T23:21:40.2740786Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6c0bf0f616c1cb1b.xml (deflated 35%) 2024-08-20T23:21:40.2742602Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fa9e4a85fce0124c.xml (deflated 35%) 2024-08-20T23:21:40.2744403Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-829bb2b7c020b394.xml (deflated 35%) 2024-08-20T23:21:40.2746213Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f04c371140856943.xml (deflated 36%) 2024-08-20T23:21:40.2748094Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5c88d51fb01a68b6.xml (deflated 35%) 2024-08-20T23:21:40.2750208Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ec13c34cd5225252.xml (deflated 55%) 2024-08-20T23:21:40.2752001Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b02dd8f9ef9b93eb.xml (deflated 55%) 2024-08-20T23:21:40.2753825Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5fca94fc07004af0.xml (deflated 36%) 2024-08-20T23:21:40.2755639Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b6ed6651301f2250.xml (deflated 35%) 2024-08-20T23:21:40.2757452Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ad362c55a19d6f8c.xml (deflated 41%) 2024-08-20T23:21:40.2759258Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-113acd4ee122bc83.xml (deflated 37%) 2024-08-20T23:21:40.2761056Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e23092056d86a92c.xml (deflated 37%) 2024-08-20T23:21:40.2762848Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-469035388612b759.xml (deflated 37%) 2024-08-20T23:21:40.2764788Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6e06b70a6bdd9504.xml (deflated 37%) 2024-08-20T23:21:40.2766586Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-14ef8f186bcc0eb9.xml (deflated 37%) 2024-08-20T23:21:40.2768393Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-62e0742b89dcf2c2.xml (deflated 37%) 2024-08-20T23:21:40.2770321Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6e8a3bb729bcc556.xml (deflated 36%) 2024-08-20T23:21:40.2772144Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8e5df436ce8d83b1.xml (deflated 37%) 2024-08-20T23:21:40.2773939Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-76f9c070324ed0ff.xml (deflated 37%) 2024-08-20T23:21:40.2775754Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b131c3aa356fc6f6.xml (deflated 36%) 2024-08-20T23:21:40.2777565Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-05317ba9a32ed1e0.xml (deflated 37%) 2024-08-20T23:21:40.2779371Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cf76b052015bedda.xml (deflated 37%) 2024-08-20T23:21:40.2781284Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cc4523ca1a2c9ac8.xml (deflated 37%) 2024-08-20T23:21:40.2783098Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-338522e1ea0ec0f6.xml (deflated 37%) 2024-08-20T23:21:40.2784912Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f23f22478f2bb319.xml (deflated 38%) 2024-08-20T23:21:40.2786718Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e83a98d53e982507.xml (deflated 37%) 2024-08-20T23:21:40.2788509Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-855beec06e539271.xml (deflated 37%) 2024-08-20T23:21:40.2790326Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ebfd7e62d92ec78e.xml (deflated 36%) 2024-08-20T23:21:40.2792149Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-aba6665c78f8a141.xml (deflated 37%) 2024-08-20T23:21:40.2794347Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6144c31a0f12003c.xml (deflated 36%) 2024-08-20T23:21:40.2796180Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6f41fc6fd221eaad.xml (deflated 36%) 2024-08-20T23:21:40.2797986Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-67cf2f5eb2516217.xml (deflated 36%) 2024-08-20T23:21:40.2799798Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-199bb6b6aa663eb5.xml (deflated 36%) 2024-08-20T23:21:40.2801614Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a2b699ee387603d1.xml (deflated 37%) 2024-08-20T23:21:40.2803419Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8784f4fb5545522b.xml (deflated 36%) 2024-08-20T23:21:40.2805399Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ffd360d24a01dd94.xml (deflated 36%) 2024-08-20T23:21:40.2807227Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a2df962a381b8e02.xml (deflated 36%) 2024-08-20T23:21:40.2809038Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-43e70d7e6b8b0df1.xml (deflated 36%) 2024-08-20T23:21:40.2828315Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4fb4f67f2fdd60f5.xml (deflated 36%) 2024-08-20T23:21:40.2830423Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fff71a1a353c2daf.xml (deflated 36%) 2024-08-20T23:21:40.2832263Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-288970fb13f7f3c8.xml (deflated 36%) 2024-08-20T23:21:40.2834093Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f3d88b366fa614b6.xml (deflated 35%) 2024-08-20T23:21:40.2835906Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b6d7641770d82c7f.xml (deflated 36%) 2024-08-20T23:21:40.2837706Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b00e1faa82054d1f.xml (deflated 36%) 2024-08-20T23:21:40.2839757Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-73bb0bab449cebec.xml (deflated 37%) 2024-08-20T23:21:40.2841561Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0ef88e302265cac2.xml (deflated 35%) 2024-08-20T23:21:40.2843368Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-62ab1936642e6b47.xml (deflated 35%) 2024-08-20T23:21:40.2845162Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-114f577e13bb9d77.xml (deflated 35%) 2024-08-20T23:21:40.2846986Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7e88b4c0fe9c9bda.xml (deflated 36%) 2024-08-20T23:21:40.2848812Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0b1ecb5a7d3d7527.xml (deflated 36%) 2024-08-20T23:21:40.2850735Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-35d8d013c423ead0.xml (deflated 35%) 2024-08-20T23:21:40.2852537Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-400e7a2fbc00f90e.xml (deflated 35%) 2024-08-20T23:21:40.2854358Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-47b4f9cf7cfe54f8.xml (deflated 36%) 2024-08-20T23:21:40.2856173Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3587950a2e261d00.xml (deflated 35%) 2024-08-20T23:21:40.2857967Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2731175ac7c01889.xml (deflated 36%) 2024-08-20T23:21:40.2859759Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d099d6595ed04387.xml (deflated 36%) 2024-08-20T23:21:40.2861552Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-752f32669bf8500e.xml (deflated 36%) 2024-08-20T23:21:40.2863359Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-58a1d63cd8088e45.xml (deflated 36%) 2024-08-20T23:21:40.2865256Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-106043da3fc8cabf.xml (deflated 36%) 2024-08-20T23:21:40.2867056Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4fce334fa6dd5c9b.xml (deflated 36%) 2024-08-20T23:21:40.2868867Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-65695df12079482f.xml (deflated 35%) 2024-08-20T23:21:40.2870665Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ab167ba891cf7a09.xml (deflated 37%) 2024-08-20T23:21:40.2872478Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-02540dc7fb29411b.xml (deflated 37%) 2024-08-20T23:21:40.2874269Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6a60e9dd91435259.xml (deflated 35%) 2024-08-20T23:21:40.2876084Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cac092ae5ff2cc72.xml (deflated 36%) 2024-08-20T23:21:40.2877893Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-38efdb6ad84089ec.xml (deflated 35%) 2024-08-20T23:21:40.2879761Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d394f465b3e24796.xml (deflated 35%) 2024-08-20T23:21:40.2881549Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ee3473054b62e3d7.xml (deflated 35%) 2024-08-20T23:21:40.2883349Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-bf5deda276f51263.xml (deflated 35%) 2024-08-20T23:21:40.2885161Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5ce122ff12ec4688.xml (deflated 35%) 2024-08-20T23:21:40.2886954Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-21187ec50ddc6e23.xml (deflated 35%) 2024-08-20T23:21:40.2888754Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-57d76d4605fa630c.xml (deflated 36%) 2024-08-20T23:21:40.2890613Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9876a7e49d2e46b8.xml (deflated 35%) 2024-08-20T23:21:40.2892692Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2825a5d3dbf1cffa.xml (deflated 36%) 2024-08-20T23:21:40.2895852Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fb62462b7e0eccaa.xml (deflated 36%) 2024-08-20T23:21:40.2898708Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-40dff15f7d632728.xml (deflated 36%) 2024-08-20T23:21:40.2902230Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5d3ee22c98e6e89f.xml (deflated 36%) 2024-08-20T23:21:40.2905240Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-571fb034cdd2766b.xml (deflated 37%) 2024-08-20T23:21:40.2907089Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f693ccf3cd6a5d63.xml (deflated 36%) 2024-08-20T23:21:40.2908894Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-93914197bb3aca0c.xml (deflated 36%) 2024-08-20T23:21:40.2910882Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4b308e9f5b958306.xml (deflated 36%) 2024-08-20T23:21:40.2912709Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0c6273ee54a4ec28.xml (deflated 38%) 2024-08-20T23:21:40.2915239Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a0058b68d093ea1f.xml (deflated 36%) 2024-08-20T23:21:40.2918684Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-efc76173229247cc.xml (deflated 36%) 2024-08-20T23:21:40.2921710Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-10579149fa67b4a4.xml (deflated 36%) 2024-08-20T23:21:40.2925125Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fb396f9b7771efa0.xml (deflated 36%) 2024-08-20T23:21:40.2927640Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b4164ebf585d3fad.xml (deflated 36%) 2024-08-20T23:21:40.2929470Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-289a0e853e812cf6.xml (deflated 36%) 2024-08-20T23:21:40.2931402Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f7354fc8323107f7.xml (deflated 37%) 2024-08-20T23:21:40.2933668Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-dcd97f5141d8db4f.xml (deflated 36%) 2024-08-20T23:21:40.2935485Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-452d7de042b7928f.xml (deflated 36%) 2024-08-20T23:21:40.2937288Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-88978ad69a086a4a.xml (deflated 36%) 2024-08-20T23:21:40.2939106Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a4ca4c1efeec5868.xml (deflated 36%) 2024-08-20T23:21:40.2940922Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-409c56ce1b11b8d2.xml (deflated 36%) 2024-08-20T23:21:40.2942728Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7f12ad897c03e32b.xml (deflated 37%) 2024-08-20T23:21:40.2944625Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5e59f7a19d255d91.xml (deflated 35%) 2024-08-20T23:21:40.2946441Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6623973daea7196e.xml (deflated 36%) 2024-08-20T23:21:40.2948303Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-053659e083707ff3.xml (deflated 37%) 2024-08-20T23:21:40.2950100Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2ca560ec64a68e91.xml (deflated 37%) 2024-08-20T23:21:40.2951890Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-90e35742f0cd5568.xml (deflated 36%) 2024-08-20T23:21:40.2953698Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9a91b223d7e60e69.xml (deflated 36%) 2024-08-20T23:21:40.2955504Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f3b2b7cf7eaf15b6.xml (deflated 36%) 2024-08-20T23:21:40.2957303Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7280113cb6004ea6.xml (deflated 36%) 2024-08-20T23:21:40.2959268Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-39381fdf0de4ffa1.xml (deflated 36%) 2024-08-20T23:21:40.2961087Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3a00821665f8c568.xml (deflated 35%) 2024-08-20T23:21:40.2962902Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1c52ed216dc418db.xml (deflated 35%) 2024-08-20T23:21:40.2964721Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e07c5d4c75aa2e04.xml (deflated 36%) 2024-08-20T23:21:40.2966522Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-045adddeb2ea74d3.xml (deflated 36%) 2024-08-20T23:21:40.2968344Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5817add5862803f8.xml (deflated 37%) 2024-08-20T23:21:40.2970227Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d79cc442cb92e3ea.xml (deflated 35%) 2024-08-20T23:21:40.2972038Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-62db6b75655b38c6.xml (deflated 36%) 2024-08-20T23:21:40.2973913Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e97dcbd5494f2686.xml (deflated 34%) 2024-08-20T23:21:40.2975725Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-188e35cbbfd016f9.xml (deflated 35%) 2024-08-20T23:21:40.2977534Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d16a1d3ab1aac639.xml (deflated 35%) 2024-08-20T23:21:40.2979399Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b09c6a89ecc59255.xml (deflated 35%) 2024-08-20T23:21:40.2982535Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d3e9e519c22c89ad.xml (deflated 35%) 2024-08-20T23:21:40.2985419Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ec395dd05506f024.xml (deflated 35%) 2024-08-20T23:21:40.2988744Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f5d104e3c2f93b66.xml (deflated 34%) 2024-08-20T23:21:40.2992120Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6a304541fbe56406.xml (deflated 35%) 2024-08-20T23:21:40.2994777Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e9593be89723a9bd.xml (deflated 34%) 2024-08-20T23:21:40.2996605Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-70560281dc923582.xml (deflated 35%) 2024-08-20T23:21:40.2998419Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2af65c94045864e6.xml (deflated 35%) 2024-08-20T23:21:40.3000242Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5de0bf3058eaee58.xml (deflated 36%) 2024-08-20T23:21:40.3002039Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f16323902aac14c8.xml (deflated 35%) 2024-08-20T23:21:40.3003842Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-776c16fdf53c64e4.xml (deflated 35%) 2024-08-20T23:21:40.3005844Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c22c912f7e8ad336.xml (deflated 35%) 2024-08-20T23:21:40.3007668Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d9e31572128a87f5.xml (deflated 35%) 2024-08-20T23:21:40.3009465Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8a606755d3a304fe.xml (deflated 35%) 2024-08-20T23:21:40.3011339Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-37d07fcc4c08478d.xml (deflated 35%) 2024-08-20T23:21:40.3013157Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b8b9dfb29bb0db00.xml (deflated 35%) 2024-08-20T23:21:40.3014978Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e34fcfed4059c4db.xml (deflated 35%) 2024-08-20T23:21:40.3016793Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-18ebb6e10874a231.xml (deflated 36%) 2024-08-20T23:21:40.3018588Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a04e93b97cbdf42f.xml (deflated 36%) 2024-08-20T23:21:40.3020402Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-dea6e34c6f9eff21.xml (deflated 34%) 2024-08-20T23:21:40.3022325Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-68d4c6a34ffd510f.xml (deflated 35%) 2024-08-20T23:21:40.3024141Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f2fd7a47be0b4880.xml (deflated 35%) 2024-08-20T23:21:40.3025943Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-183b7bfe2b2fcef2.xml (deflated 35%) 2024-08-20T23:21:40.3027758Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-acfedccc8cc44e26.xml (deflated 35%) 2024-08-20T23:21:40.3029578Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-36e52e245adb1cc6.xml (deflated 35%) 2024-08-20T23:21:40.3031395Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5a15efec7a1bc22e.xml (deflated 35%) 2024-08-20T23:21:40.3033194Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fecc4b1499cad0a9.xml (deflated 35%) 2024-08-20T23:21:40.3034987Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-682b545052be9171.xml (deflated 35%) 2024-08-20T23:21:40.3036785Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-84425997d981510c.xml (deflated 36%) 2024-08-20T23:21:40.3038581Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-bfb5c12aa2d151b7.xml (deflated 36%) 2024-08-20T23:21:40.3040984Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4604e0658fd6e86e.xml (deflated 36%) 2024-08-20T23:21:40.3042856Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-73b27b436977f14b.xml (deflated 35%) 2024-08-20T23:21:40.3044731Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0ef78b5594b67a18.xml (deflated 35%) 2024-08-20T23:21:40.3047137Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8b5d3e4ea2716453.xml (deflated 35%) 2024-08-20T23:21:40.3048967Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-502efba16d3b9d1f.xml (deflated 35%) 2024-08-20T23:21:40.3050844Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fdb3cbe0181ca3eb.xml (deflated 35%) 2024-08-20T23:21:40.3052677Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3c5ec6f2dd1a9ee6.xml (deflated 36%) 2024-08-20T23:21:40.3054498Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1f3ccf40513f55d2.xml (deflated 36%) 2024-08-20T23:21:40.3056307Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a7ed6eaf59ccc93e.xml (deflated 35%) 2024-08-20T23:21:40.3058114Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-11d9e070756f750e.xml (deflated 35%) 2024-08-20T23:21:40.3059917Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2d9f3f62b05c6682.xml (deflated 35%) 2024-08-20T23:21:40.3061721Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c799091b7f57d1d0.xml (deflated 35%) 2024-08-20T23:21:40.3063611Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0dfd26ef542c70ad.xml (deflated 35%) 2024-08-20T23:21:40.3065408Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4dca49c7a8647e1f.xml (deflated 34%) 2024-08-20T23:21:40.3067224Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b8654eefc2dc8bdc.xml (deflated 35%) 2024-08-20T23:21:40.3069060Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-133a16b4f8242cf2.xml (deflated 34%) 2024-08-20T23:21:40.3070872Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b5411c2f7f963fcd.xml (deflated 35%) 2024-08-20T23:21:40.3072679Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c7cf17ef1229a05f.xml (deflated 36%) 2024-08-20T23:21:40.3074504Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b741735e3eea1271.xml (deflated 36%) 2024-08-20T23:21:40.3076316Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a70015fd6430c7e6.xml (deflated 37%) 2024-08-20T23:21:40.3078130Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-59624f96ff822219.xml (deflated 36%) 2024-08-20T23:21:40.3079913Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8062b5708891a3b4.xml (deflated 36%) 2024-08-20T23:21:40.3081709Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7d761c7959b02835.xml (deflated 37%) 2024-08-20T23:21:40.3083513Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ded3a4cf9579e406.xml (deflated 36%) 2024-08-20T23:21:40.3085318Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3a2946f590127005.xml (deflated 37%) 2024-08-20T23:21:40.3087111Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fccf140c89ea7051.xml (deflated 37%) 2024-08-20T23:21:40.3088987Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b87134393e3d2407.xml (deflated 36%) 2024-08-20T23:21:40.3090858Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d3815a5f7287b0f0.xml (deflated 38%) 2024-08-20T23:21:40.3092678Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-84fc30216db94a42.xml (deflated 38%) 2024-08-20T23:21:40.3095091Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-39c5c4182db00c21.xml (deflated 37%) 2024-08-20T23:21:40.3096915Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7183015dc4e86555.xml (deflated 38%) 2024-08-20T23:21:40.3098745Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a7e96c03e9b2e39e.xml (deflated 37%) 2024-08-20T23:21:40.3100554Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-56f2c9d14e49f61b.xml (deflated 37%) 2024-08-20T23:21:40.3102362Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d87cffb34508dcb6.xml (deflated 38%) 2024-08-20T23:21:40.3104324Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ef322f370840ea5f.xml (deflated 39%) 2024-08-20T23:21:40.3106137Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7cf94074d8d2c4a6.xml (deflated 36%) 2024-08-20T23:21:40.3107942Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-580b36818de309d3.xml (deflated 37%) 2024-08-20T23:21:40.3109753Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b07f07762c404bbd.xml (deflated 36%) 2024-08-20T23:21:40.3111558Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-71ae07cde5859e08.xml (deflated 36%) 2024-08-20T23:21:40.3113563Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-69d077feafd5d32f.xml (deflated 36%) 2024-08-20T23:21:40.3115411Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c1b1b1ce12a337ca.xml (deflated 36%) 2024-08-20T23:21:40.3117227Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-12d13b122ceef342.xml (deflated 37%) 2024-08-20T23:21:40.3119035Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4321a996ad831e01.xml (deflated 37%) 2024-08-20T23:21:40.3120850Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f1a3489e22fb1817.xml (deflated 37%) 2024-08-20T23:21:40.3122666Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-85a5dc72e5ddf60e.xml (deflated 37%) 2024-08-20T23:21:40.3124475Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c1fa76ad08f19dc7.xml (deflated 37%) 2024-08-20T23:21:40.3126280Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4cb7c65ebfaa24f1.xml (deflated 37%) 2024-08-20T23:21:40.3128089Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-19db653b587e86d1.xml (deflated 37%) 2024-08-20T23:21:40.3130090Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9c0e95033b4a0b38.xml (deflated 37%) 2024-08-20T23:21:40.3131613Z adding: test/test-reports/python-pytest/test.run_test/test.run_test-3f0478fe07d20784.xml (deflated 65%) 2024-08-20T23:21:40.3133082Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1b4d04a89f98c5f5.xml (deflated 57%) 2024-08-20T23:21:40.3135280Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-affb5d09b2cca4d7.xml (deflated 42%) 2024-08-20T23:21:40.3137025Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f32a8e2a792424b7.xml (deflated 41%) 2024-08-20T23:21:40.3138746Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-95474189c7f6bbdf.xml (deflated 42%) 2024-08-20T23:21:40.3140404Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b9209f1d3795e22f.xml (deflated 43%) 2024-08-20T23:21:40.3142081Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7f9372d42787df06.xml (deflated 42%) 2024-08-20T23:21:40.3143748Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-16af1cdff9e2049d.xml (deflated 36%) 2024-08-20T23:21:40.3145482Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b5411520477fab9e.xml (deflated 36%) 2024-08-20T23:21:40.3147211Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5087841a9203f54d.xml (deflated 36%) 2024-08-20T23:21:40.3148882Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0037696f5194eb14.xml (deflated 36%) 2024-08-20T23:21:40.3150559Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ec0ef22b314e878d.xml (deflated 35%) 2024-08-20T23:21:40.3152227Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-097e587bc1343924.xml (deflated 43%) 2024-08-20T23:21:40.3153874Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-064edbcbbc736f6b.xml (deflated 44%) 2024-08-20T23:21:40.3155551Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f3807adb89c7fc20.xml (deflated 36%) 2024-08-20T23:21:40.3157228Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e93ac7e38ff88009.xml (deflated 35%) 2024-08-20T23:21:40.3158890Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-07e812258f65a1ed.xml (deflated 35%) 2024-08-20T23:21:40.3160546Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-33421c8c57a19b4d.xml (deflated 35%) 2024-08-20T23:21:40.3162203Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-43730876c574cda8.xml (deflated 35%) 2024-08-20T23:21:40.3163866Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2316f2e8c876f072.xml (deflated 42%) 2024-08-20T23:21:40.3165537Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a8517b2440609fe1.xml (deflated 44%) 2024-08-20T23:21:40.3167180Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fde9500c3e41400f.xml (deflated 43%) 2024-08-20T23:21:40.3168848Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cef3843891942057.xml (deflated 44%) 2024-08-20T23:21:40.3170662Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5e9b2f47e63a3cc9.xml (deflated 44%) 2024-08-20T23:21:40.3172345Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-90e5561c43e72f7a.xml (deflated 45%) 2024-08-20T23:21:40.3174001Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fb7bd79d068177be.xml (deflated 44%) 2024-08-20T23:21:40.3175680Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-490159fed1fc9db8.xml (deflated 44%) 2024-08-20T23:21:40.3177357Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-dda6e918246658f7.xml (deflated 44%) 2024-08-20T23:21:40.3179025Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ee46c7d79ea0f394.xml (deflated 44%) 2024-08-20T23:21:40.3180684Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6c649ae852a0fd1c.xml (deflated 44%) 2024-08-20T23:21:40.3182345Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-57a396b6c1b95bc2.xml (deflated 43%) 2024-08-20T23:21:40.3184011Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a0dc6751eeacc92f.xml (deflated 35%) 2024-08-20T23:21:40.3185726Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-28b6b9543b918816.xml (deflated 50%) 2024-08-20T23:21:40.3187371Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-27a5e1d98fba1d3c.xml (deflated 42%) 2024-08-20T23:21:40.3189031Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ddd15c7134e634a5.xml (deflated 42%) 2024-08-20T23:21:40.3190706Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c1a766ad133a1ed6.xml (deflated 42%) 2024-08-20T23:21:40.3192361Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7d0ac3f732bad6e4.xml (deflated 36%) 2024-08-20T23:21:40.3194228Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3a12167ac3db73e1.xml (deflated 42%) 2024-08-20T23:21:40.3195919Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-238853906f09f1b8.xml (deflated 56%) 2024-08-20T23:21:40.3197588Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cf3491ed117a0a16.xml (deflated 42%) 2024-08-20T23:21:40.3199245Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a95d4cc1d2a641d0.xml (deflated 42%) 2024-08-20T23:21:40.3200919Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d3e7b68746d23b2d.xml (deflated 42%) 2024-08-20T23:21:40.3202579Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-60e528d9ff44636a.xml (deflated 42%) 2024-08-20T23:21:40.3204241Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-13c8018c051b66d7.xml (deflated 42%) 2024-08-20T23:21:40.3205892Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f07b8633a9bcd727.xml (deflated 48%) 2024-08-20T23:21:40.3207554Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f50bd3bc36be3ba9.xml (deflated 42%) 2024-08-20T23:21:40.3209217Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-dffd0d03de828098.xml (deflated 41%) 2024-08-20T23:21:40.3211063Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7f394bcd3d11f9da.xml (deflated 42%) 2024-08-20T23:21:40.3212771Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fb6766570ba0bc84.xml (deflated 42%) 2024-08-20T23:21:40.3214441Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9408de69835cec57.xml (deflated 43%) 2024-08-20T23:21:40.3216106Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0ee018b0578b19a7.xml (deflated 42%) 2024-08-20T23:21:40.3217764Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1911fb214e88b0d8.xml (deflated 42%) 2024-08-20T23:21:40.3219411Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d36c3b2f41fe3e33.xml (deflated 44%) 2024-08-20T23:21:40.3221083Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-69037719dcd9330a.xml (deflated 43%) 2024-08-20T23:21:40.3222750Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b9266ede121d31d0.xml (deflated 42%) 2024-08-20T23:21:40.3224426Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4085321101cfec27.xml (deflated 47%) 2024-08-20T23:21:40.3226169Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-00ff80b36d200b9d.xml (deflated 42%) 2024-08-20T23:21:40.3227851Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8e3d0fb6c27243cd.xml (deflated 42%) 2024-08-20T23:21:40.3229533Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-661528d881c44e3c.xml (deflated 42%) 2024-08-20T23:21:40.3231202Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a3944107f55de12a.xml (deflated 35%) 2024-08-20T23:21:40.3232862Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-337a0bf38402a609.xml (deflated 36%) 2024-08-20T23:21:40.3234544Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bad21b3b54b4cd9d.xml (deflated 42%) 2024-08-20T23:21:40.3236230Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-62ef7da7786cce90.xml (deflated 35%) 2024-08-20T23:21:40.3237902Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9a873121fe690d42.xml (deflated 43%) 2024-08-20T23:21:40.3239559Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-09a87f206bda90ff.xml (deflated 35%) 2024-08-20T23:21:40.3241231Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-41223b51222395fb.xml (deflated 37%) 2024-08-20T23:21:40.3242897Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a4173fa00497a74e.xml (deflated 35%) 2024-08-20T23:21:40.3244572Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-eb4cea6c2fa7f7c1.xml (deflated 35%) 2024-08-20T23:21:40.3246234Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5312a9b16bb566fd.xml (deflated 36%) 2024-08-20T23:21:40.3248006Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3b20fc9c57a85f25.xml (deflated 36%) 2024-08-20T23:21:40.3249843Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-140561aa55b03882.xml (deflated 42%) 2024-08-20T23:21:40.3251522Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-60f954b074a4a741.xml (deflated 35%) 2024-08-20T23:21:40.3253176Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ec7083b2ad7857c0.xml (deflated 45%) 2024-08-20T23:21:40.3254858Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a465f77028140da2.xml (deflated 43%) 2024-08-20T23:21:40.3256524Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6e8cc9d16dab81b4.xml (deflated 42%) 2024-08-20T23:21:40.3258187Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f56ab8c7f5ca99da.xml (deflated 43%) 2024-08-20T23:21:40.3259848Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-518c0504fbbaa5d9.xml (deflated 42%) 2024-08-20T23:21:40.3261503Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d63e7f8a54a8ce76.xml (deflated 42%) 2024-08-20T23:21:40.3263161Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-434c2dc385bfdc52.xml (deflated 41%) 2024-08-20T23:21:40.3264903Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bae68111144c9f10.xml (deflated 42%) 2024-08-20T23:21:40.3266555Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-01e7408012e7a62b.xml (deflated 35%) 2024-08-20T23:21:40.3268217Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3c104496c1dc5ac2.xml (deflated 35%) 2024-08-20T23:21:40.3269888Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e9175d14f189551e.xml (deflated 36%) 2024-08-20T23:21:40.3271545Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9ac65c6752998ec8.xml (deflated 35%) 2024-08-20T23:21:40.3273191Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d3025ecd3e037a7b.xml (deflated 45%) 2024-08-20T23:21:40.3274862Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a4bac3555a03ea3b.xml (deflated 36%) 2024-08-20T23:21:40.3276522Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7cf6015e0c1ce8cc.xml (deflated 35%) 2024-08-20T23:21:40.3278183Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fde9a1ca37939513.xml (deflated 35%) 2024-08-20T23:21:40.3279832Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-01538ece20f869c9.xml (deflated 35%) 2024-08-20T23:21:40.3281493Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-42fae6bb49615892.xml (deflated 35%) 2024-08-20T23:21:40.3283151Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a55bd3910816ac35.xml (deflated 35%) 2024-08-20T23:21:40.3284813Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cf274817dbf70685.xml (deflated 42%) 2024-08-20T23:21:40.3286461Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8157a3b8d3a0b6b2.xml (deflated 42%) 2024-08-20T23:21:40.3288123Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a4931445cab44e7b.xml (deflated 57%) 2024-08-20T23:21:40.3289946Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-303e2868f81dea2d.xml (deflated 42%) 2024-08-20T23:21:40.3291618Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-be912b759d888934.xml (deflated 41%) 2024-08-20T23:21:40.3293275Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d95c69d14f9c3a41.xml (deflated 41%) 2024-08-20T23:21:40.3295140Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5ce081a478028931.xml (deflated 43%) 2024-08-20T23:21:40.3296824Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-49fc44ca0886f80e.xml (deflated 42%) 2024-08-20T23:21:40.3298503Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-813a6cebb77a5a90.xml (deflated 35%) 2024-08-20T23:21:40.3300162Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1298e5fb966da0dc.xml (deflated 35%) 2024-08-20T23:21:40.3301827Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6ebe3fbd40a570bc.xml (deflated 36%) 2024-08-20T23:21:40.3303492Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b1ac99e19918e338.xml (deflated 35%) 2024-08-20T23:21:40.3305314Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d6129f1286f6e6b4.xml (deflated 35%) 2024-08-20T23:21:40.3306969Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0bc4e63cfd371505.xml (deflated 43%) 2024-08-20T23:21:40.3308630Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7ce9a5f78acb5ef1.xml (deflated 44%) 2024-08-20T23:21:40.3310301Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c7f0153e77214e92.xml (deflated 36%) 2024-08-20T23:21:40.3311961Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8cacbb4925c8418a.xml (deflated 35%) 2024-08-20T23:21:40.3313626Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-eeba327c242bb77d.xml (deflated 36%) 2024-08-20T23:21:40.3315301Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-eb9129d99dc0b367.xml (deflated 35%) 2024-08-20T23:21:40.3316961Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d8ac9a0295d23136.xml (deflated 35%) 2024-08-20T23:21:40.3318613Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-33573fc2338b9f73.xml (deflated 42%) 2024-08-20T23:21:40.3320257Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9a59736e3fdd5c51.xml (deflated 43%) 2024-08-20T23:21:40.3321922Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d0acddaad99b7167.xml (deflated 43%) 2024-08-20T23:21:40.3323588Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ed02e8f86603defe.xml (deflated 43%) 2024-08-20T23:21:40.3325261Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-af9d5a6872f78420.xml (deflated 44%) 2024-08-20T23:21:40.3326895Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-797d2853b15e399a.xml (deflated 45%) 2024-08-20T23:21:40.3328553Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2009d77744430f9d.xml (deflated 44%) 2024-08-20T23:21:40.3330373Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8b942cfb0f22714f.xml (deflated 44%) 2024-08-20T23:21:40.3332057Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c777c97e7b210ccc.xml (deflated 44%) 2024-08-20T23:21:40.3333714Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-217b872f19e26cb7.xml (deflated 44%) 2024-08-20T23:21:40.3335387Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-83be050ddcb247c6.xml (deflated 44%) 2024-08-20T23:21:40.3337058Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1fffcc697c25cbfc.xml (deflated 43%) 2024-08-20T23:21:40.3338728Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-71e846f7c7ac4fda.xml (deflated 35%) 2024-08-20T23:21:40.3340372Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-396f7234c928621d.xml (deflated 50%) 2024-08-20T23:21:40.3342030Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6d77c1ec283d96af.xml (deflated 42%) 2024-08-20T23:21:40.3343692Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2174b31e91e5a1af.xml (deflated 42%) 2024-08-20T23:21:40.3345435Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5e0c583cdd2fdd96.xml (deflated 42%) 2024-08-20T23:21:40.3347081Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fee098872775141c.xml (deflated 35%) 2024-08-20T23:21:40.3348739Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-51434e39f81250a4.xml (deflated 42%) 2024-08-20T23:21:40.3350409Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-756ea1d15bbf27a6.xml (deflated 56%) 2024-08-20T23:21:40.3352082Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-345f1d8bd0390f61.xml (deflated 42%) 2024-08-20T23:21:40.3353727Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-eb21fc42c36e8d5f.xml (deflated 42%) 2024-08-20T23:21:40.3355395Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9754ef3c57a46673.xml (deflated 42%) 2024-08-20T23:21:40.3357094Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a7bd346549ca304e.xml (deflated 42%) 2024-08-20T23:21:40.3358760Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6ef9576f0bd25bf2.xml (deflated 42%) 2024-08-20T23:21:40.3360406Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4d957a5e8622e92a.xml (deflated 48%) 2024-08-20T23:21:40.3362064Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4de65358cdba35e3.xml (deflated 42%) 2024-08-20T23:21:40.3363729Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b370ffbb8e260400.xml (deflated 41%) 2024-08-20T23:21:40.3365377Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3e4107994582a7ee.xml (deflated 42%) 2024-08-20T23:21:40.3367038Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-29c406b5b2b97a53.xml (deflated 42%) 2024-08-20T23:21:40.3368764Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-16be3f100984a95b.xml (deflated 42%) 2024-08-20T23:21:40.3370531Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0bc13059710ed1ee.xml (deflated 42%) 2024-08-20T23:21:40.3372175Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4398c969473b6087.xml (deflated 41%) 2024-08-20T23:21:40.3373837Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-936af586aa6f4331.xml (deflated 44%) 2024-08-20T23:21:40.3375496Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-49b40013e774d281.xml (deflated 43%) 2024-08-20T23:21:40.3377158Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cbc3230588f4d7e7.xml (deflated 42%) 2024-08-20T23:21:40.3378808Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-35e4d457819e86c9.xml (deflated 47%) 2024-08-20T23:21:40.3380476Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ef5301fab34b850e.xml (deflated 42%) 2024-08-20T23:21:40.3382137Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c054640c6f018184.xml (deflated 41%) 2024-08-20T23:21:40.3383863Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d88f58c095248abf.xml (deflated 42%) 2024-08-20T23:21:40.3385501Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e1564b0509573d5d.xml (deflated 35%) 2024-08-20T23:21:40.3387165Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b0d319e565aeb325.xml (deflated 36%) 2024-08-20T23:21:40.3388841Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6ca7f91982fdbb38.xml (deflated 42%) 2024-08-20T23:21:40.3390509Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4563e0ddfd6477d7.xml (deflated 35%) 2024-08-20T23:21:40.3392170Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d0a56eb8297b26d8.xml (deflated 43%) 2024-08-20T23:21:40.3394045Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3c78bb16ffa663b5.xml (deflated 35%) 2024-08-20T23:21:40.3395722Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-65ac31a5460e635c.xml (deflated 36%) 2024-08-20T23:21:40.3397390Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cf353acc42795dd4.xml (deflated 34%) 2024-08-20T23:21:40.3399047Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-931919cceb7ffc14.xml (deflated 36%) 2024-08-20T23:21:40.3400713Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ce83d233cb816bc7.xml (deflated 36%) 2024-08-20T23:21:40.3402390Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9fac47da720bde4e.xml (deflated 35%) 2024-08-20T23:21:40.3404062Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-523ccd06078626a3.xml (deflated 42%) 2024-08-20T23:21:40.3405709Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5b86534701799f22.xml (deflated 35%) 2024-08-20T23:21:40.3407372Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0fc43adbf3fe1526.xml (deflated 45%) 2024-08-20T23:21:40.3409181Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-792ec23b2b09ebb1.xml (deflated 43%) 2024-08-20T23:21:40.3410927Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d3adb0bc4d9cc725.xml (deflated 42%) 2024-08-20T23:21:40.3412582Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bde656a698613e92.xml (deflated 43%) 2024-08-20T23:21:40.3414258Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b51fd1ffa1dc3be4.xml (deflated 42%) 2024-08-20T23:21:40.3415941Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c3ae8710ef2b9be3.xml (deflated 42%) 2024-08-20T23:21:40.3417605Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c8a2598f4ccfb74d.xml (deflated 41%) 2024-08-20T23:21:40.3419258Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-53b90b33b93f4d48.xml (deflated 42%) 2024-08-20T23:21:40.3420921Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b3dd37e55f437da0.xml (deflated 35%) 2024-08-20T23:21:40.3422581Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1371a2b7f1c54f43.xml (deflated 35%) 2024-08-20T23:21:40.3424333Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cbc569263ce79102.xml (deflated 35%) 2024-08-20T23:21:40.3425979Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-135376c6b63a4b0b.xml (deflated 36%) 2024-08-20T23:21:40.3427646Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3ae54e17677b2ef9.xml (deflated 45%) 2024-08-20T23:21:40.3429315Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-116a24dbcfca4719.xml (deflated 36%) 2024-08-20T23:21:40.3430995Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-15b8857dbe2ff3f0.xml (deflated 35%) 2024-08-20T23:21:40.3432662Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f2bdc0ae8caf3c1f.xml (deflated 35%) 2024-08-20T23:21:40.3434333Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ea01f5f6dd166760.xml (deflated 35%) 2024-08-20T23:21:40.3436001Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c555786d401c05d1.xml (deflated 35%) 2024-08-20T23:21:40.3437649Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1122d377a562cf1a.xml (deflated 35%) 2024-08-20T23:21:40.3439296Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e4830497927f7f28.xml (deflated 42%) 2024-08-20T23:21:40.3440961Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b19d789d71a0d10b.xml (deflated 42%) 2024-08-20T23:21:40.3442930Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-abb74970e3e2c896.xml (deflated 37%) 2024-08-20T23:21:40.3445197Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-024c7b5556eacac4.xml (deflated 37%) 2024-08-20T23:21:40.3447418Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-a34b96d9d64562ef.xml (deflated 48%) 2024-08-20T23:21:40.3449776Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-851b942e79646ce4.xml (deflated 48%) 2024-08-20T23:21:40.3452023Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-ccadd0c787a0b165.xml (deflated 48%) 2024-08-20T23:21:40.3454266Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-498f11c41c5a7229.xml (deflated 48%) 2024-08-20T23:21:40.3456505Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-2d25315ca82b9fd6.xml (deflated 37%) 2024-08-20T23:21:40.3458725Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-9981676a62e54ea7.xml (deflated 37%) 2024-08-20T23:21:40.3460962Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-fec4ed6574bfb190.xml (deflated 48%) 2024-08-20T23:21:40.3463226Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-3b7c54c9a65a0f26.xml (deflated 48%) 2024-08-20T23:21:40.3465464Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-dba6a8f4c3337455.xml (deflated 48%) 2024-08-20T23:21:40.3467745Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-c0d92e6a9f716d17.xml (deflated 48%) 2024-08-20T23:21:40.3496507Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-20T23:21:40.3497498Z # Remove any previous usage logs if they exist 2024-08-20T23:21:40.3498361Z rm -f logs-*.zip 2024-08-20T23:21:40.3499057Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-20T23:21:40.3499877Z # so check to see if the file exists first 2024-08-20T23:21:40.3500384Z if [ -f 'usage_log.txt' ]; then 2024-08-20T23:21:40.3500890Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-20T23:21:40.3501378Z fi 2024-08-20T23:21:40.3501754Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-20T23:21:40.3502323Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-20T23:21:40.3502827Z fi 2024-08-20T23:21:40.3508549Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:21:40.3509032Z env: 2024-08-20T23:21:40.3509278Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:21:40.3509888Z DOCKER_CONTAINER_ID: a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:40.3510738Z FILE_SUFFIX: test-distributed-1-2-amz2023.linux.2xlarge_29025446952 2024-08-20T23:21:40.3511293Z ##[endgroup] 2024-08-20T23:21:40.3601898Z adding: usage_log.txt (deflated 92%) 2024-08-20T23:21:40.3853487Z adding: test/test-reports/distributed.test_inductor_collectives_1.1_7e758b11cb23cba1_.log (deflated 90%) 2024-08-20T23:21:40.3854902Z adding: test/test-reports/distributed._tensor.test_dtensor_compile_1.1_cb0420759f8eaa45_.log (deflated 85%) 2024-08-20T23:21:40.3856431Z adding: test/test-reports/distributed.test_data_parallel_1.1_f019788f800e2435_.log (deflated 89%) 2024-08-20T23:21:40.3858981Z adding: test/test-reports/distributed.fsdp.test_fsdp_mixed_precision_1.1_9ceb524957a75a33_.log (deflated 92%) 2024-08-20T23:21:40.3860539Z adding: test/test-reports/distributed._composable.test_replicate_with_compiler_1.1_d99c76b00dfc5226_.log (deflated 78%) 2024-08-20T23:21:40.3862005Z adding: test/test-reports/distributed._tensor.debug.test_op_coverage_1.1_b10a038b6d1b625f_.log (deflated 55%) 2024-08-20T23:21:40.3863498Z adding: test/test-reports/distributed.test_c10d_functional_native_1.1_4be27fce5d78c9e7_.log (deflated 86%) 2024-08-20T23:21:40.3864740Z adding: test/test-reports/distributed.pipelining.test_transformer_1.1_83c2e6f61728cb36_.log (deflated 54%) 2024-08-20T23:21:40.5012971Z adding: test/test-reports/distributed._tensor.test_dtensor_ops_1.1_bd362836885712c6_.log (deflated 96%) 2024-08-20T23:21:40.5014420Z adding: test/test-reports/distributed.flight_recorder.test_fr_analysis_1.1_aec96f27df022b66_.log (deflated 55%) 2024-08-20T23:21:40.5015716Z adding: test/test-reports/distributed.pipelining.test_pipe_1.1_6fa8bdeb647c917c_.log (deflated 62%) 2024-08-20T23:21:40.5017110Z adding: test/test-reports/distributed.tensor.parallel.test_micro_pipeline_tp_1.1_603298d07a481f63_.log (deflated 90%) 2024-08-20T23:21:40.5018500Z adding: test/test-reports/distributed.fsdp.test_fsdp_sharded_grad_scaler_1.1_e257fd07db47c557_.log (deflated 89%) 2024-08-20T23:21:40.5019938Z adding: test/test-reports/distributed.launcher.test_run_1.1_533cd6f2282d992b_.log (deflated 89%) 2024-08-20T23:21:40.5021202Z adding: test/test-reports/distributed.pipelining.test_microbatch_1.1_43db94dfe9ada2c1_.log (deflated 58%) 2024-08-20T23:21:40.5022627Z adding: test/test-reports/distributed._composable.test_contract_1.1_6765637ff7c081a3_.log (deflated 67%) 2024-08-20T23:21:40.5023932Z adding: test/test-reports/distributed._tensor.test_common_rules_1.1_e62fb4d92b26a577_.log (deflated 76%) 2024-08-20T23:21:40.5025505Z adding: test/test-reports/distributed._tensor.test_random_ops_1.1_40fdcbc4dc7ed87f_.log (deflated 75%) 2024-08-20T23:21:40.5026714Z adding: test/test-reports/distributed.fsdp.test_fsdp_fine_tune_1.1_31f573bc48a8cfe0_.log (deflated 68%) 2024-08-20T23:21:40.5028045Z adding: test/test-reports/distributed._tensor.experimental.test_register_sharding_1.1_7e7f2678996c73b2_.log (deflated 76%) 2024-08-20T23:21:40.5029372Z adding: test/test-reports/distributed.optim.test_named_optimizer_1.1_8b27d463d086bcfa_.log (deflated 34%) 2024-08-20T23:21:40.5030539Z adding: test/test-reports/distributed._tensor.test_api_1.1_7085bda8a23e0f6d_.log (deflated 89%) 2024-08-20T23:21:40.5031727Z adding: test/test-reports/distributed._tensor.test_experimental_ops_1.1_a94ba2b7a383c63e_.log (deflated 73%) 2024-08-20T23:21:40.5033021Z adding: test/test-reports/distributed.fsdp.test_fsdp_fx_1.1_f9b99d5f60b76fc9_.log (deflated 53%) 2024-08-20T23:21:40.5034185Z adding: test/test-reports/distributed._tensor.test_tensor_ops_1.1_4017135920ceadd3_.log (deflated 95%) 2024-08-20T23:21:40.5035464Z adding: test/test-reports/distributed.test_device_mesh_1.1_cfa564fa7fd9e116_.log (deflated 83%) 2024-08-20T23:21:40.5036788Z adding: test/test-reports/distributed.fsdp.test_fsdp_traversal_1.1_bce50c9f7f3ab100_.log (deflated 53%) 2024-08-20T23:21:40.5038209Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_overlap_1.1_bb6c792a84bc09e9_.log (deflated 63%) 2024-08-20T23:21:40.5039610Z adding: test/test-reports/distributed._tensor.experimental.test_tp_transform_1.1_b362bbe64109f46a_.log (deflated 65%) 2024-08-20T23:21:40.5040936Z adding: test/test-reports/distributed.tensor.parallel.test_tp_style_1.1_f8620952d2a90112_.log (deflated 90%) 2024-08-20T23:21:40.5051528Z adding: test/test-reports/distributed._tensor.test_redistribute_1.1_d6251b308089ddb2_.log (deflated 95%) 2024-08-20T23:21:40.5052844Z adding: test/test-reports/distributed._tensor.test_matrix_ops_1.1_849579c2d450512d_.log (deflated 88%) 2024-08-20T23:21:40.5054223Z adding: test/test-reports/distributed._tensor.test_math_ops_1.1_9478f74093b3b373_.log (deflated 89%) 2024-08-20T23:21:40.5055610Z adding: test/test-reports/distributed.tensor.parallel.test_tp_examples_1.1_a45cb9773efe350f_.log (deflated 87%) 2024-08-20T23:21:40.5057069Z adding: test/test-reports/distributed.optim.test_apply_optimizer_in_backward_1.1_151ad3fa4d99e303_.log (deflated 32%) 2024-08-20T23:21:40.5058502Z adding: test/test-reports/distributed.checkpoint.e2e.test_pipeline_1.1_1bd62d1493d3fadc_.log (deflated 53%) 2024-08-20T23:21:40.5060127Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_state_1.1_2f3abab9f8e38169_.log (deflated 72%) 2024-08-20T23:21:40.5061494Z adding: test/test-reports/distributed.test_collective_utils_1.1_000784736f4f6574_.log (stored 0%) 2024-08-20T23:21:40.5062777Z adding: test/test-reports/distributed.test_pg_wrapper_1.1_4f099d5c09e51039_.log (deflated 93%) 2024-08-20T23:21:40.5064091Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_compile_1.1_88971626bfb36fbf_.log (deflated 56%) 2024-08-20T23:21:40.5065531Z adding: test/test-reports/distributed.checkpoint.test_state_dict_utils_1.1_1801ef69048d924b_.log (deflated 73%) 2024-08-20T23:21:40.5066909Z adding: test/test-reports/distributed.pipelining.test_schedule_1.1_e852c240c45bbc4f_.log (deflated 79%) 2024-08-20T23:21:40.5068192Z adding: test/test-reports/distributed._tensor.test_embedding_ops_1.1_b1e7ff9be3d32f48_.log (deflated 84%) 2024-08-20T23:21:40.5069597Z adding: test/test-reports/distributed.checkpoint.test_utils_1.1_b48de287102e45e0_.log (deflated 66%) 2024-08-20T23:21:40.5071024Z adding: test/test-reports/distributed._tensor.debug.test_comm_mode_1.1_c82556452f2dfdf9_.log (deflated 61%) 2024-08-20T23:21:40.5072395Z adding: test/test-reports/distributed.optim.test_zero_redundancy_optimizer_1.1_28eca574f53a18a6_.log (deflated 94%) 2024-08-20T23:21:40.5073722Z adding: test/test-reports/distributed.fsdp.test_shard_utils_1.1_f1d68bcca60f03e3_.log (deflated 59%) 2024-08-20T23:21:40.5075196Z adding: test/test-reports/distributed.checkpoint.test_fsdp_model_state_1.1_32130ab8f1d2e433_.log (deflated 61%) 2024-08-20T23:21:40.5076423Z adding: test/test-reports/distributed.rpc.test_share_memory_1.1_53aba8f94c312608_.log (deflated 56%) 2024-08-20T23:21:40.5077664Z adding: test/test-reports/distributed.checkpoint.test_checkpoint_1.1_d9340295738cda5b_.log (deflated 81%) 2024-08-20T23:21:40.5078792Z adding: test/test-reports/distributed.test_nccl_1.1_4d076f6693dbb7ce_.log (deflated 49%) 2024-08-20T23:21:40.5079896Z adding: test/test-reports/distributed.elastic.timer.api_test_1.1_ed2909ab65472812_.log (stored 0%) 2024-08-20T23:21:40.5081261Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_mixed_precision_1.1_c1ded751ac3d4eaa_.log (deflated 58%) 2024-08-20T23:21:40.5082729Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_tensor_ops_1.1_477a0215b26c5557_.log (deflated 71%) 2024-08-20T23:21:40.5084334Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_util_1.1_50a149cff676e4a7_.log (deflated 57%) 2024-08-20T23:21:40.5085778Z adding: test/test-reports/distributed._composable.test_replicate_1.1_0e3eb8cfe148a166_.log (deflated 81%) 2024-08-20T23:21:40.5087072Z adding: test/test-reports/distributed.checkpoint.test_state_dict_1.1_f5c68d98949b276c_.log (deflated 87%) 2024-08-20T23:21:40.5088473Z adding: test/test-reports/distributed.fsdp.test_fsdp_dtensor_state_dict_1.1_c3616971e6df82dd_.log (deflated 86%) 2024-08-20T23:21:40.5089874Z adding: test/test-reports/distributed.test_c10d_logger_1.1_9bd5c5ef8a508001_.log (deflated 60%) 2024-08-20T23:21:40.5091113Z adding: test/test-reports/distributed.fsdp.test_fsdp_state_dict_1.1_fd27c415e0b06a5c_.log (deflated 96%) 2024-08-20T23:21:40.5092633Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_autograd_1.1_556302e8ec7a2cfa_.log (deflated 72%) 2024-08-20T23:21:40.5094204Z adding: test/test-reports/distributed.checkpoint.test_file_system_checkpoint_1.1_89dfd5305deeb66c_.log (deflated 73%) 2024-08-20T23:21:40.5095477Z adding: test/test-reports/distributed.fsdp.test_fsdp_input_1.1_6c520b1858866906_.log (deflated 58%) 2024-08-20T23:21:40.5096821Z adding: test/test-reports/distributed._composable.test_composability.test_2d_composability_1.1_5fd2014a4785eea2_.log (deflated 84%) 2024-08-20T23:21:40.5098046Z adding: test/test-reports/cpp.test_cpp_rpc_1.1_df4a3e8372971af1_.log (deflated 78%) 2024-08-20T23:21:40.5099372Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_frozen_1.1_23549e75305dbde1_.log (deflated 67%) 2024-08-20T23:21:40.5100721Z adding: test/test-reports/distributed.fsdp.test_distributed_checkpoint_1.1_86e29cfd31fefa77_.log (deflated 65%) 2024-08-20T23:21:40.5102012Z adding: test/test-reports/distributed.fsdp.test_fsdp_multiple_wrapping_1.1_63daa244e5aa0b2f_.log (deflated 56%) 2024-08-20T23:21:40.5103414Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_mixed_precision_1.1_10cac1d7361e2625_.log (deflated 74%) 2024-08-20T23:21:40.5104714Z adding: test/test-reports/distributed._tensor.test_utils_1.1_ec131e458f85b19d_.log (deflated 87%) 2024-08-20T23:21:40.5105858Z adding: test/test-reports/distributed.test_distributed_spawn_2.3_a93cc3a4eadbf990_.log (deflated 37%) 2024-08-20T23:21:40.5107033Z adding: test/test-reports/distributed.test_distributed_spawn_2.3_3bc95e0ae5c78bc6_.log (deflated 37%) 2024-08-20T23:21:40.5109303Z adding: test/test-reports/distributed.test_distributed_spawn_2.3_f9b55a3a7cbb0a65_.log (deflated 97%) 2024-08-20T23:21:40.5125694Z adding: test/test-reports/distributed.test_distributed_spawn_2.3_9a430e8595b1e6a5_.log (deflated 97%) 2024-08-20T23:21:40.5127087Z adding: test/test-reports/distributed._shard.sharding_plan.test_sharding_plan_1.1_86dc68755fd3da5a_.log (deflated 66%) 2024-08-20T23:21:40.5128765Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_model_checkpoint_1.1_60404ed95c02037e_.log (deflated 73%) 2024-08-20T23:21:40.5130458Z adding: test/test-reports/distributed.test_c10d_object_collectives_1.1_1b9b51bf717fe7dd_.log (deflated 76%) 2024-08-20T23:21:40.5131910Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_init_1.1_77ee4b05cf72dc6a_.log (deflated 76%) 2024-08-20T23:21:40.5133315Z adding: test/test-reports/distributed.fsdp.test_fsdp_tp_integration_1.1_6c62be84c47ce187_.log (deflated 67%) 2024-08-20T23:21:40.5134636Z adding: test/test-reports/distributed.fsdp.test_fsdp_grad_acc_1.1_0edaddb08a08094e_.log (deflated 74%) 2024-08-20T23:21:40.5135954Z adding: test/test-reports/distributed.fsdp.test_fsdp_freezing_weights_1.1_0c05d99add36a5cd_.log (deflated 93%) 2024-08-20T23:21:40.5137182Z adding: test/test-reports/distributed.test_multi_threaded_pg_1.1_7bc2092b22dc7d99_.log (deflated 85%) 2024-08-20T23:21:40.5138580Z adding: test/test-reports/distributed.algorithms.ddp_comm_hooks.test_ddp_hooks_1.1_0ec5ab8740d987a7_.log (deflated 76%) 2024-08-20T23:21:40.5140055Z adding: test/test-reports/distributed.fsdp.test_fsdp_flatten_params_1.1_ab393392fff544d2_.log (deflated 81%) 2024-08-20T23:21:40.5141578Z adding: test/test-reports/distributed.checkpoint.test_file_system_checkpoint_cpu_1.1_efa8529bd5dcfd21_.log (deflated 94%) 2024-08-20T23:21:40.5142992Z adding: test/test-reports/distributed.test_c10d_pypg_1.1_70e5a95c9be17b9a_.log (deflated 90%) 2024-08-20T23:21:40.5144370Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_extensions_1.1_8de963b98b3ca90d_.log (deflated 67%) 2024-08-20T23:21:40.5145742Z adding: test/test-reports/distributed.fsdp.test_fsdp_memory_1.1_089ba420ec3005d5_.log (deflated 59%) 2024-08-20T23:21:40.5147162Z adding: test/test-reports/distributed.fsdp.test_hsdp_dtensor_state_dict_1.1_c60b6b43126a5031_.log (deflated 79%) 2024-08-20T23:21:40.5148481Z adding: test/test-reports/distributed.fsdp.test_fsdp_exec_order_1.1_926b3d7197af8cd9_.log (deflated 80%) 2024-08-20T23:21:40.5149828Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_runtime_1.1_4d06f7e775a93721_.log (deflated 62%) 2024-08-20T23:21:40.5151416Z adding: test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_d2ea3bb9e2c49b30_.log (deflated 14%) 2024-08-20T23:21:40.5152942Z adding: test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_3727a4fc912118c6_.log (deflated 14%) 2024-08-20T23:21:40.5154627Z adding: test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_53152eb7f3adb516_.log (deflated 90%) 2024-08-20T23:21:40.5156209Z adding: test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_875781efd6fee1e1_.log (deflated 90%) 2024-08-20T23:21:40.5157556Z adding: test/test-reports/distributed.fsdp.test_fsdp_hybrid_shard_1.1_b69dfe17348ed6ac_.log (deflated 73%) 2024-08-20T23:21:40.5158772Z adding: test/test-reports/distributed.algorithms.test_join_1.1_502c3447c24b4005_.log (deflated 81%) 2024-08-20T23:21:40.5159912Z adding: test/test-reports/distributed.fsdp.test_utils_1.1_e224f7f2b17ed7c0_.log (deflated 64%) 2024-08-20T23:21:40.5161120Z adding: test/test-reports/distributed.fsdp.test_fsdp_core_1.1_950977b690e57120_.log (deflated 92%) 2024-08-20T23:21:40.5162312Z adding: test/test-reports/distributed.fsdp.test_fsdp_checkpoint_1.1_8249b56a329e380c_.log (deflated 87%) 2024-08-20T23:21:40.5163547Z adding: test/test-reports/distributed.fsdp.test_fsdp_use_orig_params_1.1_779b6079a333d90c_.log (deflated 86%) 2024-08-20T23:21:40.5164920Z adding: test/test-reports/distributed._shard.sharding_spec.test_sharding_spec_1.1_1ea20b8dffd3c0cc_.log (deflated 78%) 2024-08-20T23:21:40.5166186Z adding: test/test-reports/distributed.fsdp.test_fsdp_apply_1.1_8f7cb1ff0823d6c1_.log (deflated 62%) 2024-08-20T23:21:40.5167573Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint_1.1_e44c8d76e1478aca_.log (deflated 65%) 2024-08-20T23:21:40.5169111Z adding: test/test-reports/distributed._shard.sharded_optim.test_sharded_optim_1.1_05930829ab77c03f_.log (deflated 62%) 2024-08-20T23:21:40.5201893Z adding: test/test-reports/distributed.rpc.test_tensorpipe_agent_1.3_2c4c626cf6aa1639_.log (deflated 97%) 2024-08-20T23:21:40.5252168Z adding: test/test-reports/distributed.rpc.test_tensorpipe_agent_3.3_9ed920af1c9514c2_.log (deflated 97%) 2024-08-20T23:21:40.5253301Z adding: test/test-reports/distributed.test_c10d_ucc_1.1_a4f035ad4a6de4dd_.log (stored 0%) 2024-08-20T23:21:40.5283831Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-20T23:21:40.5284564Z # Remove any previous debugging artifacts if they exist 2024-08-20T23:21:40.5285099Z rm -f debug-*.zip 2024-08-20T23:21:40.5285453Z if [ -d 'test/debug' ]; then 2024-08-20T23:21:40.5285921Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-20T23:21:40.5286375Z fi 2024-08-20T23:21:40.5292373Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:21:40.5292855Z env: 2024-08-20T23:21:40.5293114Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:21:40.5293948Z DOCKER_CONTAINER_ID: a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:40.5294795Z FILE_SUFFIX: test-distributed-1-2-amz2023.linux.2xlarge_29025446952 2024-08-20T23:21:40.5295359Z ##[endgroup] 2024-08-20T23:21:40.5381141Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:21:40.5381559Z with: 2024-08-20T23:21:40.5381824Z s3-bucket: gha-artifacts 2024-08-20T23:21:40.5382251Z s3-prefix: pytorch/pytorch/10479309237/1/artifact 2024-08-20T23:21:40.5382697Z retention-days: 14 2024-08-20T23:21:40.5383339Z if-no-files-found: warn 2024-08-20T23:21:40.5383675Z path: test-jsons-*.zip 2024-08-20T23:21:40.5383982Z name: artifact 2024-08-20T23:21:40.5384265Z region: us-east-1 2024-08-20T23:21:40.5384546Z env: 2024-08-20T23:21:40.5384785Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:21:40.5385389Z DOCKER_CONTAINER_ID: a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:40.5386051Z ##[endgroup] 2024-08-20T23:21:40.9116858Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:21:40.9117869Z With the provided path, there will be 1 file uploaded 2024-08-20T23:21:40.9118482Z Uploading to s3 prefix: pytorch/pytorch/10479309237/1/artifact 2024-08-20T23:21:40.9158108Z Starting upload of test-jsons-test-distributed-1-2-amz2023.linux.2xlarge_29025446952.zip 2024-08-20T23:21:41.0083359Z Finished upload of test-jsons-test-distributed-1-2-amz2023.linux.2xlarge_29025446952.zip 2024-08-20T23:21:41.0259337Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:21:41.0259782Z with: 2024-08-20T23:21:41.0260037Z s3-bucket: gha-artifacts 2024-08-20T23:21:41.0260457Z s3-prefix: pytorch/pytorch/10479309237/1/artifact 2024-08-20T23:21:41.0260921Z retention-days: 14 2024-08-20T23:21:41.0261226Z if-no-files-found: error 2024-08-20T23:21:41.0261571Z path: test-reports-*.zip 2024-08-20T23:21:41.0261916Z name: artifact 2024-08-20T23:21:41.0262187Z region: us-east-1 2024-08-20T23:21:41.0262469Z env: 2024-08-20T23:21:41.0262729Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:21:41.0263323Z DOCKER_CONTAINER_ID: a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:41.0263982Z ##[endgroup] 2024-08-20T23:21:41.3748256Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:21:41.3748859Z With the provided path, there will be 1 file uploaded 2024-08-20T23:21:41.3749476Z Uploading to s3 prefix: pytorch/pytorch/10479309237/1/artifact 2024-08-20T23:21:41.3789483Z Starting upload of test-reports-test-distributed-1-2-amz2023.linux.2xlarge_29025446952.zip 2024-08-20T23:21:41.6516150Z Finished upload of test-reports-test-distributed-1-2-amz2023.linux.2xlarge_29025446952.zip 2024-08-20T23:21:41.6687710Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:21:41.6688134Z with: 2024-08-20T23:21:41.6688399Z s3-bucket: gha-artifacts 2024-08-20T23:21:41.6688974Z s3-prefix: pytorch/pytorch/10479309237/1/artifact 2024-08-20T23:21:41.6689421Z retention-days: 14 2024-08-20T23:21:41.6690170Z if-no-files-found: ignore 2024-08-20T23:21:41.6690519Z path: logs-*.zip 2024-08-20T23:21:41.6690791Z name: artifact 2024-08-20T23:21:41.6691076Z region: us-east-1 2024-08-20T23:21:41.6691354Z env: 2024-08-20T23:21:41.6691594Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:21:41.6692203Z DOCKER_CONTAINER_ID: a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:41.6692857Z ##[endgroup] 2024-08-20T23:21:42.0153836Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:21:42.0154720Z With the provided path, there will be 1 file uploaded 2024-08-20T23:21:42.0155367Z Uploading to s3 prefix: pytorch/pytorch/10479309237/1/artifact 2024-08-20T23:21:42.0194259Z Starting upload of logs-test-distributed-1-2-amz2023.linux.2xlarge_29025446952.zip 2024-08-20T23:21:42.3238946Z Finished upload of logs-test-distributed-1-2-amz2023.linux.2xlarge_29025446952.zip 2024-08-20T23:21:42.3408170Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:21:42.3408750Z with: 2024-08-20T23:21:42.3409005Z s3-bucket: gha-artifacts 2024-08-20T23:21:42.3409433Z s3-prefix: pytorch/pytorch/10479309237/1/artifact 2024-08-20T23:21:42.3409986Z retention-days: 14 2024-08-20T23:21:42.3410294Z if-no-files-found: ignore 2024-08-20T23:21:42.3410635Z path: debug-*.zip 2024-08-20T23:21:42.3410931Z name: artifact 2024-08-20T23:21:42.3411204Z region: us-east-1 2024-08-20T23:21:42.3411483Z env: 2024-08-20T23:21:42.3411737Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:21:42.3412342Z DOCKER_CONTAINER_ID: a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:42.3413001Z ##[endgroup] 2024-08-20T23:21:42.6841405Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-20T23:21:42.7010433Z ##[group]Run # shellcheck disable=SC2156 2024-08-20T23:21:42.7010899Z # shellcheck disable=SC2156 2024-08-20T23:21:42.7011826Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-20T23:21:42.7018089Z shell: /usr/bin/bash -e {0} 2024-08-20T23:21:42.7018430Z env: 2024-08-20T23:21:42.7018686Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:21:42.7019284Z DOCKER_CONTAINER_ID: a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:42.7019939Z ##[endgroup] 2024-08-20T23:21:42.9256631Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-20T23:21:42.9257178Z with: 2024-08-20T23:21:42.9257418Z env: 2024-08-20T23:21:42.9257673Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:21:42.9258270Z DOCKER_CONTAINER_ID: a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:42.9258927Z ##[endgroup] 2024-08-20T23:21:42.9280537Z ##[group]Run set -eou pipefail 2024-08-20T23:21:42.9281185Z set -eou pipefail 2024-08-20T23:21:42.9281504Z  2024-08-20T23:21:42.9282009Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-20T23:21:42.9282665Z for _ in $(seq 1440); do 2024-08-20T23:21:42.9283107Z  # Break if no ssh session exists anymore 2024-08-20T23:21:42.9283587Z  if [ "$(who)" = "" ]; then 2024-08-20T23:21:42.9283972Z  break 2024-08-20T23:21:42.9284250Z  fi 2024-08-20T23:21:42.9284576Z  echo "." 2024-08-20T23:21:42.9284870Z  sleep 5 2024-08-20T23:21:42.9285164Z done 2024-08-20T23:21:42.9291002Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:21:42.9291487Z env: 2024-08-20T23:21:42.9291748Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:21:42.9292364Z DOCKER_CONTAINER_ID: a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:42.9293022Z ##[endgroup] 2024-08-20T23:21:42.9325630Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-20T23:21:42.9391593Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:21:42.9392513Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:21:42.9393104Z # shellcheck disable=SC2046 2024-08-20T23:21:42.9393547Z docker stop $(docker ps -q) || true 2024-08-20T23:21:42.9394227Z # Prune all of the docker images 2024-08-20T23:21:42.9394664Z docker system prune -af 2024-08-20T23:21:42.9400717Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:21:42.9401198Z env: 2024-08-20T23:21:42.9401462Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:21:42.9402077Z DOCKER_CONTAINER_ID: a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:42.9402726Z ##[endgroup] 2024-08-20T23:21:53.5086240Z a5702959bcb2 2024-08-20T23:21:53.8593567Z Deleted Containers: 2024-08-20T23:21:53.8594371Z a5702959bcb2a09f76d3d08df1362dcbac9a462937db70ba59badb0edbd01215 2024-08-20T23:21:53.8594831Z 2024-08-20T23:21:57.4038383Z Deleted Images: 2024-08-20T23:21:57.4040168Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T23:21:57.4043463Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11@sha256:9c16298ecf1ba0ec71e25268bd4244acb2b2c8c60846e6c8a5e8eb35b9eff836 2024-08-20T23:21:57.4044935Z deleted: sha256:0d70ec8288d42bb7b6bd827783d06ac33bce3462f66d9a810a9a9ddd77c887e9 2024-08-20T23:21:57.4045817Z deleted: sha256:cd85f247f1db7df3b86b5d221f7780d127d898da58566662175cbceef1eca74e 2024-08-20T23:21:57.4046691Z deleted: sha256:9b5af2ba4c6fed48eb126f4e4d26fe2f6f5bb23e7960f4cd20d0e3b8e36abb36 2024-08-20T23:21:57.4047576Z deleted: sha256:dfcabb79b518bf75986db8730806b2d6e58f847262420385a83685a50b6cc03e 2024-08-20T23:21:57.4048445Z deleted: sha256:f0dc0fc952b0d537320b816a022f8835ffdd8c001dbeebc3bfe2ae19ea67cffe 2024-08-20T23:21:57.4049313Z deleted: sha256:333fc2efa12fa987d4d921d2a57ecf4c45660a519656c1ea4603452af5bca473 2024-08-20T23:21:57.4050257Z deleted: sha256:cc1311368e6820f3097a29c3d2224b26e219cfe670357339b6bebac80e1b7791 2024-08-20T23:21:57.4051148Z deleted: sha256:73fbd71f6e938b5b8a5520c5e3b05a60bc5fe200afc590be7a63bfda9dbab816 2024-08-20T23:21:57.4052027Z deleted: sha256:86ef3feab980e3e593efc8567bb6b0c5b87ecc3d56d23f3d0c1b6a558e2498f9 2024-08-20T23:21:57.4053253Z deleted: sha256:786a1cf18988e3ca8c155174a2e39790d0895667c87826244946d9d8d6a7c7e1 2024-08-20T23:21:57.4054121Z deleted: sha256:16ab50ca7d231916c2ecf9cf4e10ebb7161b1fa9d8a9069c5d9f3a67e56e569c 2024-08-20T23:21:57.4054979Z deleted: sha256:d74e37a5061fb48a66f621cb7351e79c28a563a8d579ce1c1488870e3b4551fe 2024-08-20T23:21:57.4055828Z deleted: sha256:886fd5dabaddbb1999f54b910991db2f02021b0dcfd9645ba290e51b6bcffbd3 2024-08-20T23:21:57.4056691Z deleted: sha256:1899784bdae733da95c66a616b5a7596f5e9209c64d060321e786a2a511bbf9e 2024-08-20T23:21:57.4057544Z deleted: sha256:3603bcd51acfe96da1a1e7e20000459fa9bd6839e882d942f031f7933d2cbcf4 2024-08-20T23:21:57.4058463Z deleted: sha256:c37c9591a1ebbfb3d382ea183d82239d7c10d2912da578f8dd94f88a75ec7c7c 2024-08-20T23:21:57.4059297Z deleted: sha256:42f8742744b3b0501274d2617a4faee84f98b565158f09e44f4d0241e24fb4a5 2024-08-20T23:21:57.4060154Z deleted: sha256:e4ff0c7bcc2fdb99e17c776477d5c18e9361ac0a2736df3dfb7dfd9cb1d24cb1 2024-08-20T23:21:57.4061002Z deleted: sha256:6c0717036d579c73658382280877c62371ee7fc9611c68dc252a79a79095e90f 2024-08-20T23:21:57.4061828Z deleted: sha256:5ba6e38220f1d4cae51bc9d06235689498debea840afb3e45a8d94dbe3ed4bd6 2024-08-20T23:21:57.4062695Z deleted: sha256:52e88c711fec52ba5484db46c10757a8f67fb2faa2859cb550c2504f7ea0215c 2024-08-20T23:21:57.4063554Z deleted: sha256:ee1ee62bb43ad224da336be09c15d5152e8f893e5316196fd9255f9dfa051e6c 2024-08-20T23:21:57.4064407Z deleted: sha256:db1d9c406a3c525dfa03fc55b7533d3ad07331ce244781f0685311dbf1c99b12 2024-08-20T23:21:57.4065265Z deleted: sha256:f76f7051ca457377b0f3b2af093d25534737c5a257e1fdf7ce18d0bd9f15120d 2024-08-20T23:21:57.4066116Z deleted: sha256:9fa921f68d6cd562a37209b977aa4519c4d188384865d3d8e17cabb85804a4d9 2024-08-20T23:21:57.4067104Z deleted: sha256:4245506f8114d5ad4c2e4e8e0d48207ea965e3321354a28ffd44f5e49eb02e0c 2024-08-20T23:21:57.4067909Z deleted: sha256:82526cd40c448218884561989680eba89858c126c18f041a4493345b7ff0fcf8 2024-08-20T23:21:57.4068757Z deleted: sha256:fe5ab7211ef36083ff39d37ea5310cad2fdeeaacb86b909ca510b8b69cc8b153 2024-08-20T23:21:57.4069637Z deleted: sha256:586e23ffbd3c14fd707eac41612ac1c0e9d61106de1d7665e417a51e28dfdbbe 2024-08-20T23:21:57.4070503Z deleted: sha256:bbdbcc9407efaafdaa6a1e4c8f51956a09f0083bc1e2548770b04142919d41fb 2024-08-20T23:21:57.4071363Z deleted: sha256:f8946873b0972358e88c1a71bbc540c0d7e98014ea8554485f8a085fd8ff6144 2024-08-20T23:21:57.4072204Z deleted: sha256:a6bbaaf810f89a9617f6c49b53c95731591890d326ff813a204036013fde1202 2024-08-20T23:21:57.4073039Z deleted: sha256:6ac45f1332600d2e42bc8ba9f7c4a22a23c87ec82a9f352bb7d70db70ff63604 2024-08-20T23:21:57.4073884Z deleted: sha256:97c28e8951f9ca49081cef7723673f019132b6684834ab5cd3cc3fdb850d3376 2024-08-20T23:21:57.4074727Z deleted: sha256:18093a570d0586f6bb9ef746c707bb3743a1e1d25e27a425e1ee56cabfefeb8d 2024-08-20T23:21:57.4075568Z deleted: sha256:d374585d459bd4a169573a2495ae419b4daf62d88513d87079ab7e04f6bd9ad5 2024-08-20T23:21:57.4076390Z deleted: sha256:31766aa82d623394ca9b63b1165907ce18cd4b9a0bc5608ec8e4d52acf98bf70 2024-08-20T23:21:57.4077230Z deleted: sha256:fa07904ebfa944942135ee06844871b4f28be2904d3e8a9705e0f2f68b02bf58 2024-08-20T23:21:57.4078077Z deleted: sha256:0f3b379bdaa956509f5104a95a0a237b7d8097aae3dd00963a605d12dacaa149 2024-08-20T23:21:57.4078924Z deleted: sha256:df3610f6c5e7351a273e0da0ebf0f169c0e52fb167da3fbbb4e4ca2cb5c34b6e 2024-08-20T23:21:57.4079787Z deleted: sha256:2b7ab0dc4db180ae6346f514863502dc7d5f322a228660d40c5625487a95a2da 2024-08-20T23:21:57.4080642Z deleted: sha256:cfbe220d95c10f637822c13fc6e6d08317940d78bd9e5aab395ebc3c8458151a 2024-08-20T23:21:57.4081491Z deleted: sha256:451f06689da0d7f6b2ded1da341a74f5dac58f1f9f7d2621be96dcd31c9ea581 2024-08-20T23:21:57.4082350Z deleted: sha256:829271df625d8997753cd47c019ecc424eebbc1da2a77251ee32c91f0b9640e2 2024-08-20T23:21:57.4083196Z deleted: sha256:c1c5c1249e1640be364b0d02f8912fa9edf3409852c5b002322e0efbe3631d51 2024-08-20T23:21:57.4084047Z deleted: sha256:424e5d38f16a88c5fc5e796a17f2c492bc462b16b7c7b7ce8e5f43e455daa51a 2024-08-20T23:21:57.4084892Z deleted: sha256:5350eaa4ea8f7e6443d9faf9163978c953dbb6d88f0a5c86f2fd3fcba68f69f2 2024-08-20T23:21:57.4085885Z deleted: sha256:fc5982aded69a44ef8731310809cbe67f55cbfca5b725d9aa9dd33322c7e4f71 2024-08-20T23:21:57.4086752Z deleted: sha256:894e7fd587d0d0d4607c0def34e735d47eeef86a050ced39d36e68e83333ed09 2024-08-20T23:21:57.4087592Z deleted: sha256:05c019f928fada63a413ef98cf2950d64f8117af6a4646b8fd15c602c56d5696 2024-08-20T23:21:57.4088454Z deleted: sha256:f1cd2eb056337dedc6952c668e89891bd494b7b1adc14d1cbaf6985418401e5a 2024-08-20T23:21:57.4089369Z deleted: sha256:ed97a0021fd34f50f544d15bf384159fde43539b4b71cc6128283f0c2b80e9cf 2024-08-20T23:21:57.4090309Z deleted: sha256:20d26c62d8be66b0ceae7e28bd8733d2f08b65ac595e88f9b531eba1970b26fe 2024-08-20T23:21:57.4091182Z deleted: sha256:10f41e5e3c42cf9e9b34cd50a011b7ee51004c749762737233bba0da479b388f 2024-08-20T23:21:57.4092029Z deleted: sha256:afbb9c9059b1c72194c682c5d265fe8b17011a7b6723645656fb9f1e9811528c 2024-08-20T23:21:57.4092877Z deleted: sha256:4362c875f11aa5ca8575e2f98651b30e0da7a6229b2d2c53cdf5fdc16fba2418 2024-08-20T23:21:57.4094032Z deleted: sha256:fa5d6b0e7b97ab4610f65c718ad4d75c27e62c29c61dedd196124bf3306330af 2024-08-20T23:21:57.4094903Z deleted: sha256:9739620f8ffc3c88bc57fd988134283f8f98f5ccfde916c6dd2fbb0cd6544dff 2024-08-20T23:21:57.4095746Z deleted: sha256:4f3e8b161de7f337b674841e642253443a68ebb56a2aade58fc7b1a5abb8ca67 2024-08-20T23:21:57.4096614Z deleted: sha256:c0d7a5ae04f2c46a907186321e1ada88c2d0daddf79a151fdd3e4c4055d25be0 2024-08-20T23:21:57.4097487Z deleted: sha256:bcb1a6e4b5da8e24c2539adf1943466267389d3aa76bcb63d955acc7ead340f6 2024-08-20T23:21:57.4098336Z deleted: sha256:e42d57df38426fdd6a7c8ffab2da9367f6560411c9978d11616a9cea58dfa587 2024-08-20T23:21:57.4099333Z deleted: sha256:8afca9ff63cb45e044380ee0d46025f38b3f484c74e0f42004e5671b417fb9be 2024-08-20T23:21:57.4100174Z deleted: sha256:28d3faa820053555b9081cda360ae1216e42689428f85f59f46144def51e79be 2024-08-20T23:21:57.4101006Z deleted: sha256:23be7c82be5c08957ec124bb6cfd898227e69b98f45c044d724180cc53942549 2024-08-20T23:21:57.4101850Z deleted: sha256:57adbda996a88cbbb0e18ef3b3af2f35697929595a343828d5d2892f5a36970a 2024-08-20T23:21:57.4102699Z deleted: sha256:8d149e9e773521fc0edc53831c87226a992066d0f57b637f932e7cb444592aea 2024-08-20T23:21:57.4103542Z deleted: sha256:eeb3bdd5b20188cad891eecc42f5568d5e1b119f0b86f4f597f599c523355737 2024-08-20T23:21:57.4104381Z deleted: sha256:c79b01ec230cbd7423a4cd411f275845a35764d355516c4030314ea9e09e7a3a 2024-08-20T23:21:57.4105228Z deleted: sha256:b04f4978eb8b562edf865ef1deedc442617b9c00f41f6e81f6a01949c5890087 2024-08-20T23:21:57.4106077Z deleted: sha256:be8e63aa3a7676161561b2b2e120ffd06d8f8b689588bdc09960570ac79c6fbe 2024-08-20T23:21:57.4106913Z deleted: sha256:c175ca5d09b7e7998a791dfd390deb23a97fd38dcd8c8392efe109d2376f1821 2024-08-20T23:21:57.4107769Z deleted: sha256:9e8bb347b233c66e9a8efb2783dba814fa979846a45e190f0821ddf5208c2955 2024-08-20T23:21:57.4108619Z deleted: sha256:6ff53beb2fc2f2bd235811b56056062cf4835f20790285ee922135d343836d91 2024-08-20T23:21:57.4109477Z deleted: sha256:361a9ca88dbc01c5345f0306e00d5acfbac9bd6b4de57a2ddb01cb799618b161 2024-08-20T23:21:57.4110330Z deleted: sha256:09b4536d5fb2828d961bfc5d5fdaf07980a4882f216c14419ce9d00424e63559 2024-08-20T23:21:57.4111179Z deleted: sha256:2a3665b48a131b5839ecaf5ec1ade50edaf2562b143283ba3beea46b0b1371d0 2024-08-20T23:21:57.4125352Z deleted: sha256:5bb95609ea3347bc0627bb89f8206a2a106d9569e942e4a6791d2cd3b211eb03 2024-08-20T23:21:57.4126331Z deleted: sha256:c78da1df7a4fdef88e067a6d69d2f51d4392e535ad973726f87521057cac4360 2024-08-20T23:21:57.4127220Z deleted: sha256:8e19a4ea3c026fbed7913cedded42d6e1f27b56e8534f213c48c0eec4fb3668b 2024-08-20T23:21:57.4128109Z deleted: sha256:ba8fb2cc055ddc87770d387ef61bd973c4066f0b412d4a025ccdd02795989499 2024-08-20T23:21:57.4128987Z deleted: sha256:75ac3a791e55cc40b22fde810086fb99ac453ab12546eddfecbb9158dec2786b 2024-08-20T23:21:57.4129931Z deleted: sha256:875c1779a1b3dd96a25d75839e3c65e71ed48cfb55f28b6530ce995e46f58fb0 2024-08-20T23:21:57.4130801Z deleted: sha256:5d9456aa782f8d5721bf5da9b96fa1e03ff6d30cb4f1c95c4aec4274c8843084 2024-08-20T23:21:57.4131874Z deleted: sha256:069c7e56627070abc1eef78c626d39542b3a9dbd7db3c00ba1d2bc5d07b10943 2024-08-20T23:21:57.4132800Z deleted: sha256:a445abe17034277479cd88b87a98f1a2a1fe6e85c8b631de04e81e2606d8eb43 2024-08-20T23:21:57.4133673Z deleted: sha256:3dff28f3d7b2ceed6cb9b567c8afe27ba1382cc613051130586a2c7a332de3cc 2024-08-20T23:21:57.4134555Z deleted: sha256:19aab9b70ad33c45d9fde55fdff5b52586844ee383317c0765d12b8e193fc269 2024-08-20T23:21:57.4135408Z deleted: sha256:c274afef2a17c385253c6d33c97b39cd4d03c5b5fdd9896cd36983756de313a3 2024-08-20T23:21:57.4136285Z deleted: sha256:a8aec88c47143bef5edc07e98182ecfa5ff9e7650c3bc7162ea99d61f3ed559b 2024-08-20T23:21:57.4137172Z deleted: sha256:8ad1c79513b6cbc5e44d911b4dea5840c2701a65dd2c7f92a2baa03a05426c06 2024-08-20T23:21:57.4138015Z deleted: sha256:a2d2902da87619b52c7294787417d3238f5049bdce806f88fbf2ae0fd54b9e08 2024-08-20T23:21:57.4138875Z deleted: sha256:1b9b7346fee7abbc7f5538eaa23548bd05a45abe8daf6794024be0c8ad7d60bb 2024-08-20T23:21:57.4139394Z 2024-08-20T23:21:57.4541444Z Total reclaimed space: 21.04GB 2024-08-20T23:21:57.4625184Z Post job cleanup. 2024-08-20T23:21:57.4677313Z Post job cleanup. 2024-08-20T23:21:57.5809049Z [command]/usr/bin/git version 2024-08-20T23:21:57.5852392Z git version 2.40.1 2024-08-20T23:21:57.5891168Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/153b3321-cf68-4a1b-8ce8-eccb9f96d4a3' before making global git config changes 2024-08-20T23:21:57.5892475Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T23:21:57.5896378Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T23:21:57.5936625Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T23:21:57.5970454Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T23:21:57.6399506Z Entering 'android/libs/fbjni' 2024-08-20T23:21:57.6452099Z Entering 'third_party/FP16' 2024-08-20T23:21:57.6502481Z Entering 'third_party/FXdiv' 2024-08-20T23:21:57.6553331Z Entering 'third_party/NNPACK' 2024-08-20T23:21:57.6603991Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:21:57.6654471Z Entering 'third_party/XNNPACK' 2024-08-20T23:21:57.6721064Z Entering 'third_party/benchmark' 2024-08-20T23:21:57.6772829Z Entering 'third_party/cpp-httplib' 2024-08-20T23:21:57.6822876Z Entering 'third_party/cpuinfo' 2024-08-20T23:21:57.6874136Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:21:57.6925878Z Entering 'third_party/cutlass' 2024-08-20T23:21:57.6985960Z Entering 'third_party/eigen' 2024-08-20T23:21:57.7040678Z Entering 'third_party/fbgemm' 2024-08-20T23:21:57.7092913Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:21:57.7142244Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:21:57.7195340Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:21:57.7251714Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:21:57.7302141Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:21:57.7353294Z Entering 'third_party/flatbuffers' 2024-08-20T23:21:57.7409966Z Entering 'third_party/fmt' 2024-08-20T23:21:57.7460657Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:21:57.7511940Z Entering 'third_party/gloo' 2024-08-20T23:21:57.7561788Z Entering 'third_party/googletest' 2024-08-20T23:21:57.7612985Z Entering 'third_party/ideep' 2024-08-20T23:21:57.7660940Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:21:57.7718433Z Entering 'third_party/ittapi' 2024-08-20T23:21:57.7768904Z Entering 'third_party/kineto' 2024-08-20T23:21:57.7818984Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:21:57.7867143Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:21:57.7917079Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:21:57.7965040Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:21:57.8014096Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:21:57.8061041Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:21:57.8113046Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:21:57.8160625Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:21:57.8212722Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:21:57.8262249Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:21:57.8313743Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:21:57.8362572Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:21:57.8414362Z Entering 'third_party/mimalloc' 2024-08-20T23:21:57.8464426Z Entering 'third_party/nccl/nccl' 2024-08-20T23:21:57.8516854Z Entering 'third_party/nlohmann' 2024-08-20T23:21:57.8567997Z Entering 'third_party/onnx' 2024-08-20T23:21:57.8635465Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:21:57.8685975Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:21:57.8741419Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:21:57.8792783Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:21:57.8842970Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:21:57.8892867Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:21:57.8943011Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:21:57.8996211Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:21:57.9046176Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:21:57.9097973Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:21:57.9146914Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:21:57.9199605Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:21:57.9252804Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:21:57.9325401Z Entering 'third_party/pocketfft' 2024-08-20T23:21:57.9375987Z Entering 'third_party/protobuf' 2024-08-20T23:21:57.9431137Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:21:57.9480404Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:21:57.9531962Z Entering 'third_party/psimd' 2024-08-20T23:21:57.9582292Z Entering 'third_party/pthreadpool' 2024-08-20T23:21:57.9632755Z Entering 'third_party/pybind11' 2024-08-20T23:21:57.9682918Z Entering 'third_party/python-peachpy' 2024-08-20T23:21:57.9735923Z Entering 'third_party/sleef' 2024-08-20T23:21:57.9786749Z Entering 'third_party/tensorpipe' 2024-08-20T23:21:57.9838544Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:21:57.9888814Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:21:57.9940109Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:21:57.9990436Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:21:58.0039987Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:21:58.0113711Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T23:21:58.0142945Z http.https://github.com/.extraheader 2024-08-20T23:21:58.0151906Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-20T23:21:58.0188861Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T23:21:58.0472548Z Entering 'android/libs/fbjni' 2024-08-20T23:21:58.0508573Z http.https://github.com/.extraheader 2024-08-20T23:21:58.0541215Z Entering 'third_party/FP16' 2024-08-20T23:21:58.0576112Z http.https://github.com/.extraheader 2024-08-20T23:21:58.0607915Z Entering 'third_party/FXdiv' 2024-08-20T23:21:58.0642091Z http.https://github.com/.extraheader 2024-08-20T23:21:58.0672931Z Entering 'third_party/NNPACK' 2024-08-20T23:21:58.0706788Z http.https://github.com/.extraheader 2024-08-20T23:21:58.0738719Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:21:58.0772706Z http.https://github.com/.extraheader 2024-08-20T23:21:58.0805466Z Entering 'third_party/XNNPACK' 2024-08-20T23:21:58.0839660Z http.https://github.com/.extraheader 2024-08-20T23:21:58.0889292Z Entering 'third_party/benchmark' 2024-08-20T23:21:58.0924185Z http.https://github.com/.extraheader 2024-08-20T23:21:58.0955981Z Entering 'third_party/cpp-httplib' 2024-08-20T23:21:58.0989519Z http.https://github.com/.extraheader 2024-08-20T23:21:58.1020573Z Entering 'third_party/cpuinfo' 2024-08-20T23:21:58.1053760Z http.https://github.com/.extraheader 2024-08-20T23:21:58.1085294Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:21:58.1120687Z http.https://github.com/.extraheader 2024-08-20T23:21:58.1151845Z Entering 'third_party/cutlass' 2024-08-20T23:21:58.1185396Z http.https://github.com/.extraheader 2024-08-20T23:21:58.1225567Z Entering 'third_party/eigen' 2024-08-20T23:21:58.1258407Z http.https://github.com/.extraheader 2024-08-20T23:21:58.1290912Z Entering 'third_party/fbgemm' 2024-08-20T23:21:58.1327153Z http.https://github.com/.extraheader 2024-08-20T23:21:58.1358452Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:21:58.1392105Z http.https://github.com/.extraheader 2024-08-20T23:21:58.1423384Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:21:58.1457070Z http.https://github.com/.extraheader 2024-08-20T23:21:58.1488960Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:21:58.1523814Z http.https://github.com/.extraheader 2024-08-20T23:21:58.1562853Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:21:58.1597390Z http.https://github.com/.extraheader 2024-08-20T23:21:58.1628527Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:21:58.1661306Z http.https://github.com/.extraheader 2024-08-20T23:21:58.1694267Z Entering 'third_party/flatbuffers' 2024-08-20T23:21:58.1729032Z http.https://github.com/.extraheader 2024-08-20T23:21:58.1763375Z Entering 'third_party/fmt' 2024-08-20T23:21:58.1798116Z http.https://github.com/.extraheader 2024-08-20T23:21:58.1830025Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:21:58.1864189Z http.https://github.com/.extraheader 2024-08-20T23:21:58.1896686Z Entering 'third_party/gloo' 2024-08-20T23:21:58.1931642Z http.https://github.com/.extraheader 2024-08-20T23:21:58.1963512Z Entering 'third_party/googletest' 2024-08-20T23:21:58.1998119Z http.https://github.com/.extraheader 2024-08-20T23:21:58.2029958Z Entering 'third_party/ideep' 2024-08-20T23:21:58.2064323Z http.https://github.com/.extraheader 2024-08-20T23:21:58.2095463Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:21:58.2128880Z http.https://github.com/.extraheader 2024-08-20T23:21:58.2168340Z Entering 'third_party/ittapi' 2024-08-20T23:21:58.2204343Z http.https://github.com/.extraheader 2024-08-20T23:21:58.2235934Z Entering 'third_party/kineto' 2024-08-20T23:21:58.2270306Z http.https://github.com/.extraheader 2024-08-20T23:21:58.2302944Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:21:58.2337311Z http.https://github.com/.extraheader 2024-08-20T23:21:58.2370427Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:21:58.2404915Z http.https://github.com/.extraheader 2024-08-20T23:21:58.2439946Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:21:58.2474926Z http.https://github.com/.extraheader 2024-08-20T23:21:58.2509066Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:21:58.2542516Z http.https://github.com/.extraheader 2024-08-20T23:21:58.2574597Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:21:58.2608315Z http.https://github.com/.extraheader 2024-08-20T23:21:58.2638757Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:21:58.2672670Z http.https://github.com/.extraheader 2024-08-20T23:21:58.2706195Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:21:58.2739505Z http.https://github.com/.extraheader 2024-08-20T23:21:58.2771283Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:21:58.2804888Z http.https://github.com/.extraheader 2024-08-20T23:21:58.2836811Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:21:58.2869805Z http.https://github.com/.extraheader 2024-08-20T23:21:58.2903623Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:21:58.2936798Z http.https://github.com/.extraheader 2024-08-20T23:21:58.2970205Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:21:58.3003615Z http.https://github.com/.extraheader 2024-08-20T23:21:58.3034225Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:21:58.3066321Z http.https://github.com/.extraheader 2024-08-20T23:21:58.3098650Z Entering 'third_party/mimalloc' 2024-08-20T23:21:58.3132259Z http.https://github.com/.extraheader 2024-08-20T23:21:58.3164187Z Entering 'third_party/nccl/nccl' 2024-08-20T23:21:58.3198148Z http.https://github.com/.extraheader 2024-08-20T23:21:58.3230506Z Entering 'third_party/nlohmann' 2024-08-20T23:21:58.3263982Z http.https://github.com/.extraheader 2024-08-20T23:21:58.3296848Z Entering 'third_party/onnx' 2024-08-20T23:21:58.3330725Z http.https://github.com/.extraheader 2024-08-20T23:21:58.3380814Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:21:58.3415723Z http.https://github.com/.extraheader 2024-08-20T23:21:58.3446895Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:21:58.3480026Z http.https://github.com/.extraheader 2024-08-20T23:21:58.3514236Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:21:58.3548936Z http.https://github.com/.extraheader 2024-08-20T23:21:58.3582371Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:21:58.3615505Z http.https://github.com/.extraheader 2024-08-20T23:21:58.3646510Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:21:58.3678880Z http.https://github.com/.extraheader 2024-08-20T23:21:58.3712934Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:21:58.3744926Z http.https://github.com/.extraheader 2024-08-20T23:21:58.3775594Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:21:58.3807470Z http.https://github.com/.extraheader 2024-08-20T23:21:58.3839254Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:21:58.3870925Z http.https://github.com/.extraheader 2024-08-20T23:21:58.3901230Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:21:58.3934691Z http.https://github.com/.extraheader 2024-08-20T23:21:58.3966237Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:21:58.3999788Z http.https://github.com/.extraheader 2024-08-20T23:21:58.4031072Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:21:58.4064303Z http.https://github.com/.extraheader 2024-08-20T23:21:58.4099097Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:21:58.4133375Z http.https://github.com/.extraheader 2024-08-20T23:21:58.4167170Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:21:58.4202214Z http.https://github.com/.extraheader 2024-08-20T23:21:58.4255423Z Entering 'third_party/pocketfft' 2024-08-20T23:21:58.4290305Z http.https://github.com/.extraheader 2024-08-20T23:21:58.4322090Z Entering 'third_party/protobuf' 2024-08-20T23:21:58.4356312Z http.https://github.com/.extraheader 2024-08-20T23:21:58.4391650Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:21:58.4425437Z http.https://github.com/.extraheader 2024-08-20T23:21:58.4456833Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:21:58.4489471Z http.https://github.com/.extraheader 2024-08-20T23:21:58.4523438Z Entering 'third_party/psimd' 2024-08-20T23:21:58.4559316Z http.https://github.com/.extraheader 2024-08-20T23:21:58.4591084Z Entering 'third_party/pthreadpool' 2024-08-20T23:21:58.4627877Z http.https://github.com/.extraheader 2024-08-20T23:21:58.4659705Z Entering 'third_party/pybind11' 2024-08-20T23:21:58.4692940Z http.https://github.com/.extraheader 2024-08-20T23:21:58.4724815Z Entering 'third_party/python-peachpy' 2024-08-20T23:21:58.4758802Z http.https://github.com/.extraheader 2024-08-20T23:21:58.4789728Z Entering 'third_party/sleef' 2024-08-20T23:21:58.4824301Z http.https://github.com/.extraheader 2024-08-20T23:21:58.4861961Z Entering 'third_party/tensorpipe' 2024-08-20T23:21:58.4896608Z http.https://github.com/.extraheader 2024-08-20T23:21:58.4929258Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:21:58.4962538Z http.https://github.com/.extraheader 2024-08-20T23:21:58.4994912Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:21:58.5028328Z http.https://github.com/.extraheader 2024-08-20T23:21:58.5059764Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:21:58.5093455Z http.https://github.com/.extraheader 2024-08-20T23:21:58.5126378Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:21:58.5160407Z http.https://github.com/.extraheader 2024-08-20T23:21:58.5191989Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:21:58.5228069Z http.https://github.com/.extraheader 2024-08-20T23:21:58.5350764Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-20T23:21:58.5369199Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-20T23:21:58.5374552Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:21:58.5375058Z ##[endgroup] 2024-08-20T23:22:05.4533437Z Cleaning up orphan processes