2024-08-20T21:35:51.8752710Z Current runner version: '2.319.1' 2024-08-20T21:35:51.8759282Z Runner name: 'i-0276d496c9ebff6c0' 2024-08-20T21:35:51.8760055Z Runner group name: 'Default' 2024-08-20T21:35:51.8760796Z Machine name: 'ip-10-0-30-173' 2024-08-20T21:35:51.8776985Z Testing runner upgrade compatibility 2024-08-20T21:35:52.1575428Z ##[group]GITHUB_TOKEN Permissions 2024-08-20T21:35:52.1577668Z Actions: read 2024-08-20T21:35:52.1578207Z Attestations: read 2024-08-20T21:35:52.1578674Z Checks: read 2024-08-20T21:35:52.1579110Z Contents: read 2024-08-20T21:35:52.1579557Z Deployments: read 2024-08-20T21:35:52.1580020Z Discussions: read 2024-08-20T21:35:52.1580473Z Issues: read 2024-08-20T21:35:52.1580892Z Metadata: read 2024-08-20T21:35:52.1581338Z Packages: read 2024-08-20T21:35:52.1581828Z Pages: read 2024-08-20T21:35:52.1582252Z PullRequests: read 2024-08-20T21:35:52.1582809Z RepositoryProjects: read 2024-08-20T21:35:52.1583333Z SecurityEvents: read 2024-08-20T21:35:52.1583821Z Statuses: read 2024-08-20T21:35:52.1584468Z ##[endgroup] 2024-08-20T21:35:52.1587860Z Secret source: Actions 2024-08-20T21:35:52.1588550Z Prepare workflow directory 2024-08-20T21:35:52.5249871Z Prepare all required actions 2024-08-20T21:35:52.5418941Z Getting action download info 2024-08-20T21:35:52.7615672Z Download action repository 'pytorch/test-infra@main' (SHA:0c3a2634aaa2f638c8f640e743f03d696ce1191f) 2024-08-20T21:35:53.1416087Z Download action repository 'pytorch/pytorch@main' (SHA:15b5a0b67fc3f34fb0bf1afa6f91e0c4c2b7fd8d) 2024-08-20T21:35:56.3750252Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-20T21:35:56.5255518Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-20T21:35:56.8313885Z Getting action download info 2024-08-20T21:35:56.9304455Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-20T21:35:57.1310522Z Getting action download info 2024-08-20T21:35:57.2571376Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-20T21:35:57.4057521Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/pull/133712/merge (f2fb9405c2fa9f9502a76363091cce6fd8179736) 2024-08-20T21:35:57.4060193Z ##[group] Inputs 2024-08-20T21:35:57.4060603Z build-environment: linux-jammy-py3.8-gcc11 2024-08-20T21:35:57.4064334Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]} 2024-08-20T21:35:57.4068570Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:35:57.4069538Z sync-tag: 2024-08-20T21:35:57.4070303Z timeout-minutes: 240 2024-08-20T21:35:57.4070621Z use-gha: 2024-08-20T21:35:57.4070876Z dashboard-tag: 2024-08-20T21:35:57.4071175Z s3-bucket: gha-artifacts 2024-08-20T21:35:57.4071508Z aws-role-to-assume: 2024-08-20T21:35:57.4071809Z ##[endgroup] 2024-08-20T21:35:57.4072583Z Complete job name: linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-20T21:35:57.4618425Z A job started hook has been configured by the self-hosted runner administrator 2024-08-20T21:35:57.4759631Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-20T21:35:57.4768873Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:35:57.4769394Z ##[endgroup] 2024-08-20T21:35:59.0301630Z Runner Type: amz2023.linux.2xlarge 2024-08-20T21:35:59.0302137Z Instance Type: c5.2xlarge 2024-08-20T21:35:59.0302822Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-20T21:35:59.0303400Z AMI ID: ami-06c68f701d8090592 2024-08-20T21:36:04.5915749Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-20T21:36:04.5916357Z with: 2024-08-20T21:36:04.5917041Z github-secret: *** 2024-08-20T21:36:04.5917993Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T21:36:04.5919014Z activate-with-label: false 2024-08-20T21:36:04.5919357Z label: with-ssh 2024-08-20T21:36:04.5919644Z remove-existing-keys: true 2024-08-20T21:36:04.5919986Z fail-silently: true 2024-08-20T21:36:04.5920290Z env: 2024-08-20T21:36:04.5920528Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:36:04.5920850Z ##[endgroup] 2024-08-20T21:36:04.6748354Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-20T21:36:04.9649685Z Grabbing public ssh keys from https://github.com/pytorchmergebot.keys 2024-08-20T21:36:05.0357743Z No SSH keys found for user pytorchmergebot 2024-08-20T21:36:05.0358576Z Grabbing public ssh keys from https://github.com/XuehaiPan.keys 2024-08-20T21:36:05.1199357Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-20T21:36:05.1212915Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-20T21:36:05.1236482Z Login using: ssh ec2-user@ec2-34-206-54-124.compute-1.amazonaws.com 2024-08-20T21:36:05.1237549Z All testing is done inside the container, to start an interactive session run: 2024-08-20T21:36:05.1238528Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-20T21:36:05.1381037Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-20T21:36:05.1381663Z with: 2024-08-20T21:36:05.1381931Z submodules: recursive 2024-08-20T21:36:05.1382234Z fetch-depth: 0 2024-08-20T21:36:05.1382497Z env: 2024-08-20T21:36:05.1382747Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:36:05.1383062Z ##[endgroup] 2024-08-20T21:36:05.1569616Z ##[group]Run retry () { 2024-08-20T21:36:05.1570242Z retry () { 2024-08-20T21:36:05.1570724Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-20T21:36:05.1571270Z } 2024-08-20T21:36:05.1571555Z echo "${GITHUB_WORKSPACE}" 2024-08-20T21:36:05.1572026Z if [ -z "${NO_SUDO}" ]; then 2024-08-20T21:36:05.1572492Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T21:36:05.1572931Z else 2024-08-20T21:36:05.1573241Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-20T21:36:05.1573668Z fi 2024-08-20T21:36:05.1573983Z mkdir "${GITHUB_WORKSPACE}" 2024-08-20T21:36:05.1582354Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:36:05.1582843Z env: 2024-08-20T21:36:05.1583096Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:36:05.1583417Z NO_SUDO: 2024-08-20T21:36:05.1583653Z ##[endgroup] 2024-08-20T21:36:05.1609535Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:36:07.3298909Z ##[group]Run malfet/checkout@silent-checkout 2024-08-20T21:36:07.3299334Z with: 2024-08-20T21:36:07.3299624Z ref: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:36:07.3300054Z fetch-depth: 0 2024-08-20T21:36:07.3300341Z submodules: recursive 2024-08-20T21:36:07.3300646Z quiet-checkout: true 2024-08-20T21:36:07.3300968Z repository: pytorch/pytorch 2024-08-20T21:36:07.3301442Z token: *** 2024-08-20T21:36:07.3301693Z ssh-strict: true 2024-08-20T21:36:07.3301999Z persist-credentials: true 2024-08-20T21:36:07.3302336Z clean: true 2024-08-20T21:36:07.3302815Z sparse-checkout-cone-mode: true 2024-08-20T21:36:07.3303186Z lfs: false 2024-08-20T21:36:07.3303465Z set-safe-directory: true 2024-08-20T21:36:07.3303771Z env: 2024-08-20T21:36:07.3304024Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:36:07.3304353Z ##[endgroup] 2024-08-20T21:36:07.4229218Z Syncing repository: pytorch/pytorch 2024-08-20T21:36:07.4230834Z ##[group]Getting Git version info 2024-08-20T21:36:07.4231848Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T21:36:07.4232895Z [command]/usr/bin/git version 2024-08-20T21:36:07.4233230Z git version 2.40.1 2024-08-20T21:36:07.4234553Z ##[endgroup] 2024-08-20T21:36:07.4245779Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/bccd4bd3-7b55-48f3-8516-ed8ac02bd41b' before making global git config changes 2024-08-20T21:36:07.4247059Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T21:36:07.4248214Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:36:07.4269562Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-20T21:36:07.4272994Z ##[group]Initializing the repository 2024-08-20T21:36:07.4275658Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:36:07.4366904Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-20T21:36:07.4367911Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-20T21:36:07.4368724Z hint: of your new repositories, which will suppress this warning, call: 2024-08-20T21:36:07.4369278Z hint: 2024-08-20T21:36:07.4370147Z hint: git config --global init.defaultBranch 2024-08-20T21:36:07.4370648Z hint: 2024-08-20T21:36:07.4371165Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-20T21:36:07.4372100Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-20T21:36:07.4372988Z hint: 2024-08-20T21:36:07.4373585Z hint: git branch -m 2024-08-20T21:36:07.4374359Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-20T21:36:07.4376493Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-20T21:36:07.4419279Z ##[endgroup] 2024-08-20T21:36:07.4419815Z ##[group]Disabling automatic garbage collection 2024-08-20T21:36:07.4422291Z [command]/usr/bin/git config --local gc.auto 0 2024-08-20T21:36:07.4449320Z ##[endgroup] 2024-08-20T21:36:07.4449801Z ##[group]Setting up auth 2024-08-20T21:36:07.4455297Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T21:36:07.4484219Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T21:36:07.4731015Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T21:36:07.4758900Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T21:36:07.5006149Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T21:36:07.5059659Z ##[endgroup] 2024-08-20T21:36:07.5060186Z ##[group]Fetching the repository 2024-08-20T21:36:07.5065615Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-20T21:36:10.2493119Z remote: Enumerating objects: 1008521 2024-08-20T21:36:10.2493720Z remote: Enumerating objects: 1010108, done. 2024-08-20T21:36:10.2496182Z remote: Counting objects: 0% (1/1587) 2024-08-20T21:36:10.2496948Z remote: Counting objects: 1% (16/1587) 2024-08-20T21:36:10.2498043Z remote: Counting objects: 2% (32/1587) 2024-08-20T21:36:10.2498790Z remote: Counting objects: 3% (48/1587) 2024-08-20T21:36:10.2499278Z remote: Counting objects: 4% (64/1587) 2024-08-20T21:36:10.2500112Z remote: Counting objects: 5% (80/1587) 2024-08-20T21:36:10.2500873Z remote: Counting objects: 6% (96/1587) 2024-08-20T21:36:10.2501553Z remote: Counting objects: 7% (112/1587) 2024-08-20T21:36:10.2502362Z remote: Counting objects: 8% (127/1587) 2024-08-20T21:36:10.2503223Z remote: Counting objects: 9% (143/1587) 2024-08-20T21:36:10.2504139Z remote: Counting objects: 10% (159/1587) 2024-08-20T21:36:10.2504897Z remote: Counting objects: 11% (175/1587) 2024-08-20T21:36:10.2505703Z remote: Counting objects: 12% (191/1587) 2024-08-20T21:36:10.2506549Z remote: Counting objects: 13% (207/1587) 2024-08-20T21:36:10.2507041Z remote: Counting objects: 14% (223/1587) 2024-08-20T21:36:10.2507556Z remote: Counting objects: 15% (239/1587) 2024-08-20T21:36:10.2508020Z remote: Counting objects: 16% (254/1587) 2024-08-20T21:36:10.2508738Z remote: Counting objects: 17% (270/1587) 2024-08-20T21:36:10.2509622Z remote: Counting objects: 18% (286/1587) 2024-08-20T21:36:10.2510140Z remote: Counting objects: 19% (302/1587) 2024-08-20T21:36:10.2510618Z remote: Counting objects: 20% (318/1587) 2024-08-20T21:36:10.2511102Z remote: Counting objects: 21% (334/1587) 2024-08-20T21:36:10.2511625Z remote: Counting objects: 22% (350/1587) 2024-08-20T21:36:10.2512415Z remote: Counting objects: 23% (366/1587) 2024-08-20T21:36:10.2512963Z remote: Counting objects: 24% (381/1587) 2024-08-20T21:36:10.2513632Z remote: Counting objects: 25% (397/1587) 2024-08-20T21:36:10.2514132Z remote: Counting objects: 26% (413/1587) 2024-08-20T21:36:10.2514711Z remote: Counting objects: 27% (429/1587) 2024-08-20T21:36:10.2515561Z remote: Counting objects: 28% (445/1587) 2024-08-20T21:36:10.2516196Z remote: Counting objects: 29% (461/1587) 2024-08-20T21:36:10.2516815Z remote: Counting objects: 30% (477/1587) 2024-08-20T21:36:10.2517546Z remote: Counting objects: 31% (492/1587) 2024-08-20T21:36:10.2518280Z remote: Counting objects: 32% (508/1587) 2024-08-20T21:36:10.2518934Z remote: Counting objects: 33% (524/1587) 2024-08-20T21:36:10.2519471Z remote: Counting objects: 34% (540/1587) 2024-08-20T21:36:10.2519948Z remote: Counting objects: 35% (556/1587) 2024-08-20T21:36:10.2520412Z remote: Counting objects: 36% (572/1587) 2024-08-20T21:36:10.2520895Z remote: Counting objects: 37% (588/1587) 2024-08-20T21:36:10.2521371Z remote: Counting objects: 38% (604/1587) 2024-08-20T21:36:10.2521834Z remote: Counting objects: 39% (619/1587) 2024-08-20T21:36:10.2522315Z remote: Counting objects: 40% (635/1587) 2024-08-20T21:36:10.2522814Z remote: Counting objects: 41% (651/1587) 2024-08-20T21:36:10.2523279Z remote: Counting objects: 42% (667/1587) 2024-08-20T21:36:10.2523760Z remote: Counting objects: 43% (683/1587) 2024-08-20T21:36:10.2524241Z remote: Counting objects: 44% (699/1587) 2024-08-20T21:36:10.2524706Z remote: Counting objects: 45% (715/1587) 2024-08-20T21:36:10.2525185Z remote: Counting objects: 46% (731/1587) 2024-08-20T21:36:10.2525662Z remote: Counting objects: 47% (746/1587) 2024-08-20T21:36:10.2526127Z remote: Counting objects: 48% (762/1587) 2024-08-20T21:36:10.2526603Z remote: Counting objects: 49% (778/1587) 2024-08-20T21:36:10.2527081Z remote: Counting objects: 50% (794/1587) 2024-08-20T21:36:10.2527543Z remote: Counting objects: 51% (810/1587) 2024-08-20T21:36:10.2528021Z remote: Counting objects: 52% (826/1587) 2024-08-20T21:36:10.2528500Z remote: Counting objects: 53% (842/1587) 2024-08-20T21:36:10.2529110Z remote: Counting objects: 54% (857/1587) 2024-08-20T21:36:10.2529579Z remote: Counting objects: 55% (873/1587) 2024-08-20T21:36:10.2530061Z remote: Counting objects: 56% (889/1587) 2024-08-20T21:36:10.2530542Z remote: Counting objects: 57% (905/1587) 2024-08-20T21:36:10.2531008Z remote: Counting objects: 58% (921/1587) 2024-08-20T21:36:10.2531487Z remote: Counting objects: 59% (937/1587) 2024-08-20T21:36:10.2532069Z remote: Counting objects: 60% (953/1587) 2024-08-20T21:36:10.2532538Z remote: Counting objects: 61% (969/1587) 2024-08-20T21:36:10.2533018Z remote: Counting objects: 62% (984/1587) 2024-08-20T21:36:10.2533525Z remote: Counting objects: 63% (1000/1587) 2024-08-20T21:36:10.2534023Z remote: Counting objects: 64% (1016/1587) 2024-08-20T21:36:10.2534511Z remote: Counting objects: 65% (1032/1587) 2024-08-20T21:36:10.2535013Z remote: Counting objects: 66% (1048/1587) 2024-08-20T21:36:10.2535514Z remote: Counting objects: 67% (1064/1587) 2024-08-20T21:36:10.2535998Z remote: Counting objects: 68% (1080/1587) 2024-08-20T21:36:10.2536493Z remote: Counting objects: 69% (1096/1587) 2024-08-20T21:36:10.2536989Z remote: Counting objects: 70% (1111/1587) 2024-08-20T21:36:10.2537473Z remote: Counting objects: 71% (1127/1587) 2024-08-20T21:36:10.2537966Z remote: Counting objects: 72% (1143/1587) 2024-08-20T21:36:10.2538459Z remote: Counting objects: 73% (1159/1587) 2024-08-20T21:36:10.2538941Z remote: Counting objects: 74% (1175/1587) 2024-08-20T21:36:10.2539439Z remote: Counting objects: 75% (1191/1587) 2024-08-20T21:36:10.2539933Z remote: Counting objects: 76% (1207/1587) 2024-08-20T21:36:10.2540434Z remote: Counting objects: 77% (1222/1587) 2024-08-20T21:36:10.2541190Z remote: Counting objects: 78% (1238/1587) 2024-08-20T21:36:10.2541785Z remote: Counting objects: 79% (1254/1587) 2024-08-20T21:36:10.2542287Z remote: Counting objects: 80% (1270/1587) 2024-08-20T21:36:10.2542792Z remote: Counting objects: 81% (1286/1587) 2024-08-20T21:36:10.2543293Z remote: Counting objects: 82% (1302/1587) 2024-08-20T21:36:10.2543789Z remote: Counting objects: 83% (1318/1587) 2024-08-20T21:36:10.2544273Z remote: Counting objects: 84% (1334/1587) 2024-08-20T21:36:10.2544773Z remote: Counting objects: 85% (1349/1587) 2024-08-20T21:36:10.2545269Z remote: Counting objects: 86% (1365/1587) 2024-08-20T21:36:10.2545754Z remote: Counting objects: 87% (1381/1587) 2024-08-20T21:36:10.2546249Z remote: Counting objects: 88% (1397/1587) 2024-08-20T21:36:10.2546743Z remote: Counting objects: 89% (1413/1587) 2024-08-20T21:36:10.2547237Z remote: Counting objects: 90% (1429/1587) 2024-08-20T21:36:10.2547715Z remote: Counting objects: 91% (1445/1587) 2024-08-20T21:36:10.2548214Z remote: Counting objects: 92% (1461/1587) 2024-08-20T21:36:10.2548707Z remote: Counting objects: 93% (1476/1587) 2024-08-20T21:36:10.2549189Z remote: Counting objects: 94% (1492/1587) 2024-08-20T21:36:10.2549682Z remote: Counting objects: 95% (1508/1587) 2024-08-20T21:36:10.2550176Z remote: Counting objects: 96% (1524/1587) 2024-08-20T21:36:10.2550655Z remote: Counting objects: 97% (1540/1587) 2024-08-20T21:36:10.2551145Z remote: Counting objects: 98% (1556/1587) 2024-08-20T21:36:10.2551635Z remote: Counting objects: 99% (1572/1587) 2024-08-20T21:36:10.2552113Z remote: Counting objects: 100% (1587/1587) 2024-08-20T21:36:10.2552647Z remote: Counting objects: 100% (1587/1587), done. 2024-08-20T21:36:10.2636993Z remote: Compressing objects: 0% (1/815) 2024-08-20T21:36:10.2800850Z remote: Compressing objects: 1% (9/815) 2024-08-20T21:36:10.3266813Z remote: Compressing objects: 2% (17/815) 2024-08-20T21:36:10.3894797Z remote: Compressing objects: 3% (25/815) 2024-08-20T21:36:10.4498650Z remote: Compressing objects: 4% (33/815) 2024-08-20T21:36:10.4968824Z remote: Compressing objects: 5% (41/815) 2024-08-20T21:36:10.5343520Z remote: Compressing objects: 6% (49/815) 2024-08-20T21:36:10.5711423Z remote: Compressing objects: 7% (58/815) 2024-08-20T21:36:10.5983945Z remote: Compressing objects: 8% (66/815) 2024-08-20T21:36:10.6269476Z remote: Compressing objects: 9% (74/815) 2024-08-20T21:36:10.6511585Z remote: Compressing objects: 10% (82/815) 2024-08-20T21:36:10.6720876Z remote: Compressing objects: 11% (90/815) 2024-08-20T21:36:10.6939742Z remote: Compressing objects: 12% (98/815) 2024-08-20T21:36:10.7100873Z remote: Compressing objects: 13% (106/815) 2024-08-20T21:36:10.7218228Z remote: Compressing objects: 14% (115/815) 2024-08-20T21:36:10.7296955Z remote: Compressing objects: 15% (123/815) 2024-08-20T21:36:10.7372865Z remote: Compressing objects: 16% (131/815) 2024-08-20T21:36:10.7451095Z remote: Compressing objects: 17% (139/815) 2024-08-20T21:36:10.7511059Z remote: Compressing objects: 18% (147/815) 2024-08-20T21:36:10.7551739Z remote: Compressing objects: 19% (155/815) 2024-08-20T21:36:10.7591468Z remote: Compressing objects: 20% (163/815) 2024-08-20T21:36:10.7621818Z remote: Compressing objects: 21% (172/815) 2024-08-20T21:36:10.7640164Z remote: Compressing objects: 22% (180/815) 2024-08-20T21:36:10.7643618Z remote: Compressing objects: 23% (188/815) 2024-08-20T21:36:10.7653348Z remote: Compressing objects: 24% (196/815) 2024-08-20T21:36:10.7658704Z remote: Compressing objects: 25% (204/815) 2024-08-20T21:36:10.7666313Z remote: Compressing objects: 26% (212/815) 2024-08-20T21:36:10.7692046Z remote: Compressing objects: 27% (221/815) 2024-08-20T21:36:10.7707257Z remote: Compressing objects: 28% (229/815) 2024-08-20T21:36:10.7722154Z remote: Compressing objects: 29% (237/815) 2024-08-20T21:36:10.7728994Z remote: Compressing objects: 30% (245/815) 2024-08-20T21:36:10.7736546Z remote: Compressing objects: 31% (253/815) 2024-08-20T21:36:10.7744321Z remote: Compressing objects: 32% (261/815) 2024-08-20T21:36:10.7749369Z remote: Compressing objects: 33% (269/815) 2024-08-20T21:36:10.7757570Z remote: Compressing objects: 34% (278/815) 2024-08-20T21:36:10.7762089Z remote: Compressing objects: 35% (286/815) 2024-08-20T21:36:10.7772590Z remote: Compressing objects: 36% (294/815) 2024-08-20T21:36:10.7778225Z remote: Compressing objects: 37% (302/815) 2024-08-20T21:36:10.7784185Z remote: Compressing objects: 38% (310/815) 2024-08-20T21:36:10.7788534Z remote: Compressing objects: 39% (318/815) 2024-08-20T21:36:10.7799786Z remote: Compressing objects: 40% (326/815) 2024-08-20T21:36:10.7803230Z remote: Compressing objects: 41% (335/815) 2024-08-20T21:36:10.7807782Z remote: Compressing objects: 42% (343/815) 2024-08-20T21:36:10.7813895Z remote: Compressing objects: 43% (351/815) 2024-08-20T21:36:10.7819059Z remote: Compressing objects: 44% (359/815) 2024-08-20T21:36:10.7823255Z remote: Compressing objects: 45% (367/815) 2024-08-20T21:36:10.7826684Z remote: Compressing objects: 46% (375/815) 2024-08-20T21:36:10.7829934Z remote: Compressing objects: 47% (384/815) 2024-08-20T21:36:10.7833164Z remote: Compressing objects: 48% (392/815) 2024-08-20T21:36:10.7835544Z remote: Compressing objects: 49% (400/815) 2024-08-20T21:36:10.7838134Z remote: Compressing objects: 50% (408/815) 2024-08-20T21:36:10.7841441Z remote: Compressing objects: 51% (416/815) 2024-08-20T21:36:10.7843455Z remote: Compressing objects: 52% (424/815) 2024-08-20T21:36:10.7845709Z remote: Compressing objects: 53% (432/815) 2024-08-20T21:36:10.7847603Z remote: Compressing objects: 54% (441/815) 2024-08-20T21:36:10.7849413Z remote: Compressing objects: 55% (449/815) 2024-08-20T21:36:10.7850097Z remote: Compressing objects: 56% (457/815) 2024-08-20T21:36:10.7851014Z remote: Compressing objects: 57% (465/815) 2024-08-20T21:36:10.7851766Z remote: Compressing objects: 58% (473/815) 2024-08-20T21:36:10.7852460Z remote: Compressing objects: 59% (481/815) 2024-08-20T21:36:10.7853035Z remote: Compressing objects: 60% (489/815) 2024-08-20T21:36:10.7853548Z remote: Compressing objects: 61% (498/815) 2024-08-20T21:36:10.7854914Z remote: Compressing objects: 62% (506/815) 2024-08-20T21:36:10.7855429Z remote: Compressing objects: 63% (514/815) 2024-08-20T21:36:10.7857669Z remote: Compressing objects: 64% (522/815) 2024-08-20T21:36:10.7866567Z remote: Compressing objects: 65% (530/815) 2024-08-20T21:36:10.7869591Z remote: Compressing objects: 66% (538/815) 2024-08-20T21:36:10.7873177Z remote: Compressing objects: 67% (547/815) 2024-08-20T21:36:10.7877480Z remote: Compressing objects: 68% (555/815) 2024-08-20T21:36:10.7880814Z remote: Compressing objects: 69% (563/815) 2024-08-20T21:36:10.7885080Z remote: Compressing objects: 70% (571/815) 2024-08-20T21:36:10.7887425Z remote: Compressing objects: 71% (579/815) 2024-08-20T21:36:10.7892408Z remote: Compressing objects: 72% (587/815) 2024-08-20T21:36:10.7894303Z remote: Compressing objects: 73% (595/815) 2024-08-20T21:36:10.7897054Z remote: Compressing objects: 74% (604/815) 2024-08-20T21:36:10.7899667Z remote: Compressing objects: 75% (612/815) 2024-08-20T21:36:10.7902060Z remote: Compressing objects: 76% (620/815) 2024-08-20T21:36:10.7904876Z remote: Compressing objects: 77% (628/815) 2024-08-20T21:36:10.7906921Z remote: Compressing objects: 78% (636/815) 2024-08-20T21:36:10.7908929Z remote: Compressing objects: 79% (644/815) 2024-08-20T21:36:10.7910724Z remote: Compressing objects: 80% (652/815) 2024-08-20T21:36:10.7912764Z remote: Compressing objects: 81% (661/815) 2024-08-20T21:36:10.7914900Z remote: Compressing objects: 82% (669/815) 2024-08-20T21:36:10.7915410Z remote: Compressing objects: 83% (677/815) 2024-08-20T21:36:10.7917354Z remote: Compressing objects: 84% (685/815) 2024-08-20T21:36:10.7919361Z remote: Compressing objects: 85% (693/815) 2024-08-20T21:36:10.7921322Z remote: Compressing objects: 86% (701/815) 2024-08-20T21:36:10.7923356Z remote: Compressing objects: 87% (710/815) 2024-08-20T21:36:10.7923888Z remote: Compressing objects: 88% (718/815) 2024-08-20T21:36:10.7925922Z remote: Compressing objects: 89% (726/815) 2024-08-20T21:36:10.7927993Z remote: Compressing objects: 90% (734/815) 2024-08-20T21:36:10.7928578Z remote: Compressing objects: 91% (742/815) 2024-08-20T21:36:10.7929100Z remote: Compressing objects: 92% (750/815) 2024-08-20T21:36:10.7929601Z remote: Compressing objects: 93% (758/815) 2024-08-20T21:36:10.7930327Z remote: Compressing objects: 94% (767/815) 2024-08-20T21:36:10.7932649Z remote: Compressing objects: 95% (775/815) 2024-08-20T21:36:10.7933163Z remote: Compressing objects: 96% (783/815) 2024-08-20T21:36:10.7934830Z remote: Compressing objects: 97% (791/815) 2024-08-20T21:36:10.7935341Z remote: Compressing objects: 98% (799/815) 2024-08-20T21:36:10.7936605Z remote: Compressing objects: 99% (807/815) 2024-08-20T21:36:10.7937103Z remote: Compressing objects: 100% (815/815) 2024-08-20T21:36:10.7937658Z remote: Compressing objects: 100% (815/815), done. 2024-08-20T21:36:30.7955738Z remote: Total 1010108 (delta 1021), reused 1239 (delta 769), pack-reused 1008521 (from 1) 2024-08-20T21:36:57.0642181Z [command]/usr/bin/git rev-parse --verify --quiet 40ec5f6ddd9787aca0449b24128343ff4c4a88b3^{object} 2024-08-20T21:36:57.0668004Z 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:36:57.0671924Z ##[endgroup] 2024-08-20T21:36:57.0673776Z ##[group]Determining the checkout info 2024-08-20T21:36:57.0674406Z ##[endgroup] 2024-08-20T21:36:57.0674897Z ##[group]Checking out the ref 2024-08-20T21:36:57.0677664Z [command]/usr/bin/git checkout --quiet --force 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:36:58.4316893Z ##[endgroup] 2024-08-20T21:36:58.4319418Z ##[group]Setting up auth for fetching submodules 2024-08-20T21:36:58.4323559Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-20T21:36:58.4372141Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-20T21:36:58.4400352Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-20T21:36:58.4428170Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-20T21:36:58.4454176Z ##[endgroup] 2024-08-20T21:36:58.4454672Z ##[group]Fetching submodules 2024-08-20T21:36:58.4457744Z [command]/usr/bin/git submodule sync --recursive 2024-08-20T21:36:58.4728790Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-20T21:36:58.4994126Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-20T21:36:58.4995911Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-20T21:36:58.4997719Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-20T21:36:58.5000068Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-20T21:36:58.5003117Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-20T21:36:58.5005145Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-20T21:36:58.5007626Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-20T21:36:58.5010716Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-20T21:36:58.5013500Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-20T21:36:58.5016420Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-20T21:36:58.5019418Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-20T21:36:58.5022438Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-20T21:36:58.5025603Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-20T21:36:58.5028910Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-20T21:36:58.5032091Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-20T21:36:58.5036155Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:36:58.5041156Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-20T21:36:58.5044924Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-20T21:36:58.5048576Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-20T21:36:58.5052641Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-20T21:36:58.5056519Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-20T21:36:58.5060926Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-20T21:36:58.5065061Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-20T21:36:58.5069191Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-20T21:36:58.5073373Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-20T21:36:58.5078138Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-20T21:36:58.5082338Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-20T21:36:58.5086958Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-20T21:36:58.5091654Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-20T21:36:58.5096603Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-20T21:36:58.5101309Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-20T21:36:58.5106447Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-20T21:36:58.5111363Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-20T21:36:58.5117860Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-20T21:36:58.5143193Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-20T21:36:58.7974638Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-20T21:36:58.9946890Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-20T21:36:59.2556010Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-20T21:36:59.4854838Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-20T21:37:01.6430441Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-20T21:37:15.4887521Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-20T21:37:15.9072959Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-20T21:37:16.3782603Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-20T21:37:16.9927485Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-20T21:37:18.1161219Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-20T21:37:20.1229812Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-20T21:37:25.3979123Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-20T21:37:26.9640327Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-20T21:37:28.7603909Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-20T21:37:30.2400498Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-20T21:37:30.6682983Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-20T21:37:30.9940274Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-20T21:37:32.2233728Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-20T21:37:32.6164784Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-20T21:37:32.9217412Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-20T21:37:34.3823070Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-20T21:37:35.1386156Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-20T21:37:35.8286878Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-20T21:37:42.4091040Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-20T21:37:44.7056840Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-20T21:37:50.8469863Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-20T21:37:51.0609161Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-20T21:38:00.7229196Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-20T21:38:00.8879143Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-20T21:38:01.0717191Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-20T21:38:02.0315621Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-20T21:38:02.3735415Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-20T21:38:03.0071338Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-20T21:38:03.4428231Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-20T21:38:03.4537244Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-20T21:38:03.4619664Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-20T21:38:03.4841981Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-20T21:38:03.5178857Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-20T21:38:04.4221722Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-20T21:38:04.4441471Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-20T21:38:04.4864385Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-20T21:38:04.5785352Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-20T21:38:04.6102422Z Submodule path 'third_party/cudnn_frontend': checked out '23511ba176243f27b3b275da1fb3814ea805a171' 2024-08-20T21:38:05.0884063Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-20T21:38:05.3320468Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-20T21:38:05.4101689Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-20T21:38:05.4116897Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:38:05.4119126Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:38:05.4121575Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:38:05.4124019Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:38:05.4126604Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:38:05.4150979Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-20T21:38:06.4718646Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-20T21:38:07.1488053Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-20T21:38:09.1609769Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-20T21:38:10.3702103Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-20T21:38:10.7060727Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-20T21:38:10.7965062Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-20T21:38:11.1765402Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-20T21:38:11.2357387Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-20T21:38:11.2473336Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-20T21:38:11.3582345Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-20T21:38:11.3973325Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-20T21:38:11.4350718Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-20T21:38:11.4586526Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-20T21:38:11.5009325Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T21:38:11.5126239Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-20T21:38:11.5140377Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-20T21:38:11.5164156Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-20T21:38:25.4807912Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-20T21:38:25.4978144Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-20T21:38:25.5816937Z Submodule path 'third_party/kineto': checked out 'd9753139d181b9ff42872465aac0e5d3018be415' 2024-08-20T21:38:25.5835120Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:38:25.5837361Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:38:25.5839213Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:38:25.5863558Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-20T21:38:26.1491137Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-20T21:38:27.4893218Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-20T21:38:28.6962364Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-20T21:38:28.6977511Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:38:28.6979778Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:38:28.6982310Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:38:28.6984698Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:38:28.6987333Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:38:28.6990133Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:38:28.6992815Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:38:28.6995724Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:38:28.7021409Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-20T21:38:29.5279676Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-20T21:38:29.8946582Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-20T21:38:31.2086659Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-20T21:38:31.4866391Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-20T21:38:32.0273465Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-20T21:38:33.1427236Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-20T21:38:40.2098207Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-20T21:38:40.6329251Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-20T21:38:40.6507419Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-20T21:38:40.6857449Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-20T21:38:40.6980977Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-20T21:38:40.6995050Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:38:40.7019174Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-20T21:38:41.0820971Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-20T21:38:41.0998049Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-20T21:38:41.1385182Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-20T21:38:41.2411837Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-20T21:38:41.2565672Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-20T21:38:41.2945799Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-20T21:38:41.3504629Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-20T21:38:41.3876458Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-20T21:38:41.4102652Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-20T21:38:41.5082373Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-20T21:38:41.8580380Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-20T21:38:41.8614193Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-20T21:38:41.8616263Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-20T21:38:41.8641696Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-20T21:38:42.4785534Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-20T21:38:43.6503353Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-20T21:38:43.6842572Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-20T21:38:43.7536073Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-20T21:38:43.7554297Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:38:43.7556516Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:38:43.7560083Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:38:43.7563021Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:38:43.7565952Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:38:43.7568422Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:38:43.7572210Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:38:43.7594798Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:38:43.7598942Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-20T21:38:44.1708319Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-20T21:38:45.3119581Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-20T21:38:45.6368432Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-20T21:38:51.8970321Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-20T21:38:52.1936604Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-20T21:38:52.3806168Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-20T21:38:52.6570398Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-20T21:38:59.2134721Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-20T21:38:59.2515994Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-20T21:38:59.2661461Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-20T21:38:59.3601520Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-20T21:38:59.3727686Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-20T21:38:59.3864517Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-20T21:38:59.4010941Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-20T21:38:59.4025249Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:38:59.4027387Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:38:59.4052882Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-20T21:39:01.5528138Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-20T21:39:02.9426734Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-20T21:39:02.9866334Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-20T21:39:03.4323520Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-20T21:39:03.4436326Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-20T21:39:03.6960043Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-20T21:39:03.6979447Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:39:03.6981462Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-20T21:39:03.7005526Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-20T21:39:04.1495497Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-20T21:39:05.3090921Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-20T21:39:05.3775748Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-20T21:39:05.3859414Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-20T21:39:05.3974134Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-20T21:39:05.4324684Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-20T21:39:05.4594393Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-20T21:39:05.4991684Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-20T21:39:05.5237650Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-20T21:39:05.5253533Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:39:05.5255613Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:39:05.5258088Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:39:05.5261747Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:39:05.5284912Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-20T21:39:06.7048020Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-20T21:39:06.9399664Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-20T21:39:08.1370964Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-20T21:39:09.0735811Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-20T21:39:09.0878146Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-20T21:39:09.1533178Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-20T21:39:09.1804495Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-20T21:39:09.1818451Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:39:09.1841569Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-20T21:39:09.3732834Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-20T21:39:09.3764834Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-20T21:39:09.4031973Z Entering 'android/libs/fbjni' 2024-08-20T21:39:09.4069373Z Entering 'third_party/FP16' 2024-08-20T21:39:09.4107026Z Entering 'third_party/FXdiv' 2024-08-20T21:39:09.4144810Z Entering 'third_party/NNPACK' 2024-08-20T21:39:09.4182814Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:39:09.4219984Z Entering 'third_party/XNNPACK' 2024-08-20T21:39:09.4275715Z Entering 'third_party/benchmark' 2024-08-20T21:39:09.4313239Z Entering 'third_party/cpp-httplib' 2024-08-20T21:39:09.4350165Z Entering 'third_party/cpuinfo' 2024-08-20T21:39:09.4388142Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:39:09.4426398Z Entering 'third_party/cutlass' 2024-08-20T21:39:09.4472085Z Entering 'third_party/eigen' 2024-08-20T21:39:09.4511032Z Entering 'third_party/fbgemm' 2024-08-20T21:39:09.4549479Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:39:09.4586562Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:39:09.4623798Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:39:09.4667809Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:39:09.4703975Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:39:09.4742100Z Entering 'third_party/flatbuffers' 2024-08-20T21:39:09.4781948Z Entering 'third_party/fmt' 2024-08-20T21:39:09.4819748Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:39:09.4856585Z Entering 'third_party/gloo' 2024-08-20T21:39:09.4895276Z Entering 'third_party/googletest' 2024-08-20T21:39:09.4932616Z Entering 'third_party/ideep' 2024-08-20T21:39:09.4969028Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:39:09.5012910Z Entering 'third_party/ittapi' 2024-08-20T21:39:09.5050072Z Entering 'third_party/kineto' 2024-08-20T21:39:09.5087465Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:39:09.5124263Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:39:09.5162388Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:39:09.5199318Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:39:09.5235959Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:39:09.5272319Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:39:09.5311339Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:39:09.5347553Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:39:09.5385560Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:39:09.5423527Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:39:09.5461352Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:39:09.5497476Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:39:09.5535000Z Entering 'third_party/mimalloc' 2024-08-20T21:39:09.5572995Z Entering 'third_party/nccl/nccl' 2024-08-20T21:39:09.5609986Z Entering 'third_party/nlohmann' 2024-08-20T21:39:09.5648416Z Entering 'third_party/onnx' 2024-08-20T21:39:09.5702413Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:39:09.5740135Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:39:09.5779835Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:39:09.5819034Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:39:09.5855358Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:39:09.5891607Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:39:09.5927047Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:39:09.5964922Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:39:09.6000765Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:39:09.6037231Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:39:09.6073356Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:39:09.6111644Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:39:09.6149380Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:39:09.6204192Z Entering 'third_party/pocketfft' 2024-08-20T21:39:09.6240921Z Entering 'third_party/protobuf' 2024-08-20T21:39:09.6282302Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:39:09.6319500Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:39:09.6357418Z Entering 'third_party/psimd' 2024-08-20T21:39:09.6394725Z Entering 'third_party/pthreadpool' 2024-08-20T21:39:09.6432025Z Entering 'third_party/pybind11' 2024-08-20T21:39:09.6469536Z Entering 'third_party/python-peachpy' 2024-08-20T21:39:09.6506100Z Entering 'third_party/sleef' 2024-08-20T21:39:09.6542911Z Entering 'third_party/tensorpipe' 2024-08-20T21:39:09.6580798Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:39:09.6617361Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:39:09.6653057Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:39:09.6690075Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:39:09.6725211Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:39:09.6775847Z ##[endgroup] 2024-08-20T21:39:09.6777461Z ##[group]Persisting credentials for submodules 2024-08-20T21:39:09.6780397Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-20T21:39:09.7045074Z Entering 'android/libs/fbjni' 2024-08-20T21:39:09.7092705Z Entering 'third_party/FP16' 2024-08-20T21:39:09.7139349Z Entering 'third_party/FXdiv' 2024-08-20T21:39:09.7187541Z Entering 'third_party/NNPACK' 2024-08-20T21:39:09.7236227Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:39:09.7284097Z Entering 'third_party/XNNPACK' 2024-08-20T21:39:09.7347301Z Entering 'third_party/benchmark' 2024-08-20T21:39:09.7395310Z Entering 'third_party/cpp-httplib' 2024-08-20T21:39:09.7442494Z Entering 'third_party/cpuinfo' 2024-08-20T21:39:09.7491872Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:39:09.7539660Z Entering 'third_party/cutlass' 2024-08-20T21:39:09.7593783Z Entering 'third_party/eigen' 2024-08-20T21:39:09.7643787Z Entering 'third_party/fbgemm' 2024-08-20T21:39:09.7692776Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:39:09.7738913Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:39:09.7787111Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:39:09.7839669Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:39:09.7887509Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:39:09.7935535Z Entering 'third_party/flatbuffers' 2024-08-20T21:39:09.7989643Z Entering 'third_party/fmt' 2024-08-20T21:39:09.8039558Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:39:09.8088697Z Entering 'third_party/gloo' 2024-08-20T21:39:09.8138264Z Entering 'third_party/googletest' 2024-08-20T21:39:09.8188400Z Entering 'third_party/ideep' 2024-08-20T21:39:09.8238090Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:39:09.8293866Z Entering 'third_party/ittapi' 2024-08-20T21:39:09.8345685Z Entering 'third_party/kineto' 2024-08-20T21:39:09.8396970Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:39:09.8446573Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:39:09.8498553Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:39:09.8549704Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:39:09.8599539Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:39:09.8649074Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:39:09.8700799Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:39:09.8749417Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:39:09.8798216Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:39:09.8847694Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:39:09.8898913Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:39:09.8947135Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:39:09.8997254Z Entering 'third_party/mimalloc' 2024-08-20T21:39:09.9046443Z Entering 'third_party/nccl/nccl' 2024-08-20T21:39:09.9097174Z Entering 'third_party/nlohmann' 2024-08-20T21:39:09.9147297Z Entering 'third_party/onnx' 2024-08-20T21:39:09.9210868Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:39:09.9260130Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:39:09.9310532Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:39:09.9361516Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:39:09.9409557Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:39:09.9457371Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:39:09.9504876Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:39:09.9553737Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:39:09.9601331Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:39:09.9648227Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:39:09.9695192Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:39:09.9744970Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:39:09.9795780Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:39:09.9862391Z Entering 'third_party/pocketfft' 2024-08-20T21:39:09.9912543Z Entering 'third_party/protobuf' 2024-08-20T21:39:09.9963467Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:39:10.0010803Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:39:10.0060911Z Entering 'third_party/psimd' 2024-08-20T21:39:10.0109054Z Entering 'third_party/pthreadpool' 2024-08-20T21:39:10.0157572Z Entering 'third_party/pybind11' 2024-08-20T21:39:10.0207508Z Entering 'third_party/python-peachpy' 2024-08-20T21:39:10.0255046Z Entering 'third_party/sleef' 2024-08-20T21:39:10.0303004Z Entering 'third_party/tensorpipe' 2024-08-20T21:39:10.0350791Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:39:10.0398691Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:39:10.0445705Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:39:10.0494628Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:39:10.0541004Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:39:10.0605755Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-20T21:39:10.0872570Z Entering 'android/libs/fbjni' 2024-08-20T21:39:10.0917519Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-20T21:39:10.0932031Z Entering 'third_party/FP16' 2024-08-20T21:39:10.0978058Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-20T21:39:10.0992427Z Entering 'third_party/FXdiv' 2024-08-20T21:39:10.1038725Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-20T21:39:10.1053219Z Entering 'third_party/NNPACK' 2024-08-20T21:39:10.1098954Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-20T21:39:10.1113680Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:39:10.1159666Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-20T21:39:10.1174430Z Entering 'third_party/XNNPACK' 2024-08-20T21:39:10.1220732Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-20T21:39:10.1252693Z Entering 'third_party/benchmark' 2024-08-20T21:39:10.1298949Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:39:10.1313549Z Entering 'third_party/cpp-httplib' 2024-08-20T21:39:10.1359251Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-20T21:39:10.1374484Z Entering 'third_party/cpuinfo' 2024-08-20T21:39:10.1420005Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T21:39:10.1435476Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:39:10.1481236Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-20T21:39:10.1495628Z Entering 'third_party/cutlass' 2024-08-20T21:39:10.1542519Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-20T21:39:10.1565394Z Entering 'third_party/eigen' 2024-08-20T21:39:10.1610913Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-20T21:39:10.1627087Z Entering 'third_party/fbgemm' 2024-08-20T21:39:10.1673440Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-20T21:39:10.1687515Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:39:10.1733246Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-20T21:39:10.1746955Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:39:10.1794272Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-20T21:39:10.1808541Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:39:10.1853286Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-20T21:39:10.1873252Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:39:10.1918039Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:39:10.1931885Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:39:10.1976952Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-20T21:39:10.1992159Z Entering 'third_party/flatbuffers' 2024-08-20T21:39:10.2037854Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-20T21:39:10.2054728Z Entering 'third_party/fmt' 2024-08-20T21:39:10.2100170Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-20T21:39:10.2114718Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:39:10.2161292Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-20T21:39:10.2175918Z Entering 'third_party/gloo' 2024-08-20T21:39:10.2220956Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-20T21:39:10.2235440Z Entering 'third_party/googletest' 2024-08-20T21:39:10.2280590Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:39:10.2295070Z Entering 'third_party/ideep' 2024-08-20T21:39:10.2340594Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-20T21:39:10.2353772Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:39:10.2400387Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-20T21:39:10.2421941Z Entering 'third_party/ittapi' 2024-08-20T21:39:10.2469581Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-20T21:39:10.2483987Z Entering 'third_party/kineto' 2024-08-20T21:39:10.2529787Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-20T21:39:10.2543827Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:39:10.2590436Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-20T21:39:10.2603956Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:39:10.2649780Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-20T21:39:10.2665244Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:39:10.2710763Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-20T21:39:10.2724180Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:39:10.2769802Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-20T21:39:10.2784379Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:39:10.2830720Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-20T21:39:10.2843868Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:39:10.2890438Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-20T21:39:10.2906442Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:39:10.2951962Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-20T21:39:10.2966285Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:39:10.3012437Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:39:10.3026700Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:39:10.3072802Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-20T21:39:10.3087857Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:39:10.3133392Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-20T21:39:10.3148826Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:39:10.3194504Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-20T21:39:10.3208643Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:39:10.3253930Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-20T21:39:10.3270646Z Entering 'third_party/mimalloc' 2024-08-20T21:39:10.3315913Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-20T21:39:10.3330658Z Entering 'third_party/nccl/nccl' 2024-08-20T21:39:10.3376469Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-20T21:39:10.3391207Z Entering 'third_party/nlohmann' 2024-08-20T21:39:10.3436336Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-20T21:39:10.3452132Z Entering 'third_party/onnx' 2024-08-20T21:39:10.3497754Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-20T21:39:10.3527169Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:39:10.3572662Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:39:10.3586700Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:39:10.3631505Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-20T21:39:10.3647865Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:39:10.3695160Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-20T21:39:10.3711008Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:39:10.3756258Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:39:10.3771007Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:39:10.3815736Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:39:10.3829558Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:39:10.3874790Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-20T21:39:10.3888432Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:39:10.3932869Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-20T21:39:10.3947751Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:39:10.3993200Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-20T21:39:10.4006669Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:39:10.4051194Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-20T21:39:10.4065198Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:39:10.4110514Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-20T21:39:10.4123480Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:39:10.4170267Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-20T21:39:10.4186097Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:39:10.4231932Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-20T21:39:10.4247173Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:39:10.4295347Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-20T21:39:10.4327659Z Entering 'third_party/pocketfft' 2024-08-20T21:39:10.4373852Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-20T21:39:10.4388317Z Entering 'third_party/protobuf' 2024-08-20T21:39:10.4433566Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-20T21:39:10.4450622Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:39:10.4497117Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-20T21:39:10.4511222Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:39:10.4555947Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:39:10.4572320Z Entering 'third_party/psimd' 2024-08-20T21:39:10.4618567Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-20T21:39:10.4633016Z Entering 'third_party/pthreadpool' 2024-08-20T21:39:10.4679598Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-20T21:39:10.4694135Z Entering 'third_party/pybind11' 2024-08-20T21:39:10.4739275Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-20T21:39:10.4753616Z Entering 'third_party/python-peachpy' 2024-08-20T21:39:10.4799120Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-20T21:39:10.4813595Z Entering 'third_party/sleef' 2024-08-20T21:39:10.4858020Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-20T21:39:10.4872588Z Entering 'third_party/tensorpipe' 2024-08-20T21:39:10.4918575Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-20T21:39:10.4932533Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:39:10.4978017Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-20T21:39:10.4991896Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:39:10.5036553Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-20T21:39:10.5050186Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:39:10.5095224Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-20T21:39:10.5109342Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:39:10.5154787Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-20T21:39:10.5168128Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:39:10.5216087Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-20T21:39:10.5894160Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-20T21:39:10.6162755Z Entering 'android/libs/fbjni' 2024-08-20T21:39:10.6201016Z Entering 'third_party/FP16' 2024-08-20T21:39:10.6238713Z Entering 'third_party/FXdiv' 2024-08-20T21:39:10.6277784Z Entering 'third_party/NNPACK' 2024-08-20T21:39:10.6316295Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:39:10.6354106Z Entering 'third_party/XNNPACK' 2024-08-20T21:39:10.6407706Z Entering 'third_party/benchmark' 2024-08-20T21:39:10.6445094Z Entering 'third_party/cpp-httplib' 2024-08-20T21:39:10.6482976Z Entering 'third_party/cpuinfo' 2024-08-20T21:39:10.6520702Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:39:10.6559502Z Entering 'third_party/cutlass' 2024-08-20T21:39:10.6602936Z Entering 'third_party/eigen' 2024-08-20T21:39:10.6643374Z Entering 'third_party/fbgemm' 2024-08-20T21:39:10.6681808Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:39:10.6717969Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:39:10.6754476Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:39:10.6797667Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:39:10.6834479Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:39:10.6872315Z Entering 'third_party/flatbuffers' 2024-08-20T21:39:10.6911817Z Entering 'third_party/fmt' 2024-08-20T21:39:10.6948836Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:39:10.6987182Z Entering 'third_party/gloo' 2024-08-20T21:39:10.7024500Z Entering 'third_party/googletest' 2024-08-20T21:39:10.7062787Z Entering 'third_party/ideep' 2024-08-20T21:39:10.7098852Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:39:10.7142205Z Entering 'third_party/ittapi' 2024-08-20T21:39:10.7181001Z Entering 'third_party/kineto' 2024-08-20T21:39:10.7218298Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:39:10.7254824Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:39:10.7292472Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:39:10.7328895Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:39:10.7365098Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:39:10.7400828Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:39:10.7439248Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:39:10.7476925Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:39:10.7513313Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:39:10.7550605Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:39:10.7588989Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:39:10.7624740Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:39:10.7662732Z Entering 'third_party/mimalloc' 2024-08-20T21:39:10.7700684Z Entering 'third_party/nccl/nccl' 2024-08-20T21:39:10.7738334Z Entering 'third_party/nlohmann' 2024-08-20T21:39:10.7776907Z Entering 'third_party/onnx' 2024-08-20T21:39:10.7828827Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:39:10.7865683Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:39:10.7904341Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:39:10.7943152Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:39:10.7979547Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:39:10.8015547Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:39:10.8051508Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:39:10.8089177Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:39:10.8124934Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:39:10.8161234Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:39:10.8196992Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:39:10.8237408Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:39:10.8276432Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:39:10.8331629Z Entering 'third_party/pocketfft' 2024-08-20T21:39:10.8369602Z Entering 'third_party/protobuf' 2024-08-20T21:39:10.8409307Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:39:10.8445409Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:39:10.8484234Z Entering 'third_party/psimd' 2024-08-20T21:39:10.8521673Z Entering 'third_party/pthreadpool' 2024-08-20T21:39:10.8559453Z Entering 'third_party/pybind11' 2024-08-20T21:39:10.8596740Z Entering 'third_party/python-peachpy' 2024-08-20T21:39:10.8633747Z Entering 'third_party/sleef' 2024-08-20T21:39:10.8671166Z Entering 'third_party/tensorpipe' 2024-08-20T21:39:10.8708739Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:39:10.8745510Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:39:10.8781403Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:39:10.8817529Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:39:10.8852982Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:39:10.8906274Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-20T21:39:10.9174863Z Entering 'android/libs/fbjni' 2024-08-20T21:39:10.9212336Z Entering 'third_party/FP16' 2024-08-20T21:39:10.9249924Z Entering 'third_party/FXdiv' 2024-08-20T21:39:10.9287586Z Entering 'third_party/NNPACK' 2024-08-20T21:39:10.9325084Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T21:39:10.9363047Z Entering 'third_party/XNNPACK' 2024-08-20T21:39:10.9416053Z Entering 'third_party/benchmark' 2024-08-20T21:39:10.9453538Z Entering 'third_party/cpp-httplib' 2024-08-20T21:39:10.9492200Z Entering 'third_party/cpuinfo' 2024-08-20T21:39:10.9529714Z Entering 'third_party/cudnn_frontend' 2024-08-20T21:39:10.9567541Z Entering 'third_party/cutlass' 2024-08-20T21:39:10.9610779Z Entering 'third_party/eigen' 2024-08-20T21:39:10.9649541Z Entering 'third_party/fbgemm' 2024-08-20T21:39:10.9688598Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T21:39:10.9725392Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T21:39:10.9761683Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T21:39:10.9803804Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T21:39:10.9840000Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T21:39:10.9878168Z Entering 'third_party/flatbuffers' 2024-08-20T21:39:10.9917361Z Entering 'third_party/fmt' 2024-08-20T21:39:10.9954322Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T21:39:10.9992076Z Entering 'third_party/gloo' 2024-08-20T21:39:11.0029448Z Entering 'third_party/googletest' 2024-08-20T21:39:11.0066718Z Entering 'third_party/ideep' 2024-08-20T21:39:11.0103270Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T21:39:11.0147550Z Entering 'third_party/ittapi' 2024-08-20T21:39:11.0184905Z Entering 'third_party/kineto' 2024-08-20T21:39:11.0221892Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T21:39:11.0259514Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T21:39:11.0298643Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T21:39:11.0335727Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T21:39:11.0372075Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T21:39:11.0408239Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T21:39:11.0447103Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T21:39:11.0485908Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T21:39:11.0522930Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T21:39:11.0561307Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T21:39:11.0600043Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T21:39:11.0636803Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T21:39:11.0674974Z Entering 'third_party/mimalloc' 2024-08-20T21:39:11.0713255Z Entering 'third_party/nccl/nccl' 2024-08-20T21:39:11.0751511Z Entering 'third_party/nlohmann' 2024-08-20T21:39:11.0793072Z Entering 'third_party/onnx' 2024-08-20T21:39:11.0845557Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T21:39:11.0883157Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T21:39:11.0923255Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T21:39:11.0963087Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T21:39:11.0999020Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T21:39:11.1035223Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T21:39:11.1071605Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T21:39:11.1108832Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T21:39:11.1145358Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T21:39:11.1181190Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T21:39:11.1216661Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T21:39:11.1255262Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T21:39:11.1293687Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T21:39:11.1349415Z Entering 'third_party/pocketfft' 2024-08-20T21:39:11.1387945Z Entering 'third_party/protobuf' 2024-08-20T21:39:11.1428123Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T21:39:11.1464980Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T21:39:11.1503566Z Entering 'third_party/psimd' 2024-08-20T21:39:11.1540598Z Entering 'third_party/pthreadpool' 2024-08-20T21:39:11.1578402Z Entering 'third_party/pybind11' 2024-08-20T21:39:11.1615692Z Entering 'third_party/python-peachpy' 2024-08-20T21:39:11.1653192Z Entering 'third_party/sleef' 2024-08-20T21:39:11.1692655Z Entering 'third_party/tensorpipe' 2024-08-20T21:39:11.1729405Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T21:39:11.1766104Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T21:39:11.1801745Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T21:39:11.1837853Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T21:39:11.1875581Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T21:39:11.1924878Z ##[endgroup] 2024-08-20T21:39:11.1961199Z [command]/usr/bin/git log -1 --format='%H' 2024-08-20T21:39:11.1986108Z '40ec5f6ddd9787aca0449b24128343ff4c4a88b3' 2024-08-20T21:39:11.2170608Z Prepare all required actions 2024-08-20T21:39:11.2171052Z Getting action download info 2024-08-20T21:39:11.3895040Z ##[group]Run ./.github/actions/setup-linux 2024-08-20T21:39:11.3895451Z env: 2024-08-20T21:39:11.3895707Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:11.3896133Z ##[endgroup] 2024-08-20T21:39:11.3957475Z ##[group]Run set -euo pipefail 2024-08-20T21:39:11.3958084Z set -euo pipefail 2024-08-20T21:39:11.3958440Z function get_ec2_metadata() { 2024-08-20T21:39:11.3958955Z  # Pulled from instance metadata endpoint for EC2 2024-08-20T21:39:11.3959855Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-20T21:39:11.3960618Z  category=$1 2024-08-20T21:39:11.3961125Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-20T21:39:11.3961744Z  runner_name_str=i-0276d496c9ebff6c0 2024-08-20T21:39:11.3962220Z  if [[ -f /.inarc ]]; then 2024-08-20T21:39:11.3962695Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-20T21:39:11.3963229Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-20T21:39:11.3963912Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-20T21:39:11.3964518Z  else 2024-08-20T21:39:11.3964975Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-20T21:39:11.3965531Z  fi 2024-08-20T21:39:11.3965790Z } 2024-08-20T21:39:11.3966110Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-20T21:39:11.3966678Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-20T21:39:11.3967317Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-20T21:39:11.3967857Z echo "system info $(uname -a)" 2024-08-20T21:39:11.3975576Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:11.3976061Z env: 2024-08-20T21:39:11.3976315Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:11.3976640Z ##[endgroup] 2024-08-20T21:39:11.4100460Z ami-id: ami-06c68f701d8090592 2024-08-20T21:39:11.4149950Z instance-id: i-0276d496c9ebff6c0 2024-08-20T21:39:11.4195902Z instance-type: c5.2xlarge 2024-08-20T21:39:11.4206873Z system info Linux ip-10-0-30-173.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-20T21:39:11.4234745Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T21:39:11.4235643Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-20T21:39:11.4241718Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:11.4242204Z env: 2024-08-20T21:39:11.4242454Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:11.4242762Z ##[endgroup] 2024-08-20T21:39:11.4307100Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-20T21:39:11.4307685Z if systemctl is-active --quiet docker; then 2024-08-20T21:39:11.4308207Z  echo "Docker daemon is running..."; 2024-08-20T21:39:11.4308637Z else 2024-08-20T21:39:11.4309095Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-20T21:39:11.4309672Z fi 2024-08-20T21:39:11.4314737Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:11.4315206Z env: 2024-08-20T21:39:11.4315460Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:11.4315782Z ##[endgroup] 2024-08-20T21:39:11.4381387Z Docker daemon is running... 2024-08-20T21:39:11.4450804Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T21:39:11.4451356Z with: 2024-08-20T21:39:11.4451699Z shell: bash 2024-08-20T21:39:11.4451956Z timeout_minutes: 5 2024-08-20T21:39:11.4452254Z max_attempts: 3 2024-08-20T21:39:11.4452542Z retry_wait_seconds: 30 2024-08-20T21:39:11.4455815Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" # For LF Runners we need to make sure we also login to Meta's ECR docker registry too. META_AWS_ACCOUNT_ID=308535385114 if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" fi 2024-08-20T21:39:11.4459338Z polling_interval_seconds: 1 2024-08-20T21:39:11.4459695Z warning_on_retry: true 2024-08-20T21:39:11.4460014Z continue_on_error: false 2024-08-20T21:39:11.4460332Z env: 2024-08-20T21:39:11.4460576Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:11.4460889Z AWS_RETRY_MODE: standard 2024-08-20T21:39:11.4461213Z AWS_MAX_ATTEMPTS: 5 2024-08-20T21:39:11.4461529Z AWS_DEFAULT_REGION: us-east-1 2024-08-20T21:39:11.4461868Z ##[endgroup] 2024-08-20T21:39:12.5460419Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T21:39:12.5461262Z Configure a credential helper to remove this warning. See 2024-08-20T21:39:12.5462097Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T21:39:12.5462670Z 2024-08-20T21:39:12.5462773Z Login Succeeded 2024-08-20T21:39:13.4983026Z Command completed after 1 attempt(s). 2024-08-20T21:39:13.5030889Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T21:39:13.5031610Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T21:39:13.5032255Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-20T21:39:13.5038126Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:13.5038599Z env: 2024-08-20T21:39:13.5038857Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:13.5039183Z ##[endgroup] 2024-08-20T21:39:13.5114588Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T21:39:13.5115343Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T21:39:13.5115920Z # shellcheck disable=SC2046 2024-08-20T21:39:13.5116353Z docker stop $(docker ps -q) || true 2024-08-20T21:39:13.5116807Z # Prune all of the docker images 2024-08-20T21:39:13.5117238Z docker system prune -af 2024-08-20T21:39:13.5122796Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:13.5123262Z env: 2024-08-20T21:39:13.5123513Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:13.5123834Z ##[endgroup] 2024-08-20T21:39:13.5411052Z "docker stop" requires at least 1 argument. 2024-08-20T21:39:13.5412396Z See 'docker stop --help'. 2024-08-20T21:39:13.5412673Z 2024-08-20T21:39:13.5412888Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-20T21:39:13.5413271Z 2024-08-20T21:39:13.5413406Z Stop one or more running containers 2024-08-20T21:39:13.5552204Z Total reclaimed space: 0B 2024-08-20T21:39:13.5589050Z ##[group]Run set +e 2024-08-20T21:39:13.5589395Z set +e 2024-08-20T21:39:13.5589663Z set -x 2024-08-20T21:39:13.5589934Z  2024-08-20T21:39:13.5590229Z PT_DOMAIN=download.pytorch.org 2024-08-20T21:39:13.5591013Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-20T21:39:13.5592094Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-20T21:39:13.5592844Z # one is returned at random 2024-08-20T21:39:13.5593350Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-20T21:39:13.5593844Z  2024-08-20T21:39:13.5594135Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T21:39:13.5594721Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-20T21:39:13.5595604Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-20T21:39:13.5596149Z  2024-08-20T21:39:13.5596450Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-20T21:39:13.5596970Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-20T21:39:13.5597463Z  exit 1 2024-08-20T21:39:13.5597850Z  fi 2024-08-20T21:39:13.5598094Z fi 2024-08-20T21:39:13.5598346Z  2024-08-20T21:39:13.5598677Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-20T21:39:13.5599160Z  # Clean up any old records first 2024-08-20T21:39:13.5599651Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-20T21:39:13.5600087Z fi 2024-08-20T21:39:13.5600323Z  2024-08-20T21:39:13.5600730Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-20T21:39:13.5601262Z cat /etc/hosts 2024-08-20T21:39:13.5607036Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:13.5607522Z env: 2024-08-20T21:39:13.5607785Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:13.5608098Z ##[endgroup] 2024-08-20T21:39:13.5636388Z + PT_DOMAIN=download.pytorch.org 2024-08-20T21:39:13.5642396Z ++ dig -4 +short download.pytorch.org 2024-08-20T21:39:13.5643207Z ++ tail -n1 2024-08-20T21:39:13.6202617Z + RESOLVED_IP=18.165.98.58 2024-08-20T21:39:13.6203449Z + '[' -z 18.165.98.58 ']' 2024-08-20T21:39:13.6203861Z + grep -r download.pytorch.org /etc/hosts 2024-08-20T21:39:13.6213092Z 18.165.98.47 download.pytorch.org 2024-08-20T21:39:13.6214332Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-08-20T21:39:13.8828259Z + echo '18.165.98.58 download.pytorch.org' 2024-08-20T21:39:13.8829085Z + sudo tee -a /etc/hosts 2024-08-20T21:39:13.9218993Z 18.165.98.58 download.pytorch.org 2024-08-20T21:39:13.9232885Z + cat /etc/hosts 2024-08-20T21:39:13.9240998Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-20T21:39:13.9254118Z ::1 localhost6 localhost6.localdomain6 2024-08-20T21:39:13.9254647Z 18.165.98.58 download.pytorch.org 2024-08-20T21:39:13.9417530Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-20T21:39:13.9418122Z with: 2024-08-20T21:39:13.9419001Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:13.9420049Z docker-build-dir: .ci/docker 2024-08-20T21:39:13.9420405Z working-directory: . 2024-08-20T21:39:13.9420844Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:13.9421362Z force-push: false 2024-08-20T21:39:13.9421638Z env: 2024-08-20T21:39:13.9421871Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:13.9422191Z ##[endgroup] 2024-08-20T21:39:13.9442014Z ##[group]Run set -ex 2024-08-20T21:39:13.9442343Z set -ex 2024-08-20T21:39:13.9442615Z  2024-08-20T21:39:13.9443140Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-20T21:39:13.9444131Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-20T21:39:13.9444919Z # job could then download the pre-built image as usual 2024-08-20T21:39:13.9445651Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-20T21:39:13.9446323Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:13.9446939Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:13.9447470Z  2024-08-20T21:39:13.9447956Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-20T21:39:13.9448570Z  exit 0 2024-08-20T21:39:13.9448832Z else 2024-08-20T21:39:13.9449168Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:13.9449594Z fi 2024-08-20T21:39:13.9449826Z  2024-08-20T21:39:13.9450269Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-20T21:39:13.9451122Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-20T21:39:13.9451965Z  # use it as it is, but first let's extract the tag 2024-08-20T21:39:13.9452661Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-20T21:39:13.9453503Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:13.9454185Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:13.9454713Z else 2024-08-20T21:39:13.9455126Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-20T21:39:13.9455777Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:13.9456656Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:13.9457406Z fi 2024-08-20T21:39:13.9465852Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:13.9466320Z env: 2024-08-20T21:39:13.9466572Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:13.9466904Z REPO_NAME: pytorch 2024-08-20T21:39:13.9467809Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:13.9469045Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T21:39:13.9469526Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:13.9470028Z ##[endgroup] 2024-08-20T21:39:13.9493989Z + [[ ! -d .ci/docker ]] 2024-08-20T21:39:13.9494442Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-20T21:39:13.9494844Z + echo skip=false 2024-08-20T21:39:13.9496555Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-20T21:39:13.9502166Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:13.9503214Z ++ awk -F '[:,]' '{print $2}' 2024-08-20T21:39:13.9608311Z + DOCKER_TAG=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:13.9609091Z + echo docker-tag=f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:13.9610477Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:13.9638159Z ##[group]Run set +e 2024-08-20T21:39:13.9638496Z set +e 2024-08-20T21:39:13.9638778Z set -x 2024-08-20T21:39:13.9639044Z  2024-08-20T21:39:13.9639283Z login() { 2024-08-20T21:39:13.9639932Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T21:39:13.9640652Z } 2024-08-20T21:39:13.9640891Z  2024-08-20T21:39:13.9641144Z retry () { 2024-08-20T21:39:13.9641521Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T21:39:13.9641937Z } 2024-08-20T21:39:13.9642185Z  2024-08-20T21:39:13.9642471Z retry login "${DOCKER_REGISTRY}" 2024-08-20T21:39:13.9642861Z  2024-08-20T21:39:13.9643313Z # Check if image already exists, if it does then skip building it 2024-08-20T21:39:13.9644009Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-20T21:39:13.9644483Z  exit 0 2024-08-20T21:39:13.9644759Z fi 2024-08-20T21:39:13.9645012Z  2024-08-20T21:39:13.9645480Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-20T21:39:13.9646299Z # be empty. The default action would be to continue rebuild the image 2024-08-20T21:39:13.9647027Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-20T21:39:13.9647685Z  # if we're on the base branch then use the parent commit 2024-08-20T21:39:13.9648252Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-20T21:39:13.9648660Z else 2024-08-20T21:39:13.9649103Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-20T21:39:13.9649763Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-20T21:39:13.9650382Z fi 2024-08-20T21:39:13.9650632Z  2024-08-20T21:39:13.9650909Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-20T21:39:13.9651395Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:13.9651940Z  2024-08-20T21:39:13.9652570Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-20T21:39:13.9653334Z  exit 0 2024-08-20T21:39:13.9653613Z fi 2024-08-20T21:39:13.9653850Z  2024-08-20T21:39:13.9654265Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-20T21:39:13.9655244Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-20T21:39:13.9656039Z  exit 1 2024-08-20T21:39:13.9656313Z fi 2024-08-20T21:39:13.9656566Z  2024-08-20T21:39:13.9657027Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-20T21:39:13.9658188Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-20T21:39:13.9659060Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-20T21:39:13.9660031Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-20T21:39:13.9661119Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-20T21:39:13.9661757Z fi 2024-08-20T21:39:13.9662013Z  2024-08-20T21:39:13.9662332Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-20T21:39:13.9668144Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:13.9668635Z env: 2024-08-20T21:39:13.9668878Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:13.9669220Z DOCKER_BUILD_DIR: .ci/docker 2024-08-20T21:39:13.9669669Z BASE_REVISION: 91f3d614142df02f619e44a68e3d9e0dfeba49ec 2024-08-20T21:39:13.9670743Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:13.9671791Z DOCKER_TAG: f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:13.9672384Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:13.9672886Z ##[endgroup] 2024-08-20T21:39:13.9695246Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:13.9695868Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:13.9697932Z + aws ecr get-login-password --region us-east-1 2024-08-20T21:39:13.9699065Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:14.4849708Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T21:39:14.4850520Z Configure a credential helper to remove this warning. See 2024-08-20T21:39:14.4851442Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T21:39:14.4852281Z 2024-08-20T21:39:14.4852431Z Login Succeeded 2024-08-20T21:39:14.4865267Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:14.6799544Z { 2024-08-20T21:39:14.6799998Z "schemaVersion": 2, 2024-08-20T21:39:14.6800819Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-20T21:39:14.6801784Z "config": { 2024-08-20T21:39:14.6802743Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-20T21:39:14.6803975Z "size": 43814, 2024-08-20T21:39:14.6805209Z "digest": "sha256:0d70ec8288d42bb7b6bd827783d06ac33bce3462f66d9a810a9a9ddd77c887e9" 2024-08-20T21:39:14.6806653Z }, 2024-08-20T21:39:14.6807180Z "layers": [ 2024-08-20T21:39:14.6807744Z { 2024-08-20T21:39:14.6808564Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6809870Z "size": 30440714, 2024-08-20T21:39:14.6810960Z "digest": "sha256:762bedf4b1b784c3de6c5022c5307d63123d3b7cdd59211317e37e9d477deaa0" 2024-08-20T21:39:14.6812480Z }, 2024-08-20T21:39:14.6812992Z { 2024-08-20T21:39:14.6814046Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6815329Z "size": 1823, 2024-08-20T21:39:14.6816537Z "digest": "sha256:8c0ee87aea3c3abdeb56755560432f327745225b47799d3832637903a5ec7dee" 2024-08-20T21:39:14.6817958Z }, 2024-08-20T21:39:14.6818482Z { 2024-08-20T21:39:14.6819434Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6820686Z "size": 313164825, 2024-08-20T21:39:14.6822008Z "digest": "sha256:c28adb4ed8869daa4ae6102cbbcf69b74f140920975e4751b0dfea9b1abfd15c" 2024-08-20T21:39:14.6823462Z }, 2024-08-20T21:39:14.6823948Z { 2024-08-20T21:39:14.6824901Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6826370Z "size": 804, 2024-08-20T21:39:14.6827575Z "digest": "sha256:531dcd0b8280a361aa8bbe329afe35405c64d13abc418090cce1c5a62537b888" 2024-08-20T21:39:14.6829062Z }, 2024-08-20T21:39:14.6829584Z { 2024-08-20T21:39:14.6830528Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6831803Z "size": 106, 2024-08-20T21:39:14.6832950Z "digest": "sha256:c81ff9d20d36fbd48a567918c08f020d23bd83515aec7d23fda14cd56786be81" 2024-08-20T21:39:14.6833996Z }, 2024-08-20T21:39:14.6834348Z { 2024-08-20T21:39:14.6834996Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6835801Z "size": 544, 2024-08-20T21:39:14.6836524Z "digest": "sha256:3a2d8ecdebc5053e5cb8589cd71f956e88cc3d16f21742993bfd69b85e6d0916" 2024-08-20T21:39:14.6837157Z }, 2024-08-20T21:39:14.6837363Z { 2024-08-20T21:39:14.6837781Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6838425Z "size": 1214, 2024-08-20T21:39:14.6839101Z "digest": "sha256:8538dcd89f0cac92de23cde0e22b0e7cfde631fbef3062f6387bbc78c4b8c051" 2024-08-20T21:39:14.6840115Z }, 2024-08-20T21:39:14.6840454Z { 2024-08-20T21:39:14.6841084Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6842008Z "size": 484, 2024-08-20T21:39:14.6843150Z "digest": "sha256:4a7b11d08282dad5413dab307cc0027d2c9da8629b5c69f9139ac2c5b32884b1" 2024-08-20T21:39:14.6844691Z + exit 0 2024-08-20T21:39:14.6845200Z }, 2024-08-20T21:39:14.6845706Z { 2024-08-20T21:39:14.6846665Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6847966Z "size": 110321271, 2024-08-20T21:39:14.6849254Z "digest": "sha256:ccda2116464a17da478e5a702df1aaff473b9569444e7f8a78cf1ef549212b72" 2024-08-20T21:39:14.6850712Z }, 2024-08-20T21:39:14.6851204Z { 2024-08-20T21:39:14.6852231Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6853502Z "size": 3840, 2024-08-20T21:39:14.6854732Z "digest": "sha256:78022b868ce3065df97adce1cca995409a90a7e0321a0c0a8919ede32d1d2a04" 2024-08-20T21:39:14.6856190Z }, 2024-08-20T21:39:14.6856696Z { 2024-08-20T21:39:14.6857626Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6858774Z "size": 1909, 2024-08-20T21:39:14.6859581Z "digest": "sha256:5f75f9c1c9a3e6c8b67494920cea0a5775c08b3cbd49bd01b09b02c94261db04" 2024-08-20T21:39:14.6860639Z }, 2024-08-20T21:39:14.6861033Z { 2024-08-20T21:39:14.6862001Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6863246Z "size": 701, 2024-08-20T21:39:14.6864476Z "digest": "sha256:aaf08982c0021a1af298f0fb571c20346c1e9df09c43e44acf0e1f3588dd1039" 2024-08-20T21:39:14.6865932Z }, 2024-08-20T21:39:14.6866416Z { 2024-08-20T21:39:14.6867380Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6868657Z "size": 2663383105, 2024-08-20T21:39:14.6869909Z "digest": "sha256:906cc3fc25c79e8d91b2ff6cd961c131dafa908907006971311498636c618f4e" 2024-08-20T21:39:14.6871610Z }, 2024-08-20T21:39:14.6872106Z { 2024-08-20T21:39:14.6873052Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6874327Z "size": 32, 2024-08-20T21:39:14.6875572Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:14.6877032Z }, 2024-08-20T21:39:14.6877504Z { 2024-08-20T21:39:14.6878453Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6879734Z "size": 379, 2024-08-20T21:39:14.6880933Z "digest": "sha256:88e873d7feffb728766b8b1afd9335fba2d10983015bf5c7235d16c6e0721d83" 2024-08-20T21:39:14.6882377Z }, 2024-08-20T21:39:14.6882898Z { 2024-08-20T21:39:14.6883831Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6885097Z "size": 222878, 2024-08-20T21:39:14.6886331Z "digest": "sha256:5a604d52e1bbf5414010aece200b94d9c29be535b0c445f1c780eff46c94e660" 2024-08-20T21:39:14.6887770Z }, 2024-08-20T21:39:14.6888281Z { 2024-08-20T21:39:14.6889232Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6890484Z "size": 231, 2024-08-20T21:39:14.6891736Z "digest": "sha256:63e67358db63853afb7d84d335e022e828783ba8615f70dfa3539906b82b13ea" 2024-08-20T21:39:14.6893165Z }, 2024-08-20T21:39:14.6893648Z { 2024-08-20T21:39:14.6894599Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6895881Z "size": 2913865, 2024-08-20T21:39:14.6897138Z "digest": "sha256:3a626210afeb83b0d9b1f0dadad249a35b778b7cc7e18f5949cc1a77148ebf95" 2024-08-20T21:39:14.6898602Z }, 2024-08-20T21:39:14.6899118Z { 2024-08-20T21:39:14.6900242Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6901519Z "size": 1805, 2024-08-20T21:39:14.6902483Z "digest": "sha256:a0eb49634d885aa915f63f923ebf39fd202300a3b24f0b07394d04dd57c3c683" 2024-08-20T21:39:14.6903445Z }, 2024-08-20T21:39:14.6903867Z { 2024-08-20T21:39:14.6904536Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6905418Z "size": 105, 2024-08-20T21:39:14.6906383Z "digest": "sha256:fc983e33a069ae2891a2161f9d24540f9e2728dddc9fd0e63c5b82454ab51f61" 2024-08-20T21:39:14.6907513Z }, 2024-08-20T21:39:14.6907771Z { 2024-08-20T21:39:14.6908444Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6909364Z "size": 564, 2024-08-20T21:39:14.6910216Z "digest": "sha256:d6ba8b647425b28f2cf40f0d3e2aaf52ca28231eb984ca3bc3fdb3267d74f8d6" 2024-08-20T21:39:14.6911294Z }, 2024-08-20T21:39:14.6911694Z { 2024-08-20T21:39:14.6912433Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6913364Z "size": 32, 2024-08-20T21:39:14.6914251Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:14.6915351Z }, 2024-08-20T21:39:14.6915736Z { 2024-08-20T21:39:14.6916458Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6917404Z "size": 104, 2024-08-20T21:39:14.6918208Z "digest": "sha256:800ce5e504c0c5ef446d189679b014e4d99547deec19ea367df57f38df73ea8d" 2024-08-20T21:39:14.6918819Z }, 2024-08-20T21:39:14.6919026Z { 2024-08-20T21:39:14.6919438Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6919977Z "size": 504, 2024-08-20T21:39:14.6920469Z "digest": "sha256:8335a3887870883e9ab33c78d7922178763a5738e0581ac7d3a9d1b730e9905e" 2024-08-20T21:39:14.6921053Z }, 2024-08-20T21:39:14.6921274Z { 2024-08-20T21:39:14.6921674Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6922221Z "size": 101109848, 2024-08-20T21:39:14.6922777Z "digest": "sha256:3a7c6ce8a2c5f019bdef0c2fef6f78b1a710c9e93726696167df535316c122ca" 2024-08-20T21:39:14.6923382Z }, 2024-08-20T21:39:14.6923600Z { 2024-08-20T21:39:14.6924116Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6924642Z "size": 109, 2024-08-20T21:39:14.6925149Z "digest": "sha256:a7f2a22883c001b9e0c7f51668300802f939700131ca215ebb600e3c76f3aa95" 2024-08-20T21:39:14.6925745Z }, 2024-08-20T21:39:14.6925953Z { 2024-08-20T21:39:14.6926365Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6926906Z "size": 489, 2024-08-20T21:39:14.6927408Z "digest": "sha256:43b5054ea31a137a93ab1677c0c030c426fd54e534255dc881c08d982a5cf360" 2024-08-20T21:39:14.6928011Z }, 2024-08-20T21:39:14.6928230Z { 2024-08-20T21:39:14.6928633Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6929174Z "size": 336, 2024-08-20T21:39:14.6929704Z "digest": "sha256:8da2931100c6700d3d495b362dc3c52df3b3ef7fa3e6d1145bb9083a2c96bb0a" 2024-08-20T21:39:14.6930302Z }, 2024-08-20T21:39:14.6930521Z { 2024-08-20T21:39:14.6930932Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6931465Z "size": 103, 2024-08-20T21:39:14.6932066Z "digest": "sha256:11a81df37a9c3cafd304fb86e5cbc18b2f1223705a3afdda291f6712e2b86905" 2024-08-20T21:39:14.6932688Z }, 2024-08-20T21:39:14.6932895Z { 2024-08-20T21:39:14.6933306Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6933849Z "size": 1473, 2024-08-20T21:39:14.6934373Z "digest": "sha256:076c3c501fa9f3809ba5a9aadfb65cd50da4f6015e3a7b71b60fc9381341ab39" 2024-08-20T21:39:14.6934992Z }, 2024-08-20T21:39:14.6935213Z { 2024-08-20T21:39:14.6935611Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6936152Z "size": 611525567, 2024-08-20T21:39:14.6936904Z "digest": "sha256:1c49c3d4f7fcd12cef9c59b7a881976957d419c18ca4c544e3088ae75ba4d1be" 2024-08-20T21:39:14.6937508Z }, 2024-08-20T21:39:14.6937730Z { 2024-08-20T21:39:14.6938141Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6938676Z "size": 163, 2024-08-20T21:39:14.6939207Z "digest": "sha256:4868a90a66f1b883ee705b02862ebfe0edbfc8f6eb4ffa849e5729c41a14cfeb" 2024-08-20T21:39:14.6939823Z }, 2024-08-20T21:39:14.6940031Z { 2024-08-20T21:39:14.6940441Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6940985Z "size": 2211, 2024-08-20T21:39:14.6941501Z "digest": "sha256:4e5011c5c63b068abb700845d59ed21ac55841263d1cd90cda63570f21e69818" 2024-08-20T21:39:14.6942257Z }, 2024-08-20T21:39:14.6942488Z { 2024-08-20T21:39:14.6942916Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6943473Z "size": 152, 2024-08-20T21:39:14.6944108Z "digest": "sha256:d4e444cefea48760ec5af0ccb68377eae0087817e8827ad51a2e11ee70d4ff5f" 2024-08-20T21:39:14.6944729Z }, 2024-08-20T21:39:14.6944937Z { 2024-08-20T21:39:14.6945352Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6945894Z "size": 646, 2024-08-20T21:39:14.6946411Z "digest": "sha256:81ac92e10710a795e9b9b5c1d704febcf6b405f86d96fb9f3b119c30f5d2c7f3" 2024-08-20T21:39:14.6947025Z }, 2024-08-20T21:39:14.6947244Z { 2024-08-20T21:39:14.6947644Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6948181Z "size": 32, 2024-08-20T21:39:14.6948711Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:14.6949312Z }, 2024-08-20T21:39:14.6949529Z { 2024-08-20T21:39:14.6949937Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6950468Z "size": 165, 2024-08-20T21:39:14.6950983Z "digest": "sha256:ca1818242f05ea8f9a9668e56046c7122f1ff0523e72365f3cb5f253d38667a3" 2024-08-20T21:39:14.6951584Z }, 2024-08-20T21:39:14.6951795Z { 2024-08-20T21:39:14.6952203Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6952746Z "size": 109, 2024-08-20T21:39:14.6953245Z "digest": "sha256:025b0cee3641783843deac4a1c228846cd3332053da3f8fb2d9124744d0825c2" 2024-08-20T21:39:14.6953929Z }, 2024-08-20T21:39:14.6954151Z { 2024-08-20T21:39:14.6954546Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6955089Z "size": 102, 2024-08-20T21:39:14.6955607Z "digest": "sha256:1f0b12a5c004b373780c76332f606861cec88202fb4220c4faf2d236aa63042a" 2024-08-20T21:39:14.6956196Z }, 2024-08-20T21:39:14.6956415Z { 2024-08-20T21:39:14.6956829Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6957355Z "size": 422, 2024-08-20T21:39:14.6958055Z "digest": "sha256:d63ee2a6050488c4ff7d9569705f3bd716e1e8f305825db9f7463924859b0d84" 2024-08-20T21:39:14.6958665Z }, 2024-08-20T21:39:14.6958872Z { 2024-08-20T21:39:14.6959291Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6959837Z "size": 32, 2024-08-20T21:39:14.6960353Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:14.6960971Z }, 2024-08-20T21:39:14.6961189Z { 2024-08-20T21:39:14.6961588Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6962128Z "size": 111, 2024-08-20T21:39:14.6962648Z "digest": "sha256:d040b74545d921b180e6cbfaafbd320965a8536b804b7941ab0347b9d7b834f5" 2024-08-20T21:39:14.6963253Z }, 2024-08-20T21:39:14.6963461Z { 2024-08-20T21:39:14.6963870Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6964394Z "size": 474, 2024-08-20T21:39:14.6964914Z "digest": "sha256:13bc165424849bb5c2cd7de3f72d1a4099505ff976f11a079ba62299f4761b09" 2024-08-20T21:39:14.6965520Z }, 2024-08-20T21:39:14.6965728Z { 2024-08-20T21:39:14.6966264Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6966808Z "size": 32, 2024-08-20T21:39:14.6967321Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:14.6967935Z }, 2024-08-20T21:39:14.6968158Z { 2024-08-20T21:39:14.6968556Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6969099Z "size": 112, 2024-08-20T21:39:14.6969658Z "digest": "sha256:dfb6cbc954b5bde01bb88cb42dabfa484b63eefdae9ed7c0ddf7f56ef77c66b9" 2024-08-20T21:39:14.6970303Z }, 2024-08-20T21:39:14.6970510Z { 2024-08-20T21:39:14.6970923Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6971463Z "size": 566, 2024-08-20T21:39:14.6972037Z "digest": "sha256:5b1463b1c261bc0d99a5d8dd48cc7452fcadeece1d43010599a0fb2f8dfee9b7" 2024-08-20T21:39:14.6972656Z }, 2024-08-20T21:39:14.6972875Z { 2024-08-20T21:39:14.6973276Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6973826Z "size": 43163684, 2024-08-20T21:39:14.6974354Z "digest": "sha256:172d385449d5e4a862d9b54229c96916775d9ea683b716de04d592529bba93c5" 2024-08-20T21:39:14.6974934Z }, 2024-08-20T21:39:14.6975153Z { 2024-08-20T21:39:14.6975570Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6976096Z "size": 106, 2024-08-20T21:39:14.6976615Z "digest": "sha256:cc93353090ff2554f6d9e804b424b5f18d4d2b65c4d1d56e39fb2394e8f2ce32" 2024-08-20T21:39:14.6977216Z }, 2024-08-20T21:39:14.6977423Z { 2024-08-20T21:39:14.6977833Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6978373Z "size": 295, 2024-08-20T21:39:14.6978884Z "digest": "sha256:cf364f5f62440ba1d90661380e6f98200d23dfc3e70bab35428f5c7afbd9b912" 2024-08-20T21:39:14.6979489Z }, 2024-08-20T21:39:14.6979707Z { 2024-08-20T21:39:14.6980107Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6980681Z "size": 32, 2024-08-20T21:39:14.6981213Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:14.6981815Z }, 2024-08-20T21:39:14.6982037Z { 2024-08-20T21:39:14.6982448Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6983061Z "size": 106, 2024-08-20T21:39:14.6983600Z "digest": "sha256:8d5cbba0c8c6214d0ffb769778159ff9383c2e69b65f5f765ae11ae53875a387" 2024-08-20T21:39:14.6984217Z }, 2024-08-20T21:39:14.6984424Z { 2024-08-20T21:39:14.6984836Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6985377Z "size": 425, 2024-08-20T21:39:14.6985892Z "digest": "sha256:2e373a30f750fb90cbb430e883cce1aa23d1292246f2ea10ede17832c26f6d53" 2024-08-20T21:39:14.6986507Z }, 2024-08-20T21:39:14.6986728Z { 2024-08-20T21:39:14.6987126Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6987670Z "size": 19308720, 2024-08-20T21:39:14.6988213Z "digest": "sha256:529c734f1880e18d646a45fc8dfcc22f05e608acfe618798e202c6d96de66085" 2024-08-20T21:39:14.6988803Z }, 2024-08-20T21:39:14.6989021Z { 2024-08-20T21:39:14.6989438Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6989970Z "size": 108, 2024-08-20T21:39:14.6990489Z "digest": "sha256:9c1479550b9d4b66feec866b30965a3908c809f58cba06f362598253ba819c50" 2024-08-20T21:39:14.6991086Z }, 2024-08-20T21:39:14.6991293Z { 2024-08-20T21:39:14.6991705Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6992247Z "size": 438, 2024-08-20T21:39:14.6992743Z "digest": "sha256:292d6ba03740d88893589c73591368207eba9e59f8244794ffd9cfebca7a92b1" 2024-08-20T21:39:14.6993338Z }, 2024-08-20T21:39:14.6993560Z { 2024-08-20T21:39:14.6993957Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6994493Z "size": 701, 2024-08-20T21:39:14.6995079Z "digest": "sha256:aaf08982c0021a1af298f0fb571c20346c1e9df09c43e44acf0e1f3588dd1039" 2024-08-20T21:39:14.6995681Z }, 2024-08-20T21:39:14.6995901Z { 2024-08-20T21:39:14.6996317Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6996845Z "size": 141, 2024-08-20T21:39:14.6997368Z "digest": "sha256:81b70407002c1d04c80e5d35c577fa9acfb07012a8e71912df7c33aa99562ce8" 2024-08-20T21:39:14.6997968Z }, 2024-08-20T21:39:14.6998176Z { 2024-08-20T21:39:14.6998587Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.6999126Z "size": 135, 2024-08-20T21:39:14.6999645Z "digest": "sha256:1fc092fa7a5da990144d1006de1a9f4d219f9f2b85aec30d8cf82f60db1509aa" 2024-08-20T21:39:14.7000261Z }, 2024-08-20T21:39:14.7000479Z { 2024-08-20T21:39:14.7000880Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7001418Z "size": 32, 2024-08-20T21:39:14.7001949Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:14.7002555Z }, 2024-08-20T21:39:14.7002773Z { 2024-08-20T21:39:14.7003187Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7003716Z "size": 194, 2024-08-20T21:39:14.7004239Z "digest": "sha256:95d6f4de1332b0dc8f020f1b15a84ad4b8d61d6666a876b0b040534e88a0edcc" 2024-08-20T21:39:14.7004851Z }, 2024-08-20T21:39:14.7005057Z { 2024-08-20T21:39:14.7005466Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7006009Z "size": 1379, 2024-08-20T21:39:14.7006513Z "digest": "sha256:84b6af65e56859603f392b9cd3ffa2295934861b02eb0adc10001b6947ad4c13" 2024-08-20T21:39:14.7007113Z }, 2024-08-20T21:39:14.7007335Z { 2024-08-20T21:39:14.7007731Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7008271Z "size": 701, 2024-08-20T21:39:14.7008793Z "digest": "sha256:aaf08982c0021a1af298f0fb571c20346c1e9df09c43e44acf0e1f3588dd1039" 2024-08-20T21:39:14.7009392Z }, 2024-08-20T21:39:14.7009617Z { 2024-08-20T21:39:14.7010029Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7010558Z "size": 139, 2024-08-20T21:39:14.7011087Z "digest": "sha256:f89bdeb677945f4ff3d44f53f98f7d97d06a6e68f80ad569c95ba7a6225e2f38" 2024-08-20T21:39:14.7011827Z }, 2024-08-20T21:39:14.7012039Z { 2024-08-20T21:39:14.7012452Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7013001Z "size": 3735818322, 2024-08-20T21:39:14.7013528Z "digest": "sha256:350a7e72a699716ec997ab67d62a66f8cb1b173ce9383e5f543455cdf36777da" 2024-08-20T21:39:14.7014132Z }, 2024-08-20T21:39:14.7014352Z { 2024-08-20T21:39:14.7014750Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7015299Z "size": 156, 2024-08-20T21:39:14.7015823Z "digest": "sha256:b4e22c165860fcb12b0b96f9ffbd4f42d6928f44909a6ff246af514622ab3006" 2024-08-20T21:39:14.7016423Z }, 2024-08-20T21:39:14.7016642Z { 2024-08-20T21:39:14.7017055Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7017583Z "size": 849, 2024-08-20T21:39:14.7018115Z "digest": "sha256:b7a95f68be138948dfef44cb0d4131b38e0f46ef6248bfb83a0d9eedd4b99db7" 2024-08-20T21:39:14.7018736Z }, 2024-08-20T21:39:14.7018943Z { 2024-08-20T21:39:14.7019356Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7019899Z "size": 701, 2024-08-20T21:39:14.7020410Z "digest": "sha256:aaf08982c0021a1af298f0fb571c20346c1e9df09c43e44acf0e1f3588dd1039" 2024-08-20T21:39:14.7021020Z }, 2024-08-20T21:39:14.7021240Z { 2024-08-20T21:39:14.7021640Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7022178Z "size": 139, 2024-08-20T21:39:14.7022716Z "digest": "sha256:b7ae7e6f572a74ed87cfc6763f7db7e8ab70b00cc37e04f151b4e81b9ae00f18" 2024-08-20T21:39:14.7023324Z }, 2024-08-20T21:39:14.7023542Z { 2024-08-20T21:39:14.7024022Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7024554Z "size": 32, 2024-08-20T21:39:14.7025083Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:14.7025700Z }, 2024-08-20T21:39:14.7025911Z { 2024-08-20T21:39:14.7026327Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7026873Z "size": 159, 2024-08-20T21:39:14.7027402Z "digest": "sha256:b83dd8df6637d14b28f8fd5ecda7ccfad76d4c1a8c1d871c843ce9328193ac6c" 2024-08-20T21:39:14.7028031Z }, 2024-08-20T21:39:14.7028252Z { 2024-08-20T21:39:14.7028653Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7029197Z "size": 907, 2024-08-20T21:39:14.7029725Z "digest": "sha256:bb36ec6ca619d54831d85fe0039ab5fb58d803893da4f2d98768262e15e55984" 2024-08-20T21:39:14.7030326Z }, 2024-08-20T21:39:14.7030546Z { 2024-08-20T21:39:14.7030956Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7031485Z "size": 701, 2024-08-20T21:39:14.7032012Z "digest": "sha256:aaf08982c0021a1af298f0fb571c20346c1e9df09c43e44acf0e1f3588dd1039" 2024-08-20T21:39:14.7032622Z }, 2024-08-20T21:39:14.7032828Z { 2024-08-20T21:39:14.7033240Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7033784Z "size": 135, 2024-08-20T21:39:14.7034287Z "digest": "sha256:c52074bda1207120b6407464ecc855fe7f04e36a52d8b60efbb424a794a32beb" 2024-08-20T21:39:14.7034888Z }, 2024-08-20T21:39:14.7035107Z { 2024-08-20T21:39:14.7035506Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7036043Z "size": 32, 2024-08-20T21:39:14.7036570Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:14.7037172Z }, 2024-08-20T21:39:14.7037396Z { 2024-08-20T21:39:14.7037806Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7038331Z "size": 158, 2024-08-20T21:39:14.7038864Z "digest": "sha256:8f914a53bad07f5fd9d29cbb462aa752d67806de4b91e97ad4b5441ed805cf14" 2024-08-20T21:39:14.7039480Z }, 2024-08-20T21:39:14.7039688Z { 2024-08-20T21:39:14.7040098Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7040699Z "size": 1578, 2024-08-20T21:39:14.7041205Z "digest": "sha256:3d207949ee5a24d15c09664559300ce68d99f4e09d3d7abf927fb1e918f6c7ab" 2024-08-20T21:39:14.7041807Z }, 2024-08-20T21:39:14.7042028Z { 2024-08-20T21:39:14.7042428Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7042967Z "size": 32, 2024-08-20T21:39:14.7043493Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:14.7044092Z }, 2024-08-20T21:39:14.7044311Z { 2024-08-20T21:39:14.7044720Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7045242Z "size": 137, 2024-08-20T21:39:14.7045769Z "digest": "sha256:638224ef337df8b465596c5afc104d8c3139dceb0eb7048e29dc1cc6ae497573" 2024-08-20T21:39:14.7046375Z }, 2024-08-20T21:39:14.7046585Z { 2024-08-20T21:39:14.7046998Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7047539Z "size": 378, 2024-08-20T21:39:14.7048058Z "digest": "sha256:cf9d145104f43db76219176cf41bc07e1ca19c6bfafd89ba819a7ca1e76d1a6b" 2024-08-20T21:39:14.7048671Z }, 2024-08-20T21:39:14.7048890Z { 2024-08-20T21:39:14.7049290Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7049830Z "size": 32, 2024-08-20T21:39:14.7050355Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:14.7050957Z }, 2024-08-20T21:39:14.7051175Z { 2024-08-20T21:39:14.7051645Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7052176Z "size": 104, 2024-08-20T21:39:14.7052722Z "digest": "sha256:c4d2ce3c205c90ba7fdae94e14ecafdb42ab30440efacb631f29541e95859ca0" 2024-08-20T21:39:14.7053423Z }, 2024-08-20T21:39:14.7053631Z { 2024-08-20T21:39:14.7054044Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7054590Z "size": 1843, 2024-08-20T21:39:14.7055085Z "digest": "sha256:d417897013fc53f6de8d7b0b4214713595073871542e20c42071ba22d158052a" 2024-08-20T21:39:14.7055679Z }, 2024-08-20T21:39:14.7055902Z { 2024-08-20T21:39:14.7056300Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7056847Z "size": 7529593, 2024-08-20T21:39:14.7057372Z "digest": "sha256:1ba13f47f442f3a411f6ec25c6412d51709261fd97e71049b92964f50cc9be8f" 2024-08-20T21:39:14.7058129Z }, 2024-08-20T21:39:14.7058352Z { 2024-08-20T21:39:14.7058764Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7059295Z "size": 106, 2024-08-20T21:39:14.7059817Z "digest": "sha256:e6f28cfa301012334e1e91c6ad2d133801d9d50284153ebd9da6c9900aed1f7b" 2024-08-20T21:39:14.7060422Z }, 2024-08-20T21:39:14.7060634Z { 2024-08-20T21:39:14.7061046Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7061584Z "size": 163, 2024-08-20T21:39:14.7062099Z "digest": "sha256:305e45e19ff762cb7ee4c719fade35f7de552d2642f3116a9249586bbdad3043" 2024-08-20T21:39:14.7062720Z }, 2024-08-20T21:39:14.7062940Z { 2024-08-20T21:39:14.7063338Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7063873Z "size": 7943, 2024-08-20T21:39:14.7064396Z "digest": "sha256:78839cbb08a623ce9a097776428b4fd20a31a5726da6fedf8682f67f4d5ecb3c" 2024-08-20T21:39:14.7064986Z }, 2024-08-20T21:39:14.7065206Z { 2024-08-20T21:39:14.7065618Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7066148Z "size": 8067, 2024-08-20T21:39:14.7066667Z "digest": "sha256:97e6589defb285605e5c361be7f0ca344b9b3853b464100f6c6d9471f7984635" 2024-08-20T21:39:14.7067270Z }, 2024-08-20T21:39:14.7067478Z { 2024-08-20T21:39:14.7067893Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7068434Z "size": 303, 2024-08-20T21:39:14.7068953Z "digest": "sha256:00151a7db07be68fce69e7041b8ece2b33570eaa8ed7df74a309be4627d919cd" 2024-08-20T21:39:14.7069687Z }, 2024-08-20T21:39:14.7069909Z { 2024-08-20T21:39:14.7070306Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7070849Z "size": 32, 2024-08-20T21:39:14.7071378Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:14.7071989Z }, 2024-08-20T21:39:14.7072197Z { 2024-08-20T21:39:14.7072606Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7073145Z "size": 108, 2024-08-20T21:39:14.7073643Z "digest": "sha256:2853b72b213b163a214d97724c39b2824ba456e42b3e100cf849ebd5e8086bb0" 2024-08-20T21:39:14.7074241Z }, 2024-08-20T21:39:14.7074457Z { 2024-08-20T21:39:14.7074860Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7075399Z "size": 54145664, 2024-08-20T21:39:14.7075948Z "digest": "sha256:84ce0a95a6cef3dcd43f4e81ca3d17d58b3cc673471804a8909f1d2cc87d0c6d" 2024-08-20T21:39:14.7076555Z }, 2024-08-20T21:39:14.7076776Z { 2024-08-20T21:39:14.7077186Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-20T21:39:14.7077717Z "size": 32, 2024-08-20T21:39:14.7078245Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-20T21:39:14.7078860Z } 2024-08-20T21:39:14.7079065Z ] 2024-08-20T21:39:14.7079281Z } 2024-08-20T21:39:14.7190839Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T21:39:14.7191306Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-20T21:39:14.7191836Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-20T21:39:14.7198290Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:14.7198765Z env: 2024-08-20T21:39:14.7199025Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:14.7199978Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:14.7200972Z ##[endgroup] 2024-08-20T21:39:14.7227606Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-jammy-py3.8-gcc11-f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:14.7287780Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-20T21:39:14.7303920Z with: 2024-08-20T21:39:14.7304824Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:14.7305956Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:14.7306449Z env: 2024-08-20T21:39:14.7306700Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:14.7307024Z ##[endgroup] 2024-08-20T21:39:14.7329527Z ##[group]Run set -x 2024-08-20T21:39:14.7329841Z set -x 2024-08-20T21:39:14.7330108Z set +e 2024-08-20T21:39:14.7330372Z  2024-08-20T21:39:14.7330609Z login() { 2024-08-20T21:39:14.7331249Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-20T21:39:14.7332067Z } 2024-08-20T21:39:14.7332316Z  2024-08-20T21:39:14.7332598Z retry () { 2024-08-20T21:39:14.7332949Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-20T21:39:14.7333379Z } 2024-08-20T21:39:14.7333628Z  2024-08-20T21:39:14.7333901Z retry login "${DOCKER_REGISTRY}" 2024-08-20T21:39:14.7334296Z  2024-08-20T21:39:14.7334539Z set -e 2024-08-20T21:39:14.7334977Z # ignore output since only exit code is used for conditional 2024-08-20T21:39:14.7335662Z # only pull docker image if it's not available locally 2024-08-20T21:39:14.7336417Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-20T21:39:14.7337099Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-20T21:39:14.7337504Z fi 2024-08-20T21:39:14.7342681Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:39:14.7343159Z env: 2024-08-20T21:39:14.7343505Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:39:14.7344443Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:14.7345546Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:14.7346040Z ##[endgroup] 2024-08-20T21:39:14.7368309Z + set +e 2024-08-20T21:39:14.7369018Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:14.7369637Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:14.7372008Z + aws ecr get-login-password --region us-east-1 2024-08-20T21:39:14.7373538Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-20T21:39:15.2661792Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-20T21:39:15.2662770Z Configure a credential helper to remove this warning. See 2024-08-20T21:39:15.2663697Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-20T21:39:15.2664265Z 2024-08-20T21:39:15.2664370Z Login Succeeded 2024-08-20T21:39:15.2676482Z + set -e 2024-08-20T21:39:15.2677613Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:15.2809656Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:15.2811716Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:39:15.5200965Z f6d216893d65c7b8ae43df4daaf247db808378e9: Pulling from pytorch/pytorch-linux-jammy-py3.8-gcc11 2024-08-20T21:39:15.5218711Z 762bedf4b1b7: Pulling fs layer 2024-08-20T21:39:15.5219362Z 8c0ee87aea3c: Pulling fs layer 2024-08-20T21:39:15.5219973Z c28adb4ed886: Pulling fs layer 2024-08-20T21:39:15.5220614Z 531dcd0b8280: Pulling fs layer 2024-08-20T21:39:15.5221226Z c81ff9d20d36: Pulling fs layer 2024-08-20T21:39:15.5222096Z 3a2d8ecdebc5: Pulling fs layer 2024-08-20T21:39:15.5222727Z 8538dcd89f0c: Pulling fs layer 2024-08-20T21:39:15.5223357Z 4a7b11d08282: Pulling fs layer 2024-08-20T21:39:15.5223958Z ccda2116464a: Pulling fs layer 2024-08-20T21:39:15.5224591Z 78022b868ce3: Pulling fs layer 2024-08-20T21:39:15.5225186Z 5f75f9c1c9a3: Pulling fs layer 2024-08-20T21:39:15.5225803Z aaf08982c002: Pulling fs layer 2024-08-20T21:39:15.5226327Z 906cc3fc25c7: Pulling fs layer 2024-08-20T21:39:15.5226764Z c81ff9d20d36: Waiting 2024-08-20T21:39:15.5227379Z 4f4fb700ef54: Pulling fs layer 2024-08-20T21:39:15.5228004Z 88e873d7feff: Pulling fs layer 2024-08-20T21:39:15.5228333Z 3a2d8ecdebc5: Waiting 2024-08-20T21:39:15.5228808Z 5a604d52e1bb: Pulling fs layer 2024-08-20T21:39:15.5229257Z 8538dcd89f0c: Waiting 2024-08-20T21:39:15.5229555Z 63e67358db63: Pulling fs layer 2024-08-20T21:39:15.5229904Z 3a626210afeb: Pulling fs layer 2024-08-20T21:39:15.5230256Z 531dcd0b8280: Waiting 2024-08-20T21:39:15.5230742Z a0eb49634d88: Pulling fs layer 2024-08-20T21:39:15.5231327Z 906cc3fc25c7: Waiting 2024-08-20T21:39:15.5231820Z fc983e33a069: Pulling fs layer 2024-08-20T21:39:15.5232300Z 4f4fb700ef54: Waiting 2024-08-20T21:39:15.5232595Z 4a7b11d08282: Waiting 2024-08-20T21:39:15.5232872Z 5a604d52e1bb: Waiting 2024-08-20T21:39:15.5233179Z d6ba8b647425: Pulling fs layer 2024-08-20T21:39:15.5233530Z ccda2116464a: Waiting 2024-08-20T21:39:15.5233884Z 78022b868ce3: Waiting 2024-08-20T21:39:15.5234185Z 800ce5e504c0: Pulling fs layer 2024-08-20T21:39:15.5234522Z 5f75f9c1c9a3: Waiting 2024-08-20T21:39:15.5234812Z 8335a3887870: Pulling fs layer 2024-08-20T21:39:15.5235141Z aaf08982c002: Waiting 2024-08-20T21:39:15.5235428Z 88e873d7feff: Waiting 2024-08-20T21:39:15.5235720Z 3a7c6ce8a2c5: Pulling fs layer 2024-08-20T21:39:15.5236193Z 63e67358db63: Waiting 2024-08-20T21:39:15.5236686Z a0eb49634d88: Waiting 2024-08-20T21:39:15.5237169Z d6ba8b647425: Waiting 2024-08-20T21:39:15.5237704Z a7f2a22883c0: Pulling fs layer 2024-08-20T21:39:15.5238042Z 3a626210afeb: Waiting 2024-08-20T21:39:15.5238326Z fc983e33a069: Waiting 2024-08-20T21:39:15.5238626Z 43b5054ea31a: Pulling fs layer 2024-08-20T21:39:15.5238979Z 8da2931100c6: Pulling fs layer 2024-08-20T21:39:15.5239303Z 800ce5e504c0: Waiting 2024-08-20T21:39:15.5239590Z a7f2a22883c0: Waiting 2024-08-20T21:39:15.5239905Z 11a81df37a9c: Pulling fs layer 2024-08-20T21:39:15.5240457Z 3a7c6ce8a2c5: Waiting 2024-08-20T21:39:15.5240766Z 076c3c501fa9: Pulling fs layer 2024-08-20T21:39:15.5241156Z 8335a3887870: Waiting 2024-08-20T21:39:15.5241482Z 1c49c3d4f7fc: Pulling fs layer 2024-08-20T21:39:15.5241840Z 11a81df37a9c: Waiting 2024-08-20T21:39:15.5242371Z 4868a90a66f1: Pulling fs layer 2024-08-20T21:39:15.5242815Z 076c3c501fa9: Waiting 2024-08-20T21:39:15.5243103Z 8da2931100c6: Waiting 2024-08-20T21:39:15.5243407Z 4e5011c5c63b: Pulling fs layer 2024-08-20T21:39:15.5243763Z d4e444cefea4: Pulling fs layer 2024-08-20T21:39:15.5244319Z 81ac92e10710: Pulling fs layer 2024-08-20T21:39:15.5244948Z 43b5054ea31a: Waiting 2024-08-20T21:39:15.5245528Z ca1818242f05: Pulling fs layer 2024-08-20T21:39:15.5245895Z d4e444cefea4: Waiting 2024-08-20T21:39:15.5246204Z 025b0cee3641: Pulling fs layer 2024-08-20T21:39:15.5246546Z 1f0b12a5c004: Pulling fs layer 2024-08-20T21:39:15.5246909Z d63ee2a60504: Pulling fs layer 2024-08-20T21:39:15.5247482Z d040b74545d9: Pulling fs layer 2024-08-20T21:39:15.5248054Z 025b0cee3641: Waiting 2024-08-20T21:39:15.5248433Z 13bc16542484: Pulling fs layer 2024-08-20T21:39:15.5248766Z 81ac92e10710: Waiting 2024-08-20T21:39:15.5249064Z 1f0b12a5c004: Waiting 2024-08-20T21:39:15.5249600Z dfb6cbc954b5: Pulling fs layer 2024-08-20T21:39:15.5250176Z d63ee2a60504: Waiting 2024-08-20T21:39:15.5250840Z 5b1463b1c261: Pulling fs layer 2024-08-20T21:39:15.5251264Z ca1818242f05: Waiting 2024-08-20T21:39:15.5251633Z 4868a90a66f1: Waiting 2024-08-20T21:39:15.5252118Z 13bc16542484: Waiting 2024-08-20T21:39:15.5252756Z 172d385449d5: Pulling fs layer 2024-08-20T21:39:15.5253266Z cc93353090ff: Pulling fs layer 2024-08-20T21:39:15.5253595Z dfb6cbc954b5: Waiting 2024-08-20T21:39:15.5254055Z cf364f5f6244: Pulling fs layer 2024-08-20T21:39:15.5254397Z 5b1463b1c261: Waiting 2024-08-20T21:39:15.5254673Z 172d385449d5: Waiting 2024-08-20T21:39:15.5254980Z 8d5cbba0c8c6: Pulling fs layer 2024-08-20T21:39:15.5255319Z d040b74545d9: Waiting 2024-08-20T21:39:15.5255665Z cf364f5f6244: Waiting 2024-08-20T21:39:15.5255970Z 2e373a30f750: Pulling fs layer 2024-08-20T21:39:15.5256307Z 529c734f1880: Pulling fs layer 2024-08-20T21:39:15.5256655Z 9c1479550b9d: Pulling fs layer 2024-08-20T21:39:15.5256995Z 8d5cbba0c8c6: Waiting 2024-08-20T21:39:15.5257289Z 292d6ba03740: Pulling fs layer 2024-08-20T21:39:15.5257627Z cc93353090ff: Waiting 2024-08-20T21:39:15.5258254Z 81b70407002c: Pulling fs layer 2024-08-20T21:39:15.5258581Z 529c734f1880: Waiting 2024-08-20T21:39:15.5258870Z 9c1479550b9d: Waiting 2024-08-20T21:39:15.5259176Z 1fc092fa7a5d: Pulling fs layer 2024-08-20T21:39:15.5259499Z 292d6ba03740: Waiting 2024-08-20T21:39:15.5259811Z 95d6f4de1332: Pulling fs layer 2024-08-20T21:39:15.5260171Z 84b6af65e568: Pulling fs layer 2024-08-20T21:39:15.5260496Z 1fc092fa7a5d: Waiting 2024-08-20T21:39:15.5260804Z f89bdeb67794: Pulling fs layer 2024-08-20T21:39:15.5261140Z 84b6af65e568: Waiting 2024-08-20T21:39:15.5261416Z 81b70407002c: Waiting 2024-08-20T21:39:15.5261700Z 95d6f4de1332: Waiting 2024-08-20T21:39:15.5262003Z 350a7e72a699: Pulling fs layer 2024-08-20T21:39:15.5262325Z 2e373a30f750: Waiting 2024-08-20T21:39:15.5262628Z b4e22c165860: Pulling fs layer 2024-08-20T21:39:15.5262964Z f89bdeb67794: Waiting 2024-08-20T21:39:15.5263259Z b7a95f68be13: Pulling fs layer 2024-08-20T21:39:15.5263617Z b7ae7e6f572a: Pulling fs layer 2024-08-20T21:39:15.5263972Z b83dd8df6637: Pulling fs layer 2024-08-20T21:39:15.5264312Z bb36ec6ca619: Pulling fs layer 2024-08-20T21:39:15.5264653Z b7a95f68be13: Waiting 2024-08-20T21:39:15.5264946Z b7ae7e6f572a: Waiting 2024-08-20T21:39:15.5265242Z c52074bda120: Pulling fs layer 2024-08-20T21:39:15.5265712Z 350a7e72a699: Waiting 2024-08-20T21:39:15.5266000Z b4e22c165860: Waiting 2024-08-20T21:39:15.5266298Z 8f914a53bad0: Pulling fs layer 2024-08-20T21:39:15.5266637Z c52074bda120: Waiting 2024-08-20T21:39:15.5266940Z 3d207949ee5a: Pulling fs layer 2024-08-20T21:39:15.5267266Z bb36ec6ca619: Waiting 2024-08-20T21:39:15.5267559Z 8f914a53bad0: Waiting 2024-08-20T21:39:15.5267860Z 638224ef337d: Pulling fs layer 2024-08-20T21:39:15.5268199Z cf9d145104f4: Pulling fs layer 2024-08-20T21:39:15.5268653Z c4d2ce3c205c: Pulling fs layer 2024-08-20T21:39:15.5269005Z d417897013fc: Pulling fs layer 2024-08-20T21:39:15.5269341Z 1ba13f47f442: Pulling fs layer 2024-08-20T21:39:15.5269679Z cf9d145104f4: Waiting 2024-08-20T21:39:15.5269969Z 3d207949ee5a: Waiting 2024-08-20T21:39:15.5270258Z e6f28cfa3010: Pulling fs layer 2024-08-20T21:39:15.5270605Z 305e45e19ff7: Pulling fs layer 2024-08-20T21:39:15.5270984Z 78839cbb08a6: Pulling fs layer 2024-08-20T21:39:15.5271320Z 97e6589defb2: Pulling fs layer 2024-08-20T21:39:15.5271662Z c4d2ce3c205c: Waiting 2024-08-20T21:39:15.5271965Z 00151a7db07b: Pulling fs layer 2024-08-20T21:39:15.5272290Z e6f28cfa3010: Waiting 2024-08-20T21:39:15.5272591Z 2853b72b213b: Pulling fs layer 2024-08-20T21:39:15.5272919Z 97e6589defb2: Waiting 2024-08-20T21:39:15.5273211Z 84ce0a95a6ce: Pulling fs layer 2024-08-20T21:39:15.5273543Z 78839cbb08a6: Waiting 2024-08-20T21:39:15.5273829Z d417897013fc: Waiting 2024-08-20T21:39:15.5274102Z 00151a7db07b: Waiting 2024-08-20T21:39:15.5274386Z 2853b72b213b: Waiting 2024-08-20T21:39:15.5274673Z 305e45e19ff7: Waiting 2024-08-20T21:39:15.5274944Z 1ba13f47f442: Waiting 2024-08-20T21:39:15.6013621Z 8c0ee87aea3c: Download complete 2024-08-20T21:39:15.6847764Z 531dcd0b8280: Verifying Checksum 2024-08-20T21:39:15.6848353Z 531dcd0b8280: Download complete 2024-08-20T21:39:15.7566783Z c81ff9d20d36: Download complete 2024-08-20T21:39:15.8238829Z 3a2d8ecdebc5: Verifying Checksum 2024-08-20T21:39:15.8764795Z 3a2d8ecdebc5: Download complete 2024-08-20T21:39:15.8765416Z 762bedf4b1b7: Verifying Checksum 2024-08-20T21:39:15.8765859Z 762bedf4b1b7: Download complete 2024-08-20T21:39:15.8982382Z 8538dcd89f0c: Download complete 2024-08-20T21:39:15.9525750Z 4a7b11d08282: Verifying Checksum 2024-08-20T21:39:15.9526998Z 4a7b11d08282: Download complete 2024-08-20T21:39:16.0312574Z 78022b868ce3: Download complete 2024-08-20T21:39:16.1111056Z 5f75f9c1c9a3: Download complete 2024-08-20T21:39:16.1729217Z aaf08982c002: Verifying Checksum 2024-08-20T21:39:16.1729888Z aaf08982c002: Download complete 2024-08-20T21:39:16.8246538Z 762bedf4b1b7: Pull complete 2024-08-20T21:39:16.8393426Z 8c0ee87aea3c: Pull complete 2024-08-20T21:39:17.0490351Z ccda2116464a: Verifying Checksum 2024-08-20T21:39:17.0490922Z ccda2116464a: Download complete 2024-08-20T21:39:17.0576935Z 4f4fb700ef54: Verifying Checksum 2024-08-20T21:39:17.0577687Z 4f4fb700ef54: Download complete 2024-08-20T21:39:17.1272997Z 88e873d7feff: Verifying Checksum 2024-08-20T21:39:17.1273518Z 88e873d7feff: Download complete 2024-08-20T21:39:17.2060306Z 5a604d52e1bb: Download complete 2024-08-20T21:39:17.2866376Z 63e67358db63: Verifying Checksum 2024-08-20T21:39:17.2866911Z 63e67358db63: Download complete 2024-08-20T21:39:17.3779428Z 3a626210afeb: Verifying Checksum 2024-08-20T21:39:17.3779932Z 3a626210afeb: Download complete 2024-08-20T21:39:17.5325017Z fc983e33a069: Verifying Checksum 2024-08-20T21:39:17.5325506Z fc983e33a069: Download complete 2024-08-20T21:39:17.6091768Z d6ba8b647425: Verifying Checksum 2024-08-20T21:39:17.6092203Z d6ba8b647425: Download complete 2024-08-20T21:39:17.6774029Z 800ce5e504c0: Verifying Checksum 2024-08-20T21:39:17.6774668Z 800ce5e504c0: Download complete 2024-08-20T21:39:17.7410841Z 8335a3887870: Verifying Checksum 2024-08-20T21:39:17.7411509Z 8335a3887870: Download complete 2024-08-20T21:39:18.7124312Z c28adb4ed886: Verifying Checksum 2024-08-20T21:39:18.7125075Z c28adb4ed886: Download complete 2024-08-20T21:39:18.7757020Z a7f2a22883c0: Download complete 2024-08-20T21:39:18.8026133Z 3a7c6ce8a2c5: Verifying Checksum 2024-08-20T21:39:18.8026939Z 3a7c6ce8a2c5: Download complete 2024-08-20T21:39:18.8758437Z 8da2931100c6: Verifying Checksum 2024-08-20T21:39:18.8759140Z 8da2931100c6: Download complete 2024-08-20T21:39:18.8832415Z 43b5054ea31a: Verifying Checksum 2024-08-20T21:39:18.8833434Z 43b5054ea31a: Download complete 2024-08-20T21:39:18.9444170Z 11a81df37a9c: Download complete 2024-08-20T21:39:18.9496864Z 076c3c501fa9: Verifying Checksum 2024-08-20T21:39:18.9497495Z 076c3c501fa9: Download complete 2024-08-20T21:39:19.0228734Z 4868a90a66f1: Verifying Checksum 2024-08-20T21:39:19.0229280Z 4868a90a66f1: Download complete 2024-08-20T21:39:19.0966602Z 4e5011c5c63b: Verifying Checksum 2024-08-20T21:39:19.0967218Z 4e5011c5c63b: Download complete 2024-08-20T21:39:19.1731033Z d4e444cefea4: Download complete 2024-08-20T21:39:19.2483252Z 81ac92e10710: Verifying Checksum 2024-08-20T21:39:19.2484135Z 81ac92e10710: Download complete 2024-08-20T21:39:19.3174355Z ca1818242f05: Verifying Checksum 2024-08-20T21:39:19.3175213Z ca1818242f05: Download complete 2024-08-20T21:39:19.3842262Z 025b0cee3641: Verifying Checksum 2024-08-20T21:39:19.3843032Z 025b0cee3641: Download complete 2024-08-20T21:39:19.4575217Z 1f0b12a5c004: Verifying Checksum 2024-08-20T21:39:19.4575949Z 1f0b12a5c004: Download complete 2024-08-20T21:39:19.5283585Z d63ee2a60504: Verifying Checksum 2024-08-20T21:39:19.5284357Z d63ee2a60504: Download complete 2024-08-20T21:39:19.6056024Z d040b74545d9: Verifying Checksum 2024-08-20T21:39:19.6056735Z d040b74545d9: Download complete 2024-08-20T21:39:19.6707102Z 13bc16542484: Download complete 2024-08-20T21:39:19.7391118Z dfb6cbc954b5: Download complete 2024-08-20T21:39:19.8200698Z 5b1463b1c261: Verifying Checksum 2024-08-20T21:39:19.8201321Z 5b1463b1c261: Download complete 2024-08-20T21:39:20.2971895Z 172d385449d5: Verifying Checksum 2024-08-20T21:39:20.2972654Z 172d385449d5: Download complete 2024-08-20T21:39:20.3745111Z cc93353090ff: Verifying Checksum 2024-08-20T21:39:20.3745932Z cc93353090ff: Download complete 2024-08-20T21:39:20.4416468Z cf364f5f6244: Verifying Checksum 2024-08-20T21:39:20.4417027Z cf364f5f6244: Download complete 2024-08-20T21:39:20.5195276Z 8d5cbba0c8c6: Verifying Checksum 2024-08-20T21:39:20.5196031Z 8d5cbba0c8c6: Download complete 2024-08-20T21:39:20.5995381Z 2e373a30f750: Verifying Checksum 2024-08-20T21:39:20.5996068Z 2e373a30f750: Download complete 2024-08-20T21:39:20.8581555Z 529c734f1880: Verifying Checksum 2024-08-20T21:39:20.8582408Z 529c734f1880: Download complete 2024-08-20T21:39:20.9229234Z 9c1479550b9d: Verifying Checksum 2024-08-20T21:39:20.9229984Z 9c1479550b9d: Download complete 2024-08-20T21:39:21.0016025Z 292d6ba03740: Verifying Checksum 2024-08-20T21:39:21.0016761Z 292d6ba03740: Download complete 2024-08-20T21:39:21.0652478Z 81b70407002c: Verifying Checksum 2024-08-20T21:39:21.0653231Z 81b70407002c: Download complete 2024-08-20T21:39:21.1474168Z 1fc092fa7a5d: Verifying Checksum 2024-08-20T21:39:21.1474884Z 1fc092fa7a5d: Download complete 2024-08-20T21:39:21.2262142Z 95d6f4de1332: Verifying Checksum 2024-08-20T21:39:21.2262803Z 95d6f4de1332: Download complete 2024-08-20T21:39:21.3007621Z 84b6af65e568: Verifying Checksum 2024-08-20T21:39:21.3008326Z 84b6af65e568: Download complete 2024-08-20T21:39:21.3636950Z f89bdeb67794: Verifying Checksum 2024-08-20T21:39:21.3637644Z f89bdeb67794: Download complete 2024-08-20T21:39:25.1044852Z 1c49c3d4f7fc: Verifying Checksum 2024-08-20T21:39:25.1045660Z 1c49c3d4f7fc: Download complete 2024-08-20T21:39:25.1800488Z b4e22c165860: Verifying Checksum 2024-08-20T21:39:25.1801074Z b4e22c165860: Download complete 2024-08-20T21:39:25.2497368Z b7a95f68be13: Verifying Checksum 2024-08-20T21:39:25.2497944Z b7a95f68be13: Download complete 2024-08-20T21:39:25.3250348Z b7ae7e6f572a: Verifying Checksum 2024-08-20T21:39:25.3251066Z b7ae7e6f572a: Download complete 2024-08-20T21:39:25.4062137Z b83dd8df6637: Verifying Checksum 2024-08-20T21:39:25.4062738Z b83dd8df6637: Download complete 2024-08-20T21:39:25.4815091Z bb36ec6ca619: Verifying Checksum 2024-08-20T21:39:25.4815784Z bb36ec6ca619: Download complete 2024-08-20T21:39:25.5367146Z c52074bda120: Verifying Checksum 2024-08-20T21:39:25.5367890Z c52074bda120: Download complete 2024-08-20T21:39:25.6355360Z 8f914a53bad0: Verifying Checksum 2024-08-20T21:39:25.6355917Z 8f914a53bad0: Download complete 2024-08-20T21:39:25.7105016Z 3d207949ee5a: Download complete 2024-08-20T21:39:25.7969851Z 638224ef337d: Verifying Checksum 2024-08-20T21:39:25.7970580Z 638224ef337d: Download complete 2024-08-20T21:39:25.8743078Z cf9d145104f4: Verifying Checksum 2024-08-20T21:39:25.8743649Z cf9d145104f4: Download complete 2024-08-20T21:39:25.9391651Z c4d2ce3c205c: Verifying Checksum 2024-08-20T21:39:25.9392396Z c4d2ce3c205c: Download complete 2024-08-20T21:39:26.0076125Z d417897013fc: Verifying Checksum 2024-08-20T21:39:26.0076807Z d417897013fc: Download complete 2024-08-20T21:39:26.1372136Z 1ba13f47f442: Verifying Checksum 2024-08-20T21:39:26.1372603Z 1ba13f47f442: Download complete 2024-08-20T21:39:26.2060375Z e6f28cfa3010: Verifying Checksum 2024-08-20T21:39:26.2060932Z e6f28cfa3010: Download complete 2024-08-20T21:39:26.2816679Z 305e45e19ff7: Verifying Checksum 2024-08-20T21:39:26.2817346Z 305e45e19ff7: Download complete 2024-08-20T21:39:26.3421851Z 78839cbb08a6: Download complete 2024-08-20T21:39:26.4192556Z 97e6589defb2: Download complete 2024-08-20T21:39:26.4954322Z 00151a7db07b: Verifying Checksum 2024-08-20T21:39:26.4954880Z 00151a7db07b: Download complete 2024-08-20T21:39:26.5649145Z 2853b72b213b: Verifying Checksum 2024-08-20T21:39:26.5649705Z 2853b72b213b: Download complete 2024-08-20T21:39:27.9094884Z 84ce0a95a6ce: Verifying Checksum 2024-08-20T21:39:27.9095471Z 84ce0a95a6ce: Download complete 2024-08-20T21:39:28.9476653Z c28adb4ed886: Pull complete 2024-08-20T21:39:29.0997505Z 531dcd0b8280: Pull complete 2024-08-20T21:39:29.2470441Z c81ff9d20d36: Pull complete 2024-08-20T21:39:29.3779366Z 3a2d8ecdebc5: Pull complete 2024-08-20T21:39:29.5484687Z 8538dcd89f0c: Pull complete 2024-08-20T21:39:29.6888721Z 4a7b11d08282: Pull complete 2024-08-20T21:39:51.6849218Z ccda2116464a: Pull complete 2024-08-20T21:39:51.8775645Z 78022b868ce3: Pull complete 2024-08-20T21:39:51.9906543Z 5f75f9c1c9a3: Pull complete 2024-08-20T21:39:52.2170939Z aaf08982c002: Pull complete 2024-08-20T21:39:56.7570193Z 906cc3fc25c7: Verifying Checksum 2024-08-20T21:39:56.7571803Z 906cc3fc25c7: Download complete 2024-08-20T21:40:11.8692653Z 350a7e72a699: Verifying Checksum 2024-08-20T21:40:11.8693116Z 350a7e72a699: Download complete 2024-08-20T21:41:07.9037953Z 906cc3fc25c7: Pull complete 2024-08-20T21:41:08.1196707Z 4f4fb700ef54: Pull complete 2024-08-20T21:41:08.3206036Z 88e873d7feff: Pull complete 2024-08-20T21:41:08.5600478Z 5a604d52e1bb: Pull complete 2024-08-20T21:41:08.8291226Z 63e67358db63: Pull complete 2024-08-20T21:41:09.1697197Z 3a626210afeb: Pull complete 2024-08-20T21:41:09.2229513Z a0eb49634d88: Pull complete 2024-08-20T21:41:09.3101631Z fc983e33a069: Pull complete 2024-08-20T21:41:09.4214476Z d6ba8b647425: Pull complete 2024-08-20T21:41:09.8253533Z 800ce5e504c0: Pull complete 2024-08-20T21:41:09.9536320Z 8335a3887870: Pull complete 2024-08-20T21:41:13.8937271Z 3a7c6ce8a2c5: Pull complete 2024-08-20T21:41:14.1639480Z a7f2a22883c0: Pull complete 2024-08-20T21:41:14.4457219Z 43b5054ea31a: Pull complete 2024-08-20T21:41:14.6672128Z 8da2931100c6: Pull complete 2024-08-20T21:41:14.8453452Z 11a81df37a9c: Pull complete 2024-08-20T21:41:14.9715404Z 076c3c501fa9: Pull complete 2024-08-20T21:41:26.2798994Z 1c49c3d4f7fc: Pull complete 2024-08-20T21:41:26.3809380Z 4868a90a66f1: Pull complete 2024-08-20T21:41:26.4823669Z 4e5011c5c63b: Pull complete 2024-08-20T21:41:26.6616047Z d4e444cefea4: Pull complete 2024-08-20T21:41:26.8451892Z 81ac92e10710: Pull complete 2024-08-20T21:41:27.2776532Z ca1818242f05: Pull complete 2024-08-20T21:41:27.4734966Z 025b0cee3641: Pull complete 2024-08-20T21:41:27.6583979Z 1f0b12a5c004: Pull complete 2024-08-20T21:41:27.8396408Z d63ee2a60504: Pull complete 2024-08-20T21:41:28.1598913Z d040b74545d9: Pull complete 2024-08-20T21:41:28.3172867Z 13bc16542484: Pull complete 2024-08-20T21:41:28.6857947Z dfb6cbc954b5: Pull complete 2024-08-20T21:41:28.9119145Z 5b1463b1c261: Pull complete 2024-08-20T21:41:31.3489441Z 172d385449d5: Pull complete 2024-08-20T21:41:31.3777123Z cc93353090ff: Pull complete 2024-08-20T21:41:31.4014909Z cf364f5f6244: Pull complete 2024-08-20T21:41:31.5900163Z 8d5cbba0c8c6: Pull complete 2024-08-20T21:41:31.7359626Z 2e373a30f750: Pull complete 2024-08-20T21:41:32.5512859Z 529c734f1880: Pull complete 2024-08-20T21:41:32.6181144Z 9c1479550b9d: Pull complete 2024-08-20T21:41:32.6579957Z 292d6ba03740: Pull complete 2024-08-20T21:41:32.9076448Z 81b70407002c: Pull complete 2024-08-20T21:41:33.1251565Z 1fc092fa7a5d: Pull complete 2024-08-20T21:41:33.5123420Z 95d6f4de1332: Pull complete 2024-08-20T21:41:33.7044114Z 84b6af65e568: Pull complete 2024-08-20T21:41:34.0919342Z f89bdeb67794: Pull complete 2024-08-20T21:43:18.5780845Z 350a7e72a699: Pull complete 2024-08-20T21:43:18.5941928Z b4e22c165860: Pull complete 2024-08-20T21:43:18.6078795Z b7a95f68be13: Pull complete 2024-08-20T21:43:18.6355609Z b7ae7e6f572a: Pull complete 2024-08-20T21:43:18.6659635Z b83dd8df6637: Pull complete 2024-08-20T21:43:18.6799685Z bb36ec6ca619: Pull complete 2024-08-20T21:43:18.7100462Z c52074bda120: Pull complete 2024-08-20T21:43:18.7389383Z 8f914a53bad0: Pull complete 2024-08-20T21:43:18.7526389Z 3d207949ee5a: Pull complete 2024-08-20T21:43:18.7806493Z 638224ef337d: Pull complete 2024-08-20T21:43:18.7964020Z cf9d145104f4: Pull complete 2024-08-20T21:43:18.8243021Z c4d2ce3c205c: Pull complete 2024-08-20T21:43:18.8374686Z d417897013fc: Pull complete 2024-08-20T21:43:18.9761000Z 1ba13f47f442: Pull complete 2024-08-20T21:43:18.9912465Z e6f28cfa3010: Pull complete 2024-08-20T21:43:19.0056454Z 305e45e19ff7: Pull complete 2024-08-20T21:43:19.0190487Z 78839cbb08a6: Pull complete 2024-08-20T21:43:19.0340393Z 97e6589defb2: Pull complete 2024-08-20T21:43:19.0512115Z 00151a7db07b: Pull complete 2024-08-20T21:43:19.0814172Z 2853b72b213b: Pull complete 2024-08-20T21:43:20.6089114Z 84ce0a95a6ce: Pull complete 2024-08-20T21:43:20.6470195Z Digest: sha256:9c16298ecf1ba0ec71e25268bd4244acb2b2c8c60846e6c8a5e8eb35b9eff836 2024-08-20T21:43:20.6505211Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:43:20.6562150Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:43:20.6621907Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T21:43:20.6622868Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-20T21:43:20.6630246Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:20.6630717Z env: 2024-08-20T21:43:20.6630969Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:20.6631290Z ##[endgroup] 2024-08-20T21:43:20.6709653Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T21:43:20.6710431Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-20T21:43:20.6711111Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-20T21:43:20.6711739Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-20T21:43:20.6717838Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:20.6718324Z env: 2024-08-20T21:43:20.6718580Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:20.6718888Z ##[endgroup] 2024-08-20T21:43:20.9833974Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T21:43:21.0003694Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.9/site-packages (5.9.1) 2024-08-20T21:43:21.0010913Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.9/site-packages (11.525.84) 2024-08-20T21:43:21.1286156Z Prepare all required actions 2024-08-20T21:43:21.1286641Z Getting action download info 2024-08-20T21:43:21.2560753Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-20T21:43:21.4328183Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-20T21:43:21.5684218Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-20T21:43:21.5684699Z with: 2024-08-20T21:43:21.5684960Z name: linux-jammy-py3.8-gcc11 2024-08-20T21:43:21.5685330Z s3-bucket: gha-artifacts 2024-08-20T21:43:21.5685643Z env: 2024-08-20T21:43:21.5685880Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:21.5686203Z ##[endgroup] 2024-08-20T21:43:21.5721047Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T21:43:21.5721489Z with: 2024-08-20T21:43:21.5721749Z name: linux-jammy-py3.8-gcc11 2024-08-20T21:43:21.5722124Z s3-bucket: gha-artifacts 2024-08-20T21:43:21.5722491Z region: us-east-1 2024-08-20T21:43:21.5722767Z env: 2024-08-20T21:43:21.5723019Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:21.5723332Z ##[endgroup] 2024-08-20T21:43:22.0741886Z (node:497587) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T21:43:22.0742614Z 2024-08-20T21:43:22.0742867Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T21:43:22.0743672Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T21:43:22.0745053Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T21:43:22.1448103Z Found 1 objects with prefix pytorch/pytorch/10479309237/linux-jammy-py3.8-gcc11/ 2024-08-20T21:43:22.1449154Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T21:43:25.8933601Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-20T21:43:25.8940060Z Artifact download has finished successfully 2024-08-20T21:43:25.9124183Z ##[group]Run unzip -o artifacts.zip 2024-08-20T21:43:25.9124608Z unzip -o artifacts.zip 2024-08-20T21:43:25.9130774Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:25.9131246Z env: 2024-08-20T21:43:25.9131505Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:25.9131936Z ##[endgroup] 2024-08-20T21:43:25.9352583Z Archive: artifacts.zip 2024-08-20T21:43:25.9353472Z creating: dist/ 2024-08-20T21:43:26.8701592Z inflating: dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl 2024-08-20T21:43:26.8702703Z creating: build/custom_test_artifacts/ 2024-08-20T21:43:26.8703307Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-20T21:43:26.8704016Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-20T21:43:26.8704868Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-20T21:43:26.8706734Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T21:43:26.8707781Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-20T21:43:26.8708767Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T21:43:26.8709882Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T21:43:26.8711032Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T21:43:26.8712221Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T21:43:26.8713443Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T21:43:26.8714480Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T21:43:26.8715549Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T21:43:26.8717126Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T21:43:26.8718549Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T21:43:26.8719923Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T21:43:26.8721259Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T21:43:26.8722552Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T21:43:26.8723908Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T21:43:26.8724980Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-20T21:43:26.8725915Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-20T21:43:26.8726883Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-20T21:43:26.8727953Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-20T21:43:26.8729152Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-20T21:43:26.8730302Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-20T21:43:26.8731380Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-20T21:43:26.8732594Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-20T21:43:26.8733724Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-20T21:43:26.8734839Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-20T21:43:26.8735960Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-20T21:43:26.8737074Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-20T21:43:26.8743261Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-20T21:43:26.8871415Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-20T21:43:26.8872582Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-20T21:43:26.8873810Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-20T21:43:26.8875150Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-20T21:43:26.8876464Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-20T21:43:26.8877609Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-20T21:43:26.8878790Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-20T21:43:26.8879968Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-20T21:43:26.8881154Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-20T21:43:26.8882338Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-20T21:43:26.8883489Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-20T21:43:26.8894972Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-20T21:43:26.8969715Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-20T21:43:26.8971112Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:43:26.8972533Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-20T21:43:26.8973557Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-20T21:43:26.8974593Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-20T21:43:26.8975567Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-20T21:43:26.8976446Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-20T21:43:26.8977244Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-20T21:43:26.8978066Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-20T21:43:26.9086442Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-20T21:43:26.9144725Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-20T21:43:26.9145569Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-20T21:43:26.9146273Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-20T21:43:26.9147093Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-20T21:43:26.9148627Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T21:43:26.9149633Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-20T21:43:26.9150658Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T21:43:26.9151671Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T21:43:26.9152664Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T21:43:26.9154190Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T21:43:26.9156011Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T21:43:26.9157031Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T21:43:26.9158585Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T21:43:26.9160179Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T21:43:26.9161788Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T21:43:26.9163511Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T21:43:26.9165398Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T21:43:26.9167715Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T21:43:26.9169872Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T21:43:26.9171523Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-20T21:43:26.9172574Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-20T21:43:26.9173530Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-20T21:43:26.9174640Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-20T21:43:26.9175878Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-20T21:43:26.9177066Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-20T21:43:26.9178356Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-20T21:43:26.9179605Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-20T21:43:26.9180756Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-20T21:43:26.9181905Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-20T21:43:26.9183058Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-20T21:43:26.9184199Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-20T21:43:26.9190194Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-20T21:43:26.9248753Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-20T21:43:26.9251030Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:43:26.9252812Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-20T21:43:26.9254687Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-20T21:43:26.9255597Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-20T21:43:26.9256509Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-20T21:43:26.9257382Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-20T21:43:26.9258312Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-20T21:43:26.9259128Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-20T21:43:26.9301476Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-20T21:43:26.9302697Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-20T21:43:26.9303454Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-20T21:43:26.9304360Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-20T21:43:26.9306128Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-20T21:43:26.9307597Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-20T21:43:26.9308991Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-20T21:43:26.9310536Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-20T21:43:26.9311873Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-20T21:43:26.9313486Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-20T21:43:26.9315107Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-20T21:43:26.9316898Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-20T21:43:26.9318383Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-20T21:43:26.9320774Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-20T21:43:26.9323117Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-20T21:43:26.9325490Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-20T21:43:26.9326734Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-20T21:43:26.9328145Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-20T21:43:26.9329845Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-20T21:43:26.9331024Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-20T21:43:26.9332102Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-20T21:43:26.9333172Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-20T21:43:26.9334361Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-20T21:43:26.9335682Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-20T21:43:26.9336959Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-20T21:43:26.9338158Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-20T21:43:26.9339392Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-20T21:43:26.9340618Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-20T21:43:26.9341866Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-20T21:43:26.9343104Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-20T21:43:26.9344325Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-20T21:43:26.9345622Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-20T21:43:26.9444369Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-20T21:43:26.9446706Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-20T21:43:26.9449026Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-20T21:43:26.9450533Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-20T21:43:26.9451944Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-20T21:43:26.9453200Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-20T21:43:26.9454493Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-20T21:43:26.9455806Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-20T21:43:26.9457121Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-20T21:43:26.9458596Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-20T21:43:26.9459877Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-20T21:43:26.9468481Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-20T21:43:26.9518821Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-20T21:43:26.9521287Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:43:26.9523067Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-20T21:43:26.9524985Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-20T21:43:26.9526077Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-20T21:43:26.9527078Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-20T21:43:26.9528020Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-20T21:43:26.9528873Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-20T21:43:26.9529749Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-20T21:43:26.9622282Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-20T21:43:26.9662610Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-20T21:43:26.9663591Z creating: build/lib/ 2024-08-20T21:43:26.9664081Z inflating: build/lib/libclog.a 2024-08-20T21:43:26.9672647Z inflating: build/lib/libpthreadpool.a 2024-08-20T21:43:26.9680329Z inflating: build/lib/libcpuinfo_internals.a 2024-08-20T21:43:26.9688109Z inflating: build/lib/libcpuinfo.a 2024-08-20T21:43:26.9750545Z inflating: build/lib/libgtest.a 2024-08-20T21:43:26.9757625Z inflating: build/lib/libittnotify.a 2024-08-20T21:43:26.9836059Z inflating: build/lib/libprotobuf-lite.a 2024-08-20T21:43:26.9905676Z inflating: build/lib/libbenchmark.a 2024-08-20T21:43:26.9932060Z inflating: build/lib/libtensorpipe_uv.a 2024-08-20T21:43:26.9992566Z inflating: build/lib/libasmjit.a 2024-08-20T21:43:27.0099795Z inflating: build/lib/libgloo.a 2024-08-20T21:43:27.0519315Z inflating: build/lib/libprotobuf.a 2024-08-20T21:43:27.0538760Z inflating: build/lib/libfmt.a 2024-08-20T21:43:27.0539934Z inflating: build/lib/libtorch_global_deps.so 2024-08-20T21:43:27.0630486Z inflating: build/lib/libc10.so 2024-08-20T21:43:27.0632650Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-20T21:43:27.0650551Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-20T21:43:27.1120740Z inflating: build/lib/libprotoc.a 2024-08-20T21:43:27.1137212Z inflating: build/lib/libgmock.a 2024-08-20T21:43:27.1137986Z inflating: build/lib/libgtest_main.a 2024-08-20T21:43:27.1138851Z inflating: build/lib/libbenchmark_main.a 2024-08-20T21:43:27.1155609Z inflating: build/lib/libnnpack.a 2024-08-20T21:43:27.1675266Z inflating: build/lib/libtensorpipe.a 2024-08-20T21:43:27.2651467Z inflating: build/lib/libfbgemm.a 2024-08-20T21:43:27.2652335Z inflating: build/lib/libgmock_main.a 2024-08-20T21:43:28.1710066Z inflating: build/lib/libdnnl.a 2024-08-20T21:43:28.1980222Z inflating: build/lib/libkineto.a 2024-08-20T21:43:28.2018289Z inflating: build/lib/libonnx_proto.a 2024-08-20T21:43:28.2191420Z inflating: build/lib/libXNNPACK.a 2024-08-20T21:43:28.2842391Z inflating: build/lib/libonnx.a 2024-08-20T21:43:30.5022748Z inflating: build/lib/libtorch_cpu.so 2024-08-20T21:43:30.5023550Z inflating: build/lib/libtorch.so 2024-08-20T21:43:30.5028056Z inflating: build/lib/libunbox_lib.a 2024-08-20T21:43:30.5031626Z inflating: build/lib/libshm.so 2024-08-20T21:43:30.5099159Z inflating: build/lib/libtorchbind_test.so 2024-08-20T21:43:30.5117800Z inflating: build/lib/libjitbackend_test.so 2024-08-20T21:43:30.5141538Z inflating: build/lib/libaoti_custom_ops.so 2024-08-20T21:43:30.5165352Z inflating: build/lib/libbackend_with_compiler.so 2024-08-20T21:43:30.6966262Z inflating: build/lib/libtorch_python.so 2024-08-20T21:43:30.6998605Z inflating: build/lib/libnnapi_backend.so 2024-08-20T21:43:30.6999402Z creating: build/bin/ 2024-08-20T21:43:30.7000029Z creating: build/bin/CMakeFiles/ 2024-08-20T21:43:30.7001077Z inflating: build/bin/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-20T21:43:30.7002268Z extracting: build/bin/CMakeFiles/progress.marks 2024-08-20T21:43:30.7002997Z inflating: build/bin/Makefile 2024-08-20T21:43:30.7003448Z inflating: build/bin/cmake_install.cmake 2024-08-20T21:43:30.7004023Z inflating: build/bin/CTestTestfile.cmake 2024-08-20T21:43:30.7051825Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-20T21:43:30.7103036Z inflating: build/bin/c10_DeviceGuard_test 2024-08-20T21:43:30.7154220Z inflating: build/bin/c10_Device_test 2024-08-20T21:43:30.7213157Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-20T21:43:30.7266764Z inflating: build/bin/c10_Scalar_test 2024-08-20T21:43:30.7315730Z inflating: build/bin/c10_StreamGuard_test 2024-08-20T21:43:30.7366739Z inflating: build/bin/c10_SymInt_test 2024-08-20T21:43:30.7420782Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-20T21:43:30.7476116Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-20T21:43:30.7531296Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-20T21:43:30.7600964Z inflating: build/bin/c10_cow_test 2024-08-20T21:43:30.7653926Z inflating: build/bin/c10_Bitset_test 2024-08-20T21:43:30.7702726Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-20T21:43:30.7752333Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-20T21:43:30.7803147Z inflating: build/bin/c10_Half_test 2024-08-20T21:43:30.7858913Z inflating: build/bin/c10_LeftRight_test 2024-08-20T21:43:30.7914250Z inflating: build/bin/c10_Metaprogramming_test 2024-08-20T21:43:30.7964562Z inflating: build/bin/c10_Synchronized_test 2024-08-20T21:43:30.8019350Z inflating: build/bin/c10_ThreadLocal_test 2024-08-20T21:43:30.8070938Z inflating: build/bin/c10_TypeIndex_test 2024-08-20T21:43:30.8121878Z inflating: build/bin/c10_TypeList_test 2024-08-20T21:43:30.8170758Z inflating: build/bin/c10_TypeTraits_test 2024-08-20T21:43:30.8222172Z inflating: build/bin/c10_accumulate_test 2024-08-20T21:43:30.8277168Z inflating: build/bin/c10_bfloat16_test 2024-08-20T21:43:30.8327163Z inflating: build/bin/c10_bit_cast_test 2024-08-20T21:43:30.8383370Z inflating: build/bin/c10_complex_math_test 2024-08-20T21:43:30.8438573Z inflating: build/bin/c10_complex_test 2024-08-20T21:43:30.8491052Z inflating: build/bin/c10_exception_test 2024-08-20T21:43:30.8541248Z inflating: build/bin/c10_flags_test 2024-08-20T21:43:30.8591532Z inflating: build/bin/c10_generic_math_test 2024-08-20T21:43:30.8642063Z inflating: build/bin/c10_irange_test 2024-08-20T21:43:30.8800014Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-20T21:43:30.8853044Z inflating: build/bin/c10_lazy_test 2024-08-20T21:43:30.8910069Z inflating: build/bin/c10_logging_test 2024-08-20T21:43:30.8983891Z inflating: build/bin/c10_optional_test 2024-08-20T21:43:30.9045414Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-20T21:43:30.9098675Z inflating: build/bin/c10_registry_test 2024-08-20T21:43:30.9245421Z inflating: build/bin/c10_small_vector_test 2024-08-20T21:43:30.9296596Z inflating: build/bin/c10_ssize_test 2024-08-20T21:43:30.9348674Z inflating: build/bin/c10_string_util_test 2024-08-20T21:43:30.9407044Z inflating: build/bin/c10_string_view_test 2024-08-20T21:43:30.9457195Z inflating: build/bin/c10_tempfile_test 2024-08-20T21:43:30.9513062Z inflating: build/bin/c10_typeid_test 2024-08-20T21:43:30.9559914Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-20T21:43:30.9998067Z inflating: build/bin/protoc-3.13.0.0 2024-08-20T21:43:31.0434742Z inflating: build/bin/protoc 2024-08-20T21:43:31.0754347Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-20T21:43:31.1090889Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-20T21:43:31.1439453Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-20T21:43:31.1491501Z inflating: build/bin/BackoffTest 2024-08-20T21:43:31.1547053Z inflating: build/bin/TCPStoreTest 2024-08-20T21:43:31.1600056Z inflating: build/bin/FileStoreTest 2024-08-20T21:43:31.1653264Z inflating: build/bin/HashStoreTest 2024-08-20T21:43:31.1656338Z inflating: build/bin/example_allreduce 2024-08-20T21:43:31.1720991Z inflating: build/bin/ProcessGroupGlooTest 2024-08-20T21:43:31.1774981Z inflating: build/bin/test_dist_autograd 2024-08-20T21:43:31.1777766Z inflating: build/bin/parallel_benchmark 2024-08-20T21:43:31.1844448Z inflating: build/bin/test_cpp_rpc 2024-08-20T21:43:31.1909835Z inflating: build/bin/test_mobile_nnc 2024-08-20T21:43:31.1918413Z inflating: build/bin/aot_model_compiler_test 2024-08-20T21:43:31.1975587Z inflating: build/bin/static_runtime_bench 2024-08-20T21:43:31.2303205Z inflating: build/bin/test_lazy 2024-08-20T21:43:31.2375686Z inflating: build/bin/Dict_test 2024-08-20T21:43:31.2427784Z inflating: build/bin/Dimname_test 2024-08-20T21:43:31.2492160Z inflating: build/bin/MaybeOwned_test 2024-08-20T21:43:31.2548844Z inflating: build/bin/NamedTensor_test 2024-08-20T21:43:31.2791402Z inflating: build/bin/static_runtime_test 2024-08-20T21:43:31.2849783Z inflating: build/bin/apply_utils_test 2024-08-20T21:43:31.2907684Z inflating: build/bin/atest 2024-08-20T21:43:31.2969959Z inflating: build/bin/basic 2024-08-20T21:43:31.3024469Z inflating: build/bin/broadcast_test 2024-08-20T21:43:31.3075306Z inflating: build/bin/cpu_allocator_test 2024-08-20T21:43:31.3133083Z inflating: build/bin/cpu_generator_test 2024-08-20T21:43:31.3185779Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-20T21:43:31.3276412Z inflating: build/bin/cpu_rng_test 2024-08-20T21:43:31.3326452Z inflating: build/bin/dispatch_key_set_test 2024-08-20T21:43:31.3376786Z inflating: build/bin/dlconvertor_test 2024-08-20T21:43:31.3434056Z inflating: build/bin/extension_backend_test 2024-08-20T21:43:31.3489158Z inflating: build/bin/half_test 2024-08-20T21:43:31.3582161Z inflating: build/bin/ivalue_test 2024-08-20T21:43:31.3631823Z inflating: build/bin/lazy_tensor_test 2024-08-20T21:43:31.3685149Z inflating: build/bin/math_kernel_test 2024-08-20T21:43:31.3738623Z inflating: build/bin/memory_format_test 2024-08-20T21:43:31.3791707Z inflating: build/bin/memory_overlapping_test 2024-08-20T21:43:31.3844302Z inflating: build/bin/mobile_memory_cleanup 2024-08-20T21:43:31.4928951Z inflating: build/bin/test_api 2024-08-20T21:43:31.4984782Z inflating: build/bin/native_test 2024-08-20T21:43:31.5035303Z inflating: build/bin/operator_name_test 2024-08-20T21:43:31.5085992Z inflating: build/bin/operators_test 2024-08-20T21:43:31.5138083Z inflating: build/bin/packedtensoraccessor_test 2024-08-20T21:43:31.5204544Z inflating: build/bin/pow_test 2024-08-20T21:43:31.5261750Z inflating: build/bin/quantized_test 2024-08-20T21:43:31.5311362Z inflating: build/bin/reduce_ops_test 2024-08-20T21:43:31.5362689Z inflating: build/bin/reportMemoryUsage_test 2024-08-20T21:43:31.5419064Z inflating: build/bin/scalar_tensor_test 2024-08-20T21:43:31.5470493Z inflating: build/bin/StorageUtils_test 2024-08-20T21:43:31.5528073Z inflating: build/bin/scalar_test 2024-08-20T21:43:31.5580455Z inflating: build/bin/stride_properties_test 2024-08-20T21:43:31.5634966Z inflating: build/bin/type_ptr_test 2024-08-20T21:43:31.5637478Z inflating: build/bin/thread_init_test 2024-08-20T21:43:31.5714450Z inflating: build/bin/tensor_iterator_test 2024-08-20T21:43:31.5768166Z inflating: build/bin/test_parallel 2024-08-20T21:43:31.5820480Z inflating: build/bin/undefined_tensor_test 2024-08-20T21:43:31.5821905Z inflating: build/bin/verify_api_visibility 2024-08-20T21:43:31.5881082Z inflating: build/bin/type_test 2024-08-20T21:43:31.5949571Z inflating: build/bin/legacy_vmap_test 2024-08-20T21:43:31.6000751Z inflating: build/bin/weakref_test 2024-08-20T21:43:31.6103850Z inflating: build/bin/List_test 2024-08-20T21:43:31.6155105Z inflating: build/bin/xla_tensor_test 2024-08-20T21:43:31.6214130Z inflating: build/bin/IListRef_test 2024-08-20T21:43:31.6265635Z inflating: build/bin/wrapdim_test 2024-08-20T21:43:31.6330929Z inflating: build/bin/KernelFunction_test 2024-08-20T21:43:31.6448912Z inflating: build/bin/kernel_function_legacy_test 2024-08-20T21:43:31.6542806Z inflating: build/bin/kernel_function_test 2024-08-20T21:43:31.6665630Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-20T21:43:31.6764510Z inflating: build/bin/kernel_lambda_test 2024-08-20T21:43:31.6824783Z inflating: build/bin/kernel_stackbased_test 2024-08-20T21:43:31.6876017Z inflating: build/bin/CppSignature_test 2024-08-20T21:43:31.6969183Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-20T21:43:31.7018248Z inflating: build/bin/op_allowlist_test 2024-08-20T21:43:31.7082629Z inflating: build/bin/inline_container_test 2024-08-20T21:43:31.7137945Z inflating: build/bin/backend_fallback_test 2024-08-20T21:43:31.7436329Z inflating: build/bin/op_registration_test 2024-08-20T21:43:31.7490145Z inflating: build/bin/test_edge_op_registration 2024-08-20T21:43:31.7494878Z inflating: build/bin/torch_shm_manager 2024-08-20T21:43:31.7507750Z inflating: build/bin/tutorial_tensorexpr 2024-08-20T21:43:31.8266626Z inflating: build/bin/test_tensorexpr 2024-08-20T21:43:31.8822817Z inflating: build/bin/test_jit 2024-08-20T21:43:31.8823590Z creating: .additional_ci_files/ 2024-08-20T21:43:31.8880973Z inflating: .additional_ci_files/test-times.json 2024-08-20T21:43:31.9108949Z inflating: .additional_ci_files/test-class-times.json 2024-08-20T21:43:31.9187566Z ##[group]Run rm artifacts.zip 2024-08-20T21:43:31.9187942Z rm artifacts.zip 2024-08-20T21:43:31.9194035Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:31.9194521Z env: 2024-08-20T21:43:31.9194777Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:31.9195089Z ##[endgroup] 2024-08-20T21:43:31.9482950Z ##[group]Run df -H 2024-08-20T21:43:31.9483243Z df -H 2024-08-20T21:43:31.9488525Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:31.9488995Z env: 2024-08-20T21:43:31.9489254Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:31.9489589Z ##[endgroup] 2024-08-20T21:43:31.9561711Z Filesystem Size Used Avail Use% Mounted on 2024-08-20T21:43:31.9562301Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-20T21:43:31.9562772Z tmpfs 8.2G 16M 8.2G 1% /dev/shm 2024-08-20T21:43:31.9563220Z tmpfs 3.3G 488k 3.3G 1% /run 2024-08-20T21:43:31.9563639Z /dev/nvme0n1p1 161G 32G 130G 20% / 2024-08-20T21:43:31.9564348Z tmpfs 8.2G 41k 8.2G 1% /tmp 2024-08-20T21:43:31.9565149Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-20T21:43:31.9633391Z Prepare all required actions 2024-08-20T21:43:31.9633923Z Getting action download info 2024-08-20T21:43:32.0872501Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-20T21:43:32.0872945Z with: 2024-08-20T21:43:32.0873176Z env: 2024-08-20T21:43:32.0873427Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:32.0873737Z ##[endgroup] 2024-08-20T21:43:32.0919227Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-20T21:43:32.0919671Z with: 2024-08-20T21:43:32.0919921Z name: td_results 2024-08-20T21:43:32.0920206Z s3-bucket: gha-artifacts 2024-08-20T21:43:32.0920548Z region: us-east-1 2024-08-20T21:43:32.0920807Z env: 2024-08-20T21:43:32.0921054Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:32.0921378Z ##[endgroup] 2024-08-20T21:43:32.5532032Z (node:497606) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-20T21:43:32.5532779Z 2024-08-20T21:43:32.5533107Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-20T21:43:32.5534221Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-20T21:43:32.5535687Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-20T21:43:32.6304834Z Found 1 objects with prefix pytorch/pytorch/10479309237/td_results/ 2024-08-20T21:43:32.6305922Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-20T21:43:32.6789764Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-20T21:43:32.6795480Z Artifact download has finished successfully 2024-08-20T21:43:32.6970284Z ##[group]Run mkdir -p .additional_ci_files 2024-08-20T21:43:32.6970748Z mkdir -p .additional_ci_files 2024-08-20T21:43:32.6971298Z mv td_results.json .additional_ci_files/td_results.json 2024-08-20T21:43:32.6977728Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:32.6978205Z env: 2024-08-20T21:43:32.6978480Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:32.6978792Z ##[endgroup] 2024-08-20T21:43:32.7410815Z ##[group]Run .github/scripts/parse_ref.py 2024-08-20T21:43:32.7411270Z .github/scripts/parse_ref.py 2024-08-20T21:43:32.7416710Z shell: /usr/bin/bash -e {0} 2024-08-20T21:43:32.7417037Z env: 2024-08-20T21:43:32.7417294Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:32.7417603Z ##[endgroup] 2024-08-20T21:43:32.7937211Z Prepare all required actions 2024-08-20T21:43:32.8032262Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-20T21:43:32.8032706Z with: 2024-08-20T21:43:32.8033254Z github-token: *** 2024-08-20T21:43:32.8033520Z env: 2024-08-20T21:43:32.8033772Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:32.8034099Z ##[endgroup] 2024-08-20T21:43:32.8606263Z ##[group]Run set -eux 2024-08-20T21:43:32.8606579Z set -eux 2024-08-20T21:43:32.8607156Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-20T21:43:32.8613239Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:32.8613710Z env: 2024-08-20T21:43:32.8613964Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:32.8614475Z GITHUB_TOKEN: *** 2024-08-20T21:43:32.8614752Z ##[endgroup] 2024-08-20T21:43:32.8637540Z + python3 .github/scripts/get_workflow_job_id.py 10479309237 i-0276d496c9ebff6c0 2024-08-20T21:43:34.6550724Z setting job-id=29025447188 2024-08-20T21:43:34.6551632Z setting job-name=linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-20T21:43:34.6835084Z Prepare all required actions 2024-08-20T21:43:34.6835520Z Getting action download info 2024-08-20T21:43:34.8087788Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-20T21:43:34.8088225Z with: 2024-08-20T21:43:34.8088667Z github-token: *** 2024-08-20T21:43:34.8092347Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]} 2024-08-20T21:43:34.8096222Z job-name: linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-20T21:43:34.8096858Z env: 2024-08-20T21:43:34.8097099Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:34.8097426Z ##[endgroup] 2024-08-20T21:43:34.8193040Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-20T21:43:34.8193594Z with: 2024-08-20T21:43:34.8193838Z shell: bash 2024-08-20T21:43:34.8194107Z timeout_minutes: 10 2024-08-20T21:43:34.8194411Z max_attempts: 5 2024-08-20T21:43:34.8194702Z retry_wait_seconds: 30 2024-08-20T21:43:34.8195821Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T21:43:34.8197021Z polling_interval_seconds: 1 2024-08-20T21:43:34.8197519Z warning_on_retry: true 2024-08-20T21:43:34.8197847Z continue_on_error: false 2024-08-20T21:43:34.8198153Z env: 2024-08-20T21:43:34.8198404Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:34.8198898Z GITHUB_TOKEN: *** 2024-08-20T21:43:34.8199174Z ##[endgroup] 2024-08-20T21:43:34.9373473Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-20T21:43:35.1745446Z Defaulting to user installation because normal site-packages is not writeable 2024-08-20T21:43:35.1901962Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.9/site-packages (2.27.1) 2024-08-20T21:43:35.1906134Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.9/site-packages (6.0.1) 2024-08-20T21:43:35.2024149Z Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2.0.12) 2024-08-20T21:43:35.2028532Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-08-20T21:43:35.2037773Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-08-20T21:43:35.2044001Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2024.7.4) 2024-08-20T21:43:35.8899924Z Command completed after 1 attempt(s). 2024-08-20T21:43:35.8999841Z ##[group]Run set -x 2024-08-20T21:43:35.9000152Z set -x 2024-08-20T21:43:35.9000409Z  2024-08-20T21:43:35.9000938Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T21:43:35.9001608Z # in runner workspace 2024-08-20T21:43:35.9002102Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-20T21:43:35.9007947Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:35.9008430Z env: 2024-08-20T21:43:35.9008671Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:35.9009168Z ##[endgroup] 2024-08-20T21:43:35.9032962Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-20T21:43:35.9300144Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T21:43:35.9300666Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-20T21:43:35.9301110Z echo "Job name: ${JOB_NAME}" 2024-08-20T21:43:35.9301473Z  2024-08-20T21:43:35.9302001Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-20T21:43:35.9302676Z # in runner workspace 2024-08-20T21:43:35.9303220Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-20T21:43:35.9303862Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-20T21:43:35.9304303Z  --job-name "${JOB_NAME}" \ 2024-08-20T21:43:35.9307994Z  --test-matrix "{"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]}" \ 2024-08-20T21:43:35.9311693Z  --selected-test-configs "" \ 2024-08-20T21:43:35.9312134Z  --pr-number "${PR_NUMBER}" \ 2024-08-20T21:43:35.9312519Z  --tag "${TAG}" \ 2024-08-20T21:43:35.9312890Z  --event-name "${EVENT_NAME}" \ 2024-08-20T21:43:35.9313457Z  --schedule "${SCHEDULE}" \ 2024-08-20T21:43:35.9313855Z  --branch "${HEAD_BRANCH}" 2024-08-20T21:43:35.9319337Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:35.9319825Z env: 2024-08-20T21:43:35.9320076Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:35.9320601Z GITHUB_TOKEN: *** 2024-08-20T21:43:35.9321155Z JOB_NAME: linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-20T21:43:35.9321811Z PR_NUMBER: 133712 2024-08-20T21:43:35.9322084Z TAG: 2024-08-20T21:43:35.9322556Z EVENT_NAME: pull_request 2024-08-20T21:43:35.9322903Z SCHEDULE: 2024-08-20T21:43:35.9323175Z HEAD_BRANCH: 2024-08-20T21:43:35.9323455Z ##[endgroup] 2024-08-20T21:43:35.9346244Z Workflow: pull 2024-08-20T21:43:35.9346998Z Job name: linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-20T21:43:36.3150876Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-20T21:43:36.5195108Z ##[group]Run echo "Filtered matrix:" 2024-08-20T21:43:36.5195543Z echo "Filtered matrix:" 2024-08-20T21:43:36.5199130Z echo "{"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]}" 2024-08-20T21:43:36.5202706Z  2024-08-20T21:43:36.5203102Z echo 2024-08-20T21:43:36.5203460Z echo "Is the current job unstable? False" 2024-08-20T21:43:36.5203890Z  2024-08-20T21:43:36.5204140Z echo 2024-08-20T21:43:36.5204480Z echo "Is keep-going label set? False" 2024-08-20T21:43:36.5204893Z  2024-08-20T21:43:36.5205143Z echo 2024-08-20T21:43:36.5205438Z echo "Renabled issues? " 2024-08-20T21:43:36.5212130Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:36.5212627Z env: 2024-08-20T21:43:36.5212890Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:36.5213205Z ##[endgroup] 2024-08-20T21:43:36.5235436Z Filtered matrix: 2024-08-20T21:43:36.5240060Z {include: [{config: default, shard: 1, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: default, shard: 2, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: default, shard: 3, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: default, shard: 4, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: docs_test, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: jit_legacy, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: backwards_compat, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: distributed, shard: 1, num_shards: 2, runner: amz2023.linux.2xlarge}, {config: distributed, shard: 2, num_shards: 2, runner: amz2023.linux.2xlarge}]} 2024-08-20T21:43:36.5243443Z 2024-08-20T21:43:36.5243600Z Is the current job unstable? False 2024-08-20T21:43:36.5243868Z 2024-08-20T21:43:36.5244186Z Is keep-going label set? False 2024-08-20T21:43:36.5244446Z 2024-08-20T21:43:36.5244551Z Renabled issues? 2024-08-20T21:43:36.5355512Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T21:43:36.5356212Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-20T21:43:36.5362011Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T21:43:36.5362510Z env: 2024-08-20T21:43:36.5362936Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:36.5363271Z JOB_TIMEOUT: 240 2024-08-20T21:43:36.5363546Z ##[endgroup] 2024-08-20T21:43:36.5536645Z ##[group]Run set -x 2024-08-20T21:43:36.5537022Z set -x 2024-08-20T21:43:36.5537300Z  2024-08-20T21:43:36.5537631Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-20T21:43:36.5538156Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-20T21:43:36.5538709Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-20T21:43:36.5539206Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-20T21:43:36.5539590Z else 2024-08-20T21:43:36.5539908Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T21:43:36.5540317Z fi 2024-08-20T21:43:36.5540558Z  2024-08-20T21:43:36.5541007Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-20T21:43:36.5541758Z # TODO: Stop building test binaries as part of the build phase 2024-08-20T21:43:36.5542423Z # Used for GPU_FLAG since that doesn't play nice 2024-08-20T21:43:36.5542994Z # shellcheck disable=SC2086,SC2090 2024-08-20T21:43:36.5543432Z container_name=$(docker run \ 2024-08-20T21:43:36.5543838Z  ${GPU_FLAG:-} \ 2024-08-20T21:43:36.5544194Z  -e BUILD_ENVIRONMENT \ 2024-08-20T21:43:36.5544567Z  -e PR_NUMBER \ 2024-08-20T21:43:36.5544913Z  -e GITHUB_ACTIONS \ 2024-08-20T21:43:36.5545286Z  -e GITHUB_REPOSITORY \ 2024-08-20T21:43:36.5545658Z  -e GITHUB_WORKFLOW \ 2024-08-20T21:43:36.5546026Z  -e GITHUB_JOB \ 2024-08-20T21:43:36.5546371Z  -e GITHUB_RUN_ID \ 2024-08-20T21:43:36.5546736Z  -e GITHUB_RUN_NUMBER \ 2024-08-20T21:43:36.5547112Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-20T21:43:36.5560657Z  -e JOB_ID \ 2024-08-20T21:43:36.5560996Z  -e JOB_NAME \ 2024-08-20T21:43:36.5561322Z  -e BASE_SHA \ 2024-08-20T21:43:36.5561653Z  -e BRANCH \ 2024-08-20T21:43:36.5561951Z  -e SHA1 \ 2024-08-20T21:43:36.5562271Z  -e AWS_DEFAULT_REGION \ 2024-08-20T21:43:36.5562658Z  -e IN_WHEEL_TEST \ 2024-08-20T21:43:36.5563005Z  -e SHARD_NUMBER \ 2024-08-20T21:43:36.5563350Z  -e TEST_CONFIG \ 2024-08-20T21:43:36.5563696Z  -e NUM_TEST_SHARDS \ 2024-08-20T21:43:36.5564054Z  -e REENABLED_ISSUES \ 2024-08-20T21:43:36.5564445Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-20T21:43:36.5564848Z  -e VERBOSE_TEST_LOGS \ 2024-08-20T21:43:36.5565222Z  -e TEST_SHOWLOCALS \ 2024-08-20T21:43:36.5565596Z  -e NO_TEST_TIMEOUT \ 2024-08-20T21:43:36.5565944Z  -e NO_TD \ 2024-08-20T21:43:36.5566250Z  -e TD_DISTRIBUTED \ 2024-08-20T21:43:36.5566607Z  -e PR_LABELS \ 2024-08-20T21:43:36.5566988Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-20T21:43:36.5567425Z  -e SCCACHE_BUCKET \ 2024-08-20T21:43:36.5567790Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-20T21:43:36.5568179Z  -e XLA_CUDA \ 2024-08-20T21:43:36.5568561Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-20T21:43:36.5569037Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-20T21:43:36.5569540Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-20T21:43:36.5570038Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-20T21:43:36.5570477Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-20T21:43:36.5570908Z  -e SCRIBE_GRAPHQL_ACCESS_TOKEN \ 2024-08-20T21:43:36.5571324Z  -e DASHBOARD_TAG \ 2024-08-20T21:43:36.5571871Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-20T21:43:36.5572419Z  --security-opt seccomp=unconfined \ 2024-08-20T21:43:36.5572866Z  --cap-add=SYS_PTRACE \ 2024-08-20T21:43:36.5573225Z  --ipc=host \ 2024-08-20T21:43:36.5573559Z  --shm-size="${SHM_SIZE}" \ 2024-08-20T21:43:36.5574108Z  --tty \ 2024-08-20T21:43:36.5574387Z  --detach \ 2024-08-20T21:43:36.5574836Z  --name="${container_name}" \ 2024-08-20T21:43:36.5575242Z  --user jenkins \ 2024-08-20T21:43:36.5575700Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-20T21:43:36.5576255Z  -w /var/lib/jenkins/workspace \ 2024-08-20T21:43:36.5576674Z  "${DOCKER_IMAGE}" 2024-08-20T21:43:36.5576992Z ) 2024-08-20T21:43:36.5577374Z # Propagate download.pytorch.org IP to container 2024-08-20T21:43:36.5578302Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-20T21:43:36.5579280Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-20T21:43:36.5580182Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-20T21:43:36.5585841Z shell: /usr/bin/bash -e {0} 2024-08-20T21:43:36.5586181Z env: 2024-08-20T21:43:36.5586422Z GIT_DEFAULT_BRANCH: main 2024-08-20T21:43:36.5586819Z BUILD_ENVIRONMENT: linux-jammy-py3.8-gcc11 2024-08-20T21:43:36.5587248Z PR_NUMBER: 133712 2024-08-20T21:43:36.5587555Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-20T21:43:36.5587942Z GITHUB_WORKFLOW: pull 2024-08-20T21:43:36.5588251Z GITHUB_JOB: test 2024-08-20T21:43:36.5588527Z GITHUB_RUN_ID: 10479309237 2024-08-20T21:43:36.5588864Z GITHUB_RUN_NUMBER: 238988 2024-08-20T21:43:36.5589197Z GITHUB_RUN_ATTEMPT: 1 2024-08-20T21:43:36.5589489Z JOB_ID: 29025447188 2024-08-20T21:43:36.5590036Z JOB_NAME: linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-20T21:43:36.5590675Z BRANCH: pull/133712 2024-08-20T21:43:36.5591022Z SHA1: 40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:43:36.5591525Z BASE_SHA: 91f3d614142df02f619e44a68e3d9e0dfeba49ec 2024-08-20T21:43:36.5591979Z TEST_CONFIG: distributed 2024-08-20T21:43:36.5592290Z SHARD_NUMBER: 2 2024-08-20T21:43:36.5592577Z NUM_TEST_SHARDS: 2 2024-08-20T21:43:36.5592874Z REENABLED_ISSUES: 2024-08-20T21:43:36.5593177Z CONTINUE_THROUGH_ERROR: False 2024-08-20T21:43:36.5593541Z VERBOSE_TEST_LOGS: False 2024-08-20T21:43:36.5593878Z TEST_SHOWLOCALS: False 2024-08-20T21:43:36.5594193Z NO_TEST_TIMEOUT: False 2024-08-20T21:43:36.5594504Z NO_TD: False 2024-08-20T21:43:36.5594783Z TD_DISTRIBUTED: False 2024-08-20T21:43:36.5595169Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-20T21:43:36.5595649Z SCCACHE_S3_KEY_PREFIX: pull 2024-08-20T21:43:36.5595989Z SHM_SIZE: 1g 2024-08-20T21:43:36.5596872Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:43:36.5597824Z XLA_CUDA: 2024-08-20T21:43:36.5598295Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-20T21:43:36.5598906Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-20T21:43:36.5599312Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-20T21:43:36.5599700Z DASHBOARD_TAG: 2024-08-20T21:43:36.5600002Z HUGGING_FACE_HUB_TOKEN: 2024-08-20T21:43:36.5600332Z SCRIBE_GRAPHQL_ACCESS_TOKEN: 2024-08-20T21:43:36.5600689Z ##[endgroup] 2024-08-20T21:43:36.5622802Z + [[ distributed == \m\u\l\t\i\g\p\u ]] 2024-08-20T21:43:36.5623480Z + [[ linux-jammy-py3.8-gcc11 == *onnx* ]] 2024-08-20T21:43:36.5623916Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-20T21:43:36.5630478Z +++ nproc --ignore=2 2024-08-20T21:43:36.5737871Z ++ docker run -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=6 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10479309237 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=1g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T21:43:43.5161687Z + container_name=9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T21:43:43.5163269Z + grep download.pytorch.org /etc/hosts 2024-08-20T21:43:43.5165209Z + docker exec -i 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-20T21:43:43.6855826Z + echo DOCKER_CONTAINER_ID=9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T21:43:43.6859361Z ++ echo dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl 2024-08-20T21:43:43.6862074Z + docker exec -t 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 sh -c 'pip install dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-20T21:43:44.5721482Z Processing ./dist/torch-2.5.0a0+git40ec5f6-cp38-cp38-linux_x86_64.whl (from torch==2.5.0a0+git40ec5f6) 2024-08-20T21:43:45.0123102Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.13.1) 2024-08-20T21:43:45.0127243Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (4.12.2) 2024-08-20T21:43:45.0130964Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.8.8) 2024-08-20T21:43:45.0134923Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.1.4) 2024-08-20T21:43:45.0138792Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2024.6.1) 2024-08-20T21:43:45.0144670Z Requirement already satisfied: sympy==1.12.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.12.1) 2024-08-20T21:43:45.0164022Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy==1.12.1->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.3.0) 2024-08-20T21:43:45.0178450Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (3.3.0) 2024-08-20T21:43:45.0202229Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (1.21.2) 2024-08-20T21:43:45.0597610Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch==2.5.0a0+git40ec5f6->torch==2.5.0a0+git40ec5f6) (2.1.5) 2024-08-20T21:43:45.5804796Z Installing collected packages: torch 2024-08-20T21:43:53.1961246Z Successfully installed torch-2.5.0a0+git40ec5f6 2024-08-20T21:43:53.2579338Z + export TERM=vt100 2024-08-20T21:43:53.2579824Z + TERM=vt100 2024-08-20T21:43:53.2581303Z ++ dirname .ci/pytorch/test.sh 2024-08-20T21:43:53.2596805Z + source .ci/pytorch/common.sh 2024-08-20T21:43:53.2603278Z +++ dirname .ci/pytorch/common.sh 2024-08-20T21:43:53.2610041Z ++ source .ci/pytorch/common_utils.sh 2024-08-20T21:43:53.2615456Z +++ declare -f -t trap_add 2024-08-20T21:43:53.2619026Z ++ set -ex 2024-08-20T21:43:53.2619783Z ++ [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-20T21:43:53.2620482Z ++ BUILD_TEST_LIBTORCH=0 2024-08-20T21:43:53.2620914Z + [[ linux-jammy-py3.8-gcc11 != *rocm* ]] 2024-08-20T21:43:53.2622587Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-20T21:43:53.2643595Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-20T21:43:53.2644244Z + trap_add cleanup_workspace EXIT 2024-08-20T21:43:53.2644896Z + trap_add_cmd=cleanup_workspace 2024-08-20T21:43:53.2645249Z + shift 2024-08-20T21:43:53.2645506Z + for trap_add_name in "$@" 2024-08-20T21:43:53.2650241Z +++ trap -p EXIT 2024-08-20T21:43:53.2653118Z ++ eval 'extract_trap_cmd ' 2024-08-20T21:43:53.2653750Z +++ extract_trap_cmd 2024-08-20T21:43:53.2654119Z +++ printf '%s\n' '' 2024-08-20T21:43:53.2654487Z ++ printf '%s\n' cleanup_workspace 2024-08-20T21:43:53.2655422Z + trap -- ' 2024-08-20T21:43:53.2655823Z cleanup_workspace' EXIT 2024-08-20T21:43:53.2656280Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-20T21:43:53.8118310Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-20T21:43:53.8376862Z + echo 'Environment variables:' 2024-08-20T21:43:53.8377283Z Environment variables: 2024-08-20T21:43:53.8397550Z + env 2024-08-20T21:43:53.8398110Z INSTALLED_DB=yes 2024-08-20T21:43:53.8399408Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:43:53.8400392Z CONTINUE_THROUGH_ERROR=False 2024-08-20T21:43:53.8401190Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11 2024-08-20T21:43:53.8401854Z HOSTNAME=9645b5fa39b5 2024-08-20T21:43:53.8403215Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_a70d2a65-ffd4-4bec-8fc2-9ddfb2d9b21b 2024-08-20T21:43:53.8404719Z GITHUB_ACTION=__self 2024-08-20T21:43:53.8405288Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T21:43:53.8406012Z GITHUB_RUN_NUMBER=238988 2024-08-20T21:43:53.8406433Z TEST_CONFIG=distributed 2024-08-20T21:43:53.8406758Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T21:43:53.8407269Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T21:43:53.8407843Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T21:43:53.8408325Z GITHUB_TRIGGERING_ACTOR=pytorchmergebot 2024-08-20T21:43:53.8409093Z GITHUB_REF_TYPE=branch 2024-08-20T21:43:53.8409510Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T21:43:53.8410112Z BASE_SHA=91f3d614142df02f619e44a68e3d9e0dfeba49ec 2024-08-20T21:43:53.8410811Z XLA_CUDA= 2024-08-20T21:43:53.8411228Z HUGGING_FACE_HUB_TOKEN= 2024-08-20T21:43:53.8415369Z *** 2024-08-20T21:43:53.8415943Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T21:43:53.8416558Z GITHUB_ACTIONS=true 2024-08-20T21:43:53.8417151Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:43:53.8418022Z GITHUB_SHA=f2fb9405c2fa9f9502a76363091cce6fd8179736 2024-08-20T21:43:53.8419253Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/133712/merge 2024-08-20T21:43:53.8420385Z UCC_HOME=/usr 2024-08-20T21:43:53.8420865Z VERBOSE_TEST_LOGS=False 2024-08-20T21:43:53.8421450Z GITHUB_REF=refs/pull/133712/merge 2024-08-20T21:43:53.8422117Z SHARD_NUMBER=2 2024-08-20T21:43:53.8422617Z GITHUB_REF_PROTECTED=false 2024-08-20T21:43:53.8423212Z HOME=/var/lib/jenkins 2024-08-20T21:43:53.8423828Z GITHUB_API_URL=https://api.github.com 2024-08-20T21:43:53.8424585Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T21:43:53.8425253Z UCX_COMMIT= 2024-08-20T21:43:53.8425740Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-20T21:43:53.8426311Z NUM_TEST_SHARDS=2 2024-08-20T21:43:53.8426801Z UCX_HOME=/usr 2024-08-20T21:43:53.8428444Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_a70d2a65-ffd4-4bec-8fc2-9ddfb2d9b21b 2024-08-20T21:43:53.8430579Z JOB_NAME=linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-20T21:43:53.8432705Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_a70d2a65-ffd4-4bec-8fc2-9ddfb2d9b21b 2024-08-20T21:43:53.8434837Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T21:43:53.8436311Z GITHUB_EVENT_NAME=pull_request 2024-08-20T21:43:53.8436922Z DASHBOARD_TAG= 2024-08-20T21:43:53.8437556Z GITHUB_RUN_ID=10479309237 2024-08-20T21:43:53.8439338Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_a70d2a65-ffd4-4bec-8fc2-9ddfb2d9b21b 2024-08-20T21:43:53.8440957Z GITHUB_ACTOR=pytorchmergebot 2024-08-20T21:43:53.8441579Z PR_NUMBER=133712 2024-08-20T21:43:53.8442073Z DESIRED_CUDA= 2024-08-20T21:43:53.8442543Z GITHUB_RUN_ATTEMPT=1 2024-08-20T21:43:53.8443093Z ANACONDA_PYTHON_VERSION=3.8 2024-08-20T21:43:53.8443815Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T21:43:53.8444580Z TERM=vt100 2024-08-20T21:43:53.8445049Z INSTALLED_VISION=yes 2024-08-20T21:43:53.8445581Z BRANCH=pull/133712 2024-08-20T21:43:53.8446106Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T21:43:53.8446737Z CUDA_PATH=/usr/local/cuda 2024-08-20T21:43:53.8448173Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T21:43:53.8449568Z GITHUB_SERVER_URL=https://github.com 2024-08-20T21:43:53.8450245Z UCC_COMMIT= 2024-08-20T21:43:53.8450692Z REENABLED_ISSUES= 2024-08-20T21:43:53.8451144Z DOCS=yes 2024-08-20T21:43:53.8451563Z INSTALLED_ANDROID= 2024-08-20T21:43:53.8452140Z SHLVL=1 2024-08-20T21:43:53.8452564Z MAX_JOBS=6 2024-08-20T21:43:53.8453030Z GITHUB_ACTOR_ID=97764156 2024-08-20T21:43:53.8453793Z GITHUB_WORKFLOW_SHA=f2fb9405c2fa9f9502a76363091cce6fd8179736 2024-08-20T21:43:53.8454695Z GITHUB_REF_NAME=133712/merge 2024-08-20T21:43:53.8455844Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T21:43:53.8456818Z GITHUB_JOB=test 2024-08-20T21:43:53.8457289Z NO_TEST_TIMEOUT=False 2024-08-20T21:43:53.8462463Z TD_DISTRIBUTED=False 2024-08-20T21:43:53.8463109Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T21:43:53.8463776Z GITHUB_RETENTION_DAYS=90 2024-08-20T21:43:53.8464355Z OPENSSL_DIR=/opt/openssl 2024-08-20T21:43:53.8464902Z GITHUB_ACTION_REPOSITORY= 2024-08-20T21:43:53.8466837Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:43:53.8468899Z GITHUB_BASE_REF=gh/XuehaiPan/146/base 2024-08-20T21:43:53.8469588Z INSTALLED_ACL= 2024-08-20T21:43:53.8470027Z CI=true 2024-08-20T21:43:53.8470509Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T21:43:53.8471116Z JOB_ID=29025447188 2024-08-20T21:43:53.8471619Z INSTALLED_PROTOBUF=yes 2024-08-20T21:43:53.8472202Z GITHUB_HEAD_REF=gh/XuehaiPan/146/head 2024-08-20T21:43:53.8472901Z GITHUB_ACTION_REF= 2024-08-20T21:43:53.8473703Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T21:43:53.8474491Z TEST_SHOWLOCALS=False 2024-08-20T21:43:53.8475017Z GITHUB_WORKFLOW=pull 2024-08-20T21:43:53.8475573Z DEBIAN_FRONTEND=noninteractive 2024-08-20T21:43:53.8477408Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_a70d2a65-ffd4-4bec-8fc2-9ddfb2d9b21b 2024-08-20T21:43:53.8478966Z NO_TD=False 2024-08-20T21:43:53.8479461Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T21:43:53.8480070Z _=/usr/bin/env 2024-08-20T21:43:53.8480933Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-20T21:43:53.8657114Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch 2024-08-20T21:43:53.8658538Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-08-20T21:43:53.8659414Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib 2024-08-20T21:43:53.8660434Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/test 2024-08-20T21:43:53.8661026Z + BUILD_DIR=build 2024-08-20T21:43:53.8661327Z + BUILD_RENAMED_DIR=build_renamed 2024-08-20T21:43:53.8661683Z + BUILD_BIN_DIR=build/bin 2024-08-20T21:43:53.8661998Z + SHARD_NUMBER=2 2024-08-20T21:43:53.8662277Z + NUM_TEST_SHARDS=2 2024-08-20T21:43:53.8662558Z + export VALGRIND=ON 2024-08-20T21:43:53.8662853Z + VALGRIND=ON 2024-08-20T21:43:53.8663227Z + [[ linux-jammy-py3.8-gcc11 == *clang9* ]] 2024-08-20T21:43:53.8663818Z + [[ 0 == \1 ]] 2024-08-20T21:43:53.8664089Z + [[ False == \1 ]] 2024-08-20T21:43:53.8664582Z + [[ linux-jammy-py3.8-gcc11 != *bazel* ]] 2024-08-20T21:43:53.8665020Z ++ realpath build/custom_test_artifacts 2024-08-20T21:43:53.8677807Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-20T21:43:53.8678593Z + [[ -n '' ]] 2024-08-20T21:43:53.8678917Z + echo 'Environment variables' 2024-08-20T21:43:53.8679282Z Environment variables 2024-08-20T21:43:53.8679576Z + env 2024-08-20T21:43:53.8684841Z INSTALLED_DB=yes 2024-08-20T21:43:53.8686559Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T21:43:53.8687685Z CONTINUE_THROUGH_ERROR=False 2024-08-20T21:43:53.8691419Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11 2024-08-20T21:43:53.8692068Z HOSTNAME=9645b5fa39b5 2024-08-20T21:43:53.8692958Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_a70d2a65-ffd4-4bec-8fc2-9ddfb2d9b21b 2024-08-20T21:43:53.8693791Z GITHUB_ACTION=__self 2024-08-20T21:43:53.8694119Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-20T21:43:53.8694505Z GITHUB_RUN_NUMBER=238988 2024-08-20T21:43:53.8694815Z TEST_CONFIG=distributed 2024-08-20T21:43:53.8695153Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-20T21:43:53.8695614Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-20T21:43:53.8696013Z SCRIBE_GRAPHQL_ACCESS_TOKEN= 2024-08-20T21:43:53.8696386Z GITHUB_TRIGGERING_ACTOR=pytorchmergebot 2024-08-20T21:43:53.8696785Z GITHUB_REF_TYPE=branch 2024-08-20T21:43:53.8697091Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-20T21:43:53.8697489Z BASE_SHA=91f3d614142df02f619e44a68e3d9e0dfeba49ec 2024-08-20T21:43:53.8697914Z XLA_CUDA= 2024-08-20T21:43:53.8698159Z HUGGING_FACE_HUB_TOKEN= 2024-08-20T21:43:53.8698566Z *** 2024-08-20T21:43:53.8698818Z GITHUB_REPOSITORY_ID=65600975 2024-08-20T21:43:53.8699163Z GITHUB_ACTIONS=true 2024-08-20T21:43:53.8699496Z SHA1=40ec5f6ddd9787aca0449b24128343ff4c4a88b3 2024-08-20T21:43:53.8699992Z GITHUB_SHA=f2fb9405c2fa9f9502a76363091cce6fd8179736 2024-08-20T21:43:53.8700712Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/133712/merge 2024-08-20T21:43:53.8701323Z UCC_HOME=/usr 2024-08-20T21:43:53.8701595Z VERBOSE_TEST_LOGS=False 2024-08-20T21:43:53.8701923Z GITHUB_REF=refs/pull/133712/merge 2024-08-20T21:43:53.8702265Z SHARD_NUMBER=2 2024-08-20T21:43:53.8702548Z GITHUB_REF_PROTECTED=false 2024-08-20T21:43:53.8702873Z HOME=/var/lib/jenkins 2024-08-20T21:43:53.8703244Z GITHUB_API_URL=https://api.github.com 2024-08-20T21:43:53.8703653Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-20T21:43:53.8704019Z UCX_COMMIT= 2024-08-20T21:43:53.8704275Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-20T21:43:53.8704605Z NUM_TEST_SHARDS=2 2024-08-20T21:43:53.8704884Z UCX_HOME=/usr 2024-08-20T21:43:53.8705733Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_a70d2a65-ffd4-4bec-8fc2-9ddfb2d9b21b 2024-08-20T21:43:53.8706904Z JOB_NAME=linux-jammy-py3.8-gcc11 / test (distributed, 2, 2, amz2023.linux.2xlarge) 2024-08-20T21:43:53.8708086Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_a70d2a65-ffd4-4bec-8fc2-9ddfb2d9b21b 2024-08-20T21:43:53.8709248Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-20T21:43:53.8709897Z GITHUB_EVENT_NAME=pull_request 2024-08-20T21:43:53.8710238Z DASHBOARD_TAG= 2024-08-20T21:43:53.8710513Z GITHUB_RUN_ID=10479309237 2024-08-20T21:43:53.8711449Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_a70d2a65-ffd4-4bec-8fc2-9ddfb2d9b21b 2024-08-20T21:43:53.8712346Z GITHUB_ACTOR=pytorchmergebot 2024-08-20T21:43:53.8712688Z PR_NUMBER=133712 2024-08-20T21:43:53.8712952Z DESIRED_CUDA= 2024-08-20T21:43:53.8713221Z GITHUB_RUN_ATTEMPT=1 2024-08-20T21:43:53.8713514Z VALGRIND=ON 2024-08-20T21:43:53.8713774Z ANACONDA_PYTHON_VERSION=3.8 2024-08-20T21:43:53.8714188Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-20T21:43:53.8714764Z TERM=vt100 2024-08-20T21:43:53.8715011Z INSTALLED_VISION=yes 2024-08-20T21:43:53.8715312Z BRANCH=pull/133712 2024-08-20T21:43:53.8715691Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-20T21:43:53.8716032Z CUDA_PATH=/usr/local/cuda 2024-08-20T21:43:53.8716796Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-20T21:43:53.8717553Z GITHUB_SERVER_URL=https://github.com 2024-08-20T21:43:53.8717921Z UCC_COMMIT= 2024-08-20T21:43:53.8718181Z REENABLED_ISSUES= 2024-08-20T21:43:53.8718454Z DOCS=yes 2024-08-20T21:43:53.8718688Z INSTALLED_ANDROID= 2024-08-20T21:43:53.8718961Z SHLVL=1 2024-08-20T21:43:53.8719196Z MAX_JOBS=6 2024-08-20T21:43:53.8719443Z GITHUB_ACTOR_ID=97764156 2024-08-20T21:43:53.8719881Z GITHUB_WORKFLOW_SHA=f2fb9405c2fa9f9502a76363091cce6fd8179736 2024-08-20T21:43:53.8720389Z GITHUB_REF_NAME=133712/merge 2024-08-20T21:43:53.8720975Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-20T21:43:53.8721537Z GITHUB_JOB=test 2024-08-20T21:43:53.8721819Z NO_TEST_TIMEOUT=False 2024-08-20T21:43:53.8722110Z TD_DISTRIBUTED=False 2024-08-20T21:43:53.8722440Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-20T21:43:53.8722815Z GITHUB_RETENTION_DAYS=90 2024-08-20T21:43:53.8723126Z OPENSSL_DIR=/opt/openssl 2024-08-20T21:43:53.8723451Z GITHUB_ACTION_REPOSITORY= 2024-08-20T21:43:53.8724502Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:43:53.8725605Z GITHUB_BASE_REF=gh/XuehaiPan/146/base 2024-08-20T21:43:53.8725987Z INSTALLED_ACL= 2024-08-20T21:43:53.8726248Z CI=true 2024-08-20T21:43:53.8726503Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-20T21:43:53.8726852Z JOB_ID=29025447188 2024-08-20T21:43:53.8727138Z INSTALLED_PROTOBUF=yes 2024-08-20T21:43:53.8727458Z GITHUB_HEAD_REF=gh/XuehaiPan/146/head 2024-08-20T21:43:53.8727842Z GITHUB_ACTION_REF= 2024-08-20T21:43:53.8728268Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-20T21:43:53.8728712Z TEST_SHOWLOCALS=False 2024-08-20T21:43:53.8729110Z GITHUB_WORKFLOW=pull 2024-08-20T21:43:53.8729584Z DEBIAN_FRONTEND=noninteractive 2024-08-20T21:43:53.8731040Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_a70d2a65-ffd4-4bec-8fc2-9ddfb2d9b21b 2024-08-20T21:43:53.8732532Z NO_TD=False 2024-08-20T21:43:53.8733019Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-20T21:43:53.8733582Z _=/usr/bin/env 2024-08-20T21:43:53.8734026Z + echo 'Testing pytorch' 2024-08-20T21:43:53.8734410Z Testing pytorch 2024-08-20T21:43:53.8734705Z + export LANG=C.UTF-8 2024-08-20T21:43:53.8735018Z + LANG=C.UTF-8 2024-08-20T21:43:53.8735306Z + PR_NUMBER=133712 2024-08-20T21:43:53.8735610Z + [[ distributed == \d\e\f\a\u\l\t ]] 2024-08-20T21:43:53.8736043Z + [[ distributed == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-20T21:43:53.8736556Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-20T21:43:53.8736964Z + [[ distributed == \s\l\o\w ]] 2024-08-20T21:43:53.8737456Z + [[ linux-jammy-py3.8-gcc11 == *slow-gradcheck* ]] 2024-08-20T21:43:53.8738001Z + [[ linux-jammy-py3.8-gcc11 == *cuda* ]] 2024-08-20T21:43:53.8738477Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-20T21:43:53.8738963Z + [[ linux-jammy-py3.8-gcc11 == *xpu* ]] 2024-08-20T21:43:53.8739384Z + [[ distributed == *crossref* ]] 2024-08-20T21:43:53.8739819Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-20T21:43:53.8740301Z + [[ linux-jammy-py3.8-gcc11 == *xpu* ]] 2024-08-20T21:43:53.8740795Z + [[ linux-jammy-py3.8-gcc11 != *-bazel-* ]] 2024-08-20T21:43:53.8741268Z + pip_install --user ninja==1.10.2 2024-08-20T21:43:53.8741785Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-20T21:43:54.2791671Z Collecting ninja==1.10.2 2024-08-20T21:43:54.2889640Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-20T21:43:54.2983267Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-20T21:43:54.7230808Z Installing collected packages: ninja 2024-08-20T21:43:54.7316608Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-20T21:43:54.7317948Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-20T21:43:54.7362983Z Successfully installed ninja-1.10.2 2024-08-20T21:43:54.7964178Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:43:54.7966395Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:43:54.7967867Z + [[ linux-jammy-py3.8-gcc11 == *aarch64* ]] 2024-08-20T21:43:54.7968282Z + install_tlparse 2024-08-20T21:43:54.7968656Z + pip_install --user tlparse==0.3.25 2024-08-20T21:43:54.7969221Z + pip install --progress-bar off --user tlparse==0.3.25 2024-08-20T21:43:55.2107368Z Collecting tlparse==0.3.25 2024-08-20T21:43:55.2204329Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB) 2024-08-20T21:43:55.2367907Z Downloading tlparse-0.3.25-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-20T21:43:55.6783786Z Installing collected packages: tlparse 2024-08-20T21:43:55.7414548Z Successfully installed tlparse-0.3.25 2024-08-20T21:43:55.8010200Z ++ python -m site --user-base 2024-08-20T21:43:55.8309703Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-20T21:43:55.8311801Z + [[ linux-jammy-py3.8-gcc11 == *asan* ]] 2024-08-20T21:43:55.8312429Z + [[ linux-jammy-py3.8-gcc11 == *-debug* ]] 2024-08-20T21:43:55.8313224Z + [[ linux-jammy-py3.8-gcc11 != *-bazel-* ]] 2024-08-20T21:43:55.8314420Z + echo 'We are not in debug mode: linux-jammy-py3.8-gcc11. Expect the assertion to pass' 2024-08-20T21:43:55.8315467Z We are not in debug mode: linux-jammy-py3.8-gcc11. Expect the assertion to pass 2024-08-20T21:43:55.8316252Z + cd test 2024-08-20T21:43:55.8316814Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-20T21:43:59.2723514Z + [[ distributed == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-20T21:43:59.2724054Z + [[ distributed == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-20T21:43:59.2725372Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-20T21:43:59.2726655Z + [[ distributed == *pr_time_benchmarks* ]] 2024-08-20T21:43:59.2727521Z + [[ distributed == *dynamo_eager* ]] 2024-08-20T21:43:59.2727936Z + [[ distributed == *aot_eager* ]] 2024-08-20T21:43:59.2728332Z + [[ distributed == *aot_inductor* ]] 2024-08-20T21:43:59.2728738Z + [[ distributed == *inductor* ]] 2024-08-20T21:43:59.2729110Z + [[ distributed == *dynamic* ]] 2024-08-20T21:43:59.2729479Z + [[ distributed == *cpu* ]] 2024-08-20T21:43:59.2730093Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-20T21:43:59.2750475Z + [[ linux-jammy-py3.8-gcc11 == *libtorch* ]] 2024-08-20T21:43:59.2751070Z + [[ linux-jammy-py3.8-gcc11 == *-bazel-* ]] 2024-08-20T21:43:59.2753003Z + cd test 2024-08-20T21:43:59.2753509Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-20T21:44:00.5713933Z PyTorch built with: 2024-08-20T21:44:00.5714532Z - GCC 11.4 2024-08-20T21:44:00.5714837Z - C++ Version: 201703 2024-08-20T21:44:00.5715731Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T21:44:00.5716833Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T21:44:00.5717495Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T21:44:00.5717998Z - LAPACK is enabled (usually provided by MKL) 2024-08-20T21:44:00.5718461Z - NNPACK is enabled 2024-08-20T21:44:00.5718814Z - CPU capability usage: AVX512 2024-08-20T21:44:00.5726866Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-20T21:44:00.5733902Z 2024-08-20T21:44:00.7976117Z + cd test 2024-08-20T21:44:00.7976877Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-20T21:44:01.9755845Z ATen/Parallel: 2024-08-20T21:44:01.9756492Z at::get_num_threads() : 4 2024-08-20T21:44:01.9757148Z at::get_num_interop_threads() : 4 2024-08-20T21:44:01.9758042Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-20T21:44:01.9758582Z omp_get_max_threads() : 4 2024-08-20T21:44:01.9759608Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-20T21:44:01.9760426Z mkl_get_max_threads() : 4 2024-08-20T21:44:01.9761047Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-20T21:44:01.9761673Z std::thread::hardware_concurrency() : 8 2024-08-20T21:44:01.9762087Z Environment variables: 2024-08-20T21:44:01.9762434Z OMP_NUM_THREADS : [not set] 2024-08-20T21:44:01.9762775Z MKL_NUM_THREADS : [not set] 2024-08-20T21:44:01.9763135Z ATen parallel backend: OpenMP 2024-08-20T21:44:01.9763390Z 2024-08-20T21:44:02.1665544Z + [[ linux-jammy-py3.8-gcc11 == *aarch64* ]] 2024-08-20T21:44:02.1666411Z + [[ distributed == *backward* ]] 2024-08-20T21:44:02.1667117Z + [[ distributed == *xla* ]] 2024-08-20T21:44:02.1667647Z + [[ distributed == *executorch* ]] 2024-08-20T21:44:02.1668058Z + [[ distributed == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-20T21:44:02.1668667Z + [[ linux-jammy-py3.8-gcc11 == *libtorch* ]] 2024-08-20T21:44:02.1669175Z + [[ distributed == distributed ]] 2024-08-20T21:44:02.1669535Z + test_distributed 2024-08-20T21:44:02.1669982Z + echo 'Testing distributed python tests' 2024-08-20T21:44:02.1670433Z Testing distributed python tests 2024-08-20T21:44:02.1671065Z + python test/run_test.py --distributed-tests --shard 2 2 --verbose 2024-08-20T21:44:02.2879639Z test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-20T21:44:02.2880613Z import pkg_resources 2024-08-20T21:44:05.9539756Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-20T21:44:06.0320512Z Ignoring disabled issues: [''] 2024-08-20T21:44:06.0440169Z Found test times from artifacts 2024-08-20T21:44:06.0867348Z Found test times from artifacts 2024-08-20T21:44:06.0882727Z Running all tests 2024-08-20T21:44:06.1009937Z Running parallel tests on 3 processes 2024-08-20T21:44:06.1016402Z Name: tests to run (est. time: 59.31min) 2024-08-20T21:44:06.1017210Z Serial tests (103): 2024-08-20T21:44:06.1017887Z distributed/test_dynamo_distributed 1/1 2024-08-20T21:44:06.1018401Z distributed/_tools/test_mod_tracker 1/1 2024-08-20T21:44:06.1018879Z distributed/checkpoint/test_nested_dict 1/1 2024-08-20T21:44:06.1019577Z distributed/test_fake_pg 1/1 2024-08-20T21:44:06.1020145Z distributed/pipelining/test_backward 1/1 2024-08-20T21:44:06.1020640Z distributed/fsdp/test_checkpoint_wrapper 1/1 2024-08-20T21:44:06.1021103Z distributed/test_symmetric_memory 1/1 2024-08-20T21:44:06.1021586Z distributed/_shard/sharded_tensor/test_logger 1/1 2024-08-20T21:44:06.1022075Z distributed/_tensor/test_dtensor 1/1 2024-08-20T21:44:06.1022579Z distributed/_composable/fsdp/test_fully_shard_compile 1/1 2024-08-20T21:44:06.1023169Z distributed/pipelining/test_schedule_multiproc 1/1 2024-08-20T21:44:06.1023692Z distributed/_tensor/test_xla_integration 1/1 2024-08-20T21:44:06.1024163Z distributed/pipelining/test_unflatten 1/1 2024-08-20T21:44:06.1024644Z distributed/_tensor/test_convolution_ops 1/1 2024-08-20T21:44:06.1025091Z distributed/test_c10d_gloo 1/1 2024-08-20T21:44:06.1025499Z distributed/test_compute_comm_reordering 1/1 2024-08-20T21:44:06.1025990Z distributed/checkpoint/e2e/test_fsdp_ep 1/1 2024-08-20T21:44:06.1026458Z distributed/_tools/test_mem_tracker 1/1 2024-08-20T21:44:06.1026979Z distributed/_composable/fsdp/test_fully_shard_training 1/1 2024-08-20T21:44:06.1027543Z distributed/checkpoint/test_save_load_api 1/1 2024-08-20T21:44:06.1028042Z distributed/checkpoint/test_dedup_tensors 1/1 2024-08-20T21:44:06.1028523Z distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 2024-08-20T21:44:06.1028992Z distributed/test_c10d_spawn_gloo 1/1 2024-08-20T21:44:06.1029422Z distributed/_tensor/test_attention 1/1 2024-08-20T21:44:06.1029854Z distributed/pipelining/test_stage 1/1 2024-08-20T21:44:06.1030354Z distributed/_tensor/debug/test_comm_mode_features 1/1 2024-08-20T21:44:06.1030914Z distributed/_tensor/experimental/test_local_map 1/1 2024-08-20T21:44:06.1031419Z distributed/_tensor/test_pointwise_ops 1/1 2024-08-20T21:44:06.1031937Z distributed/tensor/parallel/test_tp_random_state 1/1 2024-08-20T21:44:06.1032509Z distributed/checkpoint/e2e/test_e2e_save_and_load 1/1 2024-08-20T21:44:06.1033029Z distributed/_composable/test_checkpoint 1/1 2024-08-20T21:44:06.1033482Z distributed/_tensor/test_init 1/1 2024-08-20T21:44:06.1033923Z distributed/_tools/test_fsdp2_mem_tracker 1/1 2024-08-20T21:44:06.1034426Z distributed/checkpoint/test_fsdp_optim_state 1/1 2024-08-20T21:44:06.1034941Z distributed/checkpoint/test_format_utils 1/1 2024-08-20T21:44:06.1035415Z distributed/checkpoint/test_fsspec 1/1 2024-08-20T21:44:06.1035875Z distributed/checkpoint/test_traverse 1/1 2024-08-20T21:44:06.1036372Z distributed/checkpoint/test_dtensor_resharding 1/1 2024-08-20T21:44:06.1036902Z distributed/checkpoint/test_tp_checkpoint 1/1 2024-08-20T21:44:06.1037414Z distributed/checkpoint/e2e/test_fine_tuning 1/1 2024-08-20T21:44:06.1037905Z distributed/_tools/test_memory_tracker 1/1 2024-08-20T21:44:06.1038426Z distributed/_composable/fsdp/test_fully_shard_init 1/1 2024-08-20T21:44:06.1038957Z distributed/checkpoint/test_planner 1/1 2024-08-20T21:44:06.1039414Z distributed/elastic/test_control_plane 1/1 2024-08-20T21:44:06.1039927Z distributed/checkpoint/test_dtensor_checkpoint 1/1 2024-08-20T21:44:06.1040508Z distributed/_composable/fsdp/test_fully_shard_logging 1/1 2024-08-20T21:44:06.1041101Z distributed/_composable/fsdp/test_fully_shard_memory 1/1 2024-08-20T21:44:06.1041632Z distributed/_tensor/test_optimizers 1/1 2024-08-20T21:44:06.1042076Z distributed/test_functional_api 1/1 2024-08-20T21:44:06.1042506Z distributed/test_control_collectives 1/1 2024-08-20T21:44:06.1043072Z distributed/checkpoint/test_fsdp_tp_checkpoint_conversion 1/1 2024-08-20T21:44:06.1043641Z distributed/_tensor/test_op_strategy 1/1 2024-08-20T21:44:06.1044117Z distributed/checkpoint/test_hsdp_checkpoint 1/1 2024-08-20T21:44:06.1044750Z distributed/_composable/test_composability/test_pp_composability 1/1 2024-08-20T21:44:06.1045379Z distributed/checkpoint/test_compatibility 1/1 2024-08-20T21:44:06.1045939Z distributed/_tensor/test_view_ops 1/1 2024-08-20T21:44:06.1046574Z distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_ 1/1 2024-08-20T21:44:06.1047212Z distributed/tensor/parallel/test_parallelize_api 1/1 2024-08-20T21:44:06.1047725Z distributed/fsdp/test_fsdp_optim_state 1/1 2024-08-20T21:44:06.1048197Z distributed/_composable/test_compose 1/1 2024-08-20T21:44:06.1048664Z distributed/fsdp/test_fsdp_pure_fp16 1/1 2024-08-20T21:44:06.1049132Z distributed/nn/jit/test_instantiator 1/1 2024-08-20T21:44:06.1049639Z distributed/_composable/fsdp/test_fully_shard_comm 1/1 2024-08-20T21:44:06.1050244Z distributed/_composable/fsdp/test_fully_shard_state_dict 1/1 2024-08-20T21:44:06.1050817Z distributed/fsdp/test_fsdp_ignored_modules 1/1 2024-08-20T21:44:06.1051287Z distributed/fsdp/test_fsdp_overlap 1/1 2024-08-20T21:44:06.1051875Z distributed/fsdp/test_fsdp_backward_prefetch 1/1 2024-08-20T21:44:06.1052373Z distributed/_shard/test_sharder 1/1 2024-08-20T21:44:06.1052800Z distributed/test_distributed_spawn 1/3 2024-08-20T21:44:06.1053251Z distributed/test_distributed_spawn 3/3 2024-08-20T21:44:06.1053679Z distributed/fsdp/test_wrap 1/1 2024-08-20T21:44:06.1054064Z distributed/fsdp/test_fsdp_misc 1/1 2024-08-20T21:44:06.1054512Z distributed/fsdp/test_fsdp_comm_hooks 1/1 2024-08-20T21:44:06.1055001Z distributed/fsdp/test_fsdp_unshard_params 1/1 2024-08-20T21:44:06.1055452Z distributed/test_c10d_common 1/1 2024-08-20T21:44:06.1055901Z distributed/fsdp/test_fsdp_clip_grad_norm 1/1 2024-08-20T21:44:06.1056370Z distributed/fsdp/test_fsdp_meta 1/1 2024-08-20T21:44:06.1056788Z distributed/fsdp/test_fsdp_uneven 1/1 2024-08-20T21:44:06.1057264Z distributed/rpc/cuda/test_tensorpipe_agent 1/1 2024-08-20T21:44:06.1057735Z distributed/test_c10d_nccl 1/1 2024-08-20T21:44:06.1058317Z distributed/rpc/test_tensorpipe_agent 2/3 2024-08-20T21:44:06.1058870Z distributed/_shard/sharded_tensor/ops/test_binary_cmp 1/1 2024-08-20T21:44:06.1059399Z distributed/rpc/test_faulty_agent 1/1 2024-08-20T21:44:06.1059860Z distributed/fsdp/test_fsdp_multiple_forward 1/1 2024-08-20T21:44:06.1060338Z distributed/fsdp/test_fsdp_comm 1/1 2024-08-20T21:44:06.1060822Z distributed/_shard/sharded_tensor/ops/test_init 1/1 2024-08-20T21:44:06.1061387Z distributed/_shard/sharded_tensor/test_sharded_tensor 1/1 2024-08-20T21:44:06.1062028Z distributed/_shard/sharded_tensor/test_sharded_tensor_reshard 1/1 2024-08-20T21:44:06.1062694Z distributed/_shard/sharded_tensor/ops/test_embedding 1/1 2024-08-20T21:44:06.1063310Z distributed/_shard/sharded_tensor/ops/test_embedding_bag 1/1 2024-08-20T21:44:06.1063822Z distributed/test_launcher 1/1 2024-08-20T21:44:06.1064197Z distributed/test_store 1/1 2024-08-20T21:44:06.1064567Z distributed/test_c10d_spawn_ucc 1/1 2024-08-20T21:44:06.1064976Z distributed/test_c10d_spawn_nccl 1/1 2024-08-20T21:44:06.1065407Z distributed/test_c10d_ops_nccl 1/1 2024-08-20T21:44:06.1065837Z distributed/elastic/metrics/api_test 1/1 2024-08-20T21:44:06.1066317Z distributed/elastic/utils/distributed_test 1/1 2024-08-20T21:44:06.1066802Z distributed/elastic/events/lib_test 1/1 2024-08-20T21:44:06.1067303Z distributed/elastic/multiprocessing/api_test 1/1 2024-08-20T21:44:06.1067829Z distributed/elastic/timer/local_timer_example 1/1 2024-08-20T21:44:06.1068352Z distributed/elastic/timer/local_timer_test 1/1 2024-08-20T21:44:06.1068835Z distributed/elastic/utils/util_test 1/1 2024-08-20T21:44:06.1069287Z distributed/elastic/utils/logging_test 1/1 2024-08-20T21:44:06.1069719Z Parallel tests (0): 2024-08-20T21:44:06.1070046Z Name: excluded (est. time: 0.0min) 2024-08-20T21:44:06.1070401Z Serial tests (0): 2024-08-20T21:44:06.1070808Z Parallel tests (0): 2024-08-20T21:44:06.1245983Z Running distributed/test_dynamo_distributed 1/1 ... [2024-08-20 21:44:06.124171] 2024-08-20T21:44:06.1247474Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:44:06.1253338Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_dynamo_distributed.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:44:06.124771] 2024-08-20T21:44:35.9954574Z 2024-08-20T21:44:35.9956596Z distributed/test_dynamo_distributed 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_dynamo_distributed_1.1_1c913bb5733b9809_.log 2024-08-20T21:44:35.9985244Z Running 48 items in this shard: test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_call_method_forward, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_hf_bert_ddp_aot_eager, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_hf_bert_ddp_inductor, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_issue90375, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_symbol_splitting, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_unbacked_symbol_splitting_direct, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_unbacked_symbol_splitting_indirect, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_unbacked_symbol_splitting_no_binding, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_unbacked_symbol_splitting_torture_multi, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_asymmetric_compilation, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_asymmetric_compilation_with_fx_cache, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_compiler_collectives_automatic_dynamic_scalar, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_compiler_collectives_automatic_dynamic_speculation_divergence, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_compiler_collectives_automatic_dynamic_tensor, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_compiler_collectives_graph_break_empty_graph_still_collective, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_ddp_activation_checkpointing, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_ddp_baseline_aot_eager_multiprocess, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_activation_checkpointing, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_aot_eager, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_inductor, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_setattr, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_aot_eager, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_aot_eager_static_graph, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_inductor, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_inductor_static_graph, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_fsdp, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_fsdp_activation_checkpointing, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_aot_autograd, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_async_subclass_no_specialize, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_custom_layer, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_ddp_baseline_aot_eager, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_ddp_baseline_inductor, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_empty_graph_inductor, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_dup_tensors_diff_source, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_dup_tensors_same_source, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_orig_params_assert, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_skip_guards, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_skip_register_attr_or_module, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_staticmethod, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_ctx_manager, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor_layout_optimizations_inference, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor_layout_optimizations_training, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor_transpose, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_higher_order_op, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_ignored_parameters, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_no_split 2024-08-20T21:44:36.0010112Z 2024-08-20T21:44:36.0010602Z Running distributed/_tools/test_mod_tracker 1/1 ... [2024-08-20 21:44:35.996288] 2024-08-20T21:44:36.0011240Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:44:36.0013103Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tools/test_mod_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:44:35.996745] 2024-08-20T21:44:38.8206189Z 2024-08-20T21:44:38.8207819Z distributed/_tools/test_mod_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_mod_tracker_1.1_3511612dabb84d9d_.log 2024-08-20T21:44:38.8211350Z Running 4 items in this shard: test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_ac, test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_bw_detection, test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_module_hierarchy, test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_user_hooks 2024-08-20T21:44:38.8214044Z 2024-08-20T21:44:38.8214563Z Running distributed/checkpoint/test_nested_dict 1/1 ... [2024-08-20 21:44:38.821148] 2024-08-20T21:44:38.8215219Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:44:38.8218864Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_nested_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:44:38.821608] 2024-08-20T21:44:41.5954778Z 2024-08-20T21:44:41.5956826Z distributed/checkpoint/test_nested_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_nested_dict_1.1_30b816a15fad5443_.log 2024-08-20T21:44:41.5959226Z Running 2 items in this shard: test/distributed/checkpoint/test_nested_dict.py::TestFlattening::test_flattening_round_trip, test/distributed/checkpoint/test_nested_dict.py::TestFlattening::test_mapping 2024-08-20T21:44:41.5960595Z 2024-08-20T21:44:41.5961332Z Running distributed/test_fake_pg 1/1 ... [2024-08-20 21:44:41.595924] 2024-08-20T21:44:41.5961908Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:44:41.5966614Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_fake_pg.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:44:41.596362] 2024-08-20T21:44:44.3702433Z 2024-08-20T21:44:44.3704810Z distributed/test_fake_pg 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_fake_pg_1.1_370d085631d7e091_.log 2024-08-20T21:44:44.3714839Z Running 13 items in this shard: test/distributed/test_fake_pg.py::TestFakePG::test_all_reduce, test/distributed/test_fake_pg.py::TestFakePG::test_allgather, test/distributed/test_fake_pg.py::TestFakePG::test_alltoall, test/distributed/test_fake_pg.py::TestFakePG::test_alltoall_base, test/distributed/test_fake_pg.py::TestFakePG::test_broadcast, test/distributed/test_fake_pg.py::TestFakePG::test_construct_fsdp, test/distributed/test_fake_pg.py::TestFakePG::test_fake_pg_tracing, test/distributed/test_fake_pg.py::TestFakePG::test_fsdp_fake_e2e, test/distributed/test_fake_pg.py::TestFakePG::test_fsdp_tp_fake_e2e, test/distributed/test_fake_pg.py::TestFakePG::test_recv, test/distributed/test_fake_pg.py::TestFakePG::test_reduce_scatter, test/distributed/test_fake_pg.py::TestFakePG::test_scatter, test/distributed/test_fake_pg.py::TestFakePG::test_send 2024-08-20T21:44:44.3722092Z 2024-08-20T21:44:44.3722881Z Running distributed/pipelining/test_backward 1/1 ... [2024-08-20 21:44:44.371002] 2024-08-20T21:44:44.3723862Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:44:44.3726728Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_backward.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:44:44.371571] 2024-08-20T21:44:47.2462033Z 2024-08-20T21:44:47.2464679Z distributed/pipelining/test_backward 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_backward_1.1_f8e00bff990f5854_.log 2024-08-20T21:44:47.2467858Z Running 1 items in this shard: test/distributed/pipelining/test_backward.py::StageBackwardTests::test_stage_backward 2024-08-20T21:44:47.2469112Z 2024-08-20T21:44:47.2471613Z Running distributed/fsdp/test_checkpoint_wrapper 1/1 ... [2024-08-20 21:44:47.246904] 2024-08-20T21:44:47.2472683Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:44:47.2477860Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_checkpoint_wrapper.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:44:47.247468] 2024-08-20T21:44:50.0714519Z 2024-08-20T21:44:50.0716473Z distributed/fsdp/test_checkpoint_wrapper 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_checkpoint_wrapper_1.1_c2236b677f52b086_.log 2024-08-20T21:44:50.0722926Z Running 8 items in this shard: test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_apply_activation_checkpointing, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_checkpoint_wrapper_args_kwargs, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_checkpoint_wrapper_cpu_offload, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_checkpoint_wrapper_kwarg_support, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_checkpoint_wrapper_parity, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_forward_missing_attributes, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_fqn, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_load_activation_checkpointed_module 2024-08-20T21:44:50.0727824Z 2024-08-20T21:44:50.0728251Z Running distributed/test_symmetric_memory 1/1 ... [2024-08-20 21:44:50.071864] 2024-08-20T21:44:50.0728882Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:44:50.0730649Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_symmetric_memory.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:44:50.072270] 2024-08-20T21:44:52.5957130Z 2024-08-20T21:44:52.5959506Z distributed/test_symmetric_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_symmetric_memory_1.1_762ac28a61c3a406_.log 2024-08-20T21:44:52.5973355Z Running 20 items in this shard: test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_cuda_nvlink_connectivity_detection, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_empty_strided_p2p, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_empty_strided_p2p_persistent, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_matmul_gather_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_matmul_gather_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_scaled_matmul_gather_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_scaled_matmul_gather_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_matmul_reduce_scatter_scatter_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_matmul_reduce_scatter_scatter_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_scaled_matmul_reduce_scatter_scatter_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_scaled_matmul_reduce_scatter_scatter_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_all_gather_symm_mem_input_False, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_all_gather_symm_mem_input_True, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_avg_symm_mem_input_False, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_avg_symm_mem_input_True, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_sum_symm_mem_input_False, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_sum_symm_mem_input_True, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_optimal_layout_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_optimal_layout_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_optimal_layout_dim_2 2024-08-20T21:44:52.5985251Z 2024-08-20T21:44:52.5985789Z Running distributed/_shard/sharded_tensor/test_logger 1/1 ... [2024-08-20 21:44:52.596212] 2024-08-20T21:44:52.5986462Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:44:52.5988297Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/test_logger.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:44:52.596656] 2024-08-20T21:44:55.3704919Z 2024-08-20T21:44:55.3707131Z distributed/_shard/sharded_tensor/test_logger 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.test_logger_1.1_13579639616a79e5_.log 2024-08-20T21:44:55.3709012Z Running 1 items in this shard: test/distributed/_shard/sharded_tensor/test_logger.py::ShardingSpecLoggerTest::test_get_or_create_logger 2024-08-20T21:44:55.3709854Z 2024-08-20T21:44:55.3711746Z Running distributed/_tensor/test_dtensor 1/1 ... [2024-08-20 21:44:55.370950] 2024-08-20T21:44:55.3712395Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:44:55.3716913Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_dtensor.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:44:55.371414] 2024-08-20T21:46:42.5548230Z 2024-08-20T21:46:42.5550879Z distributed/_tensor/test_dtensor 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_dtensor_1.1_8b0facc6c29dfcf3_.log 2024-08-20T21:46:42.5581731Z Running 32 items in this shard: test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_async_output, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_constructor, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_new_empty_strided, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_properties, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_save_load, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_spec_hash, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_spec_read_only_after_set, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_dtensor_stride, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_from_local, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_from_local_negative_dim, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_from_local_then_to_local, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_from_local_uneven_sharding, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_from_local_uneven_sharding_raise_error, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_full_tensor_grad_hint, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_full_tensor_sync, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_meta_dtensor, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_modules_w_meta_dtensor, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_to_local, test/distributed/_tensor/test_dtensor.py::DTensorTest::test_to_local_grad_hint, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_auto_implicit_replication, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_default_value_sub_mesh, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_device_mesh_nd, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_dtensor_2d_mesh, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_dtensor_api_device_mesh_context_manager, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_dtensor_device_mesh_device_conversion, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_dtensor_spec_local_shard_offset, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_from_local_sub_mesh, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_implicit_replication, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_metadata_consistency_check, test/distributed/_tensor/test_dtensor.py::DTensorMeshTest::test_redistribute_sub_mesh, test/distributed/_tensor/test_dtensor.py::TestDTensorPlacementTypes::test_split_tensor_1D, test/distributed/_tensor/test_dtensor.py::DTensorLogTest::test_dtensor_log 2024-08-20T21:46:42.5604215Z 2024-08-20T21:46:42.5605313Z Running distributed/_composable/fsdp/test_fully_shard_compile 1/1 ... [2024-08-20 21:46:42.555526] 2024-08-20T21:46:42.5606148Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:46:42.5608067Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_compile.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:46:42.556031] 2024-08-20T21:46:51.7422663Z 2024-08-20T21:46:51.7425691Z distributed/_composable/fsdp/test_fully_shard_compile 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_compile_1.1_88fc85a7e71fcc31_.log 2024-08-20T21:46:51.7441983Z Running 12 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompileCompute::test_disable_compiling_hooks, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_dynamo_trace_use_training_state, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_nested_fully_shard_backend_aot_eager, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_nested_fully_shard_backend_aot_eager_decomp_partition, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_nested_fully_shard_backend_inductor, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_simple_mlp_fullgraph_backend_aot_eager, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_simple_mlp_fullgraph_backend_aot_eager_decomp_partition, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_simple_mlp_fullgraph_backend_inductor, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_trace_fsdp_set_, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_transformer_backend_aot_eager, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_transformer_backend_aot_eager_decomp_partition, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_transformer_backend_inductor 2024-08-20T21:46:51.7455518Z 2024-08-20T21:46:51.7456367Z Running distributed/pipelining/test_schedule_multiproc 1/1 ... [2024-08-20 21:46:51.742971] 2024-08-20T21:46:51.7457393Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:46:51.7460608Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_schedule_multiproc.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:46:51.743505] 2024-08-20T21:46:54.5291518Z 2024-08-20T21:46:54.5293708Z distributed/pipelining/test_schedule_multiproc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_schedule_multiproc_1.1_0f76094012963a21_.log 2024-08-20T21:46:54.5294951Z 2024-08-20T21:46:54.5297205Z Running distributed/_tensor/test_xla_integration 1/1 ... [2024-08-20 21:46:54.529485] 2024-08-20T21:46:54.5297861Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:46:54.5301882Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_xla_integration.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:46:54.529884] 2024-08-20T21:46:57.3039513Z 2024-08-20T21:46:57.3041645Z distributed/_tensor/test_xla_integration 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_xla_integration_1.1_27a9f1646aba27c7_.log 2024-08-20T21:46:57.3045261Z Running 3 items in this shard: test/distributed/_tensor/test_xla_integration.py::DTensorXLAIntegrationTest::test_xla_distribute_tensor_1d_replicate, test/distributed/_tensor/test_xla_integration.py::DTensorXLAIntegrationTest::test_xla_distribute_tensor_1d_shard, test/distributed/_tensor/test_xla_integration.py::DTensorXLAIntegrationTest::test_xla_distribute_tensor_2d 2024-08-20T21:46:57.3047490Z 2024-08-20T21:46:57.3048281Z Running distributed/pipelining/test_unflatten 1/1 ... [2024-08-20 21:46:57.304209] 2024-08-20T21:46:57.3049100Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:46:57.3050909Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_unflatten.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:46:57.304651] 2024-08-20T21:47:00.5293932Z 2024-08-20T21:47:00.5295920Z distributed/pipelining/test_unflatten 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_unflatten_1.1_c20acd1eae87c9a9_.log 2024-08-20T21:47:00.5297587Z Running 1 items in this shard: test/distributed/pipelining/test_unflatten.py::UnflattenTests::test_unflatten 2024-08-20T21:47:00.5298551Z 2024-08-20T21:47:00.5301357Z Running distributed/_tensor/test_convolution_ops 1/1 ... [2024-08-20 21:47:00.529883] 2024-08-20T21:47:00.5302023Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:47:00.5305976Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_convolution_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:47:00.530315] 2024-08-20T21:47:24.5842946Z 2024-08-20T21:47:24.5844786Z distributed/_tensor/test_convolution_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_convolution_ops_1.1_6718a71142674185_.log 2024-08-20T21:47:24.5847978Z Running 2 items in this shard: test/distributed/_tensor/test_convolution_ops.py::DistConvolutionOpsTest::test_depthwise_convolution, test/distributed/_tensor/test_convolution_ops.py::DistConvolutionOpsTest::test_downsampling_convolution 2024-08-20T21:47:24.5849459Z 2024-08-20T21:47:24.5851022Z Running distributed/test_c10d_gloo 1/1 ... [2024-08-20 21:47:24.584835] 2024-08-20T21:47:24.5851818Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:47:24.5856432Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_gloo.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:47:24.585309] 2024-08-20T21:58:15.6048124Z 2024-08-20T21:58:15.6049640Z distributed/test_c10d_gloo 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_gloo_1.1_0011413f6c856585_.log 2024-08-20T21:58:15.6132333Z Running 120 items in this shard: test/distributed/test_c10d_gloo.py::RendezvousEnvTest::test_logging_init, test/distributed/test_c10d_gloo.py::TimeoutTest::test_default_store_timeout_gloo, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_async, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_into_tensor_coalesced, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_noncontiguous_input, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_async, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_barrier_implies_wait, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_empty_tensors, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_noncontiguous_input, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_multi_device_constructor, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_send_recv_all_to_all, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_set_gloo_pg_timeout, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_cuda_dispatched, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output_unused_param, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_weight_sharing, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_weight_sharing, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_cpu, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_gpu_gloo, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_register_just_once, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_sparse_gradients, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_init, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_return_type, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_find_unused_parameters_when_unused_parameters_empty, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_grad_is_view, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_static_graph, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_integer_list, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_torch_device_list, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_2gpu_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_4gpu_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module_grad_is_view, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output_with_unused_parameters, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_sharded_tensor, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_invalid_powerSGD_state, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_save_load_checkpoint, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients_grad_is_view, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_empty_input, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_only_empty_input, test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward, test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_optimizer, test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_unused_parameters, test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_multi_bucket, test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_single_bucket, test/distributed/test_c10d_gloo.py::ReducerTest::test_single_dtype_single_bucket, test/distributed/test_c10d_gloo.py::CommTest::test_bool_tensors, test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cpu, test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cuda, test/distributed/test_c10d_gloo.py::CommTest::test_gloo_rank_membership, test/distributed/test_c10d_gloo.py::CommTest::test_gloo_warn_not_in_group, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_default, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_subgroup, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_default_pg_gloo, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_gloo_new_group, test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_complex, test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_mismatch, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_all_to_all_single, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allgather_coalesced, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allreduce_coalesced, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_collectives, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_for_all_backends, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_optional_backend, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_monitored_barrier, test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync, test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_duplicate_pg, test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_sanity_check 2024-08-20T21:58:15.6192533Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::RendezvousEnvTest::test_logging_init 2024-08-20T21:58:15.6193684Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::TimeoutTest::test_default_store_timeout_gloo 2024-08-20T21:58:15.6194880Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics 2024-08-20T21:58:15.6196068Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics_cuda 2024-08-20T21:58:15.6197261Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_checks 2024-08-20T21:58:15.6198478Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_async 2024-08-20T21:58:15.6199749Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_checks 2024-08-20T21:58:15.6201049Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_into_tensor_coalesced 2024-08-20T21:58:15.6202373Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_noncontiguous_input 2024-08-20T21:58:15.6203628Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress 2024-08-20T21:58:15.6204816Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress_cuda 2024-08-20T21:58:15.6206007Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics 2024-08-20T21:58:15.6207199Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics_cuda 2024-08-20T21:58:15.6208382Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_checks 2024-08-20T21:58:15.6209589Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_async 2024-08-20T21:58:15.6210861Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_basics 2024-08-20T21:58:15.6212198Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks 2024-08-20T21:58:15.6213494Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks_cuda 2024-08-20T21:58:15.6214782Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_stress 2024-08-20T21:58:15.6216010Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress 2024-08-20T21:58:15.6217210Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress_cuda 2024-08-20T21:58:15.6218403Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_barrier_implies_wait 2024-08-20T21:58:15.6219600Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics 2024-08-20T21:58:15.6221025Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics_cuda 2024-08-20T21:58:15.6222225Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_checks 2024-08-20T21:58:15.6223378Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress 2024-08-20T21:58:15.6224573Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress_cuda 2024-08-20T21:58:15.6225749Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_empty_tensors 2024-08-20T21:58:15.6226885Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics 2024-08-20T21:58:15.6228032Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics_cuda 2024-08-20T21:58:15.6229208Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_checks 2024-08-20T21:58:15.6230420Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_noncontiguous_input 2024-08-20T21:58:15.6231615Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress 2024-08-20T21:58:15.6232771Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress_cuda 2024-08-20T21:58:15.6233993Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_multi_device_constructor 2024-08-20T21:58:15.6235183Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics 2024-08-20T21:58:15.6236330Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics_cuda 2024-08-20T21:58:15.6237496Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_checks 2024-08-20T21:58:15.6238663Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor 2024-08-20T21:58:15.6239937Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor_coalesced 2024-08-20T21:58:15.6241150Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress 2024-08-20T21:58:15.6242315Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress_cuda 2024-08-20T21:58:15.6243488Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics 2024-08-20T21:58:15.6244655Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics_cuda 2024-08-20T21:58:15.6245826Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_checks 2024-08-20T21:58:15.6246971Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress 2024-08-20T21:58:15.6248138Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress_cuda 2024-08-20T21:58:15.6249322Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_send_recv_all_to_all 2024-08-20T21:58:15.6250520Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_set_gloo_pg_timeout 2024-08-20T21:58:15.6251804Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics 2024-08-20T21:58:15.6253084Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics_cuda 2024-08-20T21:58:15.6254407Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_checks 2024-08-20T21:58:15.6255757Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_cuda_dispatched 2024-08-20T21:58:15.6257066Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output 2024-08-20T21:58:15.6258565Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output_unused_param 2024-08-20T21:58:15.6259981Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_module 2024-08-20T21:58:15.6261460Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_weight_sharing 2024-08-20T21:58:15.6262985Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_False 2024-08-20T21:58:15.6264504Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_True 2024-08-20T21:58:15.6266093Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_False 2024-08-20T21:58:15.6267758Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_True 2024-08-20T21:58:15.6269367Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_False 2024-08-20T21:58:15.6270886Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_True 2024-08-20T21:58:15.6272403Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_weight_sharing 2024-08-20T21:58:15.6273968Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_False 2024-08-20T21:58:15.6275583Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_True 2024-08-20T21:58:15.6277188Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_False 2024-08-20T21:58:15.6278817Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_True 2024-08-20T21:58:15.6280335Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_cpu 2024-08-20T21:58:15.6281791Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_gpu_gloo 2024-08-20T21:58:15.6283234Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_register_just_once 2024-08-20T21:58:15.6284652Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_sparse_gradients 2024-08-20T21:58:15.6286036Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_init 2024-08-20T21:58:15.6287434Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_return_type 2024-08-20T21:58:15.6288946Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_find_unused_parameters_when_unused_parameters_empty 2024-08-20T21:58:15.6290458Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad 2024-08-20T21:58:15.6292232Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_grad_is_view 2024-08-20T21:58:15.6293835Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_static_graph 2024-08-20T21:58:15.6295403Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_integer_list 2024-08-20T21:58:15.6297003Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_torch_device_list 2024-08-20T21:58:15.6298493Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_2gpu_module 2024-08-20T21:58:15.6299838Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_4gpu_module 2024-08-20T21:58:15.6301169Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module 2024-08-20T21:58:15.6302578Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module_grad_is_view 2024-08-20T21:58:15.6303927Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output 2024-08-20T21:58:15.6305296Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output_with_unused_parameters 2024-08-20T21:58:15.6306689Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_sharded_tensor 2024-08-20T21:58:15.6308007Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_invalid_powerSGD_state 2024-08-20T21:58:15.6309332Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_save_load_checkpoint 2024-08-20T21:58:15.6310602Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients 2024-08-20T21:58:15.6311941Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients_grad_is_view 2024-08-20T21:58:15.6313319Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_empty_input 2024-08-20T21:58:15.6314713Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_only_empty_input 2024-08-20T21:58:15.6315949Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward 2024-08-20T21:58:15.6317061Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_optimizer 2024-08-20T21:58:15.6318276Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_unused_parameters 2024-08-20T21:58:15.6319464Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_multi_bucket 2024-08-20T21:58:15.6320598Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_single_bucket 2024-08-20T21:58:15.6321760Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_single_dtype_single_bucket 2024-08-20T21:58:15.6322830Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_bool_tensors 2024-08-20T21:58:15.6323898Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cpu 2024-08-20T21:58:15.6325048Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cuda 2024-08-20T21:58:15.6326278Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_gloo_rank_membership 2024-08-20T21:58:15.6327364Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_gloo_warn_not_in_group 2024-08-20T21:58:15.6328516Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_default 2024-08-20T21:58:15.6329760Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_subgroup 2024-08-20T21:58:15.6331100Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_default_pg_gloo 2024-08-20T21:58:15.6332364Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_gloo_new_group 2024-08-20T21:58:15.6333482Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_complex 2024-08-20T21:58:15.6334564Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_mismatch 2024-08-20T21:58:15.6335893Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_all_to_all_single 2024-08-20T21:58:15.6337443Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allgather_coalesced 2024-08-20T21:58:15.6339003Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allreduce_coalesced 2024-08-20T21:58:15.6340526Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_collectives 2024-08-20T21:58:15.6342270Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_for_all_backends 2024-08-20T21:58:15.6343997Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_optional_backend 2024-08-20T21:58:15.6345649Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_monitored_barrier 2024-08-20T21:58:15.6346980Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync 2024-08-20T21:58:15.6348168Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_duplicate_pg 2024-08-20T21:58:15.6349411Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_sanity_check 2024-08-20T21:58:15.6350125Z 2024-08-20T21:58:15.6350707Z Running distributed/test_compute_comm_reordering 1/1 ... [2024-08-20 21:58:15.609501] 2024-08-20T21:58:15.6351359Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:58:15.6353174Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_compute_comm_reordering.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:58:15.609992] 2024-08-20T21:58:18.5363699Z 2024-08-20T21:58:18.5365845Z distributed/test_compute_comm_reordering 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_compute_comm_reordering_1.1_d3cec4fd2073a9b2_.log 2024-08-20T21:58:18.5372609Z Running 7 items in this shard: test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_grouped_scheduler_node, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_nccl_heuristics, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_raise_comms, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_reorder_compute_for_overlap, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_reorder_compute_for_overlap_custom_runtime_estimation, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_sink_waits, test/distributed/test_compute_comm_reordering.py::TestComputeCommReorderingMultiProc::test_sink_waits_raise_comms 2024-08-20T21:58:18.5377885Z 2024-08-20T21:58:18.5378442Z Running distributed/checkpoint/e2e/test_fsdp_ep 1/1 ... [2024-08-20 21:58:18.536829] 2024-08-20T21:58:18.5379176Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:58:18.5381117Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/e2e/test_fsdp_ep.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:58:18.537245] 2024-08-20T21:58:21.5115643Z 2024-08-20T21:58:21.5117500Z distributed/checkpoint/e2e/test_fsdp_ep 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.e2e.test_fsdp_ep_1.1_67a86ad84644441e_.log 2024-08-20T21:58:21.5119225Z Running 1 items in this shard: test/distributed/checkpoint/e2e/test_fsdp_ep.py::TestFSDPWithEP::test_e2e 2024-08-20T21:58:21.5119877Z 2024-08-20T21:58:21.5122374Z Running distributed/_tools/test_mem_tracker 1/1 ... [2024-08-20 21:58:21.512033] 2024-08-20T21:58:21.5123013Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:58:21.5127623Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tools/test_mem_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:58:21.512467] 2024-08-20T21:58:24.3367168Z 2024-08-20T21:58:24.3369733Z distributed/_tools/test_mem_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_mem_tracker_1.1_766901752cf3310b_.log 2024-08-20T21:58:24.3374762Z Running 3 items in this shard: test/distributed/_tools/test_mem_tracker.py::TestMemTracker::test_cuda_tracker_equivalence, test/distributed/_tools/test_mem_tracker.py::TestMemTracker::test_tracker_attribution, test/distributed/_tools/test_mem_tracker.py::TestMemTracker::test_tracker_with_activation_checkpointing 2024-08-20T21:58:24.3377774Z 2024-08-20T21:58:24.3378707Z Running distributed/_composable/fsdp/test_fully_shard_training 1/1 ... [2024-08-20 21:58:24.337494] 2024-08-20T21:58:24.3379934Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:58:24.3385067Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_training.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:58:24.338068] 2024-08-20T21:58:27.2124661Z 2024-08-20T21:58:27.2126886Z distributed/_composable/fsdp/test_fully_shard_training 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_training_1.1_3857beddeae803a3_.log 2024-08-20T21:58:27.2143054Z Running 20 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_training.py::test_compiled_fsdp, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardForwardInputs::test_root_move_forward_input_to_device, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardRegisteredParams::test_param_registration_after_backward, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardRegisteredParams::test_param_registration_after_forward, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardCastAfterInit::test_to_float64_after_init, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_explicit_prefetching, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_multi_forward_module, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_non_root_forward_backward, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_post_optim_event, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_multi_group, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_multi_group_cpu_offload_eager, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_single_group, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCompose::test_train_parity_with_activation_checkpointing, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardSharedParams::test_train_parity_with_shared_params, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardGradientAccumulation::test_1f1b_microbatching, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardGradientAccumulation::test_gradient_accumulation, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardNDTraining::test_2d_mlp_with_nd_mesh, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardHSDP3DTraining::test_3d_mlp_with_nd_mesh, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardHSDPTraining::test_train_parity_hsdp, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardCustomForwardMethod::test_register_fsdp_forward_method 2024-08-20T21:58:27.2156773Z 2024-08-20T21:58:27.2157306Z Running distributed/checkpoint/test_save_load_api 1/1 ... [2024-08-20 21:58:27.212947] 2024-08-20T21:58:27.2158184Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:58:27.2160028Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_save_load_api.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:58:27.213414] 2024-08-20T21:58:32.5410812Z 2024-08-20T21:58:32.5412754Z distributed/checkpoint/test_save_load_api 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_save_load_api_1.1_e08e5517d706b1f6_.log 2024-08-20T21:58:32.5414695Z Running 1 items in this shard: test/distributed/checkpoint/test_save_load_api.py::TestSaveAndLoadAPI::test_auto_detect 2024-08-20T21:58:32.5415490Z 2024-08-20T21:58:32.5417650Z Running distributed/checkpoint/test_dedup_tensors 1/1 ... [2024-08-20 21:58:32.541552] 2024-08-20T21:58:32.5418350Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:58:32.5423035Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_dedup_tensors.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:58:32.541998] 2024-08-20T21:58:35.3160925Z 2024-08-20T21:58:35.3162857Z distributed/checkpoint/test_dedup_tensors 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_dedup_tensors_1.1_f9d5b48325f4ba43_.log 2024-08-20T21:58:35.3164597Z Running 1 items in this shard: test/distributed/checkpoint/test_dedup_tensors.py::TestDedupTensor::test_dedup_shards 2024-08-20T21:58:35.3165319Z 2024-08-20T21:58:35.3167723Z Running distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 ... [2024-08-20 21:58:35.316560] 2024-08-20T21:58:35.3168407Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:58:35.3173039Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/fsdp/test_fsdp_dsd.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:58:35.316997] 2024-08-20T21:58:38.1409898Z 2024-08-20T21:58:38.1412278Z distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.fsdp.test_fsdp_dsd_1.1_6979e6a99699986f_.log 2024-08-20T21:58:38.1418543Z Running 6 items in this shard: test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_1d_fsdp_cpu_offload_full_model_state_dict, test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_1d_fsdp_get_model_state_dict, test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_save_with_fsdp1_and_load_with_fsdp2, test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_save_with_fsdp1_and_load_with_fsdp2_tp, test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_save_with_fsdp2_tp_and_load_with_tp, test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_save_with_tp_and_load_with_fsdp2_tp 2024-08-20T21:58:38.1423438Z 2024-08-20T21:58:38.1423880Z Running distributed/test_c10d_spawn_gloo 1/1 ... [2024-08-20 21:58:38.141424] 2024-08-20T21:58:38.1424506Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:58:38.1426343Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_spawn_gloo.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:58:38.141855] 2024-08-20T21:59:46.7754716Z 2024-08-20T21:59:46.7757210Z distributed/test_c10d_spawn_gloo 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_spawn_gloo_1.1_78e083d5415cbf62_.log 2024-08-20T21:59:46.7769974Z Running 15 items in this shard: test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allgather_chunk_gloo, test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allgather_gloo, test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allreduce_gloo, test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_broadcast_gloo, test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cpu, test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cuda, test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_rnn, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_gather, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all_single, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_allreduce, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_broadcast, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_gather, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_reduce, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_scatter 2024-08-20T21:59:46.7778912Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allgather_chunk_gloo 2024-08-20T21:59:46.7780331Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allgather_gloo 2024-08-20T21:59:46.7781722Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allreduce_gloo 2024-08-20T21:59:46.7783104Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_broadcast_gloo 2024-08-20T21:59:46.7784471Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cpu 2024-08-20T21:59:46.7786282Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cuda 2024-08-20T21:59:46.7787669Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_rnn 2024-08-20T21:59:46.7789005Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_gather 2024-08-20T21:59:46.7790323Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all 2024-08-20T21:59:46.7791660Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all_single 2024-08-20T21:59:46.7792990Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_allreduce 2024-08-20T21:59:46.7794262Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_broadcast 2024-08-20T21:59:46.7795533Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_gather 2024-08-20T21:59:46.7796793Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_reduce 2024-08-20T21:59:46.7798055Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_scatter 2024-08-20T21:59:46.7798760Z 2024-08-20T21:59:46.7799252Z Running distributed/_tensor/test_attention 1/1 ... [2024-08-20 21:59:46.776356] 2024-08-20T21:59:46.7799865Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:59:46.7801636Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_attention.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:59:46.776883] 2024-08-20T21:59:49.5125397Z 2024-08-20T21:59:49.5128012Z distributed/_tensor/test_attention 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_attention_1.1_563e8366a665e2bc_.log 2024-08-20T21:59:49.5133801Z Running 4 items in this shard: test/distributed/_tensor/test_attention.py::RingAttentionTest::test_is_causal_behavior, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_custom_transformer, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_native_transformer, test/distributed/_tensor/test_attention.py::RingAttentionTest::test_ring_attention_sdpa 2024-08-20T21:59:49.5137853Z 2024-08-20T21:59:49.5138576Z Running distributed/pipelining/test_stage 1/1 ... [2024-08-20 21:59:49.513329] 2024-08-20T21:59:49.5139616Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:59:49.5142946Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_stage.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:59:49.513892] 2024-08-20T21:59:52.2464152Z 2024-08-20T21:59:52.2465847Z distributed/pipelining/test_stage 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_stage_1.1_048ca6fd9459d43b_.log 2024-08-20T21:59:52.2466937Z 2024-08-20T21:59:52.2470075Z Running distributed/_tensor/debug/test_comm_mode_features 1/1 ... [2024-08-20 21:59:52.246790] 2024-08-20T21:59:52.2470774Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T21:59:52.2474967Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/debug/test_comm_mode_features.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 21:59:52.247205] 2024-08-20T22:00:06.4967422Z 2024-08-20T22:00:06.4969950Z distributed/_tensor/debug/test_comm_mode_features 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.debug.test_comm_mode_features_1.1_606f20b69e3c2576_.log 2024-08-20T22:00:06.4973890Z Running 4 items in this shard: test/distributed/_tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_MLPStacked_distributed_sharding_display, test/distributed/_tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_MLP_distributed_sharding_display, test/distributed/_tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_MLP_module_tracing, test/distributed/_tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_transformer_module_tracing 2024-08-20T22:00:06.4977484Z 2024-08-20T22:00:06.4978501Z Running distributed/_tensor/experimental/test_local_map 1/1 ... [2024-08-20 22:00:06.497251] 2024-08-20T22:00:06.4979683Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:00:06.4981568Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/experimental/test_local_map.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:00:06.497688] 2024-08-20T22:00:19.7381814Z 2024-08-20T22:00:19.7383723Z distributed/_tensor/experimental/test_local_map 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.experimental.test_local_map_1.1_fce5a9a22ddaf375_.log 2024-08-20T22:00:19.7387533Z Running 4 items in this shard: test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_correctness, test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_in_placements, test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_out_placements, test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_redistribute 2024-08-20T22:00:19.7390024Z 2024-08-20T22:00:19.7390491Z Running distributed/_tensor/test_pointwise_ops 1/1 ... [2024-08-20 22:00:19.738619] 2024-08-20T22:00:19.7391237Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:00:19.7393270Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_pointwise_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:00:19.739048] 2024-08-20T22:00:22.6133212Z 2024-08-20T22:00:22.6135183Z distributed/_tensor/test_pointwise_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_pointwise_ops_1.1_900217578222578d_.log 2024-08-20T22:00:22.6140183Z Running 7 items in this shard: test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_activations, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_dropout, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_dropout_backward, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_dropout_errors, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_mul_out, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_partial_add, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_partial_mul 2024-08-20T22:00:22.6143934Z 2024-08-20T22:00:22.6144511Z Running distributed/tensor/parallel/test_tp_random_state 1/1 ... [2024-08-20 22:00:22.613786] 2024-08-20T22:00:22.6145290Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:00:22.6147325Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_tp_random_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:00:22.614210] 2024-08-20T22:00:28.3441609Z 2024-08-20T22:00:28.3445169Z distributed/tensor/parallel/test_tp_random_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_tp_random_state_1.1_a3bf98274bccb9ca_.log 2024-08-20T22:00:28.3448609Z Running 1 items in this shard: test/distributed/tensor/parallel/test_tp_random_state.py::TensorParallelRandomStateTests::test_model_init 2024-08-20T22:00:28.3450064Z 2024-08-20T22:00:28.3450913Z Running distributed/checkpoint/e2e/test_e2e_save_and_load 1/1 ... [2024-08-20 22:00:28.344562] 2024-08-20T22:00:28.3452166Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:00:28.3455594Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/e2e/test_e2e_save_and_load.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:00:28.345127] 2024-08-20T22:01:10.1518967Z 2024-08-20T22:01:10.1521244Z distributed/checkpoint/e2e/test_e2e_save_and_load 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.e2e.test_e2e_save_and_load_1.1_810c1dacda091ded_.log 2024-08-20T22:01:10.1531497Z Running 14 items in this shard: test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_different_ordered_state_dict_keys, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_async_cached_cache_staged_state_dict_False, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_async_cached_cache_staged_state_dict_True, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_False_model_type0, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_False_model_type1, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_False_model_type2, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_True_model_type0, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_True_model_type1, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_True_model_type2, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_no_dist, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_overwrite, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_partial_load, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestNoCPU::test_no_cpu, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestInitStateDict::test_init_state_dict 2024-08-20T22:01:10.1540267Z 2024-08-20T22:01:10.1540740Z Running distributed/_composable/test_checkpoint 1/1 ... [2024-08-20 22:01:10.152476] 2024-08-20T22:01:10.1541397Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:01:10.1543211Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:01:10.152944] 2024-08-20T22:01:13.0272100Z 2024-08-20T22:01:13.0274118Z distributed/_composable/test_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_checkpoint_1.1_f83d4ad25e1483fe_.log 2024-08-20T22:01:13.0278258Z Running 6 items in this shard: test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_checkpoint_kwargs, test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_clears_state_on_error_in_forward, test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_multi_args, test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_random_cpu, test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_tensor_only_cpu, test/distributed/_composable/test_checkpoint.py::TestCheckpoint::test_tensor_only_gpu 2024-08-20T22:01:13.0281509Z 2024-08-20T22:01:13.0282117Z Running distributed/_tensor/test_init 1/1 ... [2024-08-20 22:01:13.027697] 2024-08-20T22:01:13.0282822Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:01:13.0284567Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_init.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:01:13.028141] 2024-08-20T22:01:35.2417142Z 2024-08-20T22:01:35.2418748Z distributed/_tensor/test_init 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_init_1.1_f1e2a20f4c842c87_.log 2024-08-20T22:01:35.2423240Z Running 7 items in this shard: test/distributed/_tensor/test_init.py::DTensorInitOpsTest::test_init_ops, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_empty, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_full, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_ones, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_zeros, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_zeros_full_mesh, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_zeros_submesh 2024-08-20T22:01:35.2426574Z 2024-08-20T22:01:35.2427067Z Running distributed/_tools/test_fsdp2_mem_tracker 1/1 ... [2024-08-20 22:01:35.242121] 2024-08-20T22:01:35.2427767Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:01:35.2429564Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tools/test_fsdp2_mem_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:01:35.242546] 2024-08-20T22:01:38.2168704Z 2024-08-20T22:01:38.2171002Z distributed/_tools/test_fsdp2_mem_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_fsdp2_mem_tracker_1.1_4289babf1c28b533_.log 2024-08-20T22:01:38.2174738Z Running 3 items in this shard: test/distributed/_tools/test_fsdp2_mem_tracker.py::TestTrackerFullyShard1DTrainingCore::test_tracker_multi_group_eager, test/distributed/_tools/test_fsdp2_mem_tracker.py::TestTrackerFullyShard1DTrainingCore::test_tracker_non_root_forward_backward, test/distributed/_tools/test_fsdp2_mem_tracker.py::TestTrackerFullyShard1DTrainingCompose::test_tracker_with_activation_checkpointing 2024-08-20T22:01:38.2177658Z 2024-08-20T22:01:38.2178186Z Running distributed/checkpoint/test_fsdp_optim_state 1/1 ... [2024-08-20 22:01:38.217394] 2024-08-20T22:01:38.2178992Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:01:38.2181307Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_fsdp_optim_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:01:38.217834] 2024-08-20T22:01:47.0076806Z 2024-08-20T22:01:47.0078940Z distributed/checkpoint/test_fsdp_optim_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_fsdp_optim_state_1.1_ae0c401806e653d4_.log 2024-08-20T22:01:47.0082470Z Running 2 items in this shard: test/distributed/checkpoint/test_fsdp_optim_state.py::FsdpOptimStateCheckpoint::test_load_sharded_optimizer_state_dict_pass_planner_False, test/distributed/checkpoint/test_fsdp_optim_state.py::FsdpOptimStateCheckpoint::test_load_sharded_optimizer_state_dict_pass_planner_True 2024-08-20T22:01:47.0084773Z 2024-08-20T22:01:47.0085278Z Running distributed/checkpoint/test_format_utils 1/1 ... [2024-08-20 22:01:47.008146] 2024-08-20T22:01:47.0085959Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:01:47.0089137Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_format_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:01:47.008558] 2024-08-20T22:01:57.9994219Z 2024-08-20T22:01:57.9996044Z distributed/checkpoint/test_format_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_format_utils_1.1_1e1ddb2654d0f2f2_.log 2024-08-20T22:01:57.9999033Z Running 3 items in this shard: test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_dcp_to_torch_save, test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_online_torch_save_to_dcp, test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_torch_save_to_dcp 2024-08-20T22:01:58.0001074Z 2024-08-20T22:01:58.0001538Z Running distributed/checkpoint/test_fsspec 1/1 ... [2024-08-20 22:01:57.999887] 2024-08-20T22:01:58.0002162Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:01:58.0006303Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_fsspec.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:01:58.000321] 2024-08-20T22:02:00.4740528Z 2024-08-20T22:02:00.4742357Z distributed/checkpoint/test_fsspec 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_fsspec_1.1_1372df8212139c74_.log 2024-08-20T22:02:00.4744343Z Running 2 items in this shard: test/distributed/checkpoint/test_fsspec.py::TestFSSpec::test_fsspec, test/distributed/checkpoint/test_fsspec.py::TestFSSpec::test_overwrite 2024-08-20T22:02:00.4745357Z 2024-08-20T22:02:00.4747410Z Running distributed/checkpoint/test_traverse 1/1 ... [2024-08-20 22:02:00.474518] 2024-08-20T22:02:00.4748066Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:02:00.4752449Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_traverse.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:02:00.474941] 2024-08-20T22:02:03.2491203Z 2024-08-20T22:02:03.2493320Z distributed/checkpoint/test_traverse 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_traverse_1.1_d00ea6594a81aaca_.log 2024-08-20T22:02:03.2498120Z Running 7 items in this shard: test/distributed/checkpoint/test_traverse.py::TestTraverse::test_get_element, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_set_element, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_doesnt_ignore_intermediate_collections, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_nested_dict, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_nested_list, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_shallow, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_with_ordered_dict 2024-08-20T22:02:03.2501973Z 2024-08-20T22:02:03.2502493Z Running distributed/checkpoint/test_dtensor_resharding 1/1 ... [2024-08-20 22:02:03.249604] 2024-08-20T22:02:03.2503193Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:02:03.2505058Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_dtensor_resharding.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:02:03.250020] 2024-08-20T22:02:20.2026268Z 2024-08-20T22:02:20.2028106Z distributed/checkpoint/test_dtensor_resharding 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_dtensor_resharding_1.1_970a6e28b62722bb_.log 2024-08-20T22:02:20.2033354Z Running 5 items in this shard: test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardPlacementChange::test_1d_to_1d_reshard_placement_change, test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardPlacementChange::test_2d_to_2d_reshard_placement_change, test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardMeshChange::test_1d_to_2d_reshard_mesh_change, test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardMeshChange::test_2d_to_1d_reshard_mesh_change, test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardMeshChange::test_dtensor_checkpoint_resharding_with_empty_shard 2024-08-20T22:02:20.2037439Z 2024-08-20T22:02:20.2037944Z Running distributed/checkpoint/test_tp_checkpoint 1/1 ... [2024-08-20 22:02:20.203080] 2024-08-20T22:02:20.2038612Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:02:20.2040446Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_tp_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:02:20.203518] 2024-08-20T22:02:28.4901848Z 2024-08-20T22:02:28.4903959Z distributed/checkpoint/test_tp_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_tp_checkpoint_1.1_3ee693d34b619e83_.log 2024-08-20T22:02:28.4906515Z Running 2 items in this shard: test/distributed/checkpoint/test_tp_checkpoint.py::TestTpCheckpoint::test_tp_checkpoint, test/distributed/checkpoint/test_tp_checkpoint.py::TestTpCheckpoint::test_tp_checkpoint_load_on_meta_device 2024-08-20T22:02:28.4908093Z 2024-08-20T22:02:28.4909268Z Running distributed/checkpoint/e2e/test_fine_tuning 1/1 ... [2024-08-20 22:02:28.490682] 2024-08-20T22:02:28.4910024Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:02:28.4915133Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/e2e/test_fine_tuning.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:02:28.491146] 2024-08-20T22:02:31.2654626Z 2024-08-20T22:02:31.2656571Z distributed/checkpoint/e2e/test_fine_tuning 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.e2e.test_fine_tuning_1.1_efdcc10651d36f7d_.log 2024-08-20T22:02:31.2658598Z Running 1 items in this shard: test/distributed/checkpoint/e2e/test_fine_tuning.py::TestFineTuning::test_fine_tuning 2024-08-20T22:02:31.2659372Z 2024-08-20T22:02:31.2661503Z Running distributed/_tools/test_memory_tracker 1/1 ... [2024-08-20 22:02:31.265947] 2024-08-20T22:02:31.2662149Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:02:31.2666788Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tools/test_memory_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:02:31.266381] 2024-08-20T22:02:34.0403845Z 2024-08-20T22:02:34.0405562Z distributed/_tools/test_memory_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_memory_tracker_1.1_234cf670a759ac1e_.log 2024-08-20T22:02:34.0407213Z Running 1 items in this shard: test/distributed/_tools/test_memory_tracker.py::TestMemoryTracker::test_local_model 2024-08-20T22:02:34.0407912Z 2024-08-20T22:02:34.0410643Z Running distributed/_composable/fsdp/test_fully_shard_init 1/1 ... [2024-08-20 22:02:34.040851] 2024-08-20T22:02:34.0411361Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:02:34.0416310Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_init.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:02:34.041308] 2024-08-20T22:02:36.8656259Z 2024-08-20T22:02:36.8659619Z distributed/_composable/fsdp/test_fully_shard_init 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_init_1.1_7bbb58b5cda03f1d_.log 2024-08-20T22:02:36.8694470Z Running 30 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardDeviceTensor::test_move_states_to_device_tensor, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardDeviceDTensor::test_move_states_to_device_dtensor_invalid, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardDeviceDTensor::test_move_states_to_device_dtensor_valid, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMeshArg::test_invalid_mesh_ndim, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_duplicate, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_list_of_mlps, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_nested, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_nested_fully_shard_and_replicate, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_single, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_states_list_of_mlps, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_states_nested_fully_shard, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_states_shared_params_and_buffers, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardParamModuleInfos::test_get_param_module_infos_duplicates, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardParamModuleInfos::test_get_param_module_infos_list_of_mlps, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardParamModuleInfos::test_get_param_module_infos_shared_params, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardedParameterTensor::test_raise_scalar_parameter, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardedParameterTensor::test_shard_tensor_parameters, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardedParameterDTensor::test_shard_dtensor_parameters, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_double_lazy_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_is_root, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_module_and_param_fqns, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_multi_module_root, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_reset_sharded_param_in_lazy_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_invalid_meta_device_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_meta_device_1d_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_meta_device_2d_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_rank0_broadcast_meta_device_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardProcessGroupInit::test_1d_process_group_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardProcessGroupInit::test_2d_process_group_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardHSDPBroadcast::test_hsdp_broadcast_across_replicas 2024-08-20T22:02:36.8723278Z 2024-08-20T22:02:36.8723944Z Running distributed/checkpoint/test_planner 1/1 ... [2024-08-20 22:02:36.866503] 2024-08-20T22:02:36.8724606Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:02:36.8726395Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_planner.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:02:36.867053] 2024-08-20T22:02:39.6926291Z 2024-08-20T22:02:39.6928231Z distributed/checkpoint/test_planner 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_planner_1.1_9bc2c8e7ab770c5b_.log 2024-08-20T22:02:39.6932891Z Running 7 items in this shard: test/distributed/checkpoint/test_planner.py::TestSavePlan::test_global_plan, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_load_with_resharding, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_load_with_world_size_diff_by_one, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_local_load_plan, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_local_plan, test/distributed/checkpoint/test_planner.py::TestPlannerHelpers::test_create_read_item_from_chunks, test/distributed/checkpoint/test_planner.py::TestLoadPlanner::test_strict 2024-08-20T22:02:39.6936403Z 2024-08-20T22:02:39.6937003Z Running distributed/elastic/test_control_plane 1/1 ... [2024-08-20 22:02:39.693108] 2024-08-20T22:02:39.6937834Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:02:39.6939669Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/test_control_plane.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:02:39.693596] 2024-08-20T22:02:45.6724695Z 2024-08-20T22:02:45.6726447Z distributed/elastic/test_control_plane 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.test_control_plane_1.1_e1ab0c9d2390bcdf_.log 2024-08-20T22:02:45.6732379Z Running 9 items in this shard: test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_nccl_trace_pickle, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_nccl_trace_pickle_with_json, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_nccl_trace_pickle_with_params, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_traceback, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_get_handler_names, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_get_handler_nonexistant, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_run_handler, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_tcp, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_worker_server 2024-08-20T22:02:45.6737159Z 2024-08-20T22:02:45.6737668Z Running distributed/checkpoint/test_dtensor_checkpoint 1/1 ... [2024-08-20 22:02:45.672929] 2024-08-20T22:02:45.6738367Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:02:45.6740219Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_dtensor_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:02:45.673411] 2024-08-20T22:02:51.6537904Z 2024-08-20T22:02:51.6539982Z distributed/checkpoint/test_dtensor_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_dtensor_checkpoint_1.1_1e6121d676fd93bd_.log 2024-08-20T22:02:51.6541896Z Running 1 items in this shard: test/distributed/checkpoint/test_dtensor_checkpoint.py::DTensorPlanner::test_distributed_tensor_planner 2024-08-20T22:02:51.6543097Z 2024-08-20T22:02:51.6544604Z Running distributed/_composable/fsdp/test_fully_shard_logging 1/1 ... [2024-08-20 22:02:51.654234] 2024-08-20T22:02:51.6545337Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:02:51.6551138Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_logging.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:02:51.654659] 2024-08-20T22:02:54.3940213Z 2024-08-20T22:02:54.3943242Z distributed/_composable/fsdp/test_fully_shard_logging 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_logging_1.1_61ab6d52f4c9ff90_.log 2024-08-20T22:02:54.3945836Z Running 0 items in this shard: 2024-08-20T22:02:54.3946244Z 2024-08-20T22:02:54.3949193Z Running distributed/_composable/fsdp/test_fully_shard_memory 1/1 ... [2024-08-20 22:02:54.394643] 2024-08-20T22:02:54.3950504Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:02:54.3956012Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_memory.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:02:54.395149] 2024-08-20T22:02:57.2695618Z 2024-08-20T22:02:57.2697988Z distributed/_composable/fsdp/test_fully_shard_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_memory_1.1_e4a465003e90d39f_.log 2024-08-20T22:02:57.2700149Z Running 1 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_memory.py::TestFullyShardMemory::test_fully_shard_training_memory 2024-08-20T22:02:57.2701105Z 2024-08-20T22:02:57.2702282Z Running distributed/_tensor/test_optimizers 1/1 ... [2024-08-20 22:02:57.270039] 2024-08-20T22:02:57.2702959Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:02:57.2707720Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_optimizers.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:02:57.270492] 2024-08-20T22:03:36.0224686Z 2024-08-20T22:03:36.0226340Z distributed/_tensor/test_optimizers 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_optimizers_1.1_b2d413c168584d7a_.log 2024-08-20T22:03:36.0233218Z Running 11 items in this shard: test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_RMSprop_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_adadelta_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_adagrad_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_adam_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_adamax_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_adamw_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_asgd_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_nadam_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_optimizer_foreach_supported_types_include_DTensor, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_radam_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_sgd_1d_sharding 2024-08-20T22:03:36.0239637Z 2024-08-20T22:03:36.0240055Z Running distributed/test_functional_api 1/1 ... [2024-08-20 22:03:36.023371] 2024-08-20T22:03:36.0240666Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:03:36.0242654Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_functional_api.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:03:36.023800] 2024-08-20T22:04:06.0933728Z 2024-08-20T22:04:06.0936711Z distributed/test_functional_api 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_functional_api_1.1_eaaecdf42ac6748f_.log 2024-08-20T22:04:06.0980890Z Running 50 items in this shard: test/distributed/test_functional_api.py::TestExpand::test_expand_1d_rank_list, test/distributed/test_functional_api.py::TestExpand::test_expand_2d_rank_list, test/distributed/test_functional_api.py::TestExpand::test_expand_device_mesh, test/distributed/test_functional_api.py::TestExpand::test_expand_device_mesh_tuple, test/distributed/test_functional_api.py::TestExpand::test_expand_process_group, test/distributed/test_functional_api.py::TestPgTag::test_find_or_create_pg, test/distributed/test_functional_api.py::TestPgTag::test_find_root_pg, test/distributed/test_functional_api.py::TestPgTag::test_pg_creation_with_tag, test/distributed/test_functional_api.py::TestPgTag::test_pg_lookup_roundtrip, test/distributed/test_functional_api.py::TestPgTag::test_pg_lookup_with_tag, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_gather_into_tensor_coalesced_device_cpu, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_gather_into_tensor_coalesced_device_cuda, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_gather_tensor_device_cpu, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_gather_tensor_device_cuda, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_reduce_coalesced_eager_device_cpu, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_reduce_coalesced_eager_device_cuda, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_reduce_eager_device_cpu, test/distributed/test_functional_api.py::TestTraceableCollectives::test_all_reduce_eager_device_cuda, test/distributed/test_functional_api.py::TestTraceableCollectives::test_broadcast_device_cpu, test/distributed/test_functional_api.py::TestTraceableCollectives::test_broadcast_device_cuda, test/distributed/test_functional_api.py::TestTraceableCollectives::test_reduce_scatter_into_tensor_coalesced_device_cpu, test/distributed/test_functional_api.py::TestTraceableCollectives::test_reduce_scatter_into_tensor_coalesced_device_cuda, test/distributed/test_functional_api.py::TestTraceableCollectives::test_reduce_scatter_tensor_device_cpu, test/distributed/test_functional_api.py::TestTraceableCollectives::test_reduce_scatter_tensor_device_cuda, test/distributed/test_functional_api.py::TestMetaCollectives::test_all_reduce, test/distributed/test_functional_api.py::TestGradCollectives::test_all_reduce, test/distributed/test_functional_api.py::TestMakeFx::test_all_reduce_tracing, test/distributed/test_functional_api.py::TestCollectivesWithNCCL::test_all_gather_into_tensor_coalesced, test/distributed/test_functional_api.py::TestCollectivesWithNCCL::test_all_to_all_single, test/distributed/test_functional_api.py::TestCollectivesWithNCCL::test_all_to_all_single_1d_input, test/distributed/test_functional_api.py::TestCollectivesWithNCCL::test_all_to_all_single_split_sizes_none, test/distributed/test_functional_api.py::TestCollectivesWithNCCL::test_tracing, test/distributed/test_functional_api.py::TestCollectivesWithNCCL::test_tracing_with_dce_code, test/distributed/test_functional_api.py::TestCollectivesWithNCCL::test_tracing_with_fakepg, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_all_gather_into_tensor_coalesced, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_all_to_all_single, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_all_to_all_single_1d_input, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_all_to_all_single_split_sizes_none, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_permute_tensor_with_sub_group, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_tracing, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_tracing_with_dce_code, test/distributed/test_functional_api.py::TestNCCLCollectivesWithWorldSize4::test_tracing_with_fakepg, test/distributed/test_functional_api.py::TestFunctionalAutograd::test_all_gather_tensor_compile_False, test/distributed/test_functional_api.py::TestFunctionalAutograd::test_all_gather_tensor_compile_True, test/distributed/test_functional_api.py::TestFunctionalAutograd::test_all_to_all_single_compile_False, test/distributed/test_functional_api.py::TestFunctionalAutograd::test_all_to_all_single_compile_True, test/distributed/test_functional_api.py::TestFunctionalAutograd::test_all_to_all_single_inductor, test/distributed/test_functional_api.py::TestFunctionalAutograd::test_reduce_scatter_tensor_compile_False, test/distributed/test_functional_api.py::TestFunctionalAutograd::test_reduce_scatter_tensor_compile_True, test/distributed/test_functional_api.py::TestFunctionalAutogradWithNCCL::test_all_to_all_single 2024-08-20T22:04:06.1011964Z 2024-08-20T22:04:06.1012520Z Running distributed/test_control_collectives 1/1 ... [2024-08-20 22:04:06.094186] 2024-08-20T22:04:06.1013165Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:04:06.1014936Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_control_collectives.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:04:06.094742] 2024-08-20T22:04:08.9702268Z 2024-08-20T22:04:08.9704847Z distributed/test_control_collectives 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_control_collectives_1.1_174bd9b769312047_.log 2024-08-20T22:04:08.9716854Z Running 13 items in this shard: test/distributed/test_control_collectives.py::TestCollectives::test_all_gather_timeout, test/distributed/test_control_collectives.py::TestCollectives::test_all_sum, test/distributed/test_control_collectives.py::TestCollectives::test_all_sum_timeout, test/distributed/test_control_collectives.py::TestCollectives::test_barrier, test/distributed/test_control_collectives.py::TestCollectives::test_barrier_timeout, test/distributed/test_control_collectives.py::TestCollectives::test_broadcast, test/distributed/test_control_collectives.py::TestCollectives::test_broadcast_timeout, test/distributed/test_control_collectives.py::TestCollectives::test_gather, test/distributed/test_control_collectives.py::TestCollectives::test_gather_timeout, test/distributed/test_control_collectives.py::TestCollectives::test_scatter, test/distributed/test_control_collectives.py::TestCollectives::test_scatter_timeout, test/distributed/test_control_collectives.py::TestCollectives::test_simple_user_func, test/distributed/test_control_collectives.py::TestCollectives::test_unique 2024-08-20T22:04:08.9726256Z 2024-08-20T22:04:08.9727225Z Running distributed/checkpoint/test_fsdp_tp_checkpoint_conversion 1/1 ... [2024-08-20 22:04:08.970987] 2024-08-20T22:04:08.9728336Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:04:08.9731010Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_fsdp_tp_checkpoint_conversion.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:04:08.971543] 2024-08-20T22:04:14.7022076Z 2024-08-20T22:04:14.7024487Z distributed/checkpoint/test_fsdp_tp_checkpoint_conversion 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_fsdp_tp_checkpoint_conversion_1.1_ac8f3a48f5de8eed_.log 2024-08-20T22:04:14.7027356Z Running 1 items in this shard: test/distributed/checkpoint/test_fsdp_tp_checkpoint_conversion.py::TestFsdpTpCheckpointConversion::test_fsdp_to_tp 2024-08-20T22:04:14.7028405Z 2024-08-20T22:04:14.7028912Z Running distributed/_tensor/test_op_strategy 1/1 ... [2024-08-20 22:04:14.702654] 2024-08-20T22:04:14.7029611Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:04:14.7033943Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_op_strategy.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:04:14.703081] 2024-08-20T22:04:17.7279685Z 2024-08-20T22:04:17.7281740Z distributed/_tensor/test_op_strategy 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_op_strategy_1.1_66a1fecae10dddae_.log 2024-08-20T22:04:17.7290230Z Running 15 items in this shard: test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_batch_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_bmm_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_free_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_mm_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_bmm_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_bmm_2d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_linearity_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_mm_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_mm_2d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_pointwise_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_bmm_strategies, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_mm_strategies, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_redistribute_cost_latency, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_redistribute_cost_mesh_1d, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_redistribute_cost_mesh_2d 2024-08-20T22:04:17.7298132Z 2024-08-20T22:04:17.7298688Z Running distributed/checkpoint/test_hsdp_checkpoint 1/1 ... [2024-08-20 22:04:17.728469] 2024-08-20T22:04:17.7299440Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:04:17.7301451Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_hsdp_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:04:17.728928] 2024-08-20T22:04:31.7262511Z 2024-08-20T22:04:31.7265212Z distributed/checkpoint/test_hsdp_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_hsdp_checkpoint_1.1_4c16db1a61d762cb_.log 2024-08-20T22:04:31.7270256Z Running 4 items in this shard: test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_checkpoint_is_even_sharded_model_False, test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_checkpoint_is_even_sharded_model_True, test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_fsdp_checkpoint_conversion_is_even_sharded_model_False, test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_fsdp_checkpoint_conversion_is_even_sharded_model_True 2024-08-20T22:04:31.7274127Z 2024-08-20T22:04:31.7274760Z Running distributed/_composable/test_composability/test_pp_composability 1/1 ... [2024-08-20 22:04:31.726712] 2024-08-20T22:04:31.7275549Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:04:31.7277757Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_composability/test_pp_composability.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:04:31.727182] 2024-08-20T22:04:34.5998827Z 2024-08-20T22:04:34.6000957Z distributed/_composable/test_composability/test_pp_composability 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_composability.test_pp_composability_1.1_0fed095348962263_.log 2024-08-20T22:04:34.6020608Z Running 20 items in this shard: test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_DDP_ScheduleClass0_use_new_runtime_False, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_DDP_ScheduleClass0_use_new_runtime_True, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_DDP_ScheduleClass1_use_new_runtime_False, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_DDP_ScheduleClass1_use_new_runtime_True, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_DDP_ScheduleClass2_use_new_runtime_False, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_DDP_ScheduleClass2_use_new_runtime_True, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_DDP_ScheduleClass3_use_new_runtime_False, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_DDP_ScheduleClass3_use_new_runtime_True, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_DDP_ScheduleClass4_use_new_runtime_False, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_DDP_ScheduleClass4_use_new_runtime_True, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_FSDP_ScheduleClass0_use_new_runtime_False, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_FSDP_ScheduleClass0_use_new_runtime_True, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_FSDP_ScheduleClass1_use_new_runtime_False, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_FSDP_ScheduleClass1_use_new_runtime_True, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_FSDP_ScheduleClass2_use_new_runtime_False, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_FSDP_ScheduleClass2_use_new_runtime_True, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_FSDP_ScheduleClass3_use_new_runtime_False, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_FSDP_ScheduleClass3_use_new_runtime_True, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_FSDP_ScheduleClass4_use_new_runtime_False, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_manual_with_data_parallel_dp_type_FSDP_ScheduleClass4_use_new_runtime_True 2024-08-20T22:04:34.6038369Z 2024-08-20T22:04:34.6038888Z Running distributed/checkpoint/test_compatibility 1/1 ... [2024-08-20 22:04:34.600341] 2024-08-20T22:04:34.6039819Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:04:34.6041749Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_compatibility.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:04:34.600757] 2024-08-20T22:04:37.3748052Z 2024-08-20T22:04:37.3749988Z distributed/checkpoint/test_compatibility 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_compatibility_1.1_6c024e15e643a0b4_.log 2024-08-20T22:04:37.3753167Z Running 3 items in this shard: test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_metadata, test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_sharded_tensor_dependency, test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_storage_meta 2024-08-20T22:04:37.3755179Z 2024-08-20T22:04:37.3755652Z Running distributed/_tensor/test_view_ops 1/1 ... [2024-08-20 22:04:37.375250] 2024-08-20T22:04:37.3756340Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:04:37.3760162Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_view_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:04:37.375687] 2024-08-20T22:05:08.5798852Z 2024-08-20T22:05:08.5801669Z distributed/_tensor/test_view_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_view_ops_1.1_4f74b70b1db9d1c9_.log 2024-08-20T22:05:08.5805810Z Running 3 items in this shard: test/distributed/_tensor/test_view_ops.py::TestViewOps::test_complex_view_ops, test/distributed/_tensor/test_view_ops.py::TestViewOps::test_view_groups, test/distributed/_tensor/test_view_ops.py::TestViewOps::test_view_ops 2024-08-20T22:05:08.5808347Z 2024-08-20T22:05:08.5809401Z Running distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_ 1/1 ... [2024-08-20 22:05:08.580536] 2024-08-20T22:05:08.5810606Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:05:08.5815554Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:05:08.581097] 2024-08-20T22:05:11.5057337Z 2024-08-20T22:05:11.5060902Z distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_ 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_clip_grad_norm__1.1_ee60f21786f6f73a_.log 2024-08-20T22:05:11.5065853Z Running 2 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_.py::TestClipGradNormWorldSize2::test_clip_grad_norm_1d, test/distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_.py::TestClipGradNormWorldSize4::test_clip_grad_norm_2d 2024-08-20T22:05:11.5068629Z 2024-08-20T22:05:11.5069546Z Running distributed/tensor/parallel/test_parallelize_api 1/1 ... [2024-08-20 22:05:11.506527] 2024-08-20T22:05:11.5070683Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:05:11.5075246Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_parallelize_api.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:05:11.507098] 2024-08-20T22:05:42.9894481Z 2024-08-20T22:05:42.9896860Z distributed/tensor/parallel/test_parallelize_api 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_parallelize_api_1.1_8239082ff858120e_.log 2024-08-20T22:05:42.9906142Z Running 10 items in this shard: test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_linear_col_wise_parallel, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_linear_row_wise_parallel, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_parallelize_mlp_with_module_api, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_parallelize_mlp_with_module_api_nested, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_parallelize_module_multi_wildcard, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_parallelize_module_with_digit, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_parallelize_module_with_question, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_parallelize_module_with_star, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_prepare_module_input, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_prepare_module_output 2024-08-20T22:05:42.9913230Z 2024-08-20T22:05:42.9913740Z Running distributed/fsdp/test_fsdp_optim_state 1/1 ... [2024-08-20 22:05:42.990001] 2024-08-20T22:05:42.9914399Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:05:42.9916295Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_optim_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:05:42.990441] 2024-08-20T22:08:48.0058014Z 2024-08-20T22:08:48.0059863Z distributed/fsdp/test_fsdp_optim_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_optim_state_1.1_5544bd2ef680ace0_.log 2024-08-20T22:08:48.0109980Z Running 60 items in this shard: test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_compatible_with_trec, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_flatten_sharded_optim_state_dict_nested, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_flatten_sharded_optim_state_dict_transformer, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_full_optim_state_dict_keys, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_full_optim_state_dict_nested_invalid, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_interface_arguments, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_no_grad, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_input_warning, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_without_param_groups, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type0_use_multiple_param_groups_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type0_use_multiple_param_groups_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type1_use_multiple_param_groups_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type1_use_multiple_param_groups_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_names, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_save_load_without_0th_param_state_state_dict_type0, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_save_load_without_0th_param_state_state_dict_type1, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_halve_world_size, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_transformer, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_halve_world_size, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_transformer, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type0_add_to_fsdp_module_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type0_add_to_fsdp_module_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type1_add_to_fsdp_module_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type1_add_to_fsdp_module_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_state_dict_with_none_tensor_state, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_use_orig_params, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_with_empty_optimizer_state, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_with_no_shard 2024-08-20T22:08:48.0157488Z 2024-08-20T22:08:48.0158160Z Running distributed/_composable/test_compose 1/1 ... [2024-08-20 22:08:48.007032] 2024-08-20T22:08:48.0167207Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:08:48.0169310Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_compose.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:08:48.007484] 2024-08-20T22:09:14.7173942Z 2024-08-20T22:09:14.7176214Z distributed/_composable/test_compose 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_compose_1.1_7441fd4413763dad_.log 2024-08-20T22:09:14.7183110Z Running 9 items in this shard: test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fsdp_submodules_non_reentrant, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fsdp_submodules_with_param, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fsdp_submodules_with_param_no_shard, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fully_shard_cast_forward_inputs, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_composable_fsdp_replicate, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_fully_shard_replicate_composability, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_fully_shard_replicate_correct_replicate_params, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_state_dict_fsdp_submodules, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_wrap_same_submodule 2024-08-20T22:09:14.7188968Z 2024-08-20T22:09:14.7189483Z Running distributed/fsdp/test_fsdp_pure_fp16 1/1 ... [2024-08-20 22:09:14.717832] 2024-08-20T22:09:14.7190198Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:09:14.7192119Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_pure_fp16.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:09:14.718241] 2024-08-20T22:09:23.4042672Z 2024-08-20T22:09:23.4045342Z distributed/fsdp/test_fsdp_pure_fp16 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_pure_fp16_1.1_492d72df1bc70dd4_.log 2024-08-20T22:09:23.4048272Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_pure_fp16.py::TestPureFP16::test_fp16_dtypes, test/distributed/fsdp/test_fsdp_pure_fp16.py::TestPureFP16::test_pure_fp16_training 2024-08-20T22:09:23.4049508Z 2024-08-20T22:09:23.4049984Z Running distributed/nn/jit/test_instantiator 1/1 ... [2024-08-20 22:09:23.404637] 2024-08-20T22:09:23.4050718Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:09:23.4053756Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/nn/jit/test_instantiator.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:09:23.405071] 2024-08-20T22:09:26.2291355Z 2024-08-20T22:09:26.2293549Z distributed/nn/jit/test_instantiator 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.nn.jit.test_instantiator_1.1_a3e4ecd82b14865d_.log 2024-08-20T22:09:26.2296637Z Running 3 items in this shard: test/distributed/nn/jit/test_instantiator.py::TestInstantiator::test_get_arg_return_types_from_interface, test/distributed/nn/jit/test_instantiator.py::TestInstantiator::test_instantiate_non_scripted_remote_module_template, test/distributed/nn/jit/test_instantiator.py::TestInstantiator::test_instantiate_scripted_remote_module_template 2024-08-20T22:09:26.2299232Z 2024-08-20T22:09:26.2299804Z Running distributed/_composable/fsdp/test_fully_shard_comm 1/1 ... [2024-08-20 22:09:26.229588] 2024-08-20T22:09:26.2300517Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:09:26.2303453Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_comm.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:09:26.230003] 2024-08-20T22:09:29.0543590Z 2024-08-20T22:09:29.0545565Z distributed/_composable/fsdp/test_fully_shard_comm 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_comm_1.1_efd3a7871b3a0081_.log 2024-08-20T22:09:29.0557237Z Running 14 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCollectiveOps::test_all_gather_fp32, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCollectiveOps::test_reduce_scatter_fp16, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCollectiveOps::test_reduce_scatter_fp32, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCommunication::test_fully_shard_communication_count, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCommunication::test_manual_reshard_with_reshard_after_forward_false, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCommunication::test_set_reduce_scatter_divide_factor, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardPrefetch::test_fully_shard_backward_prefetch, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardPrefetch::test_fully_shard_multi_module_backward_prefetch, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardPrefetch::test_fully_shard_multi_module_unused_module, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardPrefetch::test_set_modules_to_backward_prefetch, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardPrefetch::test_set_modules_to_forward_prefetch, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardUnshardMultiProcess::test_unshard_async, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardUnshardMultiThread::test_unshard_no_param_group, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardUnshardMultiThread::test_unshard_without_lazy_init 2024-08-20T22:09:29.0567347Z 2024-08-20T22:09:29.0567972Z Running distributed/_composable/fsdp/test_fully_shard_state_dict 1/1 ... [2024-08-20 22:09:29.054836] 2024-08-20T22:09:29.0568719Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:09:29.0570627Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:09:29.055245] 2024-08-20T22:09:31.8794244Z 2024-08-20T22:09:31.8796402Z distributed/_composable/fsdp/test_fully_shard_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_state_dict_1.1_7519eac36e32f3b5_.log 2024-08-20T22:09:31.8802610Z Running 6 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_2d_state_dict_correctness, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_dp_state_dict_cpu_offload, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_dp_state_dict_save_load, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_dp_tp_state_dict_save_load, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_hsdp_tp_state_dict_save_load, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiThread::test_rank0_offload_full_state_dict 2024-08-20T22:09:31.8807748Z 2024-08-20T22:09:31.8808238Z Running distributed/fsdp/test_fsdp_ignored_modules 1/1 ... [2024-08-20 22:09:31.879889] 2024-08-20T22:09:31.8808917Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:09:31.8810735Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_ignored_modules.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:09:31.880330] 2024-08-20T22:09:34.7047799Z 2024-08-20T22:09:34.7049833Z distributed/fsdp/test_fsdp_ignored_modules 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_ignored_modules_1.1_81483b6d89ed4ee5_.log 2024-08-20T22:09:34.7061483Z Running 13 items in this shard: test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_diff_ignored_modules_across_ranks, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_invalid_composable_False, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_invalid_composable_True, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_nested, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_nested_composable, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_False_composable_False, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_False_composable_True, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_True_composable_False, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_True_composable_True, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_transformer, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_transformer_composable, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_states_auto_wrap, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_states_check 2024-08-20T22:09:34.7070624Z 2024-08-20T22:09:34.7071082Z Running distributed/fsdp/test_fsdp_overlap 1/1 ... [2024-08-20 22:09:34.705270] 2024-08-20T22:09:34.7071715Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:09:34.7073501Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_overlap.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:09:34.705700] 2024-08-20T22:09:42.4871781Z 2024-08-20T22:09:42.4874551Z distributed/fsdp/test_fsdp_overlap 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_overlap_1.1_1b30ecdde56327a9_.log 2024-08-20T22:09:42.4878481Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_overlap.py::TestForwardOverlapWorldSizeOne::test_forward_overlap, test/distributed/fsdp/test_fsdp_overlap.py::TestForwardOverlapWorldSizeTwo::test_forward_overlap 2024-08-20T22:09:42.4880711Z 2024-08-20T22:09:42.4881626Z Running distributed/fsdp/test_fsdp_backward_prefetch 1/1 ... [2024-08-20 22:09:42.487526] 2024-08-20T22:09:42.4882740Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:09:42.4885802Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_backward_prefetch.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:09:42.488106] 2024-08-20T22:09:48.0164129Z 2024-08-20T22:09:48.0167063Z distributed/fsdp/test_fsdp_backward_prefetch 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_backward_prefetch_1.1_6e443f4c7d677724_.log 2024-08-20T22:09:48.0170505Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_backward_prefetch.py::TestBackwardPrefetch::test_backward_prefetch 2024-08-20T22:09:48.0171923Z 2024-08-20T22:09:48.0174015Z Running distributed/_shard/test_sharder 1/1 ... [2024-08-20 22:09:48.017151] 2024-08-20T22:09:48.0175001Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:09:48.0181166Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/test_sharder.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:09:48.017718] 2024-08-20T22:09:50.6430601Z 2024-08-20T22:09:50.6432291Z distributed/_shard/test_sharder 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.test_sharder_1.1_10347209540712c4_.log 2024-08-20T22:09:50.6434479Z Running 2 items in this shard: test/distributed/_shard/test_sharder.py::TestCustomSharder::test_custom_sharder, test/distributed/_shard/test_sharder.py::TestCustomSharder::test_custom_sharder_errors 2024-08-20T22:09:50.6435724Z 2024-08-20T22:09:50.6437231Z Running distributed/test_distributed_spawn 1/3 ... [2024-08-20 22:09:50.643502] 2024-08-20T22:09:50.6519370Z MPI not available -- MPI backend tests will be skipped 2024-08-20T22:09:50.6521248Z Running distributed tests for the test backend with env init_method 2024-08-20T22:09:50.6524399Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:09:50.6530276Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=1', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:09:50.652733] 2024-08-20T22:09:53.3368899Z 2024-08-20T22:09:53.3371140Z distributed/test_distributed_spawn 1/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_1.3_2c9725a09a4175c6_.log 2024-08-20T22:09:53.3372492Z Running 0 items in this shard: 2024-08-20T22:09:53.3372765Z 2024-08-20T22:09:53.3379048Z Running distributed tests for the test backend with file init_method 2024-08-20T22:09:53.3380583Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:09:53.3385961Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=1', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:09:53.338268] 2024-08-20T22:09:56.1874812Z 2024-08-20T22:09:56.1876851Z distributed/test_distributed_spawn 1/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_1.3_cf5dac0306bd68f9_.log 2024-08-20T22:09:56.1878119Z Running 0 items in this shard: 2024-08-20T22:09:56.1878441Z 2024-08-20T22:09:56.1883880Z Running distributed tests for the gloo backend with env init_method 2024-08-20T22:09:56.1885479Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:09:56.1890584Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=1', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:09:56.188757] 2024-08-20T22:18:56.2131934Z 2024-08-20T22:18:56.2133663Z distributed/test_distributed_spawn 1/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_1.3_d6080a3f1b6e59a9_.log 2024-08-20T22:18:56.2236621Z Running 102 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_1_level_hierarchical_model_averager_equivalent_to_periodic_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_3_level_hierarchical_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_Backend_enum_class, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_2D_Input, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Channels_Last, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_Running_Value, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_No_Affine, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_requires_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_cat_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_default_pg, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max_complex_unsupported, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo_tags, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_mixed_backend_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_list_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_object_list, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_with_logger, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping_requires_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_compile_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_create_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_forward_backward_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_pickling_powerSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_ignore_params_arg, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_cpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_ignored_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_execution_trace, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_python_error_logged, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_shared_grad_acc_unused_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_module_states, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_join_disable, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs_stop_iteration_sync_bn, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_default_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_no_find_unused, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_data_parallel_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_future, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_invalid_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_irecv, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_failure_order, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_init_abort, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_world_size_not_divisible_by_group_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_tuple_module, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_api_cpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_without_logger 2024-08-20T22:18:56.2296740Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_1_level_hierarchical_model_averager_equivalent_to_periodic_model_averager 2024-08-20T22:18:56.2298464Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_3_level_hierarchical_model_averager 2024-08-20T22:18:56.2299913Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_Backend_enum_class 2024-08-20T22:18:56.2301298Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU 2024-08-20T22:18:56.2302849Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_2D_Input 2024-08-20T22:18:56.2304530Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Channels_Last 2024-08-20T22:18:56.2306314Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_Running_Value 2024-08-20T22:18:56.2308083Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_No_Affine 2024-08-20T22:18:56.2309691Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_requires_grad 2024-08-20T22:18:56.2311202Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync 2024-08-20T22:18:56.2312823Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_hook 2024-08-20T22:18:56.2314411Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_grad_is_view 2024-08-20T22:18:56.2315842Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda 2024-08-20T22:18:56.2317223Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_cat_tensor_cuda 2024-08-20T22:18:56.2318948Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_default_pg 2024-08-20T22:18:56.2320359Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_subgroup 2024-08-20T22:18:56.2321818Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_max 2024-08-20T22:18:56.2323329Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_min 2024-08-20T22:18:56.2324852Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_product 2024-08-20T22:18:56.2326573Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_sum 2024-08-20T22:18:56.2328069Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_max 2024-08-20T22:18:56.2329535Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_min 2024-08-20T22:18:56.2330966Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max 2024-08-20T22:18:56.2332534Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max_complex_unsupported 2024-08-20T22:18:56.2334027Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_min 2024-08-20T22:18:56.2335422Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_max 2024-08-20T22:18:56.2336822Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_min 2024-08-20T22:18:56.2338192Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_max 2024-08-20T22:18:56.2339516Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_max 2024-08-20T22:18:56.2340818Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_product 2024-08-20T22:18:56.2342129Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum 2024-08-20T22:18:56.2343427Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_async 2024-08-20T22:18:56.2344804Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_async 2024-08-20T22:18:56.2346190Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group 2024-08-20T22:18:56.2347576Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group_cuda 2024-08-20T22:18:56.2348913Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group 2024-08-20T22:18:56.2350282Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split 2024-08-20T22:18:56.2351783Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda_complex 2024-08-20T22:18:56.2353323Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group 2024-08-20T22:18:56.2355037Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group_cuda 2024-08-20T22:18:56.2356632Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group 2024-08-20T22:18:56.2358550Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group_cuda 2024-08-20T22:18:56.2360001Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_full_group 2024-08-20T22:18:56.2361272Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier 2024-08-20T22:18:56.2362513Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_cuda 2024-08-20T22:18:56.2363853Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group_cuda 2024-08-20T22:18:56.2365228Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_group 2024-08-20T22:18:56.2366624Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo_tags 2024-08-20T22:18:56.2368087Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_mixed_backend_err 2024-08-20T22:18:56.2369559Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_list_err 2024-08-20T22:18:56.2370882Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast 2024-08-20T22:18:56.2372321Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_object_list 2024-08-20T22:18:56.2373681Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager 2024-08-20T22:18:56.2375053Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager_async 2024-08-20T22:18:56.2376610Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_with_logger 2024-08-20T22:18:56.2378249Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping 2024-08-20T22:18:56.2379867Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping_requires_grad 2024-08-20T22:18:56.2381403Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_compile_static_graph 2024-08-20T22:18:56.2382751Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_create_graph 2024-08-20T22:18:56.2384106Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_forward_backward_hook 2024-08-20T22:18:56.2385517Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_pickling_powerSGD 2024-08-20T22:18:56.2386906Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_ignore_params_arg 2024-08-20T22:18:56.2388249Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_cpu 2024-08-20T22:18:56.2389709Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_ignored_params 2024-08-20T22:18:56.2391405Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_execution_trace 2024-08-20T22:18:56.2392813Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_python_error_logged 2024-08-20T22:18:56.2394252Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_shared_grad_acc_unused_params 2024-08-20T22:18:56.2395684Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_module_states 2024-08-20T22:18:56.2397095Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_join_disable 2024-08-20T22:18:56.2398464Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs 2024-08-20T22:18:56.2399911Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs_stop_iteration_sync_bn 2024-08-20T22:18:56.2401466Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_default_group 2024-08-20T22:18:56.2403008Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_no_find_unused 2024-08-20T22:18:56.2404422Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_full_group 2024-08-20T22:18:56.2405714Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_group 2024-08-20T22:18:56.2407045Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_data_parallel_params 2024-08-20T22:18:56.2408368Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_future 2024-08-20T22:18:56.2409595Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank 2024-08-20T22:18:56.2410883Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_invalid_static_graph 2024-08-20T22:18:56.2412240Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_irecv 2024-08-20T22:18:56.2413534Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_autograd_profiler 2024-08-20T22:18:56.2414948Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_failure_order 2024-08-20T22:18:56.2416379Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo 2024-08-20T22:18:56.2417818Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_subgroup 2024-08-20T22:18:56.2419250Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allreduce 2024-08-20T22:18:56.2420602Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_init_abort 2024-08-20T22:18:56.2421886Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups 2024-08-20T22:18:56.2423362Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_world_size_not_divisible_by_group_size 2024-08-20T22:18:56.2424937Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_tuple_module 2024-08-20T22:18:56.2426578Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_product 2024-08-20T22:18:56.2428075Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_min 2024-08-20T22:18:56.2429379Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_product 2024-08-20T22:18:56.2430631Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum 2024-08-20T22:18:56.2431899Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda 2024-08-20T22:18:56.2433180Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda 2024-08-20T22:18:56.2434459Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl 2024-08-20T22:18:56.2435831Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_autograd_profiler 2024-08-20T22:18:56.2437300Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_torch_profiler 2024-08-20T22:18:56.2438731Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_torch_profiler 2024-08-20T22:18:56.2440190Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_autograd_profiler 2024-08-20T22:18:56.2441621Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_api_cpu 2024-08-20T22:18:56.2443077Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_without_logger 2024-08-20T22:18:56.2443932Z 2024-08-20T22:18:56.2444232Z Running distributed tests for the gloo backend with file init_method 2024-08-20T22:18:56.2444811Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:18:56.2446837Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=1', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:18:56.219290] 2024-08-20T22:27:54.6837499Z 2024-08-20T22:27:54.6839232Z distributed/test_distributed_spawn 1/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_1.3_88af2a339da3dc18_.log 2024-08-20T22:27:54.6918564Z Running 102 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_1_level_hierarchical_model_averager_equivalent_to_periodic_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_3_level_hierarchical_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_Backend_enum_class, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_2D_Input, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Channels_Last, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_Running_Value, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_No_Affine, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_requires_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_cat_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_default_pg, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max_complex_unsupported, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo_tags, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_mixed_backend_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_list_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_object_list, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_with_logger, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping_requires_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_compile_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_create_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_forward_backward_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_pickling_powerSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_ignore_params_arg, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_cpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_ignored_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_execution_trace, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_python_error_logged, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_shared_grad_acc_unused_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_module_states, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_join_disable, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs_stop_iteration_sync_bn, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_default_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_no_find_unused, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_data_parallel_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_future, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_invalid_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_irecv, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_failure_order, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_init_abort, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_world_size_not_divisible_by_group_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_tuple_module, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_api_cpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_without_logger 2024-08-20T22:27:54.6978266Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_1_level_hierarchical_model_averager_equivalent_to_periodic_model_averager 2024-08-20T22:27:54.6979980Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_3_level_hierarchical_model_averager 2024-08-20T22:27:54.6981411Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_Backend_enum_class 2024-08-20T22:27:54.6982799Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU 2024-08-20T22:27:54.6984354Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_2D_Input 2024-08-20T22:27:54.6986055Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Channels_Last 2024-08-20T22:27:54.6987849Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_Running_Value 2024-08-20T22:27:54.6989610Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_No_Affine 2024-08-20T22:27:54.6991212Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_requires_grad 2024-08-20T22:27:54.6992714Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync 2024-08-20T22:27:54.6994240Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_hook 2024-08-20T22:27:54.6995823Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_grad_is_view 2024-08-20T22:27:54.6997241Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda 2024-08-20T22:27:54.6998621Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_cat_tensor_cuda 2024-08-20T22:27:54.7000071Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_default_pg 2024-08-20T22:27:54.7001478Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_subgroup 2024-08-20T22:27:54.7002930Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_max 2024-08-20T22:27:54.7004654Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_min 2024-08-20T22:27:54.7006174Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_product 2024-08-20T22:27:54.7007689Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_sum 2024-08-20T22:27:54.7009164Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_max 2024-08-20T22:27:54.7010628Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_min 2024-08-20T22:27:54.7012132Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max 2024-08-20T22:27:54.7013630Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max_complex_unsupported 2024-08-20T22:27:54.7015127Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_min 2024-08-20T22:27:54.7016526Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_max 2024-08-20T22:27:54.7017932Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_min 2024-08-20T22:27:54.7019299Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_max 2024-08-20T22:27:54.7020618Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_max 2024-08-20T22:27:54.7021936Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_product 2024-08-20T22:27:54.7023239Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum 2024-08-20T22:27:54.7024547Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_async 2024-08-20T22:27:54.7025922Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_async 2024-08-20T22:27:54.7027288Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group 2024-08-20T22:27:54.7028674Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group_cuda 2024-08-20T22:27:54.7030015Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group 2024-08-20T22:27:54.7031398Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split 2024-08-20T22:27:54.7032896Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda_complex 2024-08-20T22:27:54.7034436Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group 2024-08-20T22:27:54.7036016Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group_cuda 2024-08-20T22:27:54.7037597Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group 2024-08-20T22:27:54.7039226Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group_cuda 2024-08-20T22:27:54.7040713Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_full_group 2024-08-20T22:27:54.7041985Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier 2024-08-20T22:27:54.7043232Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_cuda 2024-08-20T22:27:54.7044564Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group_cuda 2024-08-20T22:27:54.7045925Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_group 2024-08-20T22:27:54.7047317Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo_tags 2024-08-20T22:27:54.7048787Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_mixed_backend_err 2024-08-20T22:27:54.7050261Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_list_err 2024-08-20T22:27:54.7051668Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast 2024-08-20T22:27:54.7052965Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_object_list 2024-08-20T22:27:54.7054314Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager 2024-08-20T22:27:54.7055675Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager_async 2024-08-20T22:27:54.7057239Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_with_logger 2024-08-20T22:27:54.7059056Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping 2024-08-20T22:27:54.7060658Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping_requires_grad 2024-08-20T22:27:54.7062185Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_compile_static_graph 2024-08-20T22:27:54.7063525Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_create_graph 2024-08-20T22:27:54.7064900Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_forward_backward_hook 2024-08-20T22:27:54.7066315Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_pickling_powerSGD 2024-08-20T22:27:54.7067701Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_ignore_params_arg 2024-08-20T22:27:54.7069047Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_cpu 2024-08-20T22:27:54.7070513Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_ignored_params 2024-08-20T22:27:54.7072027Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_execution_trace 2024-08-20T22:27:54.7073429Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_python_error_logged 2024-08-20T22:27:54.7074863Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_shared_grad_acc_unused_params 2024-08-20T22:27:54.7076482Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_module_states 2024-08-20T22:27:54.7077903Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_join_disable 2024-08-20T22:27:54.7079277Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs 2024-08-20T22:27:54.7080714Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs_stop_iteration_sync_bn 2024-08-20T22:27:54.7082260Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_default_group 2024-08-20T22:27:54.7083802Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_no_find_unused 2024-08-20T22:27:54.7085223Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_full_group 2024-08-20T22:27:54.7086514Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_group 2024-08-20T22:27:54.7087847Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_data_parallel_params 2024-08-20T22:27:54.7089170Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_future 2024-08-20T22:27:54.7090389Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank 2024-08-20T22:27:54.7091751Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_invalid_static_graph 2024-08-20T22:27:54.7093029Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_irecv 2024-08-20T22:27:54.7094337Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_autograd_profiler 2024-08-20T22:27:54.7095754Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_failure_order 2024-08-20T22:27:54.7097183Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo 2024-08-20T22:27:54.7098602Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_subgroup 2024-08-20T22:27:54.7100022Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allreduce 2024-08-20T22:27:54.7101374Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_init_abort 2024-08-20T22:27:54.7102677Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups 2024-08-20T22:27:54.7104148Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_world_size_not_divisible_by_group_size 2024-08-20T22:27:54.7105719Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_tuple_module 2024-08-20T22:27:54.7107180Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_product 2024-08-20T22:27:54.7108529Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_min 2024-08-20T22:27:54.7109825Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_product 2024-08-20T22:27:54.7111131Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum 2024-08-20T22:27:54.7112446Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda 2024-08-20T22:27:54.7113732Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda 2024-08-20T22:27:54.7115013Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl 2024-08-20T22:27:54.7116384Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_autograd_profiler 2024-08-20T22:27:54.7117844Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_torch_profiler 2024-08-20T22:27:54.7119263Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_torch_profiler 2024-08-20T22:27:54.7120733Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_autograd_profiler 2024-08-20T22:27:54.7122158Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_api_cpu 2024-08-20T22:27:54.7123610Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_without_logger 2024-08-20T22:27:54.7124459Z 2024-08-20T22:27:54.7125047Z Running distributed/test_distributed_spawn 3/3 ... [2024-08-20 22:27:54.689087] 2024-08-20T22:27:54.7125817Z MPI not available -- MPI backend tests will be skipped 2024-08-20T22:27:54.7126459Z Running distributed tests for the test backend with env init_method 2024-08-20T22:27:54.7127042Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:27:54.7128899Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=3', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:27:54.696707] 2024-08-20T22:27:57.3771369Z 2024-08-20T22:27:57.3773306Z distributed/test_distributed_spawn 3/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_3.3_c78e95d266aa742d_.log 2024-08-20T22:27:57.3774504Z Running 0 items in this shard: 2024-08-20T22:27:57.3774767Z 2024-08-20T22:27:57.3780108Z Running distributed tests for the test backend with file init_method 2024-08-20T22:27:57.3783634Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:27:57.3786365Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=3', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:27:57.378330] 2024-08-20T22:28:00.0533945Z 2024-08-20T22:28:00.0535922Z distributed/test_distributed_spawn 3/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_3.3_57ec608eb9eabaed_.log 2024-08-20T22:28:00.0537122Z Running 0 items in this shard: 2024-08-20T22:28:00.0537372Z 2024-08-20T22:28:00.0542693Z Running distributed tests for the gloo backend with env init_method 2024-08-20T22:28:00.0544325Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:28:00.0549179Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=3', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:28:00.054634] 2024-08-20T22:36:22.1826167Z 2024-08-20T22:36:22.1828272Z distributed/test_distributed_spawn 3/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_3.3_3e53fe4f4e3527d0_.log 2024-08-20T22:36:22.1895085Z Running 91 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_gradient, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_non_default_stream, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_with_amp_and_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_stack_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_average_parameters, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_no_rank_zero_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_ring_exchange_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_self_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_without_logger, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_grad_as_bucket_view_false, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer_via_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_different_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_same_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce_process_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_post_localSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_gpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_num_params_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_shape_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_error, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_namedtuple, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_set_grad_to_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_no_set_grad_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_remove_autograd_hooks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_returns_tensor_with_no_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_static_graph_nested_types, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_grad_undefined, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_new_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_different_graph_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_checks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_wait_all_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allgather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_reduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_high_priority_stream, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_negative_input_rank, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_group_size_exceeds_world_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager_param_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda_twice, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_checks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_object_list, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_multi_forward, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_undefined_grad_parity_unused_parameters, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_with_logger 2024-08-20T22:36:22.1946845Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU_grad_is_view 2024-08-20T22:36:22.1948559Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_gradient 2024-08-20T22:36:22.1950255Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_non_default_stream 2024-08-20T22:36:22.1951964Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_with_amp_and_grad_is_view 2024-08-20T22:36:22.1962183Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather 2024-08-20T22:36:22.1963562Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_complex 2024-08-20T22:36:22.1965000Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_group 2024-08-20T22:36:22.1966381Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_full_group 2024-08-20T22:36:22.1967721Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_group 2024-08-20T22:36:22.1969136Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_stack_tensor_cuda 2024-08-20T22:36:22.1970640Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_product 2024-08-20T22:36:22.1972190Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_sum 2024-08-20T22:36:22.1973624Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_sum 2024-08-20T22:36:22.1975058Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_product 2024-08-20T22:36:22.1976476Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_product 2024-08-20T22:36:22.1978115Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_sum 2024-08-20T22:36:22.1979435Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_min 2024-08-20T22:36:22.1980793Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_complex 2024-08-20T22:36:22.1982190Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda_complex 2024-08-20T22:36:22.1983548Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group_cuda 2024-08-20T22:36:22.1984980Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group 2024-08-20T22:36:22.1986507Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group_cuda 2024-08-20T22:36:22.1988049Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda 2024-08-20T22:36:22.1989454Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_average_parameters 2024-08-20T22:36:22.1990788Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group 2024-08-20T22:36:22.1992123Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group_cuda 2024-08-20T22:36:22.1993505Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_full_group 2024-08-20T22:36:22.1994892Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo 2024-08-20T22:36:22.1996337Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_no_rank_zero_nccl 2024-08-20T22:36:22.1997846Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_ring_exchange_nccl 2024-08-20T22:36:22.1999305Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_self_nccl 2024-08-20T22:36:22.2000697Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_full_group 2024-08-20T22:36:22.2002264Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_without_logger 2024-08-20T22:36:22.2003998Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_grad_as_bucket_view_false 2024-08-20T22:36:22.2005557Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer_via_hook 2024-08-20T22:36:22.2006982Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce 2024-08-20T22:36:22.2008454Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_different_across_ranks 2024-08-20T22:36:22.2009975Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_same_across_ranks 2024-08-20T22:36:22.2011328Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device 2024-08-20T22:36:22.2012795Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce_process_group 2024-08-20T22:36:22.2014413Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_post_localSGD 2024-08-20T22:36:22.2015819Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_gpu 2024-08-20T22:36:22.2017249Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_num_params_across_ranks 2024-08-20T22:36:22.2018762Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_shape_across_ranks 2024-08-20T22:36:22.2020280Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_error 2024-08-20T22:36:22.2021693Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_namedtuple 2024-08-20T22:36:22.2023224Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_set_grad_to_none 2024-08-20T22:36:22.2025013Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_no_set_grad_none 2024-08-20T22:36:22.2026596Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd 2024-08-20T22:36:22.2028023Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd_static_graph 2024-08-20T22:36:22.2029459Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_remove_autograd_hooks 2024-08-20T22:36:22.2030898Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_returns_tensor_with_no_grad 2024-08-20T22:36:22.2032364Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_static_graph_nested_types 2024-08-20T22:36:22.2033865Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_grad_undefined 2024-08-20T22:36:22.2035377Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_new_group 2024-08-20T22:36:22.2036838Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_different_graph_across_ranks 2024-08-20T22:36:22.2038158Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather 2024-08-20T22:36:22.2039408Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_checks 2024-08-20T22:36:22.2040680Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object 2024-08-20T22:36:22.2042000Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_group 2024-08-20T22:36:22.2043418Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang 2024-08-20T22:36:22.2044897Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_wait_all_ranks 2024-08-20T22:36:22.2046334Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allgather 2024-08-20T22:36:22.2047750Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_reduce 2024-08-20T22:36:22.2049159Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_high_priority_stream 2024-08-20T22:36:22.2050785Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_negative_input_rank 2024-08-20T22:36:22.2052484Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_group_size_exceeds_world_size 2024-08-20T22:36:22.2054133Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager_param_group 2024-08-20T22:36:22.2055627Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity 2024-08-20T22:36:22.2057149Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_grad_is_view 2024-08-20T22:36:22.2059011Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd_grad_is_view 2024-08-20T22:36:22.2060608Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_min 2024-08-20T22:36:22.2061973Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_sum 2024-08-20T22:36:22.2063301Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_max 2024-08-20T22:36:22.2064578Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_min 2024-08-20T22:36:22.2065917Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_tensor_cuda 2024-08-20T22:36:22.2067309Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda_twice 2024-08-20T22:36:22.2068591Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter 2024-08-20T22:36:22.2069845Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_checks 2024-08-20T22:36:22.2071148Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_complex 2024-08-20T22:36:22.2072950Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda_complex 2024-08-20T22:36:22.2074290Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_full_group 2024-08-20T22:36:22.2075639Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_object_list 2024-08-20T22:36:22.2076929Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv 2024-08-20T22:36:22.2078332Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_autograd_profiler 2024-08-20T22:36:22.2079754Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag 2024-08-20T22:36:22.2081169Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_torch_profiler 2024-08-20T22:36:22.2082618Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum_cuda 2024-08-20T22:36:22.2084025Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_multi_forward 2024-08-20T22:36:22.2085509Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_undefined_grad_parity_unused_parameters 2024-08-20T22:36:22.2087632Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_with_logger 2024-08-20T22:36:22.2088474Z 2024-08-20T22:36:22.2088771Z Running distributed tests for the gloo backend with file init_method 2024-08-20T22:36:22.2089344Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:36:22.2091335Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=3', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:36:22.187022] 2024-08-20T22:44:46.5695881Z 2024-08-20T22:44:46.5697509Z distributed/test_distributed_spawn 3/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_3.3_c27d028217ec6b28_.log 2024-08-20T22:44:46.5766927Z Running 91 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_gradient, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_non_default_stream, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_with_amp_and_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_stack_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_average_parameters, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_no_rank_zero_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_ring_exchange_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_self_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_without_logger, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_grad_as_bucket_view_false, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer_via_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_different_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_same_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce_process_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_post_localSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_gpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_num_params_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_shape_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_error, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_namedtuple, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_set_grad_to_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_no_set_grad_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_remove_autograd_hooks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_returns_tensor_with_no_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_static_graph_nested_types, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_grad_undefined, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_new_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_different_graph_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_checks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_wait_all_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allgather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_reduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_high_priority_stream, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_negative_input_rank, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_group_size_exceeds_world_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager_param_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda_twice, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_checks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_object_list, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_multi_forward, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_undefined_grad_parity_unused_parameters, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_with_logger 2024-08-20T22:44:46.5819126Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU_grad_is_view 2024-08-20T22:44:46.5820893Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_gradient 2024-08-20T22:44:46.5822717Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_non_default_stream 2024-08-20T22:44:46.5824365Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_with_amp_and_grad_is_view 2024-08-20T22:44:46.5825813Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather 2024-08-20T22:44:46.5827155Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_complex 2024-08-20T22:44:46.5828589Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_group 2024-08-20T22:44:46.5829977Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_full_group 2024-08-20T22:44:46.5831317Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_group 2024-08-20T22:44:46.5832720Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_stack_tensor_cuda 2024-08-20T22:44:46.5834368Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_product 2024-08-20T22:44:46.5835846Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_sum 2024-08-20T22:44:46.5837278Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_sum 2024-08-20T22:44:46.5838707Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_product 2024-08-20T22:44:46.5840140Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_product 2024-08-20T22:44:46.5841499Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_sum 2024-08-20T22:44:46.5842896Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_min 2024-08-20T22:44:46.5844260Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_complex 2024-08-20T22:44:46.5845662Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda_complex 2024-08-20T22:44:46.5847025Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group_cuda 2024-08-20T22:44:46.5848476Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group 2024-08-20T22:44:46.5850009Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group_cuda 2024-08-20T22:44:46.5851549Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda 2024-08-20T22:44:46.5853024Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_average_parameters 2024-08-20T22:44:46.5854365Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group 2024-08-20T22:44:46.5855697Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group_cuda 2024-08-20T22:44:46.5857065Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_full_group 2024-08-20T22:44:46.5858638Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo 2024-08-20T22:44:46.5860072Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_no_rank_zero_nccl 2024-08-20T22:44:46.5861596Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_ring_exchange_nccl 2024-08-20T22:44:46.5863062Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_self_nccl 2024-08-20T22:44:46.5864457Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_full_group 2024-08-20T22:44:46.5866020Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_without_logger 2024-08-20T22:44:46.5867744Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_grad_as_bucket_view_false 2024-08-20T22:44:46.5869299Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer_via_hook 2024-08-20T22:44:46.5871463Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce 2024-08-20T22:44:46.5872950Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_different_across_ranks 2024-08-20T22:44:46.5874474Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_same_across_ranks 2024-08-20T22:44:46.5875832Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device 2024-08-20T22:44:46.5877230Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce_process_group 2024-08-20T22:44:46.5878727Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_post_localSGD 2024-08-20T22:44:46.5880140Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_gpu 2024-08-20T22:44:46.5881591Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_num_params_across_ranks 2024-08-20T22:44:46.5883099Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_shape_across_ranks 2024-08-20T22:44:46.5884606Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_error 2024-08-20T22:44:46.5886020Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_namedtuple 2024-08-20T22:44:46.5887545Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_set_grad_to_none 2024-08-20T22:44:46.5889343Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_no_set_grad_none 2024-08-20T22:44:46.5890934Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd 2024-08-20T22:44:46.5892468Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd_static_graph 2024-08-20T22:44:46.5893906Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_remove_autograd_hooks 2024-08-20T22:44:46.5895343Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_returns_tensor_with_no_grad 2024-08-20T22:44:46.5896796Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_static_graph_nested_types 2024-08-20T22:44:46.5898301Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_grad_undefined 2024-08-20T22:44:46.5899812Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_new_group 2024-08-20T22:44:46.5901282Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_different_graph_across_ranks 2024-08-20T22:44:46.5902595Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather 2024-08-20T22:44:46.5903836Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_checks 2024-08-20T22:44:46.5905095Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object 2024-08-20T22:44:46.5906513Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_group 2024-08-20T22:44:46.5907922Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang 2024-08-20T22:44:46.5909383Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_wait_all_ranks 2024-08-20T22:44:46.5910829Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allgather 2024-08-20T22:44:46.5912234Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_reduce 2024-08-20T22:44:46.5913643Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_high_priority_stream 2024-08-20T22:44:46.5915150Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_negative_input_rank 2024-08-20T22:44:46.5916760Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_group_size_exceeds_world_size 2024-08-20T22:44:46.5918302Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager_param_group 2024-08-20T22:44:46.5919792Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity 2024-08-20T22:44:46.5921317Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_grad_is_view 2024-08-20T22:44:46.5923025Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd_grad_is_view 2024-08-20T22:44:46.5924620Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_min 2024-08-20T22:44:46.5925978Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_sum 2024-08-20T22:44:46.5927297Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_max 2024-08-20T22:44:46.5928572Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_min 2024-08-20T22:44:46.5929900Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_tensor_cuda 2024-08-20T22:44:46.5931295Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda_twice 2024-08-20T22:44:46.5932669Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter 2024-08-20T22:44:46.5933930Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_checks 2024-08-20T22:44:46.5935219Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_complex 2024-08-20T22:44:46.5936549Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda_complex 2024-08-20T22:44:46.5937875Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_full_group 2024-08-20T22:44:46.5939215Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_object_list 2024-08-20T22:44:46.5940506Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv 2024-08-20T22:44:46.5941868Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_autograd_profiler 2024-08-20T22:44:46.5943575Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag 2024-08-20T22:44:46.5944985Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_torch_profiler 2024-08-20T22:44:46.5946438Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum_cuda 2024-08-20T22:44:46.5947831Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_multi_forward 2024-08-20T22:44:46.5949306Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_undefined_grad_parity_unused_parameters 2024-08-20T22:44:46.5950836Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_with_logger 2024-08-20T22:44:46.5951675Z 2024-08-20T22:44:46.5952213Z Running distributed/fsdp/test_wrap 1/1 ... [2024-08-20 22:44:46.574430] 2024-08-20T22:44:46.5952789Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:44:46.5954522Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_wrap.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:44:46.574830] 2024-08-20T22:46:01.4536027Z 2024-08-20T22:46:01.4537948Z distributed/fsdp/test_wrap 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_wrap_1.1_3a92b25a0649d83c_.log 2024-08-20T22:46:01.4568825Z Running 51 items in this shard: test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_bn_always_wrapped_individually, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_False_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_False_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_True_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_True_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_False_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_False_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_True_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_True_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_False_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_False_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_True_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_True_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_False_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_False_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_True_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_True_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_False_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_False_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_True_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_True_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_wrap_batchnorm_individually_use_or_policy_False, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_wrap_batchnorm_individually_use_or_policy_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_always_wrap, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_always_wrap_with_ignored_modules_wrap_method0, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_always_wrap_with_ignored_modules_wrap_method1, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_api, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_exclude_wrap, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_exclude_wrap_include_children, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_force_leaf, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_force_leaf_custom, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode0_cpu_offload0_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode0_cpu_offload0_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode0_cpu_offload1_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode0_cpu_offload1_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode1_cpu_offload0_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode1_cpu_offload0_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode1_cpu_offload1_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode1_cpu_offload1_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_with_ignored_modules_wrap_method0, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_with_ignored_modules_wrap_method1, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_custom_policy, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_frozen_params, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_module_wrap_policy, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_module_wrap_policy_callable, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_transformer_auto_wrap_policy, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_disabled_outside_context, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_override_defaults, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_wrap_method0, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_wrap_method1, test/distributed/fsdp/test_wrap.py::TestWrapUtils::test_validate_frozen_params 2024-08-20T22:46:01.4598438Z 2024-08-20T22:46:01.4598917Z Running distributed/fsdp/test_fsdp_misc 1/1 ... [2024-08-20 22:46:01.454083] 2024-08-20T22:46:01.4599542Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:46:01.4601299Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_misc.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:46:01.454512] 2024-08-20T22:46:41.7347249Z 2024-08-20T22:46:41.7348970Z distributed/fsdp/test_fsdp_misc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_misc_1.1_64f8a1c3ef646ecc_.log 2024-08-20T22:46:41.7368147Z Running 28 items in this shard: test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_cpu_init_with_sync_module_states, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_cpu_init_stays_on_cpu, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_cpu_training, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_device_id_use_index_False, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_device_id_use_index_True, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_not_all_outputs_used_in_loss, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_optim_overlap_no_use_orig_params_error, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_optimizer_overlap, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_zero2_eval_with_prefetch, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_cpu_gpu_module, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_device_id_auto_wrap, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_device_id_cpu_offload, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_device_id_no_move_ignored_params_and_bufs, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_ignored_module_meta, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_namedtuple, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_same_model_across_ranks, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_unsupported_module_cls, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_homogeneous_attributes, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_module_device_mismatches_device_id, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_multigpu_module, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_no_params, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscWorldSize1::test_training_device_mismatch_errors, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscWorldSize1::test_unsafe_setattr, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscWorldSize1::test_world_size_1_sharding_strategy_warning 2024-08-20T22:46:41.7385336Z 2024-08-20T22:46:41.7385911Z Running distributed/fsdp/test_fsdp_comm_hooks 1/1 ... [2024-08-20 22:46:41.735081] 2024-08-20T22:46:41.7386547Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:46:41.7388348Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_comm_hooks.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:46:41.735498] 2024-08-20T22:47:30.7528555Z 2024-08-20T22:47:30.7530319Z distributed/fsdp/test_fsdp_comm_hooks 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_comm_hooks_1.1_3846ef2b6a2460a8_.log 2024-08-20T22:47:30.7552798Z Running 28 items in this shard: test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_False_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_True_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_behavior_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_behavior_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_behavior_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_False_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_True_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_False_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_True_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_hybrid_strategy, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_non_root_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_non_root_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_non_root_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_submodules_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_submodules_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_submodules_sharding_strategy2 2024-08-20T22:47:30.7573585Z 2024-08-20T22:47:30.7574104Z Running distributed/fsdp/test_fsdp_unshard_params 1/1 ... [2024-08-20 22:47:30.753373] 2024-08-20T22:47:30.7574781Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:47:30.7576572Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_unshard_params.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:47:30.753796] 2024-08-20T22:48:12.6871848Z 2024-08-20T22:48:12.6874820Z distributed/fsdp/test_fsdp_unshard_params 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_unshard_params_1.1_a7a0f888097365bb_.log 2024-08-20T22:48:12.6887085Z Running 15 items in this shard: test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_named_parameters_and_buffers, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_param_data, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_recurse, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_respects_reshard, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_writeback, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_singleton_param_writeback, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_submodule, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_with_grads_core, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_with_grads_none_grads, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsNoShard::test_unshard_params_param_data_no_shard, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsNoShard::test_unshard_params_writeback_no_shard, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_offload_to_cpu_no_shard_raises, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_rank0_only_with_writeback_raises, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_unshard_params_from_backward_raises, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_unshard_params_from_forward_raises 2024-08-20T22:48:12.6896124Z 2024-08-20T22:48:12.6896574Z Running distributed/test_c10d_common 1/1 ... [2024-08-20 22:48:12.687629] 2024-08-20T22:48:12.6897177Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:48:12.6898977Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_common.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:48:12.688118] 2024-08-20T22:50:32.8559757Z 2024-08-20T22:50:32.8561630Z distributed/test_c10d_common 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_common_1.1_01e9c19590ff535c_.log 2024-08-20T22:50:32.8577299Z Running 23 items in this shard: test/distributed/test_c10d_common.py::TimeoutTest::test_store_based_barrier, test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_multi_limit_multi_dtype, test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_multi_limit_single_dtype, test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_single_limit_multi_dtype, test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_single_limit_single_dtype, test/distributed/test_c10d_common.py::CommTest::test_debug_level, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_backend_class_attr, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_backend_config, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_collectives, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_get_backend_name, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_init_process_group_with_multiple_backends, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_is_backend_available, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_send_recv, test/distributed/test_c10d_common.py::ProcessGroupWithDispatchedCollectivesTests::test_init_process_group_for_all_backends, test/distributed/test_c10d_common.py::ProcessGroupWithDispatchedCollectivesTests::test_init_process_group_optional_backend, test/distributed/test_c10d_common.py::ReduceOpTest::test_op_isinstance_of_reduceop, test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_copyable, test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_equal, test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_pickle, test/distributed/test_c10d_common.py::LocalRankTest::testNodeLocalRank, test/distributed/test_c10d_common.py::LocalRankTest::testNodeLocalRankOverridesFallback, test/distributed/test_c10d_common.py::LocalRankTest::testWithoutEnv, test/distributed/test_c10d_common.py::LocalRankTest::testWithoutEnvWithFallback 2024-08-20T22:50:32.8589620Z Running 1 items in this shard: test/distributed/test_c10d_common.py::TimeoutTest::test_store_based_barrier 2024-08-20T22:50:32.8590872Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_multi_limit_multi_dtype 2024-08-20T22:50:32.8592238Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_multi_limit_single_dtype 2024-08-20T22:50:32.8593619Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_single_limit_multi_dtype 2024-08-20T22:50:32.8594992Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_single_limit_single_dtype 2024-08-20T22:50:32.8596199Z Running 1 items in this shard: test/distributed/test_c10d_common.py::CommTest::test_debug_level 2024-08-20T22:50:32.8597391Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_backend_class_attr 2024-08-20T22:50:32.8598742Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_backend_config 2024-08-20T22:50:32.8600032Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_collectives 2024-08-20T22:50:32.8601358Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_get_backend_name 2024-08-20T22:50:32.8602830Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_init_process_group_with_multiple_backends 2024-08-20T22:50:32.8604324Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_is_backend_available 2024-08-20T22:50:32.8605659Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_send_recv 2024-08-20T22:50:32.8607137Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ProcessGroupWithDispatchedCollectivesTests::test_init_process_group_for_all_backends 2024-08-20T22:50:32.8608835Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ProcessGroupWithDispatchedCollectivesTests::test_init_process_group_optional_backend 2024-08-20T22:50:32.8610280Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ReduceOpTest::test_op_isinstance_of_reduceop 2024-08-20T22:50:32.8611425Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_copyable 2024-08-20T22:50:32.8612595Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_equal 2024-08-20T22:50:32.8613686Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_pickle 2024-08-20T22:50:32.8614758Z Running 1 items in this shard: test/distributed/test_c10d_common.py::LocalRankTest::testNodeLocalRank 2024-08-20T22:50:32.8615942Z Running 1 items in this shard: test/distributed/test_c10d_common.py::LocalRankTest::testNodeLocalRankOverridesFallback 2024-08-20T22:50:32.8617092Z Running 1 items in this shard: test/distributed/test_c10d_common.py::LocalRankTest::testWithoutEnv 2024-08-20T22:50:32.8618196Z Running 1 items in this shard: test/distributed/test_c10d_common.py::LocalRankTest::testWithoutEnvWithFallback 2024-08-20T22:50:32.8618941Z 2024-08-20T22:50:32.8619571Z Running distributed/fsdp/test_fsdp_clip_grad_norm 1/1 ... [2024-08-20 22:50:32.857102] 2024-08-20T22:50:32.8620236Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:50:32.8622047Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_clip_grad_norm.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:50:32.857609] 2024-08-20T22:50:47.2050155Z 2024-08-20T22:50:47.2052135Z distributed/fsdp/test_fsdp_clip_grad_norm 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_clip_grad_norm_1.1_673cf73577639c4d_.log 2024-08-20T22:50:47.2055649Z Running 4 items in this shard: test/distributed/fsdp/test_fsdp_clip_grad_norm.py::TestClipGradNorm::test_ddp_parity, test/distributed/fsdp/test_fsdp_clip_grad_norm.py::TestClipGradNorm::test_low_precision_grads, test/distributed/fsdp/test_fsdp_clip_grad_norm.py::TestClipGradNorm::test_no_gradients, test/distributed/fsdp/test_fsdp_clip_grad_norm.py::TestClipGradNorm::test_non_root 2024-08-20T22:50:47.2059195Z 2024-08-20T22:50:47.2059638Z Running distributed/fsdp/test_fsdp_meta 1/1 ... [2024-08-20 22:50:47.205489] 2024-08-20T22:50:47.2060321Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:50:47.2063394Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_meta.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:50:47.205983] 2024-08-20T22:51:11.1639024Z 2024-08-20T22:51:11.1640858Z distributed/fsdp/test_fsdp_meta 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_meta_1.1_dc6040bc516a9bb0_.log 2024-08-20T22:51:11.1652474Z Running 15 items in this shard: test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_bad_arg_meta, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_bad_arg_torchdistx, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_meta_device_with_mixed_precision, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_meta_device_default_init_auto_wrap_False, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_meta_device_default_init_auto_wrap_True, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_meta_device_reset_params_auto_wrap_False, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_meta_device_reset_params_auto_wrap_True, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_torchdistX_default_init_auto_wrap_False, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_torchdistX_default_init_auto_wrap_True, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_torchdistX_init_fn_auto_wrap_False, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_torchdistX_init_fn_auto_wrap_True, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_simple_model_with_meta_device_default_init, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_simple_model_with_meta_device_reset_params, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_simple_model_with_torchdistX_default_init, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_simple_model_with_torchdistX_init_fn 2024-08-20T22:51:11.1662382Z 2024-08-20T22:51:11.1662880Z Running distributed/fsdp/test_fsdp_uneven 1/1 ... [2024-08-20 22:51:11.164300] 2024-08-20T22:51:11.1663500Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:51:11.1665528Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_uneven.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:51:11.164723] 2024-08-20T22:51:16.8448998Z 2024-08-20T22:51:16.8450673Z distributed/fsdp/test_fsdp_uneven 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_uneven_1.1_ff5fc4f52868b006_.log 2024-08-20T22:51:16.8452468Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_uneven.py::TestUnevenParamShard::test_one_iteration 2024-08-20T22:51:16.8453180Z 2024-08-20T22:51:16.8455718Z Running distributed/rpc/cuda/test_tensorpipe_agent 1/1 ... [2024-08-20 22:51:16.845340] 2024-08-20T22:51:16.8456419Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T22:51:16.8461243Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/cuda/test_tensorpipe_agent.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 22:51:16.845800] 2024-08-20T23:01:39.4831549Z 2024-08-20T23:01:39.4833271Z distributed/rpc/cuda/test_tensorpipe_agent 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.cuda.test_tensorpipe_agent_1.1_df0cfd35ad1f7f8b_.log 2024-08-20T23:01:39.4923751Z Running 99 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRpcTest::test_profiler_remote_cuda, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_simple, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation_gpu_root, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device_script, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_invalid_devices, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_valid_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_nested_with_cuda_future, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_with_cuda_future, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_callback_changes_devices, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_sparse_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_sparse_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_sparse_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_int, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_str, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_not_cuda, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_modify_tensor_inplace, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_replace_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_value_on_bad_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_multi, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested_multi, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default_to_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_5, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_6, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_7, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_8, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_5, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_6, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_7, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_8, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default_to_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_in_options, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_local_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_remote_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_min_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_many_to_one, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_loop, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_not_timeout, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote_response, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response_loop, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu_self, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_one_to_many, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_remote, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu_self, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_wrong_worker_name, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_mismatch, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch_reverse, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization5, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_with_unpickleable_attributes, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_tensor_view_as_return_value, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_device_maps_backward_pass, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_dist_autograd_sync_streams, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_gradients_synchronizations 2024-08-20T23:01:39.4993996Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRpcTest::test_profiler_remote_cuda 2024-08-20T23:01:39.4995520Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_simple 2024-08-20T23:01:39.4997047Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation 2024-08-20T23:01:39.4998696Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation_gpu_root 2024-08-20T23:01:39.5000357Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device 2024-08-20T23:01:39.5002021Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device_script 2024-08-20T23:01:39.5003615Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_invalid_devices 2024-08-20T23:01:39.5005089Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_valid_device 2024-08-20T23:01:39.5006703Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote_gpu 2024-08-20T23:01:39.5008495Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_nested_with_cuda_future 2024-08-20T23:01:39.5010375Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_with_cuda_future 2024-08-20T23:01:39.5012230Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_callback_changes_devices 2024-08-20T23:01:39.5014057Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_sparse_tensor 2024-08-20T23:01:39.5015874Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_tensor 2024-08-20T23:01:39.5017984Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_sparse_tensor 2024-08-20T23:01:39.5019991Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_tensor 2024-08-20T23:01:39.5021939Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_sparse_tensor 2024-08-20T23:01:39.5023874Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_tensor 2024-08-20T23:01:39.5025690Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_device 2024-08-20T23:01:39.5027410Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_int 2024-08-20T23:01:39.5029083Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_str 2024-08-20T23:01:39.5030777Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_not_cuda 2024-08-20T23:01:39.5032512Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_modify_tensor_inplace 2024-08-20T23:01:39.5034242Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_replace_tensor 2024-08-20T23:01:39.5035959Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_value_on_bad_device 2024-08-20T23:01:39.5037638Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream 2024-08-20T23:01:39.5039246Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_multi 2024-08-20T23:01:39.5040885Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested 2024-08-20T23:01:39.5042544Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested_multi 2024-08-20T23:01:39.5044182Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu 2024-08-20T23:01:39.5045833Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_default 2024-08-20T23:01:39.5047594Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_non_default 2024-08-20T23:01:39.5049291Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default 2024-08-20T23:01:39.5051014Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default_to_non_default 2024-08-20T23:01:39.5052828Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_1 2024-08-20T23:01:39.5054500Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_2 2024-08-20T23:01:39.5056262Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_3 2024-08-20T23:01:39.5058216Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_4 2024-08-20T23:01:39.5059886Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_5 2024-08-20T23:01:39.5061554Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_6 2024-08-20T23:01:39.5063198Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_7 2024-08-20T23:01:39.5064855Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_8 2024-08-20T23:01:39.5066558Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_1 2024-08-20T23:01:39.5068277Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_2 2024-08-20T23:01:39.5069975Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_3 2024-08-20T23:01:39.5071672Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_4 2024-08-20T23:01:39.5073369Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_5 2024-08-20T23:01:39.5075071Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_6 2024-08-20T23:01:39.5076775Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_7 2024-08-20T23:01:39.5078470Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_8 2024-08-20T23:01:39.5080175Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default 2024-08-20T23:01:39.5081934Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default_to_default 2024-08-20T23:01:39.5083701Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_default 2024-08-20T23:01:39.5085455Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_non_default 2024-08-20T23:01:39.5087122Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_gpu 2024-08-20T23:01:39.5088755Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_in_options 2024-08-20T23:01:39.5090466Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_local_device 2024-08-20T23:01:39.5092345Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_remote_device 2024-08-20T23:01:39.5094133Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_min_device 2024-08-20T23:01:39.5096031Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_many_to_one 2024-08-20T23:01:39.5097720Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config 2024-08-20T23:01:39.5099435Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_loop 2024-08-20T23:01:39.5101220Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_not_timeout 2024-08-20T23:01:39.5103019Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote 2024-08-20T23:01:39.5104841Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote_response 2024-08-20T23:01:39.5106641Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response 2024-08-20T23:01:39.5108455Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response_loop 2024-08-20T23:01:39.5110251Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu 2024-08-20T23:01:39.5111955Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu_self 2024-08-20T23:01:39.5113627Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_one_to_many 2024-08-20T23:01:39.5115283Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_remote 2024-08-20T23:01:39.5116943Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu 2024-08-20T23:01:39.5118670Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu_self 2024-08-20T23:01:39.5120392Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_wrong_worker_name 2024-08-20T23:01:39.5122056Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_mismatch 2024-08-20T23:01:39.5123688Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch 2024-08-20T23:01:39.5125404Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch_reverse 2024-08-20T23:01:39.5127166Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization1 2024-08-20T23:01:39.5128962Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization2 2024-08-20T23:01:39.5130748Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization3 2024-08-20T23:01:39.5132626Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization4 2024-08-20T23:01:39.5134504Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization1 2024-08-20T23:01:39.5136232Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization2 2024-08-20T23:01:39.5137957Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization3 2024-08-20T23:01:39.5139684Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization4 2024-08-20T23:01:39.5141395Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization5 2024-08-20T23:01:39.5143125Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization1 2024-08-20T23:01:39.5153279Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization2 2024-08-20T23:01:39.5155118Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization3 2024-08-20T23:01:39.5156856Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization4 2024-08-20T23:01:39.5158763Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization1 2024-08-20T23:01:39.5160517Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization2 2024-08-20T23:01:39.5162284Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization3 2024-08-20T23:01:39.5164003Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization4 2024-08-20T23:01:39.5165749Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_with_unpickleable_attributes 2024-08-20T23:01:39.5167511Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_tensor_view_as_return_value 2024-08-20T23:01:39.5169232Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_device_maps_backward_pass 2024-08-20T23:01:39.5170978Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_dist_autograd_sync_streams 2024-08-20T23:01:39.5172810Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_gradients_synchronizations 2024-08-20T23:01:39.5173787Z 2024-08-20T23:01:39.5174297Z Running distributed/test_c10d_nccl 1/1 ... [2024-08-20 23:01:39.487698] 2024-08-20T23:01:39.5174880Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:01:39.5176684Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_nccl.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:01:39.488135] 2024-08-20T23:01:40.8914315Z 2024-08-20T23:01:40.8916161Z distributed/test_c10d_nccl 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_nccl_1.1_3a89adabdc6b204b_.log 2024-08-20T23:01:40.8917478Z 2024-08-20T23:01:40.8920505Z Running distributed/rpc/test_tensorpipe_agent 2/3 ... [2024-08-20 23:01:40.891824] 2024-08-20T23:01:40.8921176Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:01:40.8925362Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_tensorpipe_agent.py', '--shard-id=2', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:01:40.892251] 2024-08-20T23:15:22.6504453Z 2024-08-20T23:15:22.6506166Z distributed/rpc/test_tensorpipe_agent 2/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_tensorpipe_agent_2.3_5eb4324e92140afd_.log 2024-08-20T23:15:22.6605974Z Running 116 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_chained_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_build_rpc_profiling_key, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_multi, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_none, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_dist_init_decorator, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_duplicate_name_2, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_function_not_on_callee, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_in_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_wait_twice, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_get_worker_infos, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_rpc_twice, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_int_callee, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_builtin_remote_ret, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_non_garbage_collected_user_rref_due_to_local_circular_dependency, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_owner_rref_backward, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_async_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_remote_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_instance_method, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_multi_async_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_nested_pickle, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rpc_rref_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_reinit, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_context_debug_info, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_forward_chain, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_get_future, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_slow_init, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_scalar_add, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_add, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_rpc_arg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_send_to_rank, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_shutdown_followed_by_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rref_after_shutdown, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_raise_in_user_func, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_with_partial_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_dense, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_twice_dense, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_worker_info_pickle, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_world_size_one, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_autograd_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_tensor_dims, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_round_trips, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_node_failure, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_nested, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_script_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_self, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_unused_send_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_tensor_with_grad, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_dist_autograd_profiling, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_only_on_return_value_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_python_remote_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_nested_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_rpc_complex_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_none_grads, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_add_done_callback, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_args_and_kwargs_contain_different_types, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote_multi, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_fork_in_jit_with_profiling, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_script_function_that_not_exists_remotely_from_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_chain, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_local_rref_local_value, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_record_function_jit_end_callbacks_with_fork, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_module, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_throw, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_as_arg_and_return, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_functions_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_unexepected_kwarg_is_specified, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_dist_backward, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_get_gradients, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_jit_fork_within_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_restore_context_after_swtich_to_jit_thread, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_sync_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_send_remote_module_with_a_new_attribute_not_pickled_over_the_wire, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_train_eval, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_create_remote_module_from_module_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire_script_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_inside, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_no_ddp, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_builtin_remote_ret_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc_without_rank, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_py_udf_remote_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_my_parameter_server_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_remote_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rref_stress_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rpc_rref_args_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_get_type_timeout_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_proxy_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_py_udf_remote_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_self_remote_arg_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_set_and_get_num_worker_threads, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_no_grad_on_tensor_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_multi_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_self_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_no_grad_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_call_itself_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_multiple_backward_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_nested_backward_accumulate_grads_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_no_graph_with_tensors_not_require_grad_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_trainer_ps_sparse 2024-08-20T23:15:22.6673161Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_chained_remote 2024-08-20T23:15:22.6674573Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise_async 2024-08-20T23:15:22.6676043Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_remote 2024-08-20T23:15:22.6677455Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_build_rpc_profiling_key 2024-08-20T23:15:22.6678752Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_multi 2024-08-20T23:15:22.6679991Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_none 2024-08-20T23:15:22.6681242Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_simple 2024-08-20T23:15:22.6682500Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_dist_init_decorator 2024-08-20T23:15:22.6683788Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_duplicate_name_2 2024-08-20T23:15:22.6685088Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_function_not_on_callee 2024-08-20T23:15:22.6686360Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done 2024-08-20T23:15:22.6687621Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done_exception 2024-08-20T23:15:22.6688896Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_in_rpc 2024-08-20T23:15:22.6690144Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_wait_twice 2024-08-20T23:15:22.6691397Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_get_worker_infos 2024-08-20T23:15:22.6692750Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_rpc_twice 2024-08-20T23:15:22.6693969Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_int_callee 2024-08-20T23:15:22.6695244Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_builtin_remote_ret 2024-08-20T23:15:22.6696517Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_remote 2024-08-20T23:15:22.6698007Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_non_garbage_collected_user_rref_due_to_local_circular_dependency 2024-08-20T23:15:22.6699534Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_owner_rref_backward 2024-08-20T23:15:22.6700972Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin_single_threaded 2024-08-20T23:15:22.6702666Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_udf 2024-08-20T23:15:22.6704062Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_async_rpc 2024-08-20T23:15:22.6705488Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_remote_rpc 2024-08-20T23:15:22.6706905Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin 2024-08-20T23:15:22.6708385Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin_single_threaded 2024-08-20T23:15:22.6709855Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_instance_method 2024-08-20T23:15:22.6711192Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_multi_async_call 2024-08-20T23:15:22.6712472Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_nested_pickle 2024-08-20T23:15:22.6713729Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rpc_rref_args 2024-08-20T23:15:22.6714955Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_reinit 2024-08-20T23:15:22.6716213Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_context_debug_info 2024-08-20T23:15:22.6717513Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_forward_chain 2024-08-20T23:15:22.6718782Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_get_future 2024-08-20T23:15:22.6720060Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_slow_init 2024-08-20T23:15:22.6721311Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_scalar_add 2024-08-20T23:15:22.6722494Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_add 2024-08-20T23:15:22.6723783Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_rpc_arg 2024-08-20T23:15:22.6725079Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_send_to_rank 2024-08-20T23:15:22.6726359Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_shutdown_followed_by_rpc 2024-08-20T23:15:22.6727698Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rref_after_shutdown 2024-08-20T23:15:22.6729064Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_raise_in_user_func 2024-08-20T23:15:22.6730472Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_with_partial_exception 2024-08-20T23:15:22.6731906Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_dense 2024-08-20T23:15:22.6733266Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_twice_dense 2024-08-20T23:15:22.6734604Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_worker_info_pickle 2024-08-20T23:15:22.6735870Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_world_size_one 2024-08-20T23:15:22.6737296Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_autograd_context 2024-08-20T23:15:22.6738757Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_tensor_dims 2024-08-20T23:15:22.6740289Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_round_trips 2024-08-20T23:15:22.6741765Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_node_failure 2024-08-20T23:15:22.6743189Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_nested 2024-08-20T23:15:22.6744652Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_script_call 2024-08-20T23:15:22.6746121Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_self 2024-08-20T23:15:22.6747593Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_unused_send_function 2024-08-20T23:15:22.6749106Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_tensor_with_grad 2024-08-20T23:15:22.6750609Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_dist_autograd_profiling 2024-08-20T23:15:22.6752110Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_only_on_return_value_remote 2024-08-20T23:15:22.6753609Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_call 2024-08-20T23:15:22.6755082Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_python_remote_call 2024-08-20T23:15:22.6756521Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_nested_context 2024-08-20T23:15:22.6758077Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_rpc_complex_args 2024-08-20T23:15:22.6759504Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_none_grads 2024-08-20T23:15:22.6760872Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_add_done_callback 2024-08-20T23:15:22.6762304Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_args_and_kwargs_contain_different_types 2024-08-20T23:15:22.6763762Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote 2024-08-20T23:15:22.6765164Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote_multi 2024-08-20T23:15:22.6766590Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_fork_in_jit_with_profiling 2024-08-20T23:15:22.6768172Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_script_function_that_not_exists_remotely_from_script 2024-08-20T23:15:22.6769663Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_chain 2024-08-20T23:15:22.6770971Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_local_rref_local_value 2024-08-20T23:15:22.6772511Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_record_function_jit_end_callbacks_with_fork 2024-08-20T23:15:22.6774150Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_module 2024-08-20T23:15:22.6775504Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_throw 2024-08-20T23:15:22.6776836Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_as_arg_and_return 2024-08-20T23:15:22.6778283Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_functions_not_supported 2024-08-20T23:15:22.6779763Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_unexepected_kwarg_is_specified 2024-08-20T23:15:22.6781181Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_dist_backward 2024-08-20T23:15:22.6782563Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_get_gradients 2024-08-20T23:15:22.6784008Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_jit_fork_within_context 2024-08-20T23:15:22.6785603Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_restore_context_after_swtich_to_jit_thread 2024-08-20T23:15:22.6787151Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_sync_script 2024-08-20T23:15:22.6788779Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_send_remote_module_with_a_new_attribute_not_pickled_over_the_wire 2024-08-20T23:15:22.6790366Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_train_eval 2024-08-20T23:15:22.6791911Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_create_remote_module_from_module_rref 2024-08-20T23:15:22.6793644Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire 2024-08-20T23:15:22.6795451Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire_script_not_supported 2024-08-20T23:15:22.6797175Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_inside 2024-08-20T23:15:22.6798684Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_no_ddp 2024-08-20T23:15:22.6800225Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote 2024-08-20T23:15:22.6801789Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_builtin_remote_ret_sparse 2024-08-20T23:15:22.6803345Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc 2024-08-20T23:15:22.6804920Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc_without_rank 2024-08-20T23:15:22.6806523Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_py_udf_remote_sparse 2024-08-20T23:15:22.6808081Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_my_parameter_server_sparse 2024-08-20T23:15:22.6809622Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_remote_sparse 2024-08-20T23:15:22.6811285Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rref_stress_sparse 2024-08-20T23:15:22.6812904Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rpc_rref_args_sparse 2024-08-20T23:15:22.6814412Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_sparse 2024-08-20T23:15:22.6815970Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_get_type_timeout_blocking 2024-08-20T23:15:22.6817532Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_proxy_timeout 2024-08-20T23:15:22.6819066Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_py_udf_remote_sparse 2024-08-20T23:15:22.6820705Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_self_remote_arg_sparse 2024-08-20T23:15:22.6822366Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_set_and_get_num_worker_threads 2024-08-20T23:15:22.6824044Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_no_grad_on_tensor_sparse 2024-08-20T23:15:22.6825772Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_multi_sparse 2024-08-20T23:15:22.6827467Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_self_sparse 2024-08-20T23:15:22.6829141Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_sparse 2024-08-20T23:15:22.6830883Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_no_grad_sparse 2024-08-20T23:15:22.6832640Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_call_sparse 2024-08-20T23:15:22.6834388Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_call_itself_sparse 2024-08-20T23:15:22.6836130Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_multiple_backward_sparse 2024-08-20T23:15:22.6837883Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_nested_backward_accumulate_grads_sparse 2024-08-20T23:15:22.6839741Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_no_graph_with_tensors_not_require_grad_sparse 2024-08-20T23:15:22.6841468Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_trainer_ps_sparse 2024-08-20T23:15:22.6842365Z 2024-08-20T23:15:22.6843067Z Running distributed/_shard/sharded_tensor/ops/test_binary_cmp 1/1 ... [2024-08-20 23:15:22.657297] 2024-08-20T23:15:22.6843788Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:15:22.6845668Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_binary_cmp.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:15:22.657733] 2024-08-20T23:15:25.1318589Z 2024-08-20T23:15:25.1321099Z distributed/_shard/sharded_tensor/ops/test_binary_cmp 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_binary_cmp_1.1_457ffa507753c035_.log 2024-08-20T23:15:25.1325554Z Running 4 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_binary_cmp.py::TestShardedTensorBinaryOps::test_torch_allclose, test/distributed/_shard/sharded_tensor/ops/test_binary_cmp.py::TestShardedTensorBinaryOps::test_torch_allclose_tensor_specs, test/distributed/_shard/sharded_tensor/ops/test_binary_cmp.py::TestShardedTensorBinaryOps::test_torch_equal, test/distributed/_shard/sharded_tensor/ops/test_binary_cmp.py::TestShardedTensorBinaryOps::test_torch_equal_tensor_specs 2024-08-20T23:15:25.1328378Z 2024-08-20T23:15:25.1329111Z Running distributed/rpc/test_faulty_agent 1/1 ... [2024-08-20 23:15:25.132274] 2024-08-20T23:15:25.1329870Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:15:25.1331805Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_faulty_agent.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:15:25.132722] 2024-08-20T23:21:01.0737072Z 2024-08-20T23:21:01.0739466Z distributed/rpc/test_faulty_agent 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_faulty_agent_1.1_b657ad53c939206c_.log 2024-08-20T23:21:01.0760784Z Running 28 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_check_failed_messages, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_faulty_messages, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_messages_to_delay, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_no_faulty_messages, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_builtin_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_script_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rref_to_here_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_verify_backend_options, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_context_cleanup_tensor_with_grad, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_verify_backend_options, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_remote_timeout_to_here_in_jit, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_in_jit, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_script_func, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_to_here_timeout_in_jit, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_python, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_torchscript_function 2024-08-20T23:21:01.0778180Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout 2024-08-20T23:21:01.0779760Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout_to_self 2024-08-20T23:21:01.0781263Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_check_failed_messages 2024-08-20T23:21:01.0782652Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_faulty_messages 2024-08-20T23:21:01.0784035Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_messages_to_delay 2024-08-20T23:21:01.0785501Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_no_faulty_messages 2024-08-20T23:21:01.0786937Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout 2024-08-20T23:21:01.0788483Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout_to_self 2024-08-20T23:21:01.0790000Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle 2024-08-20T23:21:01.0791480Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle_to_self 2024-08-20T23:21:01.0793013Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout 2024-08-20T23:21:01.0794554Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout_to_self 2024-08-20T23:21:01.0796019Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_builtin_timeout 2024-08-20T23:21:01.0797346Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_script_timeout 2024-08-20T23:21:01.0798691Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rref_to_here_timeout 2024-08-20T23:21:01.0800110Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout 2024-08-20T23:21:01.0801626Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout_to_self 2024-08-20T23:21:01.0803149Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout 2024-08-20T23:21:01.0804694Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout_to_self 2024-08-20T23:21:01.0806162Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_verify_backend_options 2024-08-20T23:21:01.0807649Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_context_cleanup_tensor_with_grad 2024-08-20T23:21:01.0809192Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_verify_backend_options 2024-08-20T23:21:01.0810828Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_remote_timeout_to_here_in_jit 2024-08-20T23:21:01.0812401Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_in_jit 2024-08-20T23:21:01.0813898Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_script_func 2024-08-20T23:21:01.0815382Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_to_here_timeout_in_jit 2024-08-20T23:21:01.0816793Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_python 2024-08-20T23:21:01.0818232Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_torchscript_function 2024-08-20T23:21:01.0819060Z 2024-08-20T23:21:01.0819622Z Running distributed/fsdp/test_fsdp_multiple_forward 1/1 ... [2024-08-20 23:21:01.075181] 2024-08-20T23:21:01.0820288Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:21:01.0822127Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_multiple_forward.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:21:01.075640] 2024-08-20T23:21:06.7556606Z 2024-08-20T23:21:06.7558708Z distributed/fsdp/test_fsdp_multiple_forward 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_multiple_forward_1.1_76ca3b62fbc0d3d0_.log 2024-08-20T23:21:06.7560490Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_multiple_forward.py::TestMultiForward::test_multi_forward 2024-08-20T23:21:06.7561436Z 2024-08-20T23:21:06.7562418Z Running distributed/fsdp/test_fsdp_comm 1/1 ... [2024-08-20 23:21:06.756066] 2024-08-20T23:21:06.7563050Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:21:06.7567634Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_comm.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:21:06.756497] 2024-08-20T23:21:32.7272398Z 2024-08-20T23:21:32.7274523Z distributed/fsdp/test_fsdp_comm 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_comm_1.1_30efbf57c84d82cb_.log 2024-08-20T23:21:32.7283815Z Running 10 items in this shard: test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestExplicitUnshard::test_unshard_async_use_orig_params_False, test/distributed/fsdp/test_fsdp_comm.py::TestExplicitUnshard::test_unshard_async_use_orig_params_True 2024-08-20T23:21:32.7291980Z 2024-08-20T23:21:32.7292711Z Running distributed/_shard/sharded_tensor/ops/test_init 1/1 ... [2024-08-20 23:21:32.727910] 2024-08-20T23:21:32.7293414Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:21:32.7295281Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_init.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:21:32.728393] 2024-08-20T23:21:35.2021552Z 2024-08-20T23:21:35.2023514Z distributed/_shard/sharded_tensor/ops/test_init 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_init_1.1_8a0fa0414e86ef88_.log 2024-08-20T23:21:35.2026822Z Running 3 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_init.py::TestShardedTensorNNInit::test_init_sharded_tensor_with_kaiming_uniform, test/distributed/_shard/sharded_tensor/ops/test_init.py::TestShardedTensorNNInit::test_init_sharded_tensor_with_normal, test/distributed/_shard/sharded_tensor/ops/test_init.py::TestShardedTensorNNInit::test_init_sharded_tensor_with_uniform 2024-08-20T23:21:35.2029408Z 2024-08-20T23:21:35.2029958Z Running distributed/_shard/sharded_tensor/test_sharded_tensor 1/1 ... [2024-08-20 23:21:35.202662] 2024-08-20T23:21:35.2030687Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:21:35.2034152Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/test_sharded_tensor.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:21:35.203115] 2024-08-20T23:21:43.6888057Z 2024-08-20T23:21:43.6890233Z distributed/_shard/sharded_tensor/test_sharded_tensor 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_1.1_0cba918e2107b5ad_.log 2024-08-20T23:21:43.6936942Z Running 69 items in this shard: test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorMetadata::test_serialize_and_deserialize, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestCreateTensorFromParams::test_empty, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardParameter::test_shard_parameter, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardParameter::test_shard_parameter_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardTensor::test_shard_tensor, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardTensor::test_shard_tensor_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardTensor::test_shard_tensor_with_empty_shard, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestModuleHookApi::test_collect_local_shard, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestModuleHookApi::test_reshard_output, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestLocalTensor::test_local_tensor, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestLocalTensor::test_local_tensor_error, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_cleanup, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_complete_world_size, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_like, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_full, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_ones, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_rand, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_zeros, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_gather_even, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_gather_uneven, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_insufficient_sharding_dims, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_invalid_pg_rpc_ranks, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_invalid_sharding, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_load_state_dict_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_multiple_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_partial_world_size, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_sharded_tensor_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_sharded_tensor_sizes, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_sharding_columns, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_state_dict, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_state_dict_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_state_dict_no_sharded_tensors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_create_sharded_tensor_with_ones, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_gather_even, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_gather_uneven, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_grid_sharding, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_multiple_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_partial_world_size, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_device, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_to_cpu, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_to_cuda, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_to_test, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_uneven_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_with_rpc_names, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalTensor::test_init_from_local_tensor, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalTensor::test_init_from_local_tensor_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_and_global_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_and_global_metadata_invalid_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_pin_memory, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_property_cross_ranks, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_shards_gaps, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_shards_overlap, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_st_base_init_from_local_shards_and_global_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorCustomOps::test_custom_op, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorCustomOps::test_custom_op_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorCustomOps::test_custom_op_override, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardMetadata::test_create_shard_with_no_placement, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardMetadata::test_shard_metadata_init, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorSubGroupInit::test_sub_process_group_placement_validation, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorSubGroupInit::test_sub_process_group_sharded_tensor_init, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestCreateTensorNoProcessGroupMode::test_init_from_local_shards_and_global_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestCreateTensorNoProcessGroupMode::test_non_contiguous_local_shards 2024-08-20T23:21:43.6981478Z 2024-08-20T23:21:43.6982131Z Running distributed/_shard/sharded_tensor/test_sharded_tensor_reshard 1/1 ... [2024-08-20 23:21:43.689439] 2024-08-20T23:21:43.6982898Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:21:43.6984811Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/test_sharded_tensor_reshard.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:21:43.689942] 2024-08-20T23:21:46.1637715Z 2024-08-20T23:21:46.1639933Z distributed/_shard/sharded_tensor/test_sharded_tensor_reshard 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_reshard_1.1_f5ae028b0c328e1b_.log 2024-08-20T23:21:46.1642864Z Running 2 items in this shard: test/distributed/_shard/sharded_tensor/test_sharded_tensor_reshard.py::TestReshard::test_sharded_tensor_reshard, test/distributed/_shard/sharded_tensor/test_sharded_tensor_reshard.py::TestReshard::test_sharded_tensor_reshard_errors 2024-08-20T23:21:46.1644567Z 2024-08-20T23:21:46.1645118Z Running distributed/_shard/sharded_tensor/ops/test_embedding 1/1 ... [2024-08-20 23:21:46.164181] 2024-08-20T23:21:46.1645899Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:21:46.1648809Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_embedding.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:21:46.164584] 2024-08-20T23:21:48.6391253Z 2024-08-20T23:21:48.6393630Z distributed/_shard/sharded_tensor/ops/test_embedding 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_1.1_dd0c2dc1002782d0_.log 2024-08-20T23:21:48.6396396Z Running 2 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_embedding.py::TestShardedEmbedding::test_sharded_embedding_colwise, test/distributed/_shard/sharded_tensor/ops/test_embedding.py::TestShardedEmbedding::test_sharded_embedding_rowwise 2024-08-20T23:21:48.6398226Z 2024-08-20T23:21:48.6398804Z Running distributed/_shard/sharded_tensor/ops/test_embedding_bag 1/1 ... [2024-08-20 23:21:48.639624] 2024-08-20T23:21:48.6399562Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:21:48.6403460Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_embedding_bag.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:21:48.640054] 2024-08-20T23:21:51.1139300Z 2024-08-20T23:21:51.1141576Z distributed/_shard/sharded_tensor/ops/test_embedding_bag 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_bag_1.1_f2f742f565044c2b_.log 2024-08-20T23:21:51.1144344Z Running 2 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_embedding_bag.py::TestShardedEmbeddingBag::test_sharded_embedding_bag_colwise, test/distributed/_shard/sharded_tensor/ops/test_embedding_bag.py::TestShardedEmbeddingBag::test_sharded_embedding_bag_rowwise 2024-08-20T23:21:51.1146127Z 2024-08-20T23:21:51.1146503Z Running distributed/test_launcher 1/1 ... [2024-08-20 23:21:51.114406] 2024-08-20T23:21:51.1147083Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:21:51.1150984Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_launcher.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:21:51.114833] 2024-08-20T23:21:55.0405720Z 2024-08-20T23:21:55.0407552Z distributed/test_launcher 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_launcher_1.1_9e8fdd432cba8d9b_.log 2024-08-20T23:21:55.0409108Z Running 1 items in this shard: test/distributed/test_launcher.py::TestDistributedLaunch::test_launch_user_script 2024-08-20T23:21:55.0409797Z 2024-08-20T23:21:55.0412750Z Running distributed/test_store 1/1 ... [2024-08-20 23:21:55.041040] 2024-08-20T23:21:55.0413357Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:21:55.0418058Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_store.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:21:55.041478] 2024-08-20T23:26:19.2934591Z 2024-08-20T23:26:19.2936457Z distributed/test_store 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_store_1.1_c6dd3553b454b67b_.log 2024-08-20T23:26:19.2981902Z Running 86 items in this shard: test/distributed/test_store.py::FileStoreTest::test_append, test/distributed/test_store.py::FileStoreTest::test_compare_set, test/distributed/test_store.py::FileStoreTest::test_init_pg_and_rpc_with_same_file, test/distributed/test_store.py::FileStoreTest::test_multi_get, test/distributed/test_store.py::FileStoreTest::test_multi_set, test/distributed/test_store.py::FileStoreTest::test_refcount, test/distributed/test_store.py::FileStoreTest::test_set_get_check, test/distributed/test_store.py::FileStoreTest::test_simple_wait, test/distributed/test_store.py::HashStoreTest::test_append, test/distributed/test_store.py::HashStoreTest::test_compare_set, test/distributed/test_store.py::HashStoreTest::test_multi_get, test/distributed/test_store.py::HashStoreTest::test_multi_set, test/distributed/test_store.py::HashStoreTest::test_set_get_check, test/distributed/test_store.py::HashStoreTest::test_simple_wait, test/distributed/test_store.py::PrefixStoreTest::test_get_underlying_store, test/distributed/test_store.py::PrefixFileStoreTest::test_append, test/distributed/test_store.py::PrefixFileStoreTest::test_compare_set, test/distributed/test_store.py::PrefixFileStoreTest::test_multi_get, test/distributed/test_store.py::PrefixFileStoreTest::test_multi_set, test/distributed/test_store.py::PrefixFileStoreTest::test_set_get_check, test/distributed/test_store.py::PrefixFileStoreTest::test_simple_wait, test/distributed/test_store.py::TCPStoreTest::test_address_already_in_use, test/distributed/test_store.py::TCPStoreTest::test_append, test/distributed/test_store.py::TCPStoreTest::test_compare_set, test/distributed/test_store.py::TCPStoreTest::test_init_pg_and_rpc_with_same_socket, test/distributed/test_store.py::TCPStoreTest::test_multi_get, test/distributed/test_store.py::TCPStoreTest::test_multi_set, test/distributed/test_store.py::TCPStoreTest::test_multi_worker_with_fixed_world_size, test/distributed/test_store.py::TCPStoreTest::test_multi_worker_with_nonfixed_world_size, test/distributed/test_store.py::TCPStoreTest::test_multitenancy, test/distributed/test_store.py::TCPStoreTest::test_numkeys_delkeys, test/distributed/test_store.py::TCPStoreTest::test_repr, test/distributed/test_store.py::TCPStoreTest::test_set_get_check, test/distributed/test_store.py::TCPStoreTest::test_simple_wait, test/distributed/test_store.py::TCPStoreTest::test_store_timeout_on_missing_clients, test/distributed/test_store.py::TCPStoreTest::test_take_over_listen_socket, test/distributed/test_store.py::LibUvTCPStoreTest::test_address_already_in_use, test/distributed/test_store.py::LibUvTCPStoreTest::test_append, test/distributed/test_store.py::LibUvTCPStoreTest::test_compare_set, test/distributed/test_store.py::LibUvTCPStoreTest::test_init_pg_and_rpc_with_same_socket, test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_get, test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_set, test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_worker_with_fixed_world_size, test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_worker_with_nonfixed_world_size, test/distributed/test_store.py::LibUvTCPStoreTest::test_multitenancy, test/distributed/test_store.py::LibUvTCPStoreTest::test_numkeys_delkeys, test/distributed/test_store.py::LibUvTCPStoreTest::test_repr, test/distributed/test_store.py::LibUvTCPStoreTest::test_set_get_check, test/distributed/test_store.py::LibUvTCPStoreTest::test_simple_wait, test/distributed/test_store.py::LibUvTCPStoreTest::test_store_timeout_on_missing_clients, test/distributed/test_store.py::LibUvTCPStoreTest::test_take_over_listen_socket, test/distributed/test_store.py::PrefixTCPStoreTest::test_append, test/distributed/test_store.py::PrefixTCPStoreTest::test_compare_set, test/distributed/test_store.py::PrefixTCPStoreTest::test_multi_get, test/distributed/test_store.py::PrefixTCPStoreTest::test_multi_set, test/distributed/test_store.py::PrefixTCPStoreTest::test_set_get_check, test/distributed/test_store.py::PrefixTCPStoreTest::test_simple_wait, test/distributed/test_store.py::PrefixTCPStoreTest::test_underlying_non_prefix_store, test/distributed/test_store.py::PythonStoreTest::test_set_get, test/distributed/test_store.py::RendezvousTest::test_unknown_handler, test/distributed/test_store.py::RendezvousTest::test_url_with_node_params, test/distributed/test_store.py::RendezvousEnvTest::test_nominal, test/distributed/test_store.py::RendezvousFileTest::test_common_errors, test/distributed/test_store.py::RendezvousFileTest::test_nominal, test/distributed/test_store.py::RendezvousTCPTest::test_common_errors, test/distributed/test_store.py::RendezvousTCPTest::test_dns_timeout, test/distributed/test_store.py::RendezvousTCPTest::test_nominal, test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_timeout_doest_break_client, test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_timeout_set, test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_url_with_libuv, test/distributed/test_store.py::TestPythonStore::test_append_roundtrip, test/distributed/test_store.py::TestPythonStore::test_extended_methods_fallbacks, test/distributed/test_store.py::TestPythonStore::test_has_extended_api_passthrough, test/distributed/test_store.py::TestPythonStore::test_has_extended_api_roundtrip, test/distributed/test_store.py::TestPythonStore::test_multi_get_roundtrip, test/distributed/test_store.py::TestPythonStore::test_multi_set_roundtrip, test/distributed/test_store.py::TestPythonStore::test_optional_methods_fail, test/distributed/test_store.py::TestMultiThreadedWait::test_wait_file_store, test/distributed/test_store.py::TestMultiThreadedWait::test_wait_hash_store, test/distributed/test_store.py::TestMultiThreadedWait::test_wait_prefix_file_store, test/distributed/test_store.py::TestMultiThreadedWait::test_wait_tcp_store, test/distributed/test_store.py::TestMultiThreadedWait::test_wait_tcp_store_uv, test/distributed/test_store.py::TimeoutTest::test_interrupt_doesnt_break_wait, test/distributed/test_store.py::InitPgWithNonUvStore::test_with_env_var, test/distributed/test_store.py::InitPgWithNonUvStore::test_with_url_param, test/distributed/test_store.py::TestClientProtocol::test_client_connect 2024-08-20T23:26:19.3015829Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_append 2024-08-20T23:26:19.3016821Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_compare_set 2024-08-20T23:26:19.3017927Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_init_pg_and_rpc_with_same_file 2024-08-20T23:26:19.3019014Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_multi_get 2024-08-20T23:26:19.3019982Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_multi_set 2024-08-20T23:26:19.3020955Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_refcount 2024-08-20T23:26:19.3021944Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_set_get_check 2024-08-20T23:26:19.3022951Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_simple_wait 2024-08-20T23:26:19.3023919Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_append 2024-08-20T23:26:19.3024884Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_compare_set 2024-08-20T23:26:19.3025848Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_multi_get 2024-08-20T23:26:19.3026820Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_multi_set 2024-08-20T23:26:19.3027809Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_set_get_check 2024-08-20T23:26:19.3028797Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_simple_wait 2024-08-20T23:26:19.3029861Z Running 1 items in this shard: test/distributed/test_store.py::PrefixStoreTest::test_get_underlying_store 2024-08-20T23:26:19.3030930Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_append 2024-08-20T23:26:19.3031978Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_compare_set 2024-08-20T23:26:19.3033024Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_multi_get 2024-08-20T23:26:19.3034073Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_multi_set 2024-08-20T23:26:19.3035146Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_set_get_check 2024-08-20T23:26:19.3036216Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_simple_wait 2024-08-20T23:26:19.3037295Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_address_already_in_use 2024-08-20T23:26:19.3038436Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_append 2024-08-20T23:26:19.3039398Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_compare_set 2024-08-20T23:26:19.3040476Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_init_pg_and_rpc_with_same_socket 2024-08-20T23:26:19.3041553Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_multi_get 2024-08-20T23:26:19.3042512Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_multi_set 2024-08-20T23:26:19.3043588Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_multi_worker_with_fixed_world_size 2024-08-20T23:26:19.3044822Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_multi_worker_with_nonfixed_world_size 2024-08-20T23:26:19.3045941Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_multitenancy 2024-08-20T23:26:19.3046960Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_numkeys_delkeys 2024-08-20T23:26:19.3048011Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_repr 2024-08-20T23:26:19.3048967Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_set_get_check 2024-08-20T23:26:19.3049958Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_simple_wait 2024-08-20T23:26:19.3051039Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_store_timeout_on_missing_clients 2024-08-20T23:26:19.3052285Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_take_over_listen_socket 2024-08-20T23:26:19.3053427Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_address_already_in_use 2024-08-20T23:26:19.3054510Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_append 2024-08-20T23:26:19.3055531Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_compare_set 2024-08-20T23:26:19.3056697Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_init_pg_and_rpc_with_same_socket 2024-08-20T23:26:19.3057991Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_get 2024-08-20T23:26:19.3059030Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_set 2024-08-20T23:26:19.3060180Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_worker_with_fixed_world_size 2024-08-20T23:26:19.3061484Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_worker_with_nonfixed_world_size 2024-08-20T23:26:19.3062678Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_multitenancy 2024-08-20T23:26:19.3063761Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_numkeys_delkeys 2024-08-20T23:26:19.3064794Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_repr 2024-08-20T23:26:19.3065818Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_set_get_check 2024-08-20T23:26:19.3066879Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_simple_wait 2024-08-20T23:26:19.3068024Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_store_timeout_on_missing_clients 2024-08-20T23:26:19.3069244Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_take_over_listen_socket 2024-08-20T23:26:19.3070331Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_append 2024-08-20T23:26:19.3071351Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_compare_set 2024-08-20T23:26:19.3073157Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_multi_get 2024-08-20T23:26:19.3074197Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_multi_set 2024-08-20T23:26:19.3075248Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_set_get_check 2024-08-20T23:26:19.3076313Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_simple_wait 2024-08-20T23:26:19.3077461Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_underlying_non_prefix_store 2024-08-20T23:26:19.3078563Z Running 1 items in this shard: test/distributed/test_store.py::PythonStoreTest::test_set_get 2024-08-20T23:26:19.3079563Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTest::test_unknown_handler 2024-08-20T23:26:19.3080637Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTest::test_url_with_node_params 2024-08-20T23:26:19.3081693Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousEnvTest::test_nominal 2024-08-20T23:26:19.3082732Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousFileTest::test_common_errors 2024-08-20T23:26:19.3083763Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousFileTest::test_nominal 2024-08-20T23:26:19.3084798Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_common_errors 2024-08-20T23:26:19.3085853Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_dns_timeout 2024-08-20T23:26:19.3086861Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_nominal 2024-08-20T23:26:19.3088012Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_timeout_doest_break_client 2024-08-20T23:26:19.3089233Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_timeout_set 2024-08-20T23:26:19.3090395Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_url_with_libuv 2024-08-20T23:26:19.3091513Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_append_roundtrip 2024-08-20T23:26:19.3092714Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_extended_methods_fallbacks 2024-08-20T23:26:19.3093911Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_has_extended_api_passthrough 2024-08-20T23:26:19.3095101Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_has_extended_api_roundtrip 2024-08-20T23:26:19.3096226Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_multi_get_roundtrip 2024-08-20T23:26:19.3097333Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_multi_set_roundtrip 2024-08-20T23:26:19.3098466Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_optional_methods_fail 2024-08-20T23:26:19.3099583Z Running 1 items in this shard: test/distributed/test_store.py::TestMultiThreadedWait::test_wait_file_store 2024-08-20T23:26:19.3100712Z Running 1 items in this shard: test/distributed/test_store.py::TestMultiThreadedWait::test_wait_hash_store 2024-08-20T23:26:19.3101880Z Running 1 items in this shard: test/distributed/test_store.py::TestMultiThreadedWait::test_wait_prefix_file_store 2024-08-20T23:26:19.3103047Z Running 1 items in this shard: test/distributed/test_store.py::TestMultiThreadedWait::test_wait_tcp_store 2024-08-20T23:26:19.3104175Z Running 1 items in this shard: test/distributed/test_store.py::TestMultiThreadedWait::test_wait_tcp_store_uv 2024-08-20T23:26:19.3105319Z Running 1 items in this shard: test/distributed/test_store.py::TimeoutTest::test_interrupt_doesnt_break_wait 2024-08-20T23:26:19.3106426Z Running 1 items in this shard: test/distributed/test_store.py::InitPgWithNonUvStore::test_with_env_var 2024-08-20T23:26:19.3107651Z Running 1 items in this shard: test/distributed/test_store.py::InitPgWithNonUvStore::test_with_url_param 2024-08-20T23:26:19.3108759Z Running 1 items in this shard: test/distributed/test_store.py::TestClientProtocol::test_client_connect 2024-08-20T23:26:19.3109394Z 2024-08-20T23:26:19.3109915Z Running distributed/test_c10d_spawn_ucc 1/1 ... [2024-08-20 23:26:19.296508] 2024-08-20T23:26:19.3110524Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:26:19.3112346Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_spawn_ucc.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:26:19.296945] 2024-08-20T23:26:47.3625579Z 2024-08-20T23:26:47.3627287Z distributed/test_c10d_spawn_ucc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_spawn_ucc_1.1_f971886e41b45078_.log 2024-08-20T23:26:47.3633939Z Running 9 items in this shard: test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_allgather_ucc, test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_allreduce_ucc, test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_broadcast_ucc, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_gather, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_to_all, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_to_all_single, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_allreduce, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_broadcast, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_reduce 2024-08-20T23:26:47.3640008Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_allgather_ucc 2024-08-20T23:26:47.3641393Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_allreduce_ucc 2024-08-20T23:26:47.3642756Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_broadcast_ucc 2024-08-20T23:26:47.3644077Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_gather 2024-08-20T23:26:47.3645354Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_to_all 2024-08-20T23:26:47.3646661Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_to_all_single 2024-08-20T23:26:47.3647965Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_allreduce 2024-08-20T23:26:47.3649221Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_broadcast 2024-08-20T23:26:47.3650466Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_reduce 2024-08-20T23:26:47.3651154Z 2024-08-20T23:26:47.3651611Z Running distributed/test_c10d_spawn_nccl 1/1 ... [2024-08-20 23:26:47.363273] 2024-08-20T23:26:47.3652295Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:26:47.3654135Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_spawn_nccl.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:26:47.363740] 2024-08-20T23:27:26.6417270Z 2024-08-20T23:27:26.6419439Z distributed/test_c10d_spawn_nccl 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_spawn_nccl_1.1_f3a06736f2be1654_.log 2024-08-20T23:27:26.6429223Z Running 13 items in this shard: test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_allgather_nccl, test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_allreduce_nccl, test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_broadcast_nccl, test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_reduce_nccl, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather_base, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all_single, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_allreduce, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_broadcast, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter_non_contiguous 2024-08-20T23:27:26.6437490Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_allgather_nccl 2024-08-20T23:27:26.6438895Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_allreduce_nccl 2024-08-20T23:27:26.6440263Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_broadcast_nccl 2024-08-20T23:27:26.6441638Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_reduce_nccl 2024-08-20T23:27:26.6443072Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather 2024-08-20T23:27:26.6444409Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather_base 2024-08-20T23:27:26.6445741Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all 2024-08-20T23:27:26.6447089Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all_single 2024-08-20T23:27:26.6448423Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_allreduce 2024-08-20T23:27:26.6449695Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_broadcast 2024-08-20T23:27:26.6450972Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce 2024-08-20T23:27:26.6460484Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter 2024-08-20T23:27:26.6461958Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter_non_contiguous 2024-08-20T23:27:26.6462803Z 2024-08-20T23:27:26.6463285Z Running distributed/test_c10d_ops_nccl 1/1 ... [2024-08-20 23:27:26.642427] 2024-08-20T23:27:26.6463890Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:27:26.6465651Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_ops_nccl.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:27:26.642866] 2024-08-20T23:27:28.0297469Z 2024-08-20T23:27:28.0299346Z distributed/test_c10d_ops_nccl 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_ops_nccl_1.1_294fca59a0ee3dd0_.log 2024-08-20T23:27:28.0300653Z 2024-08-20T23:27:28.0304621Z Running distributed/elastic/metrics/api_test 1/1 ... [2024-08-20 23:27:28.030206] 2024-08-20T23:27:28.0305287Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:27:28.0309886Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/metrics/api_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:27:28.030664] 2024-08-20T23:27:30.8048900Z 2024-08-20T23:27:30.8050745Z distributed/elastic/metrics/api_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.metrics.api_test_1.1_2531c2e4b172f20e_.log 2024-08-20T23:27:30.8053499Z Running 3 items in this shard: test/distributed/elastic/metrics/api_test.py::MetricsApiTest::test_get_metric_name, test/distributed/elastic/metrics/api_test.py::MetricsApiTest::test_inheritance, test/distributed/elastic/metrics/api_test.py::MetricsApiTest::test_profile 2024-08-20T23:27:30.8055433Z 2024-08-20T23:27:30.8055979Z Running distributed/elastic/utils/distributed_test 1/1 ... [2024-08-20 23:27:30.805365] 2024-08-20T23:27:30.8056665Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:27:30.8061228Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/utils/distributed_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:27:30.805817] 2024-08-20T23:27:38.4864239Z 2024-08-20T23:27:38.4866361Z distributed/elastic/utils/distributed_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.utils.distributed_test_1.1_ed1b8966ec795815_.log 2024-08-20T23:27:38.4873114Z Running 8 items in this shard: test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_multi, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_no_port_multi, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_single_server, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_timeout_on_server, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_timeout_on_worker, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_with_libuv_support, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_port_already_in_use_on_server, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_port_already_in_use_on_worker 2024-08-20T23:27:38.4878611Z 2024-08-20T23:27:38.4879141Z Running distributed/elastic/events/lib_test 1/1 ... [2024-08-20 23:27:38.487133] 2024-08-20T23:27:38.4879834Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:27:38.4881801Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/events/lib_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:27:38.487589] 2024-08-20T23:27:41.3118194Z 2024-08-20T23:27:41.3119893Z distributed/elastic/events/lib_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.events.lib_test_1.1_4135dfad42307c52_.log 2024-08-20T23:27:41.3125855Z Running 8 items in this shard: test/distributed/elastic/events/lib_test.py::EventLibTest::test_event_created, test/distributed/elastic/events/lib_test.py::EventLibTest::test_event_deser, test/distributed/elastic/events/lib_test.py::EventLibTest::test_get_or_create_logger, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_construct_and_record_rdzv_event, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_construct_and_record_rdzv_event_does_not_run_if_invalid_dest, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_rdzv_event_created, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_rdzv_event_deserialize, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_rdzv_event_str 2024-08-20T23:27:41.3130387Z 2024-08-20T23:27:41.3130935Z Running distributed/elastic/multiprocessing/api_test 1/1 ... [2024-08-20 23:27:41.312269] 2024-08-20T23:27:41.3131739Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:27:41.3133619Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/multiprocessing/api_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:27:41.312696] 2024-08-20T23:28:11.8799018Z 2024-08-20T23:28:11.8801690Z distributed/elastic/multiprocessing/api_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.multiprocessing.api_test_1.1_8474f9cabcb93aac_.log 2024-08-20T23:28:11.8829433Z Running 40 items in this shard: test/distributed/elastic/multiprocessing/api_test.py::RunProcResultsTest::test_get_failures, test/distributed/elastic/multiprocessing/api_test.py::RunProcResultsTest::test_is_failed, test/distributed/elastic/multiprocessing/api_test.py::StdTest::test_from_str_bad_input, test/distributed/elastic/multiprocessing/api_test.py::StdTest::test_from_value, test/distributed/elastic/multiprocessing/api_test.py::StdTest::test_from_value_map, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_args_env_len_mismatch, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_binary_exit, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_binary_incorrect_entrypoint, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_binary_raises, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_function_large_ret_val, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_function_raise, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_function_with_tensor, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_invalid_log_dir, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_multiprocess_context_close, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_multiprocessing_context_poll_raises_exception, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_pcontext_wait, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_subprocess_context_close, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_to_map, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_validate_full_rank, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_void_function, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_wait_for_all_child_procs_to_exit, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_args_env_len_mismatch, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_binary, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_binary_exit, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_binary_incorrect_entrypoint, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_binary_raises, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_binary_redirect_and_tee, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_function, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_function_large_ret_val, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_function_raise, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_function_with_tensor, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_invalid_log_dir, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_multiprocess_context_close, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_multiprocessing_context_poll_raises_exception, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_pcontext_wait, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_subprocess_context_close, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_to_map, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_validate_full_rank, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_void_function, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_wait_for_all_child_procs_to_exit 2024-08-20T23:28:11.8853172Z 2024-08-20T23:28:11.8853816Z Running distributed/elastic/timer/local_timer_example 1/1 ... [2024-08-20 23:28:11.880582] 2024-08-20T23:28:11.8854527Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:28:11.8856383Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/timer/local_timer_example.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:28:11.881036] 2024-08-20T23:28:28.6820717Z 2024-08-20T23:28:28.6822559Z distributed/elastic/timer/local_timer_example 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.timer.local_timer_example_1.1_eb0dcf631ba2479b_.log 2024-08-20T23:28:28.6825063Z Running 2 items in this shard: test/distributed/elastic/timer/local_timer_example.py::LocalTimerExample::test_example_start_method_spawn, test/distributed/elastic/timer/local_timer_example.py::LocalTimerExample::test_torch_mp_example 2024-08-20T23:28:28.6826621Z 2024-08-20T23:28:28.6827374Z Running distributed/elastic/timer/local_timer_test 1/1 ... [2024-08-20 23:28:28.682543] 2024-08-20T23:28:28.6828060Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:28:28.6834268Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/timer/local_timer_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:28:28.682980] 2024-08-20T23:28:35.8134232Z 2024-08-20T23:28:35.8136179Z distributed/elastic/timer/local_timer_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.timer.local_timer_test_1.1_4d36d84ec13daf48_.log 2024-08-20T23:28:35.8145568Z Running 14 items in this shard: test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_client_interaction, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_exception_propagation, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_get_timer_recursive, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_happy_path, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_no_client, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_timer, test/distributed/elastic/timer/local_timer_test.py::MultiprocessingRequestQueueTest::test_get, test/distributed/elastic/timer/local_timer_test.py::MultiprocessingRequestQueueTest::test_get_less_than_size, test/distributed/elastic/timer/local_timer_test.py::MultiprocessingRequestQueueTest::test_get_size, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_acquire_release, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_expired_timers, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_valid_timers, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_watchdog_call_count, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_watchdog_empty_queue 2024-08-20T23:28:35.8154268Z 2024-08-20T23:28:35.8154749Z Running distributed/elastic/utils/util_test 1/1 ... [2024-08-20 23:28:35.814232] 2024-08-20T23:28:35.8155460Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:28:35.8157428Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/utils/util_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:28:35.814815] 2024-08-20T23:28:38.7894992Z 2024-08-20T23:28:38.7897690Z distributed/elastic/utils/util_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.utils.util_test_1.1_0ca2828cdcff6a93_.log 2024-08-20T23:28:38.7909393Z Running 12 items in this shard: test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_barrier, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_barrier_hash_store, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_barrier_timeout_operations, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_barrier_timeout_rank_tracing, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_get_all_rank_0, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_get_all_rank_n, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_synchronize, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_synchronize_hash_store, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger_custom_name, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger_different, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger_none 2024-08-20T23:28:38.7918264Z 2024-08-20T23:28:38.7919151Z Running distributed/elastic/utils/logging_test 1/1 ... [2024-08-20 23:28:38.790216] 2024-08-20T23:28:38.7920179Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set 2024-08-20T23:28:38.7922694Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/utils/logging_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-20 23:28:38.790784] 2024-08-20T23:28:41.6156962Z 2024-08-20T23:28:41.6159251Z distributed/elastic/utils/logging_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.utils.logging_test_1.1_d143787a7c59c076_.log 2024-08-20T23:28:41.6161601Z Running 2 items in this shard: test/distributed/elastic/utils/logging_test.py::LoggingTest::test_derive_module_name, test/distributed/elastic/utils/logging_test.py::LoggingTest::test_logger_name 2024-08-20T23:28:41.6162943Z 2024-08-20T23:28:42.0891332Z Running test batch 'tests to run' cost 6275.99 seconds 2024-08-20T23:28:42.4750385Z 2024-08-20T23:28:42.4751023Z real 104m40.308s 2024-08-20T23:28:42.4751582Z user 204m12.208s 2024-08-20T23:28:42.4752017Z sys 43m34.508s 2024-08-20T23:28:42.4752431Z + assert_git_not_dirty 2024-08-20T23:28:42.4753293Z + [[ linux-jammy-py3.8-gcc11 != *rocm* ]] 2024-08-20T23:28:42.4754134Z + [[ linux-jammy-py3.8-gcc11 != *xla* ]] 2024-08-20T23:28:42.4756793Z ++ git status --porcelain 2024-08-20T23:28:42.4758669Z ++ grep -v '?? third_party' 2024-08-20T23:29:05.2744454Z ++ true 2024-08-20T23:29:05.2745083Z + git_status= 2024-08-20T23:29:05.2745673Z + [[ -n '' ]] 2024-08-20T23:29:05.2747065Z + [[ linux-jammy-py3.8-gcc11 == *cuda* ]] 2024-08-20T23:29:05.2747735Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-20T23:29:05.2748531Z + [[ 2 == 1 ]] 2024-08-20T23:29:05.2749270Z + cleanup_workspace 2024-08-20T23:29:05.2750617Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-20T23:29:05.2752382Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-20T23:29:05.2753425Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-20T23:29:05.2754118Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-20T23:29:05.2754984Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-20T23:29:05.2755942Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-20T23:29:05.2756702Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-20T23:29:07.8827192Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-20T23:29:07.8827699Z cat test/**/*_toprint.log || true 2024-08-20T23:29:07.8874936Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:07.8875417Z env: 2024-08-20T23:29:07.8875677Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:07.8876264Z DOCKER_CONTAINER_ID: 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:07.8876909Z ##[endgroup] 2024-08-20T23:29:07.8949962Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-20T23:29:07.8981716Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:29:07.8982162Z kill "$MONITOR_SCRIPT_PID" 2024-08-20T23:29:07.8988992Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:07.8989485Z env: 2024-08-20T23:29:07.8989728Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:07.8990313Z DOCKER_CONTAINER_ID: 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:07.8990968Z MONITOR_SCRIPT_PID: 497574 2024-08-20T23:29:07.8991304Z ##[endgroup] 2024-08-20T23:29:07.9193786Z Prepare all required actions 2024-08-20T23:29:07.9194229Z Getting action download info 2024-08-20T23:29:08.1557345Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-20T23:29:08.3436751Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-20T23:29:08.3437518Z with: 2024-08-20T23:29:08.3437962Z file-suffix: test-distributed-2-2-amz2023.linux.2xlarge_29025447188 2024-08-20T23:29:08.3438543Z s3-bucket: gha-artifacts 2024-08-20T23:29:08.3438845Z env: 2024-08-20T23:29:08.3439095Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:08.3439669Z DOCKER_CONTAINER_ID: 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:08.3440289Z ##[endgroup] 2024-08-20T23:29:08.3477570Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-20T23:29:08.3478184Z # Remove any previous test jsons if they exist 2024-08-20T23:29:08.3478678Z rm -f test-jsons-*.zip 2024-08-20T23:29:08.3479208Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-20T23:29:08.3484978Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:08.3485455Z env: 2024-08-20T23:29:08.3485694Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:08.3486277Z DOCKER_CONTAINER_ID: 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:08.3487116Z FILE_SUFFIX: test-distributed-2-2-amz2023.linux.2xlarge_29025447188 2024-08-20T23:29:08.3487661Z ##[endgroup] 2024-08-20T23:29:08.3862431Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-20T23:29:08.3889922Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-20T23:29:08.3890706Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-20T23:29:08.3896957Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-20T23:29:08.3902925Z adding: test/slow_tests.json (deflated 82%) 2024-08-20T23:29:08.3918239Z adding: test/test-reports/td_exclusions-41f07381e81784445195.json (deflated 86%) 2024-08-20T23:29:08.3929566Z adding: test/.pytorch-disabled-tests.json (deflated 88%) 2024-08-20T23:29:08.3957623Z ##[group]Run # Remove any previous test reports if they exist 2024-08-20T23:29:08.3958497Z # Remove any previous test reports if they exist 2024-08-20T23:29:08.3959169Z rm -f test-reports-*.zip 2024-08-20T23:29:08.3959745Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-20T23:29:08.3965588Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:08.3966072Z env: 2024-08-20T23:29:08.3966326Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:08.3966897Z DOCKER_CONTAINER_ID: 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:08.3967721Z FILE_SUFFIX: test-distributed-2-2-amz2023.linux.2xlarge_29025447188 2024-08-20T23:29:08.3968279Z ##[endgroup] 2024-08-20T23:29:08.4247864Z adding: test/test-reports/python-pytest/distributed.test_dynamo_distributed/distributed.test_dynamo_distributed-cc595a6ded5f0e0f.xml (deflated 92%) 2024-08-20T23:29:08.4249906Z adding: test/test-reports/python-pytest/distributed._tools.test_mod_tracker/distributed._tools.test_mod_tracker-54056008b0d9e753.xml (deflated 65%) 2024-08-20T23:29:08.4252035Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_nested_dict/distributed.checkpoint.test_nested_dict-3aee52c160d39428.xml (deflated 51%) 2024-08-20T23:29:08.4253978Z adding: test/test-reports/python-pytest/distributed.test_fake_pg/distributed.test_fake_pg-6ed21f83184193f8.xml (deflated 82%) 2024-08-20T23:29:08.4256000Z adding: test/test-reports/python-pytest/distributed.pipelining.test_backward/distributed.pipelining.test_backward-437f38240d052bb8.xml (deflated 37%) 2024-08-20T23:29:08.4258385Z adding: test/test-reports/python-pytest/distributed.fsdp.test_checkpoint_wrapper/distributed.fsdp.test_checkpoint_wrapper-cf824d880d5a6021.xml (deflated 76%) 2024-08-20T23:29:08.4260693Z adding: test/test-reports/python-pytest/distributed.test_symmetric_memory/distributed.test_symmetric_memory-9026e73d1043bc05.xml (deflated 93%) 2024-08-20T23:29:08.4262837Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.test_logger/distributed._shard.sharded_tensor.test_logger-5a51f30b70aa473f.xml (deflated 37%) 2024-08-20T23:29:08.4264922Z adding: test/test-reports/python-pytest/distributed._tensor.test_dtensor/distributed._tensor.test_dtensor-4bdeca464387bd9b.xml (deflated 86%) 2024-08-20T23:29:08.4267024Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_compile/distributed._composable.fsdp.test_fully_shard_compile-b6f66e33526c37a7.xml (deflated 89%) 2024-08-20T23:29:08.4269023Z adding: test/test-reports/python-pytest/distributed._tensor.test_xla_integration/distributed._tensor.test_xla_integration-e52ce3c297e69c23.xml (deflated 74%) 2024-08-20T23:29:08.4270959Z adding: test/test-reports/python-pytest/distributed.pipelining.test_unflatten/distributed.pipelining.test_unflatten-cd5b214189d0d598.xml (deflated 41%) 2024-08-20T23:29:08.4273200Z adding: test/test-reports/python-pytest/distributed._tensor.test_convolution_ops/distributed._tensor.test_convolution_ops-9b66c9502dd4f174.xml (deflated 54%) 2024-08-20T23:29:08.4275120Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3c3650601446a4d4.xml (deflated 34%) 2024-08-20T23:29:08.4276898Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e01814e268501564.xml (deflated 37%) 2024-08-20T23:29:08.4278666Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3732680d274ef5b3.xml (deflated 35%) 2024-08-20T23:29:08.4280423Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b03c94c8d461c433.xml (deflated 42%) 2024-08-20T23:29:08.4282721Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-04a79592c8257edc.xml (deflated 35%) 2024-08-20T23:29:08.4284254Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-47967e3454c7a500.xml (deflated 35%) 2024-08-20T23:29:08.4285794Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b7f44fdcb07a7290.xml (deflated 35%) 2024-08-20T23:29:08.4287482Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-717eff9dd933631a.xml (deflated 35%) 2024-08-20T23:29:08.4289017Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-d4718fa99c96332f.xml (deflated 35%) 2024-08-20T23:29:08.4290549Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c8636992ac2c4f53.xml (deflated 35%) 2024-08-20T23:29:08.4292162Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a527741ada3c530b.xml (deflated 43%) 2024-08-20T23:29:08.4293693Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-477abecdfe04cef3.xml (deflated 36%) 2024-08-20T23:29:08.4295230Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8e78c5d843e72262.xml (deflated 42%) 2024-08-20T23:29:08.4296752Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-08784295f1680a88.xml (deflated 35%) 2024-08-20T23:29:08.4298274Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-43f9d01fbfcacd44.xml (deflated 35%) 2024-08-20T23:29:08.4299818Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-69afe8d05b6edf5b.xml (deflated 36%) 2024-08-20T23:29:08.4301353Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4f5bf2eb949f4fb0.xml (deflated 36%) 2024-08-20T23:29:08.4302981Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-babf9bb68b60cc9c.xml (deflated 42%) 2024-08-20T23:29:08.4304514Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a7c9efe678745843.xml (deflated 35%) 2024-08-20T23:29:08.4306047Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2c5f5208ccadc5b5.xml (deflated 35%) 2024-08-20T23:29:08.4307587Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-ad0ad3615a72f7c1.xml (deflated 43%) 2024-08-20T23:29:08.4309201Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-1413dfdbde8fa87b.xml (deflated 35%) 2024-08-20T23:29:08.4310727Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-068fadd45efa8a1a.xml (deflated 36%) 2024-08-20T23:29:08.4312267Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-ad9e549811343943.xml (deflated 43%) 2024-08-20T23:29:08.4313788Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-cd6bd612ae71920f.xml (deflated 35%) 2024-08-20T23:29:08.4315330Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0dfc70f3584f0d21.xml (deflated 36%) 2024-08-20T23:29:08.4316864Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-755f10d59745c2ae.xml (deflated 43%) 2024-08-20T23:29:08.4318398Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-7fab3f2c3e3cc2b8.xml (deflated 35%) 2024-08-20T23:29:08.4319921Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b7fc38d19eca4e94.xml (deflated 35%) 2024-08-20T23:29:08.4321458Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c63a2cde0ab55d79.xml (deflated 42%) 2024-08-20T23:29:08.4322990Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-6ad741481fc4a8aa.xml (deflated 35%) 2024-08-20T23:29:08.4324523Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-abfe135162892642.xml (deflated 35%) 2024-08-20T23:29:08.4326039Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f36f30d27da7146b.xml (deflated 35%) 2024-08-20T23:29:08.4327638Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-75f1bd038fc381ee.xml (deflated 42%) 2024-08-20T23:29:08.4329163Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2711fccec9346319.xml (deflated 34%) 2024-08-20T23:29:08.4330689Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c218aba070796e43.xml (deflated 35%) 2024-08-20T23:29:08.4332325Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c3ec559ef4c3bb07.xml (deflated 42%) 2024-08-20T23:29:08.4333858Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2608ae08ee994cff.xml (deflated 35%) 2024-08-20T23:29:08.4335394Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-da2adeee53cf3288.xml (deflated 35%) 2024-08-20T23:29:08.4336934Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-34926172e6883d8a.xml (deflated 35%) 2024-08-20T23:29:08.4338451Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-232fbcbed96c4760.xml (deflated 35%) 2024-08-20T23:29:08.4339979Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-563d3f81e016a6b6.xml (deflated 42%) 2024-08-20T23:29:08.4341581Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-75f0e3d59e79d4b4.xml (deflated 36%) 2024-08-20T23:29:08.4343110Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-850b24ecb572f455.xml (deflated 42%) 2024-08-20T23:29:08.4344632Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-faf5c3cdc4af1090.xml (deflated 35%) 2024-08-20T23:29:08.4346179Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0188f7c4902ca86b.xml (deflated 36%) 2024-08-20T23:29:08.4347702Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2334ae203f7170f2.xml (deflated 47%) 2024-08-20T23:29:08.4349213Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a69ecef7c480b77d.xml (deflated 35%) 2024-08-20T23:29:08.4350748Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-fe2fe723942b46fe.xml (deflated 57%) 2024-08-20T23:29:08.4352291Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2fb49d2baa392393.xml (deflated 36%) 2024-08-20T23:29:08.4353821Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4f7cf8b2806ff43d.xml (deflated 43%) 2024-08-20T23:29:08.4355334Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-80ec1435430d13bc.xml (deflated 35%) 2024-08-20T23:29:08.4356857Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-146e849707782a66.xml (deflated 42%) 2024-08-20T23:29:08.4358911Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-1ffc0fb92c3c9ce7.xml (deflated 43%) 2024-08-20T23:29:08.4360457Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-43b5ef55575824aa.xml (deflated 42%) 2024-08-20T23:29:08.4363330Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-d1220a41b9844070.xml (deflated 42%) 2024-08-20T23:29:08.4366325Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e0d16f2d7628dbb3.xml (deflated 42%) 2024-08-20T23:29:08.4369326Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-1f152115eb2eff65.xml (deflated 42%) 2024-08-20T23:29:08.4372552Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-1c06859157c965f9.xml (deflated 42%) 2024-08-20T23:29:08.4375527Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-01a2e4012843f0e2.xml (deflated 41%) 2024-08-20T23:29:08.4378482Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-849d1a260ba13096.xml (deflated 42%) 2024-08-20T23:29:08.4381457Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-91d2c8b8fa223df4.xml (deflated 42%) 2024-08-20T23:29:08.4384444Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b6d622d3bb7e2cb4.xml (deflated 42%) 2024-08-20T23:29:08.4387444Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2dd040d97e521e5b.xml (deflated 42%) 2024-08-20T23:29:08.4390444Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-85671eae6e48b4ee.xml (deflated 42%) 2024-08-20T23:29:08.4393432Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-acf6f99786d01647.xml (deflated 42%) 2024-08-20T23:29:08.4396411Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0deb0f08bc10dc64.xml (deflated 42%) 2024-08-20T23:29:08.4399389Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-edefa98bfe819e64.xml (deflated 42%) 2024-08-20T23:29:08.4402515Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-1541f662bcd6acb6.xml (deflated 36%) 2024-08-20T23:29:08.4405494Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-54ffc01236ab260d.xml (deflated 43%) 2024-08-20T23:29:08.4408473Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-5cb9f73c7419012d.xml (deflated 36%) 2024-08-20T23:29:08.4411443Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c605402bb8d7cb84.xml (deflated 36%) 2024-08-20T23:29:08.4414114Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-ffb09e4547f9fb45.xml (deflated 36%) 2024-08-20T23:29:08.4415644Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-365912363cb9f848.xml (deflated 36%) 2024-08-20T23:29:08.4417192Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-355dc2cac9e07aee.xml (deflated 45%) 2024-08-20T23:29:08.4418704Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-5f10c1e030752247.xml (deflated 43%) 2024-08-20T23:29:08.4420335Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c5d1e339ed579cf1.xml (deflated 44%) 2024-08-20T23:29:08.4421957Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-85b0b7000c89fb2e.xml (deflated 43%) 2024-08-20T23:29:08.4424582Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c3531daf549bde47.xml (deflated 43%) 2024-08-20T23:29:08.4427625Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-760e4b9974a51016.xml (deflated 44%) 2024-08-20T23:29:08.4430598Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2b0a8e59f74c68c4.xml (deflated 44%) 2024-08-20T23:29:08.4433588Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-48e9499750c6d6f1.xml (deflated 44%) 2024-08-20T23:29:08.4436529Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-482bf6f203ec3002.xml (deflated 36%) 2024-08-20T23:29:08.4439674Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-1ea9a618e4b7f6c1.xml (deflated 36%) 2024-08-20T23:29:08.4442651Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-03a146d85b6eb671.xml (deflated 36%) 2024-08-20T23:29:08.4445613Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8d5bd49702593680.xml (deflated 36%) 2024-08-20T23:29:08.4448550Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-d8fce00a9959c7b8.xml (deflated 44%) 2024-08-20T23:29:08.4451533Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-38a1f7604616caec.xml (deflated 35%) 2024-08-20T23:29:08.4454598Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3c903c6994549232.xml (deflated 43%) 2024-08-20T23:29:08.4457557Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-5b6700df2e09eede.xml (deflated 37%) 2024-08-20T23:29:08.4460691Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a1cef32c407cb41b.xml (deflated 36%) 2024-08-20T23:29:08.4463551Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-17875551ab877919.xml (deflated 42%) 2024-08-20T23:29:08.4466482Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c6a3cf2ffd90a905.xml (deflated 42%) 2024-08-20T23:29:08.4469564Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a8683a6fef1eff2a.xml (deflated 36%) 2024-08-20T23:29:08.4472475Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-66f0b187eb85b85e.xml (deflated 35%) 2024-08-20T23:29:08.4475458Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-6b484a9c79a6f0de.xml (deflated 36%) 2024-08-20T23:29:08.4478457Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-aaf3f1f616d0116c.xml (deflated 36%) 2024-08-20T23:29:08.4481450Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e3a3545085f39929.xml (deflated 35%) 2024-08-20T23:29:08.4484410Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-af174c36de3a474b.xml (deflated 36%) 2024-08-20T23:29:08.4487422Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-385bc466c4776b36.xml (deflated 36%) 2024-08-20T23:29:08.4490403Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-5240d45b4012e963.xml (deflated 36%) 2024-08-20T23:29:08.4493470Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a35be8a3d274dccd.xml (deflated 44%) 2024-08-20T23:29:08.4496369Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-00bb91e57db1051d.xml (deflated 44%) 2024-08-20T23:29:08.4499293Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3c08267ca477f593.xml (deflated 43%) 2024-08-20T23:29:08.4502204Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2673abe741f6a340.xml (deflated 43%) 2024-08-20T23:29:08.4505096Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-937ca9a9132b6f3c.xml (deflated 43%) 2024-08-20T23:29:08.4507991Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4c2202ef3d07bc19.xml (deflated 43%) 2024-08-20T23:29:08.4510900Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-5a12aad1d88dd233.xml (deflated 43%) 2024-08-20T23:29:08.4513826Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-5830053a3d5b80bf.xml (deflated 43%) 2024-08-20T23:29:08.4516891Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c52bddd536956947.xml (deflated 43%) 2024-08-20T23:29:08.4519818Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3fa20b3026e6e926.xml (deflated 34%) 2024-08-20T23:29:08.4522734Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b05e20d3caf3dd01.xml (deflated 33%) 2024-08-20T23:29:08.4525660Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4b128690d6fec8bb.xml (deflated 34%) 2024-08-20T23:29:08.4528561Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-19f5fa46cbc8ad31.xml (deflated 35%) 2024-08-20T23:29:08.4531482Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-819544f62eacff46.xml (deflated 34%) 2024-08-20T23:29:08.4534465Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f122540678c7c612.xml (deflated 34%) 2024-08-20T23:29:08.4537381Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-7c4335a4eb4b09d4.xml (deflated 33%) 2024-08-20T23:29:08.4540259Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-6de784379575ac9e.xml (deflated 34%) 2024-08-20T23:29:08.4543176Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-11be4082aff173cd.xml (deflated 34%) 2024-08-20T23:29:08.4546203Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c7b6e5d13b9b4c0d.xml (deflated 34%) 2024-08-20T23:29:08.4549473Z adding: test/test-reports/python-pytest/distributed.test_compute_comm_reordering/distributed.test_compute_comm_reordering-c80e26986f1ae155.xml (deflated 86%) 2024-08-20T23:29:08.4553070Z adding: test/test-reports/python-pytest/distributed.checkpoint.e2e.test_fsdp_ep/distributed.checkpoint.e2e.test_fsdp_ep-86dce2f9571e85fe.xml (deflated 34%) 2024-08-20T23:29:08.4556534Z adding: test/test-reports/python-pytest/distributed._tools.test_mem_tracker/distributed._tools.test_mem_tracker-54b90455d6e945b8.xml (deflated 65%) 2024-08-20T23:29:08.4560435Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_training/distributed._composable.fsdp.test_fully_shard_training-f4fc609f10ab5a3e.xml (deflated 84%) 2024-08-20T23:29:08.4564186Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_save_load_api/distributed.checkpoint.test_save_load_api-4d25a243809b735f.xml (deflated 44%) 2024-08-20T23:29:08.4567977Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_dedup_tensors/distributed.checkpoint.test_dedup_tensors-0efb7cfacc15ce47.xml (deflated 37%) 2024-08-20T23:29:08.4571896Z adding: test/test-reports/python-pytest/distributed.checkpoint.fsdp.test_fsdp_dsd/distributed.checkpoint.fsdp.test_fsdp_dsd-c2de106cff4fcab0.xml (deflated 77%) 2024-08-20T23:29:08.4575460Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-ec4eb4cb08a8cf61.xml (deflated 43%) 2024-08-20T23:29:08.4578711Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-ebe5a88a231d0734.xml (deflated 43%) 2024-08-20T23:29:08.4581958Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-0572f5d1ae0c6a36.xml (deflated 43%) 2024-08-20T23:29:08.4585216Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-ff116f9b1e448b04.xml (deflated 43%) 2024-08-20T23:29:08.4588487Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-2a01bfa12e1c4b0c.xml (deflated 36%) 2024-08-20T23:29:08.4591755Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-81f01a827b06fae4.xml (deflated 43%) 2024-08-20T23:29:08.4595186Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-a64f2b29e65a013e.xml (deflated 43%) 2024-08-20T23:29:08.4598436Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-671a6921e03beb37.xml (deflated 43%) 2024-08-20T23:29:08.4600932Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-9201676c8880bfa7.xml (deflated 44%) 2024-08-20T23:29:08.4602594Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-5f2cc71a5461c0fe.xml (deflated 44%) 2024-08-20T23:29:08.4604246Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-4d15272a5adf2e16.xml (deflated 44%) 2024-08-20T23:29:08.4605903Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-07878204ff2722e1.xml (deflated 44%) 2024-08-20T23:29:08.4607565Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-a0a7fda38c93dd75.xml (deflated 44%) 2024-08-20T23:29:08.4609271Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-459588014c8c8e9a.xml (deflated 44%) 2024-08-20T23:29:08.4610912Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-3110cee9f4414c73.xml (deflated 44%) 2024-08-20T23:29:08.4612803Z adding: test/test-reports/python-pytest/distributed._tensor.test_attention/distributed._tensor.test_attention-eb1b1ec6d7f4a179.xml (deflated 74%) 2024-08-20T23:29:08.4614721Z adding: test/test-reports/python-pytest/distributed._tensor.debug.test_comm_mode_features/distributed._tensor.debug.test_comm_mode_features-c7d45d3857da888d.xml (deflated 65%) 2024-08-20T23:29:08.4616779Z adding: test/test-reports/python-pytest/distributed._tensor.experimental.test_local_map/distributed._tensor.experimental.test_local_map-fd5af10e736fbfb2.xml (deflated 68%) 2024-08-20T23:29:08.4618698Z adding: test/test-reports/python-pytest/distributed._tensor.test_pointwise_ops/distributed._tensor.test_pointwise_ops-69bf065991e278fa.xml (deflated 70%) 2024-08-20T23:29:08.4620650Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_tp_random_state/distributed.tensor.parallel.test_tp_random_state-7e7aba49c0cbb9b7.xml (deflated 46%) 2024-08-20T23:29:08.4622789Z adding: test/test-reports/python-pytest/distributed.checkpoint.e2e.test_e2e_save_and_load/distributed.checkpoint.e2e.test_e2e_save_and_load-9c8addf959b60c38.xml (deflated 89%) 2024-08-20T23:29:08.4624754Z adding: test/test-reports/python-pytest/distributed._composable.test_checkpoint/distributed._composable.test_checkpoint-89f403031844d4f7.xml (deflated 70%) 2024-08-20T23:29:08.4626474Z adding: test/test-reports/python-pytest/distributed._tensor.test_init/distributed._tensor.test_init-125dca11f30686bb.xml (deflated 74%) 2024-08-20T23:29:08.4628198Z adding: test/test-reports/python-pytest/distributed._tools.test_fsdp2_mem_tracker/distributed._tools.test_fsdp2_mem_tracker-b77416a6619fbf6a.xml (deflated 60%) 2024-08-20T23:29:08.4630137Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_fsdp_optim_state/distributed.checkpoint.test_fsdp_optim_state-cbeebf70e78ab392.xml (deflated 65%) 2024-08-20T23:29:08.4632092Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_format_utils/distributed.checkpoint.test_format_utils-508fcae8fec829b4.xml (deflated 59%) 2024-08-20T23:29:08.4634174Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_fsspec/distributed.checkpoint.test_fsspec-e03c233b97f4cf91.xml (deflated 65%) 2024-08-20T23:29:08.4637095Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_traverse/distributed.checkpoint.test_traverse-1cffb20d65e8e08c.xml (deflated 76%) 2024-08-20T23:29:08.4639170Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_dtensor_resharding/distributed.checkpoint.test_dtensor_resharding-05160b0de1b62dcf.xml (deflated 80%) 2024-08-20T23:29:08.4641177Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_tp_checkpoint/distributed.checkpoint.test_tp_checkpoint-432cb6149de407ef.xml (deflated 65%) 2024-08-20T23:29:08.4643161Z adding: test/test-reports/python-pytest/distributed.checkpoint.e2e.test_fine_tuning/distributed.checkpoint.e2e.test_fine_tuning-bb56c001c97fbeb5.xml (deflated 37%) 2024-08-20T23:29:08.4645057Z adding: test/test-reports/python-pytest/distributed._tools.test_memory_tracker/distributed._tools.test_memory_tracker-79163cbecf667a3c.xml (deflated 44%) 2024-08-20T23:29:08.4647363Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_init/distributed._composable.fsdp.test_fully_shard_init-d73a667ecacc94cf.xml (deflated 92%) 2024-08-20T23:29:08.4649310Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_planner/distributed.checkpoint.test_planner-a243ee362a809349.xml (deflated 67%) 2024-08-20T23:29:08.4651155Z adding: test/test-reports/python-pytest/distributed.elastic.test_control_plane/distributed.elastic.test_control_plane-528dd7194802b1d0.xml (deflated 78%) 2024-08-20T23:29:08.4653201Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_dtensor_checkpoint/distributed.checkpoint.test_dtensor_checkpoint-0f39bea0e5a66da5.xml (deflated 47%) 2024-08-20T23:29:08.4655411Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_logging/distributed._composable.fsdp.test_fully_shard_logging-fa736fa39abdc9d5.xml (deflated 28%) 2024-08-20T23:29:08.4657621Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_memory/distributed._composable.fsdp.test_fully_shard_memory-95b769794d4be4bc.xml (deflated 38%) 2024-08-20T23:29:08.4659871Z adding: test/test-reports/python-pytest/distributed._tensor.test_optimizers/distributed._tensor.test_optimizers-c0e1562801fddbcc.xml (deflated 81%) 2024-08-20T23:29:08.4661593Z adding: test/test-reports/python-pytest/distributed.test_functional_api/distributed.test_functional_api-a49ad4ee982fa875.xml (deflated 87%) 2024-08-20T23:29:08.4663302Z adding: test/test-reports/python-pytest/distributed.test_control_collectives/distributed.test_control_collectives-7826172270b7ea75.xml (deflated 84%) 2024-08-20T23:29:08.4665353Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_fsdp_tp_checkpoint_conversion/distributed.checkpoint.test_fsdp_tp_checkpoint_conversion-1646fa95ad70d5a3.xml (deflated 48%) 2024-08-20T23:29:08.4667355Z adding: test/test-reports/python-pytest/distributed._tensor.test_op_strategy/distributed._tensor.test_op_strategy-45e91c760ae504d7.xml (deflated 84%) 2024-08-20T23:29:08.4669235Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_hsdp_checkpoint/distributed.checkpoint.test_hsdp_checkpoint-a402fbf346b7375f.xml (deflated 79%) 2024-08-20T23:29:08.4671463Z adding: test/test-reports/python-pytest/distributed._composable.test_composability.test_pp_composability/distributed._composable.test_composability.test_pp_composability-dcbd50aac85b27b6.xml (deflated 95%) 2024-08-20T23:29:08.4673636Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_compatibility/distributed.checkpoint.test_compatibility-42763a20318e7010.xml (deflated 66%) 2024-08-20T23:29:08.4675448Z adding: test/test-reports/python-pytest/distributed._tensor.test_view_ops/distributed._tensor.test_view_ops-f1efc33b503f73dd.xml (deflated 58%) 2024-08-20T23:29:08.4677483Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_clip_grad_norm_/distributed._composable.fsdp.test_fully_shard_clip_grad_norm_-724dfeab7dd3c270.xml (deflated 53%) 2024-08-20T23:29:08.4679672Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_parallelize_api/distributed.tensor.parallel.test_parallelize_api-747f1d4d95564413.xml (deflated 82%) 2024-08-20T23:29:08.4681764Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_optim_state/distributed.fsdp.test_fsdp_optim_state-a2797d836dfe2514.xml (deflated 96%) 2024-08-20T23:29:08.4684634Z adding: test/test-reports/python-pytest/distributed._composable.test_compose/distributed._composable.test_compose-18e54e93f5b36a32.xml (deflated 86%) 2024-08-20T23:29:08.4686429Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_pure_fp16/distributed.fsdp.test_fsdp_pure_fp16-2b88f5bc60db7e6f.xml (deflated 62%) 2024-08-20T23:29:08.4688235Z adding: test/test-reports/python-pytest/distributed.nn.jit.test_instantiator/distributed.nn.jit.test_instantiator-14ef4e5cc30271a6.xml (deflated 61%) 2024-08-20T23:29:08.4690168Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_comm/distributed._composable.fsdp.test_fully_shard_comm-54021d25277cadbb.xml (deflated 84%) 2024-08-20T23:29:08.4692416Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_state_dict/distributed._composable.fsdp.test_fully_shard_state_dict-bb33adac67a32506.xml (deflated 75%) 2024-08-20T23:29:08.4694475Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_ignored_modules/distributed.fsdp.test_fsdp_ignored_modules-40ebfdeaf0ad8bc7.xml (deflated 86%) 2024-08-20T23:29:08.4696319Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_overlap/distributed.fsdp.test_fsdp_overlap-be3c4ef7cbafe168.xml (deflated 64%) 2024-08-20T23:29:08.4698295Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_backward_prefetch/distributed.fsdp.test_fsdp_backward_prefetch-d9147f5082538c85.xml (deflated 47%) 2024-08-20T23:29:08.4700103Z adding: test/test-reports/python-pytest/distributed._shard.test_sharder/distributed._shard.test_sharder-16f8d7e996b3f13f.xml (deflated 66%) 2024-08-20T23:29:08.4701736Z adding: test/test-reports/python-pytest/distributed.fsdp.test_wrap/distributed.fsdp.test_wrap-d4645ea91e6eca82.xml (deflated 94%) 2024-08-20T23:29:08.4703381Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_misc/distributed.fsdp.test_fsdp_misc-7359add880882a97.xml (deflated 89%) 2024-08-20T23:29:08.4705117Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_comm_hooks/distributed.fsdp.test_fsdp_comm_hooks-bd44cd1b2b597e59.xml (deflated 94%) 2024-08-20T23:29:08.4706977Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_unshard_params/distributed.fsdp.test_fsdp_unshard_params-1a2082009fa30f39.xml (deflated 89%) 2024-08-20T23:29:08.4708710Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-d1d27b92be46f75d.xml (deflated 37%) 2024-08-20T23:29:08.4710300Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-42a773fe966ad547.xml (deflated 35%) 2024-08-20T23:29:08.4711874Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-5d80a6ff90586f6d.xml (deflated 35%) 2024-08-20T23:29:08.4713464Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-11b2300b7dfff817.xml (deflated 34%) 2024-08-20T23:29:08.4715041Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-0476b67b61e06b7d.xml (deflated 36%) 2024-08-20T23:29:08.4716613Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-85f2033faaf603cf.xml (deflated 36%) 2024-08-20T23:29:08.4718169Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-97a6232c7082190b.xml (deflated 34%) 2024-08-20T23:29:08.4719744Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-63c923f52fa10a6d.xml (deflated 34%) 2024-08-20T23:29:08.4721333Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-07b5fbd8eed571f3.xml (deflated 34%) 2024-08-20T23:29:08.4722991Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-454200d1aeb691ae.xml (deflated 34%) 2024-08-20T23:29:08.4724561Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-861a9a17a01ec0de.xml (deflated 35%) 2024-08-20T23:29:08.4726155Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-02f54a9934dc2cc9.xml (deflated 34%) 2024-08-20T23:29:08.4727744Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-a89b9206bb46449b.xml (deflated 34%) 2024-08-20T23:29:08.4729793Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-23c1bd954f7f19dd.xml (deflated 34%) 2024-08-20T23:29:08.4731417Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-ad5adeea00841ced.xml (deflated 34%) 2024-08-20T23:29:08.4733203Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-d5daed865ec22cb0.xml (deflated 36%) 2024-08-20T23:29:08.4734795Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-a7a51c9d2c507c5f.xml (deflated 37%) 2024-08-20T23:29:08.4736383Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-108a18daa1cd56f7.xml (deflated 35%) 2024-08-20T23:29:08.4738018Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-87b446cec7c21371.xml (deflated 36%) 2024-08-20T23:29:08.4739602Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-f0c83c45db285fb9.xml (deflated 36%) 2024-08-20T23:29:08.4741185Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-9717c5ff6351d081.xml (deflated 35%) 2024-08-20T23:29:08.4742775Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-d44f54215831e3a4.xml (deflated 34%) 2024-08-20T23:29:08.4744334Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-7f783ff667170fc5.xml (deflated 33%) 2024-08-20T23:29:08.4746067Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_clip_grad_norm/distributed.fsdp.test_fsdp_clip_grad_norm-f0983f351358f7e1.xml (deflated 77%) 2024-08-20T23:29:08.4747842Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_meta/distributed.fsdp.test_fsdp_meta-5c4fcb76b75cf458.xml (deflated 90%) 2024-08-20T23:29:08.4749555Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_uneven/distributed.fsdp.test_fsdp_uneven-413825de11c1c9ce.xml (deflated 44%) 2024-08-20T23:29:08.4751399Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b674b5c67460d7fb.xml (deflated 41%) 2024-08-20T23:29:08.4753351Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-f7c99940971e5a3b.xml (deflated 41%) 2024-08-20T23:29:08.4755299Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-351ac557bfac80cd.xml (deflated 41%) 2024-08-20T23:29:08.4757247Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-2000e11a2a1147f7.xml (deflated 41%) 2024-08-20T23:29:08.4759468Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-57e21009824ee3f2.xml (deflated 41%) 2024-08-20T23:29:08.4761430Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-2b9f65de25ec8989.xml (deflated 40%) 2024-08-20T23:29:08.4763499Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-8356be6c68d4c54f.xml (deflated 41%) 2024-08-20T23:29:08.4765456Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-df2ae10fa289522a.xml (deflated 40%) 2024-08-20T23:29:08.4767415Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-367e496a210db45a.xml (deflated 43%) 2024-08-20T23:29:08.4769359Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b659101fced1a582.xml (deflated 41%) 2024-08-20T23:29:08.4771291Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-db7e30b6e977cddc.xml (deflated 41%) 2024-08-20T23:29:08.4773338Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-148fed8c6d6bd586.xml (deflated 42%) 2024-08-20T23:29:08.4775284Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-01357e41a2080d74.xml (deflated 42%) 2024-08-20T23:29:08.4777209Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-e3d8ec6471d35cff.xml (deflated 42%) 2024-08-20T23:29:08.4779222Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-3903fe232c3f0446.xml (deflated 42%) 2024-08-20T23:29:08.4781177Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-db45be7e887feb70.xml (deflated 42%) 2024-08-20T23:29:08.4783136Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-97b1f42b54fa38cf.xml (deflated 42%) 2024-08-20T23:29:08.4785075Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-ae70ca19ac67583d.xml (deflated 42%) 2024-08-20T23:29:08.4787022Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-1f646df480aa433c.xml (deflated 42%) 2024-08-20T23:29:08.4788956Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-6b8f218b7ad3e5de.xml (deflated 42%) 2024-08-20T23:29:08.4790902Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-f3453986f5d1b53e.xml (deflated 42%) 2024-08-20T23:29:08.4792825Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-487c23d206db2a46.xml (deflated 42%) 2024-08-20T23:29:08.4794769Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-9dcecb3a70a887de.xml (deflated 42%) 2024-08-20T23:29:08.4796707Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-bc77dd55c39376d6.xml (deflated 42%) 2024-08-20T23:29:08.4798659Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-6ebad00a9bdae369.xml (deflated 42%) 2024-08-20T23:29:08.4800591Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-5f6167ab1b9b0d80.xml (deflated 42%) 2024-08-20T23:29:08.4802526Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-f2238c01c032f30d.xml (deflated 42%) 2024-08-20T23:29:08.4804468Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d20d517731f75246.xml (deflated 42%) 2024-08-20T23:29:08.4806472Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-57d80e5c7ad60ccf.xml (deflated 42%) 2024-08-20T23:29:08.4808411Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-fc773f6e5811cde4.xml (deflated 35%) 2024-08-20T23:29:08.4810347Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-33d974e6f0967b6b.xml (deflated 41%) 2024-08-20T23:29:08.4812350Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-839fa155a4d8fcca.xml (deflated 42%) 2024-08-20T23:29:08.4814303Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-98cda65a9ae926af.xml (deflated 42%) 2024-08-20T23:29:08.4816257Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-e53b202a84e7da8a.xml (deflated 42%) 2024-08-20T23:29:08.4818189Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-f057a0a3afa7db1c.xml (deflated 42%) 2024-08-20T23:29:08.4820139Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-30a0be19c0d9bf4c.xml (deflated 42%) 2024-08-20T23:29:08.4822156Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-65ce2762ce2c824c.xml (deflated 41%) 2024-08-20T23:29:08.4824097Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-9c18762501d684fa.xml (deflated 42%) 2024-08-20T23:29:08.4826022Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-7c4ab8543a972c24.xml (deflated 41%) 2024-08-20T23:29:08.4827964Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-a8b0fc40388af312.xml (deflated 41%) 2024-08-20T23:29:08.4829910Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-31380ba933a1bd71.xml (deflated 41%) 2024-08-20T23:29:08.4831851Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d19694048c2f5063.xml (deflated 41%) 2024-08-20T23:29:08.4833771Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-27424ede40c212bf.xml (deflated 42%) 2024-08-20T23:29:08.4835716Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-3d68b8fc7c97e535.xml (deflated 42%) 2024-08-20T23:29:08.4837651Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-319f95408a18296d.xml (deflated 42%) 2024-08-20T23:29:08.4839584Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0a69fbd8cbbc8255.xml (deflated 41%) 2024-08-20T23:29:08.4841517Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-e5797b51613ebe77.xml (deflated 42%) 2024-08-20T23:29:08.4843462Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d6ad52481590fa4e.xml (deflated 41%) 2024-08-20T23:29:08.4845410Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-91ddf1291296df3e.xml (deflated 41%) 2024-08-20T23:29:08.4847411Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d79021eaa1244980.xml (deflated 41%) 2024-08-20T23:29:08.4849356Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b376931caa26bbbf.xml (deflated 42%) 2024-08-20T23:29:08.4851285Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-987547ab5145509b.xml (deflated 42%) 2024-08-20T23:29:08.4853310Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-a0b60f346dd750bb.xml (deflated 41%) 2024-08-20T23:29:08.4855250Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-6f84602cefd3b05a.xml (deflated 42%) 2024-08-20T23:29:08.4857202Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0d2f0c8ed308e8c5.xml (deflated 42%) 2024-08-20T23:29:08.4859366Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-209ffce9f26ea447.xml (deflated 42%) 2024-08-20T23:29:08.4861323Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-233207d7acefa8d5.xml (deflated 42%) 2024-08-20T23:29:08.4863373Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0b4f45ba8c774e42.xml (deflated 42%) 2024-08-20T23:29:08.4865328Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-7d9aba861d2e9284.xml (deflated 42%) 2024-08-20T23:29:08.4867263Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-c0173cf4fe43929c.xml (deflated 42%) 2024-08-20T23:29:08.4869207Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-366de389b3a8dc38.xml (deflated 42%) 2024-08-20T23:29:08.4871146Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-83cbc0f07662b5de.xml (deflated 42%) 2024-08-20T23:29:08.4873088Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-85369d24bf6dc3ed.xml (deflated 42%) 2024-08-20T23:29:08.4875009Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-a9595a103c7d78aa.xml (deflated 42%) 2024-08-20T23:29:08.4876936Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-71bb5e9c747a8ee0.xml (deflated 42%) 2024-08-20T23:29:08.4878879Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d828cf914077cac1.xml (deflated 41%) 2024-08-20T23:29:08.4880821Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b71646bd2ab9ef2e.xml (deflated 41%) 2024-08-20T23:29:08.4882753Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-14d281ccbd4dd366.xml (deflated 42%) 2024-08-20T23:29:08.4884696Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-fae0998cd0734c1c.xml (deflated 42%) 2024-08-20T23:29:08.4886630Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-49e5ec63568b4c0d.xml (deflated 42%) 2024-08-20T23:29:08.4888650Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-32bda4eceebb893c.xml (deflated 42%) 2024-08-20T23:29:08.4890599Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-1339fbe7ed8966ad.xml (deflated 42%) 2024-08-20T23:29:08.4892595Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-9ec343d863b21956.xml (deflated 42%) 2024-08-20T23:29:08.4894547Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0bfb0c95231a504d.xml (deflated 42%) 2024-08-20T23:29:08.4896502Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-56f1d60a1d589c4e.xml (deflated 42%) 2024-08-20T23:29:08.4898452Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b5773031a4e3f2cd.xml (deflated 42%) 2024-08-20T23:29:08.4900385Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-cb998d81099d8d0d.xml (deflated 42%) 2024-08-20T23:29:08.4902322Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-535450110c6582d3.xml (deflated 41%) 2024-08-20T23:29:08.4904387Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-a2cf7e4f01cbf915.xml (deflated 41%) 2024-08-20T23:29:08.4906345Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-842c2d39e1d3b4a3.xml (deflated 41%) 2024-08-20T23:29:08.4908272Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-c0878e004b16a5ef.xml (deflated 41%) 2024-08-20T23:29:08.4910222Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-7eb9e7e0cf42d31a.xml (deflated 41%) 2024-08-20T23:29:08.4912172Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0ed050e6dae6ef78.xml (deflated 41%) 2024-08-20T23:29:08.4914128Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-77916fcff9f06969.xml (deflated 41%) 2024-08-20T23:29:08.4916061Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-c34e3e73dcced68f.xml (deflated 41%) 2024-08-20T23:29:08.4918003Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-2fa7b226add40d06.xml (deflated 41%) 2024-08-20T23:29:08.4919943Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-2ae7a122a2cb6149.xml (deflated 41%) 2024-08-20T23:29:08.4921878Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-91070e05d31cc58f.xml (deflated 41%) 2024-08-20T23:29:08.4923805Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-86ae2a31b60b3924.xml (deflated 41%) 2024-08-20T23:29:08.4925743Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d8a82aae06b88fa8.xml (deflated 41%) 2024-08-20T23:29:08.4927690Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-5f8f17465fedd485.xml (deflated 41%) 2024-08-20T23:29:08.4929693Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-1e55c9ede54e5371.xml (deflated 41%) 2024-08-20T23:29:08.4931629Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-7339fb939e9a426a.xml (deflated 41%) 2024-08-20T23:29:08.4933666Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b58f93c164fd76d9.xml (deflated 42%) 2024-08-20T23:29:08.4935619Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-98ffea26f4b1d5eb.xml (deflated 42%) 2024-08-20T23:29:08.4937571Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-ab02e873f2f744b4.xml (deflated 42%) 2024-08-20T23:29:08.4939514Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-66b7f3c4e05cd609.xml (deflated 42%) 2024-08-20T23:29:08.4941443Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-baf5a312f615103a.xml (deflated 43%) 2024-08-20T23:29:08.4943388Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-a477eb3b5e089eee.xml (deflated 42%) 2024-08-20T23:29:08.4945321Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0156dba44794d5f1.xml (deflated 35%) 2024-08-20T23:29:08.4947130Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a81ec391f404e89b.xml (deflated 36%) 2024-08-20T23:29:08.4948925Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3090bb297b9f5147.xml (deflated 35%) 2024-08-20T23:29:08.4950732Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-038447ccbaff07e2.xml (deflated 35%) 2024-08-20T23:29:08.4952537Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3648a9208e6b1111.xml (deflated 35%) 2024-08-20T23:29:08.4954334Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-369341ebb4b8f49d.xml (deflated 35%) 2024-08-20T23:29:08.4956114Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-28177d7f83ef999f.xml (deflated 35%) 2024-08-20T23:29:08.4958132Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e7488d50cabc95f8.xml (deflated 36%) 2024-08-20T23:29:08.4959958Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-04da2b2c1c602d7f.xml (deflated 36%) 2024-08-20T23:29:08.4961777Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-19a5d122881cc8ca.xml (deflated 35%) 2024-08-20T23:29:08.4963572Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-aadd0f95d5f3ce23.xml (deflated 35%) 2024-08-20T23:29:08.4965401Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b3b753a0951f5ac8.xml (deflated 35%) 2024-08-20T23:29:08.4967200Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-db77f59ec1f840d3.xml (deflated 35%) 2024-08-20T23:29:08.4968998Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1369979b1ac08f61.xml (deflated 35%) 2024-08-20T23:29:08.4970905Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-012ad8ada5e68c60.xml (deflated 35%) 2024-08-20T23:29:08.4972804Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-80c8b01fa26bfd11.xml (deflated 35%) 2024-08-20T23:29:08.4974606Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-069374e9ae1c879f.xml (deflated 35%) 2024-08-20T23:29:08.4976414Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2fe9e720a7ccc5f2.xml (deflated 35%) 2024-08-20T23:29:08.4978212Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-275ecfc30d85ff65.xml (deflated 35%) 2024-08-20T23:29:08.4980025Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2c0e42c3bff0691d.xml (deflated 36%) 2024-08-20T23:29:08.4981846Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d9b0b52c9bc10ad7.xml (deflated 35%) 2024-08-20T23:29:08.4984118Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a7dbec7bc5f99994.xml (deflated 35%) 2024-08-20T23:29:08.4986071Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e399fcd4f3a44d98.xml (deflated 35%) 2024-08-20T23:29:08.4987967Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-71f021f74a44a362.xml (deflated 35%) 2024-08-20T23:29:08.4989781Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fb6cc4393dcdaaeb.xml (deflated 35%) 2024-08-20T23:29:08.4991615Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-51bd615130ceca0a.xml (deflated 35%) 2024-08-20T23:29:08.4993414Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2361570c879082bf.xml (deflated 35%) 2024-08-20T23:29:08.4995208Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7b5daf7215ac22bb.xml (deflated 36%) 2024-08-20T23:29:08.4997021Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0d0e58244bd2e93b.xml (deflated 35%) 2024-08-20T23:29:08.4998836Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c1e1abfa83d48116.xml (deflated 36%) 2024-08-20T23:29:08.5000646Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6b2a0f9d9ec61726.xml (deflated 36%) 2024-08-20T23:29:08.5002445Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a054f38b7973c40a.xml (deflated 35%) 2024-08-20T23:29:08.5004254Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8f8d54f2ae82cdfd.xml (deflated 35%) 2024-08-20T23:29:08.5006060Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7867d248559d9526.xml (deflated 34%) 2024-08-20T23:29:08.5007860Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8693851ab1810a6d.xml (deflated 35%) 2024-08-20T23:29:08.5009646Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b515d0f00fe13c51.xml (deflated 35%) 2024-08-20T23:29:08.5011506Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0c5bbb70b712e90f.xml (deflated 35%) 2024-08-20T23:29:08.5013376Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fcefa4b9504b040f.xml (deflated 35%) 2024-08-20T23:29:08.5015185Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4441c04d2d7d8a7f.xml (deflated 35%) 2024-08-20T23:29:08.5016975Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9b320576f12ecf95.xml (deflated 35%) 2024-08-20T23:29:08.5018774Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c40e1dd0e98941ba.xml (deflated 35%) 2024-08-20T23:29:08.5020584Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a33ee264e72cac8a.xml (deflated 35%) 2024-08-20T23:29:08.5022387Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c9ff721022add040.xml (deflated 35%) 2024-08-20T23:29:08.5024177Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-54b06ff97a13bef2.xml (deflated 35%) 2024-08-20T23:29:08.5025980Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-10df10d4f893414d.xml (deflated 36%) 2024-08-20T23:29:08.5027859Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c82b20d20d3398e7.xml (deflated 35%) 2024-08-20T23:29:08.5029655Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-105a7488272b3393.xml (deflated 35%) 2024-08-20T23:29:08.5031438Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-be8315c7fbbacaf5.xml (deflated 35%) 2024-08-20T23:29:08.5033257Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d76e49b8f0ca148b.xml (deflated 35%) 2024-08-20T23:29:08.5035074Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e6dac7c73462baf5.xml (deflated 35%) 2024-08-20T23:29:08.5036879Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-833cbc9e8e1fe29a.xml (deflated 34%) 2024-08-20T23:29:08.5038656Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b296614f62504465.xml (deflated 35%) 2024-08-20T23:29:08.5040443Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6d4b4dd4c40620e2.xml (deflated 34%) 2024-08-20T23:29:08.5042240Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-609c4e696cbee1d1.xml (deflated 34%) 2024-08-20T23:29:08.5044038Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e2bf9e180512fba9.xml (deflated 34%) 2024-08-20T23:29:08.5045826Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-43f17ead64e22c58.xml (deflated 34%) 2024-08-20T23:29:08.5047636Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e13efe6cfca06b9c.xml (deflated 35%) 2024-08-20T23:29:08.5049436Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4914b2ecc46ff0f3.xml (deflated 36%) 2024-08-20T23:29:08.5051235Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-480b893d06373866.xml (deflated 35%) 2024-08-20T23:29:08.5053174Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e52a356bba8440a2.xml (deflated 35%) 2024-08-20T23:29:08.5054976Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3414925d2cd1b371.xml (deflated 34%) 2024-08-20T23:29:08.5056774Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-28f0ed6bdd49a3e2.xml (deflated 34%) 2024-08-20T23:29:08.5058846Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fca092c7b84b59e1.xml (deflated 34%) 2024-08-20T23:29:08.5060646Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d6fa08920e1514a2.xml (deflated 36%) 2024-08-20T23:29:08.5062460Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ac8884cc6650feac.xml (deflated 34%) 2024-08-20T23:29:08.5064269Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-60b917ce77229b6f.xml (deflated 35%) 2024-08-20T23:29:08.5066067Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3000351a56e1b056.xml (deflated 35%) 2024-08-20T23:29:08.5067949Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-42acb16a37f1fdc7.xml (deflated 35%) 2024-08-20T23:29:08.5069763Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-21bba8448a16b921.xml (deflated 34%) 2024-08-20T23:29:08.5071573Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5a371127c7dd06a1.xml (deflated 35%) 2024-08-20T23:29:08.5073381Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-46937c771d9ecb33.xml (deflated 34%) 2024-08-20T23:29:08.5075168Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1689e827424f562e.xml (deflated 35%) 2024-08-20T23:29:08.5076968Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d36eb86338da70d6.xml (deflated 34%) 2024-08-20T23:29:08.5078781Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5b0b09f0a9b13f72.xml (deflated 35%) 2024-08-20T23:29:08.5080585Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-746972eba2b2bf65.xml (deflated 34%) 2024-08-20T23:29:08.5082388Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4f11c9d48149d0df.xml (deflated 34%) 2024-08-20T23:29:08.5084185Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5d35bf3814aee56e.xml (deflated 35%) 2024-08-20T23:29:08.5085985Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fff3b635a386b7fa.xml (deflated 35%) 2024-08-20T23:29:08.5087789Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-329dfa3a08e3cfa9.xml (deflated 34%) 2024-08-20T23:29:08.5089594Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f275cca4e65439bd.xml (deflated 34%) 2024-08-20T23:29:08.5091377Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e6694df511beb76a.xml (deflated 34%) 2024-08-20T23:29:08.5093362Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c883040852bc575f.xml (deflated 34%) 2024-08-20T23:29:08.5095172Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0efb7004c154731b.xml (deflated 34%) 2024-08-20T23:29:08.5096979Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-851ca597a17c08ac.xml (deflated 36%) 2024-08-20T23:29:08.5098773Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-853949fc030383c9.xml (deflated 35%) 2024-08-20T23:29:08.5100570Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8ab26b4e82bafe05.xml (deflated 36%) 2024-08-20T23:29:08.5102376Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0e47cdaee40789bb.xml (deflated 35%) 2024-08-20T23:29:08.5104178Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-941b364ebf92b0c8.xml (deflated 36%) 2024-08-20T23:29:08.5105966Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e142a0071becfdc6.xml (deflated 55%) 2024-08-20T23:29:08.5107770Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0b124f845984eef4.xml (deflated 34%) 2024-08-20T23:29:08.5109631Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b40e91517772e033.xml (deflated 34%) 2024-08-20T23:29:08.5111430Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b872634569b4b1bf.xml (deflated 37%) 2024-08-20T23:29:08.5113225Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7ea4cb6b638c7d19.xml (deflated 37%) 2024-08-20T23:29:08.5115017Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ee28537e3612d548.xml (deflated 37%) 2024-08-20T23:29:08.5116816Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e63d4b5b8fd47275.xml (deflated 36%) 2024-08-20T23:29:08.5118608Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-901360e09673225a.xml (deflated 37%) 2024-08-20T23:29:08.5120389Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7106b7ea1e607d17.xml (deflated 37%) 2024-08-20T23:29:08.5122192Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5725b55dbbba2903.xml (deflated 37%) 2024-08-20T23:29:08.5124000Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-311e5e178bb38e69.xml (deflated 37%) 2024-08-20T23:29:08.5125797Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b4d59baab73c19dd.xml (deflated 36%) 2024-08-20T23:29:08.5127594Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c59a6d1ccb5e3c42.xml (deflated 37%) 2024-08-20T23:29:08.5129400Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8f22cbe19fd3a56a.xml (deflated 36%) 2024-08-20T23:29:08.5131206Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b630d472291fe545.xml (deflated 37%) 2024-08-20T23:29:08.5133104Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9f1663ae55616472.xml (deflated 38%) 2024-08-20T23:29:08.5134969Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d9837c7c80a81573.xml (deflated 36%) 2024-08-20T23:29:08.5136766Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0166c9011d32938b.xml (deflated 36%) 2024-08-20T23:29:08.5138575Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3d1f83df2cf832f3.xml (deflated 36%) 2024-08-20T23:29:08.5140397Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f096e48d876e8c16.xml (deflated 36%) 2024-08-20T23:29:08.5142186Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-50c659dc384e8348.xml (deflated 35%) 2024-08-20T23:29:08.5143995Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8565de345d1f98fd.xml (deflated 37%) 2024-08-20T23:29:08.5145798Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b9644b3ce347b736.xml (deflated 36%) 2024-08-20T23:29:08.5147598Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-db025c8c23f31dd6.xml (deflated 36%) 2024-08-20T23:29:08.5149428Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-07a3799700061581.xml (deflated 36%) 2024-08-20T23:29:08.5151228Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-46ee599a9f71944d.xml (deflated 36%) 2024-08-20T23:29:08.5153030Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8db901ba872b93ed.xml (deflated 37%) 2024-08-20T23:29:08.5154847Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-21d2da6c6bb8a152.xml (deflated 37%) 2024-08-20T23:29:08.5156825Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_binary_cmp/distributed._shard.sharded_tensor.ops.test_binary_cmp-83094434319fca73.xml (deflated 80%) 2024-08-20T23:29:08.5159106Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-9a8afafa04325ba1.xml (deflated 36%) 2024-08-20T23:29:08.5160833Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-90bf5184c640d426.xml (deflated 36%) 2024-08-20T23:29:08.5162543Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-186f4aa6a7d8f681.xml (deflated 36%) 2024-08-20T23:29:08.5164236Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-93d4456b0bca6317.xml (deflated 36%) 2024-08-20T23:29:08.5165939Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-5a6ed227cc95ba8d.xml (deflated 35%) 2024-08-20T23:29:08.5167655Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-c73d87392e4748ea.xml (deflated 37%) 2024-08-20T23:29:08.5169360Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-9b95a2add4b9ad63.xml (deflated 36%) 2024-08-20T23:29:08.5171060Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-dce3f608e9980852.xml (deflated 35%) 2024-08-20T23:29:08.5172832Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-c9d4e499318d6c38.xml (deflated 36%) 2024-08-20T23:29:08.5174661Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-1ed0841060d522bc.xml (deflated 36%) 2024-08-20T23:29:08.5176367Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-bbb8a892f3c73a7d.xml (deflated 36%) 2024-08-20T23:29:08.5178068Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-be4ae636453c7c4f.xml (deflated 35%) 2024-08-20T23:29:08.5179782Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-cde19ef63efb6377.xml (deflated 36%) 2024-08-20T23:29:08.5181506Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-7b955604da98f7a2.xml (deflated 36%) 2024-08-20T23:29:08.5183216Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-b6bbce1524607e9f.xml (deflated 36%) 2024-08-20T23:29:08.5184908Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-31769b4fa4fa1189.xml (deflated 36%) 2024-08-20T23:29:08.5186604Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-3f07082f881b0d52.xml (deflated 36%) 2024-08-20T23:29:08.5188306Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-a2666cefcfcf2880.xml (deflated 36%) 2024-08-20T23:29:08.5190078Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-8f3a328efc914c6a.xml (deflated 37%) 2024-08-20T23:29:08.5191769Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-be418baf2c953c27.xml (deflated 35%) 2024-08-20T23:29:08.5193465Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-5107877d8b54b8de.xml (deflated 36%) 2024-08-20T23:29:08.5195179Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-a330015c9f0ce753.xml (deflated 35%) 2024-08-20T23:29:08.5196896Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-03ddc02f968cf145.xml (deflated 36%) 2024-08-20T23:29:08.5198592Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-66bbbd2c5d2c8dcf.xml (deflated 35%) 2024-08-20T23:29:08.5200305Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-c8c4c75c4b066709.xml (deflated 36%) 2024-08-20T23:29:08.5202017Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-0dd146d71d1c6a2b.xml (deflated 35%) 2024-08-20T23:29:08.5203716Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-499c85e731d27335.xml (deflated 35%) 2024-08-20T23:29:08.5205407Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-74acc40ad73be641.xml (deflated 36%) 2024-08-20T23:29:08.5207239Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_multiple_forward/distributed.fsdp.test_fsdp_multiple_forward-537750a110c8801e.xml (deflated 46%) 2024-08-20T23:29:08.5225544Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_comm/distributed.fsdp.test_fsdp_comm-61c5094b114d402e.xml (deflated 87%) 2024-08-20T23:29:08.5227456Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_init/distributed._shard.sharded_tensor.ops.test_init-f198bf1f727c88a2.xml (deflated 75%) 2024-08-20T23:29:08.5229528Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.test_sharded_tensor/distributed._shard.sharded_tensor.test_sharded_tensor-41f6e42920145c36.xml (deflated 95%) 2024-08-20T23:29:08.5231933Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.test_sharded_tensor_reshard/distributed._shard.sharded_tensor.test_sharded_tensor_reshard-32053940a4061e59.xml (deflated 70%) 2024-08-20T23:29:08.5234156Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_embedding/distributed._shard.sharded_tensor.ops.test_embedding-102686ad240618a0.xml (deflated 68%) 2024-08-20T23:29:08.5236336Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_embedding_bag/distributed._shard.sharded_tensor.ops.test_embedding_bag-a00f310f8ece9083.xml (deflated 68%) 2024-08-20T23:29:08.5238189Z adding: test/test-reports/python-pytest/distributed.test_launcher/distributed.test_launcher-a09239a304f2bea0.xml (deflated 40%) 2024-08-20T23:29:08.5239675Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-d7c50e0482ab8e08.xml (deflated 44%) 2024-08-20T23:29:08.5241148Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-096874eb8bff192c.xml (deflated 37%) 2024-08-20T23:29:08.5242610Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-595c52da5fdb8c2e.xml (deflated 39%) 2024-08-20T23:29:08.5244075Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-1670be0cadfed4e6.xml (deflated 43%) 2024-08-20T23:29:08.5245528Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-ee330173ae6200ab.xml (deflated 44%) 2024-08-20T23:29:08.5247040Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-9a473b64021e75ed.xml (deflated 36%) 2024-08-20T23:29:08.5248504Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-3265e56c8a60ecad.xml (deflated 36%) 2024-08-20T23:29:08.5249939Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-2d5f92193b600791.xml (deflated 37%) 2024-08-20T23:29:08.5251400Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-bdf06160cc2eeca6.xml (deflated 37%) 2024-08-20T23:29:08.5252965Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-3982aca6a65dafdf.xml (deflated 37%) 2024-08-20T23:29:08.5254415Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-dea82d9b6d46269b.xml (deflated 37%) 2024-08-20T23:29:08.5255862Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-458c1fb70ab4f70d.xml (deflated 37%) 2024-08-20T23:29:08.5257310Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-d162137f20032bdf.xml (deflated 37%) 2024-08-20T23:29:08.5259025Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-3ff9c6dfc01b579e.xml (deflated 37%) 2024-08-20T23:29:08.5260493Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-f5e7edfc8381247e.xml (deflated 37%) 2024-08-20T23:29:08.5261939Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-fdb31b0c925e1ed8.xml (deflated 43%) 2024-08-20T23:29:08.5263389Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-667c574897c8401b.xml (deflated 36%) 2024-08-20T23:29:08.5264835Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-78fb0b5c1dcfd186.xml (deflated 43%) 2024-08-20T23:29:08.5266285Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-d33f77ad4dd786ac.xml (deflated 43%) 2024-08-20T23:29:08.5267737Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-5f0b032ebb7aae47.xml (deflated 36%) 2024-08-20T23:29:08.5269188Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-37e753b3a737f5d8.xml (deflated 36%) 2024-08-20T23:29:08.5270755Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-73f895283b847551.xml (deflated 35%) 2024-08-20T23:29:08.5272292Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-18a58a1c8b074fed.xml (deflated 36%) 2024-08-20T23:29:08.5273744Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-0687cdb69103a010.xml (deflated 36%) 2024-08-20T23:29:08.5275184Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-11ce4cc5979c2eb7.xml (deflated 38%) 2024-08-20T23:29:08.5276648Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-cef329d0b6521c1f.xml (deflated 35%) 2024-08-20T23:29:08.5278086Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-95178cdd9153b46f.xml (deflated 36%) 2024-08-20T23:29:08.5279529Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-5f1a1428533cd14d.xml (deflated 34%) 2024-08-20T23:29:08.5280975Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-131a446af046d60f.xml (deflated 35%) 2024-08-20T23:29:08.5282422Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-a7c837ed3a45b40b.xml (deflated 37%) 2024-08-20T23:29:08.5283862Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-1b5106cabcba5974.xml (deflated 36%) 2024-08-20T23:29:08.5285316Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-510e96611d5d89e8.xml (deflated 36%) 2024-08-20T23:29:08.5286865Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-03c9a4146bf2ab5b.xml (deflated 36%) 2024-08-20T23:29:08.5288317Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-479588949c527038.xml (deflated 36%) 2024-08-20T23:29:08.5289745Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-673cc8e2fa4a7421.xml (deflated 36%) 2024-08-20T23:29:08.5291188Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-cfacf4955794146f.xml (deflated 35%) 2024-08-20T23:29:08.5292721Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-58b6526e71adf2d0.xml (deflated 34%) 2024-08-20T23:29:08.5294160Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-c052bbcf52a748a3.xml (deflated 35%) 2024-08-20T23:29:08.5295613Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-8d96a69d8b2824aa.xml (deflated 35%) 2024-08-20T23:29:08.5297069Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-df6d50fdc68b44cb.xml (deflated 38%) 2024-08-20T23:29:08.5298522Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-7fba6fe93bc817ab.xml (deflated 34%) 2024-08-20T23:29:08.5299969Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-9eb983bb789168a6.xml (deflated 34%) 2024-08-20T23:29:08.5301413Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-fd5e49ad2ffc19f8.xml (deflated 34%) 2024-08-20T23:29:08.5302862Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-ee98162992d7529e.xml (deflated 34%) 2024-08-20T23:29:08.5304301Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-b2af30b33ad34d77.xml (deflated 35%) 2024-08-20T23:29:08.5305740Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-ba6e4d6317d359d0.xml (deflated 35%) 2024-08-20T23:29:08.5307177Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-6d2506eb66cdaa20.xml (deflated 35%) 2024-08-20T23:29:08.5308618Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-717f15b6e8f8d822.xml (deflated 35%) 2024-08-20T23:29:08.5310116Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-18121ac58cc55a56.xml (deflated 34%) 2024-08-20T23:29:08.5311562Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-c0ff97008905c190.xml (deflated 35%) 2024-08-20T23:29:08.5312991Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-54b4da2d72f2117b.xml (deflated 34%) 2024-08-20T23:29:08.5314430Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-23a32913650ddf26.xml (deflated 36%) 2024-08-20T23:29:08.5315858Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-9b9227a86be199e9.xml (deflated 35%) 2024-08-20T23:29:08.5317310Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-7d86666ad9ffaedd.xml (deflated 35%) 2024-08-20T23:29:08.5318744Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-81834e4c95652425.xml (deflated 35%) 2024-08-20T23:29:08.5320195Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-5b3829bd66a3ba15.xml (deflated 35%) 2024-08-20T23:29:08.5321624Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-a2c4ec177de40c8e.xml (deflated 35%) 2024-08-20T23:29:08.5323063Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-cd50c63609aa2000.xml (deflated 35%) 2024-08-20T23:29:08.5324508Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-c3e2df059bc6b01f.xml (deflated 37%) 2024-08-20T23:29:08.5326008Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-fd9948da96eac573.xml (deflated 35%) 2024-08-20T23:29:08.5327453Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-e73f55b01b5b3688.xml (deflated 35%) 2024-08-20T23:29:08.5328897Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-aa1f7a9178bb2366.xml (deflated 35%) 2024-08-20T23:29:08.5330347Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-4ee2fcfe035b94b3.xml (deflated 36%) 2024-08-20T23:29:08.5331886Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-7bd24c0f685c9559.xml (deflated 35%) 2024-08-20T23:29:08.5333333Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-230cb43e811a9eea.xml (deflated 36%) 2024-08-20T23:29:08.5335297Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-2bb09f097ad58e4b.xml (deflated 34%) 2024-08-20T23:29:08.5336812Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-556208fb815afe4d.xml (deflated 35%) 2024-08-20T23:29:08.5338301Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-3ae6a306d2536b0d.xml (deflated 35%) 2024-08-20T23:29:08.5339756Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-585d92e8270e0953.xml (deflated 35%) 2024-08-20T23:29:08.5341392Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-ed8eac06b5bd38c0.xml (deflated 35%) 2024-08-20T23:29:08.5342854Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-1b61828e7a110e54.xml (deflated 36%) 2024-08-20T23:29:08.5344304Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-2698f41ddee14c42.xml (deflated 35%) 2024-08-20T23:29:08.5345763Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-8118626fd90be4bd.xml (deflated 35%) 2024-08-20T23:29:08.5347196Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-f540e7b19c06b04e.xml (deflated 35%) 2024-08-20T23:29:08.5348650Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-918ff8e09bca72d4.xml (deflated 36%) 2024-08-20T23:29:08.5350202Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-1ea2ae655d57f762.xml (deflated 35%) 2024-08-20T23:29:08.5351657Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-91e909569fbcd6fa.xml (deflated 35%) 2024-08-20T23:29:08.5353098Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-15f002bf640aff72.xml (deflated 35%) 2024-08-20T23:29:08.5354547Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-2125986792ec0673.xml (deflated 35%) 2024-08-20T23:29:08.5356003Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-552a9b22c443acc5.xml (deflated 35%) 2024-08-20T23:29:08.5357445Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-590d78862d1187fc.xml (deflated 35%) 2024-08-20T23:29:08.5359154Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-c6f4ff537b78e361.xml (deflated 36%) 2024-08-20T23:29:08.5360616Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-ef707d8502ea7528.xml (deflated 36%) 2024-08-20T23:29:08.5362079Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-a20958cafa5e11c9.xml (deflated 33%) 2024-08-20T23:29:08.5363517Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-c804986aecdca243.xml (deflated 33%) 2024-08-20T23:29:08.5365068Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-66238fb8c8f175a3.xml (deflated 35%) 2024-08-20T23:29:08.5366639Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-9d363727a4befd23.xml (deflated 43%) 2024-08-20T23:29:08.5368287Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-a4371b688182bd2a.xml (deflated 43%) 2024-08-20T23:29:08.5369931Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-8e769bd3685cdad5.xml (deflated 43%) 2024-08-20T23:29:08.5371589Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-7fefda0bae002c0c.xml (deflated 45%) 2024-08-20T23:29:08.5373389Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-1f48f6d6aba5f71a.xml (deflated 45%) 2024-08-20T23:29:08.5375597Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-7edeb689781a95e7.xml (deflated 45%) 2024-08-20T23:29:08.5377285Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-eab5d9500011ae48.xml (deflated 45%) 2024-08-20T23:29:08.5379011Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-bacf6b0b4ae40010.xml (deflated 45%) 2024-08-20T23:29:08.5380750Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-821e6ca0bc590966.xml (deflated 45%) 2024-08-20T23:29:08.5382404Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-aa9077c8491e1393.xml (deflated 43%) 2024-08-20T23:29:08.5384066Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-073a82abd5a9ffa9.xml (deflated 43%) 2024-08-20T23:29:08.5385754Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-69fe9e2e29540e76.xml (deflated 43%) 2024-08-20T23:29:08.5387420Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-8183d4d91b4ceebe.xml (deflated 43%) 2024-08-20T23:29:08.5389088Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-a8fb7d229836186b.xml (deflated 45%) 2024-08-20T23:29:08.5390845Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-c7458d4c7360644b.xml (deflated 46%) 2024-08-20T23:29:08.5392507Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-e818b49bec667aa1.xml (deflated 46%) 2024-08-20T23:29:08.5394179Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-675024297d917b52.xml (deflated 46%) 2024-08-20T23:29:08.5395853Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-58f1a2131e0ab24b.xml (deflated 46%) 2024-08-20T23:29:08.5397510Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-98aae62a868af978.xml (deflated 46%) 2024-08-20T23:29:08.5399175Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-d34b78e9e15f5e60.xml (deflated 45%) 2024-08-20T23:29:08.5400850Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-b8acb9ac5d2313c4.xml (deflated 45%) 2024-08-20T23:29:08.5402522Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-e373f438cc7ba821.xml (deflated 45%) 2024-08-20T23:29:08.5404271Z adding: test/test-reports/python-pytest/distributed.elastic.metrics.api_test/distributed.elastic.metrics.api_test-c55d6bccca09699d.xml (deflated 60%) 2024-08-20T23:29:08.5406266Z adding: test/test-reports/python-pytest/distributed.elastic.utils.distributed_test/distributed.elastic.utils.distributed_test-80577c8e749a4021.xml (deflated 78%) 2024-08-20T23:29:08.5408189Z adding: test/test-reports/python-pytest/distributed.elastic.events.lib_test/distributed.elastic.events.lib_test-ad7ded1a25b44fec.xml (deflated 76%) 2024-08-20T23:29:08.5410168Z adding: test/test-reports/python-pytest/distributed.elastic.multiprocessing.api_test/distributed.elastic.multiprocessing.api_test-7eabbc8ac6ec3c5f.xml (deflated 88%) 2024-08-20T23:29:08.5412326Z adding: test/test-reports/python-pytest/distributed.elastic.timer.local_timer_example/distributed.elastic.timer.local_timer_example-08a2a86604e722fa.xml (deflated 51%) 2024-08-20T23:29:08.5414341Z adding: test/test-reports/python-pytest/distributed.elastic.timer.local_timer_test/distributed.elastic.timer.local_timer_test-464af8cc37b51a12.xml (deflated 80%) 2024-08-20T23:29:08.5416228Z adding: test/test-reports/python-pytest/distributed.elastic.utils.util_test/distributed.elastic.utils.util_test-46040bb3c98ff139.xml (deflated 81%) 2024-08-20T23:29:08.5418110Z adding: test/test-reports/python-pytest/distributed.elastic.utils.logging_test/distributed.elastic.utils.logging_test-1d09ca1e1e125dd4.xml (deflated 51%) 2024-08-20T23:29:08.5419887Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-393d8aa85e00e565.xml (deflated 42%) 2024-08-20T23:29:08.5421559Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5f108134bcb21379.xml (deflated 43%) 2024-08-20T23:29:08.5423234Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1eb92d828ad3b747.xml (deflated 38%) 2024-08-20T23:29:08.5424907Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6fdd7766b4eb78f3.xml (deflated 36%) 2024-08-20T23:29:08.5426563Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f3b505db359ccb36.xml (deflated 41%) 2024-08-20T23:29:08.5428245Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e317aa6ce35023da.xml (deflated 41%) 2024-08-20T23:29:08.5429922Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7a5ee319836b6c0d.xml (deflated 41%) 2024-08-20T23:29:08.5431648Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b7171071298eabed.xml (deflated 41%) 2024-08-20T23:29:08.5433290Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-59d509530baf1cb1.xml (deflated 36%) 2024-08-20T23:29:08.5434949Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2def0147234171e9.xml (deflated 35%) 2024-08-20T23:29:08.5436613Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bb3591124b04d92c.xml (deflated 35%) 2024-08-20T23:29:08.5438281Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-864dfbb39947c20e.xml (deflated 36%) 2024-08-20T23:29:08.5439930Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-719b1b6f26c65786.xml (deflated 43%) 2024-08-20T23:29:08.5441603Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4d39ba884c21c952.xml (deflated 45%) 2024-08-20T23:29:08.5443277Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9d08214ccf09e4e5.xml (deflated 35%) 2024-08-20T23:29:08.5444944Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9fab808d7882d0d9.xml (deflated 35%) 2024-08-20T23:29:08.5446657Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d9fdc85189b8966e.xml (deflated 35%) 2024-08-20T23:29:08.5448320Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b820e11a97255d30.xml (deflated 35%) 2024-08-20T23:29:08.5449969Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-47270ce634ca61e2.xml (deflated 36%) 2024-08-20T23:29:08.5451632Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c3787df21629464f.xml (deflated 35%) 2024-08-20T23:29:08.5453369Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c8c6145ff288ee99.xml (deflated 35%) 2024-08-20T23:29:08.5455037Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-674e8b05ca7acd76.xml (deflated 35%) 2024-08-20T23:29:08.5456714Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fe75fd37ec71dbb2.xml (deflated 36%) 2024-08-20T23:29:08.5458634Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8cab74105a90d0df.xml (deflated 36%) 2024-08-20T23:29:08.5460293Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c863361b13a4088f.xml (deflated 35%) 2024-08-20T23:29:08.5461957Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2841badc5745f2d5.xml (deflated 35%) 2024-08-20T23:29:08.5463610Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b7ae35ff1617ce88.xml (deflated 35%) 2024-08-20T23:29:08.5465267Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b0eaca32299c99c8.xml (deflated 35%) 2024-08-20T23:29:08.5466924Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b59a314d3bb98c07.xml (deflated 35%) 2024-08-20T23:29:08.5468583Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a78570ed8ef5b740.xml (deflated 35%) 2024-08-20T23:29:08.5470240Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-32090a63868fdc0a.xml (deflated 35%) 2024-08-20T23:29:08.5472015Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b82567ca02bcf22c.xml (deflated 35%) 2024-08-20T23:29:08.5473658Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7b2578cba624e969.xml (deflated 42%) 2024-08-20T23:29:08.5475316Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f7a220df0807b2c6.xml (deflated 43%) 2024-08-20T23:29:08.5476987Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-05dcd9e626b475a7.xml (deflated 43%) 2024-08-20T23:29:08.5478648Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-09fa502d707f8853.xml (deflated 43%) 2024-08-20T23:29:08.5480301Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-915996b7c11ca93d.xml (deflated 44%) 2024-08-20T23:29:08.5481955Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-09693f78e52a0fe2.xml (deflated 44%) 2024-08-20T23:29:08.5483611Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-62f914d5073f7b87.xml (deflated 44%) 2024-08-20T23:29:08.5485278Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ba9d9cf4d28c2a80.xml (deflated 44%) 2024-08-20T23:29:08.5487005Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a75a1aee5695ab79.xml (deflated 44%) 2024-08-20T23:29:08.5488669Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-80018a38db7b56dd.xml (deflated 44%) 2024-08-20T23:29:08.5490335Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3b3be159072e4a18.xml (deflated 43%) 2024-08-20T23:29:08.5492069Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-66e66cbc66ee3b9f.xml (deflated 36%) 2024-08-20T23:29:08.5493727Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6285847dc4da751a.xml (deflated 42%) 2024-08-20T23:29:08.5495399Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2afec514f458bbee.xml (deflated 42%) 2024-08-20T23:29:08.5497079Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-aa626b2db4da3a2a.xml (deflated 35%) 2024-08-20T23:29:08.5498751Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-42d3099f4df8733a.xml (deflated 35%) 2024-08-20T23:29:08.5500404Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f4091fe5f425497b.xml (deflated 42%) 2024-08-20T23:29:08.5502068Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-967a36cf52d0fed1.xml (deflated 42%) 2024-08-20T23:29:08.5503733Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-692fceb00c3589ca.xml (deflated 36%) 2024-08-20T23:29:08.5505835Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-970c1995f7584699.xml (deflated 47%) 2024-08-20T23:29:08.5507544Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0bf24f1684f526df.xml (deflated 47%) 2024-08-20T23:29:08.5509206Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2120f23240dca6d6.xml (deflated 47%) 2024-08-20T23:29:08.5510919Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-56edb5782003ec6c.xml (deflated 41%) 2024-08-20T23:29:08.5512660Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-17a72a5205112774.xml (deflated 42%) 2024-08-20T23:29:08.5514676Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4dc825b1c102e71a.xml (deflated 41%) 2024-08-20T23:29:08.5516345Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-73bb39e279fde256.xml (deflated 42%) 2024-08-20T23:29:08.5518004Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c5f8001daf0068fa.xml (deflated 36%) 2024-08-20T23:29:08.5519666Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e60eb0199a556004.xml (deflated 42%) 2024-08-20T23:29:08.5521856Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bf843b8f5c1a6878.xml (deflated 42%) 2024-08-20T23:29:08.5524025Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-72ac312b991d9b28.xml (deflated 42%) 2024-08-20T23:29:08.5525709Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1259ee247f4f0701.xml (deflated 35%) 2024-08-20T23:29:08.5527406Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ec49216e3a22a7c0.xml (deflated 42%) 2024-08-20T23:29:08.5529201Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1d4ab2362d5e4173.xml (deflated 43%) 2024-08-20T23:29:08.5530871Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-85cdbac251a5722f.xml (deflated 43%) 2024-08-20T23:29:08.5532613Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-041e5d8d75ceb96a.xml (deflated 42%) 2024-08-20T23:29:08.5534276Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e525470402263a10.xml (deflated 42%) 2024-08-20T23:29:08.5535915Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3a2067ab274006fe.xml (deflated 42%) 2024-08-20T23:29:08.5537577Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3475e60a8ee0b032.xml (deflated 44%) 2024-08-20T23:29:08.5539243Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-369b276bb0fd433d.xml (deflated 42%) 2024-08-20T23:29:08.5540895Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b447ba31f67cee6a.xml (deflated 43%) 2024-08-20T23:29:08.5542563Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0342e9d92c35e6df.xml (deflated 43%) 2024-08-20T23:29:08.5544226Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a7ae9a1f0fe20405.xml (deflated 35%) 2024-08-20T23:29:08.5545895Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d5544982ca00acf0.xml (deflated 35%) 2024-08-20T23:29:08.5547546Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8239438aba352e7f.xml (deflated 42%) 2024-08-20T23:29:08.5549210Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-27fafd0358af9caf.xml (deflated 35%) 2024-08-20T23:29:08.5550870Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-70eb3f2a48187319.xml (deflated 36%) 2024-08-20T23:29:08.5552664Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7cbb4c49eac73051.xml (deflated 42%) 2024-08-20T23:29:08.5554312Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7823ac9225efca10.xml (deflated 36%) 2024-08-20T23:29:08.5555971Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4c1beb9c737701e5.xml (deflated 35%) 2024-08-20T23:29:08.5557722Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4623ac9c0afdcd12.xml (deflated 36%) 2024-08-20T23:29:08.5559880Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ec611427a8808c83.xml (deflated 35%) 2024-08-20T23:29:08.5561546Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-45faec233108d99c.xml (deflated 35%) 2024-08-20T23:29:08.5563227Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5d51cf3d2e4d90a9.xml (deflated 44%) 2024-08-20T23:29:08.5564899Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-109aa571aaa2e120.xml (deflated 42%) 2024-08-20T23:29:08.5566566Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-651e43d1ecff6e24.xml (deflated 43%) 2024-08-20T23:29:08.5568335Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2a17e1916ece414c.xml (deflated 43%) 2024-08-20T23:29:08.5570008Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ebe8ff72d814f23a.xml (deflated 42%) 2024-08-20T23:29:08.5571681Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7aab05244c0bd7f3.xml (deflated 35%) 2024-08-20T23:29:08.5573443Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5d962fb66096806e.xml (deflated 35%) 2024-08-20T23:29:08.5575107Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b28e754f8cac79d7.xml (deflated 36%) 2024-08-20T23:29:08.5576766Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b54d2970585b8f74.xml (deflated 35%) 2024-08-20T23:29:08.5578435Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1a3948f2d053253f.xml (deflated 43%) 2024-08-20T23:29:08.5580087Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-554ce8f85e705eb6.xml (deflated 43%) 2024-08-20T23:29:08.5581732Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-55864f32a8e093eb.xml (deflated 41%) 2024-08-20T23:29:08.5583388Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-99ddf034ff71649d.xml (deflated 41%) 2024-08-20T23:29:08.5585055Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-aee90850ff0b3901.xml (deflated 41%) 2024-08-20T23:29:08.5586722Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9b2efd8a353f4ea0.xml (deflated 35%) 2024-08-20T23:29:08.5588384Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ea3f0ce8ded15985.xml (deflated 35%) 2024-08-20T23:29:08.5590055Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-def54182124f8d06.xml (deflated 35%) 2024-08-20T23:29:08.5591720Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a3f5a2f6cf53c5c6.xml (deflated 43%) 2024-08-20T23:29:08.5593476Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f371477e773c1ffe.xml (deflated 42%) 2024-08-20T23:29:08.5595126Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bede690b6ebc6106.xml (deflated 43%) 2024-08-20T23:29:08.5596790Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0ae552776f839a00.xml (deflated 37%) 2024-08-20T23:29:08.5598449Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-697ee71dec07f6a6.xml (deflated 37%) 2024-08-20T23:29:08.5600105Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-05be601c62c14ca4.xml (deflated 41%) 2024-08-20T23:29:08.5601752Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ef59aa0466402d09.xml (deflated 41%) 2024-08-20T23:29:08.5603413Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-306b20796cdc9767.xml (deflated 40%) 2024-08-20T23:29:08.5605065Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-200c9e4d954f677b.xml (deflated 41%) 2024-08-20T23:29:08.5606726Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6f4e8a2d61d3d61c.xml (deflated 36%) 2024-08-20T23:29:08.5608428Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-815b993887a4dfb2.xml (deflated 35%) 2024-08-20T23:29:08.5610082Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8f456f5361405f71.xml (deflated 35%) 2024-08-20T23:29:08.5611805Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1fb582c7b5986f0f.xml (deflated 35%) 2024-08-20T23:29:08.5613478Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8274c2ca9cc17571.xml (deflated 43%) 2024-08-20T23:29:08.5615122Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d77288fc856fed68.xml (deflated 45%) 2024-08-20T23:29:08.5616784Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a936cb2709ad6e9c.xml (deflated 35%) 2024-08-20T23:29:08.5618458Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6df8b6a2d673322d.xml (deflated 35%) 2024-08-20T23:29:08.5620120Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-850e22a08e8c3179.xml (deflated 35%) 2024-08-20T23:29:08.5621758Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-397fa8887871db26.xml (deflated 35%) 2024-08-20T23:29:08.5623425Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2fbfe0a0b3abd4e7.xml (deflated 35%) 2024-08-20T23:29:08.5625086Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1834cfc7e03c26f6.xml (deflated 35%) 2024-08-20T23:29:08.5626748Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7dd8f16b77c72074.xml (deflated 35%) 2024-08-20T23:29:08.5628397Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8fca69cc87b05371.xml (deflated 35%) 2024-08-20T23:29:08.5630056Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a485dfb04ffe0526.xml (deflated 35%) 2024-08-20T23:29:08.5631782Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6c493b1ac49442bb.xml (deflated 35%) 2024-08-20T23:29:08.5633432Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-975ec85f33761859.xml (deflated 35%) 2024-08-20T23:29:08.5635074Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-93ed0c97ce462b98.xml (deflated 35%) 2024-08-20T23:29:08.5636737Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bf4b5d0a5efadcc4.xml (deflated 35%) 2024-08-20T23:29:08.5638407Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fe0470258a68a75e.xml (deflated 35%) 2024-08-20T23:29:08.5640072Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-de1bb63b9f812b98.xml (deflated 35%) 2024-08-20T23:29:08.5641722Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4c5ab4719a974b0e.xml (deflated 35%) 2024-08-20T23:29:08.5643379Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-42e6eb8b74a6a839.xml (deflated 35%) 2024-08-20T23:29:08.5645036Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-59d60b4dea9fc937.xml (deflated 35%) 2024-08-20T23:29:08.5646701Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-29c7c23137e0dfd3.xml (deflated 41%) 2024-08-20T23:29:08.5648399Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c08f3ba6405b7c5a.xml (deflated 43%) 2024-08-20T23:29:08.5650053Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-13fff91d23b45718.xml (deflated 43%) 2024-08-20T23:29:08.5651790Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0f8ef577ed074691.xml (deflated 43%) 2024-08-20T23:29:08.5653455Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0e0b696ff2a83eac.xml (deflated 44%) 2024-08-20T23:29:08.5655105Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e08bb04e7b5e26ce.xml (deflated 44%) 2024-08-20T23:29:08.5656813Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ce2142fca24b74d6.xml (deflated 44%) 2024-08-20T23:29:08.5658844Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7537d1a307f87ed4.xml (deflated 44%) 2024-08-20T23:29:08.5660515Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-201b89b585802bae.xml (deflated 44%) 2024-08-20T23:29:08.5662239Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-08d98b7b648ff74a.xml (deflated 45%) 2024-08-20T23:29:08.5663961Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-07c832582da0d545.xml (deflated 43%) 2024-08-20T23:29:08.5665623Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0d862ef90b117193.xml (deflated 36%) 2024-08-20T23:29:08.5667291Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d1d41e512a4fada4.xml (deflated 42%) 2024-08-20T23:29:08.5668959Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ffd6bce618080a2a.xml (deflated 42%) 2024-08-20T23:29:08.5670613Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c6432420b29a8d58.xml (deflated 35%) 2024-08-20T23:29:08.5672411Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e451b1b505ac4e8d.xml (deflated 35%) 2024-08-20T23:29:08.5674081Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-64e462d6fd57e8c5.xml (deflated 42%) 2024-08-20T23:29:08.5675747Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1b7e7bf42f62f742.xml (deflated 42%) 2024-08-20T23:29:08.5677409Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d15e73edf56c286e.xml (deflated 35%) 2024-08-20T23:29:08.5679070Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-81763f52bb26158b.xml (deflated 47%) 2024-08-20T23:29:08.5680723Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-457fdf0493d6d4d4.xml (deflated 47%) 2024-08-20T23:29:08.5682391Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-92f2ef4a96fceab1.xml (deflated 47%) 2024-08-20T23:29:08.5684038Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5114e491aa98f375.xml (deflated 42%) 2024-08-20T23:29:08.5685691Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-42635bc3325c0b5e.xml (deflated 42%) 2024-08-20T23:29:08.5687434Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4dafef6cedd99c8f.xml (deflated 42%) 2024-08-20T23:29:08.5689093Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-95a61e3f0458d84e.xml (deflated 42%) 2024-08-20T23:29:08.5690737Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-857d873e7dde12e2.xml (deflated 36%) 2024-08-20T23:29:08.5692461Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a1e446d54bc3c000.xml (deflated 42%) 2024-08-20T23:29:08.5694130Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-67278944e2d4cf04.xml (deflated 42%) 2024-08-20T23:29:08.5695781Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0da3fb4cf9ef05f0.xml (deflated 42%) 2024-08-20T23:29:08.5697435Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-69e07bd855aa5a77.xml (deflated 35%) 2024-08-20T23:29:08.5699100Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2ea66a84855e211f.xml (deflated 42%) 2024-08-20T23:29:08.5700763Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9e4d62b3156edb90.xml (deflated 43%) 2024-08-20T23:29:08.5702428Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-55ec7eb020f4a991.xml (deflated 43%) 2024-08-20T23:29:08.5704080Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e44720bcaa16f7f5.xml (deflated 42%) 2024-08-20T23:29:08.5705740Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c0b2e355bc7a6895.xml (deflated 42%) 2024-08-20T23:29:08.5707387Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bc83512030864001.xml (deflated 42%) 2024-08-20T23:29:08.5709045Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8dbbf56340557107.xml (deflated 44%) 2024-08-20T23:29:08.5710684Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d11cdecc585db278.xml (deflated 42%) 2024-08-20T23:29:08.5712403Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-50b37938fc56a91d.xml (deflated 43%) 2024-08-20T23:29:08.5714055Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5ab75ff4190400be.xml (deflated 42%) 2024-08-20T23:29:08.5715718Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3affdbeb45c30edf.xml (deflated 35%) 2024-08-20T23:29:08.5717386Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a9ba42b46f8aa935.xml (deflated 35%) 2024-08-20T23:29:08.5719038Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1656fafe13d76573.xml (deflated 42%) 2024-08-20T23:29:08.5720687Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0651ecf8e7c73f7c.xml (deflated 35%) 2024-08-20T23:29:08.5722348Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7878eb9b7e2eef9a.xml (deflated 36%) 2024-08-20T23:29:08.5723997Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e068e64c3d13e372.xml (deflated 42%) 2024-08-20T23:29:08.5725651Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-10b2bfc2f4723c54.xml (deflated 36%) 2024-08-20T23:29:08.5727359Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4576ce8153f8696e.xml (deflated 35%) 2024-08-20T23:29:08.5729022Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4bbd8ccaf49fdec5.xml (deflated 36%) 2024-08-20T23:29:08.5730674Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b388805d0481c0be.xml (deflated 35%) 2024-08-20T23:29:08.5732401Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5a97720b4ae537cf.xml (deflated 35%) 2024-08-20T23:29:08.5734056Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5a6932ca542be3c6.xml (deflated 44%) 2024-08-20T23:29:08.5735711Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9b6edb7d46332011.xml (deflated 42%) 2024-08-20T23:29:08.5737367Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-391994b589f1ccde.xml (deflated 43%) 2024-08-20T23:29:08.5739028Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a67fba96ba0a7c57.xml (deflated 43%) 2024-08-20T23:29:08.5740692Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e9c5c9f11279eb1b.xml (deflated 42%) 2024-08-20T23:29:08.5742356Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f596397e3bdc6fe2.xml (deflated 35%) 2024-08-20T23:29:08.5743998Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e7dca8751f0d614c.xml (deflated 35%) 2024-08-20T23:29:08.5745655Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0cd3a36535422c69.xml (deflated 36%) 2024-08-20T23:29:08.5747301Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-04133290e86c7045.xml (deflated 35%) 2024-08-20T23:29:08.5748951Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-67565b2620c835c3.xml (deflated 43%) 2024-08-20T23:29:08.5750587Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9b632cf59ea0644a.xml (deflated 42%) 2024-08-20T23:29:08.5752303Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ad6c4372d0f33e43.xml (deflated 41%) 2024-08-20T23:29:08.5753962Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5c745664bc817a73.xml (deflated 41%) 2024-08-20T23:29:08.5755614Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c03223feff1f7e4e.xml (deflated 42%) 2024-08-20T23:29:08.5757251Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4911822997b800d1.xml (deflated 35%) 2024-08-20T23:29:08.5759248Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fe7cff1a0226468b.xml (deflated 35%) 2024-08-20T23:29:08.5760909Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5c3d8808849a0bde.xml (deflated 35%) 2024-08-20T23:29:08.5762569Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-845b256f53664cf1.xml (deflated 43%) 2024-08-20T23:29:08.5764222Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-06bb2b1b292b462c.xml (deflated 36%) 2024-08-20T23:29:08.5765882Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-dbdf1b2a7cd8648f.xml (deflated 40%) 2024-08-20T23:29:08.5767653Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f469b81d43d73289.xml (deflated 42%) 2024-08-20T23:29:08.5769323Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bef7f89f8533fcf8.xml (deflated 42%) 2024-08-20T23:29:08.5770972Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d77c1104685ca6bb.xml (deflated 35%) 2024-08-20T23:29:08.5772702Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-98a6a8f3d7bbfb6a.xml (deflated 35%) 2024-08-20T23:29:08.5774366Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e7238c281f8c5a8a.xml (deflated 35%) 2024-08-20T23:29:08.5776017Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-828a17ac2b91b073.xml (deflated 35%) 2024-08-20T23:29:08.5777670Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-273a84f40524b730.xml (deflated 35%) 2024-08-20T23:29:08.5779322Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1a7ade6214a64b57.xml (deflated 45%) 2024-08-20T23:29:08.5780994Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ff9fc4ecf2266eed.xml (deflated 36%) 2024-08-20T23:29:08.5782653Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-674f0274936f0a57.xml (deflated 35%) 2024-08-20T23:29:08.5784294Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-623aa5a1bc8486d3.xml (deflated 35%) 2024-08-20T23:29:08.5785951Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-15552a28018a5777.xml (deflated 35%) 2024-08-20T23:29:08.5787597Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f63c81ab2752f772.xml (deflated 35%) 2024-08-20T23:29:08.5789254Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d39d7b2852f25651.xml (deflated 36%) 2024-08-20T23:29:08.5790976Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-541bf2295a3bb81c.xml (deflated 35%) 2024-08-20T23:29:08.5792638Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-53ab9153e028025b.xml (deflated 42%) 2024-08-20T23:29:08.5794293Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6f585e3de96d553c.xml (deflated 43%) 2024-08-20T23:29:08.5795952Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-15780eddc3fb0718.xml (deflated 44%) 2024-08-20T23:29:08.5797592Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b075d228ea727041.xml (deflated 44%) 2024-08-20T23:29:08.5799245Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6bd4a758101a76d0.xml (deflated 44%) 2024-08-20T23:29:08.5800896Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f777662508bcd182.xml (deflated 45%) 2024-08-20T23:29:08.5802551Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fcfced50338fbb79.xml (deflated 42%) 2024-08-20T23:29:08.5804258Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-176f24c71e2d34e5.xml (deflated 35%) 2024-08-20T23:29:08.5805970Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e8a1c0b71b676e48.xml (deflated 42%) 2024-08-20T23:29:08.5807628Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ea74d83d8b1a4cd8.xml (deflated 35%) 2024-08-20T23:29:08.5809281Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b7d5ff741963dadc.xml (deflated 35%) 2024-08-20T23:29:08.5810930Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-329247504db59415.xml (deflated 42%) 2024-08-20T23:29:08.5812622Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-827bc42bfb31983a.xml (deflated 42%) 2024-08-20T23:29:08.5814286Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d677d2e9f48ab4a1.xml (deflated 42%) 2024-08-20T23:29:08.5815949Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cdd018d15745268e.xml (deflated 35%) 2024-08-20T23:29:08.5817601Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d35488080ed63d46.xml (deflated 42%) 2024-08-20T23:29:08.5819244Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9e3cd971baf04634.xml (deflated 42%) 2024-08-20T23:29:08.5820893Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-06538288fc739908.xml (deflated 41%) 2024-08-20T23:29:08.5822539Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ec9124f89359b303.xml (deflated 42%) 2024-08-20T23:29:08.5824188Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a3180ba2d174b7b1.xml (deflated 42%) 2024-08-20T23:29:08.5825821Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-63497506f77986f0.xml (deflated 42%) 2024-08-20T23:29:08.5827472Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4c6e8833cc0bea69.xml (deflated 43%) 2024-08-20T23:29:08.5829129Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b3b455141d2c0f4f.xml (deflated 42%) 2024-08-20T23:29:08.5830830Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-73973c8a22451345.xml (deflated 42%) 2024-08-20T23:29:08.5832466Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-deb0d8a41522126f.xml (deflated 42%) 2024-08-20T23:29:08.5834123Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a22cc3faf531ec39.xml (deflated 42%) 2024-08-20T23:29:08.5835783Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3ac25a753c0efc64.xml (deflated 43%) 2024-08-20T23:29:08.5837438Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bc40c49122966496.xml (deflated 42%) 2024-08-20T23:29:08.5839071Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0c87fb8fcd8326d8.xml (deflated 43%) 2024-08-20T23:29:08.5840735Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ff9ea60502b0804e.xml (deflated 42%) 2024-08-20T23:29:08.5842396Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-40c1c5289a7e1dc6.xml (deflated 42%) 2024-08-20T23:29:08.5844049Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c7dc76cac477e769.xml (deflated 42%) 2024-08-20T23:29:08.5845749Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e0c23388c60ab541.xml (deflated 42%) 2024-08-20T23:29:08.5847406Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a8dd9e3fc4acf9d2.xml (deflated 42%) 2024-08-20T23:29:08.5849070Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-47c743f9a6de1124.xml (deflated 42%) 2024-08-20T23:29:08.5850725Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cf36257b7e79d1a8.xml (deflated 42%) 2024-08-20T23:29:08.5852437Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-96166f0ae773d758.xml (deflated 43%) 2024-08-20T23:29:08.5854090Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3a80fb6bb31983d1.xml (deflated 43%) 2024-08-20T23:29:08.5855755Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ddc88b71c7bb807c.xml (deflated 42%) 2024-08-20T23:29:08.5857413Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2e5c8b4492218624.xml (deflated 35%) 2024-08-20T23:29:08.5859408Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d3c5a15df6ea8de2.xml (deflated 35%) 2024-08-20T23:29:08.5861082Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fe96510e6eb6b3be.xml (deflated 35%) 2024-08-20T23:29:08.5862739Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8574f431841d0080.xml (deflated 35%) 2024-08-20T23:29:08.5864383Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-99572b38fd1ca706.xml (deflated 42%) 2024-08-20T23:29:08.5866026Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-daad3b645c3f6c27.xml (deflated 35%) 2024-08-20T23:29:08.5867698Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1972d7af41c9dca9.xml (deflated 45%) 2024-08-20T23:29:08.5869361Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ae77c1025c4268e3.xml (deflated 45%) 2024-08-20T23:29:08.5871119Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b24c8a51acb9990c.xml (deflated 45%) 2024-08-20T23:29:08.5872767Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-66c5acf44fe06158.xml (deflated 41%) 2024-08-20T23:29:08.5874423Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0c9ad84d21ba6f42.xml (deflated 41%) 2024-08-20T23:29:08.5876079Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-49b7811bae252f07.xml (deflated 42%) 2024-08-20T23:29:08.5877737Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0f6da0fb37a14613.xml (deflated 41%) 2024-08-20T23:29:08.5879384Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4eabed8721deb0f7.xml (deflated 41%) 2024-08-20T23:29:08.5881040Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-edd33f5528746c81.xml (deflated 41%) 2024-08-20T23:29:08.5882687Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8610c5119f477a8e.xml (deflated 35%) 2024-08-20T23:29:08.5884333Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c907cf8d73ebc7e0.xml (deflated 35%) 2024-08-20T23:29:08.5886056Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-786f5af7826757ef.xml (deflated 35%) 2024-08-20T23:29:08.5887718Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f521af7be6c56506.xml (deflated 35%) 2024-08-20T23:29:08.5889371Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f1d6314164563e36.xml (deflated 45%) 2024-08-20T23:29:08.5891002Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1041f757479d0c4e.xml (deflated 42%) 2024-08-20T23:29:08.5892747Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-898b8cea4209f8e5.xml (deflated 36%) 2024-08-20T23:29:08.5894401Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3ff866de191e4dd4.xml (deflated 36%) 2024-08-20T23:29:08.5896063Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3799b30cd4b09df7.xml (deflated 35%) 2024-08-20T23:29:08.5897712Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-599ecdce056d7828.xml (deflated 42%) 2024-08-20T23:29:08.5899370Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3e0e229925ac41fc.xml (deflated 35%) 2024-08-20T23:29:08.5901017Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-667880797cafe506.xml (deflated 35%) 2024-08-20T23:29:08.5902670Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-380e90fd2e0dd06b.xml (deflated 36%) 2024-08-20T23:29:08.5904310Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-862b4b3dbdbab2be.xml (deflated 35%) 2024-08-20T23:29:08.5905969Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ec97b9cc9164642d.xml (deflated 35%) 2024-08-20T23:29:08.5907623Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-388a8cff92ffbb0d.xml (deflated 35%) 2024-08-20T23:29:08.5909338Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-969b4f18b56ab020.xml (deflated 41%) 2024-08-20T23:29:08.5910983Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-771d9b602e976a11.xml (deflated 42%) 2024-08-20T23:29:08.5912638Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f894300c78a2f4bd.xml (deflated 42%) 2024-08-20T23:29:08.5914297Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2577d3b6133aff26.xml (deflated 44%) 2024-08-20T23:29:08.5915959Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b41fbf7e770f51a6.xml (deflated 36%) 2024-08-20T23:29:08.5917608Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a244b081ce951a1c.xml (deflated 40%) 2024-08-20T23:29:08.5919267Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b4ef46373908e199.xml (deflated 42%) 2024-08-20T23:29:08.5920921Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9dcbad8a4fc5e571.xml (deflated 42%) 2024-08-20T23:29:08.5922591Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e6ae1ef32adc2adb.xml (deflated 35%) 2024-08-20T23:29:08.5924291Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-84f851534b056926.xml (deflated 35%) 2024-08-20T23:29:08.5925941Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2ae1df097ee639be.xml (deflated 35%) 2024-08-20T23:29:08.5927602Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2b5d414ced079a5a.xml (deflated 35%) 2024-08-20T23:29:08.5929264Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-98f99e30559b10c4.xml (deflated 35%) 2024-08-20T23:29:08.5930909Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2a79f84063df3fac.xml (deflated 45%) 2024-08-20T23:29:08.5932632Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-606011e8066e6565.xml (deflated 35%) 2024-08-20T23:29:08.5934290Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ad5381b815c7c48a.xml (deflated 35%) 2024-08-20T23:29:08.5935961Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-26851fb5d2d2445f.xml (deflated 35%) 2024-08-20T23:29:08.5937612Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3382c5abbbb0f716.xml (deflated 35%) 2024-08-20T23:29:08.5939271Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-66400f16387dd566.xml (deflated 35%) 2024-08-20T23:29:08.5940926Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6bd31a15c1da5fe9.xml (deflated 35%) 2024-08-20T23:29:08.5942587Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ed982f1d10ab81c4.xml (deflated 35%) 2024-08-20T23:29:08.5944237Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-171e07e4d1aa30fc.xml (deflated 41%) 2024-08-20T23:29:08.5945890Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7d60703cd307549a.xml (deflated 43%) 2024-08-20T23:29:08.5947548Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ab089d9d07142d84.xml (deflated 44%) 2024-08-20T23:29:08.5949278Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d1d27beefc36fd70.xml (deflated 44%) 2024-08-20T23:29:08.5950929Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-01b5dfda9fe23561.xml (deflated 44%) 2024-08-20T23:29:08.5952569Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c902337bf1b92472.xml (deflated 45%) 2024-08-20T23:29:08.5954231Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-55b825df35cf4bb5.xml (deflated 42%) 2024-08-20T23:29:08.5955892Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6a81239ca1c41603.xml (deflated 35%) 2024-08-20T23:29:08.5957688Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e2a7096f05207a47.xml (deflated 42%) 2024-08-20T23:29:08.5959527Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fa6fda401ae00718.xml (deflated 35%) 2024-08-20T23:29:08.5961200Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-20f9d8ddd0a03c81.xml (deflated 35%) 2024-08-20T23:29:08.5962849Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-169de8ca1a01d116.xml (deflated 41%) 2024-08-20T23:29:08.5964611Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-509dc48801367f78.xml (deflated 42%) 2024-08-20T23:29:08.5966270Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ccbb96c43929e727.xml (deflated 42%) 2024-08-20T23:29:08.5967923Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a4930b0c23658f0c.xml (deflated 35%) 2024-08-20T23:29:08.5969595Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2edf9b52cdda2ba2.xml (deflated 41%) 2024-08-20T23:29:08.5971246Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7bde39447cb5a7bb.xml (deflated 42%) 2024-08-20T23:29:08.5972961Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1172d091f50cc799.xml (deflated 42%) 2024-08-20T23:29:08.5974616Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9ea6413c70718ffe.xml (deflated 42%) 2024-08-20T23:29:08.5976275Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ed7e102df04cc012.xml (deflated 42%) 2024-08-20T23:29:08.5977929Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-beaf4f30a814f366.xml (deflated 42%) 2024-08-20T23:29:08.5979589Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1b6d001fce9a4d5a.xml (deflated 43%) 2024-08-20T23:29:08.5981244Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c5c5869a57cc3369.xml (deflated 42%) 2024-08-20T23:29:08.5982889Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a29e147288d5ec1d.xml (deflated 41%) 2024-08-20T23:29:08.5984528Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d1948582783132b9.xml (deflated 42%) 2024-08-20T23:29:08.5986172Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d2794707c1f218f0.xml (deflated 42%) 2024-08-20T23:29:08.5987821Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-49e4b141179d8a7b.xml (deflated 43%) 2024-08-20T23:29:08.5989553Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1ca179f07acf398f.xml (deflated 42%) 2024-08-20T23:29:08.5991207Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-056cf8a05a2034d7.xml (deflated 43%) 2024-08-20T23:29:08.5992854Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9c676068ec696b60.xml (deflated 42%) 2024-08-20T23:29:08.5994519Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ea00fe64596c9072.xml (deflated 42%) 2024-08-20T23:29:08.5996176Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-08477da80eff049c.xml (deflated 42%) 2024-08-20T23:29:08.5997826Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7dfa8218d1eaf295.xml (deflated 42%) 2024-08-20T23:29:08.5999482Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d3a906b6ee2079e2.xml (deflated 42%) 2024-08-20T23:29:08.6001143Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-82fc2bba40abf03c.xml (deflated 42%) 2024-08-20T23:29:08.6002810Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d054eef173d2d8cd.xml (deflated 42%) 2024-08-20T23:29:08.6004506Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-29ed6136b6d2984a.xml (deflated 42%) 2024-08-20T23:29:08.6006161Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0c6ec9e68ea9cd15.xml (deflated 43%) 2024-08-20T23:29:08.6007830Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-41d4f3ef2cea753b.xml (deflated 42%) 2024-08-20T23:29:08.6009489Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-19c83db6b6540bdf.xml (deflated 36%) 2024-08-20T23:29:08.6011134Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c95cd3ad979787b4.xml (deflated 35%) 2024-08-20T23:29:08.6012873Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6fa79988320cf3d2.xml (deflated 35%) 2024-08-20T23:29:08.6014537Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-dfa4cde43f6e2a38.xml (deflated 35%) 2024-08-20T23:29:08.6016200Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-32bc723303815a6c.xml (deflated 42%) 2024-08-20T23:29:08.6017848Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5a1fad1a983593ac.xml (deflated 35%) 2024-08-20T23:29:08.6019507Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-934acd95a3a468ef.xml (deflated 44%) 2024-08-20T23:29:08.6021163Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c61bf9400b7a9b98.xml (deflated 45%) 2024-08-20T23:29:08.6022818Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-639449fa69a36da1.xml (deflated 45%) 2024-08-20T23:29:08.6024460Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2d69c8f8bafc80c8.xml (deflated 41%) 2024-08-20T23:29:08.6026126Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9d2de8df7d38ed66.xml (deflated 41%) 2024-08-20T23:29:08.6027848Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-73d82d5d81247af8.xml (deflated 42%) 2024-08-20T23:29:08.6029491Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-85c6d1098071c5da.xml (deflated 41%) 2024-08-20T23:29:08.6031141Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f14b4bc5fa6617d6.xml (deflated 41%) 2024-08-20T23:29:08.6032805Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a1771b273b7aa14a.xml (deflated 41%) 2024-08-20T23:29:08.6034457Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5e8ba9588eb16572.xml (deflated 35%) 2024-08-20T23:29:08.6036097Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2a71b017f6eed629.xml (deflated 35%) 2024-08-20T23:29:08.6037747Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5b66f60e30d0a483.xml (deflated 35%) 2024-08-20T23:29:08.6039403Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-71d20e151b50bb14.xml (deflated 35%) 2024-08-20T23:29:08.6041058Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-46f9beed697588af.xml (deflated 45%) 2024-08-20T23:29:08.6042751Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f5bcc38a30710bf0.xml (deflated 42%) 2024-08-20T23:29:08.6044420Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b475f85bea0897b0.xml (deflated 36%) 2024-08-20T23:29:08.6046075Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f2742bc77a46bb99.xml (deflated 36%) 2024-08-20T23:29:08.6047737Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-821b8ea9c30cacdd.xml (deflated 35%) 2024-08-20T23:29:08.6049385Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b9a5336abd406b37.xml (deflated 42%) 2024-08-20T23:29:08.6051043Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bd86a75e269b47f1.xml (deflated 35%) 2024-08-20T23:29:08.6052766Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-09bcc23ee78a49cd.xml (deflated 35%) 2024-08-20T23:29:08.6054434Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6eaf6fba3169ca32.xml (deflated 35%) 2024-08-20T23:29:08.6056078Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-36903c10edd40586.xml (deflated 35%) 2024-08-20T23:29:08.6058044Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-445b73b6100ee3c4.xml (deflated 35%) 2024-08-20T23:29:08.6059717Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d769466d3ca213d0.xml (deflated 35%) 2024-08-20T23:29:08.6061369Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3d5d9c285a07daca.xml (deflated 41%) 2024-08-20T23:29:08.6063017Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-42e99f641cf1f031.xml (deflated 42%) 2024-08-20T23:29:08.6064669Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-82d36acbc193a363.xml (deflated 42%) 2024-08-20T23:29:08.6066321Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-517932a975450d80.xml (deflated 44%) 2024-08-20T23:29:08.6092183Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-20T23:29:08.6092798Z # Remove any previous usage logs if they exist 2024-08-20T23:29:08.6093265Z rm -f logs-*.zip 2024-08-20T23:29:08.6093917Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-20T23:29:08.6094694Z # so check to see if the file exists first 2024-08-20T23:29:08.6095159Z if [ -f 'usage_log.txt' ]; then 2024-08-20T23:29:08.6095663Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-20T23:29:08.6096134Z fi 2024-08-20T23:29:08.6096458Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-20T23:29:08.6097011Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-20T23:29:08.6097480Z fi 2024-08-20T23:29:08.6103022Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:08.6103503Z env: 2024-08-20T23:29:08.6103756Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:08.6104326Z DOCKER_CONTAINER_ID: 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:08.6105145Z FILE_SUFFIX: test-distributed-2-2-amz2023.linux.2xlarge_29025447188 2024-08-20T23:29:08.6105697Z ##[endgroup] 2024-08-20T23:29:08.6203890Z adding: usage_log.txt (deflated 92%) 2024-08-20T23:29:08.6478517Z adding: test/test-reports/distributed.test_dynamo_distributed_1.1_1c913bb5733b9809_.log (deflated 88%) 2024-08-20T23:29:08.6480797Z adding: test/test-reports/distributed._tools.test_mod_tracker_1.1_3511612dabb84d9d_.log (deflated 64%) 2024-08-20T23:29:08.6482805Z adding: test/test-reports/distributed.checkpoint.test_nested_dict_1.1_30b816a15fad5443_.log (deflated 59%) 2024-08-20T23:29:08.6484346Z adding: test/test-reports/distributed.test_fake_pg_1.1_370d085631d7e091_.log (deflated 74%) 2024-08-20T23:29:08.6485520Z adding: test/test-reports/distributed.pipelining.test_backward_1.1_f8e00bff990f5854_.log (deflated 54%) 2024-08-20T23:29:08.6486783Z adding: test/test-reports/distributed.fsdp.test_checkpoint_wrapper_1.1_c2236b677f52b086_.log (deflated 74%) 2024-08-20T23:29:08.6487975Z adding: test/test-reports/distributed.test_symmetric_memory_1.1_762ac28a61c3a406_.log (deflated 86%) 2024-08-20T23:29:08.6489208Z adding: test/test-reports/distributed._shard.sharded_tensor.test_logger_1.1_13579639616a79e5_.log (deflated 55%) 2024-08-20T23:29:08.6490419Z adding: test/test-reports/distributed._tensor.test_dtensor_1.1_8b0facc6c29dfcf3_.log (deflated 91%) 2024-08-20T23:29:08.6494912Z adding: test/test-reports/distributed.test_store_1.1_c6dd3553b454b67b_.log (deflated 95%) 2024-08-20T23:29:08.6496196Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_compile_1.1_88fc85a7e71fcc31_.log (deflated 82%) 2024-08-20T23:29:08.6497569Z adding: test/test-reports/distributed.pipelining.test_schedule_multiproc_1.1_0f76094012963a21_.log (deflated 2%) 2024-08-20T23:29:08.6498867Z adding: test/test-reports/distributed._tensor.test_xla_integration_1.1_27a9f1646aba27c7_.log (deflated 66%) 2024-08-20T23:29:08.6500101Z adding: test/test-reports/distributed.pipelining.test_unflatten_1.1_c20acd1eae87c9a9_.log (deflated 54%) 2024-08-20T23:29:08.6501336Z adding: test/test-reports/distributed._tensor.test_convolution_ops_1.1_6718a71142674185_.log (deflated 61%) 2024-08-20T23:29:08.6512508Z adding: test/test-reports/distributed.test_c10d_gloo_1.1_0011413f6c856585_.log (deflated 96%) 2024-08-20T23:29:08.6513680Z adding: test/test-reports/distributed.test_compute_comm_reordering_1.1_d3cec4fd2073a9b2_.log (deflated 76%) 2024-08-20T23:29:08.6514944Z adding: test/test-reports/distributed.checkpoint.e2e.test_fsdp_ep_1.1_67a86ad84644441e_.log (deflated 54%) 2024-08-20T23:29:08.6516164Z adding: test/test-reports/distributed._tools.test_mem_tracker_1.1_766901752cf3310b_.log (deflated 61%) 2024-08-20T23:29:08.6517319Z adding: test/test-reports/distributed.test_c10d_spawn_ucc_1.1_f971886e41b45078_.log (deflated 92%) 2024-08-20T23:29:08.6518746Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_training_1.1_3857beddeae803a3_.log (deflated 84%) 2024-08-20T23:29:08.6520082Z adding: test/test-reports/distributed.checkpoint.test_save_load_api_1.1_e08e5517d706b1f6_.log (deflated 53%) 2024-08-20T23:29:08.6521358Z adding: test/test-reports/distributed.checkpoint.test_dedup_tensors_1.1_f9d5b48325f4ba43_.log (deflated 52%) 2024-08-20T23:29:08.6522647Z adding: test/test-reports/distributed.checkpoint.fsdp.test_fsdp_dsd_1.1_6979e6a99699986f_.log (deflated 77%) 2024-08-20T23:29:08.6523866Z adding: test/test-reports/distributed.test_c10d_spawn_gloo_1.1_78e083d5415cbf62_.log (deflated 94%) 2024-08-20T23:29:08.6525008Z adding: test/test-reports/distributed._tensor.test_attention_1.1_563e8366a665e2bc_.log (deflated 64%) 2024-08-20T23:29:08.6526185Z adding: test/test-reports/distributed.pipelining.test_stage_1.1_048ca6fd9459d43b_.log (deflated 2%) 2024-08-20T23:29:08.6527460Z adding: test/test-reports/distributed._tensor.debug.test_comm_mode_features_1.1_606f20b69e3c2576_.log (deflated 82%) 2024-08-20T23:29:08.6528801Z adding: test/test-reports/distributed._tensor.experimental.test_local_map_1.1_fce5a9a22ddaf375_.log (deflated 78%) 2024-08-20T23:29:08.6530076Z adding: test/test-reports/distributed._tensor.test_pointwise_ops_1.1_900217578222578d_.log (deflated 69%) 2024-08-20T23:29:08.6531369Z adding: test/test-reports/distributed.tensor.parallel.test_tp_random_state_1.1_a3bf98274bccb9ca_.log (deflated 55%) 2024-08-20T23:29:08.6532923Z adding: test/test-reports/distributed.checkpoint.e2e.test_e2e_save_and_load_1.1_810c1dacda091ded_.log (deflated 94%) 2024-08-20T23:29:08.6534228Z adding: test/test-reports/distributed._composable.test_checkpoint_1.1_f83d4ad25e1483fe_.log (deflated 69%) 2024-08-20T23:29:08.6535408Z adding: test/test-reports/distributed._tensor.test_init_1.1_f1e2a20f4c842c87_.log (deflated 70%) 2024-08-20T23:29:08.6536593Z adding: test/test-reports/distributed._tools.test_fsdp2_mem_tracker_1.1_4289babf1c28b533_.log (deflated 67%) 2024-08-20T23:29:08.6537882Z adding: test/test-reports/distributed.checkpoint.test_fsdp_optim_state_1.1_ae0c401806e653d4_.log (deflated 62%) 2024-08-20T23:29:08.6539200Z adding: test/test-reports/distributed.checkpoint.test_format_utils_1.1_1e1ddb2654d0f2f2_.log (deflated 86%) 2024-08-20T23:29:08.6540517Z adding: test/test-reports/distributed.checkpoint.test_fsspec_1.1_1372df8212139c74_.log (deflated 57%) 2024-08-20T23:29:08.6541733Z adding: test/test-reports/distributed.checkpoint.test_traverse_1.1_d00ea6594a81aaca_.log (deflated 71%) 2024-08-20T23:29:08.6543020Z adding: test/test-reports/distributed.checkpoint.test_dtensor_resharding_1.1_970a6e28b62722bb_.log (deflated 71%) 2024-08-20T23:29:08.6544319Z adding: test/test-reports/distributed.checkpoint.test_tp_checkpoint_1.1_3ee693d34b619e83_.log (deflated 60%) 2024-08-20T23:29:08.6545617Z adding: test/test-reports/distributed.checkpoint.e2e.test_fine_tuning_1.1_efdcc10651d36f7d_.log (deflated 53%) 2024-08-20T23:29:08.6546894Z adding: test/test-reports/distributed._tools.test_memory_tracker_1.1_234cf670a759ac1e_.log (deflated 54%) 2024-08-20T23:29:08.6548174Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_init_1.1_7bbb58b5cda03f1d_.log (deflated 87%) 2024-08-20T23:29:08.6549467Z adding: test/test-reports/distributed.checkpoint.test_planner_1.1_9bc2c8e7ab770c5b_.log (deflated 67%) 2024-08-20T23:29:08.6550697Z adding: test/test-reports/distributed.elastic.test_control_plane_1.1_e1ab0c9d2390bcdf_.log (deflated 73%) 2024-08-20T23:29:08.6551989Z adding: test/test-reports/distributed.checkpoint.test_dtensor_checkpoint_1.1_1e6121d676fd93bd_.log (deflated 55%) 2024-08-20T23:29:08.6553210Z adding: test/test-reports/distributed.test_c10d_spawn_nccl_1.1_f3a06736f2be1654_.log (deflated 93%) 2024-08-20T23:29:08.6554489Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_logging_1.1_61ab6d52f4c9ff90_.log (deflated 53%) 2024-08-20T23:29:08.6555792Z adding: test/test-reports/distributed.test_c10d_ops_nccl_1.1_294fca59a0ee3dd0_.log (stored 0%) 2024-08-20T23:29:08.6557045Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_memory_1.1_e4a465003e90d39f_.log (deflated 55%) 2024-08-20T23:29:08.6558660Z adding: test/test-reports/distributed._tensor.test_optimizers_1.1_b2d413c168584d7a_.log (deflated 95%) 2024-08-20T23:29:08.6559825Z adding: test/test-reports/distributed.test_functional_api_1.1_eaaecdf42ac6748f_.log (deflated 86%) 2024-08-20T23:29:08.6561001Z adding: test/test-reports/distributed.test_control_collectives_1.1_174bd9b769312047_.log (deflated 78%) 2024-08-20T23:29:08.6562205Z adding: test/test-reports/distributed.elastic.utils.util_test_1.1_0ca2828cdcff6a93_.log (deflated 76%) 2024-08-20T23:29:08.6563565Z adding: test/test-reports/distributed.checkpoint.test_fsdp_tp_checkpoint_conversion_1.1_ac8f3a48f5de8eed_.log (deflated 57%) 2024-08-20T23:29:08.6564884Z adding: test/test-reports/distributed._tensor.test_op_strategy_1.1_66a1fecae10dddae_.log (deflated 79%) 2024-08-20T23:29:08.6566144Z adding: test/test-reports/distributed.checkpoint.test_hsdp_checkpoint_1.1_4c16db1a61d762cb_.log (deflated 70%) 2024-08-20T23:29:08.6567444Z adding: test/test-reports/distributed.elastic.timer.local_timer_test_1.1_4d36d84ec13daf48_.log (deflated 78%) 2024-08-20T23:29:08.6568857Z adding: test/test-reports/distributed._composable.test_composability.test_pp_composability_1.1_0fed095348962263_.log (deflated 90%) 2024-08-20T23:29:08.6570253Z adding: test/test-reports/distributed.checkpoint.test_compatibility_1.1_6c024e15e643a0b4_.log (deflated 63%) 2024-08-20T23:29:08.6571577Z adding: test/test-reports/distributed._tensor.test_view_ops_1.1_4f74b70b1db9d1c9_.log (deflated 81%) 2024-08-20T23:29:08.6572942Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_bag_1.1_f2f742f565044c2b_.log (deflated 63%) 2024-08-20T23:29:08.6574406Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_clip_grad_norm__1.1_ee60f21786f6f73a_.log (deflated 63%) 2024-08-20T23:29:08.6575841Z adding: test/test-reports/distributed.tensor.parallel.test_parallelize_api_1.1_8239082ff858120e_.log (deflated 91%) 2024-08-20T23:29:08.6577150Z adding: test/test-reports/distributed.fsdp.test_fsdp_optim_state_1.1_5544bd2ef680ace0_.log (deflated 97%) 2024-08-20T23:29:08.6578349Z adding: test/test-reports/distributed._composable.test_compose_1.1_7441fd4413763dad_.log (deflated 78%) 2024-08-20T23:29:08.6579545Z adding: test/test-reports/distributed.fsdp.test_fsdp_pure_fp16_1.1_492d72df1bc70dd4_.log (deflated 60%) 2024-08-20T23:29:08.6580757Z adding: test/test-reports/distributed.nn.jit.test_instantiator_1.1_a3e4ecd82b14865d_.log (deflated 64%) 2024-08-20T23:29:08.6582037Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_comm_1.1_efd3a7871b3a0081_.log (deflated 82%) 2024-08-20T23:29:08.6583343Z adding: test/test-reports/distributed.elastic.utils.logging_test_1.1_d143787a7c59c076_.log (deflated 58%) 2024-08-20T23:29:08.6584702Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_state_dict_1.1_7519eac36e32f3b5_.log (deflated 76%) 2024-08-20T23:29:08.6586039Z adding: test/test-reports/distributed.fsdp.test_fsdp_ignored_modules_1.1_81483b6d89ed4ee5_.log (deflated 85%) 2024-08-20T23:29:08.6587257Z adding: test/test-reports/distributed.fsdp.test_fsdp_overlap_1.1_1b30ecdde56327a9_.log (deflated 59%) 2024-08-20T23:29:08.6588508Z adding: test/test-reports/distributed.fsdp.test_fsdp_backward_prefetch_1.1_6e443f4c7d677724_.log (deflated 55%) 2024-08-20T23:29:08.6589713Z adding: test/test-reports/distributed._shard.test_sharder_1.1_10347209540712c4_.log (deflated 59%) 2024-08-20T23:29:08.6590861Z adding: test/test-reports/distributed.test_distributed_spawn_1.3_2c9725a09a4175c6_.log (deflated 37%) 2024-08-20T23:29:08.6592019Z adding: test/test-reports/distributed.test_distributed_spawn_1.3_cf5dac0306bd68f9_.log (deflated 37%) 2024-08-20T23:29:08.6597588Z adding: test/test-reports/distributed.test_distributed_spawn_1.3_d6080a3f1b6e59a9_.log (deflated 97%) 2024-08-20T23:29:08.6616290Z adding: test/test-reports/distributed.test_distributed_spawn_1.3_88af2a339da3dc18_.log (deflated 97%) 2024-08-20T23:29:08.6617475Z adding: test/test-reports/distributed.test_distributed_spawn_3.3_c78e95d266aa742d_.log (deflated 37%) 2024-08-20T23:29:08.6618628Z adding: test/test-reports/distributed.test_distributed_spawn_3.3_57ec608eb9eabaed_.log (deflated 37%) 2024-08-20T23:29:08.6633059Z adding: test/test-reports/distributed.test_distributed_spawn_3.3_3e53fe4f4e3527d0_.log (deflated 97%) 2024-08-20T23:29:08.6649202Z adding: test/test-reports/distributed.test_distributed_spawn_3.3_c27d028217ec6b28_.log (deflated 97%) 2024-08-20T23:29:08.6651139Z adding: test/test-reports/distributed.fsdp.test_wrap_1.1_3a92b25a0649d83c_.log (deflated 90%) 2024-08-20T23:29:08.6653127Z adding: test/test-reports/distributed.fsdp.test_fsdp_misc_1.1_64f8a1c3ef646ecc_.log (deflated 90%) 2024-08-20T23:29:08.6654386Z adding: test/test-reports/distributed.fsdp.test_fsdp_comm_hooks_1.1_3846ef2b6a2460a8_.log (deflated 89%) 2024-08-20T23:29:08.6655634Z adding: test/test-reports/distributed.fsdp.test_fsdp_unshard_params_1.1_a7a0f888097365bb_.log (deflated 83%) 2024-08-20T23:29:08.6658912Z adding: test/test-reports/distributed.test_c10d_common_1.1_01e9c19590ff535c_.log (deflated 94%) 2024-08-20T23:29:08.6660096Z adding: test/test-reports/distributed.fsdp.test_fsdp_clip_grad_norm_1.1_673cf73577639c4d_.log (deflated 69%) 2024-08-20T23:29:08.6661425Z adding: test/test-reports/distributed.fsdp.test_fsdp_meta_1.1_dc6040bc516a9bb0_.log (deflated 84%) 2024-08-20T23:29:08.6662602Z adding: test/test-reports/distributed.fsdp.test_fsdp_uneven_1.1_ff5fc4f52868b006_.log (deflated 54%) 2024-08-20T23:29:08.6680812Z adding: test/test-reports/distributed.rpc.cuda.test_tensorpipe_agent_1.1_df0cfd35ad1f7f8b_.log (deflated 98%) 2024-08-20T23:29:08.6681969Z adding: test/test-reports/distributed.test_c10d_nccl_1.1_3a89adabdc6b204b_.log (stored 0%) 2024-08-20T23:29:08.6716770Z adding: test/test-reports/distributed.rpc.test_tensorpipe_agent_2.3_5eb4324e92140afd_.log (deflated 97%) 2024-08-20T23:29:08.6718093Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_binary_cmp_1.1_457ffa507753c035_.log (deflated 70%) 2024-08-20T23:29:08.6724793Z adding: test/test-reports/distributed.rpc.test_faulty_agent_1.1_b657ad53c939206c_.log (deflated 96%) 2024-08-20T23:29:08.6726022Z adding: test/test-reports/distributed.fsdp.test_fsdp_multiple_forward_1.1_76ca3b62fbc0d3d0_.log (deflated 55%) 2024-08-20T23:29:08.6727245Z adding: test/test-reports/distributed.fsdp.test_fsdp_comm_1.1_30efbf57c84d82cb_.log (deflated 80%) 2024-08-20T23:29:08.6728493Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_init_1.1_8a0fa0414e86ef88_.log (deflated 67%) 2024-08-20T23:29:08.6729856Z adding: test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_1.1_0cba918e2107b5ad_.log (deflated 91%) 2024-08-20T23:29:08.6731074Z adding: test/test-reports/distributed.test_launcher_1.1_9e8fdd432cba8d9b_.log (deflated 58%) 2024-08-20T23:29:08.6732485Z adding: test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_reshard_1.1_f5ae028b0c328e1b_.log (deflated 64%) 2024-08-20T23:29:08.6733912Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_1.1_dd0c2dc1002782d0_.log (deflated 62%) 2024-08-20T23:29:08.6735237Z adding: test/test-reports/distributed.elastic.metrics.api_test_1.1_2531c2e4b172f20e_.log (deflated 62%) 2024-08-20T23:29:08.6736526Z adding: test/test-reports/distributed.elastic.utils.distributed_test_1.1_ed1b8966ec795815_.log (deflated 93%) 2024-08-20T23:29:08.6737802Z adding: test/test-reports/distributed.elastic.events.lib_test_1.1_4135dfad42307c52_.log (deflated 72%) 2024-08-20T23:29:08.6739109Z adding: test/test-reports/distributed.elastic.multiprocessing.api_test_1.1_8474f9cabcb93aac_.log (deflated 88%) 2024-08-20T23:29:08.6740568Z adding: test/test-reports/distributed.elastic.timer.local_timer_example_1.1_eb0dcf631ba2479b_.log (deflated 67%) 2024-08-20T23:29:08.7160105Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-20T23:29:08.7160813Z # Remove any previous debugging artifacts if they exist 2024-08-20T23:29:08.7161345Z rm -f debug-*.zip 2024-08-20T23:29:08.7161694Z if [ -d 'test/debug' ]; then 2024-08-20T23:29:08.7162160Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-20T23:29:08.7162612Z fi 2024-08-20T23:29:08.7168096Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:08.7168575Z env: 2024-08-20T23:29:08.7168828Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:08.7181053Z DOCKER_CONTAINER_ID: 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:08.7181905Z FILE_SUFFIX: test-distributed-2-2-amz2023.linux.2xlarge_29025447188 2024-08-20T23:29:08.7182485Z ##[endgroup] 2024-08-20T23:29:08.7267419Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:29:08.7267844Z with: 2024-08-20T23:29:08.7268084Z s3-bucket: gha-artifacts 2024-08-20T23:29:08.7268499Z s3-prefix: pytorch/pytorch/10479309237/1/artifact 2024-08-20T23:29:08.7268953Z retention-days: 14 2024-08-20T23:29:08.7269246Z if-no-files-found: warn 2024-08-20T23:29:08.7269581Z path: test-jsons-*.zip 2024-08-20T23:29:08.7269894Z name: artifact 2024-08-20T23:29:08.7270163Z region: us-east-1 2024-08-20T23:29:08.7270439Z env: 2024-08-20T23:29:08.7270687Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:08.7271371Z DOCKER_CONTAINER_ID: 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:08.7272013Z ##[endgroup] 2024-08-20T23:29:09.1194380Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:29:09.1194969Z With the provided path, there will be 1 file uploaded 2024-08-20T23:29:09.1195575Z Uploading to s3 prefix: pytorch/pytorch/10479309237/1/artifact 2024-08-20T23:29:09.2446142Z Starting upload of test-jsons-test-distributed-2-2-amz2023.linux.2xlarge_29025447188.zip 2024-08-20T23:29:09.4079943Z Finished upload of test-jsons-test-distributed-2-2-amz2023.linux.2xlarge_29025447188.zip 2024-08-20T23:29:09.4268189Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:29:09.4268610Z with: 2024-08-20T23:29:09.4268868Z s3-bucket: gha-artifacts 2024-08-20T23:29:09.4269280Z s3-prefix: pytorch/pytorch/10479309237/1/artifact 2024-08-20T23:29:09.4269724Z retention-days: 14 2024-08-20T23:29:09.4270040Z if-no-files-found: error 2024-08-20T23:29:09.4270381Z path: test-reports-*.zip 2024-08-20T23:29:09.4270694Z name: artifact 2024-08-20T23:29:09.4270983Z region: us-east-1 2024-08-20T23:29:09.4271257Z env: 2024-08-20T23:29:09.4271499Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:09.4272078Z DOCKER_CONTAINER_ID: 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:09.4272711Z ##[endgroup] 2024-08-20T23:29:09.7648054Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:29:09.7648848Z With the provided path, there will be 1 file uploaded 2024-08-20T23:29:09.7649454Z Uploading to s3 prefix: pytorch/pytorch/10479309237/1/artifact 2024-08-20T23:29:09.7689127Z Starting upload of test-reports-test-distributed-2-2-amz2023.linux.2xlarge_29025447188.zip 2024-08-20T23:29:09.9225889Z Finished upload of test-reports-test-distributed-2-2-amz2023.linux.2xlarge_29025447188.zip 2024-08-20T23:29:09.9412163Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:29:09.9412581Z with: 2024-08-20T23:29:09.9412840Z s3-bucket: gha-artifacts 2024-08-20T23:29:09.9413270Z s3-prefix: pytorch/pytorch/10479309237/1/artifact 2024-08-20T23:29:09.9413714Z retention-days: 14 2024-08-20T23:29:09.9414026Z if-no-files-found: ignore 2024-08-20T23:29:09.9414360Z path: logs-*.zip 2024-08-20T23:29:09.9414630Z name: artifact 2024-08-20T23:29:09.9414908Z region: us-east-1 2024-08-20T23:29:09.9415179Z env: 2024-08-20T23:29:09.9415421Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:09.9416194Z DOCKER_CONTAINER_ID: 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:09.9416830Z ##[endgroup] 2024-08-20T23:29:10.2801094Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-20T23:29:10.2802098Z With the provided path, there will be 1 file uploaded 2024-08-20T23:29:10.2803168Z Uploading to s3 prefix: pytorch/pytorch/10479309237/1/artifact 2024-08-20T23:29:10.2847057Z Starting upload of logs-test-distributed-2-2-amz2023.linux.2xlarge_29025447188.zip 2024-08-20T23:29:10.4837128Z Finished upload of logs-test-distributed-2-2-amz2023.linux.2xlarge_29025447188.zip 2024-08-20T23:29:10.5024387Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-20T23:29:10.5024808Z with: 2024-08-20T23:29:10.5025069Z s3-bucket: gha-artifacts 2024-08-20T23:29:10.5025481Z s3-prefix: pytorch/pytorch/10479309237/1/artifact 2024-08-20T23:29:10.5025922Z retention-days: 14 2024-08-20T23:29:10.5026232Z if-no-files-found: ignore 2024-08-20T23:29:10.5026583Z path: debug-*.zip 2024-08-20T23:29:10.5026855Z name: artifact 2024-08-20T23:29:10.5027133Z region: us-east-1 2024-08-20T23:29:10.5027404Z env: 2024-08-20T23:29:10.5027639Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:10.5028221Z DOCKER_CONTAINER_ID: 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:10.5028848Z ##[endgroup] 2024-08-20T23:29:10.8366415Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-20T23:29:10.8549693Z ##[group]Run # shellcheck disable=SC2156 2024-08-20T23:29:10.8550153Z # shellcheck disable=SC2156 2024-08-20T23:29:10.8551072Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-20T23:29:10.8557027Z shell: /usr/bin/bash -e {0} 2024-08-20T23:29:10.8557355Z env: 2024-08-20T23:29:10.8557613Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:10.8558519Z DOCKER_CONTAINER_ID: 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:10.8559170Z ##[endgroup] 2024-08-20T23:29:11.0754506Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-20T23:29:11.0755042Z with: 2024-08-20T23:29:11.0755272Z env: 2024-08-20T23:29:11.0755519Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:11.0756087Z DOCKER_CONTAINER_ID: 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:11.0756718Z ##[endgroup] 2024-08-20T23:29:11.0779292Z ##[group]Run set -eou pipefail 2024-08-20T23:29:11.0779678Z set -eou pipefail 2024-08-20T23:29:11.0779991Z  2024-08-20T23:29:11.0780483Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-20T23:29:11.0781119Z for _ in $(seq 1440); do 2024-08-20T23:29:11.0781555Z  # Break if no ssh session exists anymore 2024-08-20T23:29:11.0782033Z  if [ "$(who)" = "" ]; then 2024-08-20T23:29:11.0782421Z  break 2024-08-20T23:29:11.0782707Z  fi 2024-08-20T23:29:11.0783023Z  echo "." 2024-08-20T23:29:11.0783312Z  sleep 5 2024-08-20T23:29:11.0783601Z done 2024-08-20T23:29:11.0789128Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:11.0789633Z env: 2024-08-20T23:29:11.0789902Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:11.0790535Z DOCKER_CONTAINER_ID: 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:11.0791236Z ##[endgroup] 2024-08-20T23:29:11.0814258Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-20T23:29:11.0896105Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:29:11.0896902Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-20T23:29:11.0897681Z # shellcheck disable=SC2046 2024-08-20T23:29:11.0898198Z docker stop $(docker ps -q) || true 2024-08-20T23:29:11.0898687Z # Prune all of the docker images 2024-08-20T23:29:11.0915324Z docker system prune -af 2024-08-20T23:29:11.0921006Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:11.0921504Z env: 2024-08-20T23:29:11.0921772Z GIT_DEFAULT_BRANCH: main 2024-08-20T23:29:11.0922361Z DOCKER_CONTAINER_ID: 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:11.0923004Z ##[endgroup] 2024-08-20T23:29:21.6633837Z 9645b5fa39b5 2024-08-20T23:29:22.2372768Z Deleted Containers: 2024-08-20T23:29:22.2373369Z 9645b5fa39b5608dd4325658552b9210b6a80a879835f7fff982df0418b6a4a7 2024-08-20T23:29:22.2373898Z 2024-08-20T23:29:30.7946273Z Deleted Images: 2024-08-20T23:29:30.7948023Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:f6d216893d65c7b8ae43df4daaf247db808378e9 2024-08-20T23:29:30.7950242Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11@sha256:9c16298ecf1ba0ec71e25268bd4244acb2b2c8c60846e6c8a5e8eb35b9eff836 2024-08-20T23:29:30.7952267Z deleted: sha256:0d70ec8288d42bb7b6bd827783d06ac33bce3462f66d9a810a9a9ddd77c887e9 2024-08-20T23:29:30.7953157Z deleted: sha256:cd85f247f1db7df3b86b5d221f7780d127d898da58566662175cbceef1eca74e 2024-08-20T23:29:30.7954029Z deleted: sha256:9b5af2ba4c6fed48eb126f4e4d26fe2f6f5bb23e7960f4cd20d0e3b8e36abb36 2024-08-20T23:29:30.7954902Z deleted: sha256:dfcabb79b518bf75986db8730806b2d6e58f847262420385a83685a50b6cc03e 2024-08-20T23:29:30.7955768Z deleted: sha256:f0dc0fc952b0d537320b816a022f8835ffdd8c001dbeebc3bfe2ae19ea67cffe 2024-08-20T23:29:30.7956619Z deleted: sha256:333fc2efa12fa987d4d921d2a57ecf4c45660a519656c1ea4603452af5bca473 2024-08-20T23:29:30.7957469Z deleted: sha256:cc1311368e6820f3097a29c3d2224b26e219cfe670357339b6bebac80e1b7791 2024-08-20T23:29:30.7958515Z deleted: sha256:73fbd71f6e938b5b8a5520c5e3b05a60bc5fe200afc590be7a63bfda9dbab816 2024-08-20T23:29:30.7959393Z deleted: sha256:86ef3feab980e3e593efc8567bb6b0c5b87ecc3d56d23f3d0c1b6a558e2498f9 2024-08-20T23:29:30.7960473Z deleted: sha256:786a1cf18988e3ca8c155174a2e39790d0895667c87826244946d9d8d6a7c7e1 2024-08-20T23:29:30.7961331Z deleted: sha256:16ab50ca7d231916c2ecf9cf4e10ebb7161b1fa9d8a9069c5d9f3a67e56e569c 2024-08-20T23:29:30.7962184Z deleted: sha256:d74e37a5061fb48a66f621cb7351e79c28a563a8d579ce1c1488870e3b4551fe 2024-08-20T23:29:30.7963027Z deleted: sha256:886fd5dabaddbb1999f54b910991db2f02021b0dcfd9645ba290e51b6bcffbd3 2024-08-20T23:29:30.7963955Z deleted: sha256:1899784bdae733da95c66a616b5a7596f5e9209c64d060321e786a2a511bbf9e 2024-08-20T23:29:30.7964797Z deleted: sha256:3603bcd51acfe96da1a1e7e20000459fa9bd6839e882d942f031f7933d2cbcf4 2024-08-20T23:29:30.7965635Z deleted: sha256:c37c9591a1ebbfb3d382ea183d82239d7c10d2912da578f8dd94f88a75ec7c7c 2024-08-20T23:29:30.7966471Z deleted: sha256:42f8742744b3b0501274d2617a4faee84f98b565158f09e44f4d0241e24fb4a5 2024-08-20T23:29:30.7967318Z deleted: sha256:e4ff0c7bcc2fdb99e17c776477d5c18e9361ac0a2736df3dfb7dfd9cb1d24cb1 2024-08-20T23:29:30.7968390Z deleted: sha256:6c0717036d579c73658382280877c62371ee7fc9611c68dc252a79a79095e90f 2024-08-20T23:29:30.7969222Z deleted: sha256:5ba6e38220f1d4cae51bc9d06235689498debea840afb3e45a8d94dbe3ed4bd6 2024-08-20T23:29:30.7970078Z deleted: sha256:52e88c711fec52ba5484db46c10757a8f67fb2faa2859cb550c2504f7ea0215c 2024-08-20T23:29:30.7970934Z deleted: sha256:ee1ee62bb43ad224da336be09c15d5152e8f893e5316196fd9255f9dfa051e6c 2024-08-20T23:29:30.7971869Z deleted: sha256:db1d9c406a3c525dfa03fc55b7533d3ad07331ce244781f0685311dbf1c99b12 2024-08-20T23:29:30.7972725Z deleted: sha256:f76f7051ca457377b0f3b2af093d25534737c5a257e1fdf7ce18d0bd9f15120d 2024-08-20T23:29:30.7973571Z deleted: sha256:9fa921f68d6cd562a37209b977aa4519c4d188384865d3d8e17cabb85804a4d9 2024-08-20T23:29:30.7974392Z deleted: sha256:4245506f8114d5ad4c2e4e8e0d48207ea965e3321354a28ffd44f5e49eb02e0c 2024-08-20T23:29:30.7975208Z deleted: sha256:82526cd40c448218884561989680eba89858c126c18f041a4493345b7ff0fcf8 2024-08-20T23:29:30.7976041Z deleted: sha256:fe5ab7211ef36083ff39d37ea5310cad2fdeeaacb86b909ca510b8b69cc8b153 2024-08-20T23:29:30.7977075Z deleted: sha256:586e23ffbd3c14fd707eac41612ac1c0e9d61106de1d7665e417a51e28dfdbbe 2024-08-20T23:29:30.7978098Z deleted: sha256:bbdbcc9407efaafdaa6a1e4c8f51956a09f0083bc1e2548770b04142919d41fb 2024-08-20T23:29:30.7978956Z deleted: sha256:f8946873b0972358e88c1a71bbc540c0d7e98014ea8554485f8a085fd8ff6144 2024-08-20T23:29:30.7979792Z deleted: sha256:a6bbaaf810f89a9617f6c49b53c95731591890d326ff813a204036013fde1202 2024-08-20T23:29:30.7980626Z deleted: sha256:6ac45f1332600d2e42bc8ba9f7c4a22a23c87ec82a9f352bb7d70db70ff63604 2024-08-20T23:29:30.7981467Z deleted: sha256:97c28e8951f9ca49081cef7723673f019132b6684834ab5cd3cc3fdb850d3376 2024-08-20T23:29:30.7982299Z deleted: sha256:18093a570d0586f6bb9ef746c707bb3743a1e1d25e27a425e1ee56cabfefeb8d 2024-08-20T23:29:30.7983117Z deleted: sha256:d374585d459bd4a169573a2495ae419b4daf62d88513d87079ab7e04f6bd9ad5 2024-08-20T23:29:30.7983951Z deleted: sha256:31766aa82d623394ca9b63b1165907ce18cd4b9a0bc5608ec8e4d52acf98bf70 2024-08-20T23:29:30.7984797Z deleted: sha256:fa07904ebfa944942135ee06844871b4f28be2904d3e8a9705e0f2f68b02bf58 2024-08-20T23:29:30.7985638Z deleted: sha256:0f3b379bdaa956509f5104a95a0a237b7d8097aae3dd00963a605d12dacaa149 2024-08-20T23:29:30.7986484Z deleted: sha256:df3610f6c5e7351a273e0da0ebf0f169c0e52fb167da3fbbb4e4ca2cb5c34b6e 2024-08-20T23:29:30.7987344Z deleted: sha256:2b7ab0dc4db180ae6346f514863502dc7d5f322a228660d40c5625487a95a2da 2024-08-20T23:29:30.7988374Z deleted: sha256:cfbe220d95c10f637822c13fc6e6d08317940d78bd9e5aab395ebc3c8458151a 2024-08-20T23:29:30.7989226Z deleted: sha256:451f06689da0d7f6b2ded1da341a74f5dac58f1f9f7d2621be96dcd31c9ea581 2024-08-20T23:29:30.7990080Z deleted: sha256:829271df625d8997753cd47c019ecc424eebbc1da2a77251ee32c91f0b9640e2 2024-08-20T23:29:30.7990923Z deleted: sha256:c1c5c1249e1640be364b0d02f8912fa9edf3409852c5b002322e0efbe3631d51 2024-08-20T23:29:30.7991760Z deleted: sha256:424e5d38f16a88c5fc5e796a17f2c492bc462b16b7c7b7ce8e5f43e455daa51a 2024-08-20T23:29:30.7992623Z deleted: sha256:5350eaa4ea8f7e6443d9faf9163978c953dbb6d88f0a5c86f2fd3fcba68f69f2 2024-08-20T23:29:30.7993609Z deleted: sha256:fc5982aded69a44ef8731310809cbe67f55cbfca5b725d9aa9dd33322c7e4f71 2024-08-20T23:29:30.7994482Z deleted: sha256:894e7fd587d0d0d4607c0def34e735d47eeef86a050ced39d36e68e83333ed09 2024-08-20T23:29:30.7995322Z deleted: sha256:05c019f928fada63a413ef98cf2950d64f8117af6a4646b8fd15c602c56d5696 2024-08-20T23:29:30.7996177Z deleted: sha256:f1cd2eb056337dedc6952c668e89891bd494b7b1adc14d1cbaf6985418401e5a 2024-08-20T23:29:30.7997028Z deleted: sha256:ed97a0021fd34f50f544d15bf384159fde43539b4b71cc6128283f0c2b80e9cf 2024-08-20T23:29:30.7997869Z deleted: sha256:20d26c62d8be66b0ceae7e28bd8733d2f08b65ac595e88f9b531eba1970b26fe 2024-08-20T23:29:30.7998726Z deleted: sha256:10f41e5e3c42cf9e9b34cd50a011b7ee51004c749762737233bba0da479b388f 2024-08-20T23:29:30.7999571Z deleted: sha256:afbb9c9059b1c72194c682c5d265fe8b17011a7b6723645656fb9f1e9811528c 2024-08-20T23:29:30.8000395Z deleted: sha256:4362c875f11aa5ca8575e2f98651b30e0da7a6229b2d2c53cdf5fdc16fba2418 2024-08-20T23:29:30.8001254Z deleted: sha256:fa5d6b0e7b97ab4610f65c718ad4d75c27e62c29c61dedd196124bf3306330af 2024-08-20T23:29:30.8002117Z deleted: sha256:9739620f8ffc3c88bc57fd988134283f8f98f5ccfde916c6dd2fbb0cd6544dff 2024-08-20T23:29:30.8002967Z deleted: sha256:4f3e8b161de7f337b674841e642253443a68ebb56a2aade58fc7b1a5abb8ca67 2024-08-20T23:29:30.8003814Z deleted: sha256:c0d7a5ae04f2c46a907186321e1ada88c2d0daddf79a151fdd3e4c4055d25be0 2024-08-20T23:29:30.8004681Z deleted: sha256:bcb1a6e4b5da8e24c2539adf1943466267389d3aa76bcb63d955acc7ead340f6 2024-08-20T23:29:30.8005542Z deleted: sha256:e42d57df38426fdd6a7c8ffab2da9367f6560411c9978d11616a9cea58dfa587 2024-08-20T23:29:30.8006385Z deleted: sha256:8afca9ff63cb45e044380ee0d46025f38b3f484c74e0f42004e5671b417fb9be 2024-08-20T23:29:30.8007227Z deleted: sha256:28d3faa820053555b9081cda360ae1216e42689428f85f59f46144def51e79be 2024-08-20T23:29:30.8008069Z deleted: sha256:23be7c82be5c08957ec124bb6cfd898227e69b98f45c044d724180cc53942549 2024-08-20T23:29:30.8009011Z deleted: sha256:57adbda996a88cbbb0e18ef3b3af2f35697929595a343828d5d2892f5a36970a 2024-08-20T23:29:30.8010004Z deleted: sha256:8d149e9e773521fc0edc53831c87226a992066d0f57b637f932e7cb444592aea 2024-08-20T23:29:30.8010857Z deleted: sha256:eeb3bdd5b20188cad891eecc42f5568d5e1b119f0b86f4f597f599c523355737 2024-08-20T23:29:30.8011811Z deleted: sha256:c79b01ec230cbd7423a4cd411f275845a35764d355516c4030314ea9e09e7a3a 2024-08-20T23:29:30.8012654Z deleted: sha256:b04f4978eb8b562edf865ef1deedc442617b9c00f41f6e81f6a01949c5890087 2024-08-20T23:29:30.8013641Z deleted: sha256:be8e63aa3a7676161561b2b2e120ffd06d8f8b689588bdc09960570ac79c6fbe 2024-08-20T23:29:30.8014501Z deleted: sha256:c175ca5d09b7e7998a791dfd390deb23a97fd38dcd8c8392efe109d2376f1821 2024-08-20T23:29:30.8015348Z deleted: sha256:9e8bb347b233c66e9a8efb2783dba814fa979846a45e190f0821ddf5208c2955 2024-08-20T23:29:30.8016200Z deleted: sha256:6ff53beb2fc2f2bd235811b56056062cf4835f20790285ee922135d343836d91 2024-08-20T23:29:30.8017060Z deleted: sha256:361a9ca88dbc01c5345f0306e00d5acfbac9bd6b4de57a2ddb01cb799618b161 2024-08-20T23:29:30.8017925Z deleted: sha256:09b4536d5fb2828d961bfc5d5fdaf07980a4882f216c14419ce9d00424e63559 2024-08-20T23:29:30.8018767Z deleted: sha256:2a3665b48a131b5839ecaf5ec1ade50edaf2562b143283ba3beea46b0b1371d0 2024-08-20T23:29:30.8019615Z deleted: sha256:5bb95609ea3347bc0627bb89f8206a2a106d9569e942e4a6791d2cd3b211eb03 2024-08-20T23:29:30.8020469Z deleted: sha256:c78da1df7a4fdef88e067a6d69d2f51d4392e535ad973726f87521057cac4360 2024-08-20T23:29:30.8021328Z deleted: sha256:8e19a4ea3c026fbed7913cedded42d6e1f27b56e8534f213c48c0eec4fb3668b 2024-08-20T23:29:30.8022202Z deleted: sha256:ba8fb2cc055ddc87770d387ef61bd973c4066f0b412d4a025ccdd02795989499 2024-08-20T23:29:30.8023060Z deleted: sha256:75ac3a791e55cc40b22fde810086fb99ac453ab12546eddfecbb9158dec2786b 2024-08-20T23:29:30.8023965Z deleted: sha256:875c1779a1b3dd96a25d75839e3c65e71ed48cfb55f28b6530ce995e46f58fb0 2024-08-20T23:29:30.8024825Z deleted: sha256:5d9456aa782f8d5721bf5da9b96fa1e03ff6d30cb4f1c95c4aec4274c8843084 2024-08-20T23:29:30.8025779Z deleted: sha256:069c7e56627070abc1eef78c626d39542b3a9dbd7db3c00ba1d2bc5d07b10943 2024-08-20T23:29:30.8026637Z deleted: sha256:a445abe17034277479cd88b87a98f1a2a1fe6e85c8b631de04e81e2606d8eb43 2024-08-20T23:29:30.8027490Z deleted: sha256:3dff28f3d7b2ceed6cb9b567c8afe27ba1382cc613051130586a2c7a332de3cc 2024-08-20T23:29:30.8028359Z deleted: sha256:19aab9b70ad33c45d9fde55fdff5b52586844ee383317c0765d12b8e193fc269 2024-08-20T23:29:30.8029420Z deleted: sha256:c274afef2a17c385253c6d33c97b39cd4d03c5b5fdd9896cd36983756de313a3 2024-08-20T23:29:30.8030275Z deleted: sha256:a8aec88c47143bef5edc07e98182ecfa5ff9e7650c3bc7162ea99d61f3ed559b 2024-08-20T23:29:30.8031149Z deleted: sha256:8ad1c79513b6cbc5e44d911b4dea5840c2701a65dd2c7f92a2baa03a05426c06 2024-08-20T23:29:30.8031996Z deleted: sha256:a2d2902da87619b52c7294787417d3238f5049bdce806f88fbf2ae0fd54b9e08 2024-08-20T23:29:30.8032837Z deleted: sha256:1b9b7346fee7abbc7f5538eaa23548bd05a45abe8daf6794024be0c8ad7d60bb 2024-08-20T23:29:30.8033374Z 2024-08-20T23:29:30.8069271Z Total reclaimed space: 21.04GB 2024-08-20T23:29:30.8168719Z Post job cleanup. 2024-08-20T23:29:30.8238628Z Post job cleanup. 2024-08-20T23:29:30.9103899Z [command]/usr/bin/git version 2024-08-20T23:29:30.9158838Z git version 2.40.1 2024-08-20T23:29:30.9196421Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/28990dc0-8cef-439f-b259-263dba785f8e' before making global git config changes 2024-08-20T23:29:30.9197774Z Adding repository directory to the temporary git global config as a safe directory 2024-08-20T23:29:30.9201522Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-20T23:29:30.9249983Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-20T23:29:30.9283201Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-20T23:29:30.9675637Z Entering 'android/libs/fbjni' 2024-08-20T23:29:30.9726121Z Entering 'third_party/FP16' 2024-08-20T23:29:30.9776084Z Entering 'third_party/FXdiv' 2024-08-20T23:29:30.9825103Z Entering 'third_party/NNPACK' 2024-08-20T23:29:30.9873709Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:29:30.9922461Z Entering 'third_party/XNNPACK' 2024-08-20T23:29:30.9987562Z Entering 'third_party/benchmark' 2024-08-20T23:29:31.0036645Z Entering 'third_party/cpp-httplib' 2024-08-20T23:29:31.0084795Z Entering 'third_party/cpuinfo' 2024-08-20T23:29:31.0133567Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:29:31.0182297Z Entering 'third_party/cutlass' 2024-08-20T23:29:31.0237755Z Entering 'third_party/eigen' 2024-08-20T23:29:31.0288961Z Entering 'third_party/fbgemm' 2024-08-20T23:29:31.0341668Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:29:31.0388521Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:29:31.0436191Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:29:31.0489234Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:29:31.0537473Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:29:31.0586845Z Entering 'third_party/flatbuffers' 2024-08-20T23:29:31.0637201Z Entering 'third_party/fmt' 2024-08-20T23:29:31.0686572Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:29:31.0735713Z Entering 'third_party/gloo' 2024-08-20T23:29:31.0785421Z Entering 'third_party/googletest' 2024-08-20T23:29:31.0833552Z Entering 'third_party/ideep' 2024-08-20T23:29:31.0881212Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:29:31.0935836Z Entering 'third_party/ittapi' 2024-08-20T23:29:31.0985312Z Entering 'third_party/kineto' 2024-08-20T23:29:31.1034373Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:29:31.1081906Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:29:31.1131203Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:29:31.1180107Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:29:31.1228037Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:29:31.1275737Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:29:31.1325481Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:29:31.1374897Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:29:31.1423725Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:29:31.1473513Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:29:31.1522668Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:29:31.1570221Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:29:31.1619129Z Entering 'third_party/mimalloc' 2024-08-20T23:29:31.1668650Z Entering 'third_party/nccl/nccl' 2024-08-20T23:29:31.1717848Z Entering 'third_party/nlohmann' 2024-08-20T23:29:31.1767251Z Entering 'third_party/onnx' 2024-08-20T23:29:31.1830663Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:29:31.1879752Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:29:31.1930315Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:29:31.1981035Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:29:31.2028057Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:29:31.2075387Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:29:31.2122222Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:29:31.2172002Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:29:31.2219099Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:29:31.2266655Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:29:31.2313381Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:29:31.2362876Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:29:31.2412357Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:29:31.2480467Z Entering 'third_party/pocketfft' 2024-08-20T23:29:31.2528445Z Entering 'third_party/protobuf' 2024-08-20T23:29:31.2580830Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:29:31.2628428Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:29:31.2677975Z Entering 'third_party/psimd' 2024-08-20T23:29:31.2726798Z Entering 'third_party/pthreadpool' 2024-08-20T23:29:31.2776047Z Entering 'third_party/pybind11' 2024-08-20T23:29:31.2824654Z Entering 'third_party/python-peachpy' 2024-08-20T23:29:31.2872524Z Entering 'third_party/sleef' 2024-08-20T23:29:31.2921374Z Entering 'third_party/tensorpipe' 2024-08-20T23:29:31.2970519Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:29:31.3018229Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:29:31.3065108Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:29:31.3113216Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:29:31.3159321Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:29:31.3225413Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-20T23:29:31.3251613Z http.https://github.com/.extraheader 2024-08-20T23:29:31.3261837Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-20T23:29:31.3294718Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-20T23:29:31.3562650Z Entering 'android/libs/fbjni' 2024-08-20T23:29:31.3595608Z http.https://github.com/.extraheader 2024-08-20T23:29:31.3626941Z Entering 'third_party/FP16' 2024-08-20T23:29:31.3661082Z http.https://github.com/.extraheader 2024-08-20T23:29:31.3690234Z Entering 'third_party/FXdiv' 2024-08-20T23:29:31.3722979Z http.https://github.com/.extraheader 2024-08-20T23:29:31.3752283Z Entering 'third_party/NNPACK' 2024-08-20T23:29:31.3785457Z http.https://github.com/.extraheader 2024-08-20T23:29:31.3815971Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-20T23:29:31.3848498Z http.https://github.com/.extraheader 2024-08-20T23:29:31.3878405Z Entering 'third_party/XNNPACK' 2024-08-20T23:29:31.3911102Z http.https://github.com/.extraheader 2024-08-20T23:29:31.3957242Z Entering 'third_party/benchmark' 2024-08-20T23:29:31.3990322Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4020465Z Entering 'third_party/cpp-httplib' 2024-08-20T23:29:31.4053183Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4084485Z Entering 'third_party/cpuinfo' 2024-08-20T23:29:31.4117483Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4148065Z Entering 'third_party/cudnn_frontend' 2024-08-20T23:29:31.4181419Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4212011Z Entering 'third_party/cutlass' 2024-08-20T23:29:31.4244848Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4283114Z Entering 'third_party/eigen' 2024-08-20T23:29:31.4315794Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4348060Z Entering 'third_party/fbgemm' 2024-08-20T23:29:31.4383519Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4413746Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-20T23:29:31.4446331Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4476770Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-20T23:29:31.4508343Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4538035Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-20T23:29:31.4569666Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4605901Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-20T23:29:31.4637864Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4668499Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-20T23:29:31.4699889Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4730335Z Entering 'third_party/flatbuffers' 2024-08-20T23:29:31.4763721Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4796602Z Entering 'third_party/fmt' 2024-08-20T23:29:31.4829149Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4860225Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-20T23:29:31.4892532Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4922353Z Entering 'third_party/gloo' 2024-08-20T23:29:31.4954615Z http.https://github.com/.extraheader 2024-08-20T23:29:31.4984630Z Entering 'third_party/googletest' 2024-08-20T23:29:31.5016904Z http.https://github.com/.extraheader 2024-08-20T23:29:31.5046697Z Entering 'third_party/ideep' 2024-08-20T23:29:31.5079268Z http.https://github.com/.extraheader 2024-08-20T23:29:31.5108332Z Entering 'third_party/ideep/mkl-dnn' 2024-08-20T23:29:31.5140303Z http.https://github.com/.extraheader 2024-08-20T23:29:31.5177653Z Entering 'third_party/ittapi' 2024-08-20T23:29:31.5210498Z http.https://github.com/.extraheader 2024-08-20T23:29:31.5239950Z Entering 'third_party/kineto' 2024-08-20T23:29:31.5272913Z http.https://github.com/.extraheader 2024-08-20T23:29:31.5302806Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-20T23:29:31.5334870Z http.https://github.com/.extraheader 2024-08-20T23:29:31.5365225Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-20T23:29:31.5397646Z http.https://github.com/.extraheader 2024-08-20T23:29:31.5429013Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-20T23:29:31.5460748Z http.https://github.com/.extraheader 2024-08-20T23:29:31.5491078Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-20T23:29:31.5522586Z http.https://github.com/.extraheader 2024-08-20T23:29:31.5553225Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-20T23:29:31.5585197Z http.https://github.com/.extraheader 2024-08-20T23:29:31.5615358Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-20T23:29:31.5647384Z http.https://github.com/.extraheader 2024-08-20T23:29:31.5681421Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-20T23:29:31.5713644Z http.https://github.com/.extraheader 2024-08-20T23:29:31.5744853Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-20T23:29:31.5779777Z http.https://github.com/.extraheader 2024-08-20T23:29:31.5810310Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-20T23:29:31.5842229Z http.https://github.com/.extraheader 2024-08-20T23:29:31.5874666Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-20T23:29:31.5906905Z http.https://github.com/.extraheader 2024-08-20T23:29:31.5938446Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-20T23:29:31.5971053Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6000963Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-20T23:29:31.6032396Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6063745Z Entering 'third_party/mimalloc' 2024-08-20T23:29:31.6096088Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6126645Z Entering 'third_party/nccl/nccl' 2024-08-20T23:29:31.6159664Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6192880Z Entering 'third_party/nlohmann' 2024-08-20T23:29:31.6225623Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6256624Z Entering 'third_party/onnx' 2024-08-20T23:29:31.6289616Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6334858Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-20T23:29:31.6367629Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6398495Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-20T23:29:31.6431152Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6463727Z Entering 'third_party/opentelemetry-cpp' 2024-08-20T23:29:31.6496049Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6528144Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-20T23:29:31.6561919Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6592206Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-20T23:29:31.6624473Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6654235Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-20T23:29:31.6687299Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6716583Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-20T23:29:31.6748648Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6780658Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-20T23:29:31.6812379Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6842720Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-20T23:29:31.6875622Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6905604Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-20T23:29:31.6936945Z http.https://github.com/.extraheader 2024-08-20T23:29:31.6966378Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-20T23:29:31.6998353Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7030525Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-20T23:29:31.7062848Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7094968Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-20T23:29:31.7126691Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7178168Z Entering 'third_party/pocketfft' 2024-08-20T23:29:31.7211326Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7241945Z Entering 'third_party/protobuf' 2024-08-20T23:29:31.7275808Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7309069Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-20T23:29:31.7340658Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7371296Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-20T23:29:31.7403368Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7434464Z Entering 'third_party/psimd' 2024-08-20T23:29:31.7467600Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7497566Z Entering 'third_party/pthreadpool' 2024-08-20T23:29:31.7530053Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7560822Z Entering 'third_party/pybind11' 2024-08-20T23:29:31.7593321Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7623595Z Entering 'third_party/python-peachpy' 2024-08-20T23:29:31.7656023Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7686020Z Entering 'third_party/sleef' 2024-08-20T23:29:31.7719220Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7748965Z Entering 'third_party/tensorpipe' 2024-08-20T23:29:31.7782406Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7813172Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-20T23:29:31.7845026Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7875819Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-20T23:29:31.7907390Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7936857Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-20T23:29:31.7969207Z http.https://github.com/.extraheader 2024-08-20T23:29:31.7999534Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-20T23:29:31.8031253Z http.https://github.com/.extraheader 2024-08-20T23:29:31.8061448Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-20T23:29:31.8093281Z http.https://github.com/.extraheader 2024-08-20T23:29:31.8206672Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-20T23:29:31.8225601Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-20T23:29:31.8230906Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-20T23:29:31.8231411Z ##[endgroup] 2024-08-20T23:29:39.6818132Z Cleaning up orphan processes