2024-06-26T04:56:11.2115374Z Current runner version: '2.317.0' 2024-06-26T04:56:11.2121821Z Runner name: 'i-0b079ed883f442af4' 2024-06-26T04:56:11.2122671Z Runner group name: 'Default' 2024-06-26T04:56:11.2123504Z Machine name: 'ip-10-0-7-176' 2024-06-26T04:56:11.2128159Z ##[group]GITHUB_TOKEN Permissions 2024-06-26T04:56:11.2130275Z Actions: read 2024-06-26T04:56:11.2130823Z Attestations: read 2024-06-26T04:56:11.2131335Z Checks: read 2024-06-26T04:56:11.2131850Z Contents: read 2024-06-26T04:56:11.2132321Z Deployments: read 2024-06-26T04:56:11.2132838Z Discussions: read 2024-06-26T04:56:11.2133374Z Issues: read 2024-06-26T04:56:11.2133824Z Metadata: read 2024-06-26T04:56:11.2134322Z Packages: read 2024-06-26T04:56:11.2134833Z Pages: read 2024-06-26T04:56:11.2135291Z PullRequests: read 2024-06-26T04:56:11.2135832Z RepositoryProjects: read 2024-06-26T04:56:11.2136405Z SecurityEvents: read 2024-06-26T04:56:11.2136909Z Statuses: read 2024-06-26T04:56:11.2137393Z ##[endgroup] 2024-06-26T04:56:11.2141007Z Secret source: Actions 2024-06-26T04:56:11.2141864Z Prepare workflow directory 2024-06-26T04:56:11.3484834Z Prepare all required actions 2024-06-26T04:56:11.3649966Z Getting action download info 2024-06-26T04:56:11.5544268Z Download action repository 'pytorch/test-infra@main' (SHA:43a2ce341cc31288e9a38b65ce600a7f43021bd5) 2024-06-26T04:56:11.9346018Z Download action repository 'pytorch/pytorch@main' (SHA:6181e65cd81725efc6bc5d64ef3be607b0aa3ca1) 2024-06-26T04:56:14.8011035Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-06-26T04:56:15.0162905Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-06-26T04:56:15.2959789Z Getting action download info 2024-06-26T04:56:15.3872384Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-06-26T04:56:15.5824598Z Getting action download info 2024-06-26T04:56:15.6811364Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-06-26T04:56:15.8337127Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/pull/129470/merge (4b51b1a62a63a1add1b3a0f7882f5c7dc66b8f8d) 2024-06-26T04:56:15.8339519Z ##[group] Inputs 2024-06-26T04:56:15.8339933Z build-environment: linux-jammy-py3.8-gcc11 2024-06-26T04:56:15.8343065Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 3, "runner": "linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 3, "runner": "linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 3, "runner": "linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "linux.2xlarge"}]} 2024-06-26T04:56:15.8346684Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:56:15.8347670Z sync-tag: 2024-06-26T04:56:15.8348438Z timeout-minutes: 240 2024-06-26T04:56:15.8348765Z use-gha: 2024-06-26T04:56:15.8349033Z dashboard-tag: 2024-06-26T04:56:15.8349325Z s3-bucket: gha-artifacts 2024-06-26T04:56:15.8349664Z aws-role-to-assume: 2024-06-26T04:56:15.8349977Z ##[endgroup] 2024-06-26T04:56:15.8350705Z Complete job name: linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, linux.2xlarge) 2024-06-26T04:56:15.8892407Z A job started hook has been configured by the self-hosted runner administrator 2024-06-26T04:56:15.9035609Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T04:56:15.9047187Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T04:56:15.9047714Z ##[endgroup] 2024-06-26T04:56:16.6556226Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-06-26T04:56:16.6557022Z with: 2024-06-26T04:56:16.6557802Z github-secret: *** 2024-06-26T04:56:16.6558812Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-06-26T04:56:16.6559919Z activate-with-label: false 2024-06-26T04:56:16.6560330Z label: with-ssh 2024-06-26T04:56:16.6560683Z remove-existing-keys: true 2024-06-26T04:56:16.6561080Z fail-silently: true 2024-06-26T04:56:16.6561406Z env: 2024-06-26T04:56:16.6561706Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:56:16.6562084Z ##[endgroup] 2024-06-26T04:56:16.7458977Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-06-26T04:56:17.0262640Z Grabbing public ssh keys from https://github.com/leslie-fang-intel.keys 2024-06-26T04:56:17.0908042Z No SSH keys found for user leslie-fang-intel 2024-06-26T04:56:17.0908915Z Grabbing public ssh keys from https://github.com/leslie-fang-intel.keys 2024-06-26T04:56:17.1748517Z No SSH keys found for user leslie-fang-intel 2024-06-26T04:56:17.1864153Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-06-26T04:56:17.1864734Z with: 2024-06-26T04:56:17.1865042Z submodules: recursive 2024-06-26T04:56:17.1865406Z fetch-depth: 0 2024-06-26T04:56:17.1865708Z env: 2024-06-26T04:56:17.1866005Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:56:17.1866372Z ##[endgroup] 2024-06-26T04:56:17.2060644Z ##[group]Run retry () { 2024-06-26T04:56:17.2061071Z retry () { 2024-06-26T04:56:17.2061615Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-06-26T04:56:17.2062221Z } 2024-06-26T04:56:17.2062550Z echo "${GITHUB_WORKSPACE}" 2024-06-26T04:56:17.2063284Z if [ -z "${NO_SUDO}" ]; then 2024-06-26T04:56:17.2063805Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T04:56:17.2064286Z else 2024-06-26T04:56:17.2064677Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-06-26T04:56:17.2065144Z fi 2024-06-26T04:56:17.2065514Z mkdir "${GITHUB_WORKSPACE}" 2024-06-26T04:56:17.2073610Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T04:56:17.2074156Z env: 2024-06-26T04:56:17.2074462Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:56:17.2074822Z NO_SUDO: 2024-06-26T04:56:17.2075128Z ##[endgroup] 2024-06-26T04:56:17.2098882Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T04:56:19.0807102Z ##[group]Run malfet/checkout@silent-checkout 2024-06-26T04:56:19.0807565Z with: 2024-06-26T04:56:19.0807919Z ref: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T04:56:19.0808397Z fetch-depth: 0 2024-06-26T04:56:19.0808725Z submodules: recursive 2024-06-26T04:56:19.0809097Z quiet-checkout: true 2024-06-26T04:56:19.0809478Z repository: pytorch/pytorch 2024-06-26T04:56:19.0810003Z token: *** 2024-06-26T04:56:19.0810327Z ssh-strict: true 2024-06-26T04:56:19.0810701Z persist-credentials: true 2024-06-26T04:56:19.0811072Z clean: true 2024-06-26T04:56:19.0811436Z sparse-checkout-cone-mode: true 2024-06-26T04:56:19.0811854Z lfs: false 2024-06-26T04:56:19.0812169Z set-safe-directory: true 2024-06-26T04:56:19.0812537Z env: 2024-06-26T04:56:19.0812836Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:56:19.0813191Z ##[endgroup] 2024-06-26T04:56:19.1767169Z Syncing repository: pytorch/pytorch 2024-06-26T04:56:19.1768972Z ##[group]Getting Git version info 2024-06-26T04:56:19.1769786Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T04:56:19.1770725Z [command]/usr/bin/git version 2024-06-26T04:56:19.1771153Z git version 2.40.1 2024-06-26T04:56:19.1772783Z ##[endgroup] 2024-06-26T04:56:19.1784483Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/edd8a3eb-da3c-417c-9a23-99d18ce79b15' before making global git config changes 2024-06-26T04:56:19.1785835Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T04:56:19.1787233Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T04:56:19.1807968Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-06-26T04:56:19.1811790Z ##[group]Initializing the repository 2024-06-26T04:56:19.1814481Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T04:56:19.1948294Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-06-26T04:56:19.1949245Z hint: is subject to change. To configure the initial branch name to use in all 2024-06-26T04:56:19.1950131Z hint: of your new repositories, which will suppress this warning, call: 2024-06-26T04:56:19.1950738Z hint: 2024-06-26T04:56:19.1951390Z hint: git config --global init.defaultBranch 2024-06-26T04:56:19.1951957Z hint: 2024-06-26T04:56:19.1952515Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-06-26T04:56:19.1953656Z hint: 'development'. The just-created branch can be renamed via this command: 2024-06-26T04:56:19.1954563Z hint: 2024-06-26T04:56:19.1954925Z hint: git branch -m 2024-06-26T04:56:19.1955714Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-06-26T04:56:19.1961433Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-06-26T04:56:19.1992675Z ##[endgroup] 2024-06-26T04:56:19.1993311Z ##[group]Disabling automatic garbage collection 2024-06-26T04:56:19.1995989Z [command]/usr/bin/git config --local gc.auto 0 2024-06-26T04:56:19.2024164Z ##[endgroup] 2024-06-26T04:56:19.2024726Z ##[group]Setting up auth 2024-06-26T04:56:19.2030603Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T04:56:19.2059046Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T04:56:19.2288620Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T04:56:19.2318073Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T04:56:19.2553863Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T04:56:19.2595528Z ##[endgroup] 2024-06-26T04:56:19.2596141Z ##[group]Fetching the repository 2024-06-26T04:56:19.2601913Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-06-26T04:56:21.5183467Z remote: Enumerating objects: 987786 2024-06-26T04:56:21.5184442Z remote: Enumerating objects: 989963, done. 2024-06-26T04:56:21.5185453Z remote: Counting objects: 0% (1/2177) 2024-06-26T04:56:21.5186425Z remote: Counting objects: 1% (22/2177) 2024-06-26T04:56:21.5187386Z remote: Counting objects: 2% (44/2177) 2024-06-26T04:56:21.5188313Z remote: Counting objects: 3% (66/2177) 2024-06-26T04:56:21.5189279Z remote: Counting objects: 4% (88/2177) 2024-06-26T04:56:21.5190011Z remote: Counting objects: 5% (109/2177) 2024-06-26T04:56:21.5190750Z remote: Counting objects: 6% (131/2177) 2024-06-26T04:56:21.5191557Z remote: Counting objects: 7% (153/2177) 2024-06-26T04:56:21.5192618Z remote: Counting objects: 8% (175/2177) 2024-06-26T04:56:21.5193617Z remote: Counting objects: 9% (196/2177) 2024-06-26T04:56:21.5194577Z remote: Counting objects: 10% (218/2177) 2024-06-26T04:56:21.5195400Z remote: Counting objects: 11% (240/2177) 2024-06-26T04:56:21.5196237Z remote: Counting objects: 12% (262/2177) 2024-06-26T04:56:21.5197579Z remote: Counting objects: 13% (284/2177) 2024-06-26T04:56:21.5198297Z remote: Counting objects: 14% (305/2177) 2024-06-26T04:56:21.5198879Z remote: Counting objects: 15% (327/2177) 2024-06-26T04:56:21.5199887Z remote: Counting objects: 16% (349/2177) 2024-06-26T04:56:21.5200492Z remote: Counting objects: 17% (371/2177) 2024-06-26T04:56:21.5201080Z remote: Counting objects: 18% (392/2177) 2024-06-26T04:56:21.5201624Z remote: Counting objects: 19% (414/2177) 2024-06-26T04:56:21.5202190Z remote: Counting objects: 20% (436/2177) 2024-06-26T04:56:21.5202733Z remote: Counting objects: 21% (458/2177) 2024-06-26T04:56:21.5203273Z remote: Counting objects: 22% (479/2177) 2024-06-26T04:56:21.5203802Z remote: Counting objects: 23% (501/2177) 2024-06-26T04:56:21.5204525Z remote: Counting objects: 24% (523/2177) 2024-06-26T04:56:21.5205105Z remote: Counting objects: 25% (545/2177) 2024-06-26T04:56:21.5205712Z remote: Counting objects: 26% (567/2177) 2024-06-26T04:56:21.5206524Z remote: Counting objects: 27% (588/2177) 2024-06-26T04:56:21.5207624Z remote: Counting objects: 28% (610/2177) 2024-06-26T04:56:21.5208208Z remote: Counting objects: 29% (632/2177) 2024-06-26T04:56:21.5208738Z remote: Counting objects: 30% (654/2177) 2024-06-26T04:56:21.5209285Z remote: Counting objects: 31% (675/2177) 2024-06-26T04:56:21.5209918Z remote: Counting objects: 32% (697/2177) 2024-06-26T04:56:21.5210715Z remote: Counting objects: 33% (719/2177) 2024-06-26T04:56:21.5211427Z remote: Counting objects: 34% (741/2177) 2024-06-26T04:56:21.5211966Z remote: Counting objects: 35% (762/2177) 2024-06-26T04:56:21.5212506Z remote: Counting objects: 36% (784/2177) 2024-06-26T04:56:21.5213046Z remote: Counting objects: 37% (806/2177) 2024-06-26T04:56:21.5213587Z remote: Counting objects: 38% (828/2177) 2024-06-26T04:56:21.5214125Z remote: Counting objects: 39% (850/2177) 2024-06-26T04:56:21.5214666Z remote: Counting objects: 40% (871/2177) 2024-06-26T04:56:21.5215213Z remote: Counting objects: 41% (893/2177) 2024-06-26T04:56:21.5215802Z remote: Counting objects: 42% (915/2177) 2024-06-26T04:56:21.5216331Z remote: Counting objects: 43% (937/2177) 2024-06-26T04:56:21.5217291Z remote: Counting objects: 44% (958/2177) 2024-06-26T04:56:21.5217937Z remote: Counting objects: 45% (980/2177) 2024-06-26T04:56:21.5218472Z remote: Counting objects: 46% (1002/2177) 2024-06-26T04:56:21.5219035Z remote: Counting objects: 47% (1024/2177) 2024-06-26T04:56:21.5219592Z remote: Counting objects: 48% (1045/2177) 2024-06-26T04:56:21.5220135Z remote: Counting objects: 49% (1067/2177) 2024-06-26T04:56:21.5220690Z remote: Counting objects: 50% (1089/2177) 2024-06-26T04:56:21.5221288Z remote: Counting objects: 51% (1111/2177) 2024-06-26T04:56:21.5222195Z remote: Counting objects: 52% (1133/2177) 2024-06-26T04:56:21.5222882Z remote: Counting objects: 53% (1154/2177) 2024-06-26T04:56:21.5223433Z remote: Counting objects: 54% (1176/2177) 2024-06-26T04:56:21.5223990Z remote: Counting objects: 55% (1198/2177) 2024-06-26T04:56:21.5224571Z remote: Counting objects: 56% (1220/2177) 2024-06-26T04:56:21.5225123Z remote: Counting objects: 57% (1241/2177) 2024-06-26T04:56:21.5227217Z remote: Counting objects: 58% (1263/2177) 2024-06-26T04:56:21.5228019Z remote: Counting objects: 59% (1285/2177) 2024-06-26T04:56:21.5231254Z remote: Counting objects: 60% (1307/2177) 2024-06-26T04:56:21.5231901Z remote: Counting objects: 61% (1328/2177) 2024-06-26T04:56:21.5232444Z remote: Counting objects: 62% (1350/2177) 2024-06-26T04:56:21.5233017Z remote: Counting objects: 63% (1372/2177) 2024-06-26T04:56:21.5233575Z remote: Counting objects: 64% (1394/2177) 2024-06-26T04:56:21.5234288Z remote: Counting objects: 65% (1416/2177) 2024-06-26T04:56:21.5234836Z remote: Counting objects: 66% (1437/2177) 2024-06-26T04:56:21.5235393Z remote: Counting objects: 67% (1459/2177) 2024-06-26T04:56:21.5235950Z remote: Counting objects: 68% (1481/2177) 2024-06-26T04:56:21.5236491Z remote: Counting objects: 69% (1503/2177) 2024-06-26T04:56:21.5237048Z remote: Counting objects: 70% (1524/2177) 2024-06-26T04:56:21.5237604Z remote: Counting objects: 71% (1546/2177) 2024-06-26T04:56:21.5238157Z remote: Counting objects: 72% (1568/2177) 2024-06-26T04:56:21.5238699Z remote: Counting objects: 73% (1590/2177) 2024-06-26T04:56:21.5239253Z remote: Counting objects: 74% (1611/2177) 2024-06-26T04:56:21.5239808Z remote: Counting objects: 75% (1633/2177) 2024-06-26T04:56:21.5240349Z remote: Counting objects: 76% (1655/2177) 2024-06-26T04:56:21.5240900Z remote: Counting objects: 77% (1677/2177) 2024-06-26T04:56:21.5241455Z remote: Counting objects: 78% (1699/2177) 2024-06-26T04:56:21.5242066Z remote: Counting objects: 79% (1720/2177) 2024-06-26T04:56:21.5242627Z remote: Counting objects: 80% (1742/2177) 2024-06-26T04:56:21.5243179Z remote: Counting objects: 81% (1764/2177) 2024-06-26T04:56:21.5243719Z remote: Counting objects: 82% (1786/2177) 2024-06-26T04:56:21.5244642Z remote: Counting objects: 83% (1807/2177) 2024-06-26T04:56:21.5246883Z remote: Counting objects: 84% (1829/2177) 2024-06-26T04:56:21.5247829Z remote: Counting objects: 85% (1851/2177) 2024-06-26T04:56:21.5248347Z remote: Counting objects: 86% (1873/2177) 2024-06-26T04:56:21.5248856Z remote: Counting objects: 87% (1894/2177) 2024-06-26T04:56:21.5249367Z remote: Counting objects: 88% (1916/2177) 2024-06-26T04:56:21.5249862Z remote: Counting objects: 89% (1938/2177) 2024-06-26T04:56:21.5250360Z remote: Counting objects: 90% (1960/2177) 2024-06-26T04:56:21.5250882Z remote: Counting objects: 91% (1982/2177) 2024-06-26T04:56:21.5251382Z remote: Counting objects: 92% (2003/2177) 2024-06-26T04:56:21.5251881Z remote: Counting objects: 93% (2025/2177) 2024-06-26T04:56:21.5252380Z remote: Counting objects: 94% (2047/2177) 2024-06-26T04:56:21.5252869Z remote: Counting objects: 95% (2069/2177) 2024-06-26T04:56:21.5253374Z remote: Counting objects: 96% (2090/2177) 2024-06-26T04:56:21.5253874Z remote: Counting objects: 97% (2112/2177) 2024-06-26T04:56:21.5254372Z remote: Counting objects: 98% (2134/2177) 2024-06-26T04:56:21.5254861Z remote: Counting objects: 99% (2156/2177) 2024-06-26T04:56:21.5255362Z remote: Counting objects: 100% (2177/2177) 2024-06-26T04:56:21.5255897Z remote: Counting objects: 100% (2177/2177), done. 2024-06-26T04:56:21.5347946Z remote: Compressing objects: 0% (1/896) 2024-06-26T04:56:21.5655516Z remote: Compressing objects: 1% (9/896) 2024-06-26T04:56:21.5989392Z remote: Compressing objects: 2% (18/896) 2024-06-26T04:56:21.6287675Z remote: Compressing objects: 3% (27/896) 2024-06-26T04:56:21.6630079Z remote: Compressing objects: 4% (36/896) 2024-06-26T04:56:21.6766929Z remote: Compressing objects: 5% (45/896) 2024-06-26T04:56:21.6915850Z remote: Compressing objects: 6% (54/896) 2024-06-26T04:56:21.7096159Z remote: Compressing objects: 7% (63/896) 2024-06-26T04:56:21.7256921Z remote: Compressing objects: 8% (72/896) 2024-06-26T04:56:21.7574857Z remote: Compressing objects: 9% (81/896) 2024-06-26T04:56:21.8325554Z remote: Compressing objects: 10% (90/896) 2024-06-26T04:56:21.8921599Z remote: Compressing objects: 11% (99/896) 2024-06-26T04:56:21.9723558Z remote: Compressing objects: 12% (108/896) 2024-06-26T04:56:21.9947407Z remote: Compressing objects: 13% (117/896) 2024-06-26T04:56:22.0085918Z remote: Compressing objects: 14% (126/896) 2024-06-26T04:56:22.0195042Z remote: Compressing objects: 15% (135/896) 2024-06-26T04:56:22.0290839Z remote: Compressing objects: 16% (144/896) 2024-06-26T04:56:22.0337835Z remote: Compressing objects: 17% (153/896) 2024-06-26T04:56:22.0348723Z remote: Compressing objects: 18% (162/896) 2024-06-26T04:56:22.0356149Z remote: Compressing objects: 19% (171/896) 2024-06-26T04:56:22.0373965Z remote: Compressing objects: 20% (180/896) 2024-06-26T04:56:22.0391873Z remote: Compressing objects: 21% (189/896) 2024-06-26T04:56:22.0408620Z remote: Compressing objects: 22% (198/896) 2024-06-26T04:56:22.0415723Z remote: Compressing objects: 23% (207/896) 2024-06-26T04:56:22.0431442Z remote: Compressing objects: 24% (216/896) 2024-06-26T04:56:22.0445801Z remote: Compressing objects: 25% (224/896) 2024-06-26T04:56:22.0454315Z remote: Compressing objects: 26% (233/896) 2024-06-26T04:56:22.0460970Z remote: Compressing objects: 27% (242/896) 2024-06-26T04:56:22.0468189Z remote: Compressing objects: 28% (251/896) 2024-06-26T04:56:22.0471129Z remote: Compressing objects: 29% (260/896) 2024-06-26T04:56:22.0474330Z remote: Compressing objects: 30% (269/896) 2024-06-26T04:56:22.0478094Z remote: Compressing objects: 31% (278/896) 2024-06-26T04:56:22.0486198Z remote: Compressing objects: 32% (287/896) 2024-06-26T04:56:22.0491315Z remote: Compressing objects: 33% (296/896) 2024-06-26T04:56:22.0497135Z remote: Compressing objects: 34% (305/896) 2024-06-26T04:56:22.0503317Z remote: Compressing objects: 35% (314/896) 2024-06-26T04:56:22.0508116Z remote: Compressing objects: 36% (323/896) 2024-06-26T04:56:22.0513344Z remote: Compressing objects: 37% (332/896) 2024-06-26T04:56:22.0527017Z remote: Compressing objects: 38% (341/896) 2024-06-26T04:56:22.0536533Z remote: Compressing objects: 39% (350/896) 2024-06-26T04:56:22.0550523Z remote: Compressing objects: 40% (359/896) 2024-06-26T04:56:22.0562988Z remote: Compressing objects: 41% (368/896) 2024-06-26T04:56:22.0574900Z remote: Compressing objects: 42% (377/896) 2024-06-26T04:56:22.0594936Z remote: Compressing objects: 43% (386/896) 2024-06-26T04:56:22.0634154Z remote: Compressing objects: 44% (395/896) 2024-06-26T04:56:22.0634681Z remote: Compressing objects: 45% (404/896) 2024-06-26T04:56:22.0635202Z remote: Compressing objects: 46% (413/896) 2024-06-26T04:56:22.0647271Z remote: Compressing objects: 47% (422/896) 2024-06-26T04:56:22.0670313Z remote: Compressing objects: 48% (431/896) 2024-06-26T04:56:22.0670837Z remote: Compressing objects: 49% (440/896) 2024-06-26T04:56:22.0674624Z remote: Compressing objects: 50% (448/896) 2024-06-26T04:56:22.0679504Z remote: Compressing objects: 51% (457/896) 2024-06-26T04:56:22.0683587Z remote: Compressing objects: 52% (466/896) 2024-06-26T04:56:22.0689127Z remote: Compressing objects: 53% (475/896) 2024-06-26T04:56:22.0704239Z remote: Compressing objects: 54% (484/896) 2024-06-26T04:56:22.0717313Z remote: Compressing objects: 55% (493/896) 2024-06-26T04:56:22.0725006Z remote: Compressing objects: 56% (502/896) 2024-06-26T04:56:22.0733959Z remote: Compressing objects: 57% (511/896) 2024-06-26T04:56:22.0747178Z remote: Compressing objects: 58% (520/896) 2024-06-26T04:56:22.0759123Z remote: Compressing objects: 59% (529/896) 2024-06-26T04:56:22.0772555Z remote: Compressing objects: 60% (538/896) 2024-06-26T04:56:22.0783177Z remote: Compressing objects: 61% (547/896) 2024-06-26T04:56:22.0791707Z remote: Compressing objects: 62% (556/896) 2024-06-26T04:56:22.0798911Z remote: Compressing objects: 63% (565/896) 2024-06-26T04:56:22.0810997Z remote: Compressing objects: 64% (574/896) 2024-06-26T04:56:22.0819550Z remote: Compressing objects: 65% (583/896) 2024-06-26T04:56:22.0836985Z remote: Compressing objects: 66% (592/896) 2024-06-26T04:56:22.0839766Z remote: Compressing objects: 67% (601/896) 2024-06-26T04:56:22.0848872Z remote: Compressing objects: 68% (610/896) 2024-06-26T04:56:22.0858576Z remote: Compressing objects: 69% (619/896) 2024-06-26T04:56:22.0865074Z remote: Compressing objects: 70% (628/896) 2024-06-26T04:56:22.0870973Z remote: Compressing objects: 71% (637/896) 2024-06-26T04:56:22.0881622Z remote: Compressing objects: 72% (646/896) 2024-06-26T04:56:22.0890126Z remote: Compressing objects: 73% (655/896) 2024-06-26T04:56:22.0895614Z remote: Compressing objects: 74% (664/896) 2024-06-26T04:56:22.0899892Z remote: Compressing objects: 75% (672/896) 2024-06-26T04:56:22.0903589Z remote: Compressing objects: 76% (681/896) 2024-06-26T04:56:22.0905878Z remote: Compressing objects: 77% (690/896) 2024-06-26T04:56:22.0906922Z remote: Compressing objects: 78% (699/896) 2024-06-26T04:56:22.0908266Z remote: Compressing objects: 79% (708/896) 2024-06-26T04:56:22.0909070Z remote: Compressing objects: 80% (717/896) 2024-06-26T04:56:22.0909623Z remote: Compressing objects: 81% (726/896) 2024-06-26T04:56:22.0911856Z remote: Compressing objects: 82% (735/896) 2024-06-26T04:56:22.0916149Z remote: Compressing objects: 83% (744/896) 2024-06-26T04:56:22.0920092Z remote: Compressing objects: 84% (753/896) 2024-06-26T04:56:22.0922719Z remote: Compressing objects: 85% (762/896) 2024-06-26T04:56:22.0924565Z remote: Compressing objects: 86% (771/896) 2024-06-26T04:56:22.0926648Z remote: Compressing objects: 87% (780/896) 2024-06-26T04:56:22.0927726Z remote: Compressing objects: 88% (789/896) 2024-06-26T04:56:22.0928960Z remote: Compressing objects: 89% (798/896) 2024-06-26T04:56:22.0930047Z remote: Compressing objects: 90% (807/896) 2024-06-26T04:56:22.0931112Z remote: Compressing objects: 91% (816/896) 2024-06-26T04:56:22.0932526Z remote: Compressing objects: 92% (825/896) 2024-06-26T04:56:22.0933743Z remote: Compressing objects: 93% (834/896) 2024-06-26T04:56:22.0935722Z remote: Compressing objects: 94% (843/896) 2024-06-26T04:56:22.0936813Z remote: Compressing objects: 95% (852/896) 2024-06-26T04:56:22.0937902Z remote: Compressing objects: 96% (861/896) 2024-06-26T04:56:22.0938951Z remote: Compressing objects: 97% (870/896) 2024-06-26T04:56:22.0939763Z remote: Compressing objects: 98% (879/896) 2024-06-26T04:56:22.0940529Z remote: Compressing objects: 99% (888/896) 2024-06-26T04:56:22.0941328Z remote: Compressing objects: 100% (896/896) 2024-06-26T04:56:22.0941877Z remote: Compressing objects: 100% (896/896), done. 2024-06-26T04:56:41.9158808Z remote: Total 989963 (delta 1757), reused 1545 (delta 1280), pack-reused 987786 2024-06-26T04:57:08.0203287Z [command]/usr/bin/git rev-parse --verify --quiet b8c4c54d347aa776934c60784e35936878ef18dc^{object} 2024-06-26T04:57:08.0227948Z b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T04:57:08.0232088Z ##[endgroup] 2024-06-26T04:57:08.0232611Z ##[group]Determining the checkout info 2024-06-26T04:57:08.0234824Z ##[endgroup] 2024-06-26T04:57:08.0235339Z ##[group]Checking out the ref 2024-06-26T04:57:08.0236964Z [command]/usr/bin/git checkout --quiet --force b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T04:57:09.3385039Z ##[endgroup] 2024-06-26T04:57:09.3386930Z ##[group]Setting up auth for fetching submodules 2024-06-26T04:57:09.3391061Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-06-26T04:57:09.3439233Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-06-26T04:57:09.3465008Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-06-26T04:57:09.3491931Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-06-26T04:57:09.3515152Z ##[endgroup] 2024-06-26T04:57:09.3515650Z ##[group]Fetching submodules 2024-06-26T04:57:09.3518512Z [command]/usr/bin/git submodule sync --recursive 2024-06-26T04:57:09.3770383Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-06-26T04:57:09.4016803Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-06-26T04:57:09.4018291Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-06-26T04:57:09.4019741Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-06-26T04:57:09.4021442Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-06-26T04:57:09.4024292Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-06-26T04:57:09.4026020Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-06-26T04:57:09.4028338Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-06-26T04:57:09.4030920Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-06-26T04:57:09.4033512Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-06-26T04:57:09.4036358Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-06-26T04:57:09.4038960Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-06-26T04:57:09.4041772Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-06-26T04:57:09.4045934Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-06-26T04:57:09.4048414Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-06-26T04:57:09.4051325Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-06-26T04:57:09.4054611Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-06-26T04:57:09.4058105Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-06-26T04:57:09.4061318Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-06-26T04:57:09.4064906Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-06-26T04:57:09.4068353Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-06-26T04:57:09.4072194Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-06-26T04:57:09.4076072Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-06-26T04:57:09.4080171Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-06-26T04:57:09.4084246Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-06-26T04:57:09.4088627Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-06-26T04:57:09.4092721Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-06-26T04:57:09.4097393Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-06-26T04:57:09.4101541Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-06-26T04:57:09.4106093Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-06-26T04:57:09.4110632Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-06-26T04:57:09.4115622Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-06-26T04:57:09.4120347Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-06-26T04:57:09.4125369Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-06-26T04:57:09.4130259Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-06-26T04:57:09.4135514Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-06-26T04:57:09.4159195Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-06-26T04:57:09.6525638Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-06-26T04:57:09.8311780Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-06-26T04:57:10.0236102Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-06-26T04:57:10.2587963Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-06-26T04:57:12.5405675Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-06-26T04:57:24.5656961Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-06-26T04:57:24.9717024Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-06-26T04:57:25.5489266Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-06-26T04:57:26.1626275Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-06-26T04:57:27.3749300Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-06-26T04:57:29.1738834Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-06-26T04:57:34.6930488Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-06-26T04:57:36.2929596Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-06-26T04:57:38.0921655Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-06-26T04:57:39.4680814Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-06-26T04:57:39.6797069Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-06-26T04:57:40.0851843Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-06-26T04:57:40.4136422Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-06-26T04:57:41.5826325Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-06-26T04:57:41.9483544Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-06-26T04:57:42.2113651Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-06-26T04:57:43.7224387Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-06-26T04:57:44.4961324Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-06-26T04:57:45.1705001Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-06-26T04:57:51.4532607Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-06-26T04:57:54.1453463Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-06-26T04:58:01.2550660Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-06-26T04:58:01.5692802Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-06-26T04:58:10.6947463Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-06-26T04:58:10.8516697Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-06-26T04:58:11.0470133Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-06-26T04:58:12.1450692Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-06-26T04:58:12.4324463Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-06-26T04:58:13.0953268Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-06-26T04:58:13.5154547Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-06-26T04:58:13.5251269Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-06-26T04:58:13.5324137Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-06-26T04:58:13.5523879Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-06-26T04:58:13.5841464Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-06-26T04:58:14.4255125Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-06-26T04:58:14.4454339Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-06-26T04:58:14.4812323Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-06-26T04:58:14.5706881Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-06-26T04:58:14.6001793Z Submodule path 'third_party/cudnn_frontend': checked out 'aa3abd4bc689d6412979c7f55f9cd132848c9c6a' 2024-06-26T04:58:15.0636860Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-06-26T04:58:15.2926515Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-06-26T04:58:15.3613464Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-06-26T04:58:15.3628420Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-06-26T04:58:15.3632422Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T04:58:15.3634426Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-06-26T04:58:15.3636239Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-06-26T04:58:15.3638158Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T04:58:15.3659038Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-06-26T04:58:16.3873070Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-06-26T04:58:16.9759044Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-06-26T04:58:18.7112355Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-06-26T04:58:19.8233324Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-06-26T04:58:20.1282908Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-06-26T04:58:20.2155360Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-06-26T04:58:20.5807030Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-06-26T04:58:20.6381589Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-06-26T04:58:20.6485408Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-06-26T04:58:20.7501110Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-06-26T04:58:20.7822272Z Submodule path 'third_party/fmt': checked out 'e69e5f977d458f2650bb346dadf2ad30c5320281' 2024-06-26T04:58:20.7903909Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-06-26T04:58:20.8271243Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-06-26T04:58:20.8490494Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-06-26T04:58:20.8902221Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T04:58:20.9012007Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-06-26T04:58:20.9024456Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-06-26T04:58:20.9045287Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-06-26T04:58:34.1523383Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-06-26T04:58:34.1682993Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-06-26T04:58:34.2544961Z Submodule path 'third_party/kineto': checked out '8681ff11e1fa54da39023076c5c43eddd87b7a8a' 2024-06-26T04:58:34.2559368Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T04:58:34.2561442Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T04:58:34.2564259Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T04:58:34.2587241Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-06-26T04:58:34.7927588Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-06-26T04:58:36.1389095Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-06-26T04:58:37.2753686Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-06-26T04:58:37.2767080Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T04:58:37.2769447Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T04:58:37.2771388Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T04:58:37.2773394Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T04:58:37.2775411Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T04:58:37.2777919Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T04:58:37.2780496Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T04:58:37.2782869Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T04:58:37.2807010Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-06-26T04:58:38.2920582Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-06-26T04:58:38.6120853Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-06-26T04:58:40.0313927Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-06-26T04:58:40.2876691Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-06-26T04:58:40.7731675Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-06-26T04:58:41.8556718Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-06-26T04:58:48.5789238Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-06-26T04:58:48.9902901Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-06-26T04:58:49.0074657Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-06-26T04:58:49.0416386Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-06-26T04:58:49.0531522Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-06-26T04:58:49.0543193Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T04:58:49.0566052Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-06-26T04:58:49.3953233Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-06-26T04:58:49.4114563Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-06-26T04:58:49.4478840Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-06-26T04:58:49.5435761Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-06-26T04:58:49.5580899Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-06-26T04:58:49.5921734Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out 'a33701196adfad74917046096bf5a2aa0ab0bb50' 2024-06-26T04:58:49.6467607Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-06-26T04:58:49.6799420Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-06-26T04:58:49.7018579Z Submodule path 'third_party/nccl/nccl': checked out '48bb7fec7953112ff37499a272317f6663f8f600' 2024-06-26T04:58:49.7997822Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-06-26T04:58:50.1002594Z Submodule path 'third_party/onnx': checked out '990217f043af7222348ca8f0301e17fa7b841781' 2024-06-26T04:58:50.1033402Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-06-26T04:58:50.1035224Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-06-26T04:58:50.1058804Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-06-26T04:58:50.5255525Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-06-26T04:58:51.7279630Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-06-26T04:58:51.7583859Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-06-26T04:58:51.8158164Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-06-26T04:58:51.8174116Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T04:58:51.8176172Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T04:58:51.8178044Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T04:58:51.8179989Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T04:58:51.8182277Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T04:58:51.8184676Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T04:58:51.8186954Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T04:58:51.8188880Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T04:58:51.8211294Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-06-26T04:58:52.2727408Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-06-26T04:58:53.3862572Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-06-26T04:58:53.6704362Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-06-26T04:58:59.8879884Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-06-26T04:59:00.1437899Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-06-26T04:59:00.3272378Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-06-26T04:59:00.6041773Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-06-26T04:59:08.0307381Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-06-26T04:59:08.0668275Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-06-26T04:59:08.0805363Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-06-26T04:59:08.1723269Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-06-26T04:59:08.1837182Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-06-26T04:59:08.1958893Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-06-26T04:59:08.2085000Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-06-26T04:59:08.2098708Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T04:59:08.2101417Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T04:59:08.2120632Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-06-26T04:59:10.6068106Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-06-26T04:59:11.9147043Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-06-26T04:59:11.9563811Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-06-26T04:59:12.3435850Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-06-26T04:59:12.3538111Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-06-26T04:59:12.5936181Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-06-26T04:59:12.5955098Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-06-26T04:59:12.5957135Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-06-26T04:59:12.5980433Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-06-26T04:59:12.9716861Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-06-26T04:59:14.0650760Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-06-26T04:59:14.1307458Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-06-26T04:59:14.1380760Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-06-26T04:59:14.1481593Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-06-26T04:59:14.1782254Z Submodule path 'third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b' 2024-06-26T04:59:14.2036309Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-06-26T04:59:14.2408964Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-06-26T04:59:14.2634766Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-06-26T04:59:14.2648573Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-06-26T04:59:14.2650441Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-06-26T04:59:14.2652941Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-06-26T04:59:14.2655438Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T04:59:14.2677291Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-06-26T04:59:15.3574452Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-06-26T04:59:15.5731882Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-06-26T04:59:16.7004722Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-06-26T04:59:17.9067074Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-06-26T04:59:17.9199779Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-06-26T04:59:17.9838245Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-06-26T04:59:18.0090799Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-06-26T04:59:18.0103407Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T04:59:18.0125877Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-06-26T04:59:18.1814326Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-06-26T04:59:18.1841837Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-06-26T04:59:18.2094707Z Entering 'android/libs/fbjni' 2024-06-26T04:59:18.2126680Z Entering 'third_party/FP16' 2024-06-26T04:59:18.2158774Z Entering 'third_party/FXdiv' 2024-06-26T04:59:18.2190124Z Entering 'third_party/NNPACK' 2024-06-26T04:59:18.2224205Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T04:59:18.2256499Z Entering 'third_party/XNNPACK' 2024-06-26T04:59:18.2304551Z Entering 'third_party/benchmark' 2024-06-26T04:59:18.2336932Z Entering 'third_party/cpp-httplib' 2024-06-26T04:59:18.2368716Z Entering 'third_party/cpuinfo' 2024-06-26T04:59:18.2401226Z Entering 'third_party/cudnn_frontend' 2024-06-26T04:59:18.2433564Z Entering 'third_party/cutlass' 2024-06-26T04:59:18.2474694Z Entering 'third_party/eigen' 2024-06-26T04:59:18.2509425Z Entering 'third_party/fbgemm' 2024-06-26T04:59:18.2544084Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T04:59:18.2575798Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T04:59:18.2607444Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T04:59:18.2646204Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T04:59:18.2677393Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T04:59:18.2710549Z Entering 'third_party/flatbuffers' 2024-06-26T04:59:18.2746621Z Entering 'third_party/fmt' 2024-06-26T04:59:18.2779416Z Entering 'third_party/foxi' 2024-06-26T04:59:18.2813009Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T04:59:18.2846591Z Entering 'third_party/gloo' 2024-06-26T04:59:18.2878601Z Entering 'third_party/googletest' 2024-06-26T04:59:18.2911847Z Entering 'third_party/ideep' 2024-06-26T04:59:18.2944732Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T04:59:18.2984321Z Entering 'third_party/ittapi' 2024-06-26T04:59:18.3017628Z Entering 'third_party/kineto' 2024-06-26T04:59:18.3052159Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T04:59:18.3085054Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T04:59:18.3118295Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T04:59:18.3150119Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T04:59:18.3183178Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T04:59:18.3214414Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T04:59:18.3248923Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T04:59:18.3280823Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T04:59:18.3312696Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T04:59:18.3347152Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T04:59:18.3380641Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T04:59:18.3413529Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T04:59:18.3447277Z Entering 'third_party/mimalloc' 2024-06-26T04:59:18.3481370Z Entering 'third_party/nccl/nccl' 2024-06-26T04:59:18.3514754Z Entering 'third_party/nlohmann' 2024-06-26T04:59:18.3549432Z Entering 'third_party/onnx' 2024-06-26T04:59:18.3599123Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T04:59:18.3662423Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T04:59:18.3668637Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T04:59:18.3704456Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T04:59:18.3736048Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T04:59:18.3768619Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T04:59:18.3800455Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T04:59:18.3833098Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T04:59:18.3864804Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T04:59:18.3896314Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T04:59:18.3926941Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T04:59:18.3960776Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T04:59:18.3993688Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T04:59:18.4045872Z Entering 'third_party/pocketfft' 2024-06-26T04:59:18.4077982Z Entering 'third_party/protobuf' 2024-06-26T04:59:18.4112886Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T04:59:18.4143849Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T04:59:18.4177674Z Entering 'third_party/psimd' 2024-06-26T04:59:18.4211490Z Entering 'third_party/pthreadpool' 2024-06-26T04:59:18.4243088Z Entering 'third_party/pybind11' 2024-06-26T04:59:18.4275730Z Entering 'third_party/python-peachpy' 2024-06-26T04:59:18.4308060Z Entering 'third_party/sleef' 2024-06-26T04:59:18.4342309Z Entering 'third_party/tensorpipe' 2024-06-26T04:59:18.4375552Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T04:59:18.4407919Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T04:59:18.4438832Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T04:59:18.4470834Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T04:59:18.4502874Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T04:59:18.4548840Z ##[endgroup] 2024-06-26T04:59:18.4549759Z ##[group]Persisting credentials for submodules 2024-06-26T04:59:18.4555632Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-06-26T04:59:18.4803808Z Entering 'android/libs/fbjni' 2024-06-26T04:59:18.4844463Z Entering 'third_party/FP16' 2024-06-26T04:59:18.4886957Z Entering 'third_party/FXdiv' 2024-06-26T04:59:18.4926424Z Entering 'third_party/NNPACK' 2024-06-26T04:59:18.4967210Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T04:59:18.5007873Z Entering 'third_party/XNNPACK' 2024-06-26T04:59:18.5064085Z Entering 'third_party/benchmark' 2024-06-26T04:59:18.5105518Z Entering 'third_party/cpp-httplib' 2024-06-26T04:59:18.5145641Z Entering 'third_party/cpuinfo' 2024-06-26T04:59:18.5187071Z Entering 'third_party/cudnn_frontend' 2024-06-26T04:59:18.5228075Z Entering 'third_party/cutlass' 2024-06-26T04:59:18.5277444Z Entering 'third_party/eigen' 2024-06-26T04:59:18.5320265Z Entering 'third_party/fbgemm' 2024-06-26T04:59:18.5360625Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T04:59:18.5400073Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T04:59:18.5440528Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T04:59:18.5486226Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T04:59:18.5526070Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T04:59:18.5566920Z Entering 'third_party/flatbuffers' 2024-06-26T04:59:18.5610737Z Entering 'third_party/fmt' 2024-06-26T04:59:18.5651623Z Entering 'third_party/foxi' 2024-06-26T04:59:18.5692283Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T04:59:18.5733831Z Entering 'third_party/gloo' 2024-06-26T04:59:18.5775295Z Entering 'third_party/googletest' 2024-06-26T04:59:18.5817021Z Entering 'third_party/ideep' 2024-06-26T04:59:18.5856630Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T04:59:18.5906520Z Entering 'third_party/ittapi' 2024-06-26T04:59:18.5951223Z Entering 'third_party/kineto' 2024-06-26T04:59:18.5993983Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T04:59:18.6036212Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T04:59:18.6079469Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T04:59:18.6122779Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T04:59:18.6165725Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T04:59:18.6207417Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T04:59:18.6250367Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T04:59:18.6292058Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T04:59:18.6334670Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T04:59:18.6377512Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T04:59:18.6420679Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T04:59:18.6461854Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T04:59:18.6505734Z Entering 'third_party/mimalloc' 2024-06-26T04:59:18.6547648Z Entering 'third_party/nccl/nccl' 2024-06-26T04:59:18.6589222Z Entering 'third_party/nlohmann' 2024-06-26T04:59:18.6631248Z Entering 'third_party/onnx' 2024-06-26T04:59:18.6689327Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T04:59:18.6730290Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T04:59:18.6773253Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T04:59:18.6816667Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T04:59:18.6857232Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T04:59:18.6896903Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T04:59:18.6936905Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T04:59:18.6978105Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T04:59:18.7017803Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T04:59:18.7057481Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T04:59:18.7099150Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T04:59:18.7142997Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T04:59:18.7185111Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T04:59:18.7246774Z Entering 'third_party/pocketfft' 2024-06-26T04:59:18.7289005Z Entering 'third_party/protobuf' 2024-06-26T04:59:18.7336337Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T04:59:18.7377714Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T04:59:18.7420108Z Entering 'third_party/psimd' 2024-06-26T04:59:18.7462524Z Entering 'third_party/pthreadpool' 2024-06-26T04:59:18.7505293Z Entering 'third_party/pybind11' 2024-06-26T04:59:18.7548326Z Entering 'third_party/python-peachpy' 2024-06-26T04:59:18.7589847Z Entering 'third_party/sleef' 2024-06-26T04:59:18.7632749Z Entering 'third_party/tensorpipe' 2024-06-26T04:59:18.7675278Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T04:59:18.7715813Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T04:59:18.7757292Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T04:59:18.7797859Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T04:59:18.7836862Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T04:59:18.7893503Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-06-26T04:59:18.8145834Z Entering 'android/libs/fbjni' 2024-06-26T04:59:18.8187292Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-06-26T04:59:18.8202862Z Entering 'third_party/FP16' 2024-06-26T04:59:18.8242421Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-06-26T04:59:18.8256913Z Entering 'third_party/FXdiv' 2024-06-26T04:59:18.8296212Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-06-26T04:59:18.8310417Z Entering 'third_party/NNPACK' 2024-06-26T04:59:18.8349059Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-06-26T04:59:18.8364318Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T04:59:18.8402108Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-06-26T04:59:18.8418306Z Entering 'third_party/XNNPACK' 2024-06-26T04:59:18.8458022Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-06-26T04:59:18.8489165Z Entering 'third_party/benchmark' 2024-06-26T04:59:18.8530680Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-06-26T04:59:18.8545525Z Entering 'third_party/cpp-httplib' 2024-06-26T04:59:18.8584580Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-06-26T04:59:18.8597899Z Entering 'third_party/cpuinfo' 2024-06-26T04:59:18.8637081Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T04:59:18.8652935Z Entering 'third_party/cudnn_frontend' 2024-06-26T04:59:18.8692863Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-06-26T04:59:18.8706950Z Entering 'third_party/cutlass' 2024-06-26T04:59:18.8747343Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-06-26T04:59:18.8769972Z Entering 'third_party/eigen' 2024-06-26T04:59:18.8811099Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-06-26T04:59:18.8827274Z Entering 'third_party/fbgemm' 2024-06-26T04:59:18.8867359Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-06-26T04:59:18.8881458Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T04:59:18.8920131Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-06-26T04:59:18.8934527Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T04:59:18.8974545Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-06-26T04:59:18.8989006Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T04:59:18.9027857Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-06-26T04:59:18.9049140Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T04:59:18.9089246Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-06-26T04:59:18.9104039Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T04:59:18.9142658Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-06-26T04:59:18.9157419Z Entering 'third_party/flatbuffers' 2024-06-26T04:59:18.9197688Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-06-26T04:59:18.9215439Z Entering 'third_party/fmt' 2024-06-26T04:59:18.9254571Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-06-26T04:59:18.9268429Z Entering 'third_party/foxi' 2024-06-26T04:59:18.9307687Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-06-26T04:59:18.9321252Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T04:59:18.9359920Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-06-26T04:59:18.9374530Z Entering 'third_party/gloo' 2024-06-26T04:59:18.9415834Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-06-26T04:59:18.9430553Z Entering 'third_party/googletest' 2024-06-26T04:59:18.9470335Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-06-26T04:59:18.9486229Z Entering 'third_party/ideep' 2024-06-26T04:59:18.9523814Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-06-26T04:59:18.9538671Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T04:59:18.9577729Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-06-26T04:59:18.9599582Z Entering 'third_party/ittapi' 2024-06-26T04:59:18.9638123Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-06-26T04:59:18.9653857Z Entering 'third_party/kineto' 2024-06-26T04:59:18.9694027Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-06-26T04:59:18.9708522Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T04:59:18.9748586Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-06-26T04:59:18.9762351Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T04:59:18.9801205Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-06-26T04:59:18.9817065Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T04:59:18.9855232Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-06-26T04:59:18.9869425Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T04:59:18.9908549Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-06-26T04:59:18.9922946Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T04:59:18.9961695Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-06-26T04:59:18.9974687Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T04:59:19.0014078Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-06-26T04:59:19.0030623Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T04:59:19.0069524Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-06-26T04:59:19.0084295Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T04:59:19.0123256Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-06-26T04:59:19.0137979Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T04:59:19.0177008Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-06-26T04:59:19.0193067Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T04:59:19.0233164Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-06-26T04:59:19.0249859Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T04:59:19.0288520Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-06-26T04:59:19.0303145Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T04:59:19.0341494Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-06-26T04:59:19.0357836Z Entering 'third_party/mimalloc' 2024-06-26T04:59:19.0396545Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-06-26T04:59:19.0412216Z Entering 'third_party/nccl/nccl' 2024-06-26T04:59:19.0451683Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-06-26T04:59:19.0466200Z Entering 'third_party/nlohmann' 2024-06-26T04:59:19.0505923Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-06-26T04:59:19.0521266Z Entering 'third_party/onnx' 2024-06-26T04:59:19.0560902Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-06-26T04:59:19.0591486Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T04:59:19.0630638Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-06-26T04:59:19.0646635Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T04:59:19.0684362Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-06-26T04:59:19.0701717Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T04:59:19.0741208Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-06-26T04:59:19.0758042Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T04:59:19.0796240Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-06-26T04:59:19.0812256Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T04:59:19.0851257Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-06-26T04:59:19.0864972Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T04:59:19.0903190Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-06-26T04:59:19.0917467Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T04:59:19.0956098Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-06-26T04:59:19.0971825Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T04:59:19.1011005Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-06-26T04:59:19.1024329Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T04:59:19.1063937Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-06-26T04:59:19.1077755Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T04:59:19.1116277Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-06-26T04:59:19.1130586Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T04:59:19.1170103Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-06-26T04:59:19.1186197Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T04:59:19.1224635Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-06-26T04:59:19.1239829Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T04:59:19.1279684Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-06-26T04:59:19.1313734Z Entering 'third_party/pocketfft' 2024-06-26T04:59:19.1353619Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-06-26T04:59:19.1369213Z Entering 'third_party/protobuf' 2024-06-26T04:59:19.1408708Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-06-26T04:59:19.1426552Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T04:59:19.1467640Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-06-26T04:59:19.1482391Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T04:59:19.1520451Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-06-26T04:59:19.1536991Z Entering 'third_party/psimd' 2024-06-26T04:59:19.1575454Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-06-26T04:59:19.1589479Z Entering 'third_party/pthreadpool' 2024-06-26T04:59:19.1628326Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-06-26T04:59:19.1642578Z Entering 'third_party/pybind11' 2024-06-26T04:59:19.1681445Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-06-26T04:59:19.1696285Z Entering 'third_party/python-peachpy' 2024-06-26T04:59:19.1735608Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-06-26T04:59:19.1750383Z Entering 'third_party/sleef' 2024-06-26T04:59:19.1788225Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-06-26T04:59:19.1803550Z Entering 'third_party/tensorpipe' 2024-06-26T04:59:19.1841400Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-06-26T04:59:19.1855957Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T04:59:19.1894697Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-06-26T04:59:19.1909123Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T04:59:19.1947495Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-06-26T04:59:19.1960994Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T04:59:19.1999393Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-06-26T04:59:19.2014072Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T04:59:19.2051775Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-06-26T04:59:19.2065643Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T04:59:19.2103885Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-06-26T04:59:19.2808893Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-06-26T04:59:19.3070195Z Entering 'android/libs/fbjni' 2024-06-26T04:59:19.3105209Z Entering 'third_party/FP16' 2024-06-26T04:59:19.3139052Z Entering 'third_party/FXdiv' 2024-06-26T04:59:19.3173456Z Entering 'third_party/NNPACK' 2024-06-26T04:59:19.3208496Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T04:59:19.3241902Z Entering 'third_party/XNNPACK' 2024-06-26T04:59:19.3293773Z Entering 'third_party/benchmark' 2024-06-26T04:59:19.3328310Z Entering 'third_party/cpp-httplib' 2024-06-26T04:59:19.3361820Z Entering 'third_party/cpuinfo' 2024-06-26T04:59:19.3395896Z Entering 'third_party/cudnn_frontend' 2024-06-26T04:59:19.3430216Z Entering 'third_party/cutlass' 2024-06-26T04:59:19.3472338Z Entering 'third_party/eigen' 2024-06-26T04:59:19.3508248Z Entering 'third_party/fbgemm' 2024-06-26T04:59:19.3543554Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T04:59:19.3576527Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T04:59:19.3610375Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T04:59:19.3649886Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T04:59:19.3683011Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T04:59:19.3717194Z Entering 'third_party/flatbuffers' 2024-06-26T04:59:19.3753150Z Entering 'third_party/fmt' 2024-06-26T04:59:19.3786700Z Entering 'third_party/foxi' 2024-06-26T04:59:19.3819906Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T04:59:19.3855149Z Entering 'third_party/gloo' 2024-06-26T04:59:19.3889959Z Entering 'third_party/googletest' 2024-06-26T04:59:19.3924529Z Entering 'third_party/ideep' 2024-06-26T04:59:19.3957579Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T04:59:19.3998751Z Entering 'third_party/ittapi' 2024-06-26T04:59:19.4032453Z Entering 'third_party/kineto' 2024-06-26T04:59:19.4067403Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T04:59:19.4100966Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T04:59:19.4136286Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T04:59:19.4169834Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T04:59:19.4203654Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T04:59:19.4236951Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T04:59:19.4271937Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T04:59:19.4305819Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T04:59:19.4340658Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T04:59:19.4375380Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T04:59:19.4412013Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T04:59:19.4446229Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T04:59:19.4480670Z Entering 'third_party/mimalloc' 2024-06-26T04:59:19.4516046Z Entering 'third_party/nccl/nccl' 2024-06-26T04:59:19.4549543Z Entering 'third_party/nlohmann' 2024-06-26T04:59:19.4584537Z Entering 'third_party/onnx' 2024-06-26T04:59:19.4633580Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T04:59:19.4667544Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T04:59:19.4703180Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T04:59:19.4738363Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T04:59:19.4770871Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T04:59:19.4804412Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T04:59:19.4837461Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T04:59:19.4871339Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T04:59:19.4904235Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T04:59:19.4936228Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T04:59:19.4968650Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T04:59:19.5002692Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T04:59:19.5037224Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T04:59:19.5090545Z Entering 'third_party/pocketfft' 2024-06-26T04:59:19.5124217Z Entering 'third_party/protobuf' 2024-06-26T04:59:19.5160672Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T04:59:19.5194140Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T04:59:19.5228878Z Entering 'third_party/psimd' 2024-06-26T04:59:19.5262849Z Entering 'third_party/pthreadpool' 2024-06-26T04:59:19.5296602Z Entering 'third_party/pybind11' 2024-06-26T04:59:19.5331270Z Entering 'third_party/python-peachpy' 2024-06-26T04:59:19.5364380Z Entering 'third_party/sleef' 2024-06-26T04:59:19.5397705Z Entering 'third_party/tensorpipe' 2024-06-26T04:59:19.5431199Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T04:59:19.5465029Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T04:59:19.5497832Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T04:59:19.5531580Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T04:59:19.5564731Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T04:59:19.5613487Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-06-26T04:59:19.5868010Z Entering 'android/libs/fbjni' 2024-06-26T04:59:19.5901430Z Entering 'third_party/FP16' 2024-06-26T04:59:19.5935867Z Entering 'third_party/FXdiv' 2024-06-26T04:59:19.5969771Z Entering 'third_party/NNPACK' 2024-06-26T04:59:19.6003381Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T04:59:19.6036971Z Entering 'third_party/XNNPACK' 2024-06-26T04:59:19.6088208Z Entering 'third_party/benchmark' 2024-06-26T04:59:19.6122769Z Entering 'third_party/cpp-httplib' 2024-06-26T04:59:19.6156495Z Entering 'third_party/cpuinfo' 2024-06-26T04:59:19.6190599Z Entering 'third_party/cudnn_frontend' 2024-06-26T04:59:19.6225664Z Entering 'third_party/cutlass' 2024-06-26T04:59:19.6269044Z Entering 'third_party/eigen' 2024-06-26T04:59:19.6305225Z Entering 'third_party/fbgemm' 2024-06-26T04:59:19.6339996Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T04:59:19.6373539Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T04:59:19.6406614Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T04:59:19.6445998Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T04:59:19.6477457Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T04:59:19.6512590Z Entering 'third_party/flatbuffers' 2024-06-26T04:59:19.6550070Z Entering 'third_party/fmt' 2024-06-26T04:59:19.6586303Z Entering 'third_party/foxi' 2024-06-26T04:59:19.6619912Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T04:59:19.6654485Z Entering 'third_party/gloo' 2024-06-26T04:59:19.6688951Z Entering 'third_party/googletest' 2024-06-26T04:59:19.6722717Z Entering 'third_party/ideep' 2024-06-26T04:59:19.6755998Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T04:59:19.6796678Z Entering 'third_party/ittapi' 2024-06-26T04:59:19.6829818Z Entering 'third_party/kineto' 2024-06-26T04:59:19.6862911Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T04:59:19.6896533Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T04:59:19.6930834Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T04:59:19.6963563Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T04:59:19.6996784Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T04:59:19.7028818Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T04:59:19.7062904Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T04:59:19.7096708Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T04:59:19.7130577Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T04:59:19.7163736Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T04:59:19.7198057Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T04:59:19.7231325Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T04:59:19.7265577Z Entering 'third_party/mimalloc' 2024-06-26T04:59:19.7301625Z Entering 'third_party/nccl/nccl' 2024-06-26T04:59:19.7335577Z Entering 'third_party/nlohmann' 2024-06-26T04:59:19.7370684Z Entering 'third_party/onnx' 2024-06-26T04:59:19.7419351Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T04:59:19.7453886Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T04:59:19.7489022Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T04:59:19.7524912Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T04:59:19.7556571Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T04:59:19.7589691Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T04:59:19.7622942Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T04:59:19.7655757Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T04:59:19.7688520Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T04:59:19.7720229Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T04:59:19.7751490Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T04:59:19.7786266Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T04:59:19.7821100Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T04:59:19.7874153Z Entering 'third_party/pocketfft' 2024-06-26T04:59:19.7908947Z Entering 'third_party/protobuf' 2024-06-26T04:59:19.7945551Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T04:59:19.7977529Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T04:59:19.8012077Z Entering 'third_party/psimd' 2024-06-26T04:59:19.8044618Z Entering 'third_party/pthreadpool' 2024-06-26T04:59:19.8078559Z Entering 'third_party/pybind11' 2024-06-26T04:59:19.8112225Z Entering 'third_party/python-peachpy' 2024-06-26T04:59:19.8145123Z Entering 'third_party/sleef' 2024-06-26T04:59:19.8178840Z Entering 'third_party/tensorpipe' 2024-06-26T04:59:19.8211496Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T04:59:19.8244292Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T04:59:19.8276293Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T04:59:19.8308844Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T04:59:19.8342107Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T04:59:19.8387416Z ##[endgroup] 2024-06-26T04:59:19.8423911Z [command]/usr/bin/git log -1 --format='%H' 2024-06-26T04:59:19.8450005Z 'b8c4c54d347aa776934c60784e35936878ef18dc' 2024-06-26T04:59:19.8637990Z Prepare all required actions 2024-06-26T04:59:19.8638503Z Getting action download info 2024-06-26T04:59:19.9937424Z ##[group]Run ./.github/actions/setup-linux 2024-06-26T04:59:19.9937888Z env: 2024-06-26T04:59:19.9938197Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:59:19.9938560Z ##[endgroup] 2024-06-26T04:59:20.0003030Z ##[group]Run set -euo pipefail 2024-06-26T04:59:20.0003659Z set -euo pipefail 2024-06-26T04:59:20.0004226Z function get_ec2_metadata() { 2024-06-26T04:59:20.0004799Z  # Pulled from instance metadata endpoint for EC2 2024-06-26T04:59:20.0005754Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-06-26T04:59:20.0006575Z  category=$1 2024-06-26T04:59:20.0007130Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-06-26T04:59:20.0007798Z  runner_name_str=i-0b079ed883f442af4 2024-06-26T04:59:20.0008324Z  if [[ -f /.inarc ]]; then 2024-06-26T04:59:20.0008835Z  echo "ARC Runner, no info on ec2 metadata" 2024-06-26T04:59:20.0009433Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-06-26T04:59:20.0010171Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-06-26T04:59:20.0010819Z  else 2024-06-26T04:59:20.0011326Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-06-26T04:59:20.0011946Z  fi 2024-06-26T04:59:20.0012254Z } 2024-06-26T04:59:20.0012625Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-06-26T04:59:20.0013244Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-06-26T04:59:20.0013936Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-06-26T04:59:20.0014527Z echo "system info $(uname -a)" 2024-06-26T04:59:20.0022141Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T04:59:20.0022677Z env: 2024-06-26T04:59:20.0022977Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:59:20.0023349Z ##[endgroup] 2024-06-26T04:59:20.0103597Z ami-id: ami-0ce0c36d7a00b20e2 2024-06-26T04:59:20.0155203Z instance-id: i-0b079ed883f442af4 2024-06-26T04:59:20.0204966Z instance-type: c5.2xlarge 2024-06-26T04:59:20.0212194Z system info Linux ip-10-0-7-176.ec2.internal 4.14.336-257.562.amzn2.x86_64 #1 SMP Sat Feb 24 09:50:35 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-06-26T04:59:20.0234288Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T04:59:20.0235217Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-06-26T04:59:20.0242742Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T04:59:20.0243227Z env: 2024-06-26T04:59:20.0243466Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:59:20.0243788Z ##[endgroup] 2024-06-26T04:59:20.0331395Z ##[group]Run if systemctl is-active --quiet docker; then 2024-06-26T04:59:20.0331994Z if systemctl is-active --quiet docker; then 2024-06-26T04:59:20.0332503Z  echo "Docker daemon is running..."; 2024-06-26T04:59:20.0333181Z else 2024-06-26T04:59:20.0333654Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-06-26T04:59:20.0334229Z fi 2024-06-26T04:59:20.0341073Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T04:59:20.0341566Z env: 2024-06-26T04:59:20.0341815Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:59:20.0342128Z ##[endgroup] 2024-06-26T04:59:20.0425029Z Docker daemon is running... 2024-06-26T04:59:20.0470625Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T04:59:20.0471171Z with: 2024-06-26T04:59:20.0471414Z shell: bash 2024-06-26T04:59:20.0471673Z timeout_minutes: 5 2024-06-26T04:59:20.0472050Z max_attempts: 3 2024-06-26T04:59:20.0472343Z retry_wait_seconds: 30 2024-06-26T04:59:20.0473934Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-06-26T04:59:20.0475438Z polling_interval_seconds: 1 2024-06-26T04:59:20.0475796Z warning_on_retry: true 2024-06-26T04:59:20.0476192Z continue_on_error: false 2024-06-26T04:59:20.0476602Z env: 2024-06-26T04:59:20.0476962Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:59:20.0477297Z AWS_RETRY_MODE: standard 2024-06-26T04:59:20.0477607Z AWS_MAX_ATTEMPTS: 5 2024-06-26T04:59:20.0477993Z AWS_DEFAULT_REGION: us-east-1 2024-06-26T04:59:20.0478349Z ##[endgroup] 2024-06-26T04:59:21.2459953Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T04:59:21.2460886Z Configure a credential helper to remove this warning. See 2024-06-26T04:59:21.2461769Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T04:59:21.2462347Z 2024-06-26T04:59:21.2462499Z Login Succeeded 2024-06-26T04:59:22.1000483Z Command completed after 1 attempt(s). 2024-06-26T04:59:22.1045042Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T04:59:22.1045760Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T04:59:22.1046406Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-06-26T04:59:22.1053895Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T04:59:22.1054364Z env: 2024-06-26T04:59:22.1054618Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:59:22.1054947Z ##[endgroup] 2024-06-26T04:59:22.1125146Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T04:59:22.1125919Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T04:59:22.1126510Z # shellcheck disable=SC2046 2024-06-26T04:59:22.1126959Z docker stop $(docker ps -q) || true 2024-06-26T04:59:22.1127428Z # Prune all of the docker images 2024-06-26T04:59:22.1127867Z docker system prune -af 2024-06-26T04:59:22.1134904Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T04:59:22.1135377Z env: 2024-06-26T04:59:22.1135634Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:59:22.1135960Z ##[endgroup] 2024-06-26T04:59:22.1388421Z "docker stop" requires at least 1 argument. 2024-06-26T04:59:22.1389584Z See 'docker stop --help'. 2024-06-26T04:59:22.1390082Z 2024-06-26T04:59:22.1390462Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-06-26T04:59:22.1391063Z 2024-06-26T04:59:22.1391310Z Stop one or more running containers 2024-06-26T04:59:22.1527242Z Total reclaimed space: 0B 2024-06-26T04:59:22.1559347Z ##[group]Run set +e 2024-06-26T04:59:22.1559738Z set +e 2024-06-26T04:59:22.1560130Z set -x 2024-06-26T04:59:22.1560493Z  2024-06-26T04:59:22.1560839Z PT_DOMAIN=download.pytorch.org 2024-06-26T04:59:22.1561682Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-06-26T04:59:22.1562824Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-06-26T04:59:22.1563632Z # one is returned at random 2024-06-26T04:59:22.1564428Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-06-26T04:59:22.1564982Z  2024-06-26T04:59:22.1565334Z if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T04:59:22.1565977Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-06-26T04:59:22.1566787Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-06-26T04:59:22.1567379Z  2024-06-26T04:59:22.1567726Z  if [ -z "${RESOLVED_IP}" ]; then 2024-06-26T04:59:22.1568293Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-06-26T04:59:22.1568842Z  exit 1 2024-06-26T04:59:22.1569180Z  fi 2024-06-26T04:59:22.1569478Z fi 2024-06-26T04:59:22.1569905Z  2024-06-26T04:59:22.1570293Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-06-26T04:59:22.1570830Z  # Clean up any old records first 2024-06-26T04:59:22.1571371Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-06-26T04:59:22.1571854Z fi 2024-06-26T04:59:22.1572143Z  2024-06-26T04:59:22.1572600Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-06-26T04:59:22.1573274Z cat /etc/hosts 2024-06-26T04:59:22.1580781Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T04:59:22.1581319Z env: 2024-06-26T04:59:22.1581620Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:59:22.1581978Z ##[endgroup] 2024-06-26T04:59:22.1602753Z + PT_DOMAIN=download.pytorch.org 2024-06-26T04:59:22.1607745Z ++ dig -4 +short download.pytorch.org 2024-06-26T04:59:22.1608227Z ++ tail -n1 2024-06-26T04:59:22.2352407Z + RESOLVED_IP=18.160.10.28 2024-06-26T04:59:22.2353515Z + '[' -z 18.160.10.28 ']' 2024-06-26T04:59:22.2354300Z + grep -r download.pytorch.org /etc/hosts 2024-06-26T04:59:22.2361979Z 18.160.10.28 download.pytorch.org 2024-06-26T04:59:22.2362733Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-06-26T04:59:22.2468293Z + echo '18.160.10.28 download.pytorch.org' 2024-06-26T04:59:22.2469163Z + sudo tee -a /etc/hosts 2024-06-26T04:59:22.2892217Z 18.160.10.28 download.pytorch.org 2024-06-26T04:59:22.2904911Z + cat /etc/hosts 2024-06-26T04:59:22.2910713Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-06-26T04:59:22.2925712Z ::1 localhost6 localhost6.localdomain6 2024-06-26T04:59:22.2926285Z 18.160.10.28 download.pytorch.org 2024-06-26T04:59:22.3093130Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-06-26T04:59:22.3093773Z with: 2024-06-26T04:59:22.3094703Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:22.3095797Z docker-build-dir: .ci/docker 2024-06-26T04:59:22.3096197Z working-directory: . 2024-06-26T04:59:22.3096682Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T04:59:22.3097250Z force-push: false 2024-06-26T04:59:22.3097572Z env: 2024-06-26T04:59:22.3097854Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:59:22.3098224Z ##[endgroup] 2024-06-26T04:59:22.3119277Z ##[group]Run set -ex 2024-06-26T04:59:22.3119668Z set -ex 2024-06-26T04:59:22.3119993Z  2024-06-26T04:59:22.3120578Z # If the docker build directory or the build script doesn't exist, the action will 2024-06-26T04:59:22.3121738Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-06-26T04:59:22.3122585Z # job could then download the pre-built image as usual 2024-06-26T04:59:22.3123373Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-06-26T04:59:22.3124225Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-06-26T04:59:22.3124900Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T04:59:22.3125485Z  2024-06-26T04:59:22.3126034Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-06-26T04:59:22.3126702Z  exit 0 2024-06-26T04:59:22.3127020Z else 2024-06-26T04:59:22.3127422Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-06-26T04:59:22.3127902Z fi 2024-06-26T04:59:22.3128191Z  2024-06-26T04:59:22.3128690Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-06-26T04:59:22.3129605Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-06-26T04:59:22.3130424Z  # use it as it is, but first let's extract the tag 2024-06-26T04:59:22.3131173Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-06-26T04:59:22.3131945Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T04:59:22.3132679Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-06-26T04:59:22.3133263Z else 2024-06-26T04:59:22.3133721Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-06-26T04:59:22.3134425Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T04:59:22.3135502Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-06-26T04:59:22.3136317Z fi 2024-06-26T04:59:22.3143793Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T04:59:22.3144312Z env: 2024-06-26T04:59:22.3144615Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:59:22.3144993Z REPO_NAME: pytorch 2024-06-26T04:59:22.3145952Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:22.3147016Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T04:59:22.3147547Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T04:59:22.3148105Z ##[endgroup] 2024-06-26T04:59:22.3169990Z + [[ ! -d .ci/docker ]] 2024-06-26T04:59:22.3170409Z + [[ ! -f .ci/docker/build.sh ]] 2024-06-26T04:59:22.3170823Z + echo skip=false 2024-06-26T04:59:22.3172485Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-06-26T04:59:22.3176286Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:22.3177439Z ++ awk -F '[:,]' '{print $2}' 2024-06-26T04:59:22.3236440Z + DOCKER_TAG=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:22.3237189Z + echo docker-tag=91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:22.3238721Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:22.3268553Z ##[group]Run set +e 2024-06-26T04:59:22.3268942Z set +e 2024-06-26T04:59:22.3269274Z set -x 2024-06-26T04:59:22.3269578Z  2024-06-26T04:59:22.3269881Z login() { 2024-06-26T04:59:22.3270591Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T04:59:22.3271357Z } 2024-06-26T04:59:22.3271642Z  2024-06-26T04:59:22.3272026Z retry () { 2024-06-26T04:59:22.3272436Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T04:59:22.3272916Z } 2024-06-26T04:59:22.3273216Z  2024-06-26T04:59:22.3273540Z retry login "${DOCKER_REGISTRY}" 2024-06-26T04:59:22.3273987Z  2024-06-26T04:59:22.3274487Z # Check if image already exists, if it does then skip building it 2024-06-26T04:59:22.3275238Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-06-26T04:59:22.3275765Z  exit 0 2024-06-26T04:59:22.3276087Z fi 2024-06-26T04:59:22.3276382Z  2024-06-26T04:59:22.3276902Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-06-26T04:59:22.3277779Z # be empty. The default action would be to continue rebuild the image 2024-06-26T04:59:22.3278573Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-06-26T04:59:22.3279271Z  # if we're on the base branch then use the parent commit 2024-06-26T04:59:22.3279903Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-06-26T04:59:22.3280362Z else 2024-06-26T04:59:22.3280842Z  # otherwise we're on a PR, so use the most recent base commit 2024-06-26T04:59:22.3281569Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-06-26T04:59:22.3282109Z fi 2024-06-26T04:59:22.3282397Z  2024-06-26T04:59:22.3282747Z if [[ -z "${MERGE_BASE}" ]]; then 2024-06-26T04:59:22.3283285Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T04:59:22.3283779Z  2024-06-26T04:59:22.3284659Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-06-26T04:59:22.3285636Z  exit 0 2024-06-26T04:59:22.3285966Z fi 2024-06-26T04:59:22.3286254Z  2024-06-26T04:59:22.3286723Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-06-26T04:59:22.3287769Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-06-26T04:59:22.3288626Z  exit 1 2024-06-26T04:59:22.3288961Z fi 2024-06-26T04:59:22.3289264Z  2024-06-26T04:59:22.3289785Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-06-26T04:59:22.3290801Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-06-26T04:59:22.3291725Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-06-26T04:59:22.3292752Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-06-26T04:59:22.3293912Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-06-26T04:59:22.3294608Z fi 2024-06-26T04:59:22.3294907Z  2024-06-26T04:59:22.3295270Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-06-26T04:59:22.3302766Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T04:59:22.3303299Z env: 2024-06-26T04:59:22.3303589Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:59:22.3303981Z DOCKER_BUILD_DIR: .ci/docker 2024-06-26T04:59:22.3304475Z BASE_REVISION: 4b9c9a9cc9c9283380a011310ba180c105c3dcb9 2024-06-26T04:59:22.3305590Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:22.3306688Z DOCKER_TAG: 91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:22.3307329Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T04:59:22.3307883Z ##[endgroup] 2024-06-26T04:59:22.3364825Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T04:59:22.3365639Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T04:59:22.3366205Z + aws ecr get-login-password --region us-east-1 2024-06-26T04:59:22.3366960Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T04:59:22.7243370Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T04:59:22.7244589Z Configure a credential helper to remove this warning. See 2024-06-26T04:59:22.7245442Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T04:59:22.7245988Z 2024-06-26T04:59:22.7246104Z Login Succeeded 2024-06-26T04:59:22.7254089Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:22.9109571Z { 2024-06-26T04:59:22.9110119Z "schemaVersion": 2, 2024-06-26T04:59:22.9110700Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-06-26T04:59:22.9111500Z "config": { 2024-06-26T04:59:22.9112247Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-06-26T04:59:22.9113093Z "size": 44015, 2024-06-26T04:59:22.9113976Z "digest": "sha256:14402cfc353dbfa2ba1fa30f87c7ccf0e77a56d529c503e4a803c15ef3448962" 2024-06-26T04:59:22.9114988Z }, 2024-06-26T04:59:22.9115220Z "layers": [ 2024-06-26T04:59:22.9115468Z { 2024-06-26T04:59:22.9115878Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9116434Z "size": 30439283, 2024-06-26T04:59:22.9116988Z "digest": "sha256:2ec76a50fe7c8d5db9ec25590b9217e14e3920513c6e7b5be55db72a16b55f7c" 2024-06-26T04:59:22.9117602Z }, 2024-06-26T04:59:22.9117822Z { 2024-06-26T04:59:22.9118239Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9118781Z "size": 1823, 2024-06-26T04:59:22.9119314Z "digest": "sha256:94cf876dd15336a1bb9c68efe3c199eb689974408d8d00307e6ee7c39375add8" 2024-06-26T04:59:22.9120203Z }, 2024-06-26T04:59:22.9120415Z { 2024-06-26T04:59:22.9120832Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9121383Z "size": 313225743, 2024-06-26T04:59:22.9121978Z "digest": "sha256:844eff13770e3d7fcbb311cc117c4644a656c61edacc026aa3a4167da5c78eac" 2024-06-26T04:59:22.9122632Z }, 2024-06-26T04:59:22.9122854Z { 2024-06-26T04:59:22.9123272Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9123811Z "size": 804, 2024-06-26T04:59:22.9124532Z "digest": "sha256:0ebe1846f17bf25dd364aeebc97768509fda90340864aaaf66f18435862b5d00" 2024-06-26T04:59:22.9125158Z }, 2024-06-26T04:59:22.9125372Z { 2024-06-26T04:59:22.9125799Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9126357Z "size": 106, 2024-06-26T04:59:22.9126892Z "digest": "sha256:aab1810da80ffabe097875b02464bde8a4a3db91be2d3bbbe39ccdec81a36b2d" 2024-06-26T04:59:22.9127541Z }, 2024-06-26T04:59:22.9127763Z { 2024-06-26T04:59:22.9128169Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9128723Z "size": 545, 2024-06-26T04:59:22.9129387Z "digest": "sha256:f7a111ade856ed3b73afaea5a82f7717419e8215a08d5289c3bca760c5369c95" 2024-06-26T04:59:22.9130231Z + exit 0 2024-06-26T04:59:22.9130467Z }, 2024-06-26T04:59:22.9130687Z { 2024-06-26T04:59:22.9131090Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9131642Z "size": 1239, 2024-06-26T04:59:22.9132191Z "digest": "sha256:4521770e6b41adbbb2db5cff2e8b83a06b08caed66486df8ee6884ffa2625bf4" 2024-06-26T04:59:22.9132809Z }, 2024-06-26T04:59:22.9133029Z { 2024-06-26T04:59:22.9133444Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9133975Z "size": 484, 2024-06-26T04:59:22.9134511Z "digest": "sha256:deb25f4e99d07c3a82d02a1c780af7d3298b55434c9998f9fffd83dc97dc1077" 2024-06-26T04:59:22.9135143Z }, 2024-06-26T04:59:22.9135352Z { 2024-06-26T04:59:22.9135770Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9136322Z "size": 110320620, 2024-06-26T04:59:22.9136849Z "digest": "sha256:77b895c04768539cb607a2e35c08521bc97388f138643c1a2ff93365693ab2df" 2024-06-26T04:59:22.9137448Z }, 2024-06-26T04:59:22.9137669Z { 2024-06-26T04:59:22.9138074Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9138629Z "size": 3684, 2024-06-26T04:59:22.9139143Z "digest": "sha256:cf312583a2f806332647a141d0517260a0e45439a169dd92274ead5979f81acd" 2024-06-26T04:59:22.9139728Z }, 2024-06-26T04:59:22.9139951Z { 2024-06-26T04:59:22.9140364Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9140897Z "size": 1905, 2024-06-26T04:59:22.9141428Z "digest": "sha256:a180d8bca39de20a44602dd9e54099934988e98b0659cf7770cedfff341da77c" 2024-06-26T04:59:22.9142050Z }, 2024-06-26T04:59:22.9142261Z { 2024-06-26T04:59:22.9142681Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9143230Z "size": 702, 2024-06-26T04:59:22.9143745Z "digest": "sha256:29852573ae91b4bb7572c70c54c36797bb4bf172fa77beafdb68067f29ec5b8d" 2024-06-26T04:59:22.9144357Z }, 2024-06-26T04:59:22.9144579Z { 2024-06-26T04:59:22.9144984Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9145538Z "size": 2627181312, 2024-06-26T04:59:22.9146103Z "digest": "sha256:f32c36ef12b53151147e93ec42d342771bfad4fec642edd6a9b9e5f8326a3e77" 2024-06-26T04:59:22.9146708Z }, 2024-06-26T04:59:22.9146930Z { 2024-06-26T04:59:22.9147347Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9147880Z "size": 32, 2024-06-26T04:59:22.9148414Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T04:59:22.9149041Z }, 2024-06-26T04:59:22.9149249Z { 2024-06-26T04:59:22.9149791Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9150345Z "size": 381, 2024-06-26T04:59:22.9150865Z "digest": "sha256:a29555b55aa4775dfe3656a896ba2b217487eaf2d243c87a539bccf2d37f39c7" 2024-06-26T04:59:22.9151487Z }, 2024-06-26T04:59:22.9151717Z { 2024-06-26T04:59:22.9152227Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9152781Z "size": 220641, 2024-06-26T04:59:22.9153356Z "digest": "sha256:45d450910c56999dfec09dcf8bfbfd86105905281c6a3857b7031faf255f04b2" 2024-06-26T04:59:22.9154027Z }, 2024-06-26T04:59:22.9154247Z { 2024-06-26T04:59:22.9154666Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9155197Z "size": 232, 2024-06-26T04:59:22.9155719Z "digest": "sha256:b2dd7129275db5ceb6d3257a7e3f75711661020063a2d19616eb48365c88c9c0" 2024-06-26T04:59:22.9156325Z }, 2024-06-26T04:59:22.9156534Z { 2024-06-26T04:59:22.9156950Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9157505Z "size": 2840574, 2024-06-26T04:59:22.9158033Z "digest": "sha256:efd2aa485229a0666afce3f0965ec82713c94d4a3dba5008856716f4559c7e39" 2024-06-26T04:59:22.9158654Z }, 2024-06-26T04:59:22.9158874Z { 2024-06-26T04:59:22.9159391Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9159946Z "size": 1990, 2024-06-26T04:59:22.9160479Z "digest": "sha256:0661ae1bd510185c6176c43a2090f7fd4ad6a625ec04c827eb6149a37ae08030" 2024-06-26T04:59:22.9161079Z }, 2024-06-26T04:59:22.9161304Z { 2024-06-26T04:59:22.9161730Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9162265Z "size": 105, 2024-06-26T04:59:22.9162793Z "digest": "sha256:8e2059e3bfeba4fb9459ce01583b11644de40185026ff7c0fb758190b3dfabe1" 2024-06-26T04:59:22.9163525Z }, 2024-06-26T04:59:22.9163735Z { 2024-06-26T04:59:22.9164361Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9164979Z "size": 566, 2024-06-26T04:59:22.9165504Z "digest": "sha256:6542c0ef474e272d1a5f15b04ffd0ec7f1f536ba784ff139c58399b1c4248149" 2024-06-26T04:59:22.9166130Z }, 2024-06-26T04:59:22.9166357Z { 2024-06-26T04:59:22.9166767Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9167310Z "size": 32, 2024-06-26T04:59:22.9167845Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T04:59:22.9168465Z }, 2024-06-26T04:59:22.9168672Z { 2024-06-26T04:59:22.9169089Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9169621Z "size": 104, 2024-06-26T04:59:22.9170158Z "digest": "sha256:861b401f0ab5d6012e70dbd6ef8e5a9d5bce4689c87a635433ae7b7175ee0e16" 2024-06-26T04:59:22.9170780Z }, 2024-06-26T04:59:22.9170986Z { 2024-06-26T04:59:22.9171404Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9171949Z "size": 504, 2024-06-26T04:59:22.9172471Z "digest": "sha256:4ca9a23a73e71088cb9f0177151c841288f42938b799bf25c96e290463ba3a6c" 2024-06-26T04:59:22.9173062Z }, 2024-06-26T04:59:22.9173281Z { 2024-06-26T04:59:22.9173689Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9174245Z "size": 101109809, 2024-06-26T04:59:22.9174776Z "digest": "sha256:f6485591ba901699c177749168e8607338acdd109d977bfb67202fcf28b55842" 2024-06-26T04:59:22.9175375Z }, 2024-06-26T04:59:22.9175584Z { 2024-06-26T04:59:22.9176000Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9176541Z "size": 109, 2024-06-26T04:59:22.9177059Z "digest": "sha256:142b23cc83b8eba936d2663f8572979cd09d4d8963c1366cb87349132bbd83f3" 2024-06-26T04:59:22.9177671Z }, 2024-06-26T04:59:22.9177892Z { 2024-06-26T04:59:22.9178295Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9178842Z "size": 490, 2024-06-26T04:59:22.9179383Z "digest": "sha256:9f65d81b1aabd595f2419d33e6306b8474c43b28decbd11ca44ed37279010ff9" 2024-06-26T04:59:22.9180133Z }, 2024-06-26T04:59:22.9180357Z { 2024-06-26T04:59:22.9180778Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9181318Z "size": 336, 2024-06-26T04:59:22.9181853Z "digest": "sha256:260c90c40e762e888fcdd2a02024508f9af42ce14d46dec48e0ff34ab2cb0894" 2024-06-26T04:59:22.9182467Z }, 2024-06-26T04:59:22.9182678Z { 2024-06-26T04:59:22.9183092Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9183641Z "size": 103, 2024-06-26T04:59:22.9184145Z "digest": "sha256:b667c8726b8f413f7960e0813c9dbc93930426059a6ad0342a358d386e9c233a" 2024-06-26T04:59:22.9184747Z }, 2024-06-26T04:59:22.9184967Z { 2024-06-26T04:59:22.9185371Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9185917Z "size": 1474, 2024-06-26T04:59:22.9186448Z "digest": "sha256:9e75cc8302f3c403885811054b10f69b21cc2a4fafafac214b2b659cc9036959" 2024-06-26T04:59:22.9187054Z }, 2024-06-26T04:59:22.9187272Z { 2024-06-26T04:59:22.9187686Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9188229Z "size": 465317655, 2024-06-26T04:59:22.9188863Z "digest": "sha256:30ff689874b32b30ec30d1f7081d1f14a4e7bf0f5ed8108647f192c7ac6e7188" 2024-06-26T04:59:22.9189479Z }, 2024-06-26T04:59:22.9189691Z { 2024-06-26T04:59:22.9190110Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9190660Z "size": 160, 2024-06-26T04:59:22.9191180Z "digest": "sha256:e75ff799212e700eddfd97d957cf9000a5cb79001cdd2d1d258016fc76b38b04" 2024-06-26T04:59:22.9191888Z }, 2024-06-26T04:59:22.9192116Z { 2024-06-26T04:59:22.9192524Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9193075Z "size": 2211, 2024-06-26T04:59:22.9193615Z "digest": "sha256:319065f88688c9273cff5b495dba47e4a017feda1bf861b6b6ff997cc11ce29f" 2024-06-26T04:59:22.9194226Z }, 2024-06-26T04:59:22.9194450Z { 2024-06-26T04:59:22.9194868Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9195407Z "size": 152, 2024-06-26T04:59:22.9195954Z "digest": "sha256:a26ad32bbc0fe830ffb52ed978b971cb4c492c489ac4d80c97f472952eb6c529" 2024-06-26T04:59:22.9196580Z }, 2024-06-26T04:59:22.9196788Z { 2024-06-26T04:59:22.9197207Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9197753Z "size": 648, 2024-06-26T04:59:22.9198265Z "digest": "sha256:1347adb36d32b24469eee98e2d1e94305ea395ab49d7616021705edc304c9524" 2024-06-26T04:59:22.9198876Z }, 2024-06-26T04:59:22.9199096Z { 2024-06-26T04:59:22.9199498Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9200039Z "size": 32, 2024-06-26T04:59:22.9200572Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T04:59:22.9201181Z }, 2024-06-26T04:59:22.9201405Z { 2024-06-26T04:59:22.9201818Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9202351Z "size": 162, 2024-06-26T04:59:22.9202902Z "digest": "sha256:8b2e5004bcbab0b9c3b20aaec3aa2f3f482f8cbdc8e488d1e92a24646aebe851" 2024-06-26T04:59:22.9203544Z }, 2024-06-26T04:59:22.9203753Z { 2024-06-26T04:59:22.9204301Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9204858Z "size": 109, 2024-06-26T04:59:22.9205469Z "digest": "sha256:3ad8bad3700657dbadc73b57a9f435ab604b52af2dbb2960fec90101703c2c7c" 2024-06-26T04:59:22.9206092Z }, 2024-06-26T04:59:22.9206313Z { 2024-06-26T04:59:22.9206717Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9207262Z "size": 102, 2024-06-26T04:59:22.9207795Z "digest": "sha256:d335836e780559f38cf4cacaf5fa72a7fd7d9c4cfc1286978413360de54cd94d" 2024-06-26T04:59:22.9208402Z }, 2024-06-26T04:59:22.9208623Z { 2024-06-26T04:59:22.9209039Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9209704Z "size": 422, 2024-06-26T04:59:22.9210246Z "digest": "sha256:03e3083ae7df397a9661184e7bc0ca6bbc2e1f6854833a31fcaadd40bfeec4b2" 2024-06-26T04:59:22.9210866Z }, 2024-06-26T04:59:22.9211072Z { 2024-06-26T04:59:22.9211494Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9212050Z "size": 32, 2024-06-26T04:59:22.9212569Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T04:59:22.9213191Z }, 2024-06-26T04:59:22.9213410Z { 2024-06-26T04:59:22.9213811Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9214355Z "size": 111, 2024-06-26T04:59:22.9214897Z "digest": "sha256:97cc85cb7c4e8e1a4f40eff1a6443c10d929b413dbc2a2a69bbf4b04005e17c2" 2024-06-26T04:59:22.9215515Z }, 2024-06-26T04:59:22.9215815Z { 2024-06-26T04:59:22.9216396Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9217230Z "size": 475, 2024-06-26T04:59:22.9218156Z "digest": "sha256:d490220e4f6bc3cd20e5a46b873043c8f47ffaa718f81ef42a8e45704fc2e96c" 2024-06-26T04:59:22.9219276Z }, 2024-06-26T04:59:22.9219659Z { 2024-06-26T04:59:22.9220521Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9221094Z "size": 32, 2024-06-26T04:59:22.9221624Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T04:59:22.9222254Z }, 2024-06-26T04:59:22.9222474Z { 2024-06-26T04:59:22.9222881Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9223432Z "size": 112, 2024-06-26T04:59:22.9223943Z "digest": "sha256:04d0190bc97a111068f095e0631193a6ef218093d621a406151a284e12049861" 2024-06-26T04:59:22.9224530Z }, 2024-06-26T04:59:22.9224757Z { 2024-06-26T04:59:22.9225183Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9225716Z "size": 563, 2024-06-26T04:59:22.9226259Z "digest": "sha256:04d47328c7bd2d0987f8357de16bdf9224a7629db9fc046318942fbcee32c568" 2024-06-26T04:59:22.9226872Z }, 2024-06-26T04:59:22.9227083Z { 2024-06-26T04:59:22.9227501Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9228054Z "size": 43163951, 2024-06-26T04:59:22.9228599Z "digest": "sha256:69a3d4e29db0e7d7bd5833503abd51a03f51e2ee4dc4c5317f572a7e9ab4edda" 2024-06-26T04:59:22.9229225Z }, 2024-06-26T04:59:22.9229447Z { 2024-06-26T04:59:22.9229852Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9230401Z "size": 106, 2024-06-26T04:59:22.9230939Z "digest": "sha256:0231c909d672d2e40b9deccf81dc97dddfdd0530b2b072782fe8d2b29957aa54" 2024-06-26T04:59:22.9231549Z }, 2024-06-26T04:59:22.9231845Z { 2024-06-26T04:59:22.9232270Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9232812Z "size": 295, 2024-06-26T04:59:22.9233356Z "digest": "sha256:7af7703fd0c99d4c22c6c420c3ca9ace4646234bce2dbbc8e4a685276bd96a6b" 2024-06-26T04:59:22.9233989Z }, 2024-06-26T04:59:22.9234203Z { 2024-06-26T04:59:22.9234617Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9235203Z "size": 32, 2024-06-26T04:59:22.9235728Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T04:59:22.9236361Z }, 2024-06-26T04:59:22.9236582Z { 2024-06-26T04:59:22.9236989Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9237539Z "size": 106, 2024-06-26T04:59:22.9238052Z "digest": "sha256:027ac6996c62a5c9772f3885939b2ca1508d6342a46be3344c2eb474c0032903" 2024-06-26T04:59:22.9238644Z }, 2024-06-26T04:59:22.9238867Z { 2024-06-26T04:59:22.9239284Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9239820Z "size": 425, 2024-06-26T04:59:22.9240351Z "digest": "sha256:74a68590fdfcf793ff1148af97440cab24ff53112786632dfd31af3861d7d695" 2024-06-26T04:59:22.9241065Z }, 2024-06-26T04:59:22.9241275Z { 2024-06-26T04:59:22.9241697Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9242255Z "size": 19308854, 2024-06-26T04:59:22.9242798Z "digest": "sha256:2e867e2465c63943a6f568b03f82de03caa582dbcfb66bb8364ca6af33bf687e" 2024-06-26T04:59:22.9243413Z }, 2024-06-26T04:59:22.9243636Z { 2024-06-26T04:59:22.9244172Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9244730Z "size": 108, 2024-06-26T04:59:22.9245267Z "digest": "sha256:f51a2243c1bf3305a28ce63dcb4e55761f73abf2dc0172c25921e352761dc92e" 2024-06-26T04:59:22.9245873Z }, 2024-06-26T04:59:22.9246104Z { 2024-06-26T04:59:22.9246525Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9247066Z "size": 441, 2024-06-26T04:59:22.9247614Z "digest": "sha256:dccc1e11707a46b3ff2271ca32b160514fed1bc6a7e80dd316aba8fe7ef834dd" 2024-06-26T04:59:22.9248351Z }, 2024-06-26T04:59:22.9248560Z { 2024-06-26T04:59:22.9248976Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9249524Z "size": 702, 2024-06-26T04:59:22.9250150Z "digest": "sha256:29852573ae91b4bb7572c70c54c36797bb4bf172fa77beafdb68067f29ec5b8d" 2024-06-26T04:59:22.9250774Z }, 2024-06-26T04:59:22.9251006Z { 2024-06-26T04:59:22.9251413Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9251972Z "size": 142, 2024-06-26T04:59:22.9252517Z "digest": "sha256:fa84b128b2d8bd6d79b5e2f01e4cf834d1b1d9439d42893cd7a3f2c0f9ecd923" 2024-06-26T04:59:22.9253134Z }, 2024-06-26T04:59:22.9253356Z { 2024-06-26T04:59:22.9253770Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9254304Z "size": 135, 2024-06-26T04:59:22.9254831Z "digest": "sha256:79c5c1b4789a8061392ed80f7df55a0cf8b3f578f569025973dafc832d7bbb1b" 2024-06-26T04:59:22.9255447Z }, 2024-06-26T04:59:22.9255661Z { 2024-06-26T04:59:22.9256076Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9256618Z "size": 32, 2024-06-26T04:59:22.9257139Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T04:59:22.9257772Z }, 2024-06-26T04:59:22.9257993Z { 2024-06-26T04:59:22.9258397Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9258941Z "size": 191, 2024-06-26T04:59:22.9259470Z "digest": "sha256:585d93983ffa398cf27a97f4939a0120bc1a733f4d8b1de24126a3a1a8cca278" 2024-06-26T04:59:22.9260073Z }, 2024-06-26T04:59:22.9260296Z { 2024-06-26T04:59:22.9260712Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9261249Z "size": 1212, 2024-06-26T04:59:22.9261770Z "digest": "sha256:0e2757b3e947531b56521a63a38f869487bdaf8d434517834db7ac70c3fb4453" 2024-06-26T04:59:22.9262376Z }, 2024-06-26T04:59:22.9262584Z { 2024-06-26T04:59:22.9263005Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9263553Z "size": 702, 2024-06-26T04:59:22.9264070Z "digest": "sha256:29852573ae91b4bb7572c70c54c36797bb4bf172fa77beafdb68067f29ec5b8d" 2024-06-26T04:59:22.9264684Z }, 2024-06-26T04:59:22.9264910Z { 2024-06-26T04:59:22.9265315Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9265862Z "size": 138, 2024-06-26T04:59:22.9266382Z "digest": "sha256:017702dc3bb9e5484c17f9128a44348f83a3b8f07e0b1562e0b715ba365dcded" 2024-06-26T04:59:22.9266977Z }, 2024-06-26T04:59:22.9267200Z { 2024-06-26T04:59:22.9267620Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9268159Z "size": 1838061418, 2024-06-26T04:59:22.9268718Z "digest": "sha256:26b6032d8e72bb67615c5d0a83ba3e01d208fc78c20cace534c5bff7bbe41a59" 2024-06-26T04:59:22.9269337Z }, 2024-06-26T04:59:22.9269546Z { 2024-06-26T04:59:22.9269961Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9270631Z "size": 152, 2024-06-26T04:59:22.9271168Z "digest": "sha256:7eaba9afcf96af27f87045a1f17dbbbc83d88c47208139b136a80edf27b5b1f8" 2024-06-26T04:59:22.9271898Z }, 2024-06-26T04:59:22.9272131Z { 2024-06-26T04:59:22.9272545Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9273102Z "size": 849, 2024-06-26T04:59:22.9273643Z "digest": "sha256:4e1273fff111ab63c6def740c9fe3f405168e73ca96505750ec9560e571f877f" 2024-06-26T04:59:22.9274262Z }, 2024-06-26T04:59:22.9274473Z { 2024-06-26T04:59:22.9274890Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9275438Z "size": 702, 2024-06-26T04:59:22.9275949Z "digest": "sha256:29852573ae91b4bb7572c70c54c36797bb4bf172fa77beafdb68067f29ec5b8d" 2024-06-26T04:59:22.9276561Z }, 2024-06-26T04:59:22.9276785Z { 2024-06-26T04:59:22.9277190Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9277737Z "size": 139, 2024-06-26T04:59:22.9278265Z "digest": "sha256:095aaa194ad58bed818d7624f54a8462d2368b1d7a9177dbf1d622d158e3def8" 2024-06-26T04:59:22.9278871Z }, 2024-06-26T04:59:22.9279091Z { 2024-06-26T04:59:22.9279667Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9280208Z "size": 32, 2024-06-26T04:59:22.9280748Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T04:59:22.9281375Z }, 2024-06-26T04:59:22.9281586Z { 2024-06-26T04:59:22.9282009Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9282558Z "size": 158, 2024-06-26T04:59:22.9283075Z "digest": "sha256:e69bf94e94cb18ac9d20e45d50582f15473ce61f6365e247b604cbcd61f4ddcc" 2024-06-26T04:59:22.9283694Z }, 2024-06-26T04:59:22.9283914Z { 2024-06-26T04:59:22.9284452Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9285006Z "size": 909, 2024-06-26T04:59:22.9285554Z "digest": "sha256:5b282fb06022aaeb8cb00baf6758ebd1fc50213fa08837e6ebd8367648f0cc03" 2024-06-26T04:59:22.9286171Z }, 2024-06-26T04:59:22.9286399Z { 2024-06-26T04:59:22.9286814Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9287349Z "size": 702, 2024-06-26T04:59:22.9287877Z "digest": "sha256:29852573ae91b4bb7572c70c54c36797bb4bf172fa77beafdb68067f29ec5b8d" 2024-06-26T04:59:22.9288492Z }, 2024-06-26T04:59:22.9288701Z { 2024-06-26T04:59:22.9289119Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9289668Z "size": 135, 2024-06-26T04:59:22.9290165Z "digest": "sha256:26170244c99466a119be20c01e62387ea3245e38bca465127b3e1b5ca8683314" 2024-06-26T04:59:22.9290764Z }, 2024-06-26T04:59:22.9290984Z { 2024-06-26T04:59:22.9291390Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9291935Z "size": 32, 2024-06-26T04:59:22.9292473Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T04:59:22.9293091Z }, 2024-06-26T04:59:22.9293310Z { 2024-06-26T04:59:22.9293726Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9294261Z "size": 156, 2024-06-26T04:59:22.9294794Z "digest": "sha256:45c7879febc5552f1bb9b59969d326cc799eeb1d3e31f7d425d1d02259551a4e" 2024-06-26T04:59:22.9295412Z }, 2024-06-26T04:59:22.9295619Z { 2024-06-26T04:59:22.9296034Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9296581Z "size": 1578, 2024-06-26T04:59:22.9297116Z "digest": "sha256:f5dd85f916a3414b09232ecaddf806dfe81f2d7fba5ed61373edd807df99a262" 2024-06-26T04:59:22.9297747Z }, 2024-06-26T04:59:22.9297968Z { 2024-06-26T04:59:22.9298373Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9298918Z "size": 32, 2024-06-26T04:59:22.9299452Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T04:59:22.9300171Z }, 2024-06-26T04:59:22.9300398Z { 2024-06-26T04:59:22.9300819Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9301358Z "size": 128, 2024-06-26T04:59:22.9301907Z "digest": "sha256:00cfe467539eb476bd66ff9c6bf2bea8601f073a4ca45ecb476c3e37ed2b2d85" 2024-06-26T04:59:22.9302540Z }, 2024-06-26T04:59:22.9302747Z { 2024-06-26T04:59:22.9303163Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9303711Z "size": 379, 2024-06-26T04:59:22.9304207Z "digest": "sha256:761b58c4a7323f0fa29d02368b839830353760466f08d0a73f0d5e505d8aa44c" 2024-06-26T04:59:22.9304809Z }, 2024-06-26T04:59:22.9305032Z { 2024-06-26T04:59:22.9305434Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9305977Z "size": 32, 2024-06-26T04:59:22.9306509Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T04:59:22.9307122Z }, 2024-06-26T04:59:22.9307346Z { 2024-06-26T04:59:22.9307764Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9308296Z "size": 104, 2024-06-26T04:59:22.9308819Z "digest": "sha256:ce57725e9a4c3eadb62246f231e0f27e0535973a6f881557be320048c4552f2e" 2024-06-26T04:59:22.9309512Z }, 2024-06-26T04:59:22.9309725Z { 2024-06-26T04:59:22.9310147Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9310703Z "size": 1841, 2024-06-26T04:59:22.9311233Z "digest": "sha256:ecdd20263166bfcc5e8e25f125b05534fe8c754c82c8ea6aa5a60d733cbbf694" 2024-06-26T04:59:22.9311962Z }, 2024-06-26T04:59:22.9312190Z { 2024-06-26T04:59:22.9312598Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9313153Z "size": 7529630, 2024-06-26T04:59:22.9313709Z "digest": "sha256:f6b630e16b35df33b7f2ff25c6a3213d58d8203e1dd6c5dcfc41e96ecf26f646" 2024-06-26T04:59:22.9314325Z }, 2024-06-26T04:59:22.9314548Z { 2024-06-26T04:59:22.9314967Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9315501Z "size": 106, 2024-06-26T04:59:22.9316029Z "digest": "sha256:05462c0bf1655f0b876d0358a0831b29d5b2eaa9f4e99f9aa6be8f1a338b5d00" 2024-06-26T04:59:22.9316642Z }, 2024-06-26T04:59:22.9316853Z { 2024-06-26T04:59:22.9317266Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9317813Z "size": 165, 2024-06-26T04:59:22.9318329Z "digest": "sha256:2861e30c4419c8e6e8cbf7f943a4d0a897ecdc504f4b1383a30de43a569798bb" 2024-06-26T04:59:22.9318944Z }, 2024-06-26T04:59:22.9319166Z { 2024-06-26T04:59:22.9319568Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9320115Z "size": 7944, 2024-06-26T04:59:22.9320652Z "digest": "sha256:ec250a1df2c075bc9fcf2595945b010296b0b32813ed9a7d93c6769f3ffe6543" 2024-06-26T04:59:22.9321257Z }, 2024-06-26T04:59:22.9321479Z { 2024-06-26T04:59:22.9321894Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9322437Z "size": 8064, 2024-06-26T04:59:22.9322968Z "digest": "sha256:994a842bbc8796fcf2e4d295f0ba513718cc7c33f63ef0648913e1ec011fbe23" 2024-06-26T04:59:22.9323588Z }, 2024-06-26T04:59:22.9323794Z { 2024-06-26T04:59:22.9324340Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9324893Z "size": 303, 2024-06-26T04:59:22.9325410Z "digest": "sha256:851f80854d0a39831957ee6397e18c35b8d4fff54dccfe373a2f10ec410a4460" 2024-06-26T04:59:22.9326030Z }, 2024-06-26T04:59:22.9326257Z { 2024-06-26T04:59:22.9326665Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9327216Z "size": 32, 2024-06-26T04:59:22.9327747Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T04:59:22.9328359Z }, 2024-06-26T04:59:22.9328577Z { 2024-06-26T04:59:22.9328996Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9329641Z "size": 108, 2024-06-26T04:59:22.9330197Z "digest": "sha256:3d9bae1abce26e7cd0093200b96c41cdb09be5472c60ebaeca918e78cb6fe6b5" 2024-06-26T04:59:22.9330836Z }, 2024-06-26T04:59:22.9331046Z { 2024-06-26T04:59:22.9331471Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9332027Z "size": 54145774, 2024-06-26T04:59:22.9332587Z "digest": "sha256:a80b78a9c4343fe6bbfc10a01ea35ad79aee2ce2ba85a79bbad4b88dca8f9a98" 2024-06-26T04:59:22.9333230Z }, 2024-06-26T04:59:22.9333453Z { 2024-06-26T04:59:22.9333860Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-06-26T04:59:22.9334409Z "size": 32, 2024-06-26T04:59:22.9334941Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-06-26T04:59:22.9335554Z } 2024-06-26T04:59:22.9335773Z ] 2024-06-26T04:59:22.9335990Z } 2024-06-26T04:59:22.9443352Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T04:59:22.9444052Z tag=${ECR_DOCKER_IMAGE##*/} 2024-06-26T04:59:22.9444627Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-06-26T04:59:22.9452319Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T04:59:22.9452812Z env: 2024-06-26T04:59:22.9453063Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:59:22.9454007Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:22.9454987Z ##[endgroup] 2024-06-26T04:59:22.9477851Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-jammy-py3.8-gcc11-91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:22.9530074Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-06-26T04:59:22.9530639Z with: 2024-06-26T04:59:22.9531504Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:22.9532628Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T04:59:22.9533135Z env: 2024-06-26T04:59:22.9533385Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:59:22.9533706Z ##[endgroup] 2024-06-26T04:59:22.9551186Z ##[group]Run set -x 2024-06-26T04:59:22.9551496Z set -x 2024-06-26T04:59:22.9551867Z set +e 2024-06-26T04:59:22.9552137Z  2024-06-26T04:59:22.9552375Z login() { 2024-06-26T04:59:22.9553047Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-06-26T04:59:22.9553827Z } 2024-06-26T04:59:22.9554062Z  2024-06-26T04:59:22.9554360Z retry () { 2024-06-26T04:59:22.9554710Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-06-26T04:59:22.9555142Z } 2024-06-26T04:59:22.9555392Z  2024-06-26T04:59:22.9555666Z retry login "${DOCKER_REGISTRY}" 2024-06-26T04:59:22.9556062Z  2024-06-26T04:59:22.9556305Z set -e 2024-06-26T04:59:22.9556744Z # ignore output since only exit code is used for conditional 2024-06-26T04:59:22.9557453Z # only pull docker image if it's not available locally 2024-06-26T04:59:22.9558214Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-06-26T04:59:22.9558900Z  retry docker pull "${DOCKER_IMAGE}" 2024-06-26T04:59:22.9559304Z fi 2024-06-26T04:59:22.9566630Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T04:59:22.9567115Z env: 2024-06-26T04:59:22.9567352Z GIT_DEFAULT_BRANCH: main 2024-06-26T04:59:22.9568299Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:22.9569415Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T04:59:22.9569911Z ##[endgroup] 2024-06-26T04:59:22.9590172Z + set +e 2024-06-26T04:59:22.9590895Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T04:59:22.9591663Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T04:59:22.9593852Z + aws ecr get-login-password --region us-east-1 2024-06-26T04:59:22.9595329Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-06-26T04:59:23.3500824Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-06-26T04:59:23.3502119Z Configure a credential helper to remove this warning. See 2024-06-26T04:59:23.3502987Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-06-26T04:59:23.3503524Z 2024-06-26T04:59:23.3503639Z Login Succeeded 2024-06-26T04:59:23.3510547Z + set -e 2024-06-26T04:59:23.3512081Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:23.3631923Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:23.3633715Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T04:59:23.5789389Z 91382da70d5719cd7007b6b80b71d2f48398f6b7: Pulling from pytorch/pytorch-linux-jammy-py3.8-gcc11 2024-06-26T04:59:23.5798360Z 2ec76a50fe7c: Pulling fs layer 2024-06-26T04:59:23.5800869Z 94cf876dd153: Pulling fs layer 2024-06-26T04:59:23.5801974Z 844eff13770e: Pulling fs layer 2024-06-26T04:59:23.5802904Z 0ebe1846f17b: Pulling fs layer 2024-06-26T04:59:23.5818458Z aab1810da80f: Pulling fs layer 2024-06-26T04:59:23.5819654Z f7a111ade856: Pulling fs layer 2024-06-26T04:59:23.5820337Z 4521770e6b41: Pulling fs layer 2024-06-26T04:59:23.5820953Z deb25f4e99d0: Pulling fs layer 2024-06-26T04:59:23.5821369Z 77b895c04768: Pulling fs layer 2024-06-26T04:59:23.5821729Z cf312583a2f8: Pulling fs layer 2024-06-26T04:59:23.5822582Z a180d8bca39d: Pulling fs layer 2024-06-26T04:59:23.5823180Z 29852573ae91: Pulling fs layer 2024-06-26T04:59:23.5823559Z f32c36ef12b5: Pulling fs layer 2024-06-26T04:59:23.5823904Z aab1810da80f: Waiting 2024-06-26T04:59:23.5824205Z 4f4fb700ef54: Pulling fs layer 2024-06-26T04:59:23.5824548Z f7a111ade856: Waiting 2024-06-26T04:59:23.5824862Z a29555b55aa4: Pulling fs layer 2024-06-26T04:59:23.5825193Z deb25f4e99d0: Waiting 2024-06-26T04:59:23.5825492Z 77b895c04768: Waiting 2024-06-26T04:59:23.5825796Z 45d450910c56: Pulling fs layer 2024-06-26T04:59:23.5826121Z 4521770e6b41: Waiting 2024-06-26T04:59:23.5826429Z b2dd7129275d: Pulling fs layer 2024-06-26T04:59:23.5826770Z cf312583a2f8: Waiting 2024-06-26T04:59:23.5827067Z f32c36ef12b5: Waiting 2024-06-26T04:59:23.5827362Z a180d8bca39d: Waiting 2024-06-26T04:59:23.5827654Z 29852573ae91: Waiting 2024-06-26T04:59:23.5827988Z efd2aa485229: Pulling fs layer 2024-06-26T04:59:23.5828348Z 0661ae1bd510: Pulling fs layer 2024-06-26T04:59:23.5828708Z 8e2059e3bfeb: Pulling fs layer 2024-06-26T04:59:23.5829075Z 4f4fb700ef54: Waiting 2024-06-26T04:59:23.5829400Z a29555b55aa4: Waiting 2024-06-26T04:59:23.5829696Z 45d450910c56: Waiting 2024-06-26T04:59:23.5829993Z 6542c0ef474e: Pulling fs layer 2024-06-26T04:59:23.5830421Z b2dd7129275d: Waiting 2024-06-26T04:59:23.5830717Z efd2aa485229: Waiting 2024-06-26T04:59:23.5830997Z 0661ae1bd510: Waiting 2024-06-26T04:59:23.5831312Z 861b401f0ab5: Pulling fs layer 2024-06-26T04:59:23.5831667Z 4ca9a23a73e7: Pulling fs layer 2024-06-26T04:59:23.5832074Z f6485591ba90: Pulling fs layer 2024-06-26T04:59:23.5832434Z 142b23cc83b8: Pulling fs layer 2024-06-26T04:59:23.5832917Z 9f65d81b1aab: Pulling fs layer 2024-06-26T04:59:23.5833256Z 260c90c40e76: Pulling fs layer 2024-06-26T04:59:23.5833619Z b667c8726b8f: Pulling fs layer 2024-06-26T04:59:23.5833957Z 0ebe1846f17b: Waiting 2024-06-26T04:59:23.5834248Z 9e75cc8302f3: Pulling fs layer 2024-06-26T04:59:23.5834579Z f6485591ba90: Waiting 2024-06-26T04:59:23.5834862Z 142b23cc83b8: Waiting 2024-06-26T04:59:23.5835154Z 30ff689874b3: Pulling fs layer 2024-06-26T04:59:23.5835687Z e75ff799212e: Pulling fs layer 2024-06-26T04:59:23.5836198Z 4ca9a23a73e7: Waiting 2024-06-26T04:59:23.5836489Z 319065f88688: Pulling fs layer 2024-06-26T04:59:23.5836827Z b667c8726b8f: Waiting 2024-06-26T04:59:23.5837141Z a26ad32bbc0f: Pulling fs layer 2024-06-26T04:59:23.5837470Z 30ff689874b3: Waiting 2024-06-26T04:59:23.5837765Z 9e75cc8302f3: Waiting 2024-06-26T04:59:23.5838057Z 1347adb36d32: Pulling fs layer 2024-06-26T04:59:23.5838398Z e75ff799212e: Waiting 2024-06-26T04:59:23.5838707Z 8b2e5004bcba: Pulling fs layer 2024-06-26T04:59:23.5839057Z 3ad8bad37006: Pulling fs layer 2024-06-26T04:59:23.5839402Z a26ad32bbc0f: Waiting 2024-06-26T04:59:23.5839704Z 1347adb36d32: Waiting 2024-06-26T04:59:23.5840004Z d335836e7805: Pulling fs layer 2024-06-26T04:59:23.5840382Z 8b2e5004bcba: Waiting 2024-06-26T04:59:23.5840677Z 3ad8bad37006: Waiting 2024-06-26T04:59:23.5841008Z 03e3083ae7df: Pulling fs layer 2024-06-26T04:59:23.5841398Z d335836e7805: Waiting 2024-06-26T04:59:23.5841703Z 97cc85cb7c4e: Pulling fs layer 2024-06-26T04:59:23.5842051Z d490220e4f6b: Pulling fs layer 2024-06-26T04:59:23.5842393Z 97cc85cb7c4e: Waiting 2024-06-26T04:59:23.5842690Z d490220e4f6b: Waiting 2024-06-26T04:59:23.5842986Z 04d0190bc97a: Pulling fs layer 2024-06-26T04:59:23.5843339Z 04d47328c7bd: Pulling fs layer 2024-06-26T04:59:23.5843678Z 319065f88688: Waiting 2024-06-26T04:59:23.5844169Z 69a3d4e29db0: Pulling fs layer 2024-06-26T04:59:23.5844531Z 0231c909d672: Pulling fs layer 2024-06-26T04:59:23.5844887Z 7af7703fd0c9: Pulling fs layer 2024-06-26T04:59:23.5845211Z 0231c909d672: Waiting 2024-06-26T04:59:23.5845521Z 027ac6996c62: Pulling fs layer 2024-06-26T04:59:23.5845989Z 7af7703fd0c9: Waiting 2024-06-26T04:59:23.5846274Z 04d0190bc97a: Waiting 2024-06-26T04:59:23.5846584Z 74a68590fdfc: Pulling fs layer 2024-06-26T04:59:23.5846929Z 69a3d4e29db0: Waiting 2024-06-26T04:59:23.5847211Z 027ac6996c62: Waiting 2024-06-26T04:59:23.5847515Z 2e867e2465c6: Pulling fs layer 2024-06-26T04:59:23.5847860Z 74a68590fdfc: Waiting 2024-06-26T04:59:23.5848155Z f51a2243c1bf: Pulling fs layer 2024-06-26T04:59:23.5848507Z 04d47328c7bd: Waiting 2024-06-26T04:59:23.5848803Z 2e867e2465c6: Waiting 2024-06-26T04:59:23.5849098Z dccc1e11707a: Pulling fs layer 2024-06-26T04:59:23.5849461Z fa84b128b2d8: Pulling fs layer 2024-06-26T04:59:23.5849815Z 79c5c1b4789a: Pulling fs layer 2024-06-26T04:59:23.5850152Z 585d93983ffa: Pulling fs layer 2024-06-26T04:59:23.5850492Z fa84b128b2d8: Waiting 2024-06-26T04:59:23.5850791Z f51a2243c1bf: Waiting 2024-06-26T04:59:23.5851071Z 79c5c1b4789a: Waiting 2024-06-26T04:59:23.5851378Z 0e2757b3e947: Pulling fs layer 2024-06-26T04:59:23.5851719Z dccc1e11707a: Waiting 2024-06-26T04:59:23.5852019Z 017702dc3bb9: Pulling fs layer 2024-06-26T04:59:23.5852365Z 0e2757b3e947: Waiting 2024-06-26T04:59:23.5852670Z 26b6032d8e72: Pulling fs layer 2024-06-26T04:59:23.5853015Z 7eaba9afcf96: Pulling fs layer 2024-06-26T04:59:23.5853411Z 017702dc3bb9: Waiting 2024-06-26T04:59:23.5853727Z 4e1273fff111: Pulling fs layer 2024-06-26T04:59:23.5854053Z 26b6032d8e72: Waiting 2024-06-26T04:59:23.5854352Z 7eaba9afcf96: Waiting 2024-06-26T04:59:23.5854649Z 4e1273fff111: Waiting 2024-06-26T04:59:23.5854946Z 095aaa194ad5: Pulling fs layer 2024-06-26T04:59:23.5855305Z e69bf94e94cb: Pulling fs layer 2024-06-26T04:59:23.5855663Z 5b282fb06022: Pulling fs layer 2024-06-26T04:59:23.5855988Z 095aaa194ad5: Waiting 2024-06-26T04:59:23.5856286Z e69bf94e94cb: Waiting 2024-06-26T04:59:23.5856648Z 26170244c994: Pulling fs layer 2024-06-26T04:59:23.5857054Z 5b282fb06022: Waiting 2024-06-26T04:59:23.5857408Z 45c7879febc5: Pulling fs layer 2024-06-26T04:59:23.5857866Z f5dd85f916a3: Pulling fs layer 2024-06-26T04:59:23.5858205Z 45c7879febc5: Waiting 2024-06-26T04:59:23.5858547Z 00cfe467539e: Pulling fs layer 2024-06-26T04:59:23.5858899Z 761b58c4a732: Pulling fs layer 2024-06-26T04:59:23.5859264Z f5dd85f916a3: Waiting 2024-06-26T04:59:23.5859561Z 00cfe467539e: Waiting 2024-06-26T04:59:23.5859871Z ce57725e9a4c: Pulling fs layer 2024-06-26T04:59:23.5860198Z 26170244c994: Waiting 2024-06-26T04:59:23.5860505Z ecdd20263166: Pulling fs layer 2024-06-26T04:59:23.5860957Z 761b58c4a732: Waiting 2024-06-26T04:59:23.5861266Z f6b630e16b35: Pulling fs layer 2024-06-26T04:59:23.5861612Z ecdd20263166: Waiting 2024-06-26T04:59:23.5861906Z 05462c0bf165: Pulling fs layer 2024-06-26T04:59:23.5862257Z 2861e30c4419: Pulling fs layer 2024-06-26T04:59:23.5862607Z ec250a1df2c0: Pulling fs layer 2024-06-26T04:59:23.5862950Z 994a842bbc87: Pulling fs layer 2024-06-26T04:59:23.5863308Z 851f80854d0a: Pulling fs layer 2024-06-26T04:59:23.5863670Z 3d9bae1abce2: Pulling fs layer 2024-06-26T04:59:23.5863995Z 851f80854d0a: Waiting 2024-06-26T04:59:23.5864379Z 994a842bbc87: Waiting 2024-06-26T04:59:23.5864690Z a80b78a9c434: Pulling fs layer 2024-06-26T04:59:23.5865020Z 05462c0bf165: Waiting 2024-06-26T04:59:23.5865319Z ec250a1df2c0: Waiting 2024-06-26T04:59:23.5865615Z 2861e30c4419: Waiting 2024-06-26T04:59:23.5865898Z 3d9bae1abce2: Waiting 2024-06-26T04:59:23.5866195Z a80b78a9c434: Waiting 2024-06-26T04:59:23.6572855Z 94cf876dd153: Verifying Checksum 2024-06-26T04:59:23.6573573Z 94cf876dd153: Download complete 2024-06-26T04:59:23.7299653Z 0ebe1846f17b: Verifying Checksum 2024-06-26T04:59:23.7300125Z 0ebe1846f17b: Download complete 2024-06-26T04:59:23.8004611Z aab1810da80f: Verifying Checksum 2024-06-26T04:59:23.8005185Z aab1810da80f: Download complete 2024-06-26T04:59:23.8736644Z f7a111ade856: Verifying Checksum 2024-06-26T04:59:23.8737455Z f7a111ade856: Download complete 2024-06-26T04:59:23.9315249Z 2ec76a50fe7c: Verifying Checksum 2024-06-26T04:59:23.9315885Z 2ec76a50fe7c: Download complete 2024-06-26T04:59:23.9464005Z 4521770e6b41: Download complete 2024-06-26T04:59:23.9983644Z deb25f4e99d0: Verifying Checksum 2024-06-26T04:59:23.9984372Z deb25f4e99d0: Download complete 2024-06-26T04:59:24.0784256Z cf312583a2f8: Verifying Checksum 2024-06-26T04:59:24.0784767Z cf312583a2f8: Download complete 2024-06-26T04:59:24.1485799Z a180d8bca39d: Verifying Checksum 2024-06-26T04:59:24.1486648Z a180d8bca39d: Download complete 2024-06-26T04:59:24.2277099Z 29852573ae91: Verifying Checksum 2024-06-26T04:59:24.2277704Z 29852573ae91: Download complete 2024-06-26T04:59:24.9176091Z 2ec76a50fe7c: Pull complete 2024-06-26T04:59:24.9845603Z 94cf876dd153: Pull complete 2024-06-26T04:59:25.0997363Z 77b895c04768: Verifying Checksum 2024-06-26T04:59:25.1083863Z 77b895c04768: Download complete 2024-06-26T04:59:25.1084735Z 4f4fb700ef54: Verifying Checksum 2024-06-26T04:59:25.1085453Z 4f4fb700ef54: Download complete 2024-06-26T04:59:25.1796305Z a29555b55aa4: Download complete 2024-06-26T04:59:25.2506551Z 45d450910c56: Download complete 2024-06-26T04:59:25.3154075Z b2dd7129275d: Verifying Checksum 2024-06-26T04:59:25.3154620Z b2dd7129275d: Download complete 2024-06-26T04:59:25.4227870Z efd2aa485229: Verifying Checksum 2024-06-26T04:59:25.4228590Z efd2aa485229: Download complete 2024-06-26T04:59:25.4932373Z 0661ae1bd510: Download complete 2024-06-26T04:59:25.5558106Z 8e2059e3bfeb: Verifying Checksum 2024-06-26T04:59:25.5558928Z 8e2059e3bfeb: Download complete 2024-06-26T04:59:25.7091332Z 861b401f0ab5: Verifying Checksum 2024-06-26T04:59:25.7092357Z 861b401f0ab5: Download complete 2024-06-26T04:59:25.7744314Z 4ca9a23a73e7: Download complete 2024-06-26T04:59:26.7642058Z 844eff13770e: Verifying Checksum 2024-06-26T04:59:26.7642849Z 844eff13770e: Download complete 2024-06-26T04:59:26.8426352Z 142b23cc83b8: Verifying Checksum 2024-06-26T04:59:26.8426994Z 142b23cc83b8: Download complete 2024-06-26T04:59:26.8785032Z f6485591ba90: Verifying Checksum 2024-06-26T04:59:26.8785764Z f6485591ba90: Download complete 2024-06-26T04:59:26.9114651Z 9f65d81b1aab: Verifying Checksum 2024-06-26T04:59:26.9115309Z 9f65d81b1aab: Download complete 2024-06-26T04:59:26.9653855Z 260c90c40e76: Download complete 2024-06-26T04:59:26.9880949Z b667c8726b8f: Verifying Checksum 2024-06-26T04:59:27.0397687Z b667c8726b8f: Download complete 2024-06-26T04:59:27.0398364Z 9e75cc8302f3: Verifying Checksum 2024-06-26T04:59:27.0398937Z 9e75cc8302f3: Download complete 2024-06-26T04:59:27.1133554Z e75ff799212e: Verifying Checksum 2024-06-26T04:59:27.1134390Z e75ff799212e: Download complete 2024-06-26T04:59:27.1909191Z 319065f88688: Verifying Checksum 2024-06-26T04:59:27.1909965Z 319065f88688: Download complete 2024-06-26T04:59:27.2597715Z a26ad32bbc0f: Verifying Checksum 2024-06-26T04:59:27.2598400Z a26ad32bbc0f: Download complete 2024-06-26T04:59:27.3358047Z 1347adb36d32: Download complete 2024-06-26T04:59:27.4594049Z 8b2e5004bcba: Download complete 2024-06-26T04:59:27.5187767Z 3ad8bad37006: Verifying Checksum 2024-06-26T04:59:27.5188805Z 3ad8bad37006: Download complete 2024-06-26T04:59:27.5928588Z d335836e7805: Verifying Checksum 2024-06-26T04:59:27.5929404Z d335836e7805: Download complete 2024-06-26T04:59:27.6588209Z 03e3083ae7df: Download complete 2024-06-26T04:59:27.7225491Z 97cc85cb7c4e: Verifying Checksum 2024-06-26T04:59:27.7226387Z 97cc85cb7c4e: Download complete 2024-06-26T04:59:27.8031065Z d490220e4f6b: Download complete 2024-06-26T04:59:27.8744661Z 04d0190bc97a: Verifying Checksum 2024-06-26T04:59:27.9419600Z 04d0190bc97a: Download complete 2024-06-26T04:59:27.9420082Z 04d47328c7bd: Verifying Checksum 2024-06-26T04:59:27.9420468Z 04d47328c7bd: Download complete 2024-06-26T04:59:28.4256947Z 69a3d4e29db0: Verifying Checksum 2024-06-26T04:59:28.4257474Z 69a3d4e29db0: Download complete 2024-06-26T04:59:28.5181225Z 0231c909d672: Verifying Checksum 2024-06-26T04:59:28.5182089Z 0231c909d672: Download complete 2024-06-26T04:59:28.6078605Z 7af7703fd0c9: Download complete 2024-06-26T04:59:28.6838773Z 027ac6996c62: Verifying Checksum 2024-06-26T04:59:28.6839350Z 027ac6996c62: Download complete 2024-06-26T04:59:28.7520966Z 74a68590fdfc: Verifying Checksum 2024-06-26T04:59:28.7521734Z 74a68590fdfc: Download complete 2024-06-26T04:59:29.0277297Z 2e867e2465c6: Verifying Checksum 2024-06-26T04:59:29.0277903Z 2e867e2465c6: Download complete 2024-06-26T04:59:29.0908547Z f51a2243c1bf: Verifying Checksum 2024-06-26T04:59:29.0909460Z f51a2243c1bf: Download complete 2024-06-26T04:59:29.1635993Z dccc1e11707a: Download complete 2024-06-26T04:59:29.2407199Z fa84b128b2d8: Download complete 2024-06-26T04:59:29.3051235Z 79c5c1b4789a: Download complete 2024-06-26T04:59:29.3897569Z 585d93983ffa: Verifying Checksum 2024-06-26T04:59:29.3898243Z 585d93983ffa: Download complete 2024-06-26T04:59:29.4578417Z 0e2757b3e947: Verifying Checksum 2024-06-26T04:59:29.5351536Z 0e2757b3e947: Download complete 2024-06-26T04:59:29.5352301Z 017702dc3bb9: Verifying Checksum 2024-06-26T04:59:29.5352939Z 017702dc3bb9: Download complete 2024-06-26T04:59:31.7060433Z 30ff689874b3: Verifying Checksum 2024-06-26T04:59:31.7061038Z 30ff689874b3: Download complete 2024-06-26T04:59:31.7869583Z 7eaba9afcf96: Verifying Checksum 2024-06-26T04:59:31.7870279Z 7eaba9afcf96: Download complete 2024-06-26T04:59:31.8711023Z 4e1273fff111: Verifying Checksum 2024-06-26T04:59:31.8711556Z 4e1273fff111: Download complete 2024-06-26T04:59:31.9522751Z 095aaa194ad5: Download complete 2024-06-26T04:59:32.0152558Z e69bf94e94cb: Verifying Checksum 2024-06-26T04:59:32.0153329Z e69bf94e94cb: Download complete 2024-06-26T04:59:32.0797666Z 5b282fb06022: Download complete 2024-06-26T04:59:32.1641298Z 26170244c994: Download complete 2024-06-26T04:59:32.2366199Z 45c7879febc5: Verifying Checksum 2024-06-26T04:59:32.2366887Z 45c7879febc5: Download complete 2024-06-26T04:59:32.3094220Z f5dd85f916a3: Verifying Checksum 2024-06-26T04:59:32.3094924Z f5dd85f916a3: Download complete 2024-06-26T04:59:32.3973123Z 00cfe467539e: Download complete 2024-06-26T04:59:32.4868913Z 761b58c4a732: Verifying Checksum 2024-06-26T04:59:32.4869612Z 761b58c4a732: Download complete 2024-06-26T04:59:32.5491938Z ce57725e9a4c: Verifying Checksum 2024-06-26T04:59:32.5492677Z ce57725e9a4c: Download complete 2024-06-26T04:59:32.6246038Z ecdd20263166: Verifying Checksum 2024-06-26T04:59:32.6246589Z ecdd20263166: Download complete 2024-06-26T04:59:32.7441224Z f6b630e16b35: Verifying Checksum 2024-06-26T04:59:32.7441833Z f6b630e16b35: Download complete 2024-06-26T04:59:32.8192412Z 05462c0bf165: Download complete 2024-06-26T04:59:32.8862595Z 2861e30c4419: Download complete 2024-06-26T04:59:32.9535351Z ec250a1df2c0: Verifying Checksum 2024-06-26T04:59:32.9535847Z ec250a1df2c0: Download complete 2024-06-26T04:59:33.0354511Z 994a842bbc87: Download complete 2024-06-26T04:59:33.1047731Z 851f80854d0a: Verifying Checksum 2024-06-26T04:59:33.1048363Z 851f80854d0a: Download complete 2024-06-26T04:59:33.1788663Z 3d9bae1abce2: Download complete 2024-06-26T04:59:33.7662096Z a80b78a9c434: Verifying Checksum 2024-06-26T04:59:33.7662687Z a80b78a9c434: Download complete 2024-06-26T04:59:34.9958301Z 844eff13770e: Pull complete 2024-06-26T04:59:35.2086667Z 0ebe1846f17b: Pull complete 2024-06-26T04:59:35.4370230Z aab1810da80f: Pull complete 2024-06-26T04:59:35.7272998Z f7a111ade856: Pull complete 2024-06-26T04:59:36.1460278Z 4521770e6b41: Pull complete 2024-06-26T04:59:36.4359321Z deb25f4e99d0: Pull complete 2024-06-26T04:59:57.4494331Z 26b6032d8e72: Verifying Checksum 2024-06-26T04:59:57.4496603Z 26b6032d8e72: Download complete 2024-06-26T04:59:58.9321692Z 77b895c04768: Pull complete 2024-06-26T04:59:59.3384678Z cf312583a2f8: Pull complete 2024-06-26T04:59:59.6489487Z a180d8bca39d: Pull complete 2024-06-26T05:00:00.2346427Z 29852573ae91: Pull complete 2024-06-26T05:00:01.8281578Z f32c36ef12b5: Verifying Checksum 2024-06-26T05:00:01.8282057Z f32c36ef12b5: Download complete 2024-06-26T05:00:49.6572244Z f32c36ef12b5: Pull complete 2024-06-26T05:00:49.8646962Z 4f4fb700ef54: Pull complete 2024-06-26T05:00:50.0755141Z a29555b55aa4: Pull complete 2024-06-26T05:00:50.2723030Z 45d450910c56: Pull complete 2024-06-26T05:00:50.3526997Z b2dd7129275d: Pull complete 2024-06-26T05:00:50.5046076Z efd2aa485229: Pull complete 2024-06-26T05:00:50.6483607Z 0661ae1bd510: Pull complete 2024-06-26T05:00:50.8451384Z 8e2059e3bfeb: Pull complete 2024-06-26T05:00:51.0576784Z 6542c0ef474e: Pull complete 2024-06-26T05:00:51.1849067Z 861b401f0ab5: Pull complete 2024-06-26T05:00:51.2345454Z 4ca9a23a73e7: Pull complete 2024-06-26T05:00:53.3501556Z f6485591ba90: Pull complete 2024-06-26T05:00:53.5692941Z 142b23cc83b8: Pull complete 2024-06-26T05:00:53.7467126Z 9f65d81b1aab: Pull complete 2024-06-26T05:00:53.8954712Z 260c90c40e76: Pull complete 2024-06-26T05:00:54.0446135Z b667c8726b8f: Pull complete 2024-06-26T05:00:54.1239261Z 9e75cc8302f3: Pull complete 2024-06-26T05:01:01.1628873Z 30ff689874b3: Pull complete 2024-06-26T05:01:01.3066656Z e75ff799212e: Pull complete 2024-06-26T05:01:01.4256241Z 319065f88688: Pull complete 2024-06-26T05:01:01.5674836Z a26ad32bbc0f: Pull complete 2024-06-26T05:01:01.7626275Z 1347adb36d32: Pull complete 2024-06-26T05:01:01.9791419Z 8b2e5004bcba: Pull complete 2024-06-26T05:01:02.0143801Z 3ad8bad37006: Pull complete 2024-06-26T05:01:02.0713645Z d335836e7805: Pull complete 2024-06-26T05:01:02.1152105Z 03e3083ae7df: Pull complete 2024-06-26T05:01:02.2492583Z 97cc85cb7c4e: Pull complete 2024-06-26T05:01:02.2941150Z d490220e4f6b: Pull complete 2024-06-26T05:01:02.4141800Z 04d0190bc97a: Pull complete 2024-06-26T05:01:02.4881482Z 04d47328c7bd: Pull complete 2024-06-26T05:01:04.1174833Z 69a3d4e29db0: Pull complete 2024-06-26T05:01:04.3103395Z 0231c909d672: Pull complete 2024-06-26T05:01:04.4132952Z 7af7703fd0c9: Pull complete 2024-06-26T05:01:04.8110370Z 027ac6996c62: Pull complete 2024-06-26T05:01:05.0174035Z 74a68590fdfc: Pull complete 2024-06-26T05:01:05.4566282Z 2e867e2465c6: Pull complete 2024-06-26T05:01:05.6506806Z f51a2243c1bf: Pull complete 2024-06-26T05:01:05.7236692Z dccc1e11707a: Pull complete 2024-06-26T05:01:05.8649771Z fa84b128b2d8: Pull complete 2024-06-26T05:01:06.0219986Z 79c5c1b4789a: Pull complete 2024-06-26T05:01:06.2961944Z 585d93983ffa: Pull complete 2024-06-26T05:01:06.4928983Z 0e2757b3e947: Pull complete 2024-06-26T05:01:06.9127829Z 017702dc3bb9: Pull complete 2024-06-26T05:01:36.9926190Z 26b6032d8e72: Pull complete 2024-06-26T05:01:37.3817145Z 7eaba9afcf96: Pull complete 2024-06-26T05:01:37.6217796Z 4e1273fff111: Pull complete 2024-06-26T05:01:37.7888525Z 095aaa194ad5: Pull complete 2024-06-26T05:01:38.1521131Z e69bf94e94cb: Pull complete 2024-06-26T05:01:38.3022577Z 5b282fb06022: Pull complete 2024-06-26T05:01:38.6470957Z 26170244c994: Pull complete 2024-06-26T05:01:38.9153932Z 45c7879febc5: Pull complete 2024-06-26T05:01:39.1079606Z f5dd85f916a3: Pull complete 2024-06-26T05:01:39.4343569Z 00cfe467539e: Pull complete 2024-06-26T05:01:39.6408515Z 761b58c4a732: Pull complete 2024-06-26T05:01:40.0551580Z ce57725e9a4c: Pull complete 2024-06-26T05:01:40.2423288Z ecdd20263166: Pull complete 2024-06-26T05:01:40.5852796Z f6b630e16b35: Pull complete 2024-06-26T05:01:40.7887202Z 05462c0bf165: Pull complete 2024-06-26T05:01:40.9712463Z 2861e30c4419: Pull complete 2024-06-26T05:01:41.1640348Z ec250a1df2c0: Pull complete 2024-06-26T05:01:41.3793369Z 994a842bbc87: Pull complete 2024-06-26T05:01:41.5925038Z 851f80854d0a: Pull complete 2024-06-26T05:01:42.0583385Z 3d9bae1abce2: Pull complete 2024-06-26T05:01:43.8051560Z a80b78a9c434: Pull complete 2024-06-26T05:01:44.1541051Z Digest: sha256:1862f87fe60cbac0efa4720f6f6fe873270a315ff02a9510d48d1480259bc4e9 2024-06-26T05:01:44.1831291Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T05:01:44.2188465Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T05:01:44.2528160Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T05:01:44.2529098Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-06-26T05:01:44.4478877Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T05:01:44.4479388Z env: 2024-06-26T05:01:44.4479647Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:01:44.4479962Z ##[endgroup] 2024-06-26T05:01:44.4600323Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T05:01:44.4601093Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-06-26T05:01:44.4601788Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-06-26T05:01:44.4602456Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-06-26T05:01:44.4610011Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T05:01:44.4610499Z env: 2024-06-26T05:01:44.4610755Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:01:44.4611065Z ##[endgroup] 2024-06-26T05:01:47.9387196Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T05:01:47.9665203Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.7/site-packages (5.9.1) 2024-06-26T05:01:47.9764739Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.7/site-packages (11.525.84) 2024-06-26T05:01:48.1054576Z Prepare all required actions 2024-06-26T05:01:48.1055102Z Getting action download info 2024-06-26T05:01:48.2223416Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-06-26T05:01:48.4128324Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-06-26T05:01:48.5277668Z ##[group]Run ./.github/actions/download-build-artifacts 2024-06-26T05:01:48.5278143Z with: 2024-06-26T05:01:48.5278402Z name: linux-jammy-py3.8-gcc11 2024-06-26T05:01:48.5278777Z s3-bucket: gha-artifacts 2024-06-26T05:01:48.5279095Z env: 2024-06-26T05:01:48.5279532Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:01:48.5279858Z ##[endgroup] 2024-06-26T05:01:48.5333740Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T05:01:48.5334159Z with: 2024-06-26T05:01:48.5334433Z name: linux-jammy-py3.8-gcc11 2024-06-26T05:01:48.5334948Z s3-bucket: gha-artifacts 2024-06-26T05:01:48.5335320Z region: us-east-1 2024-06-26T05:01:48.5335588Z env: 2024-06-26T05:01:48.5335843Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:01:48.5336235Z ##[endgroup] 2024-06-26T05:01:49.0116960Z (node:17440) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T05:01:49.0117948Z 2024-06-26T05:01:49.0118208Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T05:01:49.0118945Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T05:01:49.0119956Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T05:01:49.0937739Z Found 1 objects with prefix pytorch/pytorch/9673645538/linux-jammy-py3.8-gcc11/ 2024-06-26T05:01:49.0939565Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T05:01:52.8031258Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-06-26T05:01:52.8037566Z Artifact download has finished successfully 2024-06-26T05:01:52.8200948Z ##[group]Run unzip -o artifacts.zip 2024-06-26T05:01:52.8201368Z unzip -o artifacts.zip 2024-06-26T05:01:52.8209029Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T05:01:52.8209507Z env: 2024-06-26T05:01:52.8209776Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:01:52.8210102Z ##[endgroup] 2024-06-26T05:01:52.8558281Z Archive: artifacts.zip 2024-06-26T05:01:52.8559232Z creating: dist/ 2024-06-26T05:01:53.7810973Z inflating: dist/torch-2.5.0a0+gitb8c4c54-cp38-cp38-linux_x86_64.whl 2024-06-26T05:01:53.7811644Z creating: build/custom_test_artifacts/ 2024-06-26T05:01:53.7812211Z creating: build/custom_test_artifacts/custom-op-build/ 2024-06-26T05:01:53.7812922Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-06-26T05:01:53.7813771Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-06-26T05:01:53.7814796Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T05:01:53.7815745Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-06-26T05:01:53.7816779Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T05:01:53.7817815Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T05:01:53.7818804Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T05:01:53.7819952Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T05:01:53.7821105Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T05:01:53.7822134Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T05:01:53.7823237Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T05:01:53.7824829Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T05:01:53.7826120Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T05:01:53.7827639Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T05:01:53.7829088Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T05:01:53.7830446Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T05:01:53.7831802Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T05:01:53.7832875Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-06-26T05:01:53.7833833Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-06-26T05:01:53.7834797Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-06-26T05:01:53.7835872Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-06-26T05:01:53.7837066Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-06-26T05:01:53.7838369Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-06-26T05:01:53.7839456Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-06-26T05:01:53.7840558Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-06-26T05:01:53.7841687Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-06-26T05:01:53.7842816Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-06-26T05:01:53.7844154Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-06-26T05:01:53.7845252Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-06-26T05:01:53.7850385Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-06-26T05:01:53.7972897Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-06-26T05:01:53.7974139Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-06-26T05:01:53.7975401Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-06-26T05:01:53.7976926Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-06-26T05:01:53.7978150Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-06-26T05:01:53.7979302Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-06-26T05:01:53.7980472Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-06-26T05:01:53.7981762Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-06-26T05:01:53.7983037Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-06-26T05:01:53.7984211Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-06-26T05:01:53.7985379Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-06-26T05:01:53.7996038Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-06-26T05:01:53.8072712Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-06-26T05:01:53.8074263Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T05:01:53.8075462Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-06-26T05:01:53.8076554Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-06-26T05:01:53.8077639Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-06-26T05:01:53.8078550Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-06-26T05:01:53.8079443Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-06-26T05:01:53.8080234Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-06-26T05:01:53.8081043Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-06-26T05:01:53.8184536Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-06-26T05:01:53.8245340Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-06-26T05:01:53.8246352Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-06-26T05:01:53.8247036Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-06-26T05:01:53.8248066Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-06-26T05:01:53.8249871Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T05:01:53.8251083Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-06-26T05:01:53.8252343Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T05:01:53.8253888Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T05:01:53.8255024Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T05:01:53.8256568Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T05:01:53.8258268Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T05:01:53.8259789Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T05:01:53.8261623Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T05:01:53.8263878Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T05:01:53.8266156Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T05:01:53.8267337Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T05:01:53.8268503Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T05:01:53.8269717Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T05:01:53.8270924Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T05:01:53.8271995Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-06-26T05:01:53.8272932Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-06-26T05:01:53.8273901Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-06-26T05:01:53.8275010Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-06-26T05:01:53.8276240Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-06-26T05:01:53.8277428Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-06-26T05:01:53.8278682Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-06-26T05:01:53.8279840Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-06-26T05:01:53.8280979Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-06-26T05:01:53.8282149Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-06-26T05:01:53.8283304Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-06-26T05:01:53.8285046Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-06-26T05:01:53.8286332Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-06-26T05:01:53.8345449Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-06-26T05:01:53.8347766Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T05:01:53.8349456Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-06-26T05:01:53.8350828Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-06-26T05:01:53.8351965Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-06-26T05:01:53.8352883Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-06-26T05:01:53.8353747Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-06-26T05:01:53.8354527Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-06-26T05:01:53.8355329Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-06-26T05:01:53.8398846Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-06-26T05:01:53.8399997Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-06-26T05:01:53.8400764Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-06-26T05:01:53.8401650Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-06-26T05:01:53.8403599Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-06-26T05:01:53.8404780Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-06-26T05:01:53.8406230Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-06-26T05:01:53.8407911Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-06-26T05:01:53.8409131Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-06-26T05:01:53.8411037Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-06-26T05:01:53.8412909Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-06-26T05:01:53.8414680Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-06-26T05:01:53.8416743Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-06-26T05:01:53.8419173Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-06-26T05:01:53.8420599Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-06-26T05:01:53.8421847Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-06-26T05:01:53.8423543Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-06-26T05:01:53.8424963Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-06-26T05:01:53.8426242Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-06-26T05:01:53.8427336Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-06-26T05:01:53.8428369Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-06-26T05:01:53.8429416Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-06-26T05:01:53.8430587Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-06-26T05:01:53.8431981Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-06-26T05:01:53.8433265Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-06-26T05:01:53.8434455Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-06-26T05:01:53.8435701Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-06-26T05:01:53.8436947Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-06-26T05:01:53.8438314Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-06-26T05:01:53.8439605Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-06-26T05:01:53.8440853Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-06-26T05:01:53.8442180Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-06-26T05:01:53.8541260Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-06-26T05:01:53.8543576Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-06-26T05:01:53.8545493Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-06-26T05:01:53.8546894Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-06-26T05:01:53.8548235Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-06-26T05:01:53.8549483Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-06-26T05:01:53.8550789Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-06-26T05:01:53.8552178Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-06-26T05:01:53.8553486Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-06-26T05:01:53.8554793Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-06-26T05:01:53.8556092Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-06-26T05:01:53.8564279Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-06-26T05:01:53.8615923Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-06-26T05:01:53.8618384Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T05:01:53.8620456Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-06-26T05:01:53.8621785Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-06-26T05:01:53.8622778Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-06-26T05:01:53.8623767Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-06-26T05:01:53.8624731Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-06-26T05:01:53.8625572Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-06-26T05:01:53.8626448Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-06-26T05:01:53.8720965Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-06-26T05:01:53.8762426Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-06-26T05:01:53.8763525Z creating: build/lib/ 2024-06-26T05:01:53.8763879Z inflating: build/lib/libclog.a 2024-06-26T05:01:53.8772629Z inflating: build/lib/libpthreadpool.a 2024-06-26T05:01:53.8780097Z inflating: build/lib/libcpuinfo_internals.a 2024-06-26T05:01:53.8787927Z inflating: build/lib/libcpuinfo.a 2024-06-26T05:01:53.8852262Z inflating: build/lib/libgtest.a 2024-06-26T05:01:53.8932745Z inflating: build/lib/libprotobuf-lite.a 2024-06-26T05:01:53.8939726Z inflating: build/lib/libittnotify.a 2024-06-26T05:01:53.9010835Z inflating: build/lib/libbenchmark.a 2024-06-26T05:01:53.9037491Z inflating: build/lib/libtensorpipe_uv.a 2024-06-26T05:01:53.9099343Z inflating: build/lib/libasmjit.a 2024-06-26T05:01:53.9531992Z inflating: build/lib/libprotobuf.a 2024-06-26T05:01:54.0017455Z inflating: build/lib/libgloo.a 2024-06-26T05:01:54.0038256Z inflating: build/lib/libfmt.a 2024-06-26T05:01:54.0039027Z inflating: build/lib/libfoxi_loader.a 2024-06-26T05:01:54.0040641Z inflating: build/lib/libtorch_global_deps.so 2024-06-26T05:01:54.0131741Z inflating: build/lib/libc10.so 2024-06-26T05:01:54.0150501Z inflating: build/lib/libpytorch_qnnpack.a 2024-06-26T05:01:54.0152706Z inflating: build/lib/libnnpack_reference_layers.a 2024-06-26T05:01:54.0637239Z inflating: build/lib/libprotoc.a 2024-06-26T05:01:54.0654088Z inflating: build/lib/libgmock.a 2024-06-26T05:01:54.0654890Z inflating: build/lib/libgtest_main.a 2024-06-26T05:01:54.0655754Z inflating: build/lib/libbenchmark_main.a 2024-06-26T05:01:54.0673052Z inflating: build/lib/libnnpack.a 2024-06-26T05:01:55.0011864Z inflating: build/lib/libdnnl.a 2024-06-26T05:01:55.0547111Z inflating: build/lib/libtensorpipe.a 2024-06-26T05:01:55.1556454Z inflating: build/lib/libfbgemm.a 2024-06-26T05:01:55.1557259Z inflating: build/lib/libgmock_main.a 2024-06-26T05:01:55.1837822Z inflating: build/lib/libkineto.a 2024-06-26T05:01:55.1877085Z inflating: build/lib/libonnx_proto.a 2024-06-26T05:01:55.2055588Z inflating: build/lib/libXNNPACK.a 2024-06-26T05:01:55.2724916Z inflating: build/lib/libonnx.a 2024-06-26T05:01:57.5394890Z inflating: build/lib/libtorch_cpu.so 2024-06-26T05:01:57.5395725Z inflating: build/lib/libtorch.so 2024-06-26T05:01:57.5400135Z inflating: build/lib/libunbox_lib.a 2024-06-26T05:01:57.5403815Z inflating: build/lib/libshm.so 2024-06-26T05:01:57.5423249Z inflating: build/lib/libjitbackend_test.so 2024-06-26T05:01:57.5489785Z inflating: build/lib/libtorchbind_test.so 2024-06-26T05:01:57.5514397Z inflating: build/lib/libbackend_with_compiler.so 2024-06-26T05:01:57.7343931Z inflating: build/lib/libtorch_python.so 2024-06-26T05:01:57.7377905Z inflating: build/lib/libnnapi_backend.so 2024-06-26T05:01:57.7378687Z creating: build/bin/ 2024-06-26T05:01:57.7379319Z creating: build/bin/CMakeFiles/ 2024-06-26T05:01:57.7380130Z inflating: build/bin/CMakeFiles/CMakeDirectoryInformation.cmake 2024-06-26T05:01:57.7380782Z extracting: build/bin/CMakeFiles/progress.marks 2024-06-26T05:01:57.7431303Z inflating: build/bin/reduce_ops_test 2024-06-26T05:01:57.7432192Z inflating: build/bin/Makefile 2024-06-26T05:01:57.7485180Z inflating: build/bin/reportMemoryUsage_test 2024-06-26T05:01:57.7486083Z inflating: build/bin/cmake_install.cmake 2024-06-26T05:01:57.7544663Z inflating: build/bin/quantized_test 2024-06-26T05:01:57.7545542Z inflating: build/bin/CTestTestfile.cmake 2024-06-26T05:01:57.7597329Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-06-26T05:01:57.7650258Z inflating: build/bin/c10_DeviceGuard_test 2024-06-26T05:01:57.7702227Z inflating: build/bin/c10_Device_test 2024-06-26T05:01:57.7752692Z inflating: build/bin/c10_StreamGuard_test 2024-06-26T05:01:57.7813474Z inflating: build/bin/c10_DispatchKeySet_test 2024-06-26T05:01:57.7868800Z inflating: build/bin/c10_Scalar_test 2024-06-26T05:01:57.7921514Z inflating: build/bin/c10_SymInt_test 2024-06-26T05:01:57.7977444Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-06-26T05:01:57.8034592Z inflating: build/bin/c10_InlineStreamGuard_test 2024-06-26T05:01:57.8091773Z inflating: build/bin/c10_SizesAndStrides_test 2024-06-26T05:01:57.8163923Z inflating: build/bin/c10_cow_test 2024-06-26T05:01:57.8214464Z inflating: build/bin/c10_ConstexprCrc_test 2024-06-26T05:01:57.8269302Z inflating: build/bin/c10_Bitset_test 2024-06-26T05:01:57.8320751Z inflating: build/bin/c10_DeadlockDetection_test 2024-06-26T05:01:57.8373051Z inflating: build/bin/c10_Half_test 2024-06-26T05:01:57.8430443Z inflating: build/bin/c10_LeftRight_test 2024-06-26T05:01:57.8487552Z inflating: build/bin/c10_Metaprogramming_test 2024-06-26T05:01:57.8539075Z inflating: build/bin/c10_Synchronized_test 2024-06-26T05:01:57.8595891Z inflating: build/bin/c10_ThreadLocal_test 2024-06-26T05:01:57.8649282Z inflating: build/bin/c10_TypeIndex_test 2024-06-26T05:01:57.8702018Z inflating: build/bin/c10_TypeList_test 2024-06-26T05:01:57.8752458Z inflating: build/bin/c10_TypeTraits_test 2024-06-26T05:01:57.8806014Z inflating: build/bin/c10_accumulate_test 2024-06-26T05:01:57.8862760Z inflating: build/bin/c10_bfloat16_test 2024-06-26T05:01:57.8917028Z inflating: build/bin/c10_exception_test 2024-06-26T05:01:57.8969035Z inflating: build/bin/c10_bit_cast_test 2024-06-26T05:01:57.9025638Z inflating: build/bin/c10_complex_test 2024-06-26T05:01:57.9083590Z inflating: build/bin/c10_complex_math_test 2024-06-26T05:01:57.9135629Z inflating: build/bin/c10_flags_test 2024-06-26T05:01:57.9187355Z inflating: build/bin/c10_generic_math_test 2024-06-26T05:01:57.9350000Z inflating: build/bin/c10_intrusive_ptr_test 2024-06-26T05:01:57.9402383Z inflating: build/bin/c10_irange_test 2024-06-26T05:01:57.9457532Z inflating: build/bin/c10_lazy_test 2024-06-26T05:01:57.9516509Z inflating: build/bin/c10_logging_test 2024-06-26T05:01:57.9571482Z inflating: build/bin/c10_registry_test 2024-06-26T05:01:57.9647866Z inflating: build/bin/c10_optional_test 2024-06-26T05:01:57.9711396Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-06-26T05:01:57.9862312Z inflating: build/bin/c10_small_vector_test 2024-06-26T05:01:57.9915577Z inflating: build/bin/c10_ssize_test 2024-06-26T05:01:57.9969495Z inflating: build/bin/c10_string_util_test 2024-06-26T05:01:58.0021384Z inflating: build/bin/c10_tempfile_test 2024-06-26T05:01:58.0082330Z inflating: build/bin/c10_string_view_test 2024-06-26T05:01:58.0140359Z inflating: build/bin/c10_typeid_test 2024-06-26T05:01:58.0188854Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-06-26T05:01:58.0641565Z inflating: build/bin/protoc-3.13.0.0 2024-06-26T05:01:58.1093503Z inflating: build/bin/protoc 2024-06-26T05:01:58.1423871Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-06-26T05:01:58.1771733Z inflating: build/bin/vec_test_all_types_AVX512 2024-06-26T05:01:58.2131816Z inflating: build/bin/vec_test_all_types_AVX2 2024-06-26T05:01:58.2186845Z inflating: build/bin/HashStoreTest 2024-06-26T05:01:58.2245176Z inflating: build/bin/TCPStoreTest 2024-06-26T05:01:58.2299452Z inflating: build/bin/FileStoreTest 2024-06-26T05:01:58.2366628Z inflating: build/bin/ProcessGroupGlooTest 2024-06-26T05:01:58.2369265Z inflating: build/bin/example_allreduce 2024-06-26T05:01:58.2425383Z inflating: build/bin/test_dist_autograd 2024-06-26T05:01:58.2427679Z inflating: build/bin/parallel_benchmark 2024-06-26T05:01:58.2497758Z inflating: build/bin/test_cpp_rpc 2024-06-26T05:01:58.2506687Z inflating: build/bin/aot_model_compiler_test 2024-06-26T05:01:58.2574572Z inflating: build/bin/test_mobile_nnc 2024-06-26T05:01:58.2633461Z inflating: build/bin/static_runtime_bench 2024-06-26T05:01:58.2971176Z inflating: build/bin/test_lazy 2024-06-26T05:01:58.3046105Z inflating: build/bin/Dict_test 2024-06-26T05:01:58.3100229Z inflating: build/bin/Dimname_test 2024-06-26T05:01:58.3349849Z inflating: build/bin/static_runtime_test 2024-06-26T05:01:58.3416322Z inflating: build/bin/MaybeOwned_test 2024-06-26T05:01:58.3474925Z inflating: build/bin/NamedTensor_test 2024-06-26T05:01:58.3535305Z inflating: build/bin/apply_utils_test 2024-06-26T05:01:58.3595246Z inflating: build/bin/atest 2024-06-26T05:01:58.3660178Z inflating: build/bin/basic 2024-06-26T05:01:58.3716446Z inflating: build/bin/broadcast_test 2024-06-26T05:01:58.3769032Z inflating: build/bin/cpu_allocator_test 2024-06-26T05:01:58.3828666Z inflating: build/bin/cpu_generator_test 2024-06-26T05:01:58.3882829Z inflating: build/bin/cpu_profiling_allocator_test 2024-06-26T05:01:58.3976475Z inflating: build/bin/cpu_rng_test 2024-06-26T05:01:58.4028170Z inflating: build/bin/dispatch_key_set_test 2024-06-26T05:01:58.4079776Z inflating: build/bin/dlconvertor_test 2024-06-26T05:01:58.4139072Z inflating: build/bin/extension_backend_test 2024-06-26T05:01:58.4195717Z inflating: build/bin/half_test 2024-06-26T05:01:58.5318608Z inflating: build/bin/test_api 2024-06-26T05:01:58.5414667Z inflating: build/bin/ivalue_test 2024-06-26T05:01:58.5466175Z inflating: build/bin/lazy_tensor_test 2024-06-26T05:01:58.5521013Z inflating: build/bin/math_kernel_test 2024-06-26T05:01:58.5576250Z inflating: build/bin/memory_format_test 2024-06-26T05:01:58.5628688Z inflating: build/bin/operator_name_test 2024-06-26T05:01:58.5683393Z inflating: build/bin/memory_overlapping_test 2024-06-26T05:01:58.5740914Z inflating: build/bin/native_test 2024-06-26T05:01:58.5795632Z inflating: build/bin/mobile_memory_cleanup 2024-06-26T05:01:58.5848075Z inflating: build/bin/operators_test 2024-06-26T05:01:58.5901752Z inflating: build/bin/packedtensoraccessor_test 2024-06-26T05:01:58.5969844Z inflating: build/bin/pow_test 2024-06-26T05:01:58.6028303Z inflating: build/bin/scalar_tensor_test 2024-06-26T05:01:58.6088319Z inflating: build/bin/scalar_test 2024-06-26T05:01:58.6141158Z inflating: build/bin/StorageUtils_test 2024-06-26T05:01:58.6195036Z inflating: build/bin/stride_properties_test 2024-06-26T05:01:58.6274443Z inflating: build/bin/tensor_iterator_test 2024-06-26T05:01:58.6330958Z inflating: build/bin/type_ptr_test 2024-06-26T05:01:58.6386149Z inflating: build/bin/test_parallel 2024-06-26T05:01:58.6388817Z inflating: build/bin/thread_init_test 2024-06-26T05:01:58.6450135Z inflating: build/bin/type_test 2024-06-26T05:01:58.6451729Z inflating: build/bin/verify_api_visibility 2024-06-26T05:01:58.6505563Z inflating: build/bin/undefined_tensor_test 2024-06-26T05:01:58.6576101Z inflating: build/bin/legacy_vmap_test 2024-06-26T05:01:58.6628920Z inflating: build/bin/weakref_test 2024-06-26T05:01:58.6681838Z inflating: build/bin/wrapdim_test 2024-06-26T05:01:58.6742859Z inflating: build/bin/IListRef_test 2024-06-26T05:01:58.6850127Z inflating: build/bin/List_test 2024-06-26T05:01:58.6903279Z inflating: build/bin/xla_tensor_test 2024-06-26T05:01:58.6971254Z inflating: build/bin/KernelFunction_test 2024-06-26T05:01:58.7092485Z inflating: build/bin/kernel_function_legacy_test 2024-06-26T05:01:58.7189005Z inflating: build/bin/kernel_function_test 2024-06-26T05:01:58.7315804Z inflating: build/bin/kernel_lambda_legacy_test 2024-06-26T05:01:58.7418218Z inflating: build/bin/kernel_lambda_test 2024-06-26T05:01:58.7480332Z inflating: build/bin/kernel_stackbased_test 2024-06-26T05:01:58.7576704Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-06-26T05:01:58.7629313Z inflating: build/bin/CppSignature_test 2024-06-26T05:01:58.7679676Z inflating: build/bin/op_allowlist_test 2024-06-26T05:01:58.7987231Z inflating: build/bin/op_registration_test 2024-06-26T05:01:58.8043918Z inflating: build/bin/backend_fallback_test 2024-06-26T05:01:58.8109190Z inflating: build/bin/inline_container_test 2024-06-26T05:01:58.8164153Z inflating: build/bin/test_edge_op_registration 2024-06-26T05:01:58.8168335Z inflating: build/bin/torch_shm_manager 2024-06-26T05:01:58.8181425Z inflating: build/bin/tutorial_tensorexpr 2024-06-26T05:01:58.8967838Z inflating: build/bin/test_tensorexpr 2024-06-26T05:01:58.9539132Z inflating: build/bin/test_jit 2024-06-26T05:01:58.9539864Z creating: .additional_ci_files/ 2024-06-26T05:01:58.9593617Z inflating: .additional_ci_files/test-times.json 2024-06-26T05:01:58.9809042Z inflating: .additional_ci_files/test-class-times.json 2024-06-26T05:01:58.9852282Z ##[group]Run rm artifacts.zip 2024-06-26T05:01:58.9852912Z rm artifacts.zip 2024-06-26T05:01:58.9863309Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T05:01:58.9864060Z env: 2024-06-26T05:01:58.9864455Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:01:58.9864995Z ##[endgroup] 2024-06-26T05:01:59.0327313Z ##[group]Run df -H 2024-06-26T05:01:59.0327618Z df -H 2024-06-26T05:01:59.0334911Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T05:01:59.0335388Z env: 2024-06-26T05:01:59.0335650Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:01:59.0335974Z ##[endgroup] 2024-06-26T05:01:59.0392022Z Filesystem Size Used Avail Use% Mounted on 2024-06-26T05:01:59.0392826Z devtmpfs 8.2G 0 8.2G 0% /dev 2024-06-26T05:01:59.0393729Z tmpfs 8.2G 7.9M 8.2G 1% /dev/shm 2024-06-26T05:01:59.0394503Z tmpfs 8.2G 410k 8.2G 1% /run 2024-06-26T05:01:59.0394946Z tmpfs 8.2G 0 8.2G 0% /sys/fs/cgroup 2024-06-26T05:01:59.0395402Z /dev/nvme0n1p1 162G 25G 137G 15% / 2024-06-26T05:01:59.0448412Z Prepare all required actions 2024-06-26T05:01:59.0448839Z Getting action download info 2024-06-26T05:01:59.1512457Z ##[group]Run ./.github/actions/download-td-artifacts 2024-06-26T05:01:59.1512917Z with: 2024-06-26T05:01:59.1513136Z env: 2024-06-26T05:01:59.1513387Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:01:59.1513693Z ##[endgroup] 2024-06-26T05:01:59.1547443Z ##[group]Run seemethere/download-artifact-s3@v4 2024-06-26T05:01:59.1547883Z with: 2024-06-26T05:01:59.1548118Z name: td_results 2024-06-26T05:01:59.1548416Z s3-bucket: gha-artifacts 2024-06-26T05:01:59.1548747Z region: us-east-1 2024-06-26T05:01:59.1549007Z env: 2024-06-26T05:01:59.1549259Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:01:59.1549582Z ##[endgroup] 2024-06-26T05:01:59.6238252Z (node:17458) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-06-26T05:01:59.6239171Z 2024-06-26T05:01:59.6239583Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-06-26T05:01:59.6240562Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-06-26T05:01:59.6241869Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-06-26T05:01:59.7020541Z Found 1 objects with prefix pytorch/pytorch/9673645538/td_results/ 2024-06-26T05:01:59.7021597Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-06-26T05:01:59.7518037Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-06-26T05:01:59.7524699Z Artifact download has finished successfully 2024-06-26T05:01:59.7674139Z ##[group]Run mkdir -p .additional_ci_files 2024-06-26T05:01:59.7674961Z mkdir -p .additional_ci_files 2024-06-26T05:01:59.7675945Z mv td_results.json .additional_ci_files/td_results.json 2024-06-26T05:01:59.7686655Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T05:01:59.7687485Z env: 2024-06-26T05:01:59.7687930Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:01:59.7688507Z ##[endgroup] 2024-06-26T05:01:59.7785796Z ##[group]Run .github/scripts/parse_ref.py 2024-06-26T05:01:59.7786253Z .github/scripts/parse_ref.py 2024-06-26T05:01:59.7793058Z shell: /usr/bin/bash -e {0} 2024-06-26T05:01:59.7793389Z env: 2024-06-26T05:01:59.7793639Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:01:59.7793950Z ##[endgroup] 2024-06-26T05:01:59.8061724Z Prepare all required actions 2024-06-26T05:01:59.8103015Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-06-26T05:01:59.8103465Z with: 2024-06-26T05:01:59.8104036Z github-token: *** 2024-06-26T05:01:59.8104307Z env: 2024-06-26T05:01:59.8104563Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:01:59.8105022Z ##[endgroup] 2024-06-26T05:01:59.8125128Z ##[group]Run set -eux 2024-06-26T05:01:59.8125576Z set -eux 2024-06-26T05:01:59.8126159Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-06-26T05:01:59.8133484Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T05:01:59.8133957Z env: 2024-06-26T05:01:59.8134247Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:01:59.8134762Z GITHUB_TOKEN: *** 2024-06-26T05:01:59.8135036Z ##[endgroup] 2024-06-26T05:01:59.8155381Z + python3 .github/scripts/get_workflow_job_id.py 9673645538 i-0b079ed883f442af4 2024-06-26T05:02:01.8339370Z setting job-id=26688305322 2024-06-26T05:02:01.8340740Z setting job-name=linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, linux.2xlarge) 2024-06-26T05:02:01.8567893Z Prepare all required actions 2024-06-26T05:02:01.8568560Z Getting action download info 2024-06-26T05:02:01.9819074Z ##[group]Run ./.github/actions/filter-test-configs 2024-06-26T05:02:01.9819532Z with: 2024-06-26T05:02:01.9819977Z github-token: *** 2024-06-26T05:02:01.9822902Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 3, "runner": "linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 3, "runner": "linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 3, "runner": "linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "linux.2xlarge"}]} 2024-06-26T05:02:01.9826158Z job-name: linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, linux.2xlarge) 2024-06-26T05:02:01.9826753Z env: 2024-06-26T05:02:01.9826997Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:02:01.9827326Z ##[endgroup] 2024-06-26T05:02:01.9871533Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-06-26T05:02:01.9872176Z with: 2024-06-26T05:02:01.9872425Z shell: bash 2024-06-26T05:02:01.9872690Z timeout_minutes: 10 2024-06-26T05:02:01.9872997Z max_attempts: 5 2024-06-26T05:02:01.9873289Z retry_wait_seconds: 30 2024-06-26T05:02:01.9874434Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T05:02:01.9875651Z polling_interval_seconds: 1 2024-06-26T05:02:01.9876011Z warning_on_retry: true 2024-06-26T05:02:01.9876334Z continue_on_error: false 2024-06-26T05:02:01.9876650Z env: 2024-06-26T05:02:01.9876906Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:02:01.9877426Z GITHUB_TOKEN: *** 2024-06-26T05:02:01.9877709Z ##[endgroup] 2024-06-26T05:02:02.0427861Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-06-26T05:02:02.2638064Z Defaulting to user installation because normal site-packages is not writeable 2024-06-26T05:02:02.2816172Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.7/site-packages (2.27.1) 2024-06-26T05:02:02.2954334Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.7/site-packages (6.0.1) 2024-06-26T05:02:02.2963762Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (1.26.19) 2024-06-26T05:02:02.3146664Z Requirement already satisfied: idna<4,>=2.5; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (3.7) 2024-06-26T05:02:02.3162526Z Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= "3" in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2.0.12) 2024-06-26T05:02:02.3182637Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.7/site-packages (from requests==2.27.1) (2024.6.2) 2024-06-26T05:02:03.0416340Z Command completed after 1 attempt(s). 2024-06-26T05:02:03.0457900Z ##[group]Run set -x 2024-06-26T05:02:03.0458219Z set -x 2024-06-26T05:02:03.0458495Z  2024-06-26T05:02:03.0459030Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T05:02:03.0459700Z # in runner workspace 2024-06-26T05:02:03.0460210Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-06-26T05:02:03.0467696Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T05:02:03.0468178Z env: 2024-06-26T05:02:03.0468432Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:02:03.0468757Z ##[endgroup] 2024-06-26T05:02:03.0490485Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-06-26T05:02:03.0698499Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T05:02:03.0699033Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-06-26T05:02:03.0699485Z echo "Job name: ${JOB_NAME}" 2024-06-26T05:02:03.0699881Z  2024-06-26T05:02:03.0700397Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-06-26T05:02:03.0701075Z # in runner workspace 2024-06-26T05:02:03.0701631Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-06-26T05:02:03.0702251Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-06-26T05:02:03.0702682Z  --job-name "${JOB_NAME}" \ 2024-06-26T05:02:03.0705752Z  --test-matrix "{"include": [{"config": "default", "shard": 1, "num_shards": 3, "runner": "linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 3, "runner": "linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 3, "runner": "linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "linux.2xlarge"}]}" \ 2024-06-26T05:02:03.0708821Z  --selected-test-configs "" \ 2024-06-26T05:02:03.0709255Z  --pr-number "${PR_NUMBER}" \ 2024-06-26T05:02:03.0709644Z  --tag "${TAG}" \ 2024-06-26T05:02:03.0710009Z  --event-name "${EVENT_NAME}" \ 2024-06-26T05:02:03.0710434Z  --schedule "${SCHEDULE}" \ 2024-06-26T05:02:03.0710837Z  --branch "${HEAD_BRANCH}" 2024-06-26T05:02:03.0718041Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T05:02:03.0718523Z env: 2024-06-26T05:02:03.0718772Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:02:03.0719306Z GITHUB_TOKEN: *** 2024-06-26T05:02:03.0719817Z JOB_NAME: linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, linux.2xlarge) 2024-06-26T05:02:03.0720420Z PR_NUMBER: 129470 2024-06-26T05:02:03.0720686Z TAG: 2024-06-26T05:02:03.0720953Z EVENT_NAME: pull_request 2024-06-26T05:02:03.0721278Z SCHEDULE: 2024-06-26T05:02:03.0721524Z HEAD_BRANCH: 2024-06-26T05:02:03.0721800Z ##[endgroup] 2024-06-26T05:02:03.0741819Z Workflow: pull 2024-06-26T05:02:03.0742485Z Job name: linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, linux.2xlarge) 2024-06-26T05:02:03.2932576Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-06-26T05:02:04.0578565Z ##[group]Run echo "Filtered matrix:" 2024-06-26T05:02:04.0579135Z echo "Filtered matrix:" 2024-06-26T05:02:04.0582150Z echo "{"include": [{"config": "default", "shard": 1, "num_shards": 3, "runner": "linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 3, "runner": "linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 3, "runner": "linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "linux.2xlarge"}]}" 2024-06-26T05:02:04.0585275Z  2024-06-26T05:02:04.0585528Z echo 2024-06-26T05:02:04.0585886Z echo "Is the current job unstable? False" 2024-06-26T05:02:04.0586326Z  2024-06-26T05:02:04.0586575Z echo 2024-06-26T05:02:04.0586913Z echo "Is keep-going label set? False" 2024-06-26T05:02:04.0587329Z  2024-06-26T05:02:04.0587578Z echo 2024-06-26T05:02:04.0587979Z echo "Renabled issues? " 2024-06-26T05:02:04.0595505Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T05:02:04.0596004Z env: 2024-06-26T05:02:04.0596266Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:02:04.0596584Z ##[endgroup] 2024-06-26T05:02:04.0617497Z Filtered matrix: 2024-06-26T05:02:04.0622485Z {include: [{config: default, shard: 1, num_shards: 3, runner: linux.2xlarge}, {config: default, shard: 2, num_shards: 3, runner: linux.2xlarge}, {config: default, shard: 3, num_shards: 3, runner: linux.2xlarge}, {config: docs_test, shard: 1, num_shards: 1, runner: linux.2xlarge}, {config: jit_legacy, shard: 1, num_shards: 1, runner: linux.2xlarge}, {config: backwards_compat, shard: 1, num_shards: 1, runner: linux.2xlarge}, {config: distributed, shard: 1, num_shards: 2, runner: linux.2xlarge}, {config: distributed, shard: 2, num_shards: 2, runner: linux.2xlarge}]} 2024-06-26T05:02:04.0625252Z 2024-06-26T05:02:04.0625401Z Is the current job unstable? False 2024-06-26T05:02:04.0625682Z 2024-06-26T05:02:04.0625965Z Is keep-going label set? False 2024-06-26T05:02:04.0626210Z 2024-06-26T05:02:04.0626331Z Renabled issues? 2024-06-26T05:02:04.0671392Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T05:02:04.0672174Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-06-26T05:02:04.0680048Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T05:02:04.0680544Z env: 2024-06-26T05:02:04.0680787Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:02:04.0681116Z JOB_TIMEOUT: 240 2024-06-26T05:02:04.0681402Z ##[endgroup] 2024-06-26T05:02:04.0758711Z ##[group]Run set -x 2024-06-26T05:02:04.0759097Z set -x 2024-06-26T05:02:04.0759369Z  2024-06-26T05:02:04.0759700Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-06-26T05:02:04.0760225Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-06-26T05:02:04.0760777Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-06-26T05:02:04.0761280Z  TEST_COMMAND=.ci/onnx/test.sh 2024-06-26T05:02:04.0761663Z else 2024-06-26T05:02:04.0761981Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T05:02:04.0762387Z fi 2024-06-26T05:02:04.0762625Z  2024-06-26T05:02:04.0763081Z # detached container should get cleaned up by teardown_ec2_linux 2024-06-26T05:02:04.0763853Z # TODO: Stop building test binaries as part of the build phase 2024-06-26T05:02:04.0764670Z # Used for GPU_FLAG since that doesn't play nice 2024-06-26T05:02:04.0765230Z # shellcheck disable=SC2086,SC2090 2024-06-26T05:02:04.0765679Z container_name=$(docker run \ 2024-06-26T05:02:04.0766081Z  ${GPU_FLAG:-} \ 2024-06-26T05:02:04.0766440Z  -e BUILD_ENVIRONMENT \ 2024-06-26T05:02:04.0766810Z  -e PR_NUMBER \ 2024-06-26T05:02:04.0767158Z  -e GITHUB_ACTIONS \ 2024-06-26T05:02:04.0767531Z  -e GITHUB_REPOSITORY \ 2024-06-26T05:02:04.0767905Z  -e GITHUB_WORKFLOW \ 2024-06-26T05:02:04.0768266Z  -e GITHUB_JOB \ 2024-06-26T05:02:04.0768606Z  -e GITHUB_RUN_ID \ 2024-06-26T05:02:04.0768953Z  -e GITHUB_RUN_NUMBER \ 2024-06-26T05:02:04.0769336Z  -e GITHUB_RUN_ATTEMPT \ 2024-06-26T05:02:04.0769713Z  -e JOB_ID \ 2024-06-26T05:02:04.0770189Z  -e JOB_NAME \ 2024-06-26T05:02:04.0770518Z  -e BASE_SHA \ 2024-06-26T05:02:04.0770846Z  -e BRANCH \ 2024-06-26T05:02:04.0771145Z  -e SHA1 \ 2024-06-26T05:02:04.0771476Z  -e AWS_DEFAULT_REGION \ 2024-06-26T05:02:04.0771859Z  -e IN_WHEEL_TEST \ 2024-06-26T05:02:04.0772207Z  -e SHARD_NUMBER \ 2024-06-26T05:02:04.0772554Z  -e TEST_CONFIG \ 2024-06-26T05:02:04.0772904Z  -e NUM_TEST_SHARDS \ 2024-06-26T05:02:04.0773262Z  -e REENABLED_ISSUES \ 2024-06-26T05:02:04.0773656Z  -e CONTINUE_THROUGH_ERROR \ 2024-06-26T05:02:04.0774062Z  -e VERBOSE_TEST_LOGS \ 2024-06-26T05:02:04.0774430Z  -e NO_TEST_TIMEOUT \ 2024-06-26T05:02:04.0774781Z  -e NO_TD \ 2024-06-26T05:02:04.0775100Z  -e TD_DISTRIBUTED \ 2024-06-26T05:02:04.0775446Z  -e PR_LABELS \ 2024-06-26T05:02:04.0775829Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-06-26T05:02:04.0776278Z  -e SCCACHE_BUCKET \ 2024-06-26T05:02:04.0776638Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-06-26T05:02:04.0777028Z  -e XLA_CUDA \ 2024-06-26T05:02:04.0777408Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-06-26T05:02:04.0777891Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-06-26T05:02:04.0778397Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-06-26T05:02:04.0778895Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-06-26T05:02:04.0779335Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-06-26T05:02:04.0779735Z  -e DASHBOARD_TAG \ 2024-06-26T05:02:04.0780197Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-06-26T05:02:04.0780739Z  --security-opt seccomp=unconfined \ 2024-06-26T05:02:04.0781174Z  --cap-add=SYS_PTRACE \ 2024-06-26T05:02:04.0781548Z  --ipc=host \ 2024-06-26T05:02:04.0781883Z  --shm-size="${SHM_SIZE}" \ 2024-06-26T05:02:04.0782254Z  --tty \ 2024-06-26T05:02:04.0782543Z  --detach \ 2024-06-26T05:02:04.0782876Z  --name="${container_name}" \ 2024-06-26T05:02:04.0783264Z  --user jenkins \ 2024-06-26T05:02:04.0783855Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-06-26T05:02:04.0784410Z  -w /var/lib/jenkins/workspace \ 2024-06-26T05:02:04.0784820Z  "${DOCKER_IMAGE}" 2024-06-26T05:02:04.0785151Z ) 2024-06-26T05:02:04.0785541Z # Propagate download.pytorch.org IP to container 2024-06-26T05:02:04.0786471Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-06-26T05:02:04.0787467Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-06-26T05:02:04.0788396Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-06-26T05:02:04.0796299Z shell: /usr/bin/bash -e {0} 2024-06-26T05:02:04.0796634Z env: 2024-06-26T05:02:04.0796889Z GIT_DEFAULT_BRANCH: main 2024-06-26T05:02:04.0797285Z BUILD_ENVIRONMENT: linux-jammy-py3.8-gcc11 2024-06-26T05:02:04.0797707Z PR_NUMBER: 129470 2024-06-26T05:02:04.0798033Z GITHUB_REPOSITORY: pytorch/pytorch 2024-06-26T05:02:04.0798432Z GITHUB_WORKFLOW: pull 2024-06-26T05:02:04.0798735Z GITHUB_JOB: test 2024-06-26T05:02:04.0799030Z GITHUB_RUN_ID: 9673645538 2024-06-26T05:02:04.0799368Z GITHUB_RUN_NUMBER: 219936 2024-06-26T05:02:04.0799690Z GITHUB_RUN_ATTEMPT: 1 2024-06-26T05:02:04.0800002Z JOB_ID: 26688305322 2024-06-26T05:02:04.0800516Z JOB_NAME: linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, linux.2xlarge) 2024-06-26T05:02:04.0801109Z BRANCH: pull/129470 2024-06-26T05:02:04.0801467Z SHA1: b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T05:02:04.0801968Z BASE_SHA: 4b9c9a9cc9c9283380a011310ba180c105c3dcb9 2024-06-26T05:02:04.0802413Z TEST_CONFIG: distributed 2024-06-26T05:02:04.0802855Z SHARD_NUMBER: 1 2024-06-26T05:02:04.0803142Z NUM_TEST_SHARDS: 2 2024-06-26T05:02:04.0803429Z REENABLED_ISSUES: 2024-06-26T05:02:04.0803746Z CONTINUE_THROUGH_ERROR: False 2024-06-26T05:02:04.0804642Z VERBOSE_TEST_LOGS: False 2024-06-26T05:02:04.0804977Z NO_TEST_TIMEOUT: False 2024-06-26T05:02:04.0805296Z NO_TD: False 2024-06-26T05:02:04.0805575Z TD_DISTRIBUTED: False 2024-06-26T05:02:04.0805967Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-06-26T05:02:04.0806449Z SCCACHE_S3_KEY_PREFIX: pull 2024-06-26T05:02:04.0806791Z SHM_SIZE: 1g 2024-06-26T05:02:04.0807655Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T05:02:04.0808625Z XLA_CUDA: 2024-06-26T05:02:04.0809097Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-06-26T05:02:04.0809699Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-06-26T05:02:04.0810119Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-06-26T05:02:04.0810514Z DASHBOARD_TAG: 2024-06-26T05:02:04.0810795Z HUGGING_FACE_HUB_TOKEN: 2024-06-26T05:02:04.0811126Z ##[endgroup] 2024-06-26T05:02:04.0832592Z + [[ distributed == \m\u\l\t\i\g\p\u ]] 2024-06-26T05:02:04.0833265Z + [[ linux-jammy-py3.8-gcc11 == *onnx* ]] 2024-06-26T05:02:04.0833707Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-06-26T05:02:04.0840513Z +++ nproc --ignore=2 2024-06-26T05:02:04.0869116Z ++ docker run -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=6 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_9673645538 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=1g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T05:02:10.7908963Z + container_name=a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T05:02:10.7910525Z + grep download.pytorch.org /etc/hosts 2024-06-26T05:02:10.7912187Z + docker exec -i a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d sudo bash -c '/bin/cat >> /etc/hosts' 2024-06-26T05:02:10.9420370Z + echo DOCKER_CONTAINER_ID=a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T05:02:10.9423443Z ++ echo dist/torch-2.5.0a0+gitb8c4c54-cp38-cp38-linux_x86_64.whl 2024-06-26T05:02:10.9425499Z + docker exec -t a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d sh -c 'pip install dist/torch-2.5.0a0+gitb8c4c54-cp38-cp38-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-06-26T05:02:11.7103099Z Processing ./dist/torch-2.5.0a0+gitb8c4c54-cp38-cp38-linux_x86_64.whl (from torch==2.5.0a0+gitb8c4c54) 2024-06-26T05:02:12.1477479Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.13.1) 2024-06-26T05:02:12.1483320Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (4.12.2) 2024-06-26T05:02:12.1486897Z Requirement already satisfied: sympy in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.12.1) 2024-06-26T05:02:12.1490499Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.8.8) 2024-06-26T05:02:12.1493954Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.1.4) 2024-06-26T05:02:12.1497611Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2024.2.0) 2024-06-26T05:02:12.1527522Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (3.3.0) 2024-06-26T05:02:12.1585379Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.21.2) 2024-06-26T05:02:12.2017358Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (2.1.5) 2024-06-26T05:02:12.2184903Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy->torch==2.5.0a0+gitb8c4c54->torch==2.5.0a0+gitb8c4c54) (1.3.0) 2024-06-26T05:02:13.4415038Z Installing collected packages: torch 2024-06-26T05:02:20.9945320Z Successfully installed torch-2.5.0a0+gitb8c4c54 2024-06-26T05:02:21.0715655Z ++ dirname .ci/pytorch/test.sh 2024-06-26T05:02:21.0728862Z + source .ci/pytorch/common.sh 2024-06-26T05:02:21.0734314Z +++ dirname .ci/pytorch/common.sh 2024-06-26T05:02:21.0739601Z ++ source .ci/pytorch/common_utils.sh 2024-06-26T05:02:21.0747020Z +++ declare -f -t trap_add 2024-06-26T05:02:21.0751130Z ++ set -ex 2024-06-26T05:02:21.0751653Z ++ [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-06-26T05:02:21.0752224Z ++ BUILD_TEST_LIBTORCH=0 2024-06-26T05:02:21.0752655Z + [[ linux-jammy-py3.8-gcc11 != *rocm* ]] 2024-06-26T05:02:21.0755235Z ++ stat -c %u /var/lib/jenkins/workspace 2024-06-26T05:02:21.0782117Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-06-26T05:02:21.0782820Z + trap_add cleanup_workspace EXIT 2024-06-26T05:02:21.0783230Z + trap_add_cmd=cleanup_workspace 2024-06-26T05:02:21.0783574Z + shift 2024-06-26T05:02:21.0784101Z + for trap_add_name in "$@" 2024-06-26T05:02:21.0787783Z +++ trap -p EXIT 2024-06-26T05:02:21.0790064Z ++ eval 'extract_trap_cmd ' 2024-06-26T05:02:21.0790449Z +++ extract_trap_cmd 2024-06-26T05:02:21.0790791Z +++ printf '%s\n' '' 2024-06-26T05:02:21.0791175Z ++ printf '%s\n' cleanup_workspace 2024-06-26T05:02:21.0792105Z + trap -- ' 2024-06-26T05:02:21.0792592Z cleanup_workspace' EXIT 2024-06-26T05:02:21.0793370Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-06-26T05:02:24.4480654Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-06-26T05:02:24.4714024Z + echo 'Environment variables:' 2024-06-26T05:02:24.4714570Z Environment variables: 2024-06-26T05:02:24.4715023Z + env 2024-06-26T05:02:24.4753564Z INSTALLED_DB=yes 2024-06-26T05:02:24.4754791Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T05:02:24.4755631Z CONTINUE_THROUGH_ERROR=False 2024-06-26T05:02:24.4756078Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11 2024-06-26T05:02:24.4756513Z HOSTNAME=a39c6ed03d84 2024-06-26T05:02:24.4757351Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_92007462-5464-47ab-b3e4-9a93818191f3 2024-06-26T05:02:24.4758171Z GITHUB_ACTION=__self 2024-06-26T05:02:24.4758494Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T05:02:24.4758859Z GITHUB_RUN_NUMBER=219936 2024-06-26T05:02:24.4759184Z TEST_CONFIG=distributed 2024-06-26T05:02:24.4759516Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T05:02:24.4759952Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T05:02:24.4760435Z GITHUB_TRIGGERING_ACTOR=leslie-fang-intel 2024-06-26T05:02:24.4760848Z GITHUB_REF_TYPE=branch 2024-06-26T05:02:24.4761154Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T05:02:24.4761810Z BASE_SHA=4b9c9a9cc9c9283380a011310ba180c105c3dcb9 2024-06-26T05:02:24.4762243Z XLA_CUDA= 2024-06-26T05:02:24.4762493Z HUGGING_FACE_HUB_TOKEN= 2024-06-26T05:02:24.4764465Z *** 2024-06-26T05:02:24.4764744Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T05:02:24.4765105Z GITHUB_ACTIONS=true 2024-06-26T05:02:24.4765445Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T05:02:24.4765940Z GITHUB_SHA=4b51b1a62a63a1add1b3a0f7882f5c7dc66b8f8d 2024-06-26T05:02:24.4766649Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/129470/merge 2024-06-26T05:02:24.4767284Z UCC_HOME=/usr 2024-06-26T05:02:24.4767557Z VERBOSE_TEST_LOGS=False 2024-06-26T05:02:24.4767872Z GITHUB_REF=refs/pull/129470/merge 2024-06-26T05:02:24.4768229Z SHARD_NUMBER=1 2024-06-26T05:02:24.4768511Z GITHUB_REF_PROTECTED=false 2024-06-26T05:02:24.4768823Z HOME=/var/lib/jenkins 2024-06-26T05:02:24.4769163Z GITHUB_API_URL=https://api.github.com 2024-06-26T05:02:24.4769584Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T05:02:24.4769953Z UCX_COMMIT= 2024-06-26T05:02:24.4770219Z SCCACHE_S3_KEY_PREFIX=pull 2024-06-26T05:02:24.4770543Z NUM_TEST_SHARDS=2 2024-06-26T05:02:24.4770805Z UCX_HOME=/usr 2024-06-26T05:02:24.4771681Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_92007462-5464-47ab-b3e4-9a93818191f3 2024-06-26T05:02:24.4772797Z JOB_NAME=linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, linux.2xlarge) 2024-06-26T05:02:24.4773922Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_92007462-5464-47ab-b3e4-9a93818191f3 2024-06-26T05:02:24.4775055Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T05:02:24.4775722Z GITHUB_EVENT_NAME=pull_request 2024-06-26T05:02:24.4776066Z DASHBOARD_TAG= 2024-06-26T05:02:24.4776356Z GITHUB_RUN_ID=9673645538 2024-06-26T05:02:24.4777277Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_92007462-5464-47ab-b3e4-9a93818191f3 2024-06-26T05:02:24.4778211Z GITHUB_ACTOR=leslie-fang-intel 2024-06-26T05:02:24.4778545Z PR_NUMBER=129470 2024-06-26T05:02:24.4778818Z DESIRED_CUDA= 2024-06-26T05:02:24.4779094Z GITHUB_RUN_ATTEMPT=1 2024-06-26T05:02:24.4779396Z ANACONDA_PYTHON_VERSION=3.8 2024-06-26T05:02:24.4779989Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T05:02:24.4780444Z TERM=xterm 2024-06-26T05:02:24.4780693Z INSTALLED_VISION=yes 2024-06-26T05:02:24.4780993Z BRANCH=pull/129470 2024-06-26T05:02:24.4781303Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T05:02:24.4781646Z CUDA_PATH=/usr/local/cuda 2024-06-26T05:02:24.4782409Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T05:02:24.4783168Z GITHUB_SERVER_URL=https://github.com 2024-06-26T05:02:24.4783537Z UCC_COMMIT= 2024-06-26T05:02:24.4783792Z REENABLED_ISSUES= 2024-06-26T05:02:24.4784063Z DOCS=yes 2024-06-26T05:02:24.4784298Z INSTALLED_ANDROID= 2024-06-26T05:02:24.4784570Z SHLVL=1 2024-06-26T05:02:24.4784800Z MAX_JOBS=6 2024-06-26T05:02:24.4785051Z GITHUB_ACTOR_ID=53841472 2024-06-26T05:02:24.4785492Z GITHUB_WORKFLOW_SHA=4b51b1a62a63a1add1b3a0f7882f5c7dc66b8f8d 2024-06-26T05:02:24.4785999Z GITHUB_REF_NAME=129470/merge 2024-06-26T05:02:24.4786583Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T05:02:24.4787155Z GITHUB_JOB=test 2024-06-26T05:02:24.4787433Z NO_TEST_TIMEOUT=False 2024-06-26T05:02:24.4787725Z TD_DISTRIBUTED=False 2024-06-26T05:02:24.4788046Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T05:02:24.4788424Z GITHUB_RETENTION_DAYS=90 2024-06-26T05:02:24.4788737Z OPENSSL_DIR=/opt/openssl 2024-06-26T05:02:24.4789062Z GITHUB_ACTION_REPOSITORY= 2024-06-26T05:02:24.4790122Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T05:02:24.4791328Z GITHUB_BASE_REF=gh/leslie-fang-intel/126/base 2024-06-26T05:02:24.4791841Z INSTALLED_ACL= 2024-06-26T05:02:24.4792112Z CI=true 2024-06-26T05:02:24.4792485Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T05:02:24.4792843Z JOB_ID=26688305322 2024-06-26T05:02:24.4793139Z INSTALLED_PROTOBUF=yes 2024-06-26T05:02:24.4793546Z GITHUB_HEAD_REF=gh/leslie-fang-intel/126/head 2024-06-26T05:02:24.4793978Z GITHUB_ACTION_REF= 2024-06-26T05:02:24.4794399Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T05:02:24.4794840Z GITHUB_WORKFLOW=pull 2024-06-26T05:02:24.4795155Z DEBIAN_FRONTEND=noninteractive 2024-06-26T05:02:24.4796066Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_92007462-5464-47ab-b3e4-9a93818191f3 2024-06-26T05:02:24.4796879Z NO_TD=False 2024-06-26T05:02:24.4797157Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T05:02:24.4797496Z _=/usr/bin/env 2024-06-26T05:02:24.4797936Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-06-26T05:02:24.4918640Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch 2024-06-26T05:02:24.4919567Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-06-26T05:02:24.4920601Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib 2024-06-26T05:02:24.4921600Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/test 2024-06-26T05:02:24.4922291Z + BUILD_DIR=build 2024-06-26T05:02:24.4922639Z + BUILD_RENAMED_DIR=build_renamed 2024-06-26T05:02:24.4923043Z + BUILD_BIN_DIR=build/bin 2024-06-26T05:02:24.4923361Z + SHARD_NUMBER=1 2024-06-26T05:02:24.4923645Z + NUM_TEST_SHARDS=2 2024-06-26T05:02:24.4924095Z + export VALGRIND=ON 2024-06-26T05:02:24.4924400Z + VALGRIND=ON 2024-06-26T05:02:24.4924776Z + [[ linux-jammy-py3.8-gcc11 == *clang9* ]] 2024-06-26T05:02:24.4925182Z + [[ 0 == \1 ]] 2024-06-26T05:02:24.4925453Z + [[ False == \1 ]] 2024-06-26T05:02:24.4925829Z + [[ linux-jammy-py3.8-gcc11 != *bazel* ]] 2024-06-26T05:02:24.4926261Z ++ realpath build/custom_test_artifacts 2024-06-26T05:02:24.4956544Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-06-26T05:02:24.4957370Z + [[ -n '' ]] 2024-06-26T05:02:24.4957690Z + echo 'Environment variables' 2024-06-26T05:02:24.4958125Z Environment variables 2024-06-26T05:02:24.4958417Z + env 2024-06-26T05:02:24.4962794Z INSTALLED_DB=yes 2024-06-26T05:02:24.4964115Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T05:02:24.4964883Z CONTINUE_THROUGH_ERROR=False 2024-06-26T05:02:24.4965471Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11 2024-06-26T05:02:24.4965995Z HOSTNAME=a39c6ed03d84 2024-06-26T05:02:24.4966933Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_92007462-5464-47ab-b3e4-9a93818191f3 2024-06-26T05:02:24.4967966Z GITHUB_ACTION=__self 2024-06-26T05:02:24.4968393Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-06-26T05:02:24.4969063Z GITHUB_RUN_NUMBER=219936 2024-06-26T05:02:24.4969668Z TEST_CONFIG=distributed 2024-06-26T05:02:24.4969996Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-06-26T05:02:24.4970625Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-06-26T05:02:24.4971342Z GITHUB_TRIGGERING_ACTOR=leslie-fang-intel 2024-06-26T05:02:24.4971930Z GITHUB_REF_TYPE=branch 2024-06-26T05:02:24.4972246Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-06-26T05:02:24.4972682Z BASE_SHA=4b9c9a9cc9c9283380a011310ba180c105c3dcb9 2024-06-26T05:02:24.4973346Z XLA_CUDA= 2024-06-26T05:02:24.4973760Z HUGGING_FACE_HUB_TOKEN= 2024-06-26T05:02:24.4974431Z *** 2024-06-26T05:02:24.4974884Z GITHUB_REPOSITORY_ID=65600975 2024-06-26T05:02:24.4975323Z GITHUB_ACTIONS=true 2024-06-26T05:02:24.4975922Z SHA1=b8c4c54d347aa776934c60784e35936878ef18dc 2024-06-26T05:02:24.4976547Z GITHUB_SHA=4b51b1a62a63a1add1b3a0f7882f5c7dc66b8f8d 2024-06-26T05:02:24.4977389Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/129470/merge 2024-06-26T05:02:24.4978266Z UCC_HOME=/usr 2024-06-26T05:02:24.4978690Z VERBOSE_TEST_LOGS=False 2024-06-26T05:02:24.4979148Z GITHUB_REF=refs/pull/129470/merge 2024-06-26T05:02:24.4979504Z SHARD_NUMBER=1 2024-06-26T05:02:24.4979775Z GITHUB_REF_PROTECTED=false 2024-06-26T05:02:24.4980268Z HOME=/var/lib/jenkins 2024-06-26T05:02:24.4980617Z GITHUB_API_URL=https://api.github.com 2024-06-26T05:02:24.4981031Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-06-26T05:02:24.4981413Z UCX_COMMIT= 2024-06-26T05:02:24.4981688Z SCCACHE_S3_KEY_PREFIX=pull 2024-06-26T05:02:24.4982081Z NUM_TEST_SHARDS=2 2024-06-26T05:02:24.4982455Z UCX_HOME=/usr 2024-06-26T05:02:24.4983590Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_92007462-5464-47ab-b3e4-9a93818191f3 2024-06-26T05:02:24.4984707Z JOB_NAME=linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, linux.2xlarge) 2024-06-26T05:02:24.4985817Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_92007462-5464-47ab-b3e4-9a93818191f3 2024-06-26T05:02:24.4986970Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-06-26T05:02:24.4987634Z GITHUB_EVENT_NAME=pull_request 2024-06-26T05:02:24.4987965Z DASHBOARD_TAG= 2024-06-26T05:02:24.4988244Z GITHUB_RUN_ID=9673645538 2024-06-26T05:02:24.4989163Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_92007462-5464-47ab-b3e4-9a93818191f3 2024-06-26T05:02:24.4990083Z GITHUB_ACTOR=leslie-fang-intel 2024-06-26T05:02:24.4990436Z PR_NUMBER=129470 2024-06-26T05:02:24.4990711Z DESIRED_CUDA= 2024-06-26T05:02:24.4990968Z GITHUB_RUN_ATTEMPT=1 2024-06-26T05:02:24.4991256Z VALGRIND=ON 2024-06-26T05:02:24.4991527Z ANACONDA_PYTHON_VERSION=3.8 2024-06-26T05:02:24.4992031Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-06-26T05:02:24.4992480Z TERM=xterm 2024-06-26T05:02:24.4992744Z INSTALLED_VISION=yes 2024-06-26T05:02:24.4993033Z BRANCH=pull/129470 2024-06-26T05:02:24.4993339Z OPENSSL_ROOT_DIR=/opt/openssl 2024-06-26T05:02:24.4993695Z CUDA_PATH=/usr/local/cuda 2024-06-26T05:02:24.4994446Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-06-26T05:02:24.4995208Z GITHUB_SERVER_URL=https://github.com 2024-06-26T05:02:24.4995594Z UCC_COMMIT= 2024-06-26T05:02:24.4995838Z REENABLED_ISSUES= 2024-06-26T05:02:24.4996113Z DOCS=yes 2024-06-26T05:02:24.4996362Z INSTALLED_ANDROID= 2024-06-26T05:02:24.4996622Z SHLVL=1 2024-06-26T05:02:24.4996853Z MAX_JOBS=6 2024-06-26T05:02:24.4997226Z GITHUB_ACTOR_ID=53841472 2024-06-26T05:02:24.4997666Z GITHUB_WORKFLOW_SHA=4b51b1a62a63a1add1b3a0f7882f5c7dc66b8f8d 2024-06-26T05:02:24.4998181Z GITHUB_REF_NAME=129470/merge 2024-06-26T05:02:24.4998787Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-06-26T05:02:24.4999350Z GITHUB_JOB=test 2024-06-26T05:02:24.4999634Z NO_TEST_TIMEOUT=False 2024-06-26T05:02:24.4999937Z TD_DISTRIBUTED=False 2024-06-26T05:02:24.5000244Z GITHUB_REPOSITORY=pytorch/pytorch 2024-06-26T05:02:24.5000616Z GITHUB_RETENTION_DAYS=90 2024-06-26T05:02:24.5000939Z OPENSSL_DIR=/opt/openssl 2024-06-26T05:02:24.5001250Z GITHUB_ACTION_REPOSITORY= 2024-06-26T05:02:24.5002307Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T05:02:24.5003528Z GITHUB_BASE_REF=gh/leslie-fang-intel/126/base 2024-06-26T05:02:24.5004236Z INSTALLED_ACL= 2024-06-26T05:02:24.5004529Z CI=true 2024-06-26T05:02:24.5004809Z GITHUB_REPOSITORY_OWNER=pytorch 2024-06-26T05:02:24.5005154Z JOB_ID=26688305322 2024-06-26T05:02:24.5005446Z INSTALLED_PROTOBUF=yes 2024-06-26T05:02:24.5005866Z GITHUB_HEAD_REF=gh/leslie-fang-intel/126/head 2024-06-26T05:02:24.5006275Z GITHUB_ACTION_REF= 2024-06-26T05:02:24.5006686Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-06-26T05:02:24.5007140Z GITHUB_WORKFLOW=pull 2024-06-26T05:02:24.5007442Z DEBIAN_FRONTEND=noninteractive 2024-06-26T05:02:24.5008457Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_92007462-5464-47ab-b3e4-9a93818191f3 2024-06-26T05:02:24.5009295Z NO_TD=False 2024-06-26T05:02:24.5009566Z SKIP_SCCACHE_INITIALIZATION=1 2024-06-26T05:02:24.5009915Z _=/usr/bin/env 2024-06-26T05:02:24.5010369Z + echo 'Testing pytorch' 2024-06-26T05:02:24.5010680Z Testing pytorch 2024-06-26T05:02:24.5010981Z + export LANG=C.UTF-8 2024-06-26T05:02:24.5011299Z + LANG=C.UTF-8 2024-06-26T05:02:24.5018154Z + PR_NUMBER=129470 2024-06-26T05:02:24.5018793Z + [[ distributed == \d\e\f\a\u\l\t ]] 2024-06-26T05:02:24.5019309Z + [[ distributed == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-06-26T05:02:24.5019825Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-06-26T05:02:24.5020254Z + [[ distributed == \s\l\o\w ]] 2024-06-26T05:02:24.5020744Z + [[ linux-jammy-py3.8-gcc11 == *slow-gradcheck* ]] 2024-06-26T05:02:24.5021282Z + [[ linux-jammy-py3.8-gcc11 == *cuda* ]] 2024-06-26T05:02:24.5021755Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-06-26T05:02:24.5022315Z + [[ linux-jammy-py3.8-gcc11 == *xpu* ]] 2024-06-26T05:02:24.5022738Z + [[ distributed == *crossref* ]] 2024-06-26T05:02:24.5023166Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-06-26T05:02:24.5023648Z + [[ linux-jammy-py3.8-gcc11 == *xpu* ]] 2024-06-26T05:02:24.5024144Z + [[ linux-jammy-py3.8-gcc11 != *-bazel-* ]] 2024-06-26T05:02:24.5024611Z + pip_install --user ninja==1.10.2 2024-06-26T05:02:24.5025123Z + pip install --progress-bar off --user ninja==1.10.2 2024-06-26T05:02:24.9099929Z Collecting ninja==1.10.2 2024-06-26T05:02:24.9263257Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-06-26T05:02:24.9413952Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-06-26T05:02:26.0843480Z Installing collected packages: ninja 2024-06-26T05:02:26.0926570Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-06-26T05:02:26.0927877Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-06-26T05:02:26.0973088Z Successfully installed ninja-1.10.2 2024-06-26T05:02:26.1706975Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T05:02:26.1709536Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T05:02:26.1710996Z + [[ linux-jammy-py3.8-gcc11 == *aarch64* ]] 2024-06-26T05:02:26.1711414Z + install_tlparse 2024-06-26T05:02:26.1711840Z + pip_install --user tlparse==0.3.7 2024-06-26T05:02:26.1712377Z + pip install --progress-bar off --user tlparse==0.3.7 2024-06-26T05:02:26.5695627Z Collecting tlparse==0.3.7 2024-06-26T05:02:26.5871456Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-06-26T05:02:26.5991202Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-06-26T05:02:27.7788691Z Installing collected packages: tlparse 2024-06-26T05:02:27.8197147Z Successfully installed tlparse-0.3.7 2024-06-26T05:02:27.8953626Z ++ python -m site --user-base 2024-06-26T05:02:27.9165044Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-06-26T05:02:27.9166764Z + [[ linux-jammy-py3.8-gcc11 == *asan* ]] 2024-06-26T05:02:27.9167444Z + [[ linux-jammy-py3.8-gcc11 == *-debug* ]] 2024-06-26T05:02:27.9167995Z + [[ linux-jammy-py3.8-gcc11 != *-bazel-* ]] 2024-06-26T05:02:27.9168837Z + echo 'We are not in debug mode: linux-jammy-py3.8-gcc11. Expect the assertion to pass' 2024-06-26T05:02:27.9170407Z We are not in debug mode: linux-jammy-py3.8-gcc11. Expect the assertion to pass 2024-06-26T05:02:27.9171425Z + cd test 2024-06-26T05:02:27.9172282Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-06-26T05:02:29.2809341Z + [[ distributed == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-06-26T05:02:29.2810292Z + [[ distributed == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-06-26T05:02:29.2810907Z + [[ linux-jammy-py3.8-gcc11 != *-bazel-* ]] 2024-06-26T05:02:29.2811363Z + pushd test 2024-06-26T05:02:29.2811688Z ~/workspace/test ~/workspace 2024-06-26T05:02:29.2812379Z ++ python -c 'import torch; print(torch.version.cuda)' 2024-06-26T05:02:30.6201670Z + CUDA_VERSION=None 2024-06-26T05:02:30.6202357Z + '[' None == 12.4 ']' 2024-06-26T05:02:30.6202658Z + ISCUDA124= 2024-06-26T05:02:30.6202933Z + popd 2024-06-26T05:02:30.6203171Z ~/workspace 2024-06-26T05:02:30.6207679Z + DYNAMO_BENCHMARK_FLAGS=() 2024-06-26T05:02:30.6208365Z + [[ distributed == *dynamo_eager* ]] 2024-06-26T05:02:30.6209058Z + [[ distributed == *aot_eager* ]] 2024-06-26T05:02:30.6209449Z + [[ distributed == *aot_inductor* ]] 2024-06-26T05:02:30.6209857Z + [[ distributed == *inductor* ]] 2024-06-26T05:02:30.6210246Z + [[ distributed == *dynamic* ]] 2024-06-26T05:02:30.6210643Z + [[ distributed == *cpu_inductor* ]] 2024-06-26T05:02:30.6211185Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-06-26T05:02:30.6240775Z + [[ linux-jammy-py3.8-gcc11 == *libtorch* ]] 2024-06-26T05:02:30.6241364Z + [[ linux-jammy-py3.8-gcc11 == *-bazel-* ]] 2024-06-26T05:02:30.6244179Z + cd test 2024-06-26T05:02:30.6244657Z + python -c 'import torch; print(torch.__config__.show())' 2024-06-26T05:02:31.7648283Z PyTorch built with: 2024-06-26T05:02:31.7649037Z - GCC 11.4 2024-06-26T05:02:31.7649357Z - C++ Version: 201703 2024-06-26T05:02:31.7650247Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T05:02:31.7651341Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T05:02:31.7652004Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T05:02:31.7652498Z - LAPACK is enabled (usually provided by MKL) 2024-06-26T05:02:31.7652953Z - NNPACK is enabled 2024-06-26T05:02:31.7653315Z - CPU capability usage: AVX512 2024-06-26T05:02:31.7661397Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-06-26T05:02:31.7668709Z 2024-06-26T05:02:31.9701263Z + cd test 2024-06-26T05:02:31.9702235Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-06-26T05:02:33.0898025Z ATen/Parallel: 2024-06-26T05:02:33.0898710Z at::get_num_threads() : 4 2024-06-26T05:02:33.0899400Z at::get_num_interop_threads() : 4 2024-06-26T05:02:33.0900096Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-06-26T05:02:33.0900492Z omp_get_max_threads() : 4 2024-06-26T05:02:33.0901471Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-06-26T05:02:33.0902283Z mkl_get_max_threads() : 4 2024-06-26T05:02:33.0902899Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-06-26T05:02:33.0903526Z std::thread::hardware_concurrency() : 8 2024-06-26T05:02:33.0904225Z Environment variables: 2024-06-26T05:02:33.0904556Z OMP_NUM_THREADS : [not set] 2024-06-26T05:02:33.0904910Z MKL_NUM_THREADS : [not set] 2024-06-26T05:02:33.0905259Z ATen parallel backend: OpenMP 2024-06-26T05:02:33.0905530Z 2024-06-26T05:02:33.2834820Z + [[ linux-jammy-py3.8-gcc11 == *aarch64* ]] 2024-06-26T05:02:33.2835323Z + [[ distributed == *backward* ]] 2024-06-26T05:02:33.2835765Z + [[ distributed == *xla* ]] 2024-06-26T05:02:33.2836146Z + [[ distributed == *executorch* ]] 2024-06-26T05:02:33.2836551Z + [[ distributed == \j\i\t\_\l\e\g\a\c\y ]] 2024-06-26T05:02:33.2837076Z + [[ linux-jammy-py3.8-gcc11 == *libtorch* ]] 2024-06-26T05:02:33.2837530Z + [[ distributed == distributed ]] 2024-06-26T05:02:33.2837893Z + test_distributed 2024-06-26T05:02:33.2838264Z + echo 'Testing distributed python tests' 2024-06-26T05:02:33.2838695Z Testing distributed python tests 2024-06-26T05:02:33.2839276Z + python test/run_test.py --distributed-tests --shard 1 2 --verbose 2024-06-26T05:02:33.3821981Z test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-06-26T05:02:33.3822992Z import pkg_resources 2024-06-26T05:02:36.7965444Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-06-26T05:02:36.8409542Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-06-26T05:02:36.8882042Z Ignoring disabled issues: [''] 2024-06-26T05:02:36.8983679Z Found test times from artifacts 2024-06-26T05:02:36.9372463Z Found test times from artifacts 2024-06-26T05:02:36.9386974Z Running all tests 2024-06-26T05:02:36.9513083Z Running parallel tests on 3 processes 2024-06-26T05:02:36.9520288Z Name: tests to run (est. time: 57.16min) 2024-06-26T05:02:36.9521019Z Serial tests (115): 2024-06-26T05:02:36.9521637Z distributed/test_inductor_collectives 1/1 2024-06-26T05:02:36.9522562Z distributed/test_dynamo_distributed 1/1 2024-06-26T05:02:36.9523450Z distributed/pipelining/test_backward 1/1 2024-06-26T05:02:36.9524232Z distributed/checkpoint/test_nested_dict 1/1 2024-06-26T05:02:36.9525443Z distributed/_composable/fsdp/test_fully_shard_training 1/1 2024-06-26T05:02:36.9526568Z distributed/_shard/sharded_tensor/test_logger 1/1 2024-06-26T05:02:36.9527413Z distributed/test_symmetric_memory 1/1 2024-06-26T05:02:36.9528172Z distributed/_spmd/test_graph_utils 1/1 2024-06-26T05:02:36.9528976Z distributed/_tensor/test_dtensor_compile 1/1 2024-06-26T05:02:36.9529874Z distributed/_tensor/test_dtensor_ops 1/1 2024-06-26T05:02:36.9530844Z distributed/tensor/parallel/test_micro_pipeline_tp 1/1 2024-06-26T05:02:36.9531766Z distributed/pipelining/test_pipe 1/1 2024-06-26T05:02:36.9532298Z distributed/fsdp/test_fsdp_fx 1/1 2024-06-26T05:02:36.9532985Z distributed/test_c10d_functional_native 1/1 2024-06-26T05:02:36.9533660Z distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 2024-06-26T05:02:36.9534604Z distributed/_tensor/experimental/test_tp_transform 1/1 2024-06-26T05:02:36.9535133Z distributed/_tensor/test_common_rules 1/1 2024-06-26T05:02:36.9535613Z distributed/_tensor/test_random_ops 1/1 2024-06-26T05:02:36.9536092Z distributed/_spmd/test_transformation 1/1 2024-06-26T05:02:36.9536571Z distributed/checkpoint/test_save_load_api 1/1 2024-06-26T05:02:36.9537103Z distributed/checkpoint/test_fsdp_optim_state 1/1 2024-06-26T05:02:36.9537624Z distributed/pipelining/test_unflatten 1/1 2024-06-26T05:02:36.9538102Z distributed/checkpoint/test_format_utils 1/1 2024-06-26T05:02:36.9538573Z distributed/_tensor/test_init 1/1 2024-06-26T05:02:36.9539019Z distributed/pipelining/test_microbatch 1/1 2024-06-26T05:02:36.9539502Z distributed/_tensor/test_convolution_ops 1/1 2024-06-26T05:02:36.9540011Z distributed/checkpoint/test_tp_checkpoint 1/1 2024-06-26T05:02:36.9540683Z distributed/test_device_mesh 1/1 2024-06-26T05:02:36.9541175Z distributed/_composable/fsdp/test_fully_shard_logging 1/1 2024-06-26T05:02:36.9541730Z distributed/checkpoint/test_traverse 1/1 2024-06-26T05:02:36.9542283Z distributed/_composable/fsdp/test_fully_shard_compile 1/1 2024-06-26T05:02:36.9542861Z distributed/_tensor/experimental/test_local_map 1/1 2024-06-26T05:02:36.9543395Z distributed/_tools/test_memory_tracker 1/1 2024-06-26T05:02:36.9543866Z distributed/_tensor/test_op_strategy 1/1 2024-06-26T05:02:36.9544375Z distributed/checkpoint/e2e/test_e2e_save_and_load 1/1 2024-06-26T05:02:36.9544913Z distributed/pipelining/test_schedule 1/1 2024-06-26T05:02:36.9545369Z distributed/_spmd/test_tracing 1/1 2024-06-26T05:02:36.9545844Z distributed/checkpoint/test_hsdp_checkpoint 1/1 2024-06-26T05:02:36.9546349Z distributed/_tensor/test_embedding_ops 1/1 2024-06-26T05:02:36.9546861Z distributed/checkpoint/test_state_dict_utils 1/1 2024-06-26T05:02:36.9547484Z distributed/tensor/parallel/test_fsdp_2d_parallel 1/1 2024-06-26T05:02:36.9548023Z distributed/elastic/test_control_plane 1/1 2024-06-26T05:02:36.9548510Z distributed/checkpoint/test_compatibility 1/1 2024-06-26T05:02:36.9549113Z distributed/_composable/fully_shard/test_fully_shard_util 1/1 2024-06-26T05:02:36.9549983Z distributed/_composable/fsdp/test_fully_shard_frozen 1/1 2024-06-26T05:02:36.9550529Z distributed/_composable/test_replicate 1/1 2024-06-26T05:02:36.9551003Z distributed/test_distributed_spawn 2/3 2024-06-26T05:02:36.9551552Z distributed/_composable/fsdp/test_fully_shard_state_dict 1/1 2024-06-26T05:02:36.9552201Z distributed/fsdp/test_fsdp_pure_fp16 1/1 2024-06-26T05:02:36.9552787Z distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_ 1/1 2024-06-26T05:02:36.9553386Z distributed/_composable/test_compose 1/1 2024-06-26T05:02:36.9553995Z distributed/_composable/fully_shard/test_fully_shard_mixed_precision 1/1 2024-06-26T05:02:36.9554621Z distributed/_tensor/test_view_ops 1/1 2024-06-26T05:02:36.9555054Z distributed/_tensor/test_utils 1/1 2024-06-26T05:02:36.9555468Z distributed/checkpoint/test_utils 1/1 2024-06-26T05:02:36.9555887Z distributed/test_nccl 1/1 2024-06-26T05:02:36.9556487Z distributed/_composable/fsdp/test_fully_shard_mixed_precision 1/1 2024-06-26T05:02:36.9557060Z distributed/_tensor/test_optimizers 1/1 2024-06-26T05:02:36.9557550Z distributed/fsdp/test_fsdp_ignored_modules 1/1 2024-06-26T05:02:36.9558025Z distributed/test_fake_pg 1/1 2024-06-26T05:02:36.9558512Z distributed/_composable/fully_shard/test_fully_shard_init 1/1 2024-06-26T05:02:36.9559062Z distributed/launcher/test_run 1/1 2024-06-26T05:02:36.9559534Z distributed/fsdp/test_fsdp_dtensor_state_dict 1/1 2024-06-26T05:02:36.9560069Z distributed/fsdp/test_fsdp_backward_prefetch 1/1 2024-06-26T05:02:36.9560589Z distributed/fsdp/test_fsdp_optim_state 1/1 2024-06-26T05:02:36.9561052Z distributed/rpc/test_share_memory 1/1 2024-06-26T05:02:36.9561666Z distributed/_composable/fully_shard/test_fully_shard_model_checkpoint 1/1 2024-06-26T05:02:36.9562308Z distributed/fsdp/test_fsdp_flatten_params 1/1 2024-06-26T05:02:36.9562799Z distributed/nn/jit/test_instantiator 1/1 2024-06-26T05:02:36.9563314Z distributed/fsdp/test_distributed_checkpoint 1/1 2024-06-26T05:02:36.9563807Z distributed/fsdp/test_fsdp_overlap 1/1 2024-06-26T05:02:36.9564397Z distributed/fsdp/test_fsdp_memory 1/1 2024-06-26T05:02:36.9564881Z distributed/fsdp/test_fsdp_freezing_weights 1/1 2024-06-26T05:02:36.9565363Z distributed/fsdp/test_fsdp_input 1/1 2024-06-26T05:02:36.9565802Z distributed/fsdp/test_fsdp_apply 1/1 2024-06-26T05:02:36.9566294Z distributed/fsdp/test_fsdp_multiple_wrapping 1/1 2024-06-26T05:02:36.9566798Z distributed/fsdp/test_fsdp_state_dict 1/1 2024-06-26T05:02:36.9567257Z distributed/fsdp/test_fsdp_uneven 1/1 2024-06-26T05:02:36.9567816Z distributed/_composable/fully_shard/test_fully_shard_runtime 1/1 2024-06-26T05:02:36.9568529Z distributed/fsdp/test_fsdp_comm_hooks 1/1 2024-06-26T05:02:36.9569067Z distributed/_composable/fsdp/test_fully_shard_comm 1/1 2024-06-26T05:02:36.9569617Z distributed/fsdp/test_checkpoint_wrapper 1/1 2024-06-26T05:02:36.9570190Z distributed/checkpoint/test_file_system_checkpoint_cpu 1/1 2024-06-26T05:02:36.9570822Z distributed/_composable/test_replicate_with_compiler 1/1 2024-06-26T05:02:36.9571404Z distributed/fsdp/test_fsdp_sharded_grad_scaler 1/1 2024-06-26T05:02:36.9571950Z distributed/fsdp/test_fsdp_checkpoint 1/1 2024-06-26T05:02:36.9572686Z distributed/fsdp/test_wrap 1/1 2024-06-26T05:02:36.9573485Z distributed/_composable/fsdp/test_fully_shard_extensions 1/1 2024-06-26T05:02:36.9574330Z distributed/fsdp/test_fsdp_tp_integration 1/1 2024-06-26T05:02:36.9575001Z distributed/fsdp/test_fsdp_traversal 1/1 2024-06-26T05:02:36.9575672Z distributed/fsdp/test_fsdp_fine_tune 1/1 2024-06-26T05:02:36.9576339Z distributed/test_c10d_object_collectives 1/1 2024-06-26T05:02:36.9576988Z distributed/_shard/test_sharder 1/1 2024-06-26T05:02:36.9577700Z distributed/_shard/sharded_optim/test_sharded_optim 1/1 2024-06-26T05:02:36.9578685Z distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint 1/1 2024-06-26T05:02:36.9579619Z distributed/fsdp/test_fsdp_multiple_forward 1/1 2024-06-26T05:02:36.9580344Z distributed/fsdp/test_fsdp_meta 1/1 2024-06-26T05:02:36.9581160Z distributed/_shard/sharding_spec/test_sharding_spec 1/1 2024-06-26T05:02:36.9582051Z distributed/fsdp/test_fsdp_unshard_params 1/1 2024-06-26T05:02:36.9582790Z distributed/rpc/test_tensorpipe_agent 2/3 2024-06-26T05:02:36.9583504Z distributed/rpc/cuda/test_tensorpipe_agent 1/1 2024-06-26T05:02:36.9584233Z distributed/test_pg_wrapper 1/1 2024-06-26T05:02:36.9584891Z distributed/algorithms/test_join 1/1 2024-06-26T05:02:36.9585575Z distributed/rpc/test_faulty_agent 1/1 2024-06-26T05:02:36.9586460Z distributed/_shard/sharded_tensor/test_sharded_tensor_reshard 1/1 2024-06-26T05:02:36.9587498Z distributed/_shard/sharded_tensor/ops/test_embedding 1/1 2024-06-26T05:02:36.9588575Z distributed/_shard/sharded_tensor/ops/test_embedding_bag 1/1 2024-06-26T05:02:36.9589818Z distributed/optim/test_zero_redundancy_optimizer 1/1 2024-06-26T05:02:36.9590677Z distributed/fsdp/test_fsdp_comm 1/1 2024-06-26T05:02:36.9591551Z distributed/_shard/sharded_tensor/ops/test_tensor_ops 1/1 2024-06-26T05:02:36.9592698Z distributed/_shard/sharded_tensor/ops/test_binary_cmp 1/1 2024-06-26T05:02:36.9593595Z distributed/checkpoint/test_state_dict 1/1 2024-06-26T05:02:36.9594335Z distributed/test_c10d_ucc 1/1 2024-06-26T05:02:36.9594959Z distributed/test_c10d_gloo 1/1 2024-06-26T05:02:36.9595648Z distributed/test_c10d_ops_nccl 1/1 2024-06-26T05:02:36.9596307Z Parallel tests (0): 2024-06-26T05:02:36.9596826Z Name: excluded (est. time: 0.0min) 2024-06-26T05:02:36.9597408Z Serial tests (0): 2024-06-26T05:02:36.9597866Z Parallel tests (0): 2024-06-26T05:02:36.9598801Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-06-26T05:02:36.9705168Z Running distributed/test_inductor_collectives 1/1 ... [2024-06-26 05:02:36.970111] 2024-06-26T05:02:36.9710892Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_inductor_collectives.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:02:36.970704] 2024-06-26T05:02:40.8168161Z 2024-06-26T05:02:40.8170180Z distributed/test_inductor_collectives 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_inductor_collectives_1.1_f8216589424c9da1_.log 2024-06-26T05:02:40.8202953Z Running 47 items in this shard: test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_all_to_all_single_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_all_to_all_single_inductor_split_sizes_none, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allgather_contiguous_input, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allgather_into_tensor_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allgather_output_buffer_reuse, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allreduce_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allreduce_inductor_cudagraph_trees, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_allreduce_input_buffer_reuse, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_broadcast_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_c10d_functional_tagged_pt2_compliant, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_eager_allreduce_inductor_wait, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_inductor_allreduce_eager_wait, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_permute_tensor, test/distributed/test_inductor_collectives.py::TestCollectivesMultiProc::test_reduce_scatter_tensor_inductor, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_backwards, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_get_world_group_source_GroupMember_WORLD, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_get_world_group_source__get_default_group, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_get_world_group_source_group_WORLD, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_graphbreaks_unsupported_async_op, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_pg_var, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_gather, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_gather_args_match, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_gather_list, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_all_to_all_single, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_kwargs, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_kwargs_none, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_positional, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_positional_none, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_pg_mode_unspecified, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op0, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op1, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op2, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op3, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_allreduce_reduce_op_reduce_op4, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_rewrite_dist_reduce_scatter, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_support_collective_op_with_async_op_False, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_all_gather_tensor, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_all_gather_tensor_pg, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_allgather_coalesced, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_allreduce, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_dynamo_trace_reduce_scatter_tensor, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_all_gather_coalesced, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_doesnt_mutate_shared, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_reduce_scatter_coalesced, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_single_op, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_inductor_steal_buffer, test/distributed/test_inductor_collectives.py::TestCollectivesInductor::test_meta 2024-06-26T05:02:40.8232959Z 2024-06-26T05:02:40.8233445Z Running distributed/test_dynamo_distributed 1/1 ... [2024-06-26 05:02:40.817596] 2024-06-26T05:02:40.8235448Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_dynamo_distributed.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:02:40.817990] 2024-06-26T05:03:01.3696656Z 2024-06-26T05:03:01.3698255Z distributed/test_dynamo_distributed 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_dynamo_distributed_1.1_2ff584d369a0c56e_.log 2024-06-26T05:03:01.3719593Z Running 38 items in this shard: test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_call_method_forward, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_hf_bert_ddp_aot_eager, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_hf_bert_ddp_inductor, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_issue90375, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_symbol_splitting, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_ddp_activation_checkpointing, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_ddp_baseline_aot_eager_multiprocess, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_activation_checkpointing, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_aot_eager, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_inductor, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_setattr, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_aot_eager, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_aot_eager_static_graph, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_inductor, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_inductor_static_graph, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_fsdp, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_fsdp_activation_checkpointing, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_aot_autograd, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_async_subclass_no_specialize, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_custom_layer, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_ddp_baseline_aot_eager, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_ddp_baseline_inductor, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_empty_graph_inductor, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_dup_tensors_diff_source, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_dup_tensors_same_source, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_orig_params_assert, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_skip_guards, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_skip_register_attr_or_module, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_staticmethod, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_ctx_manager, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor_layout_optimizations_inference, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor_layout_optimizations_training, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor_transpose, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_higher_order_op, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_ignored_parameters, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_no_split 2024-06-26T05:03:01.3743064Z 2024-06-26T05:03:01.3743614Z Running distributed/pipelining/test_backward 1/1 ... [2024-06-26 05:03:01.370213] 2024-06-26T05:03:01.3745656Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_backward.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:03:01.370634] 2024-06-26T05:03:04.2941980Z 2024-06-26T05:03:04.2944468Z distributed/pipelining/test_backward 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_backward_1.1_466c97da96aa55aa_.log 2024-06-26T05:03:04.2946215Z Running 1 items in this shard: test/distributed/pipelining/test_backward.py::StageBackwardTests::test_stage_backward 2024-06-26T05:03:04.2947011Z 2024-06-26T05:03:04.2949274Z Running distributed/checkpoint/test_nested_dict 1/1 ... [2024-06-26 05:03:04.294667] 2024-06-26T05:03:04.2967613Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_nested_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:03:04.296238] 2024-06-26T05:03:07.1200411Z 2024-06-26T05:03:07.1202357Z distributed/checkpoint/test_nested_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_nested_dict_1.1_93b5db7937124748_.log 2024-06-26T05:03:07.1204965Z Running 2 items in this shard: test/distributed/checkpoint/test_nested_dict.py::TestFlattening::test_flattening_round_trip, test/distributed/checkpoint/test_nested_dict.py::TestFlattening::test_mapping 2024-06-26T05:03:07.1206389Z 2024-06-26T05:03:07.1207330Z Running distributed/_composable/fsdp/test_fully_shard_training 1/1 ... [2024-06-26 05:03:07.120518] 2024-06-26T05:03:07.1212511Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_training.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:03:07.120934] 2024-06-26T05:03:09.9942095Z 2024-06-26T05:03:09.9943977Z distributed/_composable/fsdp/test_fully_shard_training 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_training_1.1_4194bf092aea2f38_.log 2024-06-26T05:03:09.9961356Z Running 22 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_training.py::test_compiled_fsdp, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardForwardInputs::test_root_move_forward_input_to_device, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardRegisteredParams::test_param_registration_after_backward, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardRegisteredParams::test_param_registration_after_forward, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardCastAfterInit::test_to_float64_after_init, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_explicit_prefetching, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_multi_forward_module, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_non_root_forward_backward, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_post_optim_event, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_multi_group, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_multi_group_cpu_offload_eager, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_single_group, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCompose::test_train_parity_with_activation_checkpointing, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardSharedParams::test_train_parity_with_shared_params, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardGradientAccumulation::test_1f1b_microbatching, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardGradientAccumulation::test_gradient_accumulation, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard2DTraining::test_tp_with_fsdp_offloading, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard2DTraining::test_train_parity_2d_mlp, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard2DTraining::test_train_parity_2d_transformer_checkpoint_resume, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardNDTraining::test_2d_mlp_with_nd_mesh, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardHSDPTraining::test_train_parity_hsdp, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardCustomForwardMethod::test_register_fsdp_forward_method 2024-06-26T05:03:09.9976944Z 2024-06-26T05:03:09.9977475Z Running distributed/_shard/sharded_tensor/test_logger 1/1 ... [2024-06-26 05:03:09.994683] 2024-06-26T05:03:09.9979557Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/test_logger.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:03:09.995088] 2024-06-26T05:03:12.7683068Z 2024-06-26T05:03:12.7685817Z distributed/_shard/sharded_tensor/test_logger 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.test_logger_1.1_507f67178a63ff85_.log 2024-06-26T05:03:12.7687733Z Running 1 items in this shard: test/distributed/_shard/sharded_tensor/test_logger.py::ShardingSpecLoggerTest::test_get_or_create_logger 2024-06-26T05:03:12.7688686Z 2024-06-26T05:03:12.7690084Z Running distributed/test_symmetric_memory 1/1 ... [2024-06-26 05:03:12.768778] 2024-06-26T05:03:12.7695460Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_symmetric_memory.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:03:12.769234] 2024-06-26T05:03:15.2419502Z 2024-06-26T05:03:15.2421353Z distributed/test_symmetric_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_symmetric_memory_1.1_5de87428d7768a0d_.log 2024-06-26T05:03:15.2423856Z Running 2 items in this shard: test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_empty_strided_p2p, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_empty_strided_p2p_persistent 2024-06-26T05:03:15.2425239Z 2024-06-26T05:03:15.2425656Z Running distributed/_spmd/test_graph_utils 1/1 ... [2024-06-26 05:03:15.242377] 2024-06-26T05:03:15.2430896Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_spmd/test_graph_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:03:15.242796] 2024-06-26T05:03:17.5861107Z 2024-06-26T05:03:17.5862902Z distributed/_spmd/test_graph_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._spmd.test_graph_utils_1.1_2198ea0818db514b_.log 2024-06-26T05:03:17.5863997Z 2024-06-26T05:03:17.5866618Z Running distributed/_tensor/test_dtensor_compile 1/1 ... [2024-06-26 05:03:17.586462] 2024-06-26T05:03:17.5871836Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_dtensor_compile.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:03:17.586889] 2024-06-26T05:03:37.7948574Z 2024-06-26T05:03:37.7951676Z distributed/_tensor/test_dtensor_compile 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_dtensor_compile_1.1_767ce764f1e40be9_.log 2024-06-26T05:03:37.8065886Z Running 24 items in this shard: test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_device_mesh_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_attribute_access_on_intermediate, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_constructor_w_dynamo_disable, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_constructor_w_graph_break, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_contiguous_dtensor_noncontiguous_local_as_tangent, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_different_gradient_placement, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_dynamo_device_mesh_attrs, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_noncontiguous_output, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_partial_placement_graph_output, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_partial_placement_redistribute_unbalanced_correct_strides, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor_from_local, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor_from_local_redistribute, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_to_local_kwargs, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_to_local_kwargs_forward_hook, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_fakify_dtensor, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_graph_input_is_async, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_placement_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_tp_compile_comm_reordering, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_2d_fsdp_tp_ac_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_2d_fsdp_tp_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_compile_dtensor_redistribute_backward, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_tp_compile_fullgraph_is_seq_parallel_False, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_tp_compile_fullgraph_is_seq_parallel_True 2024-06-26T05:03:37.8095305Z 2024-06-26T05:03:37.8096199Z Running distributed/_tensor/test_dtensor_ops 1/1 ... [2024-06-26 05:03:37.796303] 2024-06-26T05:03:37.8099863Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_dtensor_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:03:37.796852] 2024-06-26T05:04:45.4136133Z 2024-06-26T05:04:45.4137954Z distributed/_tensor/test_dtensor_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_dtensor_ops_1.1_028c9fbd3c5f64b0_.log 2024-06-26T05:04:45.4862810Z Running 663 items in this shard: test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_H_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_T_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___getitem___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___radd___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rdiv___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rmatmul___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rmod___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rmul___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rpow___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rsub___cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__batch_norm_with_update_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__chunk_cat_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__native_batch_norm_legit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__segment_reduce_lengths_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__segment_reduce_offsets_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__softmax_backward_data_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__unsafe_masked_index_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__unsafe_masked_index_put_accumulate_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__upsample_bilinear2d_aa_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_abs_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_acos_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_acosh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_add_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addbmm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addcdiv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addcmul_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addmm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addmm_decomposed_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addmv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_alias_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_all_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_allclose_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_amax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_amin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_aminmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_angle_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_any_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_arange_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argmin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argsort_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argwhere_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_partial_views_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_asin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_asinh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atan2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atan_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atanh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atleast_1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atleast_2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atleast_3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_baddbmm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bernoulli_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bfloat16_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_block_diag_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bmm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bool_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_broadcast_shapes_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_broadcast_tensors_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_broadcast_to_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bucketize_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_byte_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cartesian_prod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cat_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cauchy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cdist_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cdouble_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ceil_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cfloat_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_chalf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_char_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cholesky_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cholesky_inverse_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cholesky_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_chunk_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clamp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clamp_max_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clamp_min_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clone_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_column_stack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_combinations_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_complex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_conj_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_conj_physical_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_constant_pad_nd_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_contiguous_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_copysign_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_corrcoef_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cos_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cosh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_count_nonzero_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cov_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cross_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cummax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cummin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cumprod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cumsum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cumulative_trapezoid_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_deg2rad_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diag_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diag_embed_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagflat_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagonal_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagonal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagonal_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diff_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_digamma_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dist_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_div_floor_rounding_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_div_no_rounding_mode_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_div_trunc_rounding_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_double_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dsplit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dstack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_einsum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_permuted_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_strided_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_eq_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_equal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_erf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_erfc_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_erfinv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_exp2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_exp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_expand_as_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_expand_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_expm1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_exponential_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_eye_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fftshift_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_hfft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_hfft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_hfftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifftshift_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ihfft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ihfft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ihfftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_irfft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_irfft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_irfftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_rfft2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_rfft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_rfftn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fill_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_flatten_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_flip_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fliplr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_flipud_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_float_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_float_power_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_floor_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_floor_divide_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fmin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fmod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_frac_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_frexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_full_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_full_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_gather_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ge_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_geometric_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_geqrf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_gradient_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_grid_sampler_2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_gt_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_half_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_heaviside_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_histc_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_histogram_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_histogramdd_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_hsplit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_hstack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_hypot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_i0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_igamma_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_igammac_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_add_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_fill_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_put_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_amax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_amin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_prod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_select_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_inner_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_int_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isclose_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isfinite_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isinf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isnan_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isneginf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isposinf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isreal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_item_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_2inputs_2outputs_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_4inputs_with_extra_args_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_binary_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_binary_return_by_ref_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_unary_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_kron_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_kthvalue_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ldexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_le_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lerp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lgamma_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cholesky_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cholesky_ex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cond_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cross_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_det_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_det_singular_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_diagonal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eig_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eigh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eigvals_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eigvalsh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_householder_product_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_inv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_inv_ex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_ldl_factor_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_ldl_factor_ex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_ldl_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lstsq_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lstsq_grad_oriented_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_factor_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_factor_ex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_power_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_rank_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_rank_hermitian_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_multi_dot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_norm_subgradients_at_zero_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_pinv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_pinv_hermitian_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_pinv_singular_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_qr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_slogdet_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_solve_ex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_solve_triangular_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_svd_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_svdvals_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_tensorinv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_tensorsolve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_vander_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_vecdot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_vector_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linspace_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linspace_tensor_overload_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log10_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log1p_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_normal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_softmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_softmax_with_dtype_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logaddexp2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logaddexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logcumsumexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logdet_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_and_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_not_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_or_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_xor_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logspace_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logspace_tensor_overload_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logsumexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_long_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lt_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lu_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lu_unpack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mH_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mT_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_amax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_amin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_argmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_argmin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_cumprod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_cumsum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_fill_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_log_softmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_logaddexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_logsumexp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_median_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_normalize_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_prod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_select_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_softmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_softmin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_std_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_sum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_var_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_matmul_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_matrix_exp_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_binary_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_pool2d_with_indices_backward_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_reduction_no_dim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_reduction_with_dim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_maximum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_median_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_meshgrid_list_of_tensors_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_meshgrid_variadic_tensors_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_min_binary_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_min_reduction_no_dim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_min_reduction_with_dim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_minimum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mode_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_movedim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_msort_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mul_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_multinomial_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mv_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mvlgamma_mvlgamma_p_1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mvlgamma_mvlgamma_p_3_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mvlgamma_mvlgamma_p_5_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nan_to_num_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nanmean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nanmedian_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nanquantile_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nansum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_narrow_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_narrow_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_native_batch_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_native_dropout_backward_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_native_layer_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ne_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_neg_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_empty_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_empty_strided_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_full_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_ones_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_zeros_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nextafter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_avg_pool1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_avg_pool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_avg_pool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_max_pool1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_max_pool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_max_pool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_alpha_dropout_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_avg_pool1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_avg_pool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_avg_pool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_batch_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_bilinear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_binary_cross_entropy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_binary_cross_entropy_with_logits_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_celu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_channel_shuffle_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv_transpose1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv_transpose2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv_transpose3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_cosine_embedding_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_cosine_similarity_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_cross_entropy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_ctc_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_dropout2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_dropout3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_dropout_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_elu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_embedding_bag_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_embedding_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_feature_alpha_dropout_with_train_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_feature_alpha_dropout_without_train_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_fractional_max_pool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_fractional_max_pool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_gaussian_nll_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_gelu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_glu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_grid_sample_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_group_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardshrink_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardsigmoid_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardswish_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardtanh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hinge_embedding_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_huber_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_instance_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_area_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_bicubic_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_bilinear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_linear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_nearest-exact_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_nearest_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_trilinear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_kl_div_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_l1_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_layer_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_leaky_relu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_linear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_local_response_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_logsigmoid_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_margin_ranking_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_pool1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_pool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_pool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool1d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool1d_grad_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool2d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool2d_grad_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool3d_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool3d_grad_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_mish_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_mse_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multi_head_attention_forward_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multi_margin_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multilabel_margin_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multilabel_soft_margin_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_nll_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_normalize_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_circular_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_constant_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_reflect_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_replicate_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_replicate_negative_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pairwise_distance_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pdist_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pixel_shuffle_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pixel_unshuffle_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_poisson_nll_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_prelu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_relu6_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_relu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_rms_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_rrelu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_scaled_dot_product_attention_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_selu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_silu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_smooth_l1_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_soft_margin_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softmin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softmin_with_dtype_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softplus_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softshrink_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softsign_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_tanhshrink_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_threshold_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_triplet_margin_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_triplet_margin_with_distance_loss_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_unfold_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_upsample_bilinear_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_upsample_nearest_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nonzero_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nonzero_static_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_fro_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_inf_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_nuc_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_normal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_normal_in_place_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_normal_number_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ones_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ones_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ormqr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_outer_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_pca_lowrank_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_permute_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_pinverse_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polar_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_2_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_3_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_4_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_positive_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_pow_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_prod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_put_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_qr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_quantile_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rad2deg_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rand_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randint_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randint_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randn_like_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ravel_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_real_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_reciprocal_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_remainder_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_renorm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_repeat_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_repeat_interleave_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_reshape_as_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_reshape_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resize__cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resize_as__cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resolve_conj_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resolve_neg_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_roll_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rot90_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_decimals_0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_decimals_3_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_decimals_neg_3_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rsqrt_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rsub_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scalar_tensor_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_add_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_amax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_amin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_prod_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_sum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_searchsorted_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_select_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_select_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sgn_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_short_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sigmoid_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sign_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_bartlett_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_blackman_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_cosine_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_exponential_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_gaussian_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_general_cosine_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_general_hamming_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_hamming_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_hann_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_kaiser_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_nuttall_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signbit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sin_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sinc_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sinh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_slice_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_slice_scatter_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_softmax_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_softmax_with_dtype_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sort_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sparse_mm_reduce_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sparse_sampled_addmm_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_airy_ai_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_j0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_j1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_y0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_y1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_t_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_u_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_v_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_w_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_entr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_erfcx_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_hermite_polynomial_h_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_hermite_polynomial_he_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_i0e_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_i1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_i1e_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_laguerre_polynomial_l_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_legendre_polynomial_p_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_log_ndtr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_i0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_i1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_k0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_k1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_ndtr_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_ndtri_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_polygamma_special_polygamma_n_0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_scaled_modified_bessel_k0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_scaled_modified_bessel_k1_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_t_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_u_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_v_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_w_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_spherical_bessel_j0_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_xlog1py_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_zeta_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_list_args_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_with_sizes_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_with_sizes_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sqrt_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_square_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_squeeze_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_squeeze_multiple_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_stack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_mean_unbiased_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_unbiased_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_stft_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sub_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sum_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sum_to_size_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_svd_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_svd_lowrank_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_t_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_take_along_dim_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_take_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tan_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tanh_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tensor_split_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tensordot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tile_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_to_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_to_sparse_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_topk_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trace_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_transpose_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trapezoid_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trapz_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_triangular_solve_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tril_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_triu_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_true_divide_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trunc_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unbind_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unflatten_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unfold_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unfold_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_uniform_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unique_consecutive_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unique_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unsafe_chunk_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unsafe_split_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unsqueeze_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_mean_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_mean_unbiased_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_unbiased_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_vdot_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_as_complex_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_as_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_copy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_vsplit_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_vstack_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_where_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_xlogy_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_zero__cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_zeros_cpu_float32, test/distributed/_tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_zeros_like_cpu_float32 2024-06-26T05:04:45.5241964Z 2024-06-26T05:04:45.5242605Z Running distributed/tensor/parallel/test_micro_pipeline_tp 1/1 ... [2024-06-26 05:04:45.507518] 2024-06-26T05:04:45.5244935Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_micro_pipeline_tp.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:04:45.507978] 2024-06-26T05:04:48.0307171Z 2024-06-26T05:04:48.0309073Z distributed/tensor/parallel/test_micro_pipeline_tp 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_micro_pipeline_tp_1.1_9eba7c3de3c4ab10_.log 2024-06-26T05:04:48.0322104Z Running 15 items in this shard: test/distributed/tensor/parallel/test_micro_pipeline_tp.py::test_with_non_cuda_p2p_group, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_dtensor_seq_par_shard_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_dtensor_seq_par_shard_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_2_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_2_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_2_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_3_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_3_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_3_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_2_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_2_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_2_scatter_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_3_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_3_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_3_scatter_dim_2 2024-06-26T05:04:48.0333335Z 2024-06-26T05:04:48.0333783Z Running distributed/pipelining/test_pipe 1/1 ... [2024-06-26 05:04:48.031135] 2024-06-26T05:04:48.0335756Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_pipe.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:04:48.031552] 2024-06-26T05:04:51.5568875Z 2024-06-26T05:04:51.5570443Z distributed/pipelining/test_pipe 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_pipe_1.1_41a561095c319a08_.log 2024-06-26T05:04:51.5573037Z Running 3 items in this shard: test/distributed/pipelining/test_pipe.py::PipeTests::test_model_split_ModelClass0, test/distributed/pipelining/test_pipe.py::PipeTests::test_model_split_ModelClass1, test/distributed/pipelining/test_pipe.py::PipeTests::test_model_split_ModelClass2 2024-06-26T05:04:51.5574749Z 2024-06-26T05:04:51.5575727Z Running distributed/fsdp/test_fsdp_fx 1/1 ... [2024-06-26 05:04:51.557364] 2024-06-26T05:04:51.5580628Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_fx.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:04:51.557772] 2024-06-26T05:04:54.4310444Z 2024-06-26T05:04:54.4312511Z distributed/fsdp/test_fsdp_fx 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_fx_1.1_2b28e4f776f41678_.log 2024-06-26T05:04:54.4314455Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_fx.py::TestSymbolicTracing::test_symbolic_tracing_outputs 2024-06-26T05:04:54.4315275Z 2024-06-26T05:04:54.4317148Z Running distributed/test_c10d_functional_native 1/1 ... [2024-06-26 05:04:54.431471] 2024-06-26T05:04:54.4322501Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_functional_native.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:04:54.431892] 2024-06-26T05:04:57.0049221Z 2024-06-26T05:04:57.0051796Z distributed/test_c10d_functional_native 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_functional_native_1.1_141eb86d6286dff2_.log 2024-06-26T05:04:57.0072503Z Running 23 items in this shard: test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_gather_into_tensor_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_gather_into_tensor_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_coalesced_, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_single_, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_to_all_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_broadcast, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_reduce_scatter_tensor_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_threading, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_unwaited, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_gather_into_tensor_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_gather_into_tensor_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_reduce_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_reduce_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_to_all_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_broadcast, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_inplace_op_on_view, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reduce_scatter_tensor_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reuse_buffer_after_inplace_collective, test/distributed/test_c10d_functional_native.py::CompileTest::test_ranks_and_tag 2024-06-26T05:04:57.0091485Z 2024-06-26T05:04:57.0092259Z Running distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 ... [2024-06-26 05:04:57.005653] 2024-06-26T05:04:57.0095235Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/fsdp/test_fsdp_dsd.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:04:57.006172] 2024-06-26T05:04:59.8803502Z 2024-06-26T05:04:59.8805679Z distributed/checkpoint/fsdp/test_fsdp_dsd 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.fsdp.test_fsdp_dsd_1.1_ad53db02073acef6_.log 2024-06-26T05:04:59.8809613Z Running 3 items in this shard: test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_1d_fsdp_cpu_offload_full_model_state_dict, test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_1d_fsdp_get_model_state_dict, test/distributed/checkpoint/fsdp/test_fsdp_dsd.py::TestFullyShardWithDistributedStateDict::test_save_with_fsdp1_and_load_with_fsdp2 2024-06-26T05:04:59.8812458Z 2024-06-26T05:04:59.8813003Z Running distributed/_tensor/experimental/test_tp_transform 1/1 ... [2024-06-26 05:04:59.880770] 2024-06-26T05:04:59.8815908Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/experimental/test_tp_transform.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:04:59.881178] 2024-06-26T05:05:13.0263770Z 2024-06-26T05:05:13.0265897Z distributed/_tensor/experimental/test_tp_transform 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.experimental.test_tp_transform_1.1_827300e754bf732f_.log 2024-06-26T05:05:13.0269285Z Running 3 items in this shard: test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_e2e, test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_no_bias, test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_with_uncovered_op 2024-06-26T05:05:13.0271813Z 2024-06-26T05:05:13.0272267Z Running distributed/_tensor/test_common_rules 1/1 ... [2024-06-26 05:05:13.026796] 2024-06-26T05:05:13.0275362Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_common_rules.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:05:13.027218] 2024-06-26T05:05:43.6580780Z 2024-06-26T05:05:43.6582769Z distributed/_tensor/test_common_rules 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_common_rules_1.1_ab8a228b06d5716b_.log 2024-06-26T05:05:43.6590522Z Running 10 items in this shard: test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_basic_propagation, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_errors, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_linearity, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_merge_sharding, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_multi_sharding_on_mesh_dim, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_einop_pointwise_propagation, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_enforce_sharding_multi_sharding_on_mesh_dim, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_multi_sharding_on_mesh_dim, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_rules_broadcasting, test/distributed/_tensor/test_common_rules.py::CommonRulesTest::test_pointwise_rules_suggestion 2024-06-26T05:05:43.6596662Z 2024-06-26T05:05:43.6597135Z Running distributed/_tensor/test_random_ops 1/1 ... [2024-06-26 05:05:43.658654] 2024-06-26T05:05:43.6599105Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_random_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:05:43.659096] 2024-06-26T05:06:05.3213804Z 2024-06-26T05:06:05.3215698Z distributed/_tensor/test_random_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_random_ops_1.1_112cec47a7bafcda_.log 2024-06-26T05:06:05.3221171Z Running 7 items in this shard: test/distributed/_tensor/test_random_ops.py::DistTensorRandomInitTest::test_init_ops, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_dropout_1d, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_rand_1d, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_uniform_2d, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_manual_seed, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_meta_tensor_init, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_rng_tracker_init 2024-06-26T05:06:05.3225172Z 2024-06-26T05:06:05.3225899Z Running distributed/_spmd/test_transformation 1/1 ... [2024-06-26 05:06:05.321901] 2024-06-26T05:06:05.3227910Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_spmd/test_transformation.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:06:05.322316] 2024-06-26T05:06:08.8568719Z 2024-06-26T05:06:08.8570671Z distributed/_spmd/test_transformation 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._spmd.test_transformation_1.1_a2199cd4073012b9_.log 2024-06-26T05:06:08.8571779Z 2024-06-26T05:06:08.8575813Z Running distributed/checkpoint/test_save_load_api 1/1 ... [2024-06-26 05:06:08.857295] 2024-06-26T05:06:08.8580957Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_save_load_api.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:06:08.857780] 2024-06-26T05:06:14.3350701Z 2024-06-26T05:06:14.3352539Z distributed/checkpoint/test_save_load_api 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_save_load_api_1.1_24729426d695c01a_.log 2024-06-26T05:06:14.3354502Z Running 1 items in this shard: test/distributed/checkpoint/test_save_load_api.py::TestSaveAndLoadAPI::test_auto_detect 2024-06-26T05:06:14.3355312Z 2024-06-26T05:06:14.3357190Z Running distributed/checkpoint/test_fsdp_optim_state 1/1 ... [2024-06-26 05:06:14.335506] 2024-06-26T05:06:14.3362390Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_fsdp_optim_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:06:14.335943] 2024-06-26T05:06:22.4704548Z 2024-06-26T05:06:22.4706617Z distributed/checkpoint/test_fsdp_optim_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_fsdp_optim_state_1.1_03abacb760056b83_.log 2024-06-26T05:06:22.4709490Z Running 2 items in this shard: test/distributed/checkpoint/test_fsdp_optim_state.py::FsdpOptimStateCheckpoint::test_load_sharded_optimizer_state_dict_pass_planner_False, test/distributed/checkpoint/test_fsdp_optim_state.py::FsdpOptimStateCheckpoint::test_load_sharded_optimizer_state_dict_pass_planner_True 2024-06-26T05:06:22.4711473Z 2024-06-26T05:06:22.4711996Z Running distributed/pipelining/test_unflatten 1/1 ... [2024-06-26 05:06:22.470910] 2024-06-26T05:06:22.4716431Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_unflatten.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:06:22.471343] 2024-06-26T05:06:25.7454666Z 2024-06-26T05:06:25.7456630Z distributed/pipelining/test_unflatten 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_unflatten_1.1_48a9806a7ed57725_.log 2024-06-26T05:06:25.7458349Z Running 1 items in this shard: test/distributed/pipelining/test_unflatten.py::UnflattenTests::test_unflatten 2024-06-26T05:06:25.7459034Z 2024-06-26T05:06:25.7460996Z Running distributed/checkpoint/test_format_utils 1/1 ... [2024-06-26 05:06:25.745864] 2024-06-26T05:06:25.7466203Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_format_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:06:25.746285] 2024-06-26T05:06:36.7366727Z 2024-06-26T05:06:36.7368731Z distributed/checkpoint/test_format_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_format_utils_1.1_4b274818ffa07382_.log 2024-06-26T05:06:36.7371955Z Running 3 items in this shard: test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_dcp_to_torch_save, test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_online_torch_save_to_dcp, test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_torch_save_to_dcp 2024-06-26T05:06:36.7374280Z 2024-06-26T05:06:36.7374693Z Running distributed/_tensor/test_init 1/1 ... [2024-06-26 05:06:36.737118] 2024-06-26T05:06:36.7378398Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_init.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:06:36.737517] 2024-06-26T05:06:59.0023764Z 2024-06-26T05:06:59.0025546Z distributed/_tensor/test_init 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_init_1.1_11acf8501941fd1a_.log 2024-06-26T05:06:59.0030539Z Running 7 items in this shard: test/distributed/_tensor/test_init.py::DTensorInitOpsTest::test_init_ops, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_empty, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_full, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_ones, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_zeros, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_zeros_full_mesh, test/distributed/_tensor/test_init.py::DTensorConstructorTest::test_zeros_submesh 2024-06-26T05:06:59.0034312Z 2024-06-26T05:06:59.0034999Z Running distributed/pipelining/test_microbatch 1/1 ... [2024-06-26 05:06:59.003111] 2024-06-26T05:06:59.0039070Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_microbatch.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:06:59.003596] 2024-06-26T05:07:02.2272296Z 2024-06-26T05:07:02.2274111Z distributed/pipelining/test_microbatch 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_microbatch_1.1_aed0bcda66c908d8_.log 2024-06-26T05:07:02.2276502Z Running 2 items in this shard: test/distributed/pipelining/test_microbatch.py::MicrobatchTests::test_chunk_spec, test/distributed/pipelining/test_microbatch.py::MicrobatchTests::test_split_and_merge 2024-06-26T05:07:02.2277921Z 2024-06-26T05:07:02.2279056Z Running distributed/_tensor/test_convolution_ops 1/1 ... [2024-06-26 05:07:02.227697] 2024-06-26T05:07:02.2284288Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_convolution_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:07:02.228108] 2024-06-26T05:07:22.8900619Z 2024-06-26T05:07:22.8902248Z distributed/_tensor/test_convolution_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_convolution_ops_1.1_c3b5382b719c7a58_.log 2024-06-26T05:07:22.8904629Z Running 2 items in this shard: test/distributed/_tensor/test_convolution_ops.py::DistConvolutionOpsTest::test_depthwise_convolution, test/distributed/_tensor/test_convolution_ops.py::DistConvolutionOpsTest::test_downsampling_convolution 2024-06-26T05:07:22.8906074Z 2024-06-26T05:07:22.8908454Z Running distributed/checkpoint/test_tp_checkpoint 1/1 ... [2024-06-26 05:07:22.890536] 2024-06-26T05:07:22.8912846Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_tp_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:07:22.890944] 2024-06-26T05:07:31.2255135Z 2024-06-26T05:07:31.2257634Z distributed/checkpoint/test_tp_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_tp_checkpoint_1.1_9be8fc9a57870ea0_.log 2024-06-26T05:07:31.2261095Z Running 2 items in this shard: test/distributed/checkpoint/test_tp_checkpoint.py::TestTpCheckpoint::test_tp_checkpoint, test/distributed/checkpoint/test_tp_checkpoint.py::TestTpCheckpoint::test_tp_checkpoint_load_on_meta_device 2024-06-26T05:07:31.2262502Z 2024-06-26T05:07:31.2262941Z Running distributed/test_device_mesh 1/1 ... [2024-06-26 05:07:31.225929] 2024-06-26T05:07:31.2266535Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_device_mesh.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:07:31.226345] 2024-06-26T05:09:42.8970935Z 2024-06-26T05:09:42.8973890Z distributed/test_device_mesh 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_device_mesh_1.1_f08377f13545d0d1_.log 2024-06-26T05:09:42.8993700Z Running 38 items in this shard: test/distributed/test_device_mesh.py::DeviceMeshTest::test_assert_invalid_mesh_tensor, test/distributed/test_device_mesh.py::DeviceMeshTest::test_device_mesh_2d, test/distributed/test_device_mesh.py::DeviceMeshTest::test_device_mesh_init_backend, test/distributed/test_device_mesh.py::DeviceMeshTest::test_fake_pg_device_mesh, test/distributed/test_device_mesh.py::DeviceMeshTest::test_from_group_with_global_pg, test/distributed/test_device_mesh.py::DeviceMeshTest::test_from_group_with_invalid_mesh, test/distributed/test_device_mesh.py::DeviceMeshTest::test_get_group_and_get_all_groups, test/distributed/test_device_mesh.py::DeviceMeshTest::test_get_local_rank, test/distributed/test_device_mesh.py::DeviceMeshTest::test_get_local_rank_raises_exception, test/distributed/test_device_mesh.py::DeviceMeshTest::test_init_process_group, test/distributed/test_device_mesh.py::DeviceMeshTest::test_raises_invalid_device_type, test/distributed/test_device_mesh.py::DeviceMeshTest::test_set_mesh_dim_group_options, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_device_mesh_hash, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_device_mesh_nd, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_device_mesh_parent_child_hash, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_from_group_with_mesh_shape, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_get_local_rank_3d, test/distributed/test_device_mesh.py::InitDeviceMeshTest::test_init_device_mesh, test/distributed/test_device_mesh.py::InitDeviceMeshTest::test_raises_duplicate_mesh_dim_names, test/distributed/test_device_mesh.py::InitDeviceMeshTest::test_raises_mesh_shape_mesh_dim_names_mismatch, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_cache_and_reuse_submesh_slice_result, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_1d, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_2d, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_3d, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_raises_invalid_mesh_dim_name, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_raises_no_mesh_dim_found, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_mesh_dim_by_name, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_parent_mesh, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_parent_mesh_dim_exist, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_parent_mesh_dim_not_exist, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_all_gather_uneven, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_broadcast_1d, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_broadcast_nd, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_reduce_scatter_contiguous, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_reduce_scatter_uneven, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_scatter_1d, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_scatter_nd, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_scatter_uneven 2024-06-26T05:09:42.9011742Z 2024-06-26T05:09:42.9012335Z Running distributed/_composable/fsdp/test_fully_shard_logging 1/1 ... [2024-06-26 05:09:42.897622] 2024-06-26T05:09:42.9014511Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_logging.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:09:42.898052] 2024-06-26T05:09:45.5466264Z 2024-06-26T05:09:45.5468953Z distributed/_composable/fsdp/test_fully_shard_logging 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_logging_1.1_994b4993c41d43c3_.log 2024-06-26T05:09:45.5471346Z Running 0 items in this shard: 2024-06-26T05:09:45.5472613Z 2024-06-26T05:09:45.5473393Z Running distributed/checkpoint/test_traverse 1/1 ... [2024-06-26 05:09:45.547086] 2024-06-26T05:09:45.5479569Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_traverse.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:09:45.547517] 2024-06-26T05:09:48.3718260Z 2024-06-26T05:09:48.3720142Z distributed/checkpoint/test_traverse 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_traverse_1.1_461db3b52737f951_.log 2024-06-26T05:09:48.3725422Z Running 7 items in this shard: test/distributed/checkpoint/test_traverse.py::TestTraverse::test_get_element, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_set_element, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_doesnt_ignore_intermediate_collections, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_nested_dict, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_nested_list, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_shallow, test/distributed/checkpoint/test_traverse.py::TestTraverse::test_traverse_with_ordered_dict 2024-06-26T05:09:48.3729202Z 2024-06-26T05:09:48.3729749Z Running distributed/_composable/fsdp/test_fully_shard_compile 1/1 ... [2024-06-26 05:09:48.372261] 2024-06-26T05:09:48.3731913Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_compile.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:09:48.372700] 2024-06-26T05:09:51.2460243Z 2024-06-26T05:09:51.2462118Z distributed/_composable/fsdp/test_fully_shard_compile 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_compile_1.1_7e30047fc6b1e121_.log 2024-06-26T05:09:51.2468800Z Running 7 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompileCompute::test_disable_compiling_hooks, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_dynamo_trace_use_training_state, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_simple_mlp_fullgraph_backend_aot_eager, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_simple_mlp_fullgraph_backend_aot_eager_decomp_partition, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_simple_mlp_fullgraph_backend_inductor, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_transformer_fullgraph_backend_aot_eager, test/distributed/_composable/fsdp/test_fully_shard_compile.py::TestFullyShardCompile::test_transformer_fullgraph_backend_inductor 2024-06-26T05:09:51.2474123Z 2024-06-26T05:09:51.2474630Z Running distributed/_tensor/experimental/test_local_map 1/1 ... [2024-06-26 05:09:51.246445] 2024-06-26T05:09:51.2476744Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/experimental/test_local_map.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:09:51.246858] 2024-06-26T05:10:04.3398680Z 2024-06-26T05:10:04.3401419Z distributed/_tensor/experimental/test_local_map 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.experimental.test_local_map_1.1_6c57b83ba8b1c80a_.log 2024-06-26T05:10:04.3405188Z Running 4 items in this shard: test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_correctness, test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_in_placements, test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_out_placements, test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_redistribute 2024-06-26T05:10:04.3407709Z 2024-06-26T05:10:04.3408173Z Running distributed/_tools/test_memory_tracker 1/1 ... [2024-06-26 05:10:04.340417] 2024-06-26T05:10:04.3412213Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tools/test_memory_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:10:04.340896] 2024-06-26T05:10:07.1644090Z 2024-06-26T05:10:07.1646025Z distributed/_tools/test_memory_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_memory_tracker_1.1_ebd5e9bebcbf95c4_.log 2024-06-26T05:10:07.1647835Z Running 1 items in this shard: test/distributed/_tools/test_memory_tracker.py::TestMemoryTracker::test_local_model 2024-06-26T05:10:07.1650298Z 2024-06-26T05:10:07.1650816Z Running distributed/_tensor/test_op_strategy 1/1 ... [2024-06-26 05:10:07.164858] 2024-06-26T05:10:07.1656077Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_op_strategy.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:10:07.165296] 2024-06-26T05:10:10.0387095Z 2024-06-26T05:10:10.0388949Z distributed/_tensor/test_op_strategy 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_op_strategy_1.1_7ef4c5bdb9467d6c_.log 2024-06-26T05:10:10.0398055Z Running 15 items in this shard: test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_batch_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_bmm_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_free_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumDims::test_mm_dims, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_bmm_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_bmm_2d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_linearity_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_mm_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_mm_2d_mesh, test/distributed/_tensor/test_op_strategy.py::TestEinsumStrategies::test_pointwise_1d_mesh, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_bmm_strategies, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_mm_strategies, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_redistribute_cost_latency, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_redistribute_cost_mesh_1d, test/distributed/_tensor/test_op_strategy.py::TestCostModel::test_redistribute_cost_mesh_2d 2024-06-26T05:10:10.0406168Z 2024-06-26T05:10:10.0406760Z Running distributed/checkpoint/e2e/test_e2e_save_and_load 1/1 ... [2024-06-26 05:10:10.039137] 2024-06-26T05:10:10.0409087Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/e2e/test_e2e_save_and_load.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:10:10.039560] 2024-06-26T05:10:47.8353664Z 2024-06-26T05:10:47.8357046Z distributed/checkpoint/e2e/test_e2e_save_and_load 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.e2e.test_e2e_save_and_load_1.1_e8ba61ec4722dae8_.log 2024-06-26T05:10:47.8367199Z Running 13 items in this shard: test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_different_ordered_state_dict_keys, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_async_cached_cache_staged_state_dict_False, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_async_cached_cache_staged_state_dict_True, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_False_model_type0, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_False_model_type1, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_False_model_type2, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_True_model_type0, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_True_model_type1, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_e2e_compile_True_model_type2, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_no_dist, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_overwrite, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestE2ESaveAndLoad::test_partial_load, test/distributed/checkpoint/e2e/test_e2e_save_and_load.py::TestNoCPU::test_no_cpu 2024-06-26T05:10:47.8375701Z 2024-06-26T05:10:47.8376169Z Running distributed/pipelining/test_schedule 1/1 ... [2024-06-26 05:10:47.836012] 2024-06-26T05:10:47.8378186Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_schedule.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:10:47.836457] 2024-06-26T05:10:51.0883541Z 2024-06-26T05:10:51.0885513Z distributed/pipelining/test_schedule 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_schedule_1.1_94b6df97d9b24161_.log 2024-06-26T05:10:51.0887090Z 2024-06-26T05:10:51.0891669Z Running distributed/_spmd/test_tracing 1/1 ... [2024-06-26 05:10:51.088957] 2024-06-26T05:10:51.0896480Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_spmd/test_tracing.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:10:51.089373] 2024-06-26T05:10:53.4605108Z 2024-06-26T05:10:53.4622202Z distributed/_spmd/test_tracing 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._spmd.test_tracing_1.1_d9c72d0d45cf1749_.log 2024-06-26T05:10:53.4623675Z 2024-06-26T05:10:53.4624470Z Running distributed/checkpoint/test_hsdp_checkpoint 1/1 ... [2024-06-26 05:10:53.460925] 2024-06-26T05:10:53.4627422Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_hsdp_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:10:53.461335] 2024-06-26T05:11:07.0572366Z 2024-06-26T05:11:07.0574101Z distributed/checkpoint/test_hsdp_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_hsdp_checkpoint_1.1_5f59a5a61b68c83e_.log 2024-06-26T05:11:07.0578528Z Running 4 items in this shard: test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_checkpoint_is_even_sharded_model_False, test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_checkpoint_is_even_sharded_model_True, test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_fsdp_checkpoint_conversion_is_even_sharded_model_False, test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_fsdp_checkpoint_conversion_is_even_sharded_model_True 2024-06-26T05:11:07.0581750Z 2024-06-26T05:11:07.0582212Z Running distributed/_tensor/test_embedding_ops 1/1 ... [2024-06-26 05:11:07.057663] 2024-06-26T05:11:07.0584219Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_embedding_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:11:07.058056] 2024-06-26T05:11:18.5020800Z 2024-06-26T05:11:18.5022674Z distributed/_tensor/test_embedding_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_embedding_ops_1.1_920a2d3df7af1b0d_.log 2024-06-26T05:11:18.5025865Z Running 3 items in this shard: test/distributed/_tensor/test_embedding_ops.py::TestEmbeddingOp::test_sharded_embedding_colwise, test/distributed/_tensor/test_embedding_ops.py::TestEmbeddingOp::test_sharded_embedding_colwise_max_norm_errors, test/distributed/_tensor/test_embedding_ops.py::TestEmbeddingOp::test_sharded_embedding_rowwise 2024-06-26T05:11:18.5028061Z 2024-06-26T05:11:18.5028576Z Running distributed/checkpoint/test_state_dict_utils 1/1 ... [2024-06-26 05:11:18.502547] 2024-06-26T05:11:18.5032997Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_state_dict_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:11:18.502972] 2024-06-26T05:11:21.2761832Z 2024-06-26T05:11:21.2763841Z distributed/checkpoint/test_state_dict_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_state_dict_utils_1.1_81e06d99c56358fc_.log 2024-06-26T05:11:21.2768540Z Running 5 items in this shard: test/distributed/checkpoint/test_state_dict_utils.py::TestStateDictUtils::test_complicated_dict, test/distributed/checkpoint/test_state_dict_utils.py::TestStateDictUtils::test_cpu_and_ranks_only, test/distributed/checkpoint/test_state_dict_utils.py::TestStateDictUtils::test_create_cpu_state_dict, test/distributed/checkpoint/test_state_dict_utils.py::TestStateDictUtils::test_gather_state_dict_dtensor, test/distributed/checkpoint/test_state_dict_utils.py::TestStateDictUtils::test_gather_with_cpu_and_ranks_only 2024-06-26T05:11:21.2771625Z 2024-06-26T05:11:21.2772228Z Running distributed/tensor/parallel/test_fsdp_2d_parallel 1/1 ... [2024-06-26 05:11:21.276658] 2024-06-26T05:11:21.2774856Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_fsdp_2d_parallel.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:11:21.277104] 2024-06-26T05:11:57.1196432Z 2024-06-26T05:11:57.1198807Z distributed/tensor/parallel/test_fsdp_2d_parallel 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_fsdp_2d_parallel_1.1_f306477029b9092c_.log 2024-06-26T05:11:57.1215982Z Running 12 items in this shard: test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelTraining::test_2d_e2e_training_default, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelTraining::test_2d_e2e_training_not_use_orig_params, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelTraining::test_2d_e2e_training_use_orig_params, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelTraining::test_2d_fsdp_state_enable_extension, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelTraining::test_raise_invalid_tp_composition, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelStateDict::test_2d_load_state_dict_is_even_sharded_model_False, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelStateDict::test_2d_load_state_dict_is_even_sharded_model_True, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelStateDict::test_2d_optim_state_dict_is_even_sharded_model_False, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelStateDict::test_2d_optim_state_dict_is_even_sharded_model_True, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelStateDict::test_2d_state_dict_is_even_sharded_model_False, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelStateDict::test_2d_state_dict_is_even_sharded_model_True, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelStateDict::test_fsdp_2d_extension 2024-06-26T05:11:57.1225249Z 2024-06-26T05:11:57.1225951Z Running distributed/elastic/test_control_plane 1/1 ... [2024-06-26 05:11:57.120119] 2024-06-26T05:11:57.1227984Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/test_control_plane.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:11:57.120587] 2024-06-26T05:12:03.0485670Z 2024-06-26T05:12:03.0487390Z distributed/elastic/test_control_plane 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.test_control_plane_1.1_616aa2ed03e2b8ee_.log 2024-06-26T05:12:03.0491849Z Running 5 items in this shard: test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_nccl_trace_pickle, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_nccl_trace_pickle_with_params, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_dump_traceback, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_tcp, test/distributed/elastic/test_control_plane.py::WorkerServerTest::test_worker_server 2024-06-26T05:12:03.0495285Z 2024-06-26T05:12:03.0495811Z Running distributed/checkpoint/test_compatibility 1/1 ... [2024-06-26 05:12:03.049139] 2024-06-26T05:12:03.0498837Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_compatibility.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:12:03.049557] 2024-06-26T05:12:05.8727413Z 2024-06-26T05:12:05.8729213Z distributed/checkpoint/test_compatibility 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_compatibility_1.1_56365bb8d8f91ee8_.log 2024-06-26T05:12:05.8732087Z Running 3 items in this shard: test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_metadata, test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_sharded_tensor_dependency, test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_storage_meta 2024-06-26T05:12:05.8734493Z 2024-06-26T05:12:05.8735288Z Running distributed/_composable/fully_shard/test_fully_shard_util 1/1 ... [2024-06-26 05:12:05.873221] 2024-06-26T05:12:05.8739590Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_util.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:12:05.873628] 2024-06-26T05:12:11.3511363Z 2024-06-26T05:12:11.3513383Z distributed/_composable/fully_shard/test_fully_shard_util 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_util_1.1_2d0ca7103825c788_.log 2024-06-26T05:12:11.3515763Z Running 1 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_util.py::TestUtils::test_get_sharded_module_tree_with_module_name_to_fqns 2024-06-26T05:12:11.3516813Z 2024-06-26T05:12:11.3517700Z Running distributed/_composable/fsdp/test_fully_shard_frozen 1/1 ... [2024-06-26 05:12:11.351569] 2024-06-26T05:12:11.3523119Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_frozen.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:12:11.351990] 2024-06-26T05:12:14.2752676Z 2024-06-26T05:12:14.2754629Z distributed/_composable/fsdp/test_fully_shard_frozen 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_frozen_1.1_982b2d878f8b16df_.log 2024-06-26T05:12:14.2758102Z Running 3 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_frozen.py::TestFullyShardFrozen::test_multi_forward_mixed_requires_grad, test/distributed/_composable/fsdp/test_fully_shard_frozen.py::TestFullyShardFrozen::test_train_mixed_requires_grad_across_groups, test/distributed/_composable/fsdp/test_fully_shard_frozen.py::TestFullyShardFrozen::test_train_mixed_requires_grad_per_group 2024-06-26T05:12:14.2760796Z 2024-06-26T05:12:14.2761257Z Running distributed/_composable/test_replicate 1/1 ... [2024-06-26 05:12:14.275704] 2024-06-26T05:12:14.2764963Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_replicate.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:12:14.276159] 2024-06-26T05:12:40.5376418Z 2024-06-26T05:12:40.5378087Z distributed/_composable/test_replicate 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_replicate_1.1_4556cb96c9333fce_.log 2024-06-26T05:12:40.5384857Z Running 9 items in this shard: test/distributed/_composable/test_replicate.py::ReplicateStateDictTest::test_replicate_non_root_multiple_save_load, test/distributed/_composable/test_replicate.py::ReplicateStateDictTest::test_replicate_single_module_save_load, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_device_id, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_ignore_module, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_move_args_kwargs_to_device, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_multi_module, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_single_module, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_with_kwargs, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_wrong_device_id_type 2024-06-26T05:12:40.5389798Z 2024-06-26T05:12:40.5390291Z Running distributed/test_distributed_spawn 2/3 ... [2024-06-26 05:12:40.538247] 2024-06-26T05:12:40.5491041Z MPI not available -- MPI backend tests will be skipped 2024-06-26T05:12:40.5492121Z Running distributed tests for the test backend with env init_method 2024-06-26T05:12:40.5501588Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=2', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:12:40.549796] 2024-06-26T05:12:43.2409769Z 2024-06-26T05:12:43.2412301Z distributed/test_distributed_spawn 2/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_2.3_c919288ad32395b4_.log 2024-06-26T05:12:43.2414585Z Running 0 items in this shard: 2024-06-26T05:12:43.2415068Z 2024-06-26T05:12:43.2421310Z Running distributed tests for the test backend with file init_method 2024-06-26T05:12:43.2429017Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=2', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:12:43.242458] 2024-06-26T05:12:45.8896968Z 2024-06-26T05:12:45.8898691Z distributed/test_distributed_spawn 2/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_2.3_2f7858737262c1d6_.log 2024-06-26T05:12:45.8899912Z Running 0 items in this shard: 2024-06-26T05:12:45.8900167Z 2024-06-26T05:12:45.8906333Z Running distributed tests for the gloo backend with env init_method 2024-06-26T05:12:45.8912425Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=2', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:12:45.890946] 2024-06-26T05:20:45.5980233Z 2024-06-26T05:20:45.5983215Z distributed/test_distributed_spawn 2/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_2.3_4c0500aab60c72e1_.log 2024-06-26T05:20:45.6042579Z Running 92 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Single_Input_Per_Process, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_half, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedSampler_padding, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_SyncBatchNorm_process_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_with_then_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_simple, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_with_empty, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_v_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_complex_unsupported_ops, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_global, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_ignored_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce_return_future, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_comm_hook_logging, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device_mesh_initialization, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_grad_div_uneven_inputs, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_has_finalized, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_powerSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_inference, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_join_model_equivalence, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_err_ignore_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_no_set_grad_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_set_grad_to_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sink_noclone, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_bn_training_vs_eval, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_exception, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_unused_params_rebuild_buckets_exception, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_zero_output_features, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_detect_ddp_is_actually_static, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_dump_DDP_relevant_env_vars, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_backend, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_grads_same_across_ranks_with_no_sync, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang_wait_all_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_rank_0_timeout, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_broadcast, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_input_rank_exceeds_world_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_overlap_not_allowed, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_dict_module, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_step_reload, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_v_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_twice, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_stateless_api_with_ddp, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sync_bn_logged 2024-06-26T05:20:45.6099634Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel 2024-06-26T05:20:45.6101176Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm 2024-06-26T05:20:45.6102930Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Single_Input_Per_Process 2024-06-26T05:20:45.6104848Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_half 2024-06-26T05:20:45.6106384Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedSampler_padding 2024-06-26T05:20:45.6107846Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_SyncBatchNorm_process_group 2024-06-26T05:20:45.6109449Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_with_then_hook 2024-06-26T05:20:45.6111058Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_full_group 2024-06-26T05:20:45.6112593Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_simple 2024-06-26T05:20:45.6114074Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_with_empty 2024-06-26T05:20:45.6115499Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_complex 2024-06-26T05:20:45.6116868Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda_complex 2024-06-26T05:20:45.6118253Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_v_cuda 2024-06-26T05:20:45.6119660Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_product 2024-06-26T05:20:45.6121266Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_complex_unsupported_ops 2024-06-26T05:20:45.6122742Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_sum 2024-06-26T05:20:45.6124295Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_min 2024-06-26T05:20:45.6125688Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_complex 2024-06-26T05:20:45.6127068Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda 2024-06-26T05:20:45.6128369Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all 2024-06-26T05:20:45.6129680Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_complex 2024-06-26T05:20:45.6131020Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda 2024-06-26T05:20:45.6132459Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_complex 2024-06-26T05:20:45.6133987Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda 2024-06-26T05:20:45.6135498Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split 2024-06-26T05:20:45.6137023Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_complex 2024-06-26T05:20:45.6138621Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda_complex 2024-06-26T05:20:45.6140352Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group_cuda 2024-06-26T05:20:45.6141965Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group 2024-06-26T05:20:45.6143397Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_group 2024-06-26T05:20:45.6144705Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group 2024-06-26T05:20:45.6146042Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_global 2024-06-26T05:20:45.6147434Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_nccl 2024-06-26T05:20:45.6148850Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_err 2024-06-26T05:20:45.6150198Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_cuda 2024-06-26T05:20:45.6151515Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_group 2024-06-26T05:20:45.6152965Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward 2024-06-26T05:20:45.6154498Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_ignored_params 2024-06-26T05:20:45.6155988Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer 2024-06-26T05:20:45.6157554Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce_return_future 2024-06-26T05:20:45.6159038Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_comm_hook_logging 2024-06-26T05:20:45.6160480Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device_mesh_initialization 2024-06-26T05:20:45.6161938Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_grad_div_uneven_inputs 2024-06-26T05:20:45.6163324Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_has_finalized 2024-06-26T05:20:45.6165214Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce 2024-06-26T05:20:45.6166690Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_powerSGD 2024-06-26T05:20:45.6168053Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_inference 2024-06-26T05:20:45.6169431Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_join_model_equivalence 2024-06-26T05:20:45.6171000Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_err_ignore_params 2024-06-26T05:20:45.6172758Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_no_set_grad_none 2024-06-26T05:20:45.6174569Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_set_grad_to_none 2024-06-26T05:20:45.6176242Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_autograd_profiler 2024-06-26T05:20:45.6177812Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_torch_profiler 2024-06-26T05:20:45.6179219Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sink_noclone 2024-06-26T05:20:45.6180602Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_bn_training_vs_eval 2024-06-26T05:20:45.6182059Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_exception 2024-06-26T05:20:45.6183589Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_unused_params_rebuild_buckets_exception 2024-06-26T05:20:45.6185108Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_zero_output_features 2024-06-26T05:20:45.6186489Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_full_group 2024-06-26T05:20:45.6187828Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_group 2024-06-26T05:20:45.6189222Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_detect_ddp_is_actually_static 2024-06-26T05:20:45.6190666Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_dump_DDP_relevant_env_vars 2024-06-26T05:20:45.6192111Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_cuda 2024-06-26T05:20:45.6193453Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object_subgroup 2024-06-26T05:20:45.6194872Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_backend 2024-06-26T05:20:45.6196219Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_full_group 2024-06-26T05:20:45.6197698Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_grads_same_across_ranks_with_no_sync 2024-06-26T05:20:45.6199075Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend 2024-06-26T05:20:45.6200366Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_torch_profiler 2024-06-26T05:20:45.6201871Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang_wait_all_ranks 2024-06-26T05:20:45.6203477Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_rank_0_timeout 2024-06-26T05:20:45.6205115Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_broadcast 2024-06-26T05:20:45.6206580Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration 2024-06-26T05:20:45.6208185Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_input_rank_exceeds_world_size 2024-06-26T05:20:45.6209829Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_overlap_not_allowed 2024-06-26T05:20:45.6211339Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_dict_module 2024-06-26T05:20:45.6212799Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager 2024-06-26T05:20:45.6214432Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd 2024-06-26T05:20:45.6216057Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_step_reload 2024-06-26T05:20:45.6217521Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_max 2024-06-26T05:20:45.6218909Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_product 2024-06-26T05:20:45.6220258Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_sum 2024-06-26T05:20:45.6221556Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_max 2024-06-26T05:20:45.6222899Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_v_cuda 2024-06-26T05:20:45.6224264Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_twice 2024-06-26T05:20:45.6225571Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_group 2024-06-26T05:20:45.6226912Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source 2024-06-26T05:20:45.6228373Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_torch_profiler 2024-06-26T05:20:45.6229855Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_autograd_profiler 2024-06-26T05:20:45.6231356Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum 2024-06-26T05:20:45.6232829Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_stateless_api_with_ddp 2024-06-26T05:20:45.6234185Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sync_bn_logged 2024-06-26T05:20:45.6234915Z 2024-06-26T05:20:45.6235197Z Running distributed tests for the gloo backend with file init_method 2024-06-26T05:20:45.6237298Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=2', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:20:45.601733] 2024-06-26T05:28:43.4157823Z 2024-06-26T05:28:43.4159766Z distributed/test_distributed_spawn 2/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_2.3_7711e783c60bd07d_.log 2024-06-26T05:28:43.4256601Z Running 92 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Single_Input_Per_Process, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_half, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedSampler_padding, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_SyncBatchNorm_process_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_with_then_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_simple, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_with_empty, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_v_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_complex_unsupported_ops, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_global, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_ignored_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce_return_future, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_comm_hook_logging, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device_mesh_initialization, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_grad_div_uneven_inputs, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_has_finalized, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_powerSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_inference, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_join_model_equivalence, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_err_ignore_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_no_set_grad_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_set_grad_to_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sink_noclone, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_bn_training_vs_eval, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_exception, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_unused_params_rebuild_buckets_exception, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_zero_output_features, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_detect_ddp_is_actually_static, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_dump_DDP_relevant_env_vars, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_backend, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_grads_same_across_ranks_with_no_sync, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang_wait_all_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_rank_0_timeout, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_broadcast, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_input_rank_exceeds_world_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_overlap_not_allowed, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_dict_module, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_step_reload, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_v_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_twice, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_stateless_api_with_ddp, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sync_bn_logged 2024-06-26T05:28:43.4310632Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel 2024-06-26T05:28:43.4312252Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm 2024-06-26T05:28:43.4313976Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Single_Input_Per_Process 2024-06-26T05:28:43.4315733Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_half 2024-06-26T05:28:43.4317271Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedSampler_padding 2024-06-26T05:28:43.4318737Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_SyncBatchNorm_process_group 2024-06-26T05:28:43.4320324Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_with_then_hook 2024-06-26T05:28:43.4321938Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_full_group 2024-06-26T05:28:43.4323407Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_simple 2024-06-26T05:28:43.4325153Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_with_empty 2024-06-26T05:28:43.4326596Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_complex 2024-06-26T05:28:43.4327977Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda_complex 2024-06-26T05:28:43.4329354Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_v_cuda 2024-06-26T05:28:43.4330752Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_product 2024-06-26T05:28:43.4332249Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_complex_unsupported_ops 2024-06-26T05:28:43.4333723Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_sum 2024-06-26T05:28:43.4335142Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_min 2024-06-26T05:28:43.4336531Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_complex 2024-06-26T05:28:43.4337910Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda 2024-06-26T05:28:43.4339205Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all 2024-06-26T05:28:43.4340511Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_complex 2024-06-26T05:28:43.4341847Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda 2024-06-26T05:28:43.4343293Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_complex 2024-06-26T05:28:43.4344921Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda 2024-06-26T05:28:43.4346438Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split 2024-06-26T05:28:43.4347973Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_complex 2024-06-26T05:28:43.4349580Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda_complex 2024-06-26T05:28:43.4351300Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group_cuda 2024-06-26T05:28:43.4352974Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group 2024-06-26T05:28:43.4354405Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_group 2024-06-26T05:28:43.4355709Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group 2024-06-26T05:28:43.4357042Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_global 2024-06-26T05:28:43.4358441Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_nccl 2024-06-26T05:28:43.4359856Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_err 2024-06-26T05:28:43.4361315Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_cuda 2024-06-26T05:28:43.4362628Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_group 2024-06-26T05:28:43.4364155Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward 2024-06-26T05:28:43.4365721Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_ignored_params 2024-06-26T05:28:43.4367204Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer 2024-06-26T05:28:43.4368694Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce_return_future 2024-06-26T05:28:43.4370182Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_comm_hook_logging 2024-06-26T05:28:43.4371634Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device_mesh_initialization 2024-06-26T05:28:43.4373086Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_grad_div_uneven_inputs 2024-06-26T05:28:43.4374475Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_has_finalized 2024-06-26T05:28:43.4375860Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce 2024-06-26T05:28:43.4377292Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_powerSGD 2024-06-26T05:28:43.4378636Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_inference 2024-06-26T05:28:43.4380002Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_join_model_equivalence 2024-06-26T05:28:43.4381665Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_err_ignore_params 2024-06-26T05:28:43.4383426Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_no_set_grad_none 2024-06-26T05:28:43.4385238Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_set_grad_to_none 2024-06-26T05:28:43.4386917Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_autograd_profiler 2024-06-26T05:28:43.4388412Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_torch_profiler 2024-06-26T05:28:43.4389813Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sink_noclone 2024-06-26T05:28:43.4391200Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_bn_training_vs_eval 2024-06-26T05:28:43.4392705Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_exception 2024-06-26T05:28:43.4394233Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_unused_params_rebuild_buckets_exception 2024-06-26T05:28:43.4395758Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_zero_output_features 2024-06-26T05:28:43.4397135Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_full_group 2024-06-26T05:28:43.4398551Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_destroy_group 2024-06-26T05:28:43.4399949Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_detect_ddp_is_actually_static 2024-06-26T05:28:43.4401399Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_dump_DDP_relevant_env_vars 2024-06-26T05:28:43.4402753Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_cuda 2024-06-26T05:28:43.4404251Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object_subgroup 2024-06-26T05:28:43.4405597Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_backend 2024-06-26T05:28:43.4406939Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_full_group 2024-06-26T05:28:43.4408426Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_grads_same_across_ranks_with_no_sync 2024-06-26T05:28:43.4409803Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend 2024-06-26T05:28:43.4411102Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_torch_profiler 2024-06-26T05:28:43.4412611Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang_wait_all_ranks 2024-06-26T05:28:43.4414220Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_rank_0_timeout 2024-06-26T05:28:43.4415722Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_broadcast 2024-06-26T05:28:43.4417190Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration 2024-06-26T05:28:43.4418878Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_input_rank_exceeds_world_size 2024-06-26T05:28:43.4420520Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_overlap_not_allowed 2024-06-26T05:28:43.4422042Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_dict_module 2024-06-26T05:28:43.4423507Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager 2024-06-26T05:28:43.4425070Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd 2024-06-26T05:28:43.4426707Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_step_reload 2024-06-26T05:28:43.4428170Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_max 2024-06-26T05:28:43.4429561Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_product 2024-06-26T05:28:43.4430908Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_sum 2024-06-26T05:28:43.4432388Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_max 2024-06-26T05:28:43.4433724Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_v_cuda 2024-06-26T05:28:43.4435339Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_twice 2024-06-26T05:28:43.4436652Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_group 2024-06-26T05:28:43.4437993Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source 2024-06-26T05:28:43.4439518Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_torch_profiler 2024-06-26T05:28:43.4440997Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_autograd_profiler 2024-06-26T05:28:43.4442421Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum 2024-06-26T05:28:43.4443820Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_stateless_api_with_ddp 2024-06-26T05:28:43.4445330Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sync_bn_logged 2024-06-26T05:28:43.4446071Z 2024-06-26T05:28:43.4446702Z Running distributed/_composable/fsdp/test_fully_shard_state_dict 1/1 ... [2024-06-26 05:28:43.421082] 2024-06-26T05:28:43.4448898Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:28:43.421548] 2024-06-26T05:28:46.2462220Z 2024-06-26T05:28:46.2464492Z distributed/_composable/fsdp/test_fully_shard_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_state_dict_1.1_4531b458d89bf839_.log 2024-06-26T05:28:46.2469355Z Running 4 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_1d_state_dict_cpu_offload, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_1d_state_dict_save_load, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_2d_state_dict_save_load, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiThread::test_rank0_offload_full_state_dict 2024-06-26T05:28:46.2472883Z 2024-06-26T05:28:46.2473398Z Running distributed/fsdp/test_fsdp_pure_fp16 1/1 ... [2024-06-26 05:28:46.246703] 2024-06-26T05:28:46.2475396Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_pure_fp16.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:28:46.247118] 2024-06-26T05:28:54.5818152Z 2024-06-26T05:28:54.5819986Z distributed/fsdp/test_fsdp_pure_fp16 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_pure_fp16_1.1_18e4e1ba05e30e20_.log 2024-06-26T05:28:54.5822093Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_pure_fp16.py::TestPureFP16::test_fp16_dtypes, test/distributed/fsdp/test_fsdp_pure_fp16.py::TestPureFP16::test_pure_fp16_training 2024-06-26T05:28:54.5823264Z 2024-06-26T05:28:54.5824699Z Running distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_ 1/1 ... [2024-06-26 05:28:54.582252] 2024-06-26T05:28:54.5830159Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:28:54.582702] 2024-06-26T05:28:57.4560680Z 2024-06-26T05:28:57.4563054Z distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_ 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_clip_grad_norm__1.1_47b5d317491f7d89_.log 2024-06-26T05:28:57.4566203Z Running 2 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_.py::TestClipGradNormWorldSize2::test_clip_grad_norm_1d, test/distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_.py::TestClipGradNormWorldSize4::test_clip_grad_norm_2d 2024-06-26T05:28:57.4568316Z 2024-06-26T05:28:57.4568743Z Running distributed/_composable/test_compose 1/1 ... [2024-06-26 05:28:57.456544] 2024-06-26T05:28:57.4573089Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_compose.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:28:57.456978] 2024-06-26T05:29:24.0676308Z 2024-06-26T05:29:24.0678272Z distributed/_composable/test_compose 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_compose_1.1_c208bd5ef1ccd15c_.log 2024-06-26T05:29:24.0685778Z Running 9 items in this shard: test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fsdp_submodules_non_reentrant, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fsdp_submodules_with_param, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fsdp_submodules_with_param_no_shard, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_checkpoint_fully_shard_cast_forward_inputs, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_composable_fsdp_replicate, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_fully_shard_replicate_composability, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_fully_shard_replicate_correct_replicate_params, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_state_dict_fsdp_submodules, test/distributed/_composable/test_compose.py::TestFSDPCheckpoint::test_wrap_same_submodule 2024-06-26T05:29:24.0691517Z 2024-06-26T05:29:24.0692174Z Running distributed/_composable/fully_shard/test_fully_shard_mixed_precision 1/1 ... [2024-06-26 05:29:24.068177] 2024-06-26T05:29:24.0694501Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_mixed_precision.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:29:24.068684] 2024-06-26T05:29:29.4471258Z 2024-06-26T05:29:29.4473644Z distributed/_composable/fully_shard/test_fully_shard_mixed_precision 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_mixed_precision_1.1_dd2cfa6dcf169861_.log 2024-06-26T05:29:29.4475931Z Running 1 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_mixed_precision.py::TestMixedPrecision::test_float16_cast_forward 2024-06-26T05:29:29.4476857Z 2024-06-26T05:29:29.4478224Z Running distributed/_tensor/test_view_ops 1/1 ... [2024-06-26 05:29:29.447608] 2024-06-26T05:29:29.4483200Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_view_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:29:29.448035] 2024-06-26T05:29:59.4475305Z 2024-06-26T05:29:59.4477753Z distributed/_tensor/test_view_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_view_ops_1.1_df16645b01cee6ce_.log 2024-06-26T05:29:59.4482094Z Running 3 items in this shard: test/distributed/_tensor/test_view_ops.py::TestViewOps::test_complex_view_ops, test/distributed/_tensor/test_view_ops.py::TestViewOps::test_view_groups, test/distributed/_tensor/test_view_ops.py::TestViewOps::test_view_ops 2024-06-26T05:29:59.4485192Z 2024-06-26T05:29:59.4485881Z Running distributed/_tensor/test_utils 1/1 ... [2024-06-26 05:29:59.448269] 2024-06-26T05:29:59.4492161Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:29:59.448833] 2024-06-26T05:30:22.5628620Z 2024-06-26T05:30:22.5630113Z distributed/_tensor/test_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_utils_1.1_28a0892fdb0aa6d4_.log 2024-06-26T05:30:22.5634930Z Running 5 items in this shard: test/distributed/_tensor/test_utils.py::UtilTest::test_compute_local_shape_2d_uneven, test/distributed/_tensor/test_utils.py::UtilTest::test_compute_local_shape_and_global_offset_1D, test/distributed/_tensor/test_utils.py::UtilTest::test_compute_local_shape_and_global_offset_2D, test/distributed/_tensor/test_utils.py::Test2DStridedLocalShard::test_fsdp1_tp_2d_dtensor_local_shards_and_offsets, test/distributed/_tensor/test_utils.py::Test2DStridedLocalShard::test_fsdp2_tp_2d_dtensor_local_shards_and_offsets 2024-06-26T05:30:22.5638372Z 2024-06-26T05:30:22.5638900Z Running distributed/checkpoint/test_utils 1/1 ... [2024-06-26 05:30:22.563361] 2024-06-26T05:30:22.5641285Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:30:22.563783] 2024-06-26T05:30:25.4370938Z 2024-06-26T05:30:25.4372915Z distributed/checkpoint/test_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_utils_1.1_bdec49b544dcea65_.log 2024-06-26T05:30:25.4378158Z Running 6 items in this shard: test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_flat_data, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_index_hint_ignored_on_equals, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_index_hint_ignored_on_hash, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_init_convert_offset, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_sharded_tensor_lookup, test/distributed/checkpoint/test_utils.py::TestTensorProperties::test_create_from_tensor_correct_device 2024-06-26T05:30:25.4381839Z 2024-06-26T05:30:25.4382210Z Running distributed/test_nccl 1/1 ... [2024-06-26 05:30:25.437535] 2024-06-26T05:30:25.4384014Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_nccl.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:30:25.437948] 2024-06-26T05:30:28.1510555Z 2024-06-26T05:30:28.1512557Z distributed/test_nccl 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_nccl_1.1_05856e1ca758642b_.log 2024-06-26T05:30:28.1513644Z Running 0 items in this shard: 2024-06-26T05:30:28.1513902Z 2024-06-26T05:30:28.1516900Z Running distributed/_composable/fsdp/test_fully_shard_mixed_precision 1/1 ... [2024-06-26 05:30:28.151473] 2024-06-26T05:30:28.1522073Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_mixed_precision.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:30:28.151882] 2024-06-26T05:30:31.0750975Z 2024-06-26T05:30:31.0752937Z distributed/_composable/fsdp/test_fully_shard_mixed_precision 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_mixed_precision_1.1_8386c04f55fd1622_.log 2024-06-26T05:30:31.0760226Z Running 7 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionTraining::test_compute_dtype, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionTraining::test_grad_acc_with_reduce_dtype, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionTraining::test_reduce_dtype, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_float16_on_one_submodule, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_norm_modules_bf16, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_norm_modules_fp16, test/distributed/_composable/fsdp/test_fully_shard_mixed_precision.py::TestFullyShardMixedPrecisionCasts::test_submodules_with_external_inputs 2024-06-26T05:30:31.0766417Z 2024-06-26T05:30:31.0766867Z Running distributed/_tensor/test_optimizers 1/1 ... [2024-06-26 05:30:31.075599] 2024-06-26T05:30:31.0769026Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_optimizers.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:30:31.076038] 2024-06-26T05:31:08.7714083Z 2024-06-26T05:31:08.7717839Z distributed/_tensor/test_optimizers 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_optimizers_1.1_3ab6c51beb8d1894_.log 2024-06-26T05:31:08.7725111Z Running 11 items in this shard: test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_RMSprop_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_adadelta_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_adagrad_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_adam_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_adamax_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_adamw_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_asgd_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_nadam_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_optimizer_foreach_supported_types_include_DTensor, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_radam_1d_sharding, test/distributed/_tensor/test_optimizers.py::TestDTensorOptimizer::test_sgd_1d_sharding 2024-06-26T05:31:08.7731250Z 2024-06-26T05:31:08.7731725Z Running distributed/fsdp/test_fsdp_ignored_modules 1/1 ... [2024-06-26 05:31:08.772505] 2024-06-26T05:31:08.7733791Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_ignored_modules.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:31:08.772917] 2024-06-26T05:31:11.5964831Z 2024-06-26T05:31:11.5966593Z distributed/fsdp/test_fsdp_ignored_modules 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_ignored_modules_1.1_c3ebbb899ed7c563_.log 2024-06-26T05:31:11.5978771Z Running 13 items in this shard: test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_diff_ignored_modules_across_ranks, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_invalid_composable_False, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_invalid_composable_True, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_nested, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_nested_composable, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_False_composable_False, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_False_composable_True, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_True_composable_False, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_True_composable_True, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_transformer, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_transformer_composable, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_states_auto_wrap, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_states_check 2024-06-26T05:31:11.5988463Z 2024-06-26T05:31:11.5988858Z Running distributed/test_fake_pg 1/1 ... [2024-06-26 05:31:11.597049] 2024-06-26T05:31:11.5990696Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_fake_pg.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:31:11.597493] 2024-06-26T05:31:14.5208626Z 2024-06-26T05:31:14.5210566Z distributed/test_fake_pg 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_fake_pg_1.1_64dbc573177f040e_.log 2024-06-26T05:31:14.5216757Z Running 13 items in this shard: test/distributed/test_fake_pg.py::TestFakePG::test_all_reduce, test/distributed/test_fake_pg.py::TestFakePG::test_allgather, test/distributed/test_fake_pg.py::TestFakePG::test_alltoall, test/distributed/test_fake_pg.py::TestFakePG::test_alltoall_base, test/distributed/test_fake_pg.py::TestFakePG::test_broadcast, test/distributed/test_fake_pg.py::TestFakePG::test_construct_fsdp, test/distributed/test_fake_pg.py::TestFakePG::test_fake_pg_tracing, test/distributed/test_fake_pg.py::TestFakePG::test_fsdp_fake_e2e, test/distributed/test_fake_pg.py::TestFakePG::test_fsdp_tp_fake_e2e, test/distributed/test_fake_pg.py::TestFakePG::test_recv, test/distributed/test_fake_pg.py::TestFakePG::test_reduce_scatter, test/distributed/test_fake_pg.py::TestFakePG::test_scatter, test/distributed/test_fake_pg.py::TestFakePG::test_send 2024-06-26T05:31:14.5221552Z 2024-06-26T05:31:14.5222122Z Running distributed/_composable/fully_shard/test_fully_shard_init 1/1 ... [2024-06-26 05:31:14.521412] 2024-06-26T05:31:14.5224352Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_init.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:31:14.521874] 2024-06-26T05:31:33.1223887Z 2024-06-26T05:31:33.1226068Z distributed/_composable/fully_shard/test_fully_shard_init 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_init_1.1_1b26dd0fcb4c1cf0_.log 2024-06-26T05:31:33.1231233Z Running 6 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_device_id, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_manual_fully_shard, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_materialize_meta_module, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_nested_fully_shard_shared_state, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_policy, test/distributed/_composable/fully_shard/test_fully_shard_init.py::TestInitialization::test_sync_module_states 2024-06-26T05:31:33.1235835Z 2024-06-26T05:31:33.1236286Z Running distributed/launcher/test_run 1/1 ... [2024-06-26 05:31:33.122802] 2024-06-26T05:31:33.1238333Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/launcher/test_run.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:31:33.123208] 2024-06-26T05:32:09.5001976Z 2024-06-26T05:32:09.5006407Z distributed/launcher/test_run 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.launcher.test_run_1.1_d8eeab2b24a0b3c2_.log 2024-06-26T05:32:09.5023447Z Running 24 items in this shard: test/distributed/launcher/test_run.py::ElasticLaunchTest::test_capture_logs_using_default_logs_specs, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_init_method_env_with_torchelastic, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_init_method_tcp_with_torchelastic, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_is_not_torchelastic_launched, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_is_torchelastic_launched, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_is_torchelastic_launched_with_logs_spec_defined, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic_agent_raise_exception, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic_multiple_agents, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic_worker_raise_exception, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_run_path, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_shutdown, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_standalone, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_bash, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_default_nproc, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_python, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_python_caffe2_bc, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_with_env_vars, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_logs_logs_spec_entrypoint_must_be_defined, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_min_max_nodes_parse, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_gpu_launch_configurations, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_launch_auto_configurations, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_launch_number_configurations, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_launch_unknown_configurations 2024-06-26T05:32:09.5037411Z 2024-06-26T05:32:09.5038018Z Running distributed/fsdp/test_fsdp_dtensor_state_dict 1/1 ... [2024-06-26 05:32:09.501226] 2024-06-26T05:32:09.5040212Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_dtensor_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:32:09.501688] 2024-06-26T05:32:53.6101394Z 2024-06-26T05:32:53.6103130Z distributed/fsdp/test_fsdp_dtensor_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_dtensor_state_dict_1.1_eba1493e76bd7e2e_.log 2024-06-26T05:32:53.6119758Z Running 15 items in this shard: test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_False_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_False_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_True_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_model_load_state_dict_offload_to_cpu_True_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_False_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_False_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_True_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_optim_load_state_dict_offload_to_cpu_True_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_False_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_False_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_True_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_dtensor_sharded_tensor_state_dict_identical_offload_to_cpu_True_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_fsdp_init_with_device_mesh_is_even_sharded_model_False, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_fsdp_init_with_device_mesh_is_even_sharded_model_True, test/distributed/fsdp/test_fsdp_dtensor_state_dict.py::TestFSDPWithDeviceMeshAndDTensor::test_raises_warning_or_errors 2024-06-26T05:32:53.6134255Z 2024-06-26T05:32:53.6134784Z Running distributed/fsdp/test_fsdp_backward_prefetch 1/1 ... [2024-06-26 05:32:53.610720] 2024-06-26T05:32:53.6136862Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_backward_prefetch.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:32:53.611136] 2024-06-26T05:32:58.9390278Z 2024-06-26T05:32:58.9392435Z distributed/fsdp/test_fsdp_backward_prefetch 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_backward_prefetch_1.1_b205c3b2637ea7d1_.log 2024-06-26T05:32:58.9394355Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_backward_prefetch.py::TestBackwardPrefetch::test_backward_prefetch 2024-06-26T05:32:58.9395165Z 2024-06-26T05:32:58.9400236Z Running distributed/fsdp/test_fsdp_optim_state 1/1 ... [2024-06-26 05:32:58.939807] 2024-06-26T05:32:58.9406805Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_optim_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:32:58.940356] 2024-06-26T05:35:59.2419756Z 2024-06-26T05:35:59.2421566Z distributed/fsdp/test_fsdp_optim_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_optim_state_1.1_b444d10bf5fc0ee2_.log 2024-06-26T05:35:59.2472841Z Running 60 items in this shard: test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_compatible_with_trec, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_flatten_sharded_optim_state_dict_nested, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_flatten_sharded_optim_state_dict_transformer, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_full_optim_state_dict_keys, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_full_optim_state_dict_nested_invalid, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_interface_arguments, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_no_grad, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_input_warning, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_without_param_groups, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type0_use_multiple_param_groups_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type0_use_multiple_param_groups_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type1_use_multiple_param_groups_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type1_use_multiple_param_groups_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_names, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_save_load_without_0th_param_state_state_dict_type0, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_save_load_without_0th_param_state_state_dict_type1, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_halve_world_size, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_transformer, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_halve_world_size, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_transformer, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type0_add_to_fsdp_module_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type0_add_to_fsdp_module_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type1_add_to_fsdp_module_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type1_add_to_fsdp_module_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_state_dict_with_none_tensor_state, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_use_orig_params, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_with_empty_optimizer_state, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_with_no_shard 2024-06-26T05:35:59.2521369Z 2024-06-26T05:35:59.2521850Z Running distributed/rpc/test_share_memory 1/1 ... [2024-06-26 05:35:59.243108] 2024-06-26T05:35:59.2523883Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_share_memory.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:35:59.243524] 2024-06-26T05:36:07.4927899Z 2024-06-26T05:36:07.4930065Z distributed/rpc/test_share_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_share_memory_1.1_8d18a32aa4c33a8e_.log 2024-06-26T05:36:07.4932306Z Running 1 items in this shard: test/distributed/rpc/test_share_memory.py::TestRPCPickler::test_case 2024-06-26T05:36:07.4933684Z Running 1 items in this shard: test/distributed/rpc/test_share_memory.py::TestRPCPickler::test_case 2024-06-26T05:36:07.4934331Z 2024-06-26T05:36:07.4935264Z Running distributed/_composable/fully_shard/test_fully_shard_model_checkpoint 1/1 ... [2024-06-26 05:36:07.493304] 2024-06-26T05:36:07.4940473Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_model_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:36:07.493738] 2024-06-26T05:36:20.5323885Z 2024-06-26T05:36:20.5326162Z distributed/_composable/fully_shard/test_fully_shard_model_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_model_checkpoint_1.1_6e1f4cfca0164eaa_.log 2024-06-26T05:36:20.5331245Z Running 4 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_model_checkpoint.py::TestModelCheckpointing::test_full_state_dict_save_load_mixed_sharding, test/distributed/_composable/fully_shard/test_fully_shard_model_checkpoint.py::TestModelCheckpointing::test_state_dict_save_load_flow, test/distributed/_composable/fully_shard/test_fully_shard_model_checkpoint.py::TestModelCheckpointing::test_state_dict_save_load_root_fully_shard, test/distributed/_composable/fully_shard/test_fully_shard_model_checkpoint.py::TestModelCheckpointing::test_state_dict_save_load_submodule_fully_shard 2024-06-26T05:36:20.5334769Z 2024-06-26T05:36:20.5335228Z Running distributed/fsdp/test_fsdp_flatten_params 1/1 ... [2024-06-26 05:36:20.532941] 2024-06-26T05:36:20.5337488Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_flatten_params.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:36:20.533412] 2024-06-26T05:36:50.9461097Z 2024-06-26T05:36:50.9462917Z distributed/fsdp/test_fsdp_flatten_params 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_flatten_params_1.1_167849ad47054a32_.log 2024-06-26T05:36:50.9474750Z Running 11 items in this shard: test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_empty_module, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_flat_param_shard_metadata_aligned_full_precision, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_flat_param_shard_metadata_aligned_mixed_precision, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_flat_param_shard_metadata_unaligned, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_flatten_nothing, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_numel_with_shared_params, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_numel_without_shared_params, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_output_with_shared_params, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_output_without_shared_params, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_partial_flattening, test/distributed/fsdp/test_fsdp_flatten_params.py::TestFlattenParams::test_pnorm_after_step_with_shared_params 2024-06-26T05:36:50.9481532Z 2024-06-26T05:36:50.9482023Z Running distributed/nn/jit/test_instantiator 1/1 ... [2024-06-26 05:36:50.946553] 2024-06-26T05:36:50.9484182Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/nn/jit/test_instantiator.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:36:50.946998] 2024-06-26T05:36:53.8219849Z 2024-06-26T05:36:53.8222387Z distributed/nn/jit/test_instantiator 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.nn.jit.test_instantiator_1.1_915e48a3b31e6d90_.log 2024-06-26T05:36:53.8225705Z Running 3 items in this shard: test/distributed/nn/jit/test_instantiator.py::TestInstantiator::test_get_arg_return_types_from_interface, test/distributed/nn/jit/test_instantiator.py::TestInstantiator::test_instantiate_non_scripted_remote_module_template, test/distributed/nn/jit/test_instantiator.py::TestInstantiator::test_instantiate_scripted_remote_module_template 2024-06-26T05:36:53.8228132Z 2024-06-26T05:36:53.8228638Z Running distributed/fsdp/test_distributed_checkpoint 1/1 ... [2024-06-26 05:36:53.822400] 2024-06-26T05:36:53.8231361Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_distributed_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:36:53.822838] 2024-06-26T05:36:56.6470368Z 2024-06-26T05:36:56.6472509Z distributed/fsdp/test_distributed_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_distributed_checkpoint_1.1_ce222e4fd62f43b6_.log 2024-06-26T05:36:56.6475241Z Running 2 items in this shard: test/distributed/fsdp/test_distributed_checkpoint.py::TestDistributedCheckpoint::test_distributed_checkpoint_state_dict_type0, test/distributed/fsdp/test_distributed_checkpoint.py::TestDistributedCheckpoint::test_distributed_checkpoint_state_dict_type1 2024-06-26T05:36:56.6477306Z 2024-06-26T05:36:56.6477729Z Running distributed/fsdp/test_fsdp_overlap 1/1 ... [2024-06-26 05:36:56.647497] 2024-06-26T05:36:56.6482044Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_overlap.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:36:56.647904] 2024-06-26T05:37:04.8291484Z 2024-06-26T05:37:04.8293427Z distributed/fsdp/test_fsdp_overlap 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_overlap_1.1_8a80e1dc3a82e8b3_.log 2024-06-26T05:37:04.8295952Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_overlap.py::TestForwardOverlapWorldSizeOne::test_forward_overlap, test/distributed/fsdp/test_fsdp_overlap.py::TestForwardOverlapWorldSizeTwo::test_forward_overlap 2024-06-26T05:37:04.8297864Z 2024-06-26T05:37:04.8298368Z Running distributed/fsdp/test_fsdp_memory 1/1 ... [2024-06-26 05:37:04.829585] 2024-06-26T05:37:04.8303075Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_memory.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:37:04.830003] 2024-06-26T05:37:12.9629217Z 2024-06-26T05:37:12.9631123Z distributed/fsdp/test_fsdp_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_memory_1.1_eae8fa3f0fee61f9_.log 2024-06-26T05:37:12.9633385Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_memory.py::TestFSDPMemory::test_fsdp_memory_ckpt_ckpt, test/distributed/fsdp/test_fsdp_memory.py::TestFSDPMemory::test_fsdp_memory_ckpt_no_ckpt 2024-06-26T05:37:12.9634696Z 2024-06-26T05:37:12.9636528Z Running distributed/fsdp/test_fsdp_freezing_weights 1/1 ... [2024-06-26 05:37:12.963384] 2024-06-26T05:37:12.9641744Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_freezing_weights.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:37:12.963820] 2024-06-26T05:38:43.7376464Z 2024-06-26T05:38:43.7379835Z distributed/fsdp/test_fsdp_freezing_weights 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_freezing_weights_1.1_03184518e7618f5c_.log 2024-06-26T05:38:43.7424049Z Running 32 items in this shard: test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_True 2024-06-26T05:38:43.7463561Z 2024-06-26T05:38:43.7464025Z Running distributed/fsdp/test_fsdp_input 1/1 ... [2024-06-26 05:38:43.738338] 2024-06-26T05:38:43.7465969Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_input.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:38:43.738815] 2024-06-26T05:38:51.4233102Z 2024-06-26T05:38:51.4235478Z distributed/fsdp/test_fsdp_input 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_input_1.1_854df29ec96f1e14_.log 2024-06-26T05:38:51.4237678Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_input.py::TestInput::test_input_type_dict, test/distributed/fsdp/test_fsdp_input.py::TestInput::test_input_type_list 2024-06-26T05:38:51.4238824Z 2024-06-26T05:38:51.4239596Z Running distributed/fsdp/test_fsdp_apply 1/1 ... [2024-06-26 05:38:51.423759] 2024-06-26T05:38:51.4245313Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_apply.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:38:51.424188] 2024-06-26T05:39:01.9594200Z 2024-06-26T05:39:01.9596109Z distributed/fsdp/test_fsdp_apply 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_apply_1.1_c88332a77ca564a2_.log 2024-06-26T05:39:01.9598663Z Running 3 items in this shard: test/distributed/fsdp/test_fsdp_apply.py::TestApply::test_apply_in_summon_raises_error, test/distributed/fsdp/test_fsdp_apply.py::TestApply::test_nested_module_apply, test/distributed/fsdp/test_fsdp_apply.py::TestApply::test_transformer_module_apply 2024-06-26T05:39:01.9600394Z 2024-06-26T05:39:01.9601419Z Running distributed/fsdp/test_fsdp_multiple_wrapping 1/1 ... [2024-06-26 05:39:01.959937] 2024-06-26T05:39:01.9607137Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_multiple_wrapping.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:39:01.960383] 2024-06-26T05:39:07.5393491Z 2024-06-26T05:39:07.5395568Z distributed/fsdp/test_fsdp_multiple_wrapping 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_multiple_wrapping_1.1_cccbffc5d99c9b0a_.log 2024-06-26T05:39:07.5397462Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_multiple_wrapping.py::TestMultipleWrapping::test_multiple_wrapping 2024-06-26T05:39:07.5398282Z 2024-06-26T05:39:07.5403790Z Running distributed/fsdp/test_fsdp_state_dict 1/1 ... [2024-06-26 05:39:07.540068] 2024-06-26T05:39:07.5409390Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:39:07.540574] 2024-06-26T05:39:10.7146762Z 2024-06-26T05:39:10.7149001Z distributed/fsdp/test_fsdp_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_state_dict_1.1_9c2c45be7e044f32_.log 2024-06-26T05:39:10.7332874Z Running 179 items in this shard: test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_basic_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_fp16_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_local_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_sharded_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload0_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_False_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_buffers_save_and_load_state_dict_state_dict_type_state_dict_cpu_offload1_mixed_precision_True_state_dict_rank0_and_offload_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_keys_state_dict_type_local_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_keys_state_dict_type_sharded_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_keys_state_dict_type_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_both_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_sharded_state_dict_checkpoint_wrap_source_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_both_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_dest_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_after_wrap_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_fsdp_state_dict_with_activation_checkpoint_state_dict_type_state_dict_checkpoint_wrap_source_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_full_state_dict_missing_unexpected_keys_cleaned, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_local_state_dict_with_empty_ranks, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_False_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_False_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_True_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_local_state_dict_mixed_precision_True_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_False_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_False_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_True_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_sharded_state_dict_mixed_precision_True_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_False_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_False_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_True_state_dict_rank0_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_save_and_load_after_forward_state_dict_state_dict_type_state_dict_mixed_precision_True_state_dict_rank0_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_sharded_load_multi_backend_pg, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_shared_module_and_shared_parameter, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_False_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_False_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_True_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_sharded_state_dict_state_dict_rank0_and_offload_True_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_False_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_False_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_True_fsdp_root_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_load_into_local_module_state_dict_type_state_dict_state_dict_rank0_and_offload_True_fsdp_root_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_rank0_offload_save_load_flow_use_orig_params_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_rank0_offload_save_load_flow_use_orig_params_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_save_load_flow_state_dict_type_local_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_save_load_flow_state_dict_type_sharded_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_save_load_flow_state_dict_type_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_skip_module_state_dict_type_local_state_dict_double_nest_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_skip_module_state_dict_type_sharded_state_dict_double_nest_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_skip_module_state_dict_type_state_dict_double_nest_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_type, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_False_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_sharded_state_dict_prefix_True_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_False_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_ignored_modules_state_dict_type_state_dict_prefix_True_ignore_inner_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_sharded_state_dict_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_sharded_state_dict_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_state_dict_rank0_only_and_offload_False, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_manual_ac_wrapper_state_dict_type_state_dict_rank0_only_and_offload_True, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_shared_parameters_state_dict_type_local_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_shared_parameters_state_dict_type_sharded_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_state_dict_with_shared_parameters_state_dict_type_state_dict, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_torch_save_load, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_world_size_one, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict::test_wrong_state_dict_config, test/distributed/fsdp/test_fsdp_state_dict.py::TestFSDPStateDict4GPUs::test_local_state_dict_reshard 2024-06-26T05:39:10.7635032Z 2024-06-26T05:39:10.7635906Z Running distributed/fsdp/test_fsdp_uneven 1/1 ... [2024-06-26 05:39:10.715854] 2024-06-26T05:39:10.7639940Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_uneven.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:39:10.716334] 2024-06-26T05:39:16.1952026Z 2024-06-26T05:39:16.1953743Z distributed/fsdp/test_fsdp_uneven 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_uneven_1.1_461bcf891b0389af_.log 2024-06-26T05:39:16.1955553Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_uneven.py::TestUnevenParamShard::test_one_iteration 2024-06-26T05:39:16.1956352Z 2024-06-26T05:39:16.1958794Z Running distributed/_composable/fully_shard/test_fully_shard_runtime 1/1 ... [2024-06-26 05:39:16.195653] 2024-06-26T05:39:16.1963898Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_runtime.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:39:16.196061] 2024-06-26T05:39:19.0195274Z 2024-06-26T05:39:19.0197468Z distributed/_composable/fully_shard/test_fully_shard_runtime 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_runtime_1.1_cb898f7935a3a33c_.log 2024-06-26T05:39:19.0200058Z Running 2 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_runtime.py::TestRuntime::test_training, test/distributed/_composable/fully_shard/test_fully_shard_runtime.py::TestRuntime::test_unshard_reshard_order 2024-06-26T05:39:19.0201633Z 2024-06-26T05:39:19.0202069Z Running distributed/fsdp/test_fsdp_comm_hooks 1/1 ... [2024-06-26 05:39:19.019959] 2024-06-26T05:39:19.0207547Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_comm_hooks.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:39:19.020427] 2024-06-26T05:40:06.1956263Z 2024-06-26T05:40:06.1958127Z distributed/fsdp/test_fsdp_comm_hooks 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_comm_hooks_1.1_7e4cf05f37222e0d_.log 2024-06-26T05:40:06.1993325Z Running 28 items in this shard: test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_False_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_bf16_hook_has_wrapping_True_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_behavior_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_behavior_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_behavior_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_False_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_default_communication_hook_initialization_has_wrapping_True_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_False_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_fp16_hook_has_wrapping_True_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_hybrid_strategy, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_non_root_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_non_root_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_non_root_sharding_strategy2, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_submodules_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_submodules_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm_hooks.py::TestCommunicationHooks::test_registering_hook_submodules_sharding_strategy2 2024-06-26T05:40:06.2018033Z 2024-06-26T05:40:06.2018641Z Running distributed/_composable/fsdp/test_fully_shard_comm 1/1 ... [2024-06-26 05:40:06.196182] 2024-06-26T05:40:06.2021002Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_comm.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:40:06.196760] 2024-06-26T05:40:09.0710928Z 2024-06-26T05:40:09.0712967Z distributed/_composable/fsdp/test_fully_shard_comm 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_comm_1.1_4a5a2ff612a649d6_.log 2024-06-26T05:40:09.0722162Z Running 11 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCollectiveOps::test_all_gather_fp32, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCollectiveOps::test_reduce_scatter_fp16, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCollectiveOps::test_reduce_scatter_fp32, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCommunication::test_fully_shard_communication_count, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCommunication::test_manual_reshard_with_reshard_after_forward_false, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardPrefetch::test_fully_shard_backward_prefetch, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardPrefetch::test_set_modules_to_backward_prefetch, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardPrefetch::test_set_modules_to_forward_prefetch, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardUnshardMultiProcess::test_unshard_async, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardUnshardMultiThread::test_unshard_no_param_group, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardUnshardMultiThread::test_unshard_without_lazy_init 2024-06-26T05:40:09.0730426Z 2024-06-26T05:40:09.0730911Z Running distributed/fsdp/test_checkpoint_wrapper 1/1 ... [2024-06-26 05:40:09.071563] 2024-06-26T05:40:09.0732957Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_checkpoint_wrapper.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:40:09.071994] 2024-06-26T05:40:11.9955537Z 2024-06-26T05:40:11.9957307Z distributed/fsdp/test_checkpoint_wrapper 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_checkpoint_wrapper_1.1_7f13227bee5ed1bb_.log 2024-06-26T05:40:11.9963474Z Running 8 items in this shard: test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_apply_activation_checkpointing, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_checkpoint_wrapper_args_kwargs, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_checkpoint_wrapper_cpu_offload, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_checkpoint_wrapper_kwarg_support, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_checkpoint_wrapper_parity, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_forward_missing_attributes, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_fqn, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_load_activation_checkpointed_module 2024-06-26T05:40:11.9968688Z 2024-06-26T05:40:11.9969236Z Running distributed/checkpoint/test_file_system_checkpoint_cpu 1/1 ... [2024-06-26 05:40:11.995990] 2024-06-26T05:40:11.9971704Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_file_system_checkpoint_cpu.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:40:11.996425] 2024-06-26T05:40:41.3623974Z 2024-06-26T05:40:41.3626184Z distributed/checkpoint/test_file_system_checkpoint_cpu 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_file_system_checkpoint_cpu_1.1_72ac24a3066a17a5_.log 2024-06-26T05:40:41.3638027Z Running 12 items in this shard: test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoad::test_read_write_only_tensor_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoad::test_read_write_only_tensor_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoadWithSharedTensor::test_read_write_shard_tensor_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedStateDictSaveLoadWithSharedTensor::test_read_write_shard_tensor_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_rowwise_to_colwise_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_rowwise_to_colwise_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_with_different_shard_plan_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_load_with_different_shard_plan_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_save_load_bytes_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_save_load_bytes_thread_count_2, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_switch_between_sharded_tensor_to_tensor_thread_count_1, test/distributed/checkpoint/test_file_system_checkpoint_cpu.py::TestDistributedReshardOnLoad::test_switch_between_sharded_tensor_to_tensor_thread_count_2 2024-06-26T05:40:41.3649083Z 2024-06-26T05:40:41.3649634Z Running distributed/_composable/test_replicate_with_compiler 1/1 ... [2024-06-26 05:40:41.363210] 2024-06-26T05:40:41.3651800Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_replicate_with_compiler.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:40:41.363651] 2024-06-26T05:40:44.2372703Z 2024-06-26T05:40:44.2375197Z distributed/_composable/test_replicate_with_compiler 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_replicate_with_compiler_1.1_251b88e7457f90be_.log 2024-06-26T05:40:44.2381873Z Running 9 items in this shard: test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_bucketing_coalesced_op, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_bucketing_concat_op, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_backward_only, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_bf16, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_cpu, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_cpu_no_sync, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_fp16, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_gpu, test/distributed/_composable/test_replicate_with_compiler.py::DDP_TP_Test::test_ddp_tp 2024-06-26T05:40:44.2387219Z 2024-06-26T05:40:44.2387723Z Running distributed/fsdp/test_fsdp_sharded_grad_scaler 1/1 ... [2024-06-26 05:40:44.237694] 2024-06-26T05:40:44.2390119Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_sharded_grad_scaler.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:40:44.238110] 2024-06-26T05:41:34.3277315Z 2024-06-26T05:41:34.3279836Z distributed/fsdp/test_fsdp_sharded_grad_scaler 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_sharded_grad_scaler_1.1_3d31b516f2e6c443_.log 2024-06-26T05:41:34.3300096Z Running 20 items in this shard: test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardGradScaler::test_grad_scaling, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardGradScaler::test_inf_gradients_skip_optim_step, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardGradScaler::test_scaling_unscaling_sparse, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_none_mixed_precision_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_none_mixed_precision_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_none_none_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_none_none_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_shard_grad_op_mixed_precision_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_shard_grad_op_mixed_precision_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_shard_grad_op_none_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_false_shard_grad_op_none_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_none_mixed_precision_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_none_mixed_precision_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_none_none_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_none_none_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_shard_grad_op_mixed_precision_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_shard_grad_op_mixed_precision_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_shard_grad_op_none_none, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_fsdp_ddp_parity_with_grad_scaler_offload_true_shard_grad_op_none_use_orig_params, test/distributed/fsdp/test_fsdp_sharded_grad_scaler.py::TestShardedGradScalerParityWithDDP::test_sharded_grad_scaler_found_inf 2024-06-26T05:41:34.3318129Z 2024-06-26T05:41:34.3318608Z Running distributed/fsdp/test_fsdp_checkpoint 1/1 ... [2024-06-26 05:41:34.328589] 2024-06-26T05:41:34.3320732Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:41:34.329073] 2024-06-26T05:42:24.5014796Z 2024-06-26T05:42:24.5016455Z distributed/fsdp/test_fsdp_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_checkpoint_1.1_a6c61f83d9618dbd_.log 2024-06-26T05:42:24.5035466Z Running 17 items in this shard: test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpointSubmodule::test_checkpoint_submodule_use_reentrant_False 2024-06-26T05:42:24.5050718Z 2024-06-26T05:42:24.5051218Z Running distributed/fsdp/test_wrap 1/1 ... [2024-06-26 05:42:24.501946] 2024-06-26T05:42:24.5053137Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_wrap.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:42:24.502447] 2024-06-26T05:43:38.1356856Z 2024-06-26T05:43:38.1358988Z distributed/fsdp/test_wrap 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_wrap_1.1_b7ab1814b6d4ca67_.log 2024-06-26T05:43:38.1400000Z Running 51 items in this shard: test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_bn_always_wrapped_individually, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_False_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_False_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_True_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_True_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_False_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_False_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_True_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_True_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_False_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_False_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_True_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_True_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_False_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_False_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_True_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_True_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_False_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_False_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_True_cuda_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_True_cuda_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_wrap_batchnorm_individually_use_or_policy_False, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_wrap_batchnorm_individually_use_or_policy_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_always_wrap, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_always_wrap_with_ignored_modules_wrap_method0, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_always_wrap_with_ignored_modules_wrap_method1, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_api, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_exclude_wrap, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_exclude_wrap_include_children, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_force_leaf, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_force_leaf_custom, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode0_cpu_offload0_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode0_cpu_offload0_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode0_cpu_offload1_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode0_cpu_offload1_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode1_cpu_offload0_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode1_cpu_offload0_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode1_cpu_offload1_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_cuda_init_mode1_cpu_offload1_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_with_ignored_modules_wrap_method0, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_with_ignored_modules_wrap_method1, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_custom_policy, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_frozen_params, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_module_wrap_policy, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_module_wrap_policy_callable, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_transformer_auto_wrap_policy, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_disabled_outside_context, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_override_defaults, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_wrap_method0, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_wrap_method1, test/distributed/fsdp/test_wrap.py::TestWrapUtils::test_validate_frozen_params 2024-06-26T05:43:38.1430329Z 2024-06-26T05:43:38.1430958Z Running distributed/_composable/fsdp/test_fully_shard_extensions 1/1 ... [2024-06-26 05:43:38.136378] 2024-06-26T05:43:38.1433243Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_extensions.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:43:38.136835] 2024-06-26T05:43:43.8152858Z 2024-06-26T05:43:43.8155000Z distributed/_composable/fsdp/test_fully_shard_extensions 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_extensions_1.1_e62e45ae76f69358_.log 2024-06-26T05:43:43.8158997Z Running 3 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_extensions.py::TestFullyShardAllGatherExtensionsMultiProcess::test_all_gather_extensions_train_parity, test/distributed/_composable/fsdp/test_fully_shard_extensions.py::TestFullyShardAllGatherExtensionsMultiThread::test_all_gather_extensions_end_to_end, test/distributed/_composable/fsdp/test_fully_shard_extensions.py::TestFullyShardAllGatherExtensionsMultiThread::test_all_gather_extensions_monkey_patch 2024-06-26T05:43:43.8162064Z 2024-06-26T05:43:43.8162556Z Running distributed/fsdp/test_fsdp_tp_integration 1/1 ... [2024-06-26 05:43:43.815764] 2024-06-26T05:43:43.8165418Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_tp_integration.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:43:43.816182] 2024-06-26T05:43:54.8566733Z 2024-06-26T05:43:54.8569424Z distributed/fsdp/test_fsdp_tp_integration 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_tp_integration_1.1_ba548e68f370647a_.log 2024-06-26T05:43:54.8574735Z Running 3 items in this shard: test/distributed/fsdp/test_fsdp_tp_integration.py::TestTPFSDPIntegration::test_fsdp_tp_extension_grad, test/distributed/fsdp/test_fsdp_tp_integration.py::TestTPFSDPIntegration::test_fsdp_tp_integration, test/distributed/fsdp/test_fsdp_tp_integration.py::TestTPFSDPIntegration::test_fsdp_tp_sync_module_state 2024-06-26T05:43:54.8578118Z 2024-06-26T05:43:54.8578898Z Running distributed/fsdp/test_fsdp_traversal 1/1 ... [2024-06-26 05:43:54.857387] 2024-06-26T05:43:54.8583945Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_traversal.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:43:54.857942] 2024-06-26T05:44:00.2863848Z 2024-06-26T05:44:00.2865508Z distributed/fsdp/test_fsdp_traversal 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_traversal_1.1_44453e9b30ccf47f_.log 2024-06-26T05:44:00.2867181Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_traversal.py::TestTraversal::test_fsdp_modules 2024-06-26T05:44:00.2867862Z 2024-06-26T05:44:00.2870555Z Running distributed/fsdp/test_fsdp_fine_tune 1/1 ... [2024-06-26 05:44:00.286859] 2024-06-26T05:44:00.2875720Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_fine_tune.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:44:00.287278] 2024-06-26T05:44:03.2106590Z 2024-06-26T05:44:03.2108503Z distributed/fsdp/test_fsdp_fine_tune 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_fine_tune_1.1_020f32c8ca30608c_.log 2024-06-26T05:44:03.2111724Z Running 4 items in this shard: test/distributed/fsdp/test_fsdp_fine_tune.py::TestFSDPFineTune::test_backward_reshard_hooks, test/distributed/fsdp/test_fsdp_fine_tune.py::TestFSDPFineTune::test_hooks_multi_traversal, test/distributed/fsdp/test_fsdp_fine_tune.py::TestFSDPFineTune::test_parity_with_ddp, test/distributed/fsdp/test_fsdp_fine_tune.py::TestFSDPFineTune::test_parity_with_non_frozen_fsdp 2024-06-26T05:44:03.2114328Z 2024-06-26T05:44:03.2114774Z Running distributed/test_c10d_object_collectives 1/1 ... [2024-06-26 05:44:03.211140] 2024-06-26T05:44:03.2119147Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_object_collectives.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:44:03.211573] 2024-06-26T05:44:28.9206010Z 2024-06-26T05:44:28.9207589Z distributed/test_c10d_object_collectives 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_object_collectives_1.1_60bd2e0ca47348d9_.log 2024-06-26T05:44:28.9214266Z Running 9 items in this shard: test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_all_gather_object, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_broadcast_object_list, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_gather_object, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_scatter_object_list, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_send_recv_object_list, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_subpg_all_gather_object, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_subpg_broadcast_object, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_subpg_gather_object, test/distributed/test_c10d_object_collectives.py::TestObjectCollectives::test_subpg_scatter_object 2024-06-26T05:44:28.9219405Z 2024-06-26T05:44:28.9219810Z Running distributed/_shard/test_sharder 1/1 ... [2024-06-26 05:44:28.921084] 2024-06-26T05:44:28.9221833Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/test_sharder.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:44:28.921533] 2024-06-26T05:44:31.3440161Z 2024-06-26T05:44:31.3441940Z distributed/_shard/test_sharder 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.test_sharder_1.1_2db4f35583aebb47_.log 2024-06-26T05:44:31.3444638Z Running 2 items in this shard: test/distributed/_shard/test_sharder.py::TestCustomSharder::test_custom_sharder, test/distributed/_shard/test_sharder.py::TestCustomSharder::test_custom_sharder_errors 2024-06-26T05:44:31.3446192Z 2024-06-26T05:44:31.3447098Z Running distributed/_shard/sharded_optim/test_sharded_optim 1/1 ... [2024-06-26 05:44:31.344464] 2024-06-26T05:44:31.3451874Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_optim/test_sharded_optim.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:44:31.344891] 2024-06-26T05:44:33.8174871Z 2024-06-26T05:44:33.8176819Z distributed/_shard/sharded_optim/test_sharded_optim 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_optim.test_sharded_optim_1.1_2e9ebdbcab3f334a_.log 2024-06-26T05:44:33.8179665Z Running 2 items in this shard: test/distributed/_shard/sharded_optim/test_sharded_optim.py::TestShardedOptimizer::test_named_params_with_sharded_tensor, test/distributed/_shard/sharded_optim/test_sharded_optim.py::TestShardedOptimizer::test_sharded_optim 2024-06-26T05:44:33.8181298Z 2024-06-26T05:44:33.8182481Z Running distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint 1/1 ... [2024-06-26 05:44:33.818031] 2024-06-26T05:44:33.8188113Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:44:33.818479] 2024-06-26T05:44:41.6989869Z 2024-06-26T05:44:41.6992258Z distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint_1.1_a1a63449716358d1_.log 2024-06-26T05:44:41.6995975Z Running 2 items in this shard: test/distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint.py::TestOptimStateCheckpointing::test_optim_state_dict_save_load, test/distributed/_composable/fully_shard/test_fully_shard_optim_checkpoint.py::TestOptimStateCheckpointing::test_optim_state_dict_submodule_fully_shard 2024-06-26T05:44:41.6997861Z 2024-06-26T05:44:41.6998339Z Running distributed/fsdp/test_fsdp_multiple_forward 1/1 ... [2024-06-26 05:44:41.699459] 2024-06-26T05:44:41.7001931Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_multiple_forward.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:44:41.699877] 2024-06-26T05:44:47.5792176Z 2024-06-26T05:44:47.5794755Z distributed/fsdp/test_fsdp_multiple_forward 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_multiple_forward_1.1_237c0b3538c50cac_.log 2024-06-26T05:44:47.5796577Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_multiple_forward.py::TestMultiForward::test_multi_forward 2024-06-26T05:44:47.5797342Z 2024-06-26T05:44:47.5800115Z Running distributed/fsdp/test_fsdp_meta 1/1 ... [2024-06-26 05:44:47.579763] 2024-06-26T05:44:47.5805488Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_meta.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:44:47.580231] 2024-06-26T05:45:11.4418443Z 2024-06-26T05:45:11.4420051Z distributed/fsdp/test_fsdp_meta 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_meta_1.1_6b99eb242f0d8515_.log 2024-06-26T05:45:11.4433879Z Running 15 items in this shard: test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_bad_arg_meta, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_bad_arg_torchdistx, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_meta_device_with_mixed_precision, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_meta_device_default_init_auto_wrap_False, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_meta_device_default_init_auto_wrap_True, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_meta_device_reset_params_auto_wrap_False, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_meta_device_reset_params_auto_wrap_True, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_torchdistX_default_init_auto_wrap_False, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_torchdistX_default_init_auto_wrap_True, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_torchdistX_init_fn_auto_wrap_False, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_torchdistX_init_fn_auto_wrap_True, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_simple_model_with_meta_device_default_init, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_simple_model_with_meta_device_reset_params, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_simple_model_with_torchdistX_default_init, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_simple_model_with_torchdistX_init_fn 2024-06-26T05:45:11.4443904Z 2024-06-26T05:45:11.4444654Z Running distributed/_shard/sharding_spec/test_sharding_spec 1/1 ... [2024-06-26 05:45:11.442479] 2024-06-26T05:45:11.4446828Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharding_spec/test_sharding_spec.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:45:11.442908] 2024-06-26T05:45:17.4729182Z 2024-06-26T05:45:17.4731055Z distributed/_shard/sharding_spec/test_sharding_spec 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharding_spec.test_sharding_spec_1.1_1d5103ce7f1b51ce_.log 2024-06-26T05:45:17.4739682Z Running 11 items in this shard: test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_check_overlapping, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_chunked_sharding_spec, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_device_placement, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_enumerable_sharding_spec, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_get_chunk_sharding_params, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_get_chunked_dim_size, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_get_split_size, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestShardingSpec::test_infer_sharding_spec_from_shards_metadata, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestCustomShardingSpec::test_custom_sharding_spec, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestCustomShardingSpec::test_custom_sharding_spec_shard_tensor, test/distributed/_shard/sharding_spec/test_sharding_spec.py::TestCustomShardingSpec::test_custom_sharding_spec_tensor_ctor 2024-06-26T05:45:17.4746666Z 2024-06-26T05:45:17.4747142Z Running distributed/fsdp/test_fsdp_unshard_params 1/1 ... [2024-06-26 05:45:17.473342] 2024-06-26T05:45:17.4749192Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_unshard_params.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:45:17.473745] 2024-06-26T05:45:59.4615886Z 2024-06-26T05:45:59.4618018Z distributed/fsdp/test_fsdp_unshard_params 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_unshard_params_1.1_d8d5aa52ebaaa131_.log 2024-06-26T05:45:59.4629250Z Running 15 items in this shard: test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_named_parameters_and_buffers, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_param_data, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_recurse, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_respects_reshard, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_writeback, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_singleton_param_writeback, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_submodule, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_with_grads_core, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_with_grads_none_grads, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsNoShard::test_unshard_params_param_data_no_shard, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsNoShard::test_unshard_params_writeback_no_shard, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_offload_to_cpu_no_shard_raises, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_rank0_only_with_writeback_raises, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_unshard_params_from_backward_raises, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_unshard_params_from_forward_raises 2024-06-26T05:45:59.4638844Z 2024-06-26T05:45:59.4639335Z Running distributed/rpc/test_tensorpipe_agent 2/3 ... [2024-06-26 05:45:59.462274] 2024-06-26T05:45:59.4641580Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_tensorpipe_agent.py', '--shard-id=2', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:45:59.462798] 2024-06-26T05:59:10.7377415Z 2024-06-26T05:59:10.7379625Z distributed/rpc/test_tensorpipe_agent 2/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_tensorpipe_agent_2.3_aa8e9a201e8e3889_.log 2024-06-26T05:59:10.7471375Z Running 116 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_chained_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_build_rpc_profiling_key, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_multi, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_none, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_dist_init_decorator, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_duplicate_name_2, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_function_not_on_callee, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_in_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_wait_twice, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_get_worker_infos, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_rpc_twice, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_int_callee, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_builtin_remote_ret, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_non_garbage_collected_user_rref_due_to_local_circular_dependency, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_owner_rref_backward, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_async_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_remote_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_instance_method, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_multi_async_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_nested_pickle, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rpc_rref_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_reinit, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_context_debug_info, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_forward_chain, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_get_future, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_slow_init, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_scalar_add, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_add, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_rpc_arg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_send_to_rank, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_shutdown_followed_by_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rref_after_shutdown, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_raise_in_user_func, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_with_partial_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_dense, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_twice_dense, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_worker_info_pickle, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_world_size_one, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_autograd_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_tensor_dims, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_round_trips, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_node_failure, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_nested, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_script_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_self, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_unused_send_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_tensor_with_grad, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_dist_autograd_profiling, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_only_on_return_value_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_python_remote_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_nested_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_rpc_complex_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_none_grads, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_add_done_callback, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_args_and_kwargs_contain_different_types, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote_multi, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_fork_in_jit_with_profiling, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_script_function_that_not_exists_remotely_from_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_chain, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_local_rref_local_value, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_record_function_jit_end_callbacks_with_fork, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_module, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_throw, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_as_arg_and_return, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_functions_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_unexepected_kwarg_is_specified, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_dist_backward, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_get_gradients, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_jit_fork_within_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_restore_context_after_swtich_to_jit_thread, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_sync_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_send_remote_module_with_a_new_attribute_not_pickled_over_the_wire, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_train_eval, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_create_remote_module_from_module_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire_script_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_inside, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_no_ddp, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_builtin_remote_ret_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc_without_rank, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_py_udf_remote_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_my_parameter_server_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_remote_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rref_stress_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rpc_rref_args_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_get_type_timeout_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_proxy_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_py_udf_remote_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_self_remote_arg_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_set_and_get_num_worker_threads, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_no_grad_on_tensor_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_multi_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_self_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_no_grad_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_call_itself_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_multiple_backward_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_nested_backward_accumulate_grads_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_no_graph_with_tensors_not_require_grad_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_trainer_ps_sparse 2024-06-26T05:59:10.7540079Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_chained_remote 2024-06-26T05:59:10.7541602Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise_async 2024-06-26T05:59:10.7543092Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_remote 2024-06-26T05:59:10.7544552Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_build_rpc_profiling_key 2024-06-26T05:59:10.7545875Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_multi 2024-06-26T05:59:10.7547124Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_none 2024-06-26T05:59:10.7548396Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_simple 2024-06-26T05:59:10.7549700Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_dist_init_decorator 2024-06-26T05:59:10.7550984Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_duplicate_name_2 2024-06-26T05:59:10.7552499Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_function_not_on_callee 2024-06-26T05:59:10.7553805Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done 2024-06-26T05:59:10.7555093Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done_exception 2024-06-26T05:59:10.7556371Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_in_rpc 2024-06-26T05:59:10.7557639Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_wait_twice 2024-06-26T05:59:10.7558933Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_get_worker_infos 2024-06-26T05:59:10.7560219Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_rpc_twice 2024-06-26T05:59:10.7561451Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_int_callee 2024-06-26T05:59:10.7562745Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_builtin_remote_ret 2024-06-26T05:59:10.7564147Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_remote 2024-06-26T05:59:10.7565664Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_non_garbage_collected_user_rref_due_to_local_circular_dependency 2024-06-26T05:59:10.7567214Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_owner_rref_backward 2024-06-26T05:59:10.7568771Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin_single_threaded 2024-06-26T05:59:10.7570271Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_udf 2024-06-26T05:59:10.7571677Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_async_rpc 2024-06-26T05:59:10.7573123Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_remote_rpc 2024-06-26T05:59:10.7574570Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin 2024-06-26T05:59:10.7576088Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin_single_threaded 2024-06-26T05:59:10.7577558Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_instance_method 2024-06-26T05:59:10.7578906Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_multi_async_call 2024-06-26T05:59:10.7580211Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_nested_pickle 2024-06-26T05:59:10.7581496Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rpc_rref_args 2024-06-26T05:59:10.7582711Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_reinit 2024-06-26T05:59:10.7583985Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_context_debug_info 2024-06-26T05:59:10.7585318Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_forward_chain 2024-06-26T05:59:10.7586614Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_get_future 2024-06-26T05:59:10.7587896Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_slow_init 2024-06-26T05:59:10.7589236Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_scalar_add 2024-06-26T05:59:10.7590458Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_add 2024-06-26T05:59:10.7591750Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_rpc_arg 2024-06-26T05:59:10.7593133Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_send_to_rank 2024-06-26T05:59:10.7594450Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_shutdown_followed_by_rpc 2024-06-26T05:59:10.7595822Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rref_after_shutdown 2024-06-26T05:59:10.7597199Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_raise_in_user_func 2024-06-26T05:59:10.7598635Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_with_partial_exception 2024-06-26T05:59:10.7600037Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_dense 2024-06-26T05:59:10.7601418Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_twice_dense 2024-06-26T05:59:10.7602775Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_worker_info_pickle 2024-06-26T05:59:10.7604152Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_world_size_one 2024-06-26T05:59:10.7605587Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_autograd_context 2024-06-26T05:59:10.7607079Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_tensor_dims 2024-06-26T05:59:10.7608644Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_round_trips 2024-06-26T05:59:10.7610132Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_node_failure 2024-06-26T05:59:10.7611584Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_nested 2024-06-26T05:59:10.7613068Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_script_call 2024-06-26T05:59:10.7614558Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_self 2024-06-26T05:59:10.7616043Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_unused_send_function 2024-06-26T05:59:10.7617603Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_tensor_with_grad 2024-06-26T05:59:10.7619125Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_dist_autograd_profiling 2024-06-26T05:59:10.7620649Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_only_on_return_value_remote 2024-06-26T05:59:10.7622161Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_call 2024-06-26T05:59:10.7623687Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_python_remote_call 2024-06-26T05:59:10.7625220Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_nested_context 2024-06-26T05:59:10.7626621Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_rpc_complex_args 2024-06-26T05:59:10.7628048Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_none_grads 2024-06-26T05:59:10.7629447Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_add_done_callback 2024-06-26T05:59:10.7630900Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_args_and_kwargs_contain_different_types 2024-06-26T05:59:10.7632431Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote 2024-06-26T05:59:10.7633857Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote_multi 2024-06-26T05:59:10.7635332Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_fork_in_jit_with_profiling 2024-06-26T05:59:10.7636960Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_script_function_that_not_exists_remotely_from_script 2024-06-26T05:59:10.7638477Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_chain 2024-06-26T05:59:10.7639825Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_local_rref_local_value 2024-06-26T05:59:10.7641321Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_record_function_jit_end_callbacks_with_fork 2024-06-26T05:59:10.7642909Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_module 2024-06-26T05:59:10.7644383Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_throw 2024-06-26T05:59:10.7645764Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_as_arg_and_return 2024-06-26T05:59:10.7647230Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_functions_not_supported 2024-06-26T05:59:10.7648748Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_unexepected_kwarg_is_specified 2024-06-26T05:59:10.7650190Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_dist_backward 2024-06-26T05:59:10.7651608Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_get_gradients 2024-06-26T05:59:10.7653074Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_jit_fork_within_context 2024-06-26T05:59:10.7654685Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_restore_context_after_swtich_to_jit_thread 2024-06-26T05:59:10.7656518Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_sync_script 2024-06-26T05:59:10.7658195Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_send_remote_module_with_a_new_attribute_not_pickled_over_the_wire 2024-06-26T05:59:10.7659884Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_train_eval 2024-06-26T05:59:10.7661463Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_create_remote_module_from_module_rref 2024-06-26T05:59:10.7663329Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire 2024-06-26T05:59:10.7665189Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire_script_not_supported 2024-06-26T05:59:10.7666950Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_inside 2024-06-26T05:59:10.7668489Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_no_ddp 2024-06-26T05:59:10.7670036Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote 2024-06-26T05:59:10.7671653Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_builtin_remote_ret_sparse 2024-06-26T05:59:10.7673292Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc 2024-06-26T05:59:10.7674897Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc_without_rank 2024-06-26T05:59:10.7676524Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_multi_py_udf_remote_sparse 2024-06-26T05:59:10.7678127Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_my_parameter_server_sparse 2024-06-26T05:59:10.7679702Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_remote_sparse 2024-06-26T05:59:10.7681359Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rref_stress_sparse 2024-06-26T05:59:10.7682933Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rpc_rref_args_sparse 2024-06-26T05:59:10.7684579Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_sparse 2024-06-26T05:59:10.7686172Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_get_type_timeout_blocking 2024-06-26T05:59:10.7687752Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_proxy_timeout 2024-06-26T05:59:10.7689288Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_py_udf_remote_sparse 2024-06-26T05:59:10.7690963Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_self_remote_arg_sparse 2024-06-26T05:59:10.7692668Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_set_and_get_num_worker_threads 2024-06-26T05:59:10.7694376Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_no_grad_on_tensor_sparse 2024-06-26T05:59:10.7696121Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_multi_sparse 2024-06-26T05:59:10.7697856Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_self_sparse 2024-06-26T05:59:10.7699567Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_sparse 2024-06-26T05:59:10.7701418Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_no_grad_sparse 2024-06-26T05:59:10.7703200Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_call_sparse 2024-06-26T05:59:10.7705008Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_call_itself_sparse 2024-06-26T05:59:10.7706803Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_multiple_backward_sparse 2024-06-26T05:59:10.7708599Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_nested_backward_accumulate_grads_sparse 2024-06-26T05:59:10.7710490Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_no_graph_with_tensors_not_require_grad_sparse 2024-06-26T05:59:10.7712360Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_trainer_ps_sparse 2024-06-26T05:59:10.7713280Z 2024-06-26T05:59:10.7713838Z Running distributed/rpc/cuda/test_tensorpipe_agent 1/1 ... [2024-06-26 05:59:10.743360] 2024-06-26T05:59:10.7715990Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/cuda/test_tensorpipe_agent.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 05:59:10.743780] 2024-06-26T06:09:18.9739375Z 2024-06-26T06:09:18.9742152Z distributed/rpc/cuda/test_tensorpipe_agent 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.cuda.test_tensorpipe_agent_1.1_986ba45a62725a6a_.log 2024-06-26T06:09:18.9875842Z Running 99 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRpcTest::test_profiler_remote_cuda, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_simple, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation_gpu_root, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device_script, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_invalid_devices, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_valid_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_nested_with_cuda_future, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_with_cuda_future, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_callback_changes_devices, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_sparse_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_sparse_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_sparse_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_int, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_str, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_not_cuda, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_modify_tensor_inplace, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_replace_tensor, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_value_on_bad_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_multi, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested_multi, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default_to_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_5, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_6, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_7, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_8, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_5, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_6, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_7, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_8, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default_to_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_non_default, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_in_options, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_local_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_remote_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_min_device, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_many_to_one, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_loop, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_not_timeout, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote_response, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response_loop, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu_self, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_one_to_many, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_remote, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu_self, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_wrong_worker_name, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_mismatch, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch_reverse, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization5, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization1, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization2, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization3, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization4, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_with_unpickleable_attributes, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_tensor_view_as_return_value, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_device_maps_backward_pass, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_dist_autograd_sync_streams, test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_gradients_synchronizations 2024-06-26T06:09:18.9947407Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRpcTest::test_profiler_remote_cuda 2024-06-26T06:09:18.9948906Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_simple 2024-06-26T06:09:18.9950547Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation 2024-06-26T06:09:18.9952266Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDistAutogradTest::test_gpu_to_cpu_continuation_gpu_root 2024-06-26T06:09:18.9953953Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device 2024-06-26T06:09:18.9955650Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_input_moved_to_cuda_device_script 2024-06-26T06:09:18.9957266Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_invalid_devices 2024-06-26T06:09:18.9958771Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaRemoteModuleTest::test_valid_device 2024-06-26T06:09:18.9960389Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeCudaDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote_gpu 2024-06-26T06:09:18.9962216Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_nested_with_cuda_future 2024-06-26T06:09:18.9964157Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_async_execution_with_cuda_future 2024-06-26T06:09:18.9966153Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_callback_changes_devices 2024-06-26T06:09:18.9968011Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_sparse_tensor 2024-06-26T06:09:18.9969857Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_cuda_tensor 2024-06-26T06:09:18.9971811Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_sparse_tensor 2024-06-26T06:09:18.9973859Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_custom_class_with_cuda_tensor 2024-06-26T06:09:18.9975847Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_sparse_tensor 2024-06-26T06:09:18.9977808Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_can_extract_list_with_cuda_tensor 2024-06-26T06:09:18.9979636Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_device 2024-06-26T06:09:18.9981365Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_int 2024-06-26T06:09:18.9983065Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_as_str 2024-06-26T06:09:18.9984894Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_device_not_cuda 2024-06-26T06:09:18.9986650Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_modify_tensor_inplace 2024-06-26T06:09:18.9988405Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_replace_tensor 2024-06-26T06:09:18.9990138Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_cuda_future_value_on_bad_device 2024-06-26T06:09:18.9991883Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream 2024-06-26T06:09:18.9993523Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_multi 2024-06-26T06:09:18.9995195Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested 2024-06-26T06:09:18.9996892Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_custom_stream_nested_multi 2024-06-26T06:09:18.9998563Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu 2024-06-26T06:09:19.0000248Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_default 2024-06-26T06:09:19.0002032Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_cpu_to_gpu_non_default 2024-06-26T06:09:19.0003786Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default 2024-06-26T06:09:19.0005769Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_default_to_non_default 2024-06-26T06:09:19.0007555Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_1 2024-06-26T06:09:19.0009248Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_2 2024-06-26T06:09:19.0010933Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_3 2024-06-26T06:09:19.0012599Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_4 2024-06-26T06:09:19.0014288Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_5 2024-06-26T06:09:19.0015970Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_6 2024-06-26T06:09:19.0017663Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_7 2024-06-26T06:09:19.0019330Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_8 2024-06-26T06:09:19.0021045Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_1 2024-06-26T06:09:19.0022879Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_2 2024-06-26T06:09:19.0024616Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_3 2024-06-26T06:09:19.0026342Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_4 2024-06-26T06:09:19.0028089Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_5 2024-06-26T06:09:19.0029827Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_6 2024-06-26T06:09:19.0031558Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_7 2024-06-26T06:09:19.0033361Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_mixed_self_8 2024-06-26T06:09:19.0035100Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default 2024-06-26T06:09:19.0036887Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_non_default_to_default 2024-06-26T06:09:19.0038684Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_default 2024-06-26T06:09:19.0040448Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_map_gpu_to_cpu_non_default 2024-06-26T06:09:19.0042166Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_gpu 2024-06-26T06:09:19.0043885Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_in_options 2024-06-26T06:09:19.0045820Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_local_device 2024-06-26T06:09:19.0047650Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_max_remote_device 2024-06-26T06:09:19.0049478Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_invalid_min_device 2024-06-26T06:09:19.0051215Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_many_to_one 2024-06-26T06:09:19.0052941Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config 2024-06-26T06:09:19.0054675Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_loop 2024-06-26T06:09:19.0056491Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_not_timeout 2024-06-26T06:09:19.0058319Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote 2024-06-26T06:09:19.0060170Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_remote_response 2024-06-26T06:09:19.0062118Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response 2024-06-26T06:09:19.0063968Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_missing_config_response_loop 2024-06-26T06:09:19.0065745Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu 2024-06-26T06:09:19.0067455Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_multi_gpu_self 2024-06-26T06:09:19.0069158Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_one_to_many 2024-06-26T06:09:19.0070840Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_remote 2024-06-26T06:09:19.0072581Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu 2024-06-26T06:09:19.0074334Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_return_to_gpu_self 2024-06-26T06:09:19.0076138Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_maps_wrong_worker_name 2024-06-26T06:09:19.0077828Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_device_mismatch 2024-06-26T06:09:19.0079486Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch 2024-06-26T06:09:19.0081226Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_devices_option_mismatch_reverse 2024-06-26T06:09:19.0083097Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization1 2024-06-26T06:09:19.0085044Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization2 2024-06-26T06:09:19.0086867Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization3 2024-06-26T06:09:19.0088681Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_owner_rref_forward_synchronization4 2024-06-26T06:09:19.0090457Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization1 2024-06-26T06:09:19.0092209Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization2 2024-06-26T06:09:19.0093954Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization3 2024-06-26T06:09:19.0095708Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization4 2024-06-26T06:09:19.0097459Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_as_arg_synchronization5 2024-06-26T06:09:19.0099208Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization1 2024-06-26T06:09:19.0100971Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization2 2024-06-26T06:09:19.0102848Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization3 2024-06-26T06:09:19.0104606Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_forward_synchronization4 2024-06-26T06:09:19.0106349Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization1 2024-06-26T06:09:19.0108103Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization2 2024-06-26T06:09:19.0109873Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization3 2024-06-26T06:09:19.0111638Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_to_here_synchronization4 2024-06-26T06:09:19.0113490Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_rref_with_unpickleable_attributes 2024-06-26T06:09:19.0115263Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentCudaRpcTest::test_tensor_view_as_return_value 2024-06-26T06:09:19.0117024Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_device_maps_backward_pass 2024-06-26T06:09:19.0118812Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_dist_autograd_sync_streams 2024-06-26T06:09:19.0120589Z Running 1 items in this shard: test/distributed/rpc/cuda/test_tensorpipe_agent.py::TensorPipeTensorPipeCudaDistAutogradTest::test_gradients_synchronizations 2024-06-26T06:09:19.0121575Z 2024-06-26T06:09:19.0122049Z Running distributed/test_pg_wrapper 1/1 ... [2024-06-26 06:09:18.979491] 2024-06-26T06:09:19.0124223Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_pg_wrapper.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:09:18.980168] 2024-06-26T06:10:34.8333236Z 2024-06-26T06:10:34.8335487Z distributed/test_pg_wrapper 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_pg_wrapper_1.1_207b1911e44452fb_.log 2024-06-26T06:10:34.8347429Z Running 17 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_coalescing_manager_debug_mode_detail, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_hang, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_detail, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_off, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_debug_level_detail_no_gloo, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_new_group_no_gloo, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_hang, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode_off, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_debug_mode 2024-06-26T06:10:34.8358566Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_coalescing_manager_debug_mode_detail 2024-06-26T06:10:34.8359983Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_hang 2024-06-26T06:10:34.8361430Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_detail 2024-06-26T06:10:34.8363010Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_off 2024-06-26T06:10:34.8365060Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch 2024-06-26T06:10:34.8367007Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch_debug_mode 2024-06-26T06:10:34.8368468Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_debug_level_detail_no_gloo 2024-06-26T06:10:34.8369846Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_new_group_no_gloo 2024-06-26T06:10:34.8371132Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_hang 2024-06-26T06:10:34.8372509Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda 2024-06-26T06:10:34.8374190Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda_debug_mode 2024-06-26T06:10:34.8375744Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode 2024-06-26T06:10:34.8377270Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode_off 2024-06-26T06:10:34.8378745Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch 2024-06-26T06:10:34.8380149Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda 2024-06-26T06:10:34.8381646Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda_debug_mode 2024-06-26T06:10:34.8387641Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_debug_mode 2024-06-26T06:10:34.8388526Z 2024-06-26T06:10:34.8389024Z Running distributed/algorithms/test_join 1/1 ... [2024-06-26 06:10:34.834089] 2024-06-26T06:10:34.8390986Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/algorithms/test_join.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:10:34.834509] 2024-06-26T06:11:01.0453060Z 2024-06-26T06:11:01.0454585Z distributed/algorithms/test_join 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.algorithms.test_join_1.1_fd3c909982d13987_.log 2024-06-26T06:11:01.0460536Z Running 9 items in this shard: test/distributed/algorithms/test_join.py::TestJoin::test_join_kwargs, test/distributed/algorithms/test_join.py::TestJoin::test_multiple_joinable_disable, test/distributed/algorithms/test_join.py::TestJoin::test_multiple_joinables, test/distributed/algorithms/test_join.py::TestJoin::test_multiple_joinables_throw, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_disable, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_main_hooks, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_post_hooks, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_throw 2024-06-26T06:11:01.0465189Z 2024-06-26T06:11:01.0465669Z Running distributed/rpc/test_faulty_agent 1/1 ... [2024-06-26 06:11:01.045848] 2024-06-26T06:11:01.0467868Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_faulty_agent.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:11:01.046287] 2024-06-26T06:16:33.3695178Z 2024-06-26T06:16:33.3697128Z distributed/rpc/test_faulty_agent 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_faulty_agent_1.1_c88edb2570941c31_.log 2024-06-26T06:16:33.3730746Z Running 28 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_check_failed_messages, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_faulty_messages, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_messages_to_delay, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_no_faulty_messages, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_builtin_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_script_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rref_to_here_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_verify_backend_options, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_context_cleanup_tensor_with_grad, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_verify_backend_options, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_remote_timeout_to_here_in_jit, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_in_jit, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_script_func, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_to_here_timeout_in_jit, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_python, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_torchscript_function 2024-06-26T06:16:33.3750705Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout 2024-06-26T06:16:33.3754593Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout_to_self 2024-06-26T06:16:33.3758159Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_check_failed_messages 2024-06-26T06:16:33.3761477Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_faulty_messages 2024-06-26T06:16:33.3764948Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_messages_to_delay 2024-06-26T06:16:33.3768195Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_no_faulty_messages 2024-06-26T06:16:33.3771596Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout 2024-06-26T06:16:33.3774384Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout_to_self 2024-06-26T06:16:33.3775940Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle 2024-06-26T06:16:33.3777441Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle_to_self 2024-06-26T06:16:33.3778999Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout 2024-06-26T06:16:33.3780715Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout_to_self 2024-06-26T06:16:33.3782207Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_builtin_timeout 2024-06-26T06:16:33.3783554Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_script_timeout 2024-06-26T06:16:33.3784915Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rref_to_here_timeout 2024-06-26T06:16:33.3786352Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout 2024-06-26T06:16:33.3787878Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout_to_self 2024-06-26T06:16:33.3789440Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout 2024-06-26T06:16:33.3791009Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout_to_self 2024-06-26T06:16:33.3792583Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_verify_backend_options 2024-06-26T06:16:33.3794081Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_context_cleanup_tensor_with_grad 2024-06-26T06:16:33.3795655Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_verify_backend_options 2024-06-26T06:16:33.3797177Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_remote_timeout_to_here_in_jit 2024-06-26T06:16:33.3798769Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_in_jit 2024-06-26T06:16:33.3800269Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_script_func 2024-06-26T06:16:33.3801784Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_to_here_timeout_in_jit 2024-06-26T06:16:33.3803220Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_python 2024-06-26T06:16:33.3804761Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_torchscript_function 2024-06-26T06:16:33.3805604Z 2024-06-26T06:16:33.3806253Z Running distributed/_shard/sharded_tensor/test_sharded_tensor_reshard 1/1 ... [2024-06-26 06:16:33.371129] 2024-06-26T06:16:33.3808517Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/test_sharded_tensor_reshard.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:16:33.371551] 2024-06-26T06:16:35.9450857Z 2024-06-26T06:16:35.9454116Z distributed/_shard/sharded_tensor/test_sharded_tensor_reshard 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_reshard_1.1_3d7a675a0cd844d2_.log 2024-06-26T06:16:35.9458997Z Running 2 items in this shard: test/distributed/_shard/sharded_tensor/test_sharded_tensor_reshard.py::TestReshard::test_sharded_tensor_reshard, test/distributed/_shard/sharded_tensor/test_sharded_tensor_reshard.py::TestReshard::test_sharded_tensor_reshard_errors 2024-06-26T06:16:35.9461640Z 2024-06-26T06:16:35.9462551Z Running distributed/_shard/sharded_tensor/ops/test_embedding 1/1 ... [2024-06-26 06:16:35.945831] 2024-06-26T06:16:35.9468067Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_embedding.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:16:35.946412] 2024-06-26T06:16:38.5195255Z 2024-06-26T06:16:38.5197427Z distributed/_shard/sharded_tensor/ops/test_embedding 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_1.1_627769cedf804af2_.log 2024-06-26T06:16:38.5200097Z Running 2 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_embedding.py::TestShardedEmbedding::test_sharded_embedding_colwise, test/distributed/_shard/sharded_tensor/ops/test_embedding.py::TestShardedEmbedding::test_sharded_embedding_rowwise 2024-06-26T06:16:38.5202056Z 2024-06-26T06:16:38.5202611Z Running distributed/_shard/sharded_tensor/ops/test_embedding_bag 1/1 ... [2024-06-26 06:16:38.519950] 2024-06-26T06:16:38.5207615Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_embedding_bag.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:16:38.520369] 2024-06-26T06:16:41.0431877Z 2024-06-26T06:16:41.0434059Z distributed/_shard/sharded_tensor/ops/test_embedding_bag 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_bag_1.1_7957dccbde1acd72_.log 2024-06-26T06:16:41.0437183Z Running 2 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_embedding_bag.py::TestShardedEmbeddingBag::test_sharded_embedding_bag_colwise, test/distributed/_shard/sharded_tensor/ops/test_embedding_bag.py::TestShardedEmbeddingBag::test_sharded_embedding_bag_rowwise 2024-06-26T06:16:41.0439328Z 2024-06-26T06:16:41.0439905Z Running distributed/optim/test_zero_redundancy_optimizer 1/1 ... [2024-06-26 06:16:41.043643] 2024-06-26T06:16:41.0443753Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/optim/test_zero_redundancy_optimizer.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:16:41.044071] 2024-06-26T06:17:51.7707013Z 2024-06-26T06:17:51.7711771Z distributed/optim/test_zero_redundancy_optimizer 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.optim.test_zero_redundancy_optimizer_1.1_aee6ef40716bed63_.log 2024-06-26T06:17:51.7753623Z Running 42 items in this shard: test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_constructor, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_lr_scheduler, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_same_dense_param_type, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_state_dict, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_step_with_extra_inner_key, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_step_with_kwargs, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_step_without_closure, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_zero_grad, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_add_param_group, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_collect_shards, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_False_static_graph_False_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_False_static_graph_False_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_False_static_graph_True_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_False_static_graph_True_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_True_static_graph_False_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_True_static_graph_False_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_True_static_graph_True_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_True_static_graph_True_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_False_static_graph_False_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_False_static_graph_False_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_False_static_graph_True_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_False_static_graph_True_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_True_static_graph_False_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_True_static_graph_False_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_True_static_graph_True_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_True_static_graph_True_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_AdamW_maximize_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_AdamW_maximize_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_Adam_maximize_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_Adam_maximize_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_SGD_maximize_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_SGD_maximize_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_lr_scheduler, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_multiple_param_groups, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_nondefault_process_group, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_sharding, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_step, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_step_with_closure, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_zero_join_cpu, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_zero_join_gpu, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_zero_model_parallel_parameters_as_bucket_view_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_zero_model_parallel_parameters_as_bucket_view_True 2024-06-26T06:17:51.7792729Z 2024-06-26T06:17:51.7793193Z Running distributed/fsdp/test_fsdp_comm 1/1 ... [2024-06-26 06:17:51.771787] 2024-06-26T06:17:51.7795140Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_comm.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:17:51.772236] 2024-06-26T06:18:17.2926033Z 2024-06-26T06:18:17.2927651Z distributed/fsdp/test_fsdp_comm 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_comm_1.1_b6229472297a4e82_.log 2024-06-26T06:18:17.2936771Z Running 10 items in this shard: test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestExplicitUnshard::test_unshard_async_use_orig_params_False, test/distributed/fsdp/test_fsdp_comm.py::TestExplicitUnshard::test_unshard_async_use_orig_params_True 2024-06-26T06:18:17.2944398Z 2024-06-26T06:18:17.2944932Z Running distributed/_shard/sharded_tensor/ops/test_tensor_ops 1/1 ... [2024-06-26 06:18:17.293173] 2024-06-26T06:18:17.2947381Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_tensor_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:18:17.293607] 2024-06-26T06:18:19.8670712Z 2024-06-26T06:18:19.8672541Z distributed/_shard/sharded_tensor/ops/test_tensor_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_tensor_ops_1.1_a3bfb58078273d4a_.log 2024-06-26T06:18:19.8677158Z Running 5 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_clone, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_deep_copy, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_detach, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_inplace_copy, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_set_requires_grad 2024-06-26T06:18:19.8680433Z 2024-06-26T06:18:19.8681123Z Running distributed/_shard/sharded_tensor/ops/test_binary_cmp 1/1 ... [2024-06-26 06:18:19.867595] 2024-06-26T06:18:19.8683490Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_binary_cmp.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:18:19.868024] 2024-06-26T06:18:22.3910282Z 2024-06-26T06:18:22.3912221Z distributed/_shard/sharded_tensor/ops/test_binary_cmp 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_binary_cmp_1.1_a89b440ca1a00325_.log 2024-06-26T06:18:22.3916267Z Running 4 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_binary_cmp.py::TestShardedTensorBinaryOps::test_torch_allclose, test/distributed/_shard/sharded_tensor/ops/test_binary_cmp.py::TestShardedTensorBinaryOps::test_torch_allclose_tensor_specs, test/distributed/_shard/sharded_tensor/ops/test_binary_cmp.py::TestShardedTensorBinaryOps::test_torch_equal, test/distributed/_shard/sharded_tensor/ops/test_binary_cmp.py::TestShardedTensorBinaryOps::test_torch_equal_tensor_specs 2024-06-26T06:18:22.3919488Z 2024-06-26T06:18:22.3920111Z Running distributed/checkpoint/test_state_dict 1/1 ... [2024-06-26 06:18:22.391510] 2024-06-26T06:18:22.3922648Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:18:22.391943] 2024-06-26T06:18:28.4220428Z 2024-06-26T06:18:28.4222463Z distributed/checkpoint/test_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_state_dict_1.1_ec0dd4cb8b7d8ce9_.log 2024-06-26T06:18:28.4236251Z Running 23 items in this shard: test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_activation_ckpt_fqns_ddp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_activation_ckpt_fqns_fsdp1, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_broadcast_from_rank0, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_broadcast_from_rank0_hsdp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_compiled_fsdp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_cpu_offload_full_state_dict, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_ddp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_deprecate_fsdp_api, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_deprecate_partial, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_extra_state, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_flattened_osd, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_frozen_parameters, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_fsdp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_fsdp2, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_fsdp_ddp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_fsdp_root_not_initialized, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_non_persistent_buffers, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_optim_state_dict_param_matching, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_optim_state_dict_tensor_matching, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_shared_weight, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_single_gpu, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_strict, test/distributed/checkpoint/test_state_dict.py::TestNoComm::test_no_dist 2024-06-26T06:18:28.4248827Z 2024-06-26T06:18:28.4249217Z Running distributed/test_c10d_ucc 1/1 ... [2024-06-26 06:18:28.422518] 2024-06-26T06:18:28.4251343Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_ucc.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:18:28.422933] 2024-06-26T06:18:29.7755021Z 2024-06-26T06:18:29.7756530Z distributed/test_c10d_ucc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_ucc_1.1_240c333d8940a9af_.log 2024-06-26T06:18:29.7757533Z 2024-06-26T06:18:29.7761464Z Running distributed/test_c10d_gloo 1/1 ... [2024-06-26 06:18:29.775891] 2024-06-26T06:18:29.7765990Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_gloo.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:18:29.776295] 2024-06-26T06:30:25.8570623Z 2024-06-26T06:30:25.8573005Z distributed/test_c10d_gloo 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_gloo_1.1_64d0e83d84573936_.log 2024-06-26T06:30:25.8654126Z Running 133 items in this shard: test/distributed/test_c10d_gloo.py::RendezvousEnvTest::test_logging_init, test/distributed/test_c10d_gloo.py::TimeoutTest::test_default_store_timeout_gloo, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_async, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_into_tensor_coalesced, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_noncontiguous_input, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_async, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_barrier_implies_wait, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_empty_tensors, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_noncontiguous_input, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_multi_device_constructor, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_round_robin, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_round_robin_create_destroy, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_send_recv_all_to_all, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_set_gloo_pg_timeout, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_cuda_dispatched, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output_unused_param, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_weight_sharing, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_weight_sharing, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_cpu, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_gpu_gloo, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_register_just_once, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_sparse_gradients, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_init, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_return_type, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_find_unused_parameters_when_unused_parameters_empty, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_grad_is_view, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_static_graph, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_integer_list, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_torch_device_list, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_2gpu_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_4gpu_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module_grad_is_view, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output_with_unused_parameters, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_sharded_tensor, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_invalid_powerSGD_state, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_save_load_checkpoint, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients_grad_is_view, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_empty_input, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_only_empty_input, test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward, test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_optimizer, test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_unused_parameters, test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_multi_bucket, test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_single_bucket, test/distributed/test_c10d_gloo.py::ReducerTest::test_single_dtype_single_bucket, test/distributed/test_c10d_gloo.py::CommTest::test_bool_tensors, test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cpu, test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cuda, test/distributed/test_c10d_gloo.py::CommTest::test_gloo_rank_membership, test/distributed/test_c10d_gloo.py::CommTest::test_gloo_warn_not_in_group, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_default, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_subgroup, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_default_pg_gloo, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_gloo_new_group, test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_complex, test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_mismatch, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_all_to_all_single, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allgather_coalesced, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allreduce_coalesced, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_collectives, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_for_all_backends, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_optional_backend, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_monitored_barrier, test/distributed/test_c10d_gloo.py::CompilerTest::test_allgather_work_wait_cpu, test/distributed/test_c10d_gloo.py::CompilerTest::test_allgather_work_wait_gpu, test/distributed/test_c10d_gloo.py::CompilerTest::test_allreduce_work_wait_cpu, test/distributed/test_c10d_gloo.py::CompilerTest::test_allreduce_work_wait_gpu, test/distributed/test_c10d_gloo.py::CompilerTest::test_broadcast_work_wait_cpu, test/distributed/test_c10d_gloo.py::CompilerTest::test_broadcast_work_wait_gpu, test/distributed/test_c10d_gloo.py::CompilerTest::test_consecutive_comm_work_wait_cpu, test/distributed/test_c10d_gloo.py::CompilerTest::test_consecutive_comm_work_wait_gpu, test/distributed/test_c10d_gloo.py::CompilerTest::test_nested_comm_tensor_wrapping, test/distributed/test_c10d_gloo.py::CompilerTest::test_scatter_work_wait_cpu, test/distributed/test_c10d_gloo.py::CompilerTest::test_scatter_work_wait_gpu, test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync, test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_duplicate_pg, test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_sanity_check 2024-06-26T06:30:25.8721153Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::RendezvousEnvTest::test_logging_init 2024-06-26T06:30:25.8722324Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::TimeoutTest::test_default_store_timeout_gloo 2024-06-26T06:30:25.8723517Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics 2024-06-26T06:30:25.8724836Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics_cuda 2024-06-26T06:30:25.8726034Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_checks 2024-06-26T06:30:25.8727277Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_async 2024-06-26T06:30:25.8728573Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_checks 2024-06-26T06:30:25.8729896Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_into_tensor_coalesced 2024-06-26T06:30:25.8731227Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_noncontiguous_input 2024-06-26T06:30:25.8732487Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress 2024-06-26T06:30:25.8733701Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress_cuda 2024-06-26T06:30:25.8734904Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics 2024-06-26T06:30:25.8736123Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics_cuda 2024-06-26T06:30:25.8737457Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_checks 2024-06-26T06:30:25.8738701Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_async 2024-06-26T06:30:25.8739980Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_basics 2024-06-26T06:30:25.8741272Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks 2024-06-26T06:30:25.8742598Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks_cuda 2024-06-26T06:30:25.8743931Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_stress 2024-06-26T06:30:25.8745153Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress 2024-06-26T06:30:25.8746372Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress_cuda 2024-06-26T06:30:25.8747605Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_barrier_implies_wait 2024-06-26T06:30:25.8748803Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics 2024-06-26T06:30:25.8750013Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics_cuda 2024-06-26T06:30:25.8751220Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_checks 2024-06-26T06:30:25.8752479Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress 2024-06-26T06:30:25.8753774Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress_cuda 2024-06-26T06:30:25.8754971Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_empty_tensors 2024-06-26T06:30:25.8756127Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics 2024-06-26T06:30:25.8757304Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics_cuda 2024-06-26T06:30:25.8758469Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_checks 2024-06-26T06:30:25.8759696Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_noncontiguous_input 2024-06-26T06:30:25.8760920Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress 2024-06-26T06:30:25.8762085Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress_cuda 2024-06-26T06:30:25.8763327Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_multi_device_constructor 2024-06-26T06:30:25.8764678Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics 2024-06-26T06:30:25.8765857Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics_cuda 2024-06-26T06:30:25.8767022Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_checks 2024-06-26T06:30:25.8768215Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor 2024-06-26T06:30:25.8769507Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor_coalesced 2024-06-26T06:30:25.8770766Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress 2024-06-26T06:30:25.8772016Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress_cuda 2024-06-26T06:30:25.8773191Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_round_robin 2024-06-26T06:30:25.8774400Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_round_robin_create_destroy 2024-06-26T06:30:25.8775611Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics 2024-06-26T06:30:25.8776794Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics_cuda 2024-06-26T06:30:25.8777978Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_checks 2024-06-26T06:30:25.8779149Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress 2024-06-26T06:30:25.8780328Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress_cuda 2024-06-26T06:30:25.8781558Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_send_recv_all_to_all 2024-06-26T06:30:25.8782781Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_set_gloo_pg_timeout 2024-06-26T06:30:25.8784005Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics 2024-06-26T06:30:25.8785295Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics_cuda 2024-06-26T06:30:25.8786579Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_checks 2024-06-26T06:30:25.8787975Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_cuda_dispatched 2024-06-26T06:30:25.8789293Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output 2024-06-26T06:30:25.8790651Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output_unused_param 2024-06-26T06:30:25.8792146Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_module 2024-06-26T06:30:25.8793649Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_weight_sharing 2024-06-26T06:30:25.8795180Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_False 2024-06-26T06:30:25.8796723Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_True 2024-06-26T06:30:25.8798351Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_False 2024-06-26T06:30:25.8800035Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_True 2024-06-26T06:30:25.8801647Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_False 2024-06-26T06:30:25.8803206Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_True 2024-06-26T06:30:25.8804875Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_weight_sharing 2024-06-26T06:30:25.8806459Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_False 2024-06-26T06:30:25.8808183Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_True 2024-06-26T06:30:25.8809836Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_False 2024-06-26T06:30:25.8811497Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_True 2024-06-26T06:30:25.8813044Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_cpu 2024-06-26T06:30:25.8814511Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_gpu_gloo 2024-06-26T06:30:25.8815994Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_register_just_once 2024-06-26T06:30:25.8817439Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_sparse_gradients 2024-06-26T06:30:25.8818851Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_init 2024-06-26T06:30:25.8820265Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_return_type 2024-06-26T06:30:25.8821819Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_find_unused_parameters_when_unused_parameters_empty 2024-06-26T06:30:25.8823359Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad 2024-06-26T06:30:25.8824884Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_grad_is_view 2024-06-26T06:30:25.8826569Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_static_graph 2024-06-26T06:30:25.8828181Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_integer_list 2024-06-26T06:30:25.8829803Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_torch_device_list 2024-06-26T06:30:25.8845199Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_2gpu_module 2024-06-26T06:30:25.8846718Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_4gpu_module 2024-06-26T06:30:25.8848100Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module 2024-06-26T06:30:25.8849533Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module_grad_is_view 2024-06-26T06:30:25.8850911Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output 2024-06-26T06:30:25.8852289Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output_with_unused_parameters 2024-06-26T06:30:25.8853721Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_sharded_tensor 2024-06-26T06:30:25.8855070Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_invalid_powerSGD_state 2024-06-26T06:30:25.8856411Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_save_load_checkpoint 2024-06-26T06:30:25.8857709Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients 2024-06-26T06:30:25.8859263Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients_grad_is_view 2024-06-26T06:30:25.8860688Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_empty_input 2024-06-26T06:30:25.8862112Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_only_empty_input 2024-06-26T06:30:25.8863358Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward 2024-06-26T06:30:25.8864491Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_optimizer 2024-06-26T06:30:25.8865718Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_unused_parameters 2024-06-26T06:30:25.8866938Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_multi_bucket 2024-06-26T06:30:25.8868095Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_single_bucket 2024-06-26T06:30:25.8869267Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_single_dtype_single_bucket 2024-06-26T06:30:25.8870350Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_bool_tensors 2024-06-26T06:30:25.8871420Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cpu 2024-06-26T06:30:25.8872674Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cuda 2024-06-26T06:30:25.8873806Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_gloo_rank_membership 2024-06-26T06:30:25.8874999Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_gloo_warn_not_in_group 2024-06-26T06:30:25.8876171Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_default 2024-06-26T06:30:25.8877447Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_subgroup 2024-06-26T06:30:25.8878687Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_default_pg_gloo 2024-06-26T06:30:25.8879872Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_gloo_new_group 2024-06-26T06:30:25.8881013Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_complex 2024-06-26T06:30:25.8882111Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_mismatch 2024-06-26T06:30:25.8883445Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_all_to_all_single 2024-06-26T06:30:25.8885187Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allgather_coalesced 2024-06-26T06:30:25.8886785Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allreduce_coalesced 2024-06-26T06:30:25.8888338Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_collectives 2024-06-26T06:30:25.8889977Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_for_all_backends 2024-06-26T06:30:25.8891730Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_optional_backend 2024-06-26T06:30:25.8893401Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_monitored_barrier 2024-06-26T06:30:25.8894766Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CompilerTest::test_allgather_work_wait_cpu 2024-06-26T06:30:25.8896018Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CompilerTest::test_allgather_work_wait_gpu 2024-06-26T06:30:25.8897177Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CompilerTest::test_allreduce_work_wait_cpu 2024-06-26T06:30:25.8898339Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CompilerTest::test_allreduce_work_wait_gpu 2024-06-26T06:30:25.8899500Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CompilerTest::test_broadcast_work_wait_cpu 2024-06-26T06:30:25.8900648Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CompilerTest::test_broadcast_work_wait_gpu 2024-06-26T06:30:25.8901849Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CompilerTest::test_consecutive_comm_work_wait_cpu 2024-06-26T06:30:25.8903085Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CompilerTest::test_consecutive_comm_work_wait_gpu 2024-06-26T06:30:25.8904300Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CompilerTest::test_nested_comm_tensor_wrapping 2024-06-26T06:30:25.8905454Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CompilerTest::test_scatter_work_wait_cpu 2024-06-26T06:30:25.8906588Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CompilerTest::test_scatter_work_wait_gpu 2024-06-26T06:30:25.8907727Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync 2024-06-26T06:30:25.8908937Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_duplicate_pg 2024-06-26T06:30:25.8910197Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_sanity_check 2024-06-26T06:30:25.8910995Z 2024-06-26T06:30:25.8911472Z Running distributed/test_c10d_ops_nccl 1/1 ... [2024-06-26 06:30:25.861214] 2024-06-26T06:30:25.8913452Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_ops_nccl.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-06-26 06:30:25.861633] 2024-06-26T06:30:27.2064622Z 2024-06-26T06:30:27.2066459Z distributed/test_c10d_ops_nccl 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_ops_nccl_1.1_0d2e6b2adc6a5f3e_.log 2024-06-26T06:30:27.2067511Z 2024-06-26T06:30:28.0970351Z 2024-06-26T06:30:28.0970915Z real 87m54.813s 2024-06-26T06:30:28.0971303Z user 193m5.492s 2024-06-26T06:30:28.0973579Z sys 37m48.125s 2024-06-26T06:30:28.0974071Z + assert_git_not_dirty 2024-06-26T06:30:28.0974811Z + [[ linux-jammy-py3.8-gcc11 != *rocm* ]] 2024-06-26T06:30:28.0975556Z + [[ linux-jammy-py3.8-gcc11 != *xla* ]] 2024-06-26T06:30:28.0976224Z ++ git status --porcelain 2024-06-26T06:30:28.0977751Z ++ grep -v '?? third_party' 2024-06-26T06:30:51.1738217Z ++ true 2024-06-26T06:30:51.1740191Z + git_status= 2024-06-26T06:30:51.1741596Z + [[ -n '' ]] 2024-06-26T06:30:51.1742169Z + [[ linux-jammy-py3.8-gcc11 == *cuda* ]] 2024-06-26T06:30:51.1748346Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-06-26T06:30:51.1749136Z + [[ 1 == 1 ]] 2024-06-26T06:30:51.1749406Z + test_rpc 2024-06-26T06:30:51.1749709Z + echo 'Testing RPC C++ tests' 2024-06-26T06:30:51.1750071Z Testing RPC C++ tests 2024-06-26T06:30:51.1754726Z + ln -sf /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libtorch.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libtorch_global_deps.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libtorch_python.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libtorchbind_test.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-06-26T06:30:51.1787883Z + ln -sf /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libc10.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-06-26T06:30:51.1795659Z + CPP_TESTS_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-06-26T06:30:51.1796574Z + python test/run_test.py --cpp --verbose -i cpp/test_cpp_rpc 2024-06-26T06:30:51.2729639Z test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-06-26T06:30:51.2730647Z import pkg_resources 2024-06-26T06:30:54.2763609Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-06-26T06:30:54.2766928Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-06-26T06:30:54.2905371Z Found test times from artifacts 2024-06-26T06:30:54.3414576Z Found test times from artifacts 2024-06-26T06:30:54.3429801Z Running all tests 2024-06-26T06:30:54.3433328Z Running parallel tests on 3 processes 2024-06-26T06:30:54.3433976Z Name: tests to run (est. time: 0.0min) 2024-06-26T06:30:54.3434382Z Serial tests (1): 2024-06-26T06:30:54.3434750Z cpp/test_cpp_rpc 1/1 2024-06-26T06:30:54.3435081Z Parallel tests (0): 2024-06-26T06:30:54.3435471Z Name: excluded (est. time: 0.0min) 2024-06-26T06:30:54.3435828Z Serial tests (0): 2024-06-26T06:30:54.3436196Z Parallel tests (0): 2024-06-26T06:30:54.3436902Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-06-26T06:30:54.3618573Z Running cpp/test_cpp_rpc 1/1 ... [2024-06-26 06:30:54.361527] 2024-06-26T06:30:54.3628070Z Executing ['pytest', '/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin/test_cpp_rpc', '-v', '-vv', '-rfEX', '-n', '3', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-4a6781a992a36c2d.xml', '-x', '--reruns=2'] ... [2024-06-26 06:30:54.362426] 2024-06-26T06:30:58.9927155Z 2024-06-26T06:30:58.9928862Z cpp/test_cpp_rpc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.test_cpp_rpc_1.1_4c4e9924744f297f_.log 2024-06-26T06:30:58.9929773Z 2024-06-26T06:30:59.8272591Z + cleanup_workspace 2024-06-26T06:30:59.8273804Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-06-26T06:30:59.8274952Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-06-26T06:30:59.8275893Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-06-26T06:30:59.8276529Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-06-26T06:30:59.8277390Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-06-26T06:30:59.8278309Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-06-26T06:30:59.8278999Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-06-26T06:31:02.0847683Z ##[group]Run cat test/**/*_toprint.log || true 2024-06-26T06:31:02.0848275Z cat test/**/*_toprint.log || true 2024-06-26T06:31:02.1012928Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:31:02.1013431Z env: 2024-06-26T06:31:02.1013677Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:02.1014292Z DOCKER_CONTAINER_ID: a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:02.1014947Z ##[endgroup] 2024-06-26T06:31:02.1079418Z cat: test/**/*_toprint.log: No such file or directory 2024-06-26T06:31:02.1109951Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-06-26T06:31:02.1110381Z kill "$MONITOR_SCRIPT_PID" 2024-06-26T06:31:02.1117643Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:31:02.1118133Z env: 2024-06-26T06:31:02.1118375Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:02.1118991Z DOCKER_CONTAINER_ID: a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:02.1119661Z MONITOR_SCRIPT_PID: 17434 2024-06-26T06:31:02.1119981Z ##[endgroup] 2024-06-26T06:31:02.1322034Z Prepare all required actions 2024-06-26T06:31:02.1322503Z Getting action download info 2024-06-26T06:31:02.3384426Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-06-26T06:31:02.5067819Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-06-26T06:31:02.5068269Z with: 2024-06-26T06:31:02.5068658Z file-suffix: test-distributed-1-2-linux.2xlarge_26688305322 2024-06-26T06:31:02.5069187Z s3-bucket: gha-artifacts 2024-06-26T06:31:02.5069487Z env: 2024-06-26T06:31:02.5069737Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:02.5070335Z DOCKER_CONTAINER_ID: a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:02.5070970Z ##[endgroup] 2024-06-26T06:31:02.5097337Z ##[group]Run # Remove any previous test jsons if they exist 2024-06-26T06:31:02.5097945Z # Remove any previous test jsons if they exist 2024-06-26T06:31:02.5098441Z rm -f test-jsons-*.zip 2024-06-26T06:31:02.5098976Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-06-26T06:31:02.5106684Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:31:02.5107174Z env: 2024-06-26T06:31:02.5107414Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:02.5108009Z DOCKER_CONTAINER_ID: a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:02.5108801Z FILE_SUFFIX: test-distributed-1-2-linux.2xlarge_26688305322 2024-06-26T06:31:02.5109295Z ##[endgroup] 2024-06-26T06:31:02.5379949Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-06-26T06:31:02.5406947Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-06-26T06:31:02.5407712Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-06-26T06:31:02.5413406Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-06-26T06:31:02.5420522Z adding: test/test-reports/td_exclusions-f74d67a2187f182ef68c.json (deflated 86%) 2024-06-26T06:31:02.5421568Z adding: test/test-reports/td_exclusions-35a496d6f7d1ad9f349c.json (deflated 18%) 2024-06-26T06:31:02.5422360Z adding: test/.pytorch-slow-tests.json (deflated 77%) 2024-06-26T06:31:02.5428645Z adding: test/.pytorch-disabled-tests.json (deflated 84%) 2024-06-26T06:31:02.5461265Z ##[group]Run # Remove any previous test reports if they exist 2024-06-26T06:31:02.5461912Z # Remove any previous test reports if they exist 2024-06-26T06:31:02.5462431Z rm -f test-reports-*.zip 2024-06-26T06:31:02.5463012Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-06-26T06:31:02.5470307Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:31:02.5470801Z env: 2024-06-26T06:31:02.5471056Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:02.5471771Z DOCKER_CONTAINER_ID: a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:02.5472555Z FILE_SUFFIX: test-distributed-1-2-linux.2xlarge_26688305322 2024-06-26T06:31:02.5473078Z ##[endgroup] 2024-06-26T06:31:02.5726995Z adding: test/test-reports/python-pytest/distributed.test_inductor_collectives/distributed.test_inductor_collectives-fb2a2b63ecd44a74.xml (deflated 94%) 2024-06-26T06:31:02.5730498Z adding: test/test-reports/python-pytest/distributed.test_dynamo_distributed/distributed.test_dynamo_distributed-e5b18ca04f1004c6.xml (deflated 91%) 2024-06-26T06:31:02.5733779Z adding: test/test-reports/python-pytest/distributed.pipelining.test_backward/distributed.pipelining.test_backward-79e6b6e4aface813.xml (deflated 37%) 2024-06-26T06:31:02.5737198Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_nested_dict/distributed.checkpoint.test_nested_dict-ee336de4edb1a06c.xml (deflated 51%) 2024-06-26T06:31:02.5740951Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_training/distributed._composable.fsdp.test_fully_shard_training-b890981c1ead46bf.xml (deflated 84%) 2024-06-26T06:31:02.5745050Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.test_logger/distributed._shard.sharded_tensor.test_logger-088ca61829cc1da8.xml (deflated 37%) 2024-06-26T06:31:02.5748471Z adding: test/test-reports/python-pytest/distributed.test_symmetric_memory/distributed.test_symmetric_memory-8129eec1c8fce30f.xml (deflated 65%) 2024-06-26T06:31:02.5751917Z adding: test/test-reports/python-pytest/distributed._tensor.test_dtensor_compile/distributed._tensor.test_dtensor_compile-a27f985ab6703a20.xml (deflated 83%) 2024-06-26T06:31:02.5755424Z adding: test/test-reports/python-pytest/distributed._tensor.test_dtensor_ops/distributed._tensor.test_dtensor_ops-a528d912784a005d.xml (deflated 96%) 2024-06-26T06:31:02.5759244Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_micro_pipeline_tp/distributed.tensor.parallel.test_micro_pipeline_tp-5c69520d7f1ce9de.xml (deflated 92%) 2024-06-26T06:31:02.5763021Z adding: test/test-reports/python-pytest/distributed.pipelining.test_pipe/distributed.pipelining.test_pipe-6a4b04ed8089d672.xml (deflated 72%) 2024-06-26T06:31:02.5766471Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_fx/distributed.fsdp.test_fsdp_fx-cd7809333a12b900.xml (deflated 37%) 2024-06-26T06:31:02.5769833Z adding: test/test-reports/python-pytest/distributed.test_c10d_functional_native/distributed.test_c10d_functional_native-3f7accca7b24acb0.xml (deflated 93%) 2024-06-26T06:31:02.5773581Z adding: test/test-reports/python-pytest/distributed.checkpoint.fsdp.test_fsdp_dsd/distributed.checkpoint.fsdp.test_fsdp_dsd-1e3b0704c2ade112.xml (deflated 62%) 2024-06-26T06:31:02.5777582Z adding: test/test-reports/python-pytest/distributed._tensor.experimental.test_tp_transform/distributed._tensor.experimental.test_tp_transform-45f308a9f7d677aa.xml (deflated 62%) 2024-06-26T06:31:02.5781554Z adding: test/test-reports/python-pytest/distributed._tensor.test_common_rules/distributed._tensor.test_common_rules-8258d4a70dbc21fc.xml (deflated 79%) 2024-06-26T06:31:02.5784920Z adding: test/test-reports/python-pytest/distributed._tensor.test_random_ops/distributed._tensor.test_random_ops-c9a6046d6ac70dbe.xml (deflated 82%) 2024-06-26T06:31:02.5788459Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_save_load_api/distributed.checkpoint.test_save_load_api-2fbaf90daf7e489a.xml (deflated 44%) 2024-06-26T06:31:02.5792294Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_fsdp_optim_state/distributed.checkpoint.test_fsdp_optim_state-08ce5b6bb70438da.xml (deflated 65%) 2024-06-26T06:31:02.5795957Z adding: test/test-reports/python-pytest/distributed.pipelining.test_unflatten/distributed.pipelining.test_unflatten-e273a71b3210e524.xml (deflated 40%) 2024-06-26T06:31:02.5799483Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_format_utils/distributed.checkpoint.test_format_utils-7dc75d2605a266d6.xml (deflated 59%) 2024-06-26T06:31:02.5802885Z adding: test/test-reports/python-pytest/distributed._tensor.test_init/distributed._tensor.test_init-ffa0d7fef413301e.xml (deflated 74%) 2024-06-26T06:31:02.5806289Z adding: test/test-reports/python-pytest/distributed.pipelining.test_microbatch/distributed.pipelining.test_microbatch-d57d22256ffa0a90.xml (deflated 48%) 2024-06-26T06:31:02.5809826Z adding: test/test-reports/python-pytest/distributed._tensor.test_convolution_ops/distributed._tensor.test_convolution_ops-2552ff1d781da1cc.xml (deflated 61%) 2024-06-26T06:31:02.5813518Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_tp_checkpoint/distributed.checkpoint.test_tp_checkpoint-e91d5c06be8aa551.xml (deflated 65%) 2024-06-26T06:31:02.5816929Z adding: test/test-reports/python-pytest/distributed.test_device_mesh/distributed.test_device_mesh-fb90d7c987eccf22.xml (deflated 85%) 2024-06-26T06:31:02.5820644Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_logging/distributed._composable.fsdp.test_fully_shard_logging-e7a6760003f89fb5.xml (deflated 28%) 2024-06-26T06:31:02.5824715Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_traverse/distributed.checkpoint.test_traverse-ef8042881b95444f.xml (deflated 75%) 2024-06-26T06:31:02.5828674Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_compile/distributed._composable.fsdp.test_fully_shard_compile-c2248bf5a6b8b12d.xml (deflated 79%) 2024-06-26T06:31:02.5832888Z adding: test/test-reports/python-pytest/distributed._tensor.experimental.test_local_map/distributed._tensor.experimental.test_local_map-a70e25246a0f03dc.xml (deflated 68%) 2024-06-26T06:31:02.5836668Z adding: test/test-reports/python-pytest/distributed._tools.test_memory_tracker/distributed._tools.test_memory_tracker-4564849866cfa8bf.xml (deflated 44%) 2024-06-26T06:31:02.5840109Z adding: test/test-reports/python-pytest/distributed._tensor.test_op_strategy/distributed._tensor.test_op_strategy-435982d53f915ca4.xml (deflated 84%) 2024-06-26T06:31:02.5844058Z adding: test/test-reports/python-pytest/distributed.checkpoint.e2e.test_e2e_save_and_load/distributed.checkpoint.e2e.test_e2e_save_and_load-c21c02bb4bb12299.xml (deflated 89%) 2024-06-26T06:31:02.5846613Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_hsdp_checkpoint/distributed.checkpoint.test_hsdp_checkpoint-4ca59991522f972e.xml (deflated 79%) 2024-06-26T06:31:02.5848538Z adding: test/test-reports/python-pytest/distributed._tensor.test_embedding_ops/distributed._tensor.test_embedding_ops-1322192d731f7f25.xml (deflated 62%) 2024-06-26T06:31:02.5850445Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_state_dict_utils/distributed.checkpoint.test_state_dict_utils-f064ff5bf0014f38.xml (deflated 71%) 2024-06-26T06:31:02.5852513Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_fsdp_2d_parallel/distributed.tensor.parallel.test_fsdp_2d_parallel-880f88181b8da4e5.xml (deflated 89%) 2024-06-26T06:31:02.5854715Z adding: test/test-reports/python-pytest/distributed.elastic.test_control_plane/distributed.elastic.test_control_plane-8e321b35e4ade585.xml (deflated 72%) 2024-06-26T06:31:02.5856640Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_compatibility/distributed.checkpoint.test_compatibility-deb4a45c9ce3a72f.xml (deflated 62%) 2024-06-26T06:31:02.5858759Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_util/distributed._composable.fully_shard.test_fully_shard_util-63fdc77d9b443102.xml (deflated 47%) 2024-06-26T06:31:02.5861017Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_frozen/distributed._composable.fsdp.test_fully_shard_frozen-359367019f71f97a.xml (deflated 62%) 2024-06-26T06:31:02.5862988Z adding: test/test-reports/python-pytest/distributed._composable.test_replicate/distributed._composable.test_replicate-15d11654fc884914.xml (deflated 79%) 2024-06-26T06:31:02.5865024Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_state_dict/distributed._composable.fsdp.test_fully_shard_state_dict-f3a6309d0fbb247c.xml (deflated 68%) 2024-06-26T06:31:02.5867008Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_pure_fp16/distributed.fsdp.test_fsdp_pure_fp16-e8ef4b5cf1273181.xml (deflated 63%) 2024-06-26T06:31:02.5869076Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_clip_grad_norm_/distributed._composable.fsdp.test_fully_shard_clip_grad_norm_-937fdc27d3a52daa.xml (deflated 53%) 2024-06-26T06:31:02.5871092Z adding: test/test-reports/python-pytest/distributed._composable.test_compose/distributed._composable.test_compose-e6362a47897a6a48.xml (deflated 86%) 2024-06-26T06:31:02.5873320Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_mixed_precision/distributed._composable.fully_shard.test_fully_shard_mixed_precision-3af473048523c464.xml (deflated 48%) 2024-06-26T06:31:02.5875464Z adding: test/test-reports/python-pytest/distributed._tensor.test_view_ops/distributed._tensor.test_view_ops-fe0e7f30b673c1d1.xml (deflated 58%) 2024-06-26T06:31:02.5877119Z adding: test/test-reports/python-pytest/distributed._tensor.test_utils/distributed._tensor.test_utils-8a213d8517a331e2.xml (deflated 69%) 2024-06-26T06:31:02.5878833Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_utils/distributed.checkpoint.test_utils-e2305bd8c0cabc15.xml (deflated 69%) 2024-06-26T06:31:02.5880426Z adding: test/test-reports/python-pytest/distributed.test_nccl/distributed.test_nccl-e759f95a7bc96014.xml (deflated 28%) 2024-06-26T06:31:02.5882325Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_mixed_precision/distributed._composable.fsdp.test_fully_shard_mixed_precision-cc20d977b16eea32.xml (deflated 83%) 2024-06-26T06:31:02.5884573Z adding: test/test-reports/python-pytest/distributed._tensor.test_optimizers/distributed._tensor.test_optimizers-0c9169ca2934f834.xml (deflated 81%) 2024-06-26T06:31:02.5886430Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_ignored_modules/distributed.fsdp.test_fsdp_ignored_modules-412b0cbe5484a886.xml (deflated 86%) 2024-06-26T06:31:02.5888142Z adding: test/test-reports/python-pytest/distributed.test_fake_pg/distributed.test_fake_pg-a5adfde8587ca832.xml (deflated 82%) 2024-06-26T06:31:02.5890015Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_init/distributed._composable.fully_shard.test_fully_shard_init-47a6ba6faf849f15.xml (deflated 83%) 2024-06-26T06:31:02.5891937Z adding: test/test-reports/python-pytest/distributed.launcher.test_run/distributed.launcher.test_run-c5269684bcf9a2f5.xml (deflated 84%) 2024-06-26T06:31:02.5893769Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_dtensor_state_dict/distributed.fsdp.test_fsdp_dtensor_state_dict-d441228d80c8021b.xml (deflated 92%) 2024-06-26T06:31:02.5895854Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_backward_prefetch/distributed.fsdp.test_fsdp_backward_prefetch-e7504c141ce3b3d3.xml (deflated 47%) 2024-06-26T06:31:02.5897739Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_optim_state/distributed.fsdp.test_fsdp_optim_state-75d4df2e358ee03f.xml (deflated 96%) 2024-06-26T06:31:02.5899524Z adding: test/test-reports/python-pytest/distributed.rpc.test_share_memory/distributed.rpc.test_share_memory-47f0c122007ab6f1.xml (deflated 47%) 2024-06-26T06:31:02.5901651Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_model_checkpoint/distributed._composable.fully_shard.test_fully_shard_model_checkpoint-d896b5f1a9dc917e.xml (deflated 80%) 2024-06-26T06:31:02.5903840Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_flatten_params/distributed.fsdp.test_fsdp_flatten_params-ca21900aae4af5c3.xml (deflated 83%) 2024-06-26T06:31:02.5905706Z adding: test/test-reports/python-pytest/distributed.nn.jit.test_instantiator/distributed.nn.jit.test_instantiator-d5095abf19f00cd7.xml (deflated 61%) 2024-06-26T06:31:02.5907622Z adding: test/test-reports/python-pytest/distributed.fsdp.test_distributed_checkpoint/distributed.fsdp.test_distributed_checkpoint-e0ff2d5b0fe373c8.xml (deflated 77%) 2024-06-26T06:31:02.5909493Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_overlap/distributed.fsdp.test_fsdp_overlap-60ae43b33d9c6ec9.xml (deflated 64%) 2024-06-26T06:31:02.5911242Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_memory/distributed.fsdp.test_fsdp_memory-1e02a283547c8b4c.xml (deflated 64%) 2024-06-26T06:31:02.5913167Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_freezing_weights/distributed.fsdp.test_fsdp_freezing_weights-13d04170c25f0ed3.xml (deflated 96%) 2024-06-26T06:31:02.5914978Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_input/distributed.fsdp.test_fsdp_input-73101d5b0cefdbd1.xml (deflated 64%) 2024-06-26T06:31:02.5916788Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_apply/distributed.fsdp.test_fsdp_apply-1bf5ca64f4eeec05.xml (deflated 71%) 2024-06-26T06:31:02.5918516Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_uneven/distributed.fsdp.test_fsdp_uneven-3484de2370fdcf72.xml (deflated 44%) 2024-06-26T06:31:02.5920382Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_multiple_wrapping/distributed.fsdp.test_fsdp_multiple_wrapping-d5f5c4d4fbf46d0c.xml (deflated 47%) 2024-06-26T06:31:02.5922263Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_state_dict/distributed.fsdp.test_fsdp_state_dict-f8f9a0fe38b07249.xml (deflated 97%) 2024-06-26T06:31:02.5924432Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_runtime/distributed._composable.fully_shard.test_fully_shard_runtime-544183b3c0c786b9.xml (deflated 54%) 2024-06-26T06:31:02.5926472Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_comm_hooks/distributed.fsdp.test_fsdp_comm_hooks-909b9cc81f9edd47.xml (deflated 94%) 2024-06-26T06:31:02.5928446Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_comm/distributed._composable.fsdp.test_fully_shard_comm-aecbdc96dfc97de0.xml (deflated 82%) 2024-06-26T06:31:02.5930432Z adding: test/test-reports/python-pytest/distributed.fsdp.test_checkpoint_wrapper/distributed.fsdp.test_checkpoint_wrapper-f0e9f37f63635a06.xml (deflated 74%) 2024-06-26T06:31:02.5932499Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_file_system_checkpoint_cpu/distributed.checkpoint.test_file_system_checkpoint_cpu-7d6bdb79461cf827.xml (deflated 73%) 2024-06-26T06:31:02.5934670Z adding: test/test-reports/python-pytest/distributed._composable.test_replicate_with_compiler/distributed._composable.test_replicate_with_compiler-7106356e385a4c75.xml (deflated 84%) 2024-06-26T06:31:02.5936814Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_sharded_grad_scaler/distributed.fsdp.test_fsdp_sharded_grad_scaler-725a3b13c382fd09.xml (deflated 93%) 2024-06-26T06:31:02.5938735Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_checkpoint/distributed.fsdp.test_fsdp_checkpoint-5532b52d7f3e1ffe.xml (deflated 93%) 2024-06-26T06:31:02.5940426Z adding: test/test-reports/python-pytest/distributed.fsdp.test_wrap/distributed.fsdp.test_wrap-ebf1185adf3ba9ad.xml (deflated 94%) 2024-06-26T06:31:02.5942366Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_extensions/distributed._composable.fsdp.test_fully_shard_extensions-982e934d805f0ed1.xml (deflated 71%) 2024-06-26T06:31:02.5944425Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_tp_integration/distributed.fsdp.test_fsdp_tp_integration-a8172194cdbf2154.xml (deflated 72%) 2024-06-26T06:31:02.5946294Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_traversal/distributed.fsdp.test_fsdp_traversal-70bfcec152ebd49e.xml (deflated 46%) 2024-06-26T06:31:02.5948091Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_fine_tune/distributed.fsdp.test_fsdp_fine_tune-0c6594a99b6f1f9d.xml (deflated 64%) 2024-06-26T06:31:02.5949912Z adding: test/test-reports/python-pytest/distributed.test_c10d_object_collectives/distributed.test_c10d_object_collectives-965170d04c0242ec.xml (deflated 81%) 2024-06-26T06:31:02.5951744Z adding: test/test-reports/python-pytest/distributed._shard.test_sharder/distributed._shard.test_sharder-892a4977fd914484.xml (deflated 66%) 2024-06-26T06:31:02.5953635Z adding: test/test-reports/python-pytest/distributed._shard.sharded_optim.test_sharded_optim/distributed._shard.sharded_optim.test_sharded_optim-98951e6e4f412442.xml (deflated 67%) 2024-06-26T06:31:02.5955922Z adding: test/test-reports/python-pytest/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint-ebd9f1443a987220.xml (deflated 67%) 2024-06-26T06:31:02.5958231Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_multiple_forward/distributed.fsdp.test_fsdp_multiple_forward-691886f2d774b341.xml (deflated 46%) 2024-06-26T06:31:02.5960041Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_meta/distributed.fsdp.test_fsdp_meta-50e1991d994af640.xml (deflated 90%) 2024-06-26T06:31:02.5961949Z adding: test/test-reports/python-pytest/distributed._shard.sharding_spec.test_sharding_spec/distributed._shard.sharding_spec.test_sharding_spec-7f7e9fae5ec0cad9.xml (deflated 83%) 2024-06-26T06:31:02.5964043Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_unshard_params/distributed.fsdp.test_fsdp_unshard_params-56468c76d6eaba74.xml (deflated 89%) 2024-06-26T06:31:02.5965978Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-52a9b591bb00c223.xml (deflated 36%) 2024-06-26T06:31:02.5967803Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-da6d871a7550ad95.xml (deflated 36%) 2024-06-26T06:31:02.5969629Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a8c51772e4cea062.xml (deflated 35%) 2024-06-26T06:31:02.5971445Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-02ee10848297b786.xml (deflated 35%) 2024-06-26T06:31:02.5973247Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-547009b29b6d96ef.xml (deflated 35%) 2024-06-26T06:31:02.5975062Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9972daa535536b68.xml (deflated 36%) 2024-06-26T06:31:02.5976874Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-40213c7636f18d66.xml (deflated 36%) 2024-06-26T06:31:02.5978805Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9a8a8c8300802ac0.xml (deflated 36%) 2024-06-26T06:31:02.5980618Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2d10cb0ccd45fe30.xml (deflated 36%) 2024-06-26T06:31:02.5982437Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-13b183d60b28649d.xml (deflated 36%) 2024-06-26T06:31:02.5984257Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-bebcaebc61d7d486.xml (deflated 35%) 2024-06-26T06:31:02.5986077Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f8601421fb35af32.xml (deflated 35%) 2024-06-26T06:31:02.5987870Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-439fa074809659a3.xml (deflated 35%) 2024-06-26T06:31:02.5989677Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1f35e0480ae8f3c5.xml (deflated 35%) 2024-06-26T06:31:02.5991491Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-217fa5564cab4d21.xml (deflated 36%) 2024-06-26T06:31:02.5993911Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7b85a3c6e7db0d86.xml (deflated 35%) 2024-06-26T06:31:02.5996064Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-749e3a1346970d38.xml (deflated 35%) 2024-06-26T06:31:02.5998336Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0df330f5551ec172.xml (deflated 36%) 2024-06-26T06:31:02.6000683Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-289f4f3a9f847d0a.xml (deflated 35%) 2024-06-26T06:31:02.6002909Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ad772441e1ceddbe.xml (deflated 36%) 2024-06-26T06:31:02.6005155Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ab4f67f19c61503b.xml (deflated 35%) 2024-06-26T06:31:02.6007169Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f1b776b021437cdc.xml (deflated 35%) 2024-06-26T06:31:02.6009206Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c3fdb39b2d386ca9.xml (deflated 35%) 2024-06-26T06:31:02.6011219Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-57a584c69eadcdb4.xml (deflated 35%) 2024-06-26T06:31:02.6013236Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-719796e5c408881a.xml (deflated 35%) 2024-06-26T06:31:02.6015241Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0c09022280e58976.xml (deflated 35%) 2024-06-26T06:31:02.6017249Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e3fe80e3a07f2f1f.xml (deflated 35%) 2024-06-26T06:31:02.6019282Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4b028d5ea68d6563.xml (deflated 36%) 2024-06-26T06:31:02.6021288Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-214b2089029c6bc8.xml (deflated 35%) 2024-06-26T06:31:02.6023440Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-db3a646ce540c14b.xml (deflated 36%) 2024-06-26T06:31:02.6025457Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-023d2381bf86edac.xml (deflated 36%) 2024-06-26T06:31:02.6027487Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-52ac9a4683b85bbf.xml (deflated 36%) 2024-06-26T06:31:02.6029344Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b8c038491a0971fd.xml (deflated 35%) 2024-06-26T06:31:02.6031170Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ed38cd64fa0d72ab.xml (deflated 36%) 2024-06-26T06:31:02.6033060Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-063525ff0ea0bf2a.xml (deflated 35%) 2024-06-26T06:31:02.6035697Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-36914625777c09a9.xml (deflated 35%) 2024-06-26T06:31:02.6037507Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b4ffa30bf1a80bed.xml (deflated 36%) 2024-06-26T06:31:02.6039330Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fab9f4552af0a8d7.xml (deflated 36%) 2024-06-26T06:31:02.6041144Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2f50f3645b2adba8.xml (deflated 35%) 2024-06-26T06:31:02.6042959Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8415ed8947da5b91.xml (deflated 36%) 2024-06-26T06:31:02.6044932Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-425e29bef48e429e.xml (deflated 35%) 2024-06-26T06:31:02.6046852Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-071032f928032244.xml (deflated 35%) 2024-06-26T06:31:02.6048666Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-279236ac6e7fc02f.xml (deflated 35%) 2024-06-26T06:31:02.6050483Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4b8ff5eee4e3ae8e.xml (deflated 35%) 2024-06-26T06:31:02.6052298Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-22422b5091beaff3.xml (deflated 35%) 2024-06-26T06:31:02.6054091Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7ea6a4d179eecfbf.xml (deflated 35%) 2024-06-26T06:31:02.6055912Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-53afd9939cbc2200.xml (deflated 35%) 2024-06-26T06:31:02.6057718Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-699b2eb886e3cbe3.xml (deflated 35%) 2024-06-26T06:31:02.6059520Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-419f07c5fd185926.xml (deflated 35%) 2024-06-26T06:31:02.6061317Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-bc2a0d09101f518a.xml (deflated 35%) 2024-06-26T06:31:02.6063126Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9283e5dfac64149f.xml (deflated 34%) 2024-06-26T06:31:02.6065033Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c3b2baefbd529cb6.xml (deflated 35%) 2024-06-26T06:31:02.6066849Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-11c6a3d64dfec548.xml (deflated 35%) 2024-06-26T06:31:02.6068654Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b7dee9a7a7afe77a.xml (deflated 35%) 2024-06-26T06:31:02.6070461Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-30ab8d3572e51020.xml (deflated 34%) 2024-06-26T06:31:02.6072332Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6cf95808469000b0.xml (deflated 35%) 2024-06-26T06:31:02.6074147Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5c5e00e026aa0ec7.xml (deflated 35%) 2024-06-26T06:31:02.6075951Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-64e15a4827a9aa15.xml (deflated 36%) 2024-06-26T06:31:02.6077757Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c255162ad25b40f1.xml (deflated 35%) 2024-06-26T06:31:02.6079568Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fd407b660b7a2a55.xml (deflated 35%) 2024-06-26T06:31:02.6081376Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7cde8f2ef50e4351.xml (deflated 35%) 2024-06-26T06:31:02.6083171Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-81ca12bee7337090.xml (deflated 34%) 2024-06-26T06:31:02.6085085Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cbbd0def99bbaa28.xml (deflated 35%) 2024-06-26T06:31:02.6086994Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-781a98dc2f14478d.xml (deflated 36%) 2024-06-26T06:31:02.6088806Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7313d2141b97e523.xml (deflated 35%) 2024-06-26T06:31:02.6090598Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4a36d92963b57a53.xml (deflated 35%) 2024-06-26T06:31:02.6092408Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9519a07db536d788.xml (deflated 35%) 2024-06-26T06:31:02.6094210Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7f49dadbd4507017.xml (deflated 35%) 2024-06-26T06:31:02.6096025Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ea35b8ee4704089a.xml (deflated 34%) 2024-06-26T06:31:02.6097820Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6ba39c3764697cea.xml (deflated 35%) 2024-06-26T06:31:02.6099629Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a4ab4604d4da44db.xml (deflated 34%) 2024-06-26T06:31:02.6101430Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5c4e422bd220cf71.xml (deflated 35%) 2024-06-26T06:31:02.6103243Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ad9fc6a04df61656.xml (deflated 35%) 2024-06-26T06:31:02.6105033Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-135d689531c2fc18.xml (deflated 35%) 2024-06-26T06:31:02.6106972Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3eb0571ff3e283f4.xml (deflated 35%) 2024-06-26T06:31:02.6108781Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-73ab42047c7d097b.xml (deflated 35%) 2024-06-26T06:31:02.6110590Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cf54825ce79470f6.xml (deflated 35%) 2024-06-26T06:31:02.6112454Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-08c00417f5f874ea.xml (deflated 35%) 2024-06-26T06:31:02.6114253Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-425940fb6c67baa2.xml (deflated 34%) 2024-06-26T06:31:02.6116069Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7d7ec08f4123a5e3.xml (deflated 35%) 2024-06-26T06:31:02.6117889Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-91dc1417d48d2dbd.xml (deflated 34%) 2024-06-26T06:31:02.6119679Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-61302b84196befc8.xml (deflated 34%) 2024-06-26T06:31:02.6121477Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-10b5ff99bb7181fe.xml (deflated 34%) 2024-06-26T06:31:02.6123278Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f9e9277fe5fbb13c.xml (deflated 36%) 2024-06-26T06:31:02.6125271Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d0d903beb8ea9acf.xml (deflated 35%) 2024-06-26T06:31:02.6127158Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7bafc887596d180a.xml (deflated 36%) 2024-06-26T06:31:02.6128982Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a4dbe01fb51b5aff.xml (deflated 35%) 2024-06-26T06:31:02.6130797Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1f463c69b8930bb7.xml (deflated 36%) 2024-06-26T06:31:02.6132611Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-bba6f529ec1a685f.xml (deflated 55%) 2024-06-26T06:31:02.6134418Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-192689578ef2d442.xml (deflated 34%) 2024-06-26T06:31:02.6136204Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c8646b4869312c3b.xml (deflated 35%) 2024-06-26T06:31:02.6138018Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e0f45e153845c1f6.xml (deflated 36%) 2024-06-26T06:31:02.6139830Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ed14174d166ab950.xml (deflated 37%) 2024-06-26T06:31:02.6141636Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ba2884e3a68f8e87.xml (deflated 37%) 2024-06-26T06:31:02.6143435Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0303bf2a7d5c3a45.xml (deflated 36%) 2024-06-26T06:31:02.6145240Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6cc2073c5b05c805.xml (deflated 37%) 2024-06-26T06:31:02.6147124Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1b96ee87964a202d.xml (deflated 37%) 2024-06-26T06:31:02.6148929Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1796c5938c26b140.xml (deflated 37%) 2024-06-26T06:31:02.6150716Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-38260b943c87a677.xml (deflated 37%) 2024-06-26T06:31:02.6152587Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d3ce9795b13381bd.xml (deflated 37%) 2024-06-26T06:31:02.6154406Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d5dd2799c8f9310f.xml (deflated 37%) 2024-06-26T06:31:02.6156212Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1ef237a9c325dcc9.xml (deflated 37%) 2024-06-26T06:31:02.6158015Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8090cf9b01d5b702.xml (deflated 36%) 2024-06-26T06:31:02.6159830Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c692a2ab1efd5b5f.xml (deflated 38%) 2024-06-26T06:31:02.6161637Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f759929a83efabaa.xml (deflated 36%) 2024-06-26T06:31:02.6163451Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0461e21f92d532a4.xml (deflated 37%) 2024-06-26T06:31:02.6165331Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-56602a6974ad2688.xml (deflated 36%) 2024-06-26T06:31:02.6167151Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-859b1b613a470431.xml (deflated 36%) 2024-06-26T06:31:02.6169032Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-35594fe961fc1258.xml (deflated 36%) 2024-06-26T06:31:02.6170845Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-db09fcf4dc36b43c.xml (deflated 37%) 2024-06-26T06:31:02.6172869Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-864a6a4987c05753.xml (deflated 36%) 2024-06-26T06:31:02.6174672Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-253c0e7d93788869.xml (deflated 37%) 2024-06-26T06:31:02.6176476Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e4f187a76f96ed89.xml (deflated 36%) 2024-06-26T06:31:02.6178289Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-080f486bcee01146.xml (deflated 37%) 2024-06-26T06:31:02.6180096Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9ea209e33ce1965a.xml (deflated 37%) 2024-06-26T06:31:02.6181906Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8eaa897fa9ae3d84.xml (deflated 36%) 2024-06-26T06:31:02.6183790Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-13587c164522f478.xml (deflated 41%) 2024-06-26T06:31:02.6185744Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-9879296dd9a0b862.xml (deflated 42%) 2024-06-26T06:31:02.6187777Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-92f0fde5f5864ba3.xml (deflated 41%) 2024-06-26T06:31:02.6189734Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-008d3910e8cd5330.xml (deflated 41%) 2024-06-26T06:31:02.6191743Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-41a943c4cae6381d.xml (deflated 41%) 2024-06-26T06:31:02.6193712Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-edc457a965ecc797.xml (deflated 41%) 2024-06-26T06:31:02.6195673Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d9d976e897adae0c.xml (deflated 42%) 2024-06-26T06:31:02.6197614Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-512f1c5face05768.xml (deflated 41%) 2024-06-26T06:31:02.6199586Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-1ce04c6e267a8eaa.xml (deflated 43%) 2024-06-26T06:31:02.6201546Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-ef2697c31baf1dd3.xml (deflated 42%) 2024-06-26T06:31:02.6203492Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-04127d2ab596f8e7.xml (deflated 41%) 2024-06-26T06:31:02.6205669Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b1c8f11821043a6d.xml (deflated 42%) 2024-06-26T06:31:02.6207615Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-6608076032218b94.xml (deflated 42%) 2024-06-26T06:31:02.6209643Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-48b754334f030a96.xml (deflated 42%) 2024-06-26T06:31:02.6211592Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-f0224ba275f8a68a.xml (deflated 42%) 2024-06-26T06:31:02.6213529Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-980d6b25880b1490.xml (deflated 42%) 2024-06-26T06:31:02.6215472Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-8e53d0e4fb8eab30.xml (deflated 42%) 2024-06-26T06:31:02.6217416Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b207202058067efb.xml (deflated 42%) 2024-06-26T06:31:02.6219567Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0ac51edfea9d2d68.xml (deflated 43%) 2024-06-26T06:31:02.6221524Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-110f4550ab7fc6b2.xml (deflated 42%) 2024-06-26T06:31:02.6223542Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-a8ab93d537ff6e2c.xml (deflated 42%) 2024-06-26T06:31:02.6225536Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-8d97e17c3cbaf3f4.xml (deflated 42%) 2024-06-26T06:31:02.6227517Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-57cf691375adec06.xml (deflated 42%) 2024-06-26T06:31:02.6229504Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-ca4a79af4bed111e.xml (deflated 42%) 2024-06-26T06:31:02.6231551Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-24ee842799420a54.xml (deflated 42%) 2024-06-26T06:31:02.6233549Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b7d49287a479b318.xml (deflated 42%) 2024-06-26T06:31:02.6235476Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-d3f6d67db4014971.xml (deflated 42%) 2024-06-26T06:31:02.6237421Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-f18a914e07646dcc.xml (deflated 42%) 2024-06-26T06:31:02.6239360Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-43b455690e570fd2.xml (deflated 42%) 2024-06-26T06:31:02.6241313Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-460a3b7c1d73ff6c.xml (deflated 36%) 2024-06-26T06:31:02.6243247Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-e47b6a971a6c61c4.xml (deflated 42%) 2024-06-26T06:31:02.6245265Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-01aeef24ae3feb4f.xml (deflated 42%) 2024-06-26T06:31:02.6247209Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-99a7fa732d21fd43.xml (deflated 42%) 2024-06-26T06:31:02.6249158Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0ee29ba96c2a63a4.xml (deflated 42%) 2024-06-26T06:31:02.6251093Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-e95ddb0db4959708.xml (deflated 42%) 2024-06-26T06:31:02.6253137Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-95389f1c088bd445.xml (deflated 42%) 2024-06-26T06:31:02.6255093Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-7a8dbff4378c094c.xml (deflated 42%) 2024-06-26T06:31:02.6257036Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-1978c9e438bbf4e8.xml (deflated 42%) 2024-06-26T06:31:02.6258969Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-27bb3000e92e42c4.xml (deflated 42%) 2024-06-26T06:31:02.6260911Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-9b50d27aea87e1b5.xml (deflated 42%) 2024-06-26T06:31:02.6262861Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-867d4f6958d8e88b.xml (deflated 42%) 2024-06-26T06:31:02.6264805Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-330b4077918c19c3.xml (deflated 42%) 2024-06-26T06:31:02.6266728Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0e955c94d039f245.xml (deflated 42%) 2024-06-26T06:31:02.6268673Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-3ca2969d2fe75dbb.xml (deflated 42%) 2024-06-26T06:31:02.6270612Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-2cb0cf2d3e795d48.xml (deflated 42%) 2024-06-26T06:31:02.6272704Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-13c785a0cc49a6a2.xml (deflated 42%) 2024-06-26T06:31:02.6274648Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-c0840e5e333e4046.xml (deflated 42%) 2024-06-26T06:31:02.6276575Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0d107f7e4a43c74e.xml (deflated 42%) 2024-06-26T06:31:02.6278503Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-77307a5686744465.xml (deflated 42%) 2024-06-26T06:31:02.6280437Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-553cd2a2208883a3.xml (deflated 42%) 2024-06-26T06:31:02.6282385Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-3f37bf8aad572185.xml (deflated 42%) 2024-06-26T06:31:02.6284445Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-282be848b20df33e.xml (deflated 42%) 2024-06-26T06:31:02.6286477Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-c29ca87fbf897669.xml (deflated 42%) 2024-06-26T06:31:02.6288423Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-55324137ddb85594.xml (deflated 42%) 2024-06-26T06:31:02.6290358Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-35be14e32aa272e9.xml (deflated 42%) 2024-06-26T06:31:02.6292284Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b5f7d23c7f739a5b.xml (deflated 42%) 2024-06-26T06:31:02.6294317Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-5c199f6b21d56639.xml (deflated 42%) 2024-06-26T06:31:02.6296263Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-144b2118a5f635de.xml (deflated 42%) 2024-06-26T06:31:02.6298205Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-1d946dd15c1685c5.xml (deflated 42%) 2024-06-26T06:31:02.6300139Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-3e4c8e815d83e366.xml (deflated 42%) 2024-06-26T06:31:02.6302086Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-9b7e6cd63f966fb0.xml (deflated 42%) 2024-06-26T06:31:02.6304041Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-6faa3371c1dddd5d.xml (deflated 42%) 2024-06-26T06:31:02.6305992Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-c7c2b6c76dca2461.xml (deflated 42%) 2024-06-26T06:31:02.6307941Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-021a2d6c3e1e35e2.xml (deflated 42%) 2024-06-26T06:31:02.6309875Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-183c783ae4d1967e.xml (deflated 42%) 2024-06-26T06:31:02.6311888Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-1c6543c2c2ece257.xml (deflated 42%) 2024-06-26T06:31:02.6313915Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-ee45b95f2b77dd8a.xml (deflated 42%) 2024-06-26T06:31:02.6315860Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-76643c592b72f01e.xml (deflated 42%) 2024-06-26T06:31:02.6317787Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-479384cef5e21724.xml (deflated 42%) 2024-06-26T06:31:02.6319723Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-32029d37ab52dd5d.xml (deflated 42%) 2024-06-26T06:31:02.6321672Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-16f97e667f9c2c3c.xml (deflated 42%) 2024-06-26T06:31:02.6323620Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-afade541b15f4904.xml (deflated 42%) 2024-06-26T06:31:02.6325635Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-cb97b85437063fb5.xml (deflated 42%) 2024-06-26T06:31:02.6327576Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0a0675287b1fec21.xml (deflated 42%) 2024-06-26T06:31:02.6329518Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0b9bc8f9bcf4513d.xml (deflated 42%) 2024-06-26T06:31:02.6331474Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-8be04acc992b6d06.xml (deflated 42%) 2024-06-26T06:31:02.6333570Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-7ee717ca076301d6.xml (deflated 42%) 2024-06-26T06:31:02.6335676Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-b00376a91e3ade6d.xml (deflated 41%) 2024-06-26T06:31:02.6337687Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-18aec208fb518e9c.xml (deflated 41%) 2024-06-26T06:31:02.6339642Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-610ad0736adeaa26.xml (deflated 41%) 2024-06-26T06:31:02.6341586Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-1f7cc3f038901d55.xml (deflated 41%) 2024-06-26T06:31:02.6343517Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-e428fe8f0e8028f5.xml (deflated 41%) 2024-06-26T06:31:02.6345473Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-6d74614620e9b33b.xml (deflated 42%) 2024-06-26T06:31:02.6347414Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-c30ca3f938af25f2.xml (deflated 41%) 2024-06-26T06:31:02.6349361Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-817ba17535f8ebc6.xml (deflated 41%) 2024-06-26T06:31:02.6351312Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0a2715a602da854f.xml (deflated 41%) 2024-06-26T06:31:02.6353318Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-01665c5ea8cdd59f.xml (deflated 41%) 2024-06-26T06:31:02.6355337Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-4f81cd9b5a52cd75.xml (deflated 41%) 2024-06-26T06:31:02.6357291Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-1acc60c87acc8bfc.xml (deflated 41%) 2024-06-26T06:31:02.6359243Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-fd7faf75b5d73225.xml (deflated 41%) 2024-06-26T06:31:02.6361184Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-5328ab7c32099c6f.xml (deflated 42%) 2024-06-26T06:31:02.6363129Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-2c7c37ac03845e22.xml (deflated 41%) 2024-06-26T06:31:02.6365148Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-9abe0a77df5695cd.xml (deflated 41%) 2024-06-26T06:31:02.6367120Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-85519ebbfddc663a.xml (deflated 41%) 2024-06-26T06:31:02.6369045Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-06ce426378323333.xml (deflated 42%) 2024-06-26T06:31:02.6370995Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-0fac917220afaaaf.xml (deflated 42%) 2024-06-26T06:31:02.6372950Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-3a9da260e6daeea5.xml (deflated 42%) 2024-06-26T06:31:02.6374899Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-2c558a79261b6ec6.xml (deflated 43%) 2024-06-26T06:31:02.6376908Z adding: test/test-reports/python-pytest/distributed.rpc.cuda.test_tensorpipe_agent/distributed.rpc.cuda.test_tensorpipe_agent-dedbd8b9bb8bc920.xml (deflated 42%) 2024-06-26T06:31:02.6378682Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-9e7a3ac1d0d1fce4.xml (deflated 44%) 2024-06-26T06:31:02.6380272Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-21d28fdbff6adbd4.xml (deflated 44%) 2024-06-26T06:31:02.6381856Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-7b838d343a1b58e2.xml (deflated 44%) 2024-06-26T06:31:02.6383415Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-da82e424ddd90e7a.xml (deflated 43%) 2024-06-26T06:31:02.6384993Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-e68584db696d4331.xml (deflated 44%) 2024-06-26T06:31:02.6386558Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-37dba3d814357df0.xml (deflated 44%) 2024-06-26T06:31:02.6388129Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-9a8b161663ec9db5.xml (deflated 44%) 2024-06-26T06:31:02.6389688Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-63e17de01cd0a80d.xml (deflated 44%) 2024-06-26T06:31:02.6391258Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-ad5fe61ff946a8b4.xml (deflated 35%) 2024-06-26T06:31:02.6392898Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-20673f02ff9c85a3.xml (deflated 42%) 2024-06-26T06:31:02.6394468Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-7b7bffec52c79611.xml (deflated 42%) 2024-06-26T06:31:02.6396174Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-5b87e2f1e134b9a9.xml (deflated 35%) 2024-06-26T06:31:02.6397755Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-24a0b9071fa3354b.xml (deflated 35%) 2024-06-26T06:31:02.6399321Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-1f5142e1baf6d626.xml (deflated 35%) 2024-06-26T06:31:02.6400891Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-b296ec0857d3c254.xml (deflated 43%) 2024-06-26T06:31:02.6402448Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-9cb784c2eb920663.xml (deflated 42%) 2024-06-26T06:31:02.6404092Z adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-b821fd2c83122905.xml (deflated 35%) 2024-06-26T06:31:02.6405757Z adding: test/test-reports/python-pytest/distributed.algorithms.test_join/distributed.algorithms.test_join-4cff50abd7301c07.xml (deflated 79%) 2024-06-26T06:31:02.6407509Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-704fe7a0cf0ec12e.xml (deflated 36%) 2024-06-26T06:31:02.6409227Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-443c868e58aba8fe.xml (deflated 36%) 2024-06-26T06:31:02.6410944Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-30ce138b76fedbde.xml (deflated 36%) 2024-06-26T06:31:02.6412659Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-6c62435c798e0512.xml (deflated 36%) 2024-06-26T06:31:02.6414361Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-4555f31baa6a5794.xml (deflated 36%) 2024-06-26T06:31:02.6416058Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-e10c93fd940482bf.xml (deflated 36%) 2024-06-26T06:31:02.6417879Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-ca5c53e82136fff9.xml (deflated 36%) 2024-06-26T06:31:02.6419603Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-2a42c7362be677fd.xml (deflated 36%) 2024-06-26T06:31:02.6421308Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-b0e3800193e3787e.xml (deflated 36%) 2024-06-26T06:31:02.6422999Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-2902666a93c53ca8.xml (deflated 36%) 2024-06-26T06:31:02.6424715Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-4a124177a917f4e2.xml (deflated 36%) 2024-06-26T06:31:02.6426430Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-3253cced3057fb1f.xml (deflated 36%) 2024-06-26T06:31:02.6428140Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-8d6c3859277b9620.xml (deflated 35%) 2024-06-26T06:31:02.6429832Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-fca2d767f697260b.xml (deflated 36%) 2024-06-26T06:31:02.6431552Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-48deeef9c82c8727.xml (deflated 36%) 2024-06-26T06:31:02.6433337Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-e2e4dd20cab69af3.xml (deflated 36%) 2024-06-26T06:31:02.6435054Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-25bdcb3549d6944a.xml (deflated 36%) 2024-06-26T06:31:02.6436832Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-2deed269e050077d.xml (deflated 36%) 2024-06-26T06:31:02.6438553Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-6fda1ed216e0cdb5.xml (deflated 36%) 2024-06-26T06:31:02.6440262Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-20963c0bc0794517.xml (deflated 35%) 2024-06-26T06:31:02.6441963Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-50242a85b5cec8ff.xml (deflated 36%) 2024-06-26T06:31:02.6443656Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-85b1c22394a16fe1.xml (deflated 36%) 2024-06-26T06:31:02.6445452Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-cd7edbaca351c1b4.xml (deflated 35%) 2024-06-26T06:31:02.6447181Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-56fbb90de030f453.xml (deflated 35%) 2024-06-26T06:31:02.6448892Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-9c2baa6627b94991.xml (deflated 35%) 2024-06-26T06:31:02.6450584Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-938b5de33bd05931.xml (deflated 35%) 2024-06-26T06:31:02.6452302Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-d82ce272cabe4bcb.xml (deflated 35%) 2024-06-26T06:31:02.6454018Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-2c7348ec7b9ffa7d.xml (deflated 35%) 2024-06-26T06:31:02.6456059Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.test_sharded_tensor_reshard/distributed._shard.sharded_tensor.test_sharded_tensor_reshard-68d92f076f1a62a8.xml (deflated 70%) 2024-06-26T06:31:02.6458384Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_embedding/distributed._shard.sharded_tensor.ops.test_embedding-26aab299f3bc641b.xml (deflated 68%) 2024-06-26T06:31:02.6460591Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_embedding_bag/distributed._shard.sharded_tensor.ops.test_embedding_bag-32eb99b5fd5cd5fb.xml (deflated 68%) 2024-06-26T06:31:02.6462732Z adding: test/test-reports/python-pytest/distributed.optim.test_zero_redundancy_optimizer/distributed.optim.test_zero_redundancy_optimizer-154faf77b65b4cef.xml (deflated 94%) 2024-06-26T06:31:02.6464607Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_comm/distributed.fsdp.test_fsdp_comm-b78a3ba90445139e.xml (deflated 87%) 2024-06-26T06:31:02.6466518Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_tensor_ops/distributed._shard.sharded_tensor.ops.test_tensor_ops-f7f4cf761b585b35.xml (deflated 83%) 2024-06-26T06:31:02.6468676Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_binary_cmp/distributed._shard.sharded_tensor.ops.test_binary_cmp-116c4b124aebc8ec.xml (deflated 80%) 2024-06-26T06:31:02.6470678Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_state_dict/distributed.checkpoint.test_state_dict-5108bccfe07b0822.xml (deflated 84%) 2024-06-26T06:31:02.6472458Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-ddb43b03082f0338.xml (deflated 34%) 2024-06-26T06:31:02.6473983Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-d0b9b27188925eac.xml (deflated 37%) 2024-06-26T06:31:02.6475530Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-885e14c94cf2b18c.xml (deflated 35%) 2024-06-26T06:31:02.6477147Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-aca4c49e6ce51b3d.xml (deflated 43%) 2024-06-26T06:31:02.6478691Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c8a358b3374f8046.xml (deflated 34%) 2024-06-26T06:31:02.6480215Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-ab3781ffedce5f0d.xml (deflated 35%) 2024-06-26T06:31:02.6481751Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c1167061a52c1eef.xml (deflated 35%) 2024-06-26T06:31:02.6483276Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-429108a29edf554e.xml (deflated 36%) 2024-06-26T06:31:02.6484884Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e49073eca47bb622.xml (deflated 34%) 2024-06-26T06:31:02.6486401Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3a439d1830ce99c0.xml (deflated 35%) 2024-06-26T06:31:02.6487938Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c51e7b10b38a0fea.xml (deflated 43%) 2024-06-26T06:31:02.6489483Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b7e4a3bcbdff3f74.xml (deflated 35%) 2024-06-26T06:31:02.6491022Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0b986d4235f1f8fb.xml (deflated 43%) 2024-06-26T06:31:02.6492547Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-beab15a0df4ae1f6.xml (deflated 35%) 2024-06-26T06:31:02.6494084Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2ec9f863125520a2.xml (deflated 35%) 2024-06-26T06:31:02.6495615Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a033ff309955220c.xml (deflated 36%) 2024-06-26T06:31:02.6497142Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c5855af2814b35c3.xml (deflated 36%) 2024-06-26T06:31:02.6498735Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8af0a2922b577956.xml (deflated 43%) 2024-06-26T06:31:02.6500270Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4b74a04e57d8ae4b.xml (deflated 36%) 2024-06-26T06:31:02.6501804Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2511f015864ab001.xml (deflated 36%) 2024-06-26T06:31:02.6503316Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4c71cf0cf8b25441.xml (deflated 43%) 2024-06-26T06:31:02.6504861Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f2b4d2a72516a4e0.xml (deflated 35%) 2024-06-26T06:31:02.6506401Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9603174b519985b2.xml (deflated 36%) 2024-06-26T06:31:02.6507932Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e91be7b06c2b138c.xml (deflated 43%) 2024-06-26T06:31:02.6509448Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0c7ce1123447f6eb.xml (deflated 34%) 2024-06-26T06:31:02.6510979Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e67e68c6b05064da.xml (deflated 36%) 2024-06-26T06:31:02.6512584Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9e5db2c855f3ab82.xml (deflated 43%) 2024-06-26T06:31:02.6514120Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9b3194c970ccee6f.xml (deflated 36%) 2024-06-26T06:31:02.6515644Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f1b60eb67cd5339a.xml (deflated 36%) 2024-06-26T06:31:02.6517258Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-d82406950c410d14.xml (deflated 43%) 2024-06-26T06:31:02.6518789Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f4268b1c9446415c.xml (deflated 35%) 2024-06-26T06:31:02.6520314Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c523966750fac52c.xml (deflated 35%) 2024-06-26T06:31:02.6521826Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c41218eca58f3c4e.xml (deflated 35%) 2024-06-26T06:31:02.6523362Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-17663bba6e181ba1.xml (deflated 43%) 2024-06-26T06:31:02.6524958Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e692bb66b07b4424.xml (deflated 35%) 2024-06-26T06:31:02.6526498Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-ad38d731fb49b7ed.xml (deflated 36%) 2024-06-26T06:31:02.6528024Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c2c7ddaf185e6fc0.xml (deflated 43%) 2024-06-26T06:31:02.6529554Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-d46068b29c53211d.xml (deflated 36%) 2024-06-26T06:31:02.6531076Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-131cc09f9c587b78.xml (deflated 35%) 2024-06-26T06:31:02.6532582Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-5dd57703e4e69def.xml (deflated 36%) 2024-06-26T06:31:02.6534113Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e93e3b4abb8949bd.xml (deflated 36%) 2024-06-26T06:31:02.6535648Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3a8d3872e9ed95e8.xml (deflated 43%) 2024-06-26T06:31:02.6537258Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-1833593ad99b6328.xml (deflated 35%) 2024-06-26T06:31:02.6538775Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-392c60a3f3a559ef.xml (deflated 36%) 2024-06-26T06:31:02.6540303Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-606d44f3185f9057.xml (deflated 37%) 2024-06-26T06:31:02.6541827Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-6cb883df5904a3bd.xml (deflated 43%) 2024-06-26T06:31:02.6543365Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-cd1ec0847d0fb1b9.xml (deflated 36%) 2024-06-26T06:31:02.6544873Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-436a665c188dfcb7.xml (deflated 36%) 2024-06-26T06:31:02.6546413Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-dd127d94f7d0c36e.xml (deflated 47%) 2024-06-26T06:31:02.6547945Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-432a91eda71f97c5.xml (deflated 36%) 2024-06-26T06:31:02.6549476Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b2f44503f1ad934a.xml (deflated 57%) 2024-06-26T06:31:02.6550995Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b43fabae549d95d4.xml (deflated 36%) 2024-06-26T06:31:02.6552613Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-02454a50166e6773.xml (deflated 43%) 2024-06-26T06:31:02.6554132Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-1934de190506ff45.xml (deflated 37%) 2024-06-26T06:31:02.6555742Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0b48389a86d3f52f.xml (deflated 43%) 2024-06-26T06:31:02.6557261Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-bc478f6130dc0107.xml (deflated 43%) 2024-06-26T06:31:02.6558787Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-ff15b5567651376a.xml (deflated 43%) 2024-06-26T06:31:02.6560309Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-5f124f088e8141c7.xml (deflated 43%) 2024-06-26T06:31:02.6561836Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e509b75584d182fd.xml (deflated 42%) 2024-06-26T06:31:02.6563355Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a4bc9348b501c8c9.xml (deflated 42%) 2024-06-26T06:31:02.6565051Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9b03e0b1fdbd9488.xml (deflated 42%) 2024-06-26T06:31:02.6566607Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-bc0fa483c6a179c1.xml (deflated 41%) 2024-06-26T06:31:02.6568143Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-d47d0c6f0ab5021d.xml (deflated 42%) 2024-06-26T06:31:02.6569673Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0dc0754ccd881093.xml (deflated 42%) 2024-06-26T06:31:02.6571203Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-539798c76f0746c7.xml (deflated 42%) 2024-06-26T06:31:02.6572729Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e063a5e16b6c8678.xml (deflated 42%) 2024-06-26T06:31:02.6574248Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-d13e6c2a03ae538c.xml (deflated 42%) 2024-06-26T06:31:02.6575790Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-aaaffa7e6f4ea5d2.xml (deflated 42%) 2024-06-26T06:31:02.6577409Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0e21d46630211184.xml (deflated 42%) 2024-06-26T06:31:02.6578943Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2fd1541a7d1ea1ca.xml (deflated 42%) 2024-06-26T06:31:02.6580465Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-75cabfce6e03c651.xml (deflated 37%) 2024-06-26T06:31:02.6582005Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-93e6c629141c8edc.xml (deflated 43%) 2024-06-26T06:31:02.6583523Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-847857c258701a20.xml (deflated 36%) 2024-06-26T06:31:02.6585063Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8eb324961aab2a8a.xml (deflated 36%) 2024-06-26T06:31:02.6586591Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c42742f4c6654b01.xml (deflated 36%) 2024-06-26T06:31:02.6588107Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-d0686178690adedd.xml (deflated 36%) 2024-06-26T06:31:02.6589630Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0d831afc67038225.xml (deflated 45%) 2024-06-26T06:31:02.6591154Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0e20c68bfeedb3ee.xml (deflated 44%) 2024-06-26T06:31:02.6592755Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4407a292a082ff2a.xml (deflated 44%) 2024-06-26T06:31:02.6594293Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-ea11f387af1baaf8.xml (deflated 44%) 2024-06-26T06:31:02.6595923Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-95e343674a6c54a2.xml (deflated 43%) 2024-06-26T06:31:02.6597459Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b91e4c4e720928ed.xml (deflated 44%) 2024-06-26T06:31:02.6598977Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-acceb54f02ad13fc.xml (deflated 44%) 2024-06-26T06:31:02.6600516Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-5f80378b6361de08.xml (deflated 44%) 2024-06-26T06:31:02.6602045Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-04003b95a58d9f18.xml (deflated 37%) 2024-06-26T06:31:02.6603560Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9a5416ac98aed6a6.xml (deflated 36%) 2024-06-26T06:31:02.6605182Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-20d8d2dd6c76546e.xml (deflated 36%) 2024-06-26T06:31:02.6606717Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3126c2bc01c6839f.xml (deflated 36%) 2024-06-26T06:31:02.6608247Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-027f376f22aa40cf.xml (deflated 44%) 2024-06-26T06:31:02.6609767Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3085be86b7681373.xml (deflated 36%) 2024-06-26T06:31:02.6611297Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-ba51e1dc04ed8cb6.xml (deflated 44%) 2024-06-26T06:31:02.6612839Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2f7befe55e97e0f3.xml (deflated 37%) 2024-06-26T06:31:02.6614369Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-784c834486710e38.xml (deflated 36%) 2024-06-26T06:31:02.6615949Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a251ec54e78f8ead.xml (deflated 42%) 2024-06-26T06:31:02.6617480Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4a671aaf5a6ffb54.xml (deflated 42%) 2024-06-26T06:31:02.6619007Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4724bb8448f0d80e.xml (deflated 36%) 2024-06-26T06:31:02.6620536Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8971bfc4557aa4c0.xml (deflated 36%) 2024-06-26T06:31:02.6622058Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8bd903264c1cd1a5.xml (deflated 36%) 2024-06-26T06:31:02.6623592Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-85b2f2fa8545b929.xml (deflated 36%) 2024-06-26T06:31:02.6625129Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-cddeedc12c7b2c85.xml (deflated 36%) 2024-06-26T06:31:02.6626671Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c2a4b72db4f3a420.xml (deflated 37%) 2024-06-26T06:31:02.6628187Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-34dc60523c18e249.xml (deflated 36%) 2024-06-26T06:31:02.6629712Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-7fcf2fdca9336642.xml (deflated 37%) 2024-06-26T06:31:02.6631247Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-7bf91c68805d3f11.xml (deflated 44%) 2024-06-26T06:31:02.6632837Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4d83b544308f80a1.xml (deflated 44%) 2024-06-26T06:31:02.6634428Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-ac3975c6e5f07c2b.xml (deflated 43%) 2024-06-26T06:31:02.6635964Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4601c85334b60049.xml (deflated 43%) 2024-06-26T06:31:02.6637480Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-541aa38eccadfe15.xml (deflated 43%) 2024-06-26T06:31:02.6639005Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-869ad73ee19cacbd.xml (deflated 44%) 2024-06-26T06:31:02.6640537Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8f2796a5d1af7112.xml (deflated 43%) 2024-06-26T06:31:02.6642064Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-673a19d24122c4bb.xml (deflated 44%) 2024-06-26T06:31:02.6643588Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-946be76916d91be6.xml (deflated 43%) 2024-06-26T06:31:02.6645192Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3c400f8ce24492b9.xml (deflated 34%) 2024-06-26T06:31:02.6646774Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-98a4c42f9c89818a.xml (deflated 34%) 2024-06-26T06:31:02.6648303Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-7039e334fa67cec7.xml (deflated 34%) 2024-06-26T06:31:02.6649833Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-67104c91c0b3ea4e.xml (deflated 35%) 2024-06-26T06:31:02.6651349Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9fd67e5e517bf9d9.xml (deflated 34%) 2024-06-26T06:31:02.6652873Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-082e2bc286c37e11.xml (deflated 34%) 2024-06-26T06:31:02.6654482Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-aa2f7f95e8bf7a5c.xml (deflated 33%) 2024-06-26T06:31:02.6656012Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a188b56f7272158b.xml (deflated 36%) 2024-06-26T06:31:02.6657532Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-367499fa566e4495.xml (deflated 43%) 2024-06-26T06:31:02.6659066Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-667fd6cece08a283.xml (deflated 35%) 2024-06-26T06:31:02.6660597Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8b59a8645d431c81.xml (deflated 44%) 2024-06-26T06:31:02.6662131Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b68a3c10b9c822af.xml (deflated 35%) 2024-06-26T06:31:02.6663648Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f0378844f5066ee8.xml (deflated 43%) 2024-06-26T06:31:02.6665179Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-21d4bbef2f5ae5d1.xml (deflated 35%) 2024-06-26T06:31:02.6666716Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-bc7c6ee50b423b8f.xml (deflated 43%) 2024-06-26T06:31:02.6668254Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-98e2689eb8655e67.xml (deflated 36%) 2024-06-26T06:31:02.6669771Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b9a8a850b50e7fbc.xml (deflated 35%) 2024-06-26T06:31:02.6671302Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2851cd4a37061565.xml (deflated 43%) 2024-06-26T06:31:02.6672992Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-7d2a9cb50e8f5b54.xml (deflated 35%) 2024-06-26T06:31:02.6674517Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-7c8bc716c8438694.xml (deflated 35%) 2024-06-26T06:31:02.6676042Z adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-400be90c429c0d9f.xml (deflated 34%) 2024-06-26T06:31:02.6677403Z adding: test/test-reports/python-pytest/test.run_test/test.run_test-4a6781a992a36c2d.xml (deflated 64%) 2024-06-26T06:31:02.6678868Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-53d1e2d28cbeb9b2.xml (deflated 57%) 2024-06-26T06:31:02.6680528Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-72cc6b09afc213de.xml (deflated 42%) 2024-06-26T06:31:02.6682207Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4f0c1649e80a4932.xml (deflated 41%) 2024-06-26T06:31:02.6683883Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-11c252fbae5b25fc.xml (deflated 42%) 2024-06-26T06:31:02.6685638Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b4a0620d55e8d32c.xml (deflated 43%) 2024-06-26T06:31:02.6687292Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e3eee61e420e2903.xml (deflated 42%) 2024-06-26T06:31:02.6688965Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-22a44e0c71e5eda5.xml (deflated 36%) 2024-06-26T06:31:02.6690642Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-509f608575dde93c.xml (deflated 36%) 2024-06-26T06:31:02.6692307Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bbc15f4b02173518.xml (deflated 36%) 2024-06-26T06:31:02.6694039Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8e3f9a752cdcb0e1.xml (deflated 36%) 2024-06-26T06:31:02.6695708Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-24991fb5c1702ec2.xml (deflated 35%) 2024-06-26T06:31:02.6697386Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f69812b90bfca9b9.xml (deflated 43%) 2024-06-26T06:31:02.6699054Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-574f785bcafc2c7d.xml (deflated 44%) 2024-06-26T06:31:02.6700708Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7efb0f06b004a9c1.xml (deflated 36%) 2024-06-26T06:31:02.6702374Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ae37e2837b661b3a.xml (deflated 36%) 2024-06-26T06:31:02.6704050Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3cec6a265c3d6fcc.xml (deflated 35%) 2024-06-26T06:31:02.6705718Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-17e07edbaa02e682.xml (deflated 35%) 2024-06-26T06:31:02.6707367Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0f0ef2ad07f7683d.xml (deflated 35%) 2024-06-26T06:31:02.6709030Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-217c50cf7db98c1e.xml (deflated 42%) 2024-06-26T06:31:02.6733744Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e9e3a7e08c0b6868.xml (deflated 43%) 2024-06-26T06:31:02.6735609Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a5c044c56d763d81.xml (deflated 43%) 2024-06-26T06:31:02.6737472Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3496ddb9324926bd.xml (deflated 44%) 2024-06-26T06:31:02.6739147Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-25920f0349db2400.xml (deflated 44%) 2024-06-26T06:31:02.6740817Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c1e16bbbe92057c4.xml (deflated 45%) 2024-06-26T06:31:02.6742476Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-10d4c940e6b47771.xml (deflated 44%) 2024-06-26T06:31:02.6744151Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bb67f0b35a5feafe.xml (deflated 44%) 2024-06-26T06:31:02.6745840Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b4f810ea53ff3412.xml (deflated 44%) 2024-06-26T06:31:02.6747514Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9f1f69339c24309a.xml (deflated 44%) 2024-06-26T06:31:02.6749171Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1df7e3f8cca8648a.xml (deflated 44%) 2024-06-26T06:31:02.6750840Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-85a5a0a50e33eb37.xml (deflated 43%) 2024-06-26T06:31:02.6752596Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1ad43118ccbdb17e.xml (deflated 36%) 2024-06-26T06:31:02.6754273Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4b01614e9436e0bf.xml (deflated 50%) 2024-06-26T06:31:02.6755936Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c8fdf5dc60cba549.xml (deflated 42%) 2024-06-26T06:31:02.6757713Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-20cdd245758a36ba.xml (deflated 42%) 2024-06-26T06:31:02.6759390Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a13c6330d82ba888.xml (deflated 42%) 2024-06-26T06:31:02.6761068Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-06b60fe20461dad7.xml (deflated 35%) 2024-06-26T06:31:02.6762736Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e0dde0a98bcc7793.xml (deflated 42%) 2024-06-26T06:31:02.6764527Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a9e68be27ced0552.xml (deflated 56%) 2024-06-26T06:31:02.6766208Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-39a00ac4ee83e681.xml (deflated 42%) 2024-06-26T06:31:02.6767881Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fb6a145e458342c6.xml (deflated 42%) 2024-06-26T06:31:02.6769530Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5175a117434d364d.xml (deflated 42%) 2024-06-26T06:31:02.6771190Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-49a3123d0dd4d2ae.xml (deflated 42%) 2024-06-26T06:31:02.6772863Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-67bd27b85b917ff8.xml (deflated 42%) 2024-06-26T06:31:02.6774529Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5ba2c8d745f26853.xml (deflated 48%) 2024-06-26T06:31:02.6776266Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-29ca0b70abeca819.xml (deflated 42%) 2024-06-26T06:31:02.6777936Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-579611d6d6325cca.xml (deflated 42%) 2024-06-26T06:31:02.6779601Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-28e9a96eee682ed2.xml (deflated 42%) 2024-06-26T06:31:02.6781262Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cf87ec8d40d28672.xml (deflated 42%) 2024-06-26T06:31:02.6782919Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0f0caa57155124e6.xml (deflated 42%) 2024-06-26T06:31:02.6784585Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6339e56729f47634.xml (deflated 42%) 2024-06-26T06:31:02.6786252Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-59a9279eaad399cf.xml (deflated 42%) 2024-06-26T06:31:02.6787915Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-448fab0241b3e31e.xml (deflated 44%) 2024-06-26T06:31:02.6789579Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-87f1db2a61009c7a.xml (deflated 43%) 2024-06-26T06:31:02.6791244Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9f01b03ee30f8166.xml (deflated 42%) 2024-06-26T06:31:02.6792999Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d1d610a7f06ac025.xml (deflated 47%) 2024-06-26T06:31:02.6794660Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e28a6897bb87f9a7.xml (deflated 42%) 2024-06-26T06:31:02.6796337Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7e295a416a51a304.xml (deflated 41%) 2024-06-26T06:31:02.6798079Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0ad54411d895c383.xml (deflated 42%) 2024-06-26T06:31:02.6799754Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b6fe01387c487e6b.xml (deflated 35%) 2024-06-26T06:31:02.6801406Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f538e02777467617.xml (deflated 36%) 2024-06-26T06:31:02.6803074Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9b0f0917fb0bebcf.xml (deflated 42%) 2024-06-26T06:31:02.6804823Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0939643c604e194d.xml (deflated 35%) 2024-06-26T06:31:02.6806485Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f7f2cdb862328104.xml (deflated 43%) 2024-06-26T06:31:02.6808143Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d35902880f383315.xml (deflated 35%) 2024-06-26T06:31:02.6809809Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-46dd247a125d6528.xml (deflated 36%) 2024-06-26T06:31:02.6811463Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7378864f97337878.xml (deflated 35%) 2024-06-26T06:31:02.6813127Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-67665b873c781e5f.xml (deflated 36%) 2024-06-26T06:31:02.6814781Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0c645d18c1b2b062.xml (deflated 36%) 2024-06-26T06:31:02.6816529Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4e06086d876428d1.xml (deflated 35%) 2024-06-26T06:31:02.6818197Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b17a94c8105cd2e2.xml (deflated 42%) 2024-06-26T06:31:02.6819860Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-934eead81e4f0925.xml (deflated 35%) 2024-06-26T06:31:02.6821512Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c94e76a984d0edee.xml (deflated 45%) 2024-06-26T06:31:02.6823177Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-750dba84f57549dc.xml (deflated 43%) 2024-06-26T06:31:02.6824840Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f8409ab796bf61da.xml (deflated 43%) 2024-06-26T06:31:02.6826522Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e5e65d4fec339365.xml (deflated 43%) 2024-06-26T06:31:02.6828180Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4bd6273af48b004b.xml (deflated 42%) 2024-06-26T06:31:02.6829849Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8e6e385db6960b85.xml (deflated 42%) 2024-06-26T06:31:02.6831517Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-aee2b7e5fc1c0891.xml (deflated 41%) 2024-06-26T06:31:02.6833254Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-dfedaa4b3af0288c.xml (deflated 42%) 2024-06-26T06:31:02.6834908Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a4c755f4d4b2ade7.xml (deflated 35%) 2024-06-26T06:31:02.6836665Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0b7d0d72dbca81fb.xml (deflated 36%) 2024-06-26T06:31:02.6838338Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-db80a02d7f9ea806.xml (deflated 35%) 2024-06-26T06:31:02.6840001Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2096c63670868b93.xml (deflated 35%) 2024-06-26T06:31:02.6841647Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7cd00f53b424969e.xml (deflated 44%) 2024-06-26T06:31:02.6843303Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-37395abe1730244e.xml (deflated 35%) 2024-06-26T06:31:02.6845049Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8c2cf7974dc94a63.xml (deflated 35%) 2024-06-26T06:31:02.6846715Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-765ced398a8b4107.xml (deflated 35%) 2024-06-26T06:31:02.6848362Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9735729120efb3e6.xml (deflated 35%) 2024-06-26T06:31:02.6850026Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4a2ceed4af07d925.xml (deflated 35%) 2024-06-26T06:31:02.6851694Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4ddfad7d9dcf5ec8.xml (deflated 36%) 2024-06-26T06:31:02.6853358Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-50995239015039de.xml (deflated 43%) 2024-06-26T06:31:02.6855003Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e991f917f9ed0965.xml (deflated 42%) 2024-06-26T06:31:02.6856733Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-909d5109f2f56abf.xml (deflated 57%) 2024-06-26T06:31:02.6858384Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0c43f4a9c0885474.xml (deflated 42%) 2024-06-26T06:31:02.6860038Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-01b4b6eb519f493b.xml (deflated 41%) 2024-06-26T06:31:02.6861687Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-772cb8b330c0b906.xml (deflated 42%) 2024-06-26T06:31:02.6863343Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c8774f6c914d9ad7.xml (deflated 43%) 2024-06-26T06:31:02.6865005Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7e3d92b3c0c907ac.xml (deflated 42%) 2024-06-26T06:31:02.6866679Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-454b88cfb2ed491b.xml (deflated 36%) 2024-06-26T06:31:02.6868329Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-16e5ef130325334d.xml (deflated 35%) 2024-06-26T06:31:02.6869975Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-73571ac0adf7759a.xml (deflated 35%) 2024-06-26T06:31:02.6871692Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6c9ce9413b942e33.xml (deflated 35%) 2024-06-26T06:31:02.6873352Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d9aed6ae491b8455.xml (deflated 35%) 2024-06-26T06:31:02.6875008Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9fdec39c8dad1d39.xml (deflated 43%) 2024-06-26T06:31:02.6876750Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-664c1672fa4029b8.xml (deflated 44%) 2024-06-26T06:31:02.6878404Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-74b9f83e27bc37a9.xml (deflated 36%) 2024-06-26T06:31:02.6880066Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e30df68267f798bd.xml (deflated 36%) 2024-06-26T06:31:02.6881704Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e1136223f8926d72.xml (deflated 36%) 2024-06-26T06:31:02.6883356Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e6252dd0f519d6b1.xml (deflated 35%) 2024-06-26T06:31:02.6885096Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-79d68fb1e018f5a6.xml (deflated 35%) 2024-06-26T06:31:02.6886752Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-67a93a6981184b8c.xml (deflated 42%) 2024-06-26T06:31:02.6888400Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f5c22fe22d5b78ab.xml (deflated 43%) 2024-06-26T06:31:02.6890055Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-10e21366349700e0.xml (deflated 43%) 2024-06-26T06:31:02.6891723Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ee2f4d792dc656af.xml (deflated 44%) 2024-06-26T06:31:02.6893390Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-878c6e21a2c8fbf6.xml (deflated 44%) 2024-06-26T06:31:02.6895132Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-97c755daa17546d8.xml (deflated 45%) 2024-06-26T06:31:02.6896795Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-51d395fc28388aae.xml (deflated 44%) 2024-06-26T06:31:02.6898462Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-76d209adb8c1302f.xml (deflated 44%) 2024-06-26T06:31:02.6900127Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8aacfa03878a05db.xml (deflated 44%) 2024-06-26T06:31:02.6901782Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-74ea8a73dc048b87.xml (deflated 44%) 2024-06-26T06:31:02.6903444Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-123fc6415668d931.xml (deflated 44%) 2024-06-26T06:31:02.6905105Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1131e675c888e425.xml (deflated 43%) 2024-06-26T06:31:02.6906767Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-807d530bb7e1a585.xml (deflated 35%) 2024-06-26T06:31:02.6908408Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-67363c9e0029d5b5.xml (deflated 50%) 2024-06-26T06:31:02.6910068Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6325e5d8fe123ca7.xml (deflated 42%) 2024-06-26T06:31:02.6911800Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6afb7652a45d15e7.xml (deflated 42%) 2024-06-26T06:31:02.6913468Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9e8525e030786b19.xml (deflated 42%) 2024-06-26T06:31:02.6915124Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-35aa5caf75fc56ac.xml (deflated 35%) 2024-06-26T06:31:02.6916868Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c2bf3ee1bd6c54ba.xml (deflated 42%) 2024-06-26T06:31:02.6918554Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-43b2ab67803f089c.xml (deflated 56%) 2024-06-26T06:31:02.6920210Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-00c07f0555ef7199.xml (deflated 42%) 2024-06-26T06:31:02.6921864Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bd6646029824ac8b.xml (deflated 42%) 2024-06-26T06:31:02.6923532Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4eb8915cab2cb2fa.xml (deflated 42%) 2024-06-26T06:31:02.6925294Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1595613839913b72.xml (deflated 42%) 2024-06-26T06:31:02.6926962Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cbe15defe2bc3e9d.xml (deflated 42%) 2024-06-26T06:31:02.6928629Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ecddd40c9dfdbfb5.xml (deflated 48%) 2024-06-26T06:31:02.6930306Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b97536cefc04a9a4.xml (deflated 42%) 2024-06-26T06:31:02.6931983Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4b07ce1861bd3755.xml (deflated 42%) 2024-06-26T06:31:02.6933641Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e724f709368c371b.xml (deflated 42%) 2024-06-26T06:31:02.6935378Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e304ec97ff7bcc95.xml (deflated 42%) 2024-06-26T06:31:02.6937047Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-686d035e52aa0dca.xml (deflated 42%) 2024-06-26T06:31:02.6938702Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-280593c98065834f.xml (deflated 42%) 2024-06-26T06:31:02.6940352Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b22e37f9c86a7bac.xml (deflated 42%) 2024-06-26T06:31:02.6942023Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ea3dc805ef8260ef.xml (deflated 44%) 2024-06-26T06:31:02.6943690Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6682cfeb78df37a7.xml (deflated 43%) 2024-06-26T06:31:02.6945367Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f2cc8a8040574a80.xml (deflated 42%) 2024-06-26T06:31:02.6947012Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3a63624ae8b40847.xml (deflated 47%) 2024-06-26T06:31:02.6948675Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1d47b196c2d4b00f.xml (deflated 42%) 2024-06-26T06:31:02.6950345Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c53c9edd093e772f.xml (deflated 42%) 2024-06-26T06:31:02.6952074Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8c485209917fd3c0.xml (deflated 42%) 2024-06-26T06:31:02.6953726Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-29feb233a910753b.xml (deflated 35%) 2024-06-26T06:31:02.6955469Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9373d64692f81ba9.xml (deflated 36%) 2024-06-26T06:31:02.6957143Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bef5218c5ed06e02.xml (deflated 42%) 2024-06-26T06:31:02.6958821Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6a79da75815f8fc5.xml (deflated 35%) 2024-06-26T06:31:02.6960483Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-992d20d175c9d8e3.xml (deflated 43%) 2024-06-26T06:31:02.6962147Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3c820f199b5edf23.xml (deflated 35%) 2024-06-26T06:31:02.6963815Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-01e842c76bbbecd4.xml (deflated 37%) 2024-06-26T06:31:02.6965580Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-02ec98c1b1bd047e.xml (deflated 35%) 2024-06-26T06:31:02.6967239Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4f334aa4bb6c3117.xml (deflated 36%) 2024-06-26T06:31:02.6968910Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9f7d42aeb88672b1.xml (deflated 36%) 2024-06-26T06:31:02.6970579Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-04d9fadd82e46a0a.xml (deflated 36%) 2024-06-26T06:31:02.6972254Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1aa454399f0f91e3.xml (deflated 42%) 2024-06-26T06:31:02.6973903Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3a1987b59a3768e0.xml (deflated 35%) 2024-06-26T06:31:02.6975658Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-42418c47f79740e8.xml (deflated 45%) 2024-06-26T06:31:02.6977318Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c62bf2361f2672b6.xml (deflated 43%) 2024-06-26T06:31:02.6978981Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b65d0cc5b221b54c.xml (deflated 43%) 2024-06-26T06:31:02.6980630Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-42ad56da939f71ee.xml (deflated 43%) 2024-06-26T06:31:02.6982298Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-077c42bf53140a91.xml (deflated 42%) 2024-06-26T06:31:02.6983966Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-556a2ef5a6b966e7.xml (deflated 42%) 2024-06-26T06:31:02.6985626Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-951f0829002382b3.xml (deflated 41%) 2024-06-26T06:31:02.6987271Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6c971e8c500d0084.xml (deflated 42%) 2024-06-26T06:31:02.6988936Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e166ce859e4c8a2d.xml (deflated 35%) 2024-06-26T06:31:02.6990598Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1254768dc05f2362.xml (deflated 35%) 2024-06-26T06:31:02.6992322Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b6b7bc9255d63850.xml (deflated 35%) 2024-06-26T06:31:02.6993988Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cd454f7b4e104e5a.xml (deflated 35%) 2024-06-26T06:31:02.6995719Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f243278087c11d24.xml (deflated 45%) 2024-06-26T06:31:02.6997392Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b17841828f75cd5d.xml (deflated 36%) 2024-06-26T06:31:02.6999064Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f668a2a73cb7565b.xml (deflated 35%) 2024-06-26T06:31:02.7000715Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f1b17e882712b02b.xml (deflated 35%) 2024-06-26T06:31:02.7002376Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-532523f517988175.xml (deflated 35%) 2024-06-26T06:31:02.7004107Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c6d8f98d07651ee2.xml (deflated 36%) 2024-06-26T06:31:02.7005777Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0a617ba8091587ac.xml (deflated 35%) 2024-06-26T06:31:02.7007431Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8d63df103b6226fe.xml (deflated 42%) 2024-06-26T06:31:02.7009095Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-31908ed84b09d891.xml (deflated 42%) 2024-06-26T06:31:02.7034855Z ##[group]Run # Remove any previous usage logs if they exist 2024-06-26T06:31:02.7035481Z # Remove any previous usage logs if they exist 2024-06-26T06:31:02.7035960Z rm -f logs-*.zip 2024-06-26T06:31:02.7036626Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-06-26T06:31:02.7037572Z # so check to see if the file exists first 2024-06-26T06:31:02.7038057Z if [ -f 'usage_log.txt' ]; then 2024-06-26T06:31:02.7038579Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-06-26T06:31:02.7039058Z fi 2024-06-26T06:31:02.7039388Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-06-26T06:31:02.7039958Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-06-26T06:31:02.7040434Z fi 2024-06-26T06:31:02.7047767Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:31:02.7048263Z env: 2024-06-26T06:31:02.7048521Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:02.7049110Z DOCKER_CONTAINER_ID: a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:02.7049907Z FILE_SUFFIX: test-distributed-1-2-linux.2xlarge_26688305322 2024-06-26T06:31:02.7050419Z ##[endgroup] 2024-06-26T06:31:02.7127671Z adding: usage_log.txt (deflated 92%) 2024-06-26T06:31:02.7388065Z adding: test/test-reports/distributed.test_device_mesh_1.1_f08377f13545d0d1_.log (deflated 83%) 2024-06-26T06:31:02.7389719Z adding: test/test-reports/distributed.test_inductor_collectives_1.1_f8216589424c9da1_.log (deflated 90%) 2024-06-26T06:31:02.7406750Z adding: test/test-reports/distributed.test_distributed_spawn_2.3_4c0500aab60c72e1_.log (deflated 96%) 2024-06-26T06:31:02.7408928Z adding: test/test-reports/distributed.test_dynamo_distributed_1.1_2ff584d369a0c56e_.log (deflated 86%) 2024-06-26T06:31:02.7410740Z adding: test/test-reports/distributed._tools.test_memory_tracker_1.1_ebd5e9bebcbf95c4_.log (deflated 54%) 2024-06-26T06:31:02.7412491Z adding: test/test-reports/distributed.pipelining.test_backward_1.1_466c97da96aa55aa_.log (deflated 54%) 2024-06-26T06:31:02.7414540Z adding: test/test-reports/distributed._tensor.test_op_strategy_1.1_7ef4c5bdb9467d6c_.log (deflated 79%) 2024-06-26T06:31:02.7415777Z adding: test/test-reports/distributed.checkpoint.test_nested_dict_1.1_93b5db7937124748_.log (deflated 59%) 2024-06-26T06:31:02.7417062Z adding: test/test-reports/distributed.checkpoint.test_format_utils_1.1_4b274818ffa07382_.log (deflated 89%) 2024-06-26T06:31:02.7418585Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_training_1.1_4194bf092aea2f38_.log (deflated 85%) 2024-06-26T06:31:02.7419846Z adding: test/test-reports/distributed._tensor.test_init_1.1_11acf8501941fd1a_.log (deflated 69%) 2024-06-26T06:31:02.7421063Z adding: test/test-reports/distributed._shard.sharded_tensor.test_logger_1.1_507f67178a63ff85_.log (deflated 55%) 2024-06-26T06:31:02.7430617Z adding: test/test-reports/distributed.test_distributed_spawn_2.3_7711e783c60bd07d_.log (deflated 96%) 2024-06-26T06:31:02.7432600Z adding: test/test-reports/distributed.test_symmetric_memory_1.1_5de87428d7768a0d_.log (deflated 60%) 2024-06-26T06:31:02.7434180Z adding: test/test-reports/distributed._tensor.test_view_ops_1.1_df16645b01cee6ce_.log (deflated 81%) 2024-06-26T06:31:02.7435341Z adding: test/test-reports/distributed._spmd.test_graph_utils_1.1_2198ea0818db514b_.log (stored 0%) 2024-06-26T06:31:02.7436534Z adding: test/test-reports/distributed.pipelining.test_schedule_1.1_94b6df97d9b24161_.log (deflated 89%) 2024-06-26T06:31:02.7437763Z adding: test/test-reports/distributed._tensor.test_dtensor_compile_1.1_767ce764f1e40be9_.log (deflated 83%) 2024-06-26T06:31:02.7438925Z adding: test/test-reports/distributed._spmd.test_tracing_1.1_d9c72d0d45cf1749_.log (stored 0%) 2024-06-26T06:31:02.8619385Z adding: test/test-reports/distributed._tensor.test_dtensor_ops_1.1_028c9fbd3c5f64b0_.log (deflated 96%) 2024-06-26T06:31:02.8621815Z adding: test/test-reports/distributed.pipelining.test_microbatch_1.1_aed0bcda66c908d8_.log (deflated 58%) 2024-06-26T06:31:02.8624174Z adding: test/test-reports/distributed.tensor.parallel.test_micro_pipeline_tp_1.1_9eba7c3de3c4ab10_.log (deflated 85%) 2024-06-26T06:31:02.8626601Z adding: test/test-reports/distributed._tensor.test_utils_1.1_28a0892fdb0aa6d4_.log (deflated 88%) 2024-06-26T06:31:02.8629106Z adding: test/test-reports/distributed.pipelining.test_pipe_1.1_41a561095c319a08_.log (deflated 62%) 2024-06-26T06:31:02.8631385Z adding: test/test-reports/distributed.checkpoint.test_utils_1.1_bdec49b544dcea65_.log (deflated 65%) 2024-06-26T06:31:02.8633656Z adding: test/test-reports/distributed.fsdp.test_fsdp_fx_1.1_2b28e4f776f41678_.log (deflated 53%) 2024-06-26T06:31:02.8635882Z adding: test/test-reports/distributed._tensor.test_embedding_ops_1.1_920a2d3df7af1b0d_.log (deflated 82%) 2024-06-26T06:31:02.8637404Z adding: test/test-reports/distributed.test_c10d_functional_native_1.1_141eb86d6286dff2_.log (deflated 85%) 2024-06-26T06:31:02.8639539Z adding: test/test-reports/distributed.elastic.test_control_plane_1.1_616aa2ed03e2b8ee_.log (deflated 68%) 2024-06-26T06:31:02.8641910Z adding: test/test-reports/distributed.checkpoint.fsdp.test_fsdp_dsd_1.1_ad53db02073acef6_.log (deflated 67%) 2024-06-26T06:31:02.8644169Z adding: test/test-reports/distributed._tensor.test_convolution_ops_1.1_c3b5382b719c7a58_.log (deflated 60%) 2024-06-26T06:31:02.8646679Z adding: test/test-reports/distributed._tensor.experimental.test_tp_transform_1.1_827300e754bf732f_.log (deflated 65%) 2024-06-26T06:31:02.8648333Z adding: test/test-reports/distributed.checkpoint.test_compatibility_1.1_56365bb8d8f91ee8_.log (deflated 63%) 2024-06-26T06:31:02.8650332Z adding: test/test-reports/distributed._tensor.test_common_rules_1.1_ab8a228b06d5716b_.log (deflated 75%) 2024-06-26T06:31:02.8651448Z adding: test/test-reports/distributed.test_nccl_1.1_05856e1ca758642b_.log (deflated 49%) 2024-06-26T06:31:02.8652564Z adding: test/test-reports/distributed._tensor.test_random_ops_1.1_112cec47a7bafcda_.log (deflated 75%) 2024-06-26T06:31:02.8653765Z adding: test/test-reports/distributed._composable.test_replicate_1.1_4556cb96c9333fce_.log (deflated 74%) 2024-06-26T06:31:02.8654960Z adding: test/test-reports/distributed._spmd.test_transformation_1.1_a2199cd4073012b9_.log (stored 0%) 2024-06-26T06:31:02.8656131Z adding: test/test-reports/distributed.test_distributed_spawn_2.3_c919288ad32395b4_.log (deflated 37%) 2024-06-26T06:31:02.8657494Z adding: test/test-reports/distributed.checkpoint.test_save_load_api_1.1_24729426d695c01a_.log (deflated 53%) 2024-06-26T06:31:02.8659127Z adding: test/test-reports/distributed.checkpoint.test_tp_checkpoint_1.1_9be8fc9a57870ea0_.log (deflated 61%) 2024-06-26T06:31:02.8661534Z adding: test/test-reports/distributed.checkpoint.test_fsdp_optim_state_1.1_03abacb760056b83_.log (deflated 63%) 2024-06-26T06:31:02.8662828Z adding: test/test-reports/distributed.test_distributed_spawn_2.3_2f7858737262c1d6_.log (deflated 37%) 2024-06-26T06:31:02.8664443Z adding: test/test-reports/distributed.pipelining.test_unflatten_1.1_48a9806a7ed57725_.log (deflated 54%) 2024-06-26T06:31:02.8666396Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_logging_1.1_994b4993c41d43c3_.log (deflated 53%) 2024-06-26T06:31:02.8667715Z adding: test/test-reports/distributed.checkpoint.test_traverse_1.1_461db3b52737f951_.log (deflated 71%) 2024-06-26T06:31:02.8669038Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_compile_1.1_7e30047fc6b1e121_.log (deflated 76%) 2024-06-26T06:31:02.8670395Z adding: test/test-reports/distributed._tensor.experimental.test_local_map_1.1_6c57b83ba8b1c80a_.log (deflated 79%) 2024-06-26T06:31:02.8671893Z adding: test/test-reports/distributed.checkpoint.e2e.test_e2e_save_and_load_1.1_e8ba61ec4722dae8_.log (deflated 94%) 2024-06-26T06:31:02.8674038Z adding: test/test-reports/distributed.checkpoint.test_hsdp_checkpoint_1.1_5f59a5a61b68c83e_.log (deflated 71%) 2024-06-26T06:31:02.8676167Z adding: test/test-reports/distributed.checkpoint.test_state_dict_utils_1.1_81e06d99c56358fc_.log (deflated 73%) 2024-06-26T06:31:02.8678759Z adding: test/test-reports/distributed.tensor.parallel.test_fsdp_2d_parallel_1.1_f306477029b9092c_.log (deflated 81%) 2024-06-26T06:31:02.8680707Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_util_1.1_2d0ca7103825c788_.log (deflated 57%) 2024-06-26T06:31:02.8682970Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_frozen_1.1_982b2d878f8b16df_.log (deflated 67%) 2024-06-26T06:31:02.8684473Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_state_dict_1.1_4531b458d89bf839_.log (deflated 71%) 2024-06-26T06:31:02.8685791Z adding: test/test-reports/distributed.fsdp.test_fsdp_pure_fp16_1.1_18e4e1ba05e30e20_.log (deflated 60%) 2024-06-26T06:31:02.8687009Z adding: test/test-reports/distributed.fsdp.test_fsdp_checkpoint_1.1_a6c61f83d9618dbd_.log (deflated 87%) 2024-06-26T06:31:02.8688373Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_clip_grad_norm__1.1_47b5d317491f7d89_.log (deflated 63%) 2024-06-26T06:31:02.8689694Z adding: test/test-reports/distributed._composable.test_compose_1.1_c208bd5ef1ccd15c_.log (deflated 78%) 2024-06-26T06:31:02.8692044Z adding: test/test-reports/distributed._composable.test_replicate_with_compiler_1.1_251b88e7457f90be_.log (deflated 77%) 2024-06-26T06:31:02.8694377Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_mixed_precision_1.1_dd2cfa6dcf169861_.log (deflated 58%) 2024-06-26T06:31:02.8696138Z adding: test/test-reports/distributed.fsdp.test_wrap_1.1_b7ab1814b6d4ca67_.log (deflated 90%) 2024-06-26T06:31:02.8697449Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_mixed_precision_1.1_8386c04f55fd1622_.log (deflated 74%) 2024-06-26T06:31:02.8699245Z adding: test/test-reports/distributed._tensor.test_optimizers_1.1_3ab6c51beb8d1894_.log (deflated 95%) 2024-06-26T06:31:02.8701576Z adding: test/test-reports/distributed.fsdp.test_fsdp_ignored_modules_1.1_c3ebbb899ed7c563_.log (deflated 85%) 2024-06-26T06:31:02.8703791Z adding: test/test-reports/distributed.test_fake_pg_1.1_64dbc573177f040e_.log (deflated 73%) 2024-06-26T06:31:02.8705921Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_init_1.1_1b26dd0fcb4c1cf0_.log (deflated 75%) 2024-06-26T06:31:02.8707688Z adding: test/test-reports/distributed.launcher.test_run_1.1_d8eeab2b24a0b3c2_.log (deflated 88%) 2024-06-26T06:31:02.8708930Z adding: test/test-reports/distributed.fsdp.test_fsdp_dtensor_state_dict_1.1_eba1493e76bd7e2e_.log (deflated 86%) 2024-06-26T06:31:02.8710233Z adding: test/test-reports/distributed.fsdp.test_fsdp_backward_prefetch_1.1_b205c3b2637ea7d1_.log (deflated 55%) 2024-06-26T06:31:02.8711502Z adding: test/test-reports/distributed.fsdp.test_fsdp_optim_state_1.1_b444d10bf5fc0ee2_.log (deflated 97%) 2024-06-26T06:31:02.8712774Z adding: test/test-reports/distributed.rpc.test_share_memory_1.1_8d18a32aa4c33a8e_.log (deflated 56%) 2024-06-26T06:31:02.8714028Z adding: test/test-reports/distributed.fsdp.test_fsdp_sharded_grad_scaler_1.1_3d31b516f2e6c443_.log (deflated 89%) 2024-06-26T06:31:02.8715465Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_model_checkpoint_1.1_6e1f4cfca0164eaa_.log (deflated 73%) 2024-06-26T06:31:02.8716876Z adding: test/test-reports/distributed.fsdp.test_fsdp_flatten_params_1.1_167849ad47054a32_.log (deflated 81%) 2024-06-26T06:31:02.8718118Z adding: test/test-reports/distributed.nn.jit.test_instantiator_1.1_915e48a3b31e6d90_.log (deflated 63%) 2024-06-26T06:31:02.8719374Z adding: test/test-reports/distributed.fsdp.test_distributed_checkpoint_1.1_ce222e4fd62f43b6_.log (deflated 66%) 2024-06-26T06:31:02.8720622Z adding: test/test-reports/distributed.fsdp.test_fsdp_overlap_1.1_8a80e1dc3a82e8b3_.log (deflated 59%) 2024-06-26T06:31:02.8721814Z adding: test/test-reports/distributed.fsdp.test_fsdp_memory_1.1_eae8fa3f0fee61f9_.log (deflated 60%) 2024-06-26T06:31:02.8723056Z adding: test/test-reports/distributed.fsdp.test_fsdp_freezing_weights_1.1_03184518e7618f5c_.log (deflated 93%) 2024-06-26T06:31:02.8724367Z adding: test/test-reports/distributed.fsdp.test_fsdp_input_1.1_854df29ec96f1e14_.log (deflated 58%) 2024-06-26T06:31:02.8725654Z adding: test/test-reports/distributed.fsdp.test_fsdp_apply_1.1_c88332a77ca564a2_.log (deflated 63%) 2024-06-26T06:31:02.8726905Z adding: test/test-reports/distributed.fsdp.test_fsdp_multiple_wrapping_1.1_cccbffc5d99c9b0a_.log (deflated 56%) 2024-06-26T06:31:02.8728173Z adding: test/test-reports/distributed.fsdp.test_fsdp_state_dict_1.1_9c2c45be7e044f32_.log (deflated 96%) 2024-06-26T06:31:02.8729356Z adding: test/test-reports/distributed.fsdp.test_fsdp_uneven_1.1_461bcf891b0389af_.log (deflated 54%) 2024-06-26T06:31:02.8730678Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_extensions_1.1_e62e45ae76f69358_.log (deflated 67%) 2024-06-26T06:31:02.8732134Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_runtime_1.1_cb898f7935a3a33c_.log (deflated 62%) 2024-06-26T06:31:02.8733455Z adding: test/test-reports/distributed.fsdp.test_fsdp_comm_hooks_1.1_7e4cf05f37222e0d_.log (deflated 89%) 2024-06-26T06:31:02.8734757Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_comm_1.1_4a5a2ff612a649d6_.log (deflated 80%) 2024-06-26T06:31:02.8736077Z adding: test/test-reports/distributed.fsdp.test_checkpoint_wrapper_1.1_7f13227bee5ed1bb_.log (deflated 74%) 2024-06-26T06:31:02.8737430Z adding: test/test-reports/distributed.checkpoint.test_file_system_checkpoint_cpu_1.1_72ac24a3066a17a5_.log (deflated 95%) 2024-06-26T06:31:02.8738778Z adding: test/test-reports/distributed.fsdp.test_fsdp_tp_integration_1.1_ba548e68f370647a_.log (deflated 67%) 2024-06-26T06:31:02.8740006Z adding: test/test-reports/distributed.fsdp.test_fsdp_traversal_1.1_44453e9b30ccf47f_.log (deflated 53%) 2024-06-26T06:31:02.8741210Z adding: test/test-reports/distributed.fsdp.test_fsdp_fine_tune_1.1_020f32c8ca30608c_.log (deflated 68%) 2024-06-26T06:31:02.8742892Z adding: test/test-reports/distributed.test_c10d_object_collectives_1.1_60bd2e0ca47348d9_.log (deflated 76%) 2024-06-26T06:31:02.8744072Z adding: test/test-reports/distributed._shard.test_sharder_1.1_2db4f35583aebb47_.log (deflated 59%) 2024-06-26T06:31:02.8745342Z adding: test/test-reports/distributed._shard.sharded_optim.test_sharded_optim_1.1_2e9ebdbcab3f334a_.log (deflated 62%) 2024-06-26T06:31:02.8746915Z adding: test/test-reports/distributed._composable.fully_shard.test_fully_shard_optim_checkpoint_1.1_a1a63449716358d1_.log (deflated 65%) 2024-06-26T06:31:02.8748324Z adding: test/test-reports/distributed.fsdp.test_fsdp_multiple_forward_1.1_237c0b3538c50cac_.log (deflated 56%) 2024-06-26T06:31:02.8749543Z adding: test/test-reports/distributed.fsdp.test_fsdp_meta_1.1_6b99eb242f0d8515_.log (deflated 84%) 2024-06-26T06:31:02.8750819Z adding: test/test-reports/distributed._shard.sharding_spec.test_sharding_spec_1.1_1d5103ce7f1b51ce_.log (deflated 78%) 2024-06-26T06:31:02.8752205Z adding: test/test-reports/distributed.fsdp.test_fsdp_unshard_params_1.1_d8d5aa52ebaaa131_.log (deflated 83%) 2024-06-26T06:31:02.8753446Z adding: test/test-reports/distributed.rpc.test_tensorpipe_agent_2.3_aa8e9a201e8e3889_.log (deflated 97%) 2024-06-26T06:31:02.8761410Z adding: test/test-reports/distributed.rpc.cuda.test_tensorpipe_agent_1.1_986ba45a62725a6a_.log (deflated 98%) 2024-06-26T06:31:02.8763206Z adding: test/test-reports/distributed.test_pg_wrapper_1.1_207b1911e44452fb_.log (deflated 93%) 2024-06-26T06:31:02.8764915Z adding: test/test-reports/distributed.algorithms.test_join_1.1_fd3c909982d13987_.log (deflated 81%) 2024-06-26T06:31:02.8771826Z adding: test/test-reports/distributed.rpc.test_faulty_agent_1.1_c88edb2570941c31_.log (deflated 96%) 2024-06-26T06:31:02.8773839Z adding: test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_reshard_1.1_3d7a675a0cd844d2_.log (deflated 64%) 2024-06-26T06:31:02.8775280Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_1.1_627769cedf804af2_.log (deflated 62%) 2024-06-26T06:31:02.8777602Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_bag_1.1_7957dccbde1acd72_.log (deflated 63%) 2024-06-26T06:31:02.8779804Z adding: test/test-reports/distributed.optim.test_zero_redundancy_optimizer_1.1_aee6ef40716bed63_.log (deflated 93%) 2024-06-26T06:31:02.8781334Z adding: test/test-reports/distributed.fsdp.test_fsdp_comm_1.1_b6229472297a4e82_.log (deflated 80%) 2024-06-26T06:31:02.8782630Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_tensor_ops_1.1_a3bfb58078273d4a_.log (deflated 71%) 2024-06-26T06:31:02.8784045Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_binary_cmp_1.1_a89b440ca1a00325_.log (deflated 71%) 2024-06-26T06:31:02.8785364Z adding: test/test-reports/distributed.checkpoint.test_state_dict_1.1_ec0dd4cb8b7d8ce9_.log (deflated 87%) 2024-06-26T06:31:02.8786509Z adding: test/test-reports/distributed.test_c10d_ucc_1.1_240c333d8940a9af_.log (stored 0%) 2024-06-26T06:31:02.8796787Z adding: test/test-reports/distributed.test_c10d_gloo_1.1_64d0e83d84573936_.log (deflated 96%) 2024-06-26T06:31:02.8798280Z adding: test/test-reports/distributed.test_c10d_ops_nccl_1.1_0d2e6b2adc6a5f3e_.log (stored 0%) 2024-06-26T06:31:02.8799295Z adding: test/test-reports/cpp.test_cpp_rpc_1.1_4c4e9924744f297f_.log (deflated 78%) 2024-06-26T06:31:02.8827489Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-06-26T06:31:02.8828201Z # Remove any previous debugging artifacts if they exist 2024-06-26T06:31:02.8828737Z rm -f debug-*.zip 2024-06-26T06:31:02.8829092Z if [ -d 'test/debug' ]; then 2024-06-26T06:31:02.8829559Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-06-26T06:31:02.8830020Z fi 2024-06-26T06:31:02.8841197Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:31:02.8841675Z env: 2024-06-26T06:31:02.8841930Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:02.8842531Z DOCKER_CONTAINER_ID: a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:02.8843316Z FILE_SUFFIX: test-distributed-1-2-linux.2xlarge_26688305322 2024-06-26T06:31:02.8843838Z ##[endgroup] 2024-06-26T06:31:02.8922246Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T06:31:02.8922659Z with: 2024-06-26T06:31:02.8923021Z s3-bucket: gha-artifacts 2024-06-26T06:31:02.8923432Z s3-prefix: pytorch/pytorch/9673645538/1/artifact 2024-06-26T06:31:02.8923874Z retention-days: 14 2024-06-26T06:31:02.8924345Z if-no-files-found: warn 2024-06-26T06:31:02.8924684Z path: test-jsons-*.zip 2024-06-26T06:31:02.8924987Z name: artifact 2024-06-26T06:31:02.8925269Z region: us-east-1 2024-06-26T06:31:02.8925547Z env: 2024-06-26T06:31:02.8925783Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:02.8926381Z DOCKER_CONTAINER_ID: a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:02.8927025Z ##[endgroup] 2024-06-26T06:31:03.2636938Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T06:31:03.2637677Z With the provided path, there will be 1 file uploaded 2024-06-26T06:31:03.2638293Z Uploading to s3 prefix: pytorch/pytorch/9673645538/1/artifact 2024-06-26T06:31:03.3846145Z Starting upload of test-jsons-test-distributed-1-2-linux.2xlarge_26688305322.zip 2024-06-26T06:31:03.5315577Z Finished upload of test-jsons-test-distributed-1-2-linux.2xlarge_26688305322.zip 2024-06-26T06:31:03.5449983Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T06:31:03.5450401Z with: 2024-06-26T06:31:03.5450695Z s3-bucket: gha-artifacts 2024-06-26T06:31:03.5451151Z s3-prefix: pytorch/pytorch/9673645538/1/artifact 2024-06-26T06:31:03.5451589Z retention-days: 14 2024-06-26T06:31:03.5451902Z if-no-files-found: error 2024-06-26T06:31:03.5452245Z path: test-reports-*.zip 2024-06-26T06:31:03.5452556Z name: artifact 2024-06-26T06:31:03.5452831Z region: us-east-1 2024-06-26T06:31:03.5453102Z env: 2024-06-26T06:31:03.5453333Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:03.5453921Z DOCKER_CONTAINER_ID: a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:03.5454566Z ##[endgroup] 2024-06-26T06:31:03.8875604Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T06:31:03.8876583Z With the provided path, there will be 1 file uploaded 2024-06-26T06:31:03.8877694Z Uploading to s3 prefix: pytorch/pytorch/9673645538/1/artifact 2024-06-26T06:31:03.8914495Z Starting upload of test-reports-test-distributed-1-2-linux.2xlarge_26688305322.zip 2024-06-26T06:31:04.0143716Z Finished upload of test-reports-test-distributed-1-2-linux.2xlarge_26688305322.zip 2024-06-26T06:31:04.0293777Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T06:31:04.0294208Z with: 2024-06-26T06:31:04.0294453Z s3-bucket: gha-artifacts 2024-06-26T06:31:04.0294858Z s3-prefix: pytorch/pytorch/9673645538/1/artifact 2024-06-26T06:31:04.0295310Z retention-days: 14 2024-06-26T06:31:04.0295611Z if-no-files-found: ignore 2024-06-26T06:31:04.0295944Z path: logs-*.zip 2024-06-26T06:31:04.0296221Z name: artifact 2024-06-26T06:31:04.0296483Z region: us-east-1 2024-06-26T06:31:04.0296752Z env: 2024-06-26T06:31:04.0297013Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:04.0297594Z DOCKER_CONTAINER_ID: a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:04.0298243Z ##[endgroup] 2024-06-26T06:31:04.3742171Z NOTE: s3-prefix specified, ignoring name parameter 2024-06-26T06:31:04.3742770Z With the provided path, there will be 1 file uploaded 2024-06-26T06:31:04.3743379Z Uploading to s3 prefix: pytorch/pytorch/9673645538/1/artifact 2024-06-26T06:31:04.3778343Z Starting upload of logs-test-distributed-1-2-linux.2xlarge_26688305322.zip 2024-06-26T06:31:04.5358510Z Finished upload of logs-test-distributed-1-2-linux.2xlarge_26688305322.zip 2024-06-26T06:31:04.5494157Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-06-26T06:31:04.5494572Z with: 2024-06-26T06:31:04.5494830Z s3-bucket: gha-artifacts 2024-06-26T06:31:04.5495236Z s3-prefix: pytorch/pytorch/9673645538/1/artifact 2024-06-26T06:31:04.5495678Z retention-days: 14 2024-06-26T06:31:04.5495990Z if-no-files-found: ignore 2024-06-26T06:31:04.5496338Z path: debug-*.zip 2024-06-26T06:31:04.5496611Z name: artifact 2024-06-26T06:31:04.5496889Z region: us-east-1 2024-06-26T06:31:04.5497163Z env: 2024-06-26T06:31:04.5497398Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:04.5498135Z DOCKER_CONTAINER_ID: a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:04.5498786Z ##[endgroup] 2024-06-26T06:31:04.8919100Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-06-26T06:31:04.9054951Z ##[group]Run # shellcheck disable=SC2156 2024-06-26T06:31:04.9055719Z # shellcheck disable=SC2156 2024-06-26T06:31:04.9056530Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-06-26T06:31:04.9064436Z shell: /usr/bin/bash -e {0} 2024-06-26T06:31:04.9064769Z env: 2024-06-26T06:31:04.9065020Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:04.9065607Z DOCKER_CONTAINER_ID: a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:04.9066277Z ##[endgroup] 2024-06-26T06:31:05.1585617Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-06-26T06:31:05.1586159Z with: 2024-06-26T06:31:05.1586392Z env: 2024-06-26T06:31:05.1586656Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:05.1587242Z DOCKER_CONTAINER_ID: a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:05.1588158Z ##[endgroup] 2024-06-26T06:31:05.1612568Z ##[group]Run set -eou pipefail 2024-06-26T06:31:05.1613149Z set -eou pipefail 2024-06-26T06:31:05.1613571Z  2024-06-26T06:31:05.1614071Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-06-26T06:31:05.1614695Z for _ in $(seq 1440); do 2024-06-26T06:31:05.1615148Z  # Break if no ssh session exists anymore 2024-06-26T06:31:05.1615629Z  if [ "$(who)" = "" ]; then 2024-06-26T06:31:05.1616002Z  break 2024-06-26T06:31:05.1616470Z  fi 2024-06-26T06:31:05.1616782Z  echo "." 2024-06-26T06:31:05.1617092Z  sleep 5 2024-06-26T06:31:05.1617385Z done 2024-06-26T06:31:05.1624597Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:31:05.1625081Z env: 2024-06-26T06:31:05.1625334Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:05.1625919Z DOCKER_CONTAINER_ID: a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:05.1626574Z ##[endgroup] 2024-06-26T06:31:05.1655347Z Holding runner for 2 hours until all ssh sessions have logged out 2024-06-26T06:31:05.1733917Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T06:31:05.1734769Z # ignore expansion of "docker ps -q" since it could be empty 2024-06-26T06:31:05.1735357Z # shellcheck disable=SC2046 2024-06-26T06:31:05.1735796Z docker stop $(docker ps -q) || true 2024-06-26T06:31:05.1736246Z # Prune all of the docker images 2024-06-26T06:31:05.1736688Z docker system prune -af 2024-06-26T06:31:05.1743621Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:31:05.1744091Z env: 2024-06-26T06:31:05.1744343Z GIT_DEFAULT_BRANCH: main 2024-06-26T06:31:05.1744942Z DOCKER_CONTAINER_ID: a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:05.1745585Z ##[endgroup] 2024-06-26T06:31:15.5603511Z a39c6ed03d84 2024-06-26T06:31:16.1976122Z Deleted Containers: 2024-06-26T06:31:16.1976710Z a39c6ed03d845535427f977d68abcd301d3ed42938f9b9870d212dbd05e7d94d 2024-06-26T06:31:16.1977167Z 2024-06-26T06:31:23.3932548Z Deleted Images: 2024-06-26T06:31:23.3934076Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:91382da70d5719cd7007b6b80b71d2f48398f6b7 2024-06-26T06:31:23.3937240Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11@sha256:1862f87fe60cbac0efa4720f6f6fe873270a315ff02a9510d48d1480259bc4e9 2024-06-26T06:31:23.3940720Z deleted: sha256:14402cfc353dbfa2ba1fa30f87c7ccf0e77a56d529c503e4a803c15ef3448962 2024-06-26T06:31:23.3942913Z deleted: sha256:c4b668a1ad3fb3b763ceeb3a66547f5fc09b8bd764f0062a1387f2f4f842e340 2024-06-26T06:31:23.3943954Z deleted: sha256:9e0c9058a83b79b9786d4e5bb0470c0262850b1d6aea22df279b8f3e1c2950c4 2024-06-26T06:31:23.3944817Z deleted: sha256:281571e65d91d16ba049b2833b2dde3aa1c2664a356d6157f6b35c113a3c8649 2024-06-26T06:31:23.3945675Z deleted: sha256:3d66dd8ed8d0784a5d245bb02e6f167a04c6655845cd5644370ba5a62e842272 2024-06-26T06:31:23.3946547Z deleted: sha256:a15ea8fd1fa6e97d11fd359096c95df8bc8a9af9cf8a51a3facf0eb7bad9fe1b 2024-06-26T06:31:23.3947424Z deleted: sha256:295dc0948a636fa3924b05041866db6626f9cc88d6cfa76f3532f61a00480e23 2024-06-26T06:31:23.3948356Z deleted: sha256:38a680ead4b5c015c28132d8223ee60983f1eac2b95b532e36fddb578ce42482 2024-06-26T06:31:23.3949203Z deleted: sha256:e5a82c91f9129d20c07608447221c64bf076d6f3fe2bb07faf516812e410da8f 2024-06-26T06:31:23.3950350Z deleted: sha256:d73f714a077af50db024bc1fd2444f16c27557cc96e93bad30376148147b415f 2024-06-26T06:31:23.3951349Z deleted: sha256:728c6da8b9d60375b1dc81787d3ccdc0d5c307bb373c6c00125cb845d37d5c10 2024-06-26T06:31:23.3952285Z deleted: sha256:0cd9350a208360297b2ce527623c50a3bf68186eb5399c56b69374cd6086cc50 2024-06-26T06:31:23.3953157Z deleted: sha256:5bc9ef03bfcd8d77a0d8af6b5989fcd2b1003e30f6fa659996b66c1d20d21025 2024-06-26T06:31:23.3954041Z deleted: sha256:78a6aa8bae559223884c5032eff98c7cca73d512fe223d614b2dcefdf3371d8b 2024-06-26T06:31:23.3954920Z deleted: sha256:0a2f3f76f96bb9a9b9140cb2711bcebaff2408a668284502d5ecfdfb92da4c50 2024-06-26T06:31:23.3955779Z deleted: sha256:35b68f9a682c7418f4e832da53b6bfd44889bfa5f7631f3d5eaf305a9e7ee312 2024-06-26T06:31:23.3956649Z deleted: sha256:5e5329bf94ea01ef9d0ce70d0e5c4e1d4f2512a430641c87e67d47573ebfcf7b 2024-06-26T06:31:23.3957504Z deleted: sha256:5609ea1cf24c59768b94b369598a390890a4e5f5544dc3b677f2e221e6a5ecb3 2024-06-26T06:31:23.3958336Z deleted: sha256:315afb4287c19fa61b991e8199421be2e242d1a29fbae56598625e66e7a9bc53 2024-06-26T06:31:23.3959327Z deleted: sha256:af642579485beff363b047a6737376cdc544eafa4e1db8c6cf7d8d875cde7a25 2024-06-26T06:31:23.3960203Z deleted: sha256:c835609211bf9fc1da173d4bcacbd0d04d57fdeb2e8036cd5dee8fa707cf9e4d 2024-06-26T06:31:23.3961112Z deleted: sha256:29a2e9fadb40fbf33f0ff1e16123fce6e8dd3090cb5c7bd755e8662936c7e7bb 2024-06-26T06:31:23.3961985Z deleted: sha256:839efd3e816ebb0385fb0b0394d64f44ed16af4626057798c1cfa28122f9bf7d 2024-06-26T06:31:23.3962852Z deleted: sha256:3f2502c497a5baa23c57bc8fc7c10abd8a6a448c0752b3bf29d4a0d82f6efc49 2024-06-26T06:31:23.3963729Z deleted: sha256:6cf9f28692acff1d9a0e46d0e00604b0eb1cca90d61287b7c2d4b2ee6dbc5167 2024-06-26T06:31:23.3965002Z deleted: sha256:eb36b3a0e2e450b56dc06d2655f4b091a56785af6a9e717e81ab54092c5c2100 2024-06-26T06:31:23.3965861Z deleted: sha256:5d435930e444677f1118f27453a281e8dcaaa746eb2eb5e1c74eb284c95e96e0 2024-06-26T06:31:23.3966717Z deleted: sha256:972bf551c6baa2c33f452c367d1ce2b4b56e6648e91a793fd9e9cec4308a0cde 2024-06-26T06:31:23.3967586Z deleted: sha256:c871181017d9cfde42ca3023b0dda86c9b3e33b4d734fd46a9aad894ba1ece49 2024-06-26T06:31:23.3968461Z deleted: sha256:31fcc18e5c72471e6d104fb96490aaf5f86d80b8959abd1649bf6bc37c7ff303 2024-06-26T06:31:23.3969324Z deleted: sha256:bb334a6aba089f108e2cb9937f20597e3c892b0d75b3f9d1bc0573215f64be64 2024-06-26T06:31:23.3970169Z deleted: sha256:b305692cd297d69d10a08bc107ff39cc6c1038a3b533fa5b7d798e90f777ff1a 2024-06-26T06:31:23.3971027Z deleted: sha256:4712c5b4638b608b8ddca9f6b337c58e7d396b872f365141b0772af2b1b63013 2024-06-26T06:31:23.3971881Z deleted: sha256:0b18957705c2ee8e3be564aa8aa57af17955b83616634f2dc2a3933e1bb9dbaf 2024-06-26T06:31:23.3972736Z deleted: sha256:e27827ac7ce7d5848f06b77890bd8c8becc86569eac77781e3c7f2301381f831 2024-06-26T06:31:23.3973572Z deleted: sha256:246ee1ea5f1498d711167f1822133c2e300501f725c64dcc0e973fe22feb4897 2024-06-26T06:31:23.3974410Z deleted: sha256:7620cd23ed588e59112e0f5578e30448c6d9f4de0e875929870da42d3dd060da 2024-06-26T06:31:23.3975250Z deleted: sha256:49e765a674b1865297af41869bdd809828c64b944b65469a14900b74c98943e6 2024-06-26T06:31:23.3976075Z deleted: sha256:53d422d72602186278f9d69dfb36ea243d49d1ef308d5aa6969b5c5ae2b3d5ec 2024-06-26T06:31:23.3976924Z deleted: sha256:b230252ce03a3e8af90d627f05b25a28e766298328ba5741b029cedd06b87eda 2024-06-26T06:31:23.3977767Z deleted: sha256:e02478373f52c109b749357f927fce863a248bbde8fa1b580933dfa336ec42e6 2024-06-26T06:31:23.3978618Z deleted: sha256:d5f1cfecd483d58b4d099db4f5b8d1aa64ac1b63b92e33b279549f55a8c427d7 2024-06-26T06:31:23.3979486Z deleted: sha256:9d7b0f79146aaf690751949446c7898f23d95ddaa419ffe66bbbe600c5e398fd 2024-06-26T06:31:23.3980326Z deleted: sha256:133da26422b32811689cf2cf0957386562d3cb5535adc94b5e57c8df987e043e 2024-06-26T06:31:23.3981170Z deleted: sha256:f2c71578e29cb9026fe9befb3e22645227b75f80f2a23a33d10b2db23ed2ffec 2024-06-26T06:31:23.3982021Z deleted: sha256:1b6f7c8ed5076766bbd0b62cf3828933328dd04dd3ac7f67c893e0ff97fae6a4 2024-06-26T06:31:23.3982891Z deleted: sha256:9597de82a03b18ac50bd0e6cb67679b2a2e7474ab6f2d3e94e0cd791debb59f1 2024-06-26T06:31:23.3983879Z deleted: sha256:3025c8170cf78d17d90b5bd8f433b41bd56c1d36f3e4f61c525189b8a4e63ae9 2024-06-26T06:31:23.3984730Z deleted: sha256:da61c466d61ad41a696250119e7b15ce0affe6f7c05938edd7878134807fdc80 2024-06-26T06:31:23.3985598Z deleted: sha256:f9d649f7c74f30382fe8c4934c19523b530e703abc60dbed49cad3fb17439166 2024-06-26T06:31:23.3986477Z deleted: sha256:ad91cb5ba8ab9a60d6271cec45b17cf019a826de82f473e9f1d1c02be0bc87d7 2024-06-26T06:31:23.3987364Z deleted: sha256:66a3cccbab7f9aa795dd6525c9238f778ae8fd7cfee31ffaa5fa334c6930214e 2024-06-26T06:31:23.3988251Z deleted: sha256:d5b66fb279a5b5596d9677908d317ef8f9c930cd1678f1ed7bfec0bf3d7a2c7f 2024-06-26T06:31:23.3989117Z deleted: sha256:de19e642015fb5277f1c9618adb9e0ae59cb73f3167d6cd1cf85977e430c1c8a 2024-06-26T06:31:23.3989996Z deleted: sha256:23ebfc51638f59d3cf2afcdebe5ad50a1a3325f9a25644c6f44b4ca9cf63cb56 2024-06-26T06:31:23.3990854Z deleted: sha256:c66c35c24cb2016e5bdbd103a7d88d169e0912c20082f6026bf28572302bca2a 2024-06-26T06:31:23.3991880Z deleted: sha256:9df358fe90e28361509b9ccf28e28e988a01f3f012f1b8308b0436e63113e00e 2024-06-26T06:31:23.3992746Z deleted: sha256:77df668148b1ee21bda8c56873f9e699b450760d826ec7ba4ee5a3decffa4c4c 2024-06-26T06:31:23.3993592Z deleted: sha256:20c85858580ea8c74bb79583a0fa268920ab56eb74e2d3f8a16c99d04d993dbd 2024-06-26T06:31:23.3994461Z deleted: sha256:c29b6190a2dc284ffeb328c07d4cc6c08a62aa5bc906a8af2d3f4e9e7d616101 2024-06-26T06:31:23.3995327Z deleted: sha256:354a706e4175a494886b3fc00497a52e61e0b9182e65faafb81ca602fc50b9f8 2024-06-26T06:31:23.3996164Z deleted: sha256:b475fa4d78c647583c017557e0d52b89b02263bd0cbdc4e55513511e0389af5d 2024-06-26T06:31:23.3997091Z deleted: sha256:df429193461053897fd469ddc8575e0d22fd27195147467cd22a8de72b252572 2024-06-26T06:31:23.3997938Z deleted: sha256:546680950a9ec09b3d1fff5bb37f8f5a7be287361564edd1ab44b691a07170c0 2024-06-26T06:31:23.3998827Z deleted: sha256:30e429e6fa1090247d58a8d8fcf7c10ed315d655c2882862d371cb1233f5fbe5 2024-06-26T06:31:23.3999675Z deleted: sha256:879a64e7fbc80c207de24373169c42cd4600fbd269614eb0631b70a56ac31c85 2024-06-26T06:31:23.4000526Z deleted: sha256:1f56afc7e6f4e91762a39d84b1ac902311d44683310fed82cfda29946d3b4f25 2024-06-26T06:31:23.4001361Z deleted: sha256:36c2783a09c6f553169a309d5191749c61296036f773a179e627284eab60f566 2024-06-26T06:31:23.4002184Z deleted: sha256:91ca7f1473e6c57eb999257df6da385e3c8a232d4f80b3e6c41ae986a14f08ea 2024-06-26T06:31:23.4003032Z deleted: sha256:6251f30f7423f7450abe933294854c29f56aeb24bcbdab4c6a06865e26e891ce 2024-06-26T06:31:23.4003881Z deleted: sha256:772d8b14c387cf9f67eb73651e69b3684aea6ac72bbfb8631dc2236831f532d9 2024-06-26T06:31:23.4004847Z deleted: sha256:b8c522eb18282cc42a40578f402f1714ac062f5ce58a3732c5a0b6609e371260 2024-06-26T06:31:23.4005707Z deleted: sha256:88fd863ad524e67157a3707a0e9a59ced4facd9994ccb4989f82b2b0c23fbadd 2024-06-26T06:31:23.4006558Z deleted: sha256:46b008f910d099393c32e0861ef497123fe9e38cc7858e6a78ff14d8ccf2433c 2024-06-26T06:31:23.4007398Z deleted: sha256:a87e255b7d2582715a3500969c08d892a7ff7bc89f8f26ae3264f1d85f8d50e7 2024-06-26T06:31:23.4008242Z deleted: sha256:e05af56d19ba382a9328ae93507d49fb99d75f392ed1b99236cfc53f407cd14e 2024-06-26T06:31:23.4009121Z deleted: sha256:621f607deeddafca7a75985f6fe803edddbd213ab6e757ecb93defd50850b3e0 2024-06-26T06:31:23.4010012Z deleted: sha256:fa4053ca3b4a7c3fc53056830ba5d1a41c6e9a2da86e90677c1de6cd8636699c 2024-06-26T06:31:23.4010860Z deleted: sha256:d955717953fc80b77138de68c47f0f7f417576fd714ce7ebbc2cb917df7f6268 2024-06-26T06:31:23.4011732Z deleted: sha256:f9cf157ccc61ef3b6fdfc8f28e3dffb58f608b21828ec1a75642a0d9badb4d74 2024-06-26T06:31:23.4012602Z deleted: sha256:9261bd664cae980bd33b489dd2229fcd22847263f763b43832c1a015128cde41 2024-06-26T06:31:23.4013460Z deleted: sha256:e8cbc6c23a323bd7a81fbb2abc078c62657cf6e4bbc024ce4f063c77dfa50f58 2024-06-26T06:31:23.4014348Z deleted: sha256:fc8041ead5e87fe711b5f8dcc3fcf3fb4e4378c1418b75cf248ec6ad01741146 2024-06-26T06:31:23.4015239Z deleted: sha256:c329a10c164aabb156ea9b5b7dd8ee139b33adcd80d02bddeb3b3ea0f4616840 2024-06-26T06:31:23.4016223Z deleted: sha256:08d22634590731d16719d71fa2d6119d947039f3a0ddba37975efcc18ddb1507 2024-06-26T06:31:23.4017073Z deleted: sha256:f52ec5d2b34fa58954c6dcfd982e9105eada4bba8d778e92e9a5c9e35974d195 2024-06-26T06:31:23.4017965Z deleted: sha256:7ebf40b5d1e427bdc6bc5ba2f4ba8006c0751f355c0a9c490f30dce5e99eeb84 2024-06-26T06:31:23.4018836Z deleted: sha256:99d7f78132e9c56c11631e93df699c00b711b24dac63bd7647482aa09bbb2499 2024-06-26T06:31:23.4019687Z deleted: sha256:e1c9e2fcf63c9e2d26bcadb74d498117b52d26b81430bb4bd8c96fb10b56ce20 2024-06-26T06:31:23.4020577Z deleted: sha256:afeb82ca9283ee35825ff0076df5b812574ec06cb7be63890358fa5763711749 2024-06-26T06:31:23.4021446Z deleted: sha256:b74cfe809a40a9e9adabaca01482497767195c2918e8987122c5774242e36bed 2024-06-26T06:31:23.4022305Z deleted: sha256:ab9798e1fd9865efbbe7dd0681d45687b38a292b7bdbc1d69c625999508e28fe 2024-06-26T06:31:23.4023170Z deleted: sha256:0b9c994b0484c0bc61f9de7c28a58745a504704254c5e8ed12349ebee3393a66 2024-06-26T06:31:23.4023785Z 2024-06-26T06:31:23.4023935Z Total reclaimed space: 15.04GB 2024-06-26T06:31:23.4108797Z Post job cleanup. 2024-06-26T06:31:23.4179036Z Post job cleanup. 2024-06-26T06:31:23.5081558Z [command]/usr/bin/git version 2024-06-26T06:31:23.5338483Z git version 2.40.1 2024-06-26T06:31:23.5376563Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/8251b6e7-f189-4c29-b068-2ec8592b718c' before making global git config changes 2024-06-26T06:31:23.5377869Z Adding repository directory to the temporary git global config as a safe directory 2024-06-26T06:31:23.5381328Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-06-26T06:31:23.5424980Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-06-26T06:31:23.5453550Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-06-26T06:31:23.5862727Z Entering 'android/libs/fbjni' 2024-06-26T06:31:23.5915276Z Entering 'third_party/FP16' 2024-06-26T06:31:23.5960383Z Entering 'third_party/FXdiv' 2024-06-26T06:31:23.6013412Z Entering 'third_party/NNPACK' 2024-06-26T06:31:23.6059475Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T06:31:23.6100603Z Entering 'third_party/XNNPACK' 2024-06-26T06:31:23.6227684Z Entering 'third_party/benchmark' 2024-06-26T06:31:23.6273469Z Entering 'third_party/cpp-httplib' 2024-06-26T06:31:23.6314406Z Entering 'third_party/cpuinfo' 2024-06-26T06:31:23.6361731Z Entering 'third_party/cudnn_frontend' 2024-06-26T06:31:23.6403162Z Entering 'third_party/cutlass' 2024-06-26T06:31:23.6452776Z Entering 'third_party/eigen' 2024-06-26T06:31:23.6512853Z Entering 'third_party/fbgemm' 2024-06-26T06:31:23.6573639Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T06:31:23.6634852Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T06:31:23.6700341Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T06:31:23.6798296Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T06:31:23.6866894Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T06:31:23.6916539Z Entering 'third_party/flatbuffers' 2024-06-26T06:31:23.6960828Z Entering 'third_party/fmt' 2024-06-26T06:31:23.7006692Z Entering 'third_party/foxi' 2024-06-26T06:31:23.7052661Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T06:31:23.7098017Z Entering 'third_party/gloo' 2024-06-26T06:31:23.7143667Z Entering 'third_party/googletest' 2024-06-26T06:31:23.7190231Z Entering 'third_party/ideep' 2024-06-26T06:31:23.7257816Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T06:31:23.7352102Z Entering 'third_party/ittapi' 2024-06-26T06:31:23.7392833Z Entering 'third_party/kineto' 2024-06-26T06:31:23.7433929Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T06:31:23.7472712Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T06:31:23.7514002Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T06:31:23.7553714Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T06:31:23.7593577Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T06:31:23.7632568Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T06:31:23.7673124Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T06:31:23.7712983Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T06:31:23.7752932Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T06:31:23.7794158Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T06:31:23.7835517Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T06:31:23.7876175Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T06:31:23.7916731Z Entering 'third_party/mimalloc' 2024-06-26T06:31:23.7958834Z Entering 'third_party/nccl/nccl' 2024-06-26T06:31:23.8009412Z Entering 'third_party/nlohmann' 2024-06-26T06:31:23.8052125Z Entering 'third_party/onnx' 2024-06-26T06:31:23.8187178Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T06:31:23.8242584Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T06:31:23.8300093Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T06:31:23.8342395Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T06:31:23.8383083Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T06:31:23.8423321Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T06:31:23.8462881Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T06:31:23.8504074Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T06:31:23.8543369Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T06:31:23.8583152Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T06:31:23.8624785Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T06:31:23.8666655Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T06:31:23.8707927Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T06:31:23.8768326Z Entering 'third_party/pocketfft' 2024-06-26T06:31:23.8808919Z Entering 'third_party/protobuf' 2024-06-26T06:31:23.8882941Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T06:31:23.8945951Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T06:31:23.8997442Z Entering 'third_party/psimd' 2024-06-26T06:31:23.9040922Z Entering 'third_party/pthreadpool' 2024-06-26T06:31:23.9085609Z Entering 'third_party/pybind11' 2024-06-26T06:31:23.9135780Z Entering 'third_party/python-peachpy' 2024-06-26T06:31:23.9180672Z Entering 'third_party/sleef' 2024-06-26T06:31:23.9227381Z Entering 'third_party/tensorpipe' 2024-06-26T06:31:23.9287825Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T06:31:23.9355540Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T06:31:23.9411889Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T06:31:23.9487889Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T06:31:23.9545463Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T06:31:23.9620867Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-06-26T06:31:23.9644585Z http.https://github.com/.extraheader 2024-06-26T06:31:23.9652803Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-06-26T06:31:23.9681967Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-06-26T06:31:23.9929335Z Entering 'android/libs/fbjni' 2024-06-26T06:31:23.9955545Z http.https://github.com/.extraheader 2024-06-26T06:31:23.9982962Z Entering 'third_party/FP16' 2024-06-26T06:31:24.0010525Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0037574Z Entering 'third_party/FXdiv' 2024-06-26T06:31:24.0064236Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0091694Z Entering 'third_party/NNPACK' 2024-06-26T06:31:24.0118284Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0145559Z Entering 'third_party/VulkanMemoryAllocator' 2024-06-26T06:31:24.0172357Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0199156Z Entering 'third_party/XNNPACK' 2024-06-26T06:31:24.0225710Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0269413Z Entering 'third_party/benchmark' 2024-06-26T06:31:24.0296735Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0323469Z Entering 'third_party/cpp-httplib' 2024-06-26T06:31:24.0350122Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0376728Z Entering 'third_party/cpuinfo' 2024-06-26T06:31:24.0402840Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0429895Z Entering 'third_party/cudnn_frontend' 2024-06-26T06:31:24.0456434Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0482073Z Entering 'third_party/cutlass' 2024-06-26T06:31:24.0508095Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0543517Z Entering 'third_party/eigen' 2024-06-26T06:31:24.0570619Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0599661Z Entering 'third_party/fbgemm' 2024-06-26T06:31:24.0625964Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0652667Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-06-26T06:31:24.0678762Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0706377Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-06-26T06:31:24.0732067Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0758587Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-06-26T06:31:24.0784086Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0817279Z Entering 'third_party/fbgemm/third_party/googletest' 2024-06-26T06:31:24.0842177Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0868756Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-06-26T06:31:24.0894900Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0922739Z Entering 'third_party/flatbuffers' 2024-06-26T06:31:24.0949090Z http.https://github.com/.extraheader 2024-06-26T06:31:24.0978734Z Entering 'third_party/fmt' 2024-06-26T06:31:24.1004750Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1031735Z Entering 'third_party/foxi' 2024-06-26T06:31:24.1058483Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1084621Z Entering 'third_party/gemmlowp/gemmlowp' 2024-06-26T06:31:24.1111730Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1137849Z Entering 'third_party/gloo' 2024-06-26T06:31:24.1163730Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1190546Z Entering 'third_party/googletest' 2024-06-26T06:31:24.1217375Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1245296Z Entering 'third_party/ideep' 2024-06-26T06:31:24.1271057Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1297502Z Entering 'third_party/ideep/mkl-dnn' 2024-06-26T06:31:24.1323303Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1358553Z Entering 'third_party/ittapi' 2024-06-26T06:31:24.1385165Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1411614Z Entering 'third_party/kineto' 2024-06-26T06:31:24.1438536Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1464296Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-06-26T06:31:24.1489718Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1515977Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-06-26T06:31:24.1541936Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1570282Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-06-26T06:31:24.1596474Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1623379Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-06-26T06:31:24.1649659Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1676604Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-06-26T06:31:24.1702606Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1729619Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-06-26T06:31:24.1754449Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1783083Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-06-26T06:31:24.1808590Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1835307Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-06-26T06:31:24.1860693Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1887348Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-06-26T06:31:24.1912918Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1941646Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-06-26T06:31:24.1967305Z http.https://github.com/.extraheader 2024-06-26T06:31:24.1995186Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-06-26T06:31:24.2020567Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2048104Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-06-26T06:31:24.2073571Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2101493Z Entering 'third_party/mimalloc' 2024-06-26T06:31:24.2127759Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2154558Z Entering 'third_party/nccl/nccl' 2024-06-26T06:31:24.2180566Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2208696Z Entering 'third_party/nlohmann' 2024-06-26T06:31:24.2234332Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2261498Z Entering 'third_party/onnx' 2024-06-26T06:31:24.2288613Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2330772Z Entering 'third_party/onnx/third_party/benchmark' 2024-06-26T06:31:24.2356510Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2383108Z Entering 'third_party/onnx/third_party/pybind11' 2024-06-26T06:31:24.2409349Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2437562Z Entering 'third_party/opentelemetry-cpp' 2024-06-26T06:31:24.2464078Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2492393Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-06-26T06:31:24.2517566Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2544404Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-06-26T06:31:24.2570549Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2596210Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-06-26T06:31:24.2622247Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2648522Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-06-26T06:31:24.2673729Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2701146Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-06-26T06:31:24.2727005Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2753499Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-06-26T06:31:24.2778860Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2805183Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-06-26T06:31:24.2830348Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2856435Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-06-26T06:31:24.2880956Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2908721Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-06-26T06:31:24.2933489Z http.https://github.com/.extraheader 2024-06-26T06:31:24.2961031Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-06-26T06:31:24.2987215Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3032017Z Entering 'third_party/pocketfft' 2024-06-26T06:31:24.3058726Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3086500Z Entering 'third_party/protobuf' 2024-06-26T06:31:24.3111976Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3141322Z Entering 'third_party/protobuf/third_party/benchmark' 2024-06-26T06:31:24.3167866Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3193763Z Entering 'third_party/protobuf/third_party/googletest' 2024-06-26T06:31:24.3220156Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3248214Z Entering 'third_party/psimd' 2024-06-26T06:31:24.3274367Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3300893Z Entering 'third_party/pthreadpool' 2024-06-26T06:31:24.3328494Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3355067Z Entering 'third_party/pybind11' 2024-06-26T06:31:24.3382372Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3409307Z Entering 'third_party/python-peachpy' 2024-06-26T06:31:24.3435660Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3462916Z Entering 'third_party/sleef' 2024-06-26T06:31:24.3489151Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3516515Z Entering 'third_party/tensorpipe' 2024-06-26T06:31:24.3542995Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3571860Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-06-26T06:31:24.3599161Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3626341Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-06-26T06:31:24.3653025Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3679228Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-06-26T06:31:24.3705102Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3732721Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-06-26T06:31:24.3758206Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3785602Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-06-26T06:31:24.3812182Z http.https://github.com/.extraheader 2024-06-26T06:31:24.3916220Z A job completed hook has been configured by the self-hosted runner administrator 2024-06-26T06:31:24.3935923Z ##[group]Run '/home/ec2-user/runner-scripts/cleanup.sh' 2024-06-26T06:31:24.3942810Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-06-26T06:31:24.3943315Z ##[endgroup] 2024-06-26T06:31:25.7781501Z Cleaning up orphan processes