2024-08-07T17:31:31.5434668Z Current runner version: '2.318.0' 2024-08-07T17:31:31.5442998Z Runner name: 'i-0efccfdf668c75c22' 2024-08-07T17:31:31.5443934Z Runner group name: 'Default' 2024-08-07T17:31:31.5445034Z Machine name: 'ip-10-0-77-114' 2024-08-07T17:31:31.5449667Z ##[group]GITHUB_TOKEN Permissions 2024-08-07T17:31:31.5452688Z Actions: read 2024-08-07T17:31:31.5453413Z Attestations: read 2024-08-07T17:31:31.5454130Z Checks: read 2024-08-07T17:31:31.5454697Z Contents: read 2024-08-07T17:31:31.5455308Z Deployments: read 2024-08-07T17:31:31.5456017Z Discussions: read 2024-08-07T17:31:31.5456594Z Issues: read 2024-08-07T17:31:31.5457191Z Metadata: read 2024-08-07T17:31:31.5457893Z Packages: read 2024-08-07T17:31:31.5458476Z Pages: read 2024-08-07T17:31:31.5459070Z PullRequests: read 2024-08-07T17:31:31.5459801Z RepositoryProjects: read 2024-08-07T17:31:31.5460453Z SecurityEvents: read 2024-08-07T17:31:31.5461143Z Statuses: read 2024-08-07T17:31:31.5461817Z ##[endgroup] 2024-08-07T17:31:31.5465317Z Secret source: Actions 2024-08-07T17:31:31.5466405Z Prepare workflow directory 2024-08-07T17:31:31.8571152Z Prepare all required actions 2024-08-07T17:31:31.8616288Z Getting action download info 2024-08-07T17:31:32.0190543Z Download action repository 'pytorch/test-infra@main' (SHA:a1f5a89251fc4258ab59806094fe3108f7d6741a) 2024-08-07T17:31:33.4802784Z Download action repository 'pytorch/pytorch@main' (SHA:32a284c27549449c19e3b1455f531692439d6316) 2024-08-07T17:31:44.1447372Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0) 2024-08-07T17:31:44.3724775Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a) 2024-08-07T17:31:44.6454990Z Getting action download info 2024-08-07T17:31:44.7321575Z Download action repository 'malfet/checkout@silent-checkout' (SHA:e07af140b3ccefc05679e3755b9db68f4ee4589c) 2024-08-07T17:31:44.9856427Z Getting action download info 2024-08-07T17:31:45.0806575Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482) 2024-08-07T17:31:45.2423970Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/pull/131248/merge (f779f6b7738020e244184bded4026b37de3f9f24) 2024-08-07T17:31:45.2426117Z ##[group] Inputs 2024-08-07T17:31:45.2426492Z build-environment: linux-jammy-py3.8-gcc11 2024-08-07T17:31:45.2429369Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]} 2024-08-07T17:31:45.2432592Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:31:45.2433349Z sync-tag: 2024-08-07T17:31:45.2434185Z timeout-minutes: 240 2024-08-07T17:31:45.2434474Z use-gha: 2024-08-07T17:31:45.2434729Z dashboard-tag: 2024-08-07T17:31:45.2434977Z s3-bucket: gha-artifacts 2024-08-07T17:31:45.2435269Z aws-role-to-assume: 2024-08-07T17:31:45.2435550Z ##[endgroup] 2024-08-07T17:31:45.2435980Z Complete job name: linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-07T17:31:45.2879864Z A job started hook has been configured by the self-hosted runner administrator 2024-08-07T17:31:45.2986921Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh' 2024-08-07T17:31:45.2996696Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:31:45.2997404Z ##[endgroup] 2024-08-07T17:31:46.9867739Z Runner Type: amz2023.linux.2xlarge 2024-08-07T17:31:46.9868303Z Instance Type: c5.2xlarge 2024-08-07T17:31:46.9868647Z AMI Name: al2023-ami-2023.5.20240701.0-kernel-6.1-x86_64 2024-08-07T17:31:46.9869048Z AMI ID: ami-06c68f701d8090592 2024-08-07T17:31:52.9891350Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main 2024-08-07T17:31:52.9891839Z with: 2024-08-07T17:31:52.9892543Z github-secret: *** 2024-08-07T17:31:52.9893285Z instructions: All testing is done inside the container, to start an interactive session run: docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-07T17:31:52.9894086Z activate-with-label: false 2024-08-07T17:31:52.9894379Z label: with-ssh 2024-08-07T17:31:52.9894636Z remove-existing-keys: true 2024-08-07T17:31:52.9894930Z fail-silently: true 2024-08-07T17:31:52.9895168Z env: 2024-08-07T17:31:52.9895450Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:31:52.9895731Z ##[endgroup] 2024-08-07T17:31:53.0753618Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info. 2024-08-07T17:31:53.3747133Z Grabbing public ssh keys from https://github.com/zdevito.keys 2024-08-07T17:31:53.4548865Z ~/.ssh/authorized_keys file found on node, removing ~/.ssh and starting fresh 2024-08-07T17:31:53.4562014Z Public keys pulled and installed to /home/ec2-user/.ssh/authorized_keys 2024-08-07T17:31:53.4588716Z Login using: ssh ec2-user@ec2-100-28-128-78.compute-1.amazonaws.com 2024-08-07T17:31:53.4589350Z All testing is done inside the container, to start an interactive session run: 2024-08-07T17:31:53.4589964Z docker exec -it $(docker container ps --format '{{.ID}}') bash 2024-08-07T17:31:53.4707179Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main 2024-08-07T17:31:53.4707640Z with: 2024-08-07T17:31:53.4707881Z submodules: recursive 2024-08-07T17:31:53.4708157Z fetch-depth: 0 2024-08-07T17:31:53.4708408Z env: 2024-08-07T17:31:53.4708638Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:31:53.4708899Z ##[endgroup] 2024-08-07T17:31:53.4791578Z ##[group]Run retry () { 2024-08-07T17:31:53.4791934Z retry () { 2024-08-07T17:31:53.4792284Z  $* || (sleep 1 && $*) || (sleep 2 && $*) || (sleep 4 && $*) || (sleep 8 && $*) 2024-08-07T17:31:53.4792698Z } 2024-08-07T17:31:53.4792955Z echo "${GITHUB_WORKSPACE}" 2024-08-07T17:31:53.4793271Z if [ -z "${NO_SUDO}" ]; then 2024-08-07T17:31:53.4793630Z  retry sudo rm -rf "${GITHUB_WORKSPACE}" 2024-08-07T17:31:53.4793978Z else 2024-08-07T17:31:53.4794233Z  retry rm -rf "${GITHUB_WORKSPACE}" 2024-08-07T17:31:53.4794563Z fi 2024-08-07T17:31:53.4794867Z mkdir "${GITHUB_WORKSPACE}" 2024-08-07T17:31:53.4803272Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:31:53.4803675Z env: 2024-08-07T17:31:53.4803905Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:31:53.4804187Z NO_SUDO: 2024-08-07T17:31:53.4804412Z ##[endgroup] 2024-08-07T17:31:53.4831112Z /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:31:55.7555794Z ##[group]Run malfet/checkout@silent-checkout 2024-08-07T17:31:55.7556154Z with: 2024-08-07T17:31:55.7556412Z ref: 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:31:55.7556739Z fetch-depth: 0 2024-08-07T17:31:55.7556994Z submodules: recursive 2024-08-07T17:31:55.7557263Z quiet-checkout: true 2024-08-07T17:31:55.7557527Z repository: pytorch/pytorch 2024-08-07T17:31:55.7557940Z token: *** 2024-08-07T17:31:55.7558173Z ssh-strict: true 2024-08-07T17:31:55.7558418Z persist-credentials: true 2024-08-07T17:31:55.7558792Z clean: true 2024-08-07T17:31:55.7559068Z sparse-checkout-cone-mode: true 2024-08-07T17:31:55.7559357Z lfs: false 2024-08-07T17:31:55.7559598Z set-safe-directory: true 2024-08-07T17:31:55.7559866Z env: 2024-08-07T17:31:55.7560073Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:31:55.7560573Z ##[endgroup] 2024-08-07T17:31:55.8548473Z Syncing repository: pytorch/pytorch 2024-08-07T17:31:55.8550723Z ##[group]Getting Git version info 2024-08-07T17:31:55.8551558Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-07T17:31:55.8552706Z [command]/usr/bin/git version 2024-08-07T17:31:55.8553207Z git version 2.40.1 2024-08-07T17:31:55.8568992Z ##[endgroup] 2024-08-07T17:31:55.8585556Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/18c8de1b-a337-477b-99be-361004e76f56' before making global git config changes 2024-08-07T17:31:55.8587301Z Adding repository directory to the temporary git global config as a safe directory 2024-08-07T17:31:55.8591857Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:31:55.8633825Z Deleting the contents of '/home/ec2-user/actions-runner/_work/pytorch/pytorch' 2024-08-07T17:31:55.8638535Z ##[group]Initializing the repository 2024-08-07T17:31:55.8642261Z [command]/usr/bin/git init /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:31:55.8673495Z hint: Using 'master' as the name for the initial branch. This default branch name 2024-08-07T17:31:55.8674215Z hint: is subject to change. To configure the initial branch name to use in all 2024-08-07T17:31:55.8674806Z hint: of your new repositories, which will suppress this warning, call: 2024-08-07T17:31:55.8675227Z hint: 2024-08-07T17:31:55.8675545Z hint: git config --global init.defaultBranch 2024-08-07T17:31:55.8675920Z hint: 2024-08-07T17:31:55.8676418Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-08-07T17:31:55.8677434Z hint: 'development'. The just-created branch can be renamed via this command: 2024-08-07T17:31:55.8678515Z hint: 2024-08-07T17:31:55.8678994Z hint: git branch -m 2024-08-07T17:31:55.8679909Z Initialized empty Git repository in /home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/ 2024-08-07T17:31:55.8686203Z [command]/usr/bin/git remote add origin https://github.com/pytorch/pytorch 2024-08-07T17:31:55.8718033Z ##[endgroup] 2024-08-07T17:31:55.8718843Z ##[group]Disabling automatic garbage collection 2024-08-07T17:31:55.8722343Z [command]/usr/bin/git config --local gc.auto 0 2024-08-07T17:31:55.8752126Z ##[endgroup] 2024-08-07T17:31:55.8752818Z ##[group]Setting up auth 2024-08-07T17:31:55.8759195Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-07T17:31:55.8792100Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-07T17:31:55.9052426Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-07T17:31:55.9086270Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-07T17:31:55.9346884Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-07T17:31:55.9395092Z ##[endgroup] 2024-08-07T17:31:55.9395883Z ##[group]Fetching the repository 2024-08-07T17:31:55.9402523Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --progress --no-recurse-submodules --quiet origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/* 2024-08-07T17:31:58.3222249Z remote: Enumerating objects: 1008194 2024-08-07T17:31:58.3222955Z remote: Enumerating objects: 1011031, done. 2024-08-07T17:31:58.3225850Z remote: Counting objects: 0% (1/2837) 2024-08-07T17:31:58.3228412Z remote: Counting objects: 1% (29/2837) 2024-08-07T17:31:58.3228915Z remote: Counting objects: 2% (57/2837) 2024-08-07T17:31:58.3229305Z remote: Counting objects: 3% (86/2837) 2024-08-07T17:31:58.3230963Z remote: Counting objects: 4% (114/2837) 2024-08-07T17:31:58.3231670Z remote: Counting objects: 5% (142/2837) 2024-08-07T17:31:58.3233493Z remote: Counting objects: 6% (171/2837) 2024-08-07T17:31:58.3233900Z remote: Counting objects: 7% (199/2837) 2024-08-07T17:31:58.3234273Z remote: Counting objects: 8% (227/2837) 2024-08-07T17:31:58.3236352Z remote: Counting objects: 9% (256/2837) 2024-08-07T17:31:58.3237012Z remote: Counting objects: 10% (284/2837) 2024-08-07T17:31:58.3237616Z remote: Counting objects: 11% (313/2837) 2024-08-07T17:31:58.3238072Z remote: Counting objects: 12% (341/2837) 2024-08-07T17:31:58.3238670Z remote: Counting objects: 13% (369/2837) 2024-08-07T17:31:58.3239264Z remote: Counting objects: 14% (398/2837) 2024-08-07T17:31:58.3240072Z remote: Counting objects: 15% (426/2837) 2024-08-07T17:31:58.3240538Z remote: Counting objects: 16% (454/2837) 2024-08-07T17:31:58.3240978Z remote: Counting objects: 17% (483/2837) 2024-08-07T17:31:58.3241446Z remote: Counting objects: 18% (511/2837) 2024-08-07T17:31:58.3242138Z remote: Counting objects: 19% (540/2837) 2024-08-07T17:31:58.3242761Z remote: Counting objects: 20% (568/2837) 2024-08-07T17:31:58.3243161Z remote: Counting objects: 21% (596/2837) 2024-08-07T17:31:58.3243601Z remote: Counting objects: 22% (625/2837) 2024-08-07T17:31:58.3244237Z remote: Counting objects: 23% (653/2837) 2024-08-07T17:31:58.3244709Z remote: Counting objects: 24% (681/2837) 2024-08-07T17:31:58.3245096Z remote: Counting objects: 25% (710/2837) 2024-08-07T17:31:58.3245488Z remote: Counting objects: 26% (738/2837) 2024-08-07T17:31:58.3245945Z remote: Counting objects: 27% (766/2837) 2024-08-07T17:31:58.3246833Z remote: Counting objects: 28% (795/2837) 2024-08-07T17:31:58.3247539Z remote: Counting objects: 29% (823/2837) 2024-08-07T17:31:58.3247911Z remote: Counting objects: 30% (852/2837) 2024-08-07T17:31:58.3248483Z remote: Counting objects: 31% (880/2837) 2024-08-07T17:31:58.3249073Z remote: Counting objects: 32% (908/2837) 2024-08-07T17:31:58.3249662Z remote: Counting objects: 33% (937/2837) 2024-08-07T17:31:58.3250078Z remote: Counting objects: 34% (965/2837) 2024-08-07T17:31:58.3250468Z remote: Counting objects: 35% (993/2837) 2024-08-07T17:31:58.3250868Z remote: Counting objects: 36% (1022/2837) 2024-08-07T17:31:58.3251385Z remote: Counting objects: 37% (1050/2837) 2024-08-07T17:31:58.3252065Z remote: Counting objects: 38% (1079/2837) 2024-08-07T17:31:58.3252727Z remote: Counting objects: 39% (1107/2837) 2024-08-07T17:31:58.3253372Z remote: Counting objects: 40% (1135/2837) 2024-08-07T17:31:58.3253969Z remote: Counting objects: 41% (1164/2837) 2024-08-07T17:31:58.3254593Z remote: Counting objects: 42% (1192/2837) 2024-08-07T17:31:58.3255101Z remote: Counting objects: 43% (1220/2837) 2024-08-07T17:31:58.3255505Z remote: Counting objects: 44% (1249/2837) 2024-08-07T17:31:58.3255894Z remote: Counting objects: 45% (1277/2837) 2024-08-07T17:31:58.3256344Z remote: Counting objects: 46% (1306/2837) 2024-08-07T17:31:58.3256959Z remote: Counting objects: 47% (1334/2837) 2024-08-07T17:31:58.3257606Z remote: Counting objects: 48% (1362/2837) 2024-08-07T17:31:58.3257995Z remote: Counting objects: 49% (1391/2837) 2024-08-07T17:31:58.3258381Z remote: Counting objects: 50% (1419/2837) 2024-08-07T17:31:58.3258767Z remote: Counting objects: 51% (1447/2837) 2024-08-07T17:31:58.3259159Z remote: Counting objects: 52% (1476/2837) 2024-08-07T17:31:58.3259542Z remote: Counting objects: 53% (1504/2837) 2024-08-07T17:31:58.3259928Z remote: Counting objects: 54% (1532/2837) 2024-08-07T17:31:58.3260356Z remote: Counting objects: 55% (1561/2837) 2024-08-07T17:31:58.3260901Z remote: Counting objects: 56% (1589/2837) 2024-08-07T17:31:58.3261287Z remote: Counting objects: 57% (1618/2837) 2024-08-07T17:31:58.3261680Z remote: Counting objects: 58% (1646/2837) 2024-08-07T17:31:58.3262052Z remote: Counting objects: 59% (1674/2837) 2024-08-07T17:31:58.3262441Z remote: Counting objects: 60% (1703/2837) 2024-08-07T17:31:58.3262902Z remote: Counting objects: 61% (1731/2837) 2024-08-07T17:31:58.3263405Z remote: Counting objects: 62% (1759/2837) 2024-08-07T17:31:58.3264023Z remote: Counting objects: 63% (1788/2837) 2024-08-07T17:31:58.3264506Z remote: Counting objects: 64% (1816/2837) 2024-08-07T17:31:58.3264886Z remote: Counting objects: 65% (1845/2837) 2024-08-07T17:31:58.3265288Z remote: Counting objects: 66% (1873/2837) 2024-08-07T17:31:58.3265772Z remote: Counting objects: 67% (1901/2837) 2024-08-07T17:31:58.3266386Z remote: Counting objects: 68% (1930/2837) 2024-08-07T17:31:58.3266962Z remote: Counting objects: 69% (1958/2837) 2024-08-07T17:31:58.3267352Z remote: Counting objects: 70% (1986/2837) 2024-08-07T17:31:58.3267723Z remote: Counting objects: 71% (2015/2837) 2024-08-07T17:31:58.3268112Z remote: Counting objects: 72% (2043/2837) 2024-08-07T17:31:58.3268754Z remote: Counting objects: 73% (2072/2837) 2024-08-07T17:31:58.3269207Z remote: Counting objects: 74% (2100/2837) 2024-08-07T17:31:58.3269575Z remote: Counting objects: 75% (2128/2837) 2024-08-07T17:31:58.3269960Z remote: Counting objects: 76% (2157/2837) 2024-08-07T17:31:58.3270346Z remote: Counting objects: 77% (2185/2837) 2024-08-07T17:31:58.3270716Z remote: Counting objects: 78% (2213/2837) 2024-08-07T17:31:58.3271222Z remote: Counting objects: 79% (2242/2837) 2024-08-07T17:31:58.3271608Z remote: Counting objects: 80% (2270/2837) 2024-08-07T17:31:58.3271975Z remote: Counting objects: 81% (2298/2837) 2024-08-07T17:31:58.3272364Z remote: Counting objects: 82% (2327/2837) 2024-08-07T17:31:58.3272746Z remote: Counting objects: 83% (2355/2837) 2024-08-07T17:31:58.3273114Z remote: Counting objects: 84% (2384/2837) 2024-08-07T17:31:58.3273499Z remote: Counting objects: 85% (2412/2837) 2024-08-07T17:31:58.3273882Z remote: Counting objects: 86% (2440/2837) 2024-08-07T17:31:58.3274252Z remote: Counting objects: 87% (2469/2837) 2024-08-07T17:31:58.3274635Z remote: Counting objects: 88% (2497/2837) 2024-08-07T17:31:58.3275021Z remote: Counting objects: 89% (2525/2837) 2024-08-07T17:31:58.3275392Z remote: Counting objects: 90% (2554/2837) 2024-08-07T17:31:58.3275778Z remote: Counting objects: 91% (2582/2837) 2024-08-07T17:31:58.3276178Z remote: Counting objects: 92% (2611/2837) 2024-08-07T17:31:58.3276545Z remote: Counting objects: 93% (2639/2837) 2024-08-07T17:31:58.3276929Z remote: Counting objects: 94% (2667/2837) 2024-08-07T17:31:58.3277350Z remote: Counting objects: 95% (2696/2837) 2024-08-07T17:31:58.3277733Z remote: Counting objects: 96% (2724/2837) 2024-08-07T17:31:58.3278102Z remote: Counting objects: 97% (2752/2837) 2024-08-07T17:31:58.3278483Z remote: Counting objects: 98% (2781/2837) 2024-08-07T17:31:58.3278959Z remote: Counting objects: 99% (2809/2837) 2024-08-07T17:31:58.3279331Z remote: Counting objects: 100% (2837/2837) 2024-08-07T17:31:58.3279753Z remote: Counting objects: 100% (2837/2837), done. 2024-08-07T17:31:58.3862691Z remote: Compressing objects: 0% (1/1487) 2024-08-07T17:31:58.4137175Z remote: Compressing objects: 1% (15/1487) 2024-08-07T17:31:58.4675074Z remote: Compressing objects: 2% (30/1487) 2024-08-07T17:31:58.6477650Z remote: Compressing objects: 3% (45/1487) 2024-08-07T17:31:58.7603328Z remote: Compressing objects: 4% (60/1487) 2024-08-07T17:31:58.8627868Z remote: Compressing objects: 5% (75/1487) 2024-08-07T17:31:58.9367415Z remote: Compressing objects: 6% (90/1487) 2024-08-07T17:31:58.9873749Z remote: Compressing objects: 7% (105/1487) 2024-08-07T17:31:59.0337985Z remote: Compressing objects: 8% (119/1487) 2024-08-07T17:31:59.0555026Z remote: Compressing objects: 9% (134/1487) 2024-08-07T17:31:59.0799269Z remote: Compressing objects: 10% (149/1487) 2024-08-07T17:31:59.1039031Z remote: Compressing objects: 11% (164/1487) 2024-08-07T17:31:59.1248335Z remote: Compressing objects: 12% (179/1487) 2024-08-07T17:31:59.1451446Z remote: Compressing objects: 13% (194/1487) 2024-08-07T17:31:59.1602810Z remote: Compressing objects: 14% (209/1487) 2024-08-07T17:31:59.1718328Z remote: Compressing objects: 15% (224/1487) 2024-08-07T17:31:59.1802089Z remote: Compressing objects: 16% (238/1487) 2024-08-07T17:31:59.1877174Z remote: Compressing objects: 17% (253/1487) 2024-08-07T17:31:59.1930872Z remote: Compressing objects: 18% (268/1487) 2024-08-07T17:31:59.1963506Z remote: Compressing objects: 19% (283/1487) 2024-08-07T17:31:59.1984154Z remote: Compressing objects: 20% (298/1487) 2024-08-07T17:31:59.1993675Z remote: Compressing objects: 21% (313/1487) 2024-08-07T17:31:59.2010369Z remote: Compressing objects: 22% (328/1487) 2024-08-07T17:31:59.2024596Z remote: Compressing objects: 23% (343/1487) 2024-08-07T17:31:59.2048788Z remote: Compressing objects: 24% (357/1487) 2024-08-07T17:31:59.2062641Z remote: Compressing objects: 25% (372/1487) 2024-08-07T17:31:59.2079647Z remote: Compressing objects: 26% (387/1487) 2024-08-07T17:31:59.2104963Z remote: Compressing objects: 27% (402/1487) 2024-08-07T17:31:59.2117256Z remote: Compressing objects: 28% (417/1487) 2024-08-07T17:31:59.2134250Z remote: Compressing objects: 29% (432/1487) 2024-08-07T17:31:59.2155389Z remote: Compressing objects: 30% (447/1487) 2024-08-07T17:31:59.2170865Z remote: Compressing objects: 31% (461/1487) 2024-08-07T17:31:59.2185611Z remote: Compressing objects: 32% (476/1487) 2024-08-07T17:31:59.2229490Z remote: Compressing objects: 33% (491/1487) 2024-08-07T17:31:59.2230266Z remote: Compressing objects: 34% (506/1487) 2024-08-07T17:31:59.2230856Z remote: Compressing objects: 35% (521/1487) 2024-08-07T17:31:59.2231463Z remote: Compressing objects: 36% (536/1487) 2024-08-07T17:31:59.2243251Z remote: Compressing objects: 37% (551/1487) 2024-08-07T17:31:59.2254606Z remote: Compressing objects: 38% (566/1487) 2024-08-07T17:31:59.2262856Z remote: Compressing objects: 39% (580/1487) 2024-08-07T17:31:59.2273547Z remote: Compressing objects: 40% (595/1487) 2024-08-07T17:31:59.2279760Z remote: Compressing objects: 41% (610/1487) 2024-08-07T17:31:59.2285748Z remote: Compressing objects: 42% (625/1487) 2024-08-07T17:31:59.2293466Z remote: Compressing objects: 43% (640/1487) 2024-08-07T17:31:59.2298224Z remote: Compressing objects: 44% (655/1487) 2024-08-07T17:31:59.2305697Z remote: Compressing objects: 45% (670/1487) 2024-08-07T17:31:59.2310342Z remote: Compressing objects: 46% (685/1487) 2024-08-07T17:31:59.2315906Z remote: Compressing objects: 47% (699/1487) 2024-08-07T17:31:59.2320310Z remote: Compressing objects: 48% (714/1487) 2024-08-07T17:31:59.2325471Z remote: Compressing objects: 49% (729/1487) 2024-08-07T17:31:59.2329591Z remote: Compressing objects: 50% (744/1487) 2024-08-07T17:31:59.2333808Z remote: Compressing objects: 51% (759/1487) 2024-08-07T17:31:59.2337653Z remote: Compressing objects: 52% (774/1487) 2024-08-07T17:31:59.2340785Z remote: Compressing objects: 53% (789/1487) 2024-08-07T17:31:59.2342888Z remote: Compressing objects: 54% (803/1487) 2024-08-07T17:31:59.2344591Z remote: Compressing objects: 55% (818/1487) 2024-08-07T17:31:59.2346946Z remote: Compressing objects: 56% (833/1487) 2024-08-07T17:31:59.2347935Z remote: Compressing objects: 57% (848/1487) 2024-08-07T17:31:59.2348850Z remote: Compressing objects: 58% (863/1487) 2024-08-07T17:31:59.2349366Z remote: Compressing objects: 59% (878/1487) 2024-08-07T17:31:59.2349968Z remote: Compressing objects: 60% (893/1487) 2024-08-07T17:31:59.2351370Z remote: Compressing objects: 61% (908/1487) 2024-08-07T17:31:59.2351852Z remote: Compressing objects: 62% (922/1487) 2024-08-07T17:31:59.2352686Z remote: Compressing objects: 63% (937/1487) 2024-08-07T17:31:59.2355781Z remote: Compressing objects: 64% (952/1487) 2024-08-07T17:31:59.2373315Z remote: Compressing objects: 65% (967/1487) 2024-08-07T17:31:59.2384947Z remote: Compressing objects: 66% (982/1487) 2024-08-07T17:31:59.2394100Z remote: Compressing objects: 67% (997/1487) 2024-08-07T17:31:59.2404389Z remote: Compressing objects: 68% (1012/1487) 2024-08-07T17:31:59.2410914Z remote: Compressing objects: 69% (1027/1487) 2024-08-07T17:31:59.2417791Z remote: Compressing objects: 70% (1041/1487) 2024-08-07T17:31:59.2423256Z remote: Compressing objects: 71% (1056/1487) 2024-08-07T17:31:59.2428846Z remote: Compressing objects: 72% (1071/1487) 2024-08-07T17:31:59.2432683Z remote: Compressing objects: 73% (1086/1487) 2024-08-07T17:31:59.2437226Z remote: Compressing objects: 74% (1101/1487) 2024-08-07T17:31:59.2441500Z remote: Compressing objects: 75% (1116/1487) 2024-08-07T17:31:59.2445248Z remote: Compressing objects: 76% (1131/1487) 2024-08-07T17:31:59.2449244Z remote: Compressing objects: 77% (1145/1487) 2024-08-07T17:31:59.2453878Z remote: Compressing objects: 78% (1160/1487) 2024-08-07T17:31:59.2457860Z remote: Compressing objects: 79% (1175/1487) 2024-08-07T17:31:59.2461512Z remote: Compressing objects: 80% (1190/1487) 2024-08-07T17:31:59.2465432Z remote: Compressing objects: 81% (1205/1487) 2024-08-07T17:31:59.2469475Z remote: Compressing objects: 82% (1220/1487) 2024-08-07T17:31:59.2472936Z remote: Compressing objects: 83% (1235/1487) 2024-08-07T17:31:59.2475932Z remote: Compressing objects: 84% (1250/1487) 2024-08-07T17:31:59.2479795Z remote: Compressing objects: 85% (1264/1487) 2024-08-07T17:31:59.2482185Z remote: Compressing objects: 86% (1279/1487) 2024-08-07T17:31:59.2484678Z remote: Compressing objects: 87% (1294/1487) 2024-08-07T17:31:59.2487217Z remote: Compressing objects: 88% (1309/1487) 2024-08-07T17:31:59.2489777Z remote: Compressing objects: 89% (1324/1487) 2024-08-07T17:31:59.2492454Z remote: Compressing objects: 90% (1339/1487) 2024-08-07T17:31:59.2495036Z remote: Compressing objects: 91% (1354/1487) 2024-08-07T17:31:59.2497580Z remote: Compressing objects: 92% (1369/1487) 2024-08-07T17:31:59.2500080Z remote: Compressing objects: 93% (1383/1487) 2024-08-07T17:31:59.2500516Z remote: Compressing objects: 94% (1398/1487) 2024-08-07T17:31:59.2502487Z remote: Compressing objects: 95% (1413/1487) 2024-08-07T17:31:59.2505601Z remote: Compressing objects: 96% (1428/1487) 2024-08-07T17:31:59.2508155Z remote: Compressing objects: 97% (1443/1487) 2024-08-07T17:31:59.2510775Z remote: Compressing objects: 98% (1458/1487) 2024-08-07T17:31:59.2511268Z remote: Compressing objects: 99% (1473/1487) 2024-08-07T17:31:59.2511671Z remote: Compressing objects: 100% (1487/1487) 2024-08-07T17:31:59.2512114Z remote: Compressing objects: 100% (1487/1487), done. 2024-08-07T17:32:19.9727811Z remote: Total 1011031 (delta 1809), reused 2212 (delta 1343), pack-reused 1008194 2024-08-07T17:32:47.3289520Z [command]/usr/bin/git rev-parse --verify --quiet 016588f53c6904b840aa56aa86f95460b4d9c996^{object} 2024-08-07T17:32:47.3317383Z 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:32:47.3321903Z ##[endgroup] 2024-08-07T17:32:47.3322349Z ##[group]Determining the checkout info 2024-08-07T17:32:47.3323135Z ##[endgroup] 2024-08-07T17:32:47.3323712Z ##[group]Checking out the ref 2024-08-07T17:32:47.3326340Z [command]/usr/bin/git checkout --quiet --force 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:32:48.7211743Z ##[endgroup] 2024-08-07T17:32:48.7212216Z ##[group]Setting up auth for fetching submodules 2024-08-07T17:32:48.7215115Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic *** 2024-08-07T17:32:48.7266960Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf 2024-08-07T17:32:48.7299272Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com: 2024-08-07T17:32:48.7329717Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com: 2024-08-07T17:32:48.7356901Z ##[endgroup] 2024-08-07T17:32:48.7357312Z ##[group]Fetching submodules 2024-08-07T17:32:48.7361025Z [command]/usr/bin/git submodule sync --recursive 2024-08-07T17:32:48.7653103Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive 2024-08-07T17:32:48.7944084Z Submodule 'android/libs/fbjni' (https://github.com/facebookincubator/fbjni.git) registered for path 'android/libs/fbjni' 2024-08-07T17:32:48.7945160Z Submodule 'third_party/NNPACK_deps/FP16' (https://github.com/Maratyszcza/FP16.git) registered for path 'third_party/FP16' 2024-08-07T17:32:48.7946228Z Submodule 'third_party/NNPACK_deps/FXdiv' (https://github.com/Maratyszcza/FXdiv.git) registered for path 'third_party/FXdiv' 2024-08-07T17:32:48.7949127Z Submodule 'third_party/NNPACK' (https://github.com/Maratyszcza/NNPACK.git) registered for path 'third_party/NNPACK' 2024-08-07T17:32:48.7951344Z Submodule 'third_party/VulkanMemoryAllocator' (https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git) registered for path 'third_party/VulkanMemoryAllocator' 2024-08-07T17:32:48.7953512Z Submodule 'third_party/XNNPACK' (https://github.com/google/XNNPACK.git) registered for path 'third_party/XNNPACK' 2024-08-07T17:32:48.7956132Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/benchmark' 2024-08-07T17:32:48.7958905Z Submodule 'third_party/cpp-httplib' (https://github.com/yhirose/cpp-httplib.git) registered for path 'third_party/cpp-httplib' 2024-08-07T17:32:48.7961828Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo.git) registered for path 'third_party/cpuinfo' 2024-08-07T17:32:48.7965445Z Submodule 'third_party/cudnn_frontend' (https://github.com/NVIDIA/cudnn-frontend.git) registered for path 'third_party/cudnn_frontend' 2024-08-07T17:32:48.7968412Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/cutlass' 2024-08-07T17:32:48.7971472Z Submodule 'third_party/eigen' (https://gitlab.com/libeigen/eigen.git) registered for path 'third_party/eigen' 2024-08-07T17:32:48.7974623Z Submodule 'third_party/fbgemm' (https://github.com/pytorch/fbgemm) registered for path 'third_party/fbgemm' 2024-08-07T17:32:48.7977976Z Submodule 'third_party/flatbuffers' (https://github.com/google/flatbuffers.git) registered for path 'third_party/flatbuffers' 2024-08-07T17:32:48.7981323Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/fmt' 2024-08-07T17:32:48.7985072Z Submodule 'third_party/foxi' (https://github.com/houseroad/foxi.git) registered for path 'third_party/foxi' 2024-08-07T17:32:48.7989986Z Submodule 'third_party/gemmlowp/gemmlowp' (https://github.com/google/gemmlowp.git) registered for path 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:32:48.7993601Z Submodule 'third_party/gloo' (https://github.com/facebookincubator/gloo) registered for path 'third_party/gloo' 2024-08-07T17:32:48.7997360Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/googletest' 2024-08-07T17:32:48.8001408Z Submodule 'third_party/ideep' (https://github.com/intel/ideep) registered for path 'third_party/ideep' 2024-08-07T17:32:48.8005338Z Submodule 'third_party/ittapi' (https://github.com/intel/ittapi.git) registered for path 'third_party/ittapi' 2024-08-07T17:32:48.8009326Z Submodule 'third_party/kineto' (https://github.com/pytorch/kineto) registered for path 'third_party/kineto' 2024-08-07T17:32:48.8013511Z Submodule 'third_party/mimalloc' (https://github.com/microsoft/mimalloc.git) registered for path 'third_party/mimalloc' 2024-08-07T17:32:48.8017619Z Submodule 'third_party/nccl/nccl' (https://github.com/NVIDIA/nccl) registered for path 'third_party/nccl/nccl' 2024-08-07T17:32:48.8021902Z Submodule 'third_party/nlohmann' (https://github.com/nlohmann/json.git) registered for path 'third_party/nlohmann' 2024-08-07T17:32:48.8026258Z Submodule 'third_party/onnx' (https://github.com/onnx/onnx.git) registered for path 'third_party/onnx' 2024-08-07T17:32:48.8031036Z Submodule 'third_party/opentelemetry-cpp' (https://github.com/open-telemetry/opentelemetry-cpp.git) registered for path 'third_party/opentelemetry-cpp' 2024-08-07T17:32:48.8035473Z Submodule 'third_party/pocketfft' (https://github.com/mreineck/pocketfft) registered for path 'third_party/pocketfft' 2024-08-07T17:32:48.8040450Z Submodule 'third_party/protobuf' (https://github.com/protocolbuffers/protobuf.git) registered for path 'third_party/protobuf' 2024-08-07T17:32:48.8045382Z Submodule 'third_party/NNPACK_deps/psimd' (https://github.com/Maratyszcza/psimd.git) registered for path 'third_party/psimd' 2024-08-07T17:32:48.8050499Z Submodule 'third_party/NNPACK_deps/pthreadpool' (https://github.com/Maratyszcza/pthreadpool.git) registered for path 'third_party/pthreadpool' 2024-08-07T17:32:48.8055425Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/pybind11' 2024-08-07T17:32:48.8060492Z Submodule 'third_party/python-peachpy' (https://github.com/malfet/PeachPy.git) registered for path 'third_party/python-peachpy' 2024-08-07T17:32:48.8066853Z Submodule 'third_party/sleef' (https://github.com/shibatch/sleef) registered for path 'third_party/sleef' 2024-08-07T17:32:48.8072368Z Submodule 'third_party/tensorpipe' (https://github.com/pytorch/tensorpipe.git) registered for path 'third_party/tensorpipe' 2024-08-07T17:32:48.8102839Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/android/libs/fbjni'... 2024-08-07T17:32:49.1953791Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FP16'... 2024-08-07T17:32:49.3849826Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/FXdiv'... 2024-08-07T17:32:49.5718231Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/NNPACK'... 2024-08-07T17:32:49.8512859Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/VulkanMemoryAllocator'... 2024-08-07T17:32:52.2084712Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/XNNPACK'... 2024-08-07T17:33:06.2362809Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/benchmark'... 2024-08-07T17:33:06.6509483Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpp-httplib'... 2024-08-07T17:33:07.1632062Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cpuinfo'... 2024-08-07T17:33:07.7612880Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cudnn_frontend'... 2024-08-07T17:33:08.9673689Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/cutlass'... 2024-08-07T17:33:10.9416237Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/eigen'... 2024-08-07T17:33:16.8008213Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm'... 2024-08-07T17:33:18.3790817Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/flatbuffers'... 2024-08-07T17:33:20.2735600Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fmt'... 2024-08-07T17:33:21.8964085Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/foxi'... 2024-08-07T17:33:22.0933559Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gemmlowp/gemmlowp'... 2024-08-07T17:33:22.5341142Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/gloo'... 2024-08-07T17:33:22.8863226Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/googletest'... 2024-08-07T17:33:24.0810136Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep'... 2024-08-07T17:33:24.4573945Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ittapi'... 2024-08-07T17:33:24.7420595Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto'... 2024-08-07T17:33:26.2647967Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/mimalloc'... 2024-08-07T17:33:27.0429575Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nccl/nccl'... 2024-08-07T17:33:27.7126504Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/nlohmann'... 2024-08-07T17:33:34.0902516Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx'... 2024-08-07T17:33:36.3292737Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp'... 2024-08-07T17:33:42.4531128Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pocketfft'... 2024-08-07T17:33:42.7266655Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf'... 2024-08-07T17:33:52.1470253Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/psimd'... 2024-08-07T17:33:52.3306960Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pthreadpool'... 2024-08-07T17:33:52.5428945Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/pybind11'... 2024-08-07T17:33:53.4827316Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/python-peachpy'... 2024-08-07T17:33:53.7838357Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/sleef'... 2024-08-07T17:33:54.4164572Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe'... 2024-08-07T17:33:54.8529260Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f' 2024-08-07T17:33:54.8651955Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3' 2024-08-07T17:33:54.8744213Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1' 2024-08-07T17:33:54.8988269Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73' 2024-08-07T17:33:54.9353188Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191' 2024-08-07T17:33:55.8923390Z Submodule path 'third_party/XNNPACK': checked out 'fcbf55af6cf28a4627bcd1f703ab7ad843f0f3a2' 2024-08-07T17:33:55.9166493Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415' 2024-08-07T17:33:55.9618903Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006' 2024-08-07T17:33:56.0592843Z Submodule path 'third_party/cpuinfo': checked out '3c8b1533ac03dd6531ab6e7b9245d488f13a82a5' 2024-08-07T17:33:56.0932843Z Submodule path 'third_party/cudnn_frontend': checked out '98ca4e1941fe3263f128f74f10063a3ea35c7019' 2024-08-07T17:33:56.5921893Z Submodule path 'third_party/cutlass': checked out 'bbe579a9e3beb6ea6626d9227ec32d0dae119a49' 2024-08-07T17:33:56.8435390Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c' 2024-08-07T17:33:56.9252352Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e' 2024-08-07T17:33:56.9271218Z Submodule 'third_party/asmjit' (https://github.com/asmjit/asmjit.git) registered for path 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:33:56.9272922Z Submodule 'third_party/cpuinfo' (https://github.com/pytorch/cpuinfo) registered for path 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:33:56.9275317Z Submodule 'third_party/cutlass' (https://github.com/NVIDIA/cutlass.git) registered for path 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:33:56.9277820Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:33:56.9280523Z Submodule 'third_party/hipify_torch' (https://github.com/ROCmSoftwarePlatform/hipify_torch.git) registered for path 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:33:56.9307109Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/asmjit'... 2024-08-07T17:33:58.0272194Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cpuinfo'... 2024-08-07T17:33:58.7345196Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/cutlass'... 2024-08-07T17:34:00.7264183Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/googletest'... 2024-08-07T17:34:01.9159685Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/fbgemm/third_party/hipify_torch'... 2024-08-07T17:34:02.2891318Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81' 2024-08-07T17:34:02.3847958Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3' 2024-08-07T17:34:02.7884967Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6' 2024-08-07T17:34:02.8516689Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796' 2024-08-07T17:34:02.8651071Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1' 2024-08-07T17:34:02.9842652Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b' 2024-08-07T17:34:03.0262172Z Submodule path 'third_party/fmt': checked out '0c9fce2ffefecfdce794e1859584e25877b7b592' 2024-08-07T17:34:03.0359752Z Submodule path 'third_party/foxi': checked out 'c278588e34e535f0bb8f00df3880d26928038cad' 2024-08-07T17:34:03.0767073Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350' 2024-08-07T17:34:03.1023618Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509' 2024-08-07T17:34:03.1490109Z Submodule path 'third_party/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-07T17:34:03.1621386Z Submodule path 'third_party/ideep': checked out '55ca0191687aaf19aca5cdb7881c791e3bea442b' 2024-08-07T17:34:03.1637386Z Submodule 'mkl-dnn' (https://github.com/intel/mkl-dnn.git) registered for path 'third_party/ideep/mkl-dnn' 2024-08-07T17:34:03.1662425Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/ideep/mkl-dnn'... 2024-08-07T17:34:17.1022400Z Submodule path 'third_party/ideep/mkl-dnn': checked out '1137e04ec0b5251ca2b4400a4fd3c667ce843d67' 2024-08-07T17:34:17.1203277Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42' 2024-08-07T17:34:17.2072361Z Submodule path 'third_party/kineto': checked out 'da2f2682cabaf95d601fa2a9b7e0979f84fe7667' 2024-08-07T17:34:17.2089254Z Submodule 'libkineto/third_party/dynolog' (https://github.com/facebookincubator/dynolog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:34:17.2091695Z Submodule 'libkineto/third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:34:17.2093663Z Submodule 'libkineto/third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:34:17.2121253Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog'... 2024-08-07T17:34:17.8189553Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/fmt'... 2024-08-07T17:34:19.1503780Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/googletest'... 2024-08-07T17:34:20.4265646Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e' 2024-08-07T17:34:20.4282660Z Submodule 'third_party/DCGM' (https://github.com/NVIDIA/DCGM.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:34:20.4284254Z Submodule 'third_party/cpr' (https://github.com/libcpr/cpr.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:34:20.4286731Z Submodule 'third_party/fmt' (https://github.com/fmtlib/fmt.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:34:20.4289306Z Submodule 'third_party/gflags' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:34:20.4291913Z Submodule 'third_party/glog' (https://github.com/google/glog.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:34:20.4295152Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:34:20.4297539Z Submodule 'third_party/json' (https://github.com/nlohmann/json.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:34:20.4300533Z Submodule 'third_party/pfs' (https://github.com/dtrugman/pfs.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:34:20.4328400Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'... 2024-08-07T17:34:21.4057204Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'... 2024-08-07T17:34:21.7718261Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'... 2024-08-07T17:34:23.0980523Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'... 2024-08-07T17:34:23.4270536Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/glog'... 2024-08-07T17:34:23.9881990Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'... 2024-08-07T17:34:25.2266100Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/json'... 2024-08-07T17:34:32.1919353Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'... 2024-08-07T17:34:32.6056690Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9' 2024-08-07T17:34:32.6243714Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400' 2024-08-07T17:34:32.6618351Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05' 2024-08-07T17:34:32.6758372Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067' 2024-08-07T17:34:32.6774047Z Submodule 'doc' (https://github.com/gflags/gflags.git) registered for path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:34:32.6800311Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'... 2024-08-07T17:34:33.0178534Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4' 2024-08-07T17:34:33.0363875Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446' 2024-08-07T17:34:33.0772758Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850' 2024-08-07T17:34:33.1759218Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5' 2024-08-07T17:34:33.1921246Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150' 2024-08-07T17:34:33.2326155Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164' 2024-08-07T17:34:33.2912430Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347' 2024-08-07T17:34:33.3284078Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78' 2024-08-07T17:34:33.3526973Z Submodule path 'third_party/nccl/nccl': checked out 'ab2b89c4c339bd7f816fbc114a4b05d386b66290' 2024-08-07T17:34:33.4559155Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806' 2024-08-07T17:34:33.8060737Z Submodule path 'third_party/onnx': checked out '3bf92c03a9f27eba3bda1e5b9e63ea20ec213557' 2024-08-07T17:34:33.8095711Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/onnx/third_party/benchmark' 2024-08-07T17:34:33.8097634Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/onnx/third_party/pybind11' 2024-08-07T17:34:33.8126193Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/benchmark'... 2024-08-07T17:34:34.2419642Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/onnx/third_party/pybind11'... 2024-08-07T17:34:35.2490244Z Submodule path 'third_party/onnx/third_party/benchmark': checked out '2dd015dfef425c866d9a43f2c67d8b52d709acb6' 2024-08-07T17:34:35.2838644Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '5b0a6fc2017fcc176545afe3e09c9f9885283242' 2024-08-07T17:34:35.3552801Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878' 2024-08-07T17:34:35.3572868Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark) registered for path 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:34:35.3574829Z Submodule 'third_party/googletest' (https://github.com/google/googletest) registered for path 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:34:35.3577204Z Submodule 'third_party/ms-gsl' (https://github.com/microsoft/GSL) registered for path 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:34:35.3579800Z Submodule 'third_party/nlohmann-json' (https://github.com/nlohmann/json) registered for path 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:34:35.3582802Z Submodule 'third_party/opentelemetry-proto' (https://github.com/open-telemetry/opentelemetry-proto) registered for path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:34:35.3585203Z Submodule 'third_party/opentracing-cpp' (https://github.com/opentracing/opentracing-cpp.git) registered for path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:34:35.3587927Z Submodule 'third_party/prometheus-cpp' (https://github.com/jupp0r/prometheus-cpp) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:34:35.3590571Z Submodule 'tools/vcpkg' (https://github.com/Microsoft/vcpkg) registered for path 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:34:35.3618134Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/benchmark'... 2024-08-07T17:34:35.7909232Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/googletest'... 2024-08-07T17:34:37.0084709Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/ms-gsl'... 2024-08-07T17:34:37.3328380Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/nlohmann-json'... 2024-08-07T17:34:43.7152283Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentelemetry-proto'... 2024-08-07T17:34:43.9846701Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/opentracing-cpp'... 2024-08-07T17:34:44.2175439Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp'... 2024-08-07T17:34:44.5581869Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/tools/vcpkg'... 2024-08-07T17:34:50.6794213Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2' 2024-08-07T17:34:50.7202355Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1' 2024-08-07T17:34:50.7362462Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa' 2024-08-07T17:34:50.8370529Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d' 2024-08-07T17:34:50.8508355Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce' 2024-08-07T17:34:50.8669477Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5' 2024-08-07T17:34:50.8831237Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d' 2024-08-07T17:34:50.8849174Z Submodule 'civetweb' (https://github.com/civetweb/civetweb.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:34:50.8851232Z Submodule 'googletest' (https://github.com/google/googletest.git) registered for path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:34:50.8878047Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'... 2024-08-07T17:34:53.1910429Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'... 2024-08-07T17:34:54.6705409Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4' 2024-08-07T17:34:54.7168365Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929' 2024-08-07T17:34:55.1750721Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50' 2024-08-07T17:34:55.1870693Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe' 2024-08-07T17:34:55.4515562Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a' 2024-08-07T17:34:55.4536555Z Submodule 'third_party/benchmark' (https://github.com/google/benchmark.git) registered for path 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:34:55.4538531Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/protobuf/third_party/googletest' 2024-08-07T17:34:55.4565303Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/benchmark'... 2024-08-07T17:34:55.9233489Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/protobuf/third_party/googletest'... 2024-08-07T17:34:57.1331048Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8' 2024-08-07T17:34:57.2047273Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081' 2024-08-07T17:34:57.2138119Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900' 2024-08-07T17:34:57.2257762Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8' 2024-08-07T17:34:57.2629107Z Submodule path 'third_party/pybind11': checked out '941f45bcb51457884fa1afd6e24a67377d70f75c' 2024-08-07T17:34:57.2923454Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67' 2024-08-07T17:34:57.3342537Z Submodule path 'third_party/sleef': checked out '60e76d2bce17d278b439d9da17177c8f957a9e9b' 2024-08-07T17:34:57.3608649Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e' 2024-08-07T17:34:57.3626312Z Submodule 'third_party/googletest' (https://github.com/google/googletest.git) registered for path 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:34:57.3628328Z Submodule 'third_party/libnop' (https://github.com/google/libnop.git) registered for path 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:34:57.3630687Z Submodule 'third_party/libuv' (https://github.com/libuv/libuv.git) registered for path 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:34:57.3633116Z Submodule 'third_party/pybind11' (https://github.com/pybind/pybind11.git) registered for path 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:34:57.3659715Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/googletest'... 2024-08-07T17:34:58.5479768Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libnop'... 2024-08-07T17:34:58.7610124Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/libuv'... 2024-08-07T17:34:59.9764029Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11'... 2024-08-07T17:35:00.9778082Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e' 2024-08-07T17:35:00.9931902Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281' 2024-08-07T17:35:01.0617353Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242' 2024-08-07T17:35:01.0906104Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef' 2024-08-07T17:35:01.0921676Z Submodule 'tools/clang' (https://github.com/wjakob/clang-cindex-python3) registered for path 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:35:01.0947811Z Cloning into '/home/ec2-user/actions-runner/_work/pytorch/pytorch/third_party/tensorpipe/third_party/pybind11/tools/clang'... 2024-08-07T17:35:01.3051785Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5' 2024-08-07T17:35:01.3086361Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0 2024-08-07T17:35:01.3378082Z Entering 'android/libs/fbjni' 2024-08-07T17:35:01.3419469Z Entering 'third_party/FP16' 2024-08-07T17:35:01.3460729Z Entering 'third_party/FXdiv' 2024-08-07T17:35:01.3499782Z Entering 'third_party/NNPACK' 2024-08-07T17:35:01.3540347Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:35:01.3581145Z Entering 'third_party/XNNPACK' 2024-08-07T17:35:01.3640464Z Entering 'third_party/benchmark' 2024-08-07T17:35:01.3678054Z Entering 'third_party/cpp-httplib' 2024-08-07T17:35:01.3718223Z Entering 'third_party/cpuinfo' 2024-08-07T17:35:01.3758057Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:35:01.3796838Z Entering 'third_party/cutlass' 2024-08-07T17:35:01.3846688Z Entering 'third_party/eigen' 2024-08-07T17:35:01.3889736Z Entering 'third_party/fbgemm' 2024-08-07T17:35:01.3929806Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:35:01.3969076Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:35:01.4008091Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:35:01.4053747Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:35:01.4093064Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:35:01.4132324Z Entering 'third_party/flatbuffers' 2024-08-07T17:35:01.4176323Z Entering 'third_party/fmt' 2024-08-07T17:35:01.4215653Z Entering 'third_party/foxi' 2024-08-07T17:35:01.4255791Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:35:01.4296753Z Entering 'third_party/gloo' 2024-08-07T17:35:01.4336775Z Entering 'third_party/googletest' 2024-08-07T17:35:01.4377561Z Entering 'third_party/ideep' 2024-08-07T17:35:01.4415947Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:35:01.4464407Z Entering 'third_party/ittapi' 2024-08-07T17:35:01.4505286Z Entering 'third_party/kineto' 2024-08-07T17:35:01.4545536Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:35:01.4585926Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:35:01.4626960Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:35:01.4666668Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:35:01.4705095Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:35:01.4743714Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:35:01.4784565Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:35:01.4823434Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:35:01.4863299Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:35:01.4903047Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:35:01.4946046Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:35:01.4983970Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:35:01.5023123Z Entering 'third_party/mimalloc' 2024-08-07T17:35:01.5064127Z Entering 'third_party/nccl/nccl' 2024-08-07T17:35:01.5103693Z Entering 'third_party/nlohmann' 2024-08-07T17:35:01.5146212Z Entering 'third_party/onnx' 2024-08-07T17:35:01.5202048Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:35:01.5242599Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:35:01.5283814Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:35:01.5326287Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:35:01.5367367Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:35:01.5405940Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:35:01.5445748Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:35:01.5485753Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:35:01.5523928Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:35:01.5562000Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:35:01.5599508Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:35:01.5640719Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:35:01.5680999Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:35:01.5740928Z Entering 'third_party/pocketfft' 2024-08-07T17:35:01.5781499Z Entering 'third_party/protobuf' 2024-08-07T17:35:01.5824610Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:35:01.5863305Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:35:01.5903985Z Entering 'third_party/psimd' 2024-08-07T17:35:01.5944778Z Entering 'third_party/pthreadpool' 2024-08-07T17:35:01.5983296Z Entering 'third_party/pybind11' 2024-08-07T17:35:01.6022083Z Entering 'third_party/python-peachpy' 2024-08-07T17:35:01.6060991Z Entering 'third_party/sleef' 2024-08-07T17:35:01.6099583Z Entering 'third_party/tensorpipe' 2024-08-07T17:35:01.6138774Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:35:01.6177524Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:35:01.6215312Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:35:01.6253898Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:35:01.6291155Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:35:01.6346659Z ##[endgroup] 2024-08-07T17:35:01.6349321Z ##[group]Persisting credentials for submodules 2024-08-07T17:35:01.6353150Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :" 2024-08-07T17:35:01.6637996Z Entering 'android/libs/fbjni' 2024-08-07T17:35:01.6690537Z Entering 'third_party/FP16' 2024-08-07T17:35:01.6741329Z Entering 'third_party/FXdiv' 2024-08-07T17:35:01.6792513Z Entering 'third_party/NNPACK' 2024-08-07T17:35:01.6844038Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:35:01.6895315Z Entering 'third_party/XNNPACK' 2024-08-07T17:35:01.6963825Z Entering 'third_party/benchmark' 2024-08-07T17:35:01.7014848Z Entering 'third_party/cpp-httplib' 2024-08-07T17:35:01.7066108Z Entering 'third_party/cpuinfo' 2024-08-07T17:35:01.7117857Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:35:01.7169565Z Entering 'third_party/cutlass' 2024-08-07T17:35:01.7228904Z Entering 'third_party/eigen' 2024-08-07T17:35:01.7282582Z Entering 'third_party/fbgemm' 2024-08-07T17:35:01.7333483Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:35:01.7385120Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:35:01.7435289Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:35:01.7493396Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:35:01.7545178Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:35:01.7598725Z Entering 'third_party/flatbuffers' 2024-08-07T17:35:01.7654548Z Entering 'third_party/fmt' 2024-08-07T17:35:01.7706508Z Entering 'third_party/foxi' 2024-08-07T17:35:01.7758387Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:35:01.7809911Z Entering 'third_party/gloo' 2024-08-07T17:35:01.7861232Z Entering 'third_party/googletest' 2024-08-07T17:35:01.7912472Z Entering 'third_party/ideep' 2024-08-07T17:35:01.7962258Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:35:01.8021148Z Entering 'third_party/ittapi' 2024-08-07T17:35:01.8072036Z Entering 'third_party/kineto' 2024-08-07T17:35:01.8123248Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:35:01.8177443Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:35:01.8230034Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:35:01.8284679Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:35:01.8336373Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:35:01.8387572Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:35:01.8443803Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:35:01.8494626Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:35:01.8546545Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:35:01.8599404Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:35:01.8653455Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:35:01.8704680Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:35:01.8756847Z Entering 'third_party/mimalloc' 2024-08-07T17:35:01.8810725Z Entering 'third_party/nccl/nccl' 2024-08-07T17:35:01.8863070Z Entering 'third_party/nlohmann' 2024-08-07T17:35:01.8916240Z Entering 'third_party/onnx' 2024-08-07T17:35:01.8985241Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:35:01.9038227Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:35:01.9093071Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:35:01.9147794Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:35:01.9199204Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:35:01.9250016Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:35:01.9300538Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:35:01.9353832Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:35:01.9405647Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:35:01.9458166Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:35:01.9509084Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:35:01.9563096Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:35:01.9616619Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:35:01.9691727Z Entering 'third_party/pocketfft' 2024-08-07T17:35:01.9746875Z Entering 'third_party/protobuf' 2024-08-07T17:35:01.9803046Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:35:01.9854892Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:35:01.9906997Z Entering 'third_party/psimd' 2024-08-07T17:35:01.9959355Z Entering 'third_party/pthreadpool' 2024-08-07T17:35:02.0010429Z Entering 'third_party/pybind11' 2024-08-07T17:35:02.0061348Z Entering 'third_party/python-peachpy' 2024-08-07T17:35:02.0112454Z Entering 'third_party/sleef' 2024-08-07T17:35:02.0165708Z Entering 'third_party/tensorpipe' 2024-08-07T17:35:02.0217094Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:35:02.0272263Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:35:02.0324111Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:35:02.0375107Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:35:02.0424106Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:35:02.0493381Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url" 2024-08-07T17:35:02.0785182Z Entering 'android/libs/fbjni' 2024-08-07T17:35:02.0836196Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config remote.origin.url 2024-08-07T17:35:02.0850989Z Entering 'third_party/FP16' 2024-08-07T17:35:02.0899910Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config remote.origin.url 2024-08-07T17:35:02.0914296Z Entering 'third_party/FXdiv' 2024-08-07T17:35:02.0963930Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config remote.origin.url 2024-08-07T17:35:02.0979136Z Entering 'third_party/NNPACK' 2024-08-07T17:35:02.1028451Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config remote.origin.url 2024-08-07T17:35:02.1044703Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:35:02.1094737Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config remote.origin.url 2024-08-07T17:35:02.1110309Z Entering 'third_party/XNNPACK' 2024-08-07T17:35:02.1161025Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config remote.origin.url 2024-08-07T17:35:02.1193319Z Entering 'third_party/benchmark' 2024-08-07T17:35:02.1241114Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:35:02.1256972Z Entering 'third_party/cpp-httplib' 2024-08-07T17:35:02.1307203Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config remote.origin.url 2024-08-07T17:35:02.1323066Z Entering 'third_party/cpuinfo' 2024-08-07T17:35:02.1371239Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config remote.origin.url 2024-08-07T17:35:02.1388362Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:35:02.1436194Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config remote.origin.url 2024-08-07T17:35:02.1452626Z Entering 'third_party/cutlass' 2024-08-07T17:35:02.1500269Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config remote.origin.url 2024-08-07T17:35:02.1524673Z Entering 'third_party/eigen' 2024-08-07T17:35:02.1573724Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config remote.origin.url 2024-08-07T17:35:02.1591177Z Entering 'third_party/fbgemm' 2024-08-07T17:35:02.1641983Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config remote.origin.url 2024-08-07T17:35:02.1657460Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:35:02.1706142Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config remote.origin.url 2024-08-07T17:35:02.1720990Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:35:02.1768432Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config remote.origin.url 2024-08-07T17:35:02.1783874Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:35:02.1830381Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config remote.origin.url 2024-08-07T17:35:02.1852937Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:35:02.1899856Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:35:02.1914497Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:35:02.1962482Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config remote.origin.url 2024-08-07T17:35:02.1979372Z Entering 'third_party/flatbuffers' 2024-08-07T17:35:02.2029332Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config remote.origin.url 2024-08-07T17:35:02.2046868Z Entering 'third_party/fmt' 2024-08-07T17:35:02.2095596Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config remote.origin.url 2024-08-07T17:35:02.2111067Z Entering 'third_party/foxi' 2024-08-07T17:35:02.2159521Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/foxi/config remote.origin.url 2024-08-07T17:35:02.2176045Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:35:02.2222610Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config remote.origin.url 2024-08-07T17:35:02.2238575Z Entering 'third_party/gloo' 2024-08-07T17:35:02.2286277Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config remote.origin.url 2024-08-07T17:35:02.2302022Z Entering 'third_party/googletest' 2024-08-07T17:35:02.2351577Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:35:02.2367930Z Entering 'third_party/ideep' 2024-08-07T17:35:02.2417136Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config remote.origin.url 2024-08-07T17:35:02.2432339Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:35:02.2481686Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config remote.origin.url 2024-08-07T17:35:02.2505089Z Entering 'third_party/ittapi' 2024-08-07T17:35:02.2553735Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config remote.origin.url 2024-08-07T17:35:02.2570452Z Entering 'third_party/kineto' 2024-08-07T17:35:02.2618803Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config remote.origin.url 2024-08-07T17:35:02.2635019Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:35:02.2685428Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config remote.origin.url 2024-08-07T17:35:02.2700229Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:35:02.2750384Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config remote.origin.url 2024-08-07T17:35:02.2766103Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:35:02.2814942Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config remote.origin.url 2024-08-07T17:35:02.2829098Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:35:02.2878257Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config remote.origin.url 2024-08-07T17:35:02.2893787Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:35:02.2943891Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config remote.origin.url 2024-08-07T17:35:02.2958104Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:35:02.3006755Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config remote.origin.url 2024-08-07T17:35:02.3024212Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:35:02.3073058Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config remote.origin.url 2024-08-07T17:35:02.3088599Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:35:02.3137075Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:35:02.3153171Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:35:02.3202458Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config remote.origin.url 2024-08-07T17:35:02.3218676Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:35:02.3267338Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config remote.origin.url 2024-08-07T17:35:02.3284465Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:35:02.3332235Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config remote.origin.url 2024-08-07T17:35:02.3347634Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:35:02.3395205Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config remote.origin.url 2024-08-07T17:35:02.3412304Z Entering 'third_party/mimalloc' 2024-08-07T17:35:02.3462945Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config remote.origin.url 2024-08-07T17:35:02.3478860Z Entering 'third_party/nccl/nccl' 2024-08-07T17:35:02.3525782Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nccl/nccl/config remote.origin.url 2024-08-07T17:35:02.3543313Z Entering 'third_party/nlohmann' 2024-08-07T17:35:02.3591277Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config remote.origin.url 2024-08-07T17:35:02.3608717Z Entering 'third_party/onnx' 2024-08-07T17:35:02.3656329Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config remote.origin.url 2024-08-07T17:35:02.3688946Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:35:02.3736957Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:35:02.3753288Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:35:02.3800136Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config remote.origin.url 2024-08-07T17:35:02.3818454Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:35:02.3868534Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config remote.origin.url 2024-08-07T17:35:02.3885146Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:35:02.3933442Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:35:02.3948500Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:35:02.3995698Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:35:02.4010229Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:35:02.4057741Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config remote.origin.url 2024-08-07T17:35:02.4071995Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:35:02.4118112Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config remote.origin.url 2024-08-07T17:35:02.4133881Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:35:02.4180427Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config remote.origin.url 2024-08-07T17:35:02.4194626Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:35:02.4241427Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config remote.origin.url 2024-08-07T17:35:02.4256769Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:35:02.4305458Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config remote.origin.url 2024-08-07T17:35:02.4319719Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:35:02.4369680Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config remote.origin.url 2024-08-07T17:35:02.4387086Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:35:02.4434783Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config remote.origin.url 2024-08-07T17:35:02.4451617Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:35:02.4498471Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config remote.origin.url 2024-08-07T17:35:02.4536887Z Entering 'third_party/pocketfft' 2024-08-07T17:35:02.4587537Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config remote.origin.url 2024-08-07T17:35:02.4603087Z Entering 'third_party/protobuf' 2024-08-07T17:35:02.4651186Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config remote.origin.url 2024-08-07T17:35:02.4670047Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:35:02.4717859Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config remote.origin.url 2024-08-07T17:35:02.4732772Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:35:02.4780963Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:35:02.4797854Z Entering 'third_party/psimd' 2024-08-07T17:35:02.4845443Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config remote.origin.url 2024-08-07T17:35:02.4861119Z Entering 'third_party/pthreadpool' 2024-08-07T17:35:02.4907884Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config remote.origin.url 2024-08-07T17:35:02.4923798Z Entering 'third_party/pybind11' 2024-08-07T17:35:02.4970665Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config remote.origin.url 2024-08-07T17:35:02.4986783Z Entering 'third_party/python-peachpy' 2024-08-07T17:35:02.5033238Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config remote.origin.url 2024-08-07T17:35:02.5050917Z Entering 'third_party/sleef' 2024-08-07T17:35:02.5098035Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config remote.origin.url 2024-08-07T17:35:02.5113583Z Entering 'third_party/tensorpipe' 2024-08-07T17:35:02.5161754Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config remote.origin.url 2024-08-07T17:35:02.5177270Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:35:02.5224738Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config remote.origin.url 2024-08-07T17:35:02.5240153Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:35:02.5287030Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config remote.origin.url 2024-08-07T17:35:02.5302137Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:35:02.5350705Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config remote.origin.url 2024-08-07T17:35:02.5365788Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:35:02.5412215Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config remote.origin.url 2024-08-07T17:35:02.5426255Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:35:02.5475284Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config remote.origin.url 2024-08-07T17:35:02.6379132Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:' 2024-08-07T17:35:02.6670137Z Entering 'android/libs/fbjni' 2024-08-07T17:35:02.6710991Z Entering 'third_party/FP16' 2024-08-07T17:35:02.6751272Z Entering 'third_party/FXdiv' 2024-08-07T17:35:02.6791566Z Entering 'third_party/NNPACK' 2024-08-07T17:35:02.6831271Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:35:02.6871464Z Entering 'third_party/XNNPACK' 2024-08-07T17:35:02.6928879Z Entering 'third_party/benchmark' 2024-08-07T17:35:02.6970466Z Entering 'third_party/cpp-httplib' 2024-08-07T17:35:02.7010291Z Entering 'third_party/cpuinfo' 2024-08-07T17:35:02.7050099Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:35:02.7089478Z Entering 'third_party/cutlass' 2024-08-07T17:35:02.7136235Z Entering 'third_party/eigen' 2024-08-07T17:35:02.7178712Z Entering 'third_party/fbgemm' 2024-08-07T17:35:02.7218294Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:35:02.7257020Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:35:02.7295851Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:35:02.7340362Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:35:02.7378648Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:35:02.7418856Z Entering 'third_party/flatbuffers' 2024-08-07T17:35:02.7462848Z Entering 'third_party/fmt' 2024-08-07T17:35:02.7502779Z Entering 'third_party/foxi' 2024-08-07T17:35:02.7542239Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:35:02.7581378Z Entering 'third_party/gloo' 2024-08-07T17:35:02.7620914Z Entering 'third_party/googletest' 2024-08-07T17:35:02.7660883Z Entering 'third_party/ideep' 2024-08-07T17:35:02.7699579Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:35:02.7746723Z Entering 'third_party/ittapi' 2024-08-07T17:35:02.7786097Z Entering 'third_party/kineto' 2024-08-07T17:35:02.7825290Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:35:02.7864462Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:35:02.7905791Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:35:02.7945178Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:35:02.7984541Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:35:02.8022953Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:35:02.8062563Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:35:02.8101082Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:35:02.8138698Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:35:02.8178722Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:35:02.8218395Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:35:02.8257397Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:35:02.8299525Z Entering 'third_party/mimalloc' 2024-08-07T17:35:02.8341231Z Entering 'third_party/nccl/nccl' 2024-08-07T17:35:02.8379027Z Entering 'third_party/nlohmann' 2024-08-07T17:35:02.8422401Z Entering 'third_party/onnx' 2024-08-07T17:35:02.8478672Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:35:02.8518234Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:35:02.8561454Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:35:02.8605402Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:35:02.8644403Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:35:02.8683660Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:35:02.8723174Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:35:02.8763257Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:35:02.8801471Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:35:02.8839630Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:35:02.8878797Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:35:02.8919664Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:35:02.8960374Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:35:02.9019291Z Entering 'third_party/pocketfft' 2024-08-07T17:35:02.9059297Z Entering 'third_party/protobuf' 2024-08-07T17:35:02.9101290Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:35:02.9140585Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:35:02.9182500Z Entering 'third_party/psimd' 2024-08-07T17:35:02.9222683Z Entering 'third_party/pthreadpool' 2024-08-07T17:35:02.9261662Z Entering 'third_party/pybind11' 2024-08-07T17:35:02.9301282Z Entering 'third_party/python-peachpy' 2024-08-07T17:35:02.9341248Z Entering 'third_party/sleef' 2024-08-07T17:35:02.9382411Z Entering 'third_party/tensorpipe' 2024-08-07T17:35:02.9421210Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:35:02.9459895Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:35:02.9498322Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:35:02.9536213Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:35:02.9574562Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:35:02.9629248Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:' 2024-08-07T17:35:02.9916035Z Entering 'android/libs/fbjni' 2024-08-07T17:35:02.9955952Z Entering 'third_party/FP16' 2024-08-07T17:35:02.9996245Z Entering 'third_party/FXdiv' 2024-08-07T17:35:03.0035053Z Entering 'third_party/NNPACK' 2024-08-07T17:35:03.0078919Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T17:35:03.0117424Z Entering 'third_party/XNNPACK' 2024-08-07T17:35:03.0174365Z Entering 'third_party/benchmark' 2024-08-07T17:35:03.0213643Z Entering 'third_party/cpp-httplib' 2024-08-07T17:35:03.0253317Z Entering 'third_party/cpuinfo' 2024-08-07T17:35:03.0293051Z Entering 'third_party/cudnn_frontend' 2024-08-07T17:35:03.0332998Z Entering 'third_party/cutlass' 2024-08-07T17:35:03.0382576Z Entering 'third_party/eigen' 2024-08-07T17:35:03.0424385Z Entering 'third_party/fbgemm' 2024-08-07T17:35:03.0465157Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T17:35:03.0503424Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T17:35:03.0541993Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T17:35:03.0587694Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T17:35:03.0626207Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T17:35:03.0666306Z Entering 'third_party/flatbuffers' 2024-08-07T17:35:03.0709039Z Entering 'third_party/fmt' 2024-08-07T17:35:03.0748780Z Entering 'third_party/foxi' 2024-08-07T17:35:03.0788384Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T17:35:03.0828413Z Entering 'third_party/gloo' 2024-08-07T17:35:03.0869886Z Entering 'third_party/googletest' 2024-08-07T17:35:03.0910244Z Entering 'third_party/ideep' 2024-08-07T17:35:03.0949398Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T17:35:03.0995889Z Entering 'third_party/ittapi' 2024-08-07T17:35:03.1034181Z Entering 'third_party/kineto' 2024-08-07T17:35:03.1073614Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T17:35:03.1113149Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T17:35:03.1153458Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T17:35:03.1192641Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T17:35:03.1231567Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T17:35:03.1270337Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T17:35:03.1310931Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T17:35:03.1350657Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T17:35:03.1390150Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T17:35:03.1430535Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T17:35:03.1472070Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T17:35:03.1510945Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T17:35:03.1551340Z Entering 'third_party/mimalloc' 2024-08-07T17:35:03.1591819Z Entering 'third_party/nccl/nccl' 2024-08-07T17:35:03.1632342Z Entering 'third_party/nlohmann' 2024-08-07T17:35:03.1673370Z Entering 'third_party/onnx' 2024-08-07T17:35:03.1729580Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T17:35:03.1768656Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T17:35:03.1810115Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T17:35:03.1852285Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T17:35:03.1890102Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T17:35:03.1928692Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T17:35:03.1966820Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T17:35:03.2007326Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T17:35:03.2045413Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T17:35:03.2083538Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T17:35:03.2120999Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T17:35:03.2161379Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T17:35:03.2203382Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T17:35:03.2263158Z Entering 'third_party/pocketfft' 2024-08-07T17:35:03.2303575Z Entering 'third_party/protobuf' 2024-08-07T17:35:03.2349449Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T17:35:03.2388017Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T17:35:03.2428687Z Entering 'third_party/psimd' 2024-08-07T17:35:03.2474544Z Entering 'third_party/pthreadpool' 2024-08-07T17:35:03.2514803Z Entering 'third_party/pybind11' 2024-08-07T17:35:03.2554599Z Entering 'third_party/python-peachpy' 2024-08-07T17:35:03.2593712Z Entering 'third_party/sleef' 2024-08-07T17:35:03.2632946Z Entering 'third_party/tensorpipe' 2024-08-07T17:35:03.2671878Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T17:35:03.2710821Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T17:35:03.2748335Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T17:35:03.2786868Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T17:35:03.2824246Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T17:35:03.2877720Z ##[endgroup] 2024-08-07T17:35:03.2919028Z [command]/usr/bin/git log -1 --format='%H' 2024-08-07T17:35:03.2954119Z '016588f53c6904b840aa56aa86f95460b4d9c996' 2024-08-07T17:35:03.3110859Z Prepare all required actions 2024-08-07T17:35:03.3111417Z Getting action download info 2024-08-07T17:35:03.4478386Z ##[group]Run ./.github/actions/setup-linux 2024-08-07T17:35:03.4478817Z env: 2024-08-07T17:35:03.4479042Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:03.4479326Z ##[endgroup] 2024-08-07T17:35:03.4525652Z ##[group]Run set -euo pipefail 2024-08-07T17:35:03.4526017Z set -euo pipefail 2024-08-07T17:35:03.4526313Z function get_ec2_metadata() { 2024-08-07T17:35:03.4526706Z  # Pulled from instance metadata endpoint for EC2 2024-08-07T17:35:03.4527686Z  # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html 2024-08-07T17:35:03.4528399Z  category=$1 2024-08-07T17:35:03.4528781Z  # If it is GCP runner (runner name contains gcp), do not run this 2024-08-07T17:35:03.4529246Z  runner_name_str=i-0efccfdf668c75c22 2024-08-07T17:35:03.4529651Z  if [[ -f /.inarc ]]; then 2024-08-07T17:35:03.4530015Z  echo "ARC Runner, no info on ec2 metadata" 2024-08-07T17:35:03.4530403Z  elif [[ $runner_name_str == *"gcp"* ]]; then 2024-08-07T17:35:03.4530893Z  echo "Runner is from Google Cloud Platform, No info on ec2 metadata" 2024-08-07T17:35:03.4531341Z  else 2024-08-07T17:35:03.4531684Z  curl -fsSL "http://169.254.169.254/latest/meta-data/${category}" 2024-08-07T17:35:03.4532113Z  fi 2024-08-07T17:35:03.4532346Z } 2024-08-07T17:35:03.4532889Z echo "ami-id: $(get_ec2_metadata ami-id)" 2024-08-07T17:35:03.4533595Z echo "instance-id: $(get_ec2_metadata instance-id)" 2024-08-07T17:35:03.4534372Z echo "instance-type: $(get_ec2_metadata instance-type)" 2024-08-07T17:35:03.4534918Z echo "system info $(uname -a)" 2024-08-07T17:35:03.4545143Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:03.4545731Z env: 2024-08-07T17:35:03.4546052Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:03.4546448Z ##[endgroup] 2024-08-07T17:35:03.4641886Z ami-id: ami-06c68f701d8090592 2024-08-07T17:35:03.4698010Z instance-id: i-0efccfdf668c75c22 2024-08-07T17:35:03.4748082Z instance-type: c5.2xlarge 2024-08-07T17:35:03.4758020Z system info Linux ip-10-0-77-114.ec2.internal 6.1.94-99.176.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Jun 18 14:57:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 2024-08-07T17:35:03.4788289Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-07T17:35:03.4788991Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> $GITHUB_OUTPUT 2024-08-07T17:35:03.4795596Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:03.4795993Z env: 2024-08-07T17:35:03.4796223Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:03.4796486Z ##[endgroup] 2024-08-07T17:35:03.4872560Z ##[group]Run if systemctl is-active --quiet docker; then 2024-08-07T17:35:03.4873054Z if systemctl is-active --quiet docker; then 2024-08-07T17:35:03.4873439Z  echo "Docker daemon is running..."; 2024-08-07T17:35:03.4873782Z else 2024-08-07T17:35:03.4874149Z  echo "Starting docker deamon..." && sudo systemctl start docker; 2024-08-07T17:35:03.4874571Z fi 2024-08-07T17:35:03.4880263Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:03.4880662Z env: 2024-08-07T17:35:03.4880877Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:03.4881156Z ##[endgroup] 2024-08-07T17:35:03.4953225Z Docker daemon is running... 2024-08-07T17:35:03.5002920Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-07T17:35:03.5003357Z with: 2024-08-07T17:35:03.5003583Z shell: bash 2024-08-07T17:35:03.5003822Z timeout_minutes: 5 2024-08-07T17:35:03.5004073Z max_attempts: 3 2024-08-07T17:35:03.5004325Z retry_wait_seconds: 30 2024-08-07T17:35:03.5005617Z command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\") aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \ --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com" 2024-08-07T17:35:03.5006740Z polling_interval_seconds: 1 2024-08-07T17:35:03.5007027Z warning_on_retry: true 2024-08-07T17:35:03.5007305Z continue_on_error: false 2024-08-07T17:35:03.5007578Z env: 2024-08-07T17:35:03.5007789Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:03.5008067Z AWS_RETRY_MODE: standard 2024-08-07T17:35:03.5008344Z AWS_MAX_ATTEMPTS: 5 2024-08-07T17:35:03.5008613Z AWS_DEFAULT_REGION: us-east-1 2024-08-07T17:35:03.5008903Z ##[endgroup] 2024-08-07T17:35:04.7281798Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-07T17:35:04.7282842Z Configure a credential helper to remove this warning. See 2024-08-07T17:35:04.7283436Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-07T17:35:04.7283832Z 2024-08-07T17:35:04.7284026Z Login Succeeded 2024-08-07T17:35:05.5586341Z Command completed after 1 attempt(s). 2024-08-07T17:35:05.5642700Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-07T17:35:05.5643315Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-07T17:35:05.5643783Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}" 2024-08-07T17:35:05.5649711Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:05.5650114Z env: 2024-08-07T17:35:05.5650330Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:05.5650619Z ##[endgroup] 2024-08-07T17:35:05.5736265Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T17:35:05.5736858Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T17:35:05.5737305Z # shellcheck disable=SC2046 2024-08-07T17:35:05.5737660Z docker stop $(docker ps -q) || true 2024-08-07T17:35:05.5738026Z # Prune all of the docker images 2024-08-07T17:35:05.5738371Z docker system prune -af 2024-08-07T17:35:05.5744606Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:05.5744986Z env: 2024-08-07T17:35:05.5745217Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:05.5745496Z ##[endgroup] 2024-08-07T17:35:05.6023798Z "docker stop" requires at least 1 argument. 2024-08-07T17:35:05.6024248Z See 'docker stop --help'. 2024-08-07T17:35:05.6024430Z 2024-08-07T17:35:05.6024613Z Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] 2024-08-07T17:35:05.6024890Z 2024-08-07T17:35:05.6025027Z Stop one or more running containers 2024-08-07T17:35:05.6184019Z Total reclaimed space: 0B 2024-08-07T17:35:05.6263460Z ##[group]Run set +e 2024-08-07T17:35:05.6263786Z set +e 2024-08-07T17:35:05.6264038Z set -x 2024-08-07T17:35:05.6264263Z  2024-08-07T17:35:05.6264530Z PT_DOMAIN=download.pytorch.org 2024-08-07T17:35:05.6265145Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400, 2024-08-07T17:35:05.6265924Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last 2024-08-07T17:35:05.6266456Z # one is returned at random 2024-08-07T17:35:05.6266861Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1) 2024-08-07T17:35:05.6267253Z  2024-08-07T17:35:05.6267482Z if [ -z "${RESOLVED_IP}" ]; then 2024-08-07T17:35:05.6267932Z  echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..." 2024-08-07T17:35:05.6268625Z  RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1) 2024-08-07T17:35:05.6269021Z  2024-08-07T17:35:05.6269271Z  if [ -z "${RESOLVED_IP}" ]; then 2024-08-07T17:35:05.6269673Z  echo "Couldn't resolve ${PT_DOMAIN}, exiting..." 2024-08-07T17:35:05.6270036Z  exit 1 2024-08-07T17:35:05.6270289Z  fi 2024-08-07T17:35:05.6270646Z fi 2024-08-07T17:35:05.6270855Z  2024-08-07T17:35:05.6271128Z if grep -r "${PT_DOMAIN}" /etc/hosts; then 2024-08-07T17:35:05.6271513Z  # Clean up any old records first 2024-08-07T17:35:05.6271870Z  sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts 2024-08-07T17:35:05.6272208Z fi 2024-08-07T17:35:05.6272427Z  2024-08-07T17:35:05.6272737Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts 2024-08-07T17:35:05.6273150Z cat /etc/hosts 2024-08-07T17:35:05.6279128Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:05.6279575Z env: 2024-08-07T17:35:05.6279813Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:05.6280076Z ##[endgroup] 2024-08-07T17:35:05.6303605Z + PT_DOMAIN=download.pytorch.org 2024-08-07T17:35:05.6309743Z ++ dig -4 +short download.pytorch.org 2024-08-07T17:35:05.6310378Z ++ tail -n1 2024-08-07T17:35:05.6521233Z + RESOLVED_IP=18.160.10.76 2024-08-07T17:35:05.6521631Z + '[' -z 18.160.10.76 ']' 2024-08-07T17:35:05.6521943Z + grep -r download.pytorch.org /etc/hosts 2024-08-07T17:35:05.6533051Z 18.165.98.69 download.pytorch.org 2024-08-07T17:35:05.6534338Z + sudo sed -i /download.pytorch.org/d /etc/hosts 2024-08-07T17:35:05.8863821Z + echo '18.160.10.76 download.pytorch.org' 2024-08-07T17:35:05.8864769Z + sudo tee -a /etc/hosts 2024-08-07T17:35:05.9279650Z 18.160.10.76 download.pytorch.org 2024-08-07T17:35:05.9295896Z + cat /etc/hosts 2024-08-07T17:35:05.9305460Z 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 2024-08-07T17:35:05.9311334Z ::1 localhost6 localhost6.localdomain6 2024-08-07T17:35:05.9311731Z 18.160.10.76 download.pytorch.org 2024-08-07T17:35:05.9449196Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main 2024-08-07T17:35:05.9449689Z with: 2024-08-07T17:35:05.9450367Z docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:05.9451156Z docker-build-dir: .ci/docker 2024-08-07T17:35:05.9451461Z working-directory: . 2024-08-07T17:35:05.9451823Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:05.9452221Z force-push: false 2024-08-07T17:35:05.9452494Z env: 2024-08-07T17:35:05.9452709Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:05.9452985Z ##[endgroup] 2024-08-07T17:35:05.9480966Z ##[group]Run set -ex 2024-08-07T17:35:05.9481293Z set -ex 2024-08-07T17:35:05.9481534Z  2024-08-07T17:35:05.9481946Z # If the docker build directory or the build script doesn't exist, the action will 2024-08-07T17:35:05.9482703Z # gracefully return the docker image name as it is. Pulling docker image in Linux 2024-08-07T17:35:05.9483272Z # job could then download the pre-built image as usual 2024-08-07T17:35:05.9483800Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then 2024-08-07T17:35:05.9484289Z  echo "skip=true" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:05.9484752Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:05.9485158Z  2024-08-07T17:35:05.9485520Z  echo "There is no Docker build script in ${REPO_NAME} repo, skipping..." 2024-08-07T17:35:05.9485975Z  exit 0 2024-08-07T17:35:05.9486197Z else 2024-08-07T17:35:05.9486473Z  echo "skip=false" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:05.9486815Z fi 2024-08-07T17:35:05.9487023Z  2024-08-07T17:35:05.9487535Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then 2024-08-07T17:35:05.9488161Z  # The docker image name already includes the ECR prefix and tag, so we can just 2024-08-07T17:35:05.9488701Z  # use it as it is, but first let's extract the tag 2024-08-07T17:35:05.9489204Z  DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}') 2024-08-07T17:35:05.9489729Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:05.9490227Z  echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:05.9490652Z else 2024-08-07T17:35:05.9490980Z  DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}") 2024-08-07T17:35:05.9491442Z  echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:05.9492102Z  echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:05.9492685Z fi 2024-08-07T17:35:05.9501009Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:05.9501394Z env: 2024-08-07T17:35:05.9501622Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:05.9501888Z REPO_NAME: pytorch 2024-08-07T17:35:05.9502601Z DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:05.9503376Z DOCKER_BUILD_DIR: .ci/docker 2024-08-07T17:35:05.9503764Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:05.9504153Z ##[endgroup] 2024-08-07T17:35:05.9529556Z + [[ ! -d .ci/docker ]] 2024-08-07T17:35:05.9529863Z + [[ ! -f .ci/docker/build.sh ]] 2024-08-07T17:35:05.9530163Z + echo skip=false 2024-08-07T17:35:05.9531186Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]] 2024-08-07T17:35:05.9537587Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:05.9538363Z ++ awk -F '[:,]' '{print $2}' 2024-08-07T17:35:05.9557804Z + DOCKER_TAG=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:05.9558676Z + echo docker-tag=02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:05.9559712Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:05.9595587Z ##[group]Run set +e 2024-08-07T17:35:05.9595915Z set +e 2024-08-07T17:35:05.9596166Z set -x 2024-08-07T17:35:05.9596406Z  2024-08-07T17:35:05.9596616Z login() { 2024-08-07T17:35:05.9597112Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-07T17:35:05.9597656Z } 2024-08-07T17:35:05.9597879Z  2024-08-07T17:35:05.9598100Z retry () { 2024-08-07T17:35:05.9598392Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-07T17:35:05.9598797Z } 2024-08-07T17:35:05.9599020Z  2024-08-07T17:35:05.9599271Z retry login "${DOCKER_REGISTRY}" 2024-08-07T17:35:05.9599576Z  2024-08-07T17:35:05.9599918Z # Check if image already exists, if it does then skip building it 2024-08-07T17:35:05.9600428Z if docker manifest inspect "${DOCKER_IMAGE}"; then 2024-08-07T17:35:05.9600794Z  exit 0 2024-08-07T17:35:05.9601026Z fi 2024-08-07T17:35:05.9601245Z  2024-08-07T17:35:05.9601592Z # NB: This part requires a full checkout. Otherwise, the merge base will 2024-08-07T17:35:05.9602180Z # be empty. The default action would be to continue rebuild the image 2024-08-07T17:35:05.9602711Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then 2024-08-07T17:35:05.9603332Z  # if we're on the base branch then use the parent commit 2024-08-07T17:35:05.9603754Z  MERGE_BASE=$(git rev-parse HEAD~) 2024-08-07T17:35:05.9604084Z else 2024-08-07T17:35:05.9604429Z  # otherwise we're on a PR, so use the most recent base commit 2024-08-07T17:35:05.9604910Z  MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION") 2024-08-07T17:35:05.9605288Z fi 2024-08-07T17:35:05.9605511Z  2024-08-07T17:35:05.9605742Z if [[ -z "${MERGE_BASE}" ]]; then 2024-08-07T17:35:05.9606110Z  echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:05.9606453Z  2024-08-07T17:35:05.9606975Z  echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..." 2024-08-07T17:35:05.9607541Z  exit 0 2024-08-07T17:35:05.9607774Z fi 2024-08-07T17:35:05.9607980Z  2024-08-07T17:35:05.9608296Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then 2024-08-07T17:35:05.9609005Z  echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit" 2024-08-07T17:35:05.9609604Z  exit 1 2024-08-07T17:35:05.9609841Z fi 2024-08-07T17:35:05.9610064Z  2024-08-07T17:35:05.9610426Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}") 2024-08-07T17:35:05.9611099Z # If no image exists but the hash is the same as the previous hash then we should error out here 2024-08-07T17:35:05.9611704Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then 2024-08-07T17:35:05.9612430Z  echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch" 2024-08-07T17:35:05.9613214Z  echo " Will re-build docker image to store in local cache, TTS may be longer" 2024-08-07T17:35:05.9613686Z fi 2024-08-07T17:35:05.9613910Z  2024-08-07T17:35:05.9614163Z echo "rebuild=true" >> "${GITHUB_OUTPUT}" 2024-08-07T17:35:05.9620147Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:05.9620530Z env: 2024-08-07T17:35:05.9620762Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:05.9621052Z DOCKER_BUILD_DIR: .ci/docker 2024-08-07T17:35:05.9621390Z BASE_REVISION: 6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T17:35:05.9622199Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:05.9623006Z DOCKER_TAG: 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:05.9623472Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:05.9623859Z ##[endgroup] 2024-08-07T17:35:05.9648499Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:05.9649210Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:05.9651279Z + aws ecr get-login-password --region us-east-1 2024-08-07T17:35:05.9652895Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:06.5534382Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-07T17:35:06.5535114Z Configure a credential helper to remove this warning. See 2024-08-07T17:35:06.5536081Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-07T17:35:06.5536568Z 2024-08-07T17:35:06.5536679Z Login Succeeded 2024-08-07T17:35:06.5548453Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:06.7602996Z { 2024-08-07T17:35:06.7603513Z "schemaVersion": 2, 2024-08-07T17:35:06.7604169Z "mediaType": "application/vnd.docker.distribution.manifest.v2+json", 2024-08-07T17:35:06.7604675Z "config": { 2024-08-07T17:35:06.7605061Z "mediaType": "application/vnd.docker.container.image.v1+json", 2024-08-07T17:35:06.7605791Z "size": 43802, 2024-08-07T17:35:06.7606263Z "digest": "sha256:eb2e8636b63e804e38c5e56190d8b860c4bf8a9e0794f88825eff958ed6b1e76" 2024-08-07T17:35:06.7607106Z }, 2024-08-07T17:35:06.7607412Z "layers": [ 2024-08-07T17:35:06.7607620Z { 2024-08-07T17:35:06.7607961Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7608423Z "size": 30439866, 2024-08-07T17:35:06.7609185Z "digest": "sha256:9b857f539cb142c9aa2201a17bb8e1cd5cf12edd4a65adf5732fe9f4343964cf" 2024-08-07T17:35:06.7609688Z }, 2024-08-07T17:35:06.7609881Z { 2024-08-07T17:35:06.7610299Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7610981Z "size": 1820, 2024-08-07T17:35:06.7611625Z "digest": "sha256:ceceb076946ee95300822daccb5d3e4e45e6857d34996ef89305d9a078410f8e" 2024-08-07T17:35:06.7612353Z }, 2024-08-07T17:35:06.7612667Z { 2024-08-07T17:35:06.7613271Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7614073Z "size": 313165371, 2024-08-07T17:35:06.7614514Z "digest": "sha256:042461771f00f834b40891f079173616ad761e9e3b2ec2eb25affa3041d71a3d" 2024-08-07T17:35:06.7614977Z }, 2024-08-07T17:35:06.7615178Z { 2024-08-07T17:35:06.7615513Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7615925Z "size": 802, 2024-08-07T17:35:06.7616348Z "digest": "sha256:bac001ac2fee5df3367ff9332bc1fcd706a8de3ad05a1885903f338197619520" 2024-08-07T17:35:06.7616835Z }, 2024-08-07T17:35:06.7617021Z { 2024-08-07T17:35:06.7617356Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7617784Z "size": 106, 2024-08-07T17:35:06.7618183Z "digest": "sha256:4a587220513dc642f8fe95f4ae1b2b643225db8ae8753fa25b68a4f84488c4ab" 2024-08-07T17:35:06.7618662Z }, 2024-08-07T17:35:06.7618864Z { 2024-08-07T17:35:06.7619187Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7619622Z "size": 542, 2024-08-07T17:35:06.7620224Z "digest": "sha256:af8081143f008c5cb1bd61f6d92fde22553cfc007b6a77c2fd2aaf6a9f0088a9" 2024-08-07T17:35:06.7620700Z }, 2024-08-07T17:35:06.7620910Z { 2024-08-07T17:35:06.7621254Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7621698Z "size": 1242, 2024-08-07T17:35:06.7622326Z "digest": "sha256:e7a423dcab0171df0317d2cf65153a533a002a33dce81ae5b0972b06d0838e26" 2024-08-07T17:35:06.7623197Z }, 2024-08-07T17:35:06.7623523Z + exit 0 2024-08-07T17:35:06.7623741Z { 2024-08-07T17:35:06.7624086Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7624504Z "size": 483, 2024-08-07T17:35:06.7624921Z "digest": "sha256:ad26542e0d01609840496a12691483f2c50eb8e23002a205becb5b695f9159fb" 2024-08-07T17:35:06.7625394Z }, 2024-08-07T17:35:06.7625584Z { 2024-08-07T17:35:06.7625923Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7626363Z "size": 110321946, 2024-08-07T17:35:06.7626782Z "digest": "sha256:9e1d4f8784a0033c6a3605f0126d93af125fed0dd0839965c97924ba177e350e" 2024-08-07T17:35:06.7627262Z }, 2024-08-07T17:35:06.7627465Z { 2024-08-07T17:35:06.7627791Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7628215Z "size": 3679, 2024-08-07T17:35:06.7628628Z "digest": "sha256:b1f815d162740bbe496657c7b85acd702ea1818a4d93554f8a822bd964920100" 2024-08-07T17:35:06.7629088Z }, 2024-08-07T17:35:06.7629289Z { 2024-08-07T17:35:06.7629627Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7630036Z "size": 1907, 2024-08-07T17:35:06.7630472Z "digest": "sha256:91f9dddffbee46981d9e74382704c1b53c87f1808350cf1dd518687805d3aa59" 2024-08-07T17:35:06.7630991Z }, 2024-08-07T17:35:06.7631184Z { 2024-08-07T17:35:06.7631527Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7631959Z "size": 699, 2024-08-07T17:35:06.7632475Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-07T17:35:06.7632972Z }, 2024-08-07T17:35:06.7633182Z { 2024-08-07T17:35:06.7633509Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7633945Z "size": 2627946145, 2024-08-07T17:35:06.7634382Z "digest": "sha256:8385f3c14a81343c1f329fe1449447035a2acfc7b7e7c552321cded4206f1e1d" 2024-08-07T17:35:06.7634851Z }, 2024-08-07T17:35:06.7635063Z { 2024-08-07T17:35:06.7635405Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7635822Z "size": 32, 2024-08-07T17:35:06.7636245Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:06.7636732Z }, 2024-08-07T17:35:06.7636923Z { 2024-08-07T17:35:06.7637259Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7637684Z "size": 378, 2024-08-07T17:35:06.7638092Z "digest": "sha256:202c71391ade27762d5e9cafde793db67b4d5e4d6869debe2643d5092101c461" 2024-08-07T17:35:06.7638577Z }, 2024-08-07T17:35:06.7638868Z { 2024-08-07T17:35:06.7639194Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7639620Z "size": 222877, 2024-08-07T17:35:06.7640249Z "digest": "sha256:240ee294d88b3c8decdc8b379b5bfae83e2224ed365151fe2e8026fca366d621" 2024-08-07T17:35:06.7640727Z }, 2024-08-07T17:35:06.7640929Z { 2024-08-07T17:35:06.7641249Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7641677Z "size": 231, 2024-08-07T17:35:06.7642123Z "digest": "sha256:7a823d3d7147e0cae46f715bfd2d9ffd953c6cadae532c88c8c857092e6456da" 2024-08-07T17:35:06.7642593Z }, 2024-08-07T17:35:06.7642796Z { 2024-08-07T17:35:06.7643136Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7643551Z "size": 2933351, 2024-08-07T17:35:06.7643999Z "digest": "sha256:a1b9e8856530ee2f7a1abe6b877bf49121fe149f66fbb6ba309902c35f139fb6" 2024-08-07T17:35:06.7644520Z }, 2024-08-07T17:35:06.7644715Z { 2024-08-07T17:35:06.7645203Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7645637Z "size": 1804, 2024-08-07T17:35:06.7646047Z "digest": "sha256:9c5b585a81167e4e7befcddfd54e7b522e6ed96b7c9a9773ef7099c23000ca20" 2024-08-07T17:35:06.7646543Z }, 2024-08-07T17:35:06.7646749Z { 2024-08-07T17:35:06.7647073Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7647504Z "size": 105, 2024-08-07T17:35:06.7647929Z "digest": "sha256:9cd1bcd9f117daa712ab4916cef270b8b4ecf401bed1df3784d55147457c5de5" 2024-08-07T17:35:06.7648401Z }, 2024-08-07T17:35:06.7648606Z { 2024-08-07T17:35:06.7648946Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7649393Z "size": 563, 2024-08-07T17:35:06.7649812Z "digest": "sha256:2b9731d295d60278e361d491fc274e5d558a89ec5adeb9fdf342d7a8cc17bfec" 2024-08-07T17:35:06.7650301Z }, 2024-08-07T17:35:06.7650490Z { 2024-08-07T17:35:06.7650833Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7651259Z "size": 32, 2024-08-07T17:35:06.7651666Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:06.7652179Z }, 2024-08-07T17:35:06.7652379Z { 2024-08-07T17:35:06.7652699Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7653128Z "size": 104, 2024-08-07T17:35:06.7653561Z "digest": "sha256:8d223c48f5429ef9db659237511d2d5f73345f1f217d316591adc008e8abb5a0" 2024-08-07T17:35:06.7654027Z }, 2024-08-07T17:35:06.7654227Z { 2024-08-07T17:35:06.7654564Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7654977Z "size": 502, 2024-08-07T17:35:06.7655387Z "digest": "sha256:49a0e418a175ac12f1b4b503e80e22679bc321e6163ed5b203af848204f0606a" 2024-08-07T17:35:06.7655862Z }, 2024-08-07T17:35:06.7656049Z { 2024-08-07T17:35:06.7656485Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7656901Z "size": 101109806, 2024-08-07T17:35:06.7657334Z "digest": "sha256:51ed474f6c3273503d24a4b965ff3b7e49e8684501e10413499ecc7249337a86" 2024-08-07T17:35:06.7657804Z }, 2024-08-07T17:35:06.7657994Z { 2024-08-07T17:35:06.7658330Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7658755Z "size": 109, 2024-08-07T17:35:06.7659155Z "digest": "sha256:1c01ecb575c64355bc7f0f73f60b8d212e7a83cd58347d573d0074c2a5926b57" 2024-08-07T17:35:06.7659641Z }, 2024-08-07T17:35:06.7659840Z { 2024-08-07T17:35:06.7660163Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7660590Z "size": 487, 2024-08-07T17:35:06.7661011Z "digest": "sha256:a2630e5fe166ae8ff146e4c14db0bdc1299ed2b6120108351c961adcd2375288" 2024-08-07T17:35:06.7661479Z }, 2024-08-07T17:35:06.7661684Z { 2024-08-07T17:35:06.7662022Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7662442Z "size": 340, 2024-08-07T17:35:06.7662878Z "digest": "sha256:554a42b2a2e61462bb2c2c08ed88f895679ad08a1e9295d040c239217102dc70" 2024-08-07T17:35:06.7663358Z }, 2024-08-07T17:35:06.7663549Z { 2024-08-07T17:35:06.7663887Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7664315Z "size": 103, 2024-08-07T17:35:06.7664722Z "digest": "sha256:c9a20a1fda169765309cb8a927a7ad545bddb403e1ed7db164678f1c0c3b5552" 2024-08-07T17:35:06.7665206Z }, 2024-08-07T17:35:06.7665415Z { 2024-08-07T17:35:06.7665738Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7666165Z "size": 1470, 2024-08-07T17:35:06.7666589Z "digest": "sha256:2c2e35b979f5fc0ecccef6efc157a97f33db01a77958fa33855dc367544a5c46" 2024-08-07T17:35:06.7667142Z }, 2024-08-07T17:35:06.7667457Z { 2024-08-07T17:35:06.7667855Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7668533Z "size": 629993175, 2024-08-07T17:35:06.7669121Z "digest": "sha256:0e3c02dff025006601de33f0303b8c2f62aefdee1d0c34db3a60419ebe1a7e30" 2024-08-07T17:35:06.7669692Z }, 2024-08-07T17:35:06.7670058Z { 2024-08-07T17:35:06.7670446Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7670961Z "size": 160, 2024-08-07T17:35:06.7671539Z "digest": "sha256:2c2cb9893903884e0cf968da484f3b97fb62e1a881f8d68b437c240c0f2f25d1" 2024-08-07T17:35:06.7672072Z }, 2024-08-07T17:35:06.7672349Z { 2024-08-07T17:35:06.7672836Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7673316Z "size": 2209, 2024-08-07T17:35:06.7673814Z "digest": "sha256:ec0a42784a49a934dcd7d8c99dbe058a3828080e6716a12cba391210891574f3" 2024-08-07T17:35:06.7674445Z }, 2024-08-07T17:35:06.7674703Z { 2024-08-07T17:35:06.7675115Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7675702Z "size": 150, 2024-08-07T17:35:06.7676172Z "digest": "sha256:a20886f4e5372f586ef3e2728653d1969c6d898d8eae0667de17908891ac278f" 2024-08-07T17:35:06.7676728Z }, 2024-08-07T17:35:06.7677081Z { 2024-08-07T17:35:06.7677473Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7677986Z "size": 643, 2024-08-07T17:35:06.7678563Z "digest": "sha256:af1fb4fa4eed1c385edc3660525fa1e46533ec917477c141d430c07c735ff8da" 2024-08-07T17:35:06.7679201Z }, 2024-08-07T17:35:06.7679496Z { 2024-08-07T17:35:06.7680005Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7680483Z "size": 32, 2024-08-07T17:35:06.7680990Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:06.7681623Z }, 2024-08-07T17:35:06.7681877Z { 2024-08-07T17:35:06.7682302Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7682877Z "size": 162, 2024-08-07T17:35:06.7683482Z "digest": "sha256:de13eba39cb5160281708ab7fda3d1c4e26a9edb7793e8fddc3355c37a2c2bb6" 2024-08-07T17:35:06.7684061Z }, 2024-08-07T17:35:06.7684412Z { 2024-08-07T17:35:06.7684811Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7685319Z "size": 109, 2024-08-07T17:35:06.7685900Z "digest": "sha256:ac851722c4ad33e4e6ad7a9580ff1f61e91dc5ec40f5e5ff5629cd32c8a6dff9" 2024-08-07T17:35:06.7686455Z }, 2024-08-07T17:35:06.7686741Z { 2024-08-07T17:35:06.7687238Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7687725Z "size": 102, 2024-08-07T17:35:06.7688218Z "digest": "sha256:225d8e1484f0c72016ef99e76b8309a3472d6fb528b34633eb25c00394e350d9" 2024-08-07T17:35:06.7688857Z }, 2024-08-07T17:35:06.7689115Z { 2024-08-07T17:35:06.7689543Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7690140Z "size": 421, 2024-08-07T17:35:06.7690607Z "digest": "sha256:0b92275d203788d5d990e589228173aa2fe9709737328fd78f2aa02bf159a285" 2024-08-07T17:35:06.7691206Z }, 2024-08-07T17:35:06.7691572Z { 2024-08-07T17:35:06.7691972Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7692480Z "size": 32, 2024-08-07T17:35:06.7693051Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:06.7693593Z }, 2024-08-07T17:35:06.7693876Z { 2024-08-07T17:35:06.7694389Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7694869Z "size": 111, 2024-08-07T17:35:06.7695369Z "digest": "sha256:86d521d46977bce4fd8bbc492e3f0310b7f26ffc5e36fafe97225a7ca8fa9ffa" 2024-08-07T17:35:06.7696005Z }, 2024-08-07T17:35:06.7696297Z { 2024-08-07T17:35:06.7696727Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7697301Z "size": 471, 2024-08-07T17:35:06.7697778Z "digest": "sha256:3914906be0cb7fd68abac1a779b275af42993ccb4ed1bd1015f467e170d0b1f4" 2024-08-07T17:35:06.7698354Z }, 2024-08-07T17:35:06.7698692Z { 2024-08-07T17:35:06.7699178Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7699694Z "size": 32, 2024-08-07T17:35:06.7700256Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:06.7700850Z }, 2024-08-07T17:35:06.7701086Z { 2024-08-07T17:35:06.7701565Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7702087Z "size": 112, 2024-08-07T17:35:06.7702579Z "digest": "sha256:193407c5262c69087b764527f63b3d7e47e029f4f0ddf8f6a3fe4baaa7f29c57" 2024-08-07T17:35:06.7703209Z }, 2024-08-07T17:35:06.7703477Z { 2024-08-07T17:35:06.7703897Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7704465Z "size": 562, 2024-08-07T17:35:06.7704944Z "digest": "sha256:adf9d181cfbca5b0544f23797995d4ea6500cc4a635db9f7e18954c9044e9f4a" 2024-08-07T17:35:06.7705512Z }, 2024-08-07T17:35:06.7705878Z { 2024-08-07T17:35:06.7706264Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7706775Z "size": 43163921, 2024-08-07T17:35:06.7707558Z "digest": "sha256:736c4af6f5277292080e42bfc2c7edaf8412ef1b795cf6fcbf2e95beffbb7ed2" 2024-08-07T17:35:06.7708150Z }, 2024-08-07T17:35:06.7708383Z { 2024-08-07T17:35:06.7708879Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7709402Z "size": 106, 2024-08-07T17:35:06.7709847Z "digest": "sha256:801521a42094555983e4a2674ddbb195243f6fe0d15ad1aeaab4aa88d81d22c4" 2024-08-07T17:35:06.7710501Z }, 2024-08-07T17:35:06.7710798Z { 2024-08-07T17:35:06.7711166Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7711793Z "size": 294, 2024-08-07T17:35:06.7712317Z "digest": "sha256:ed7b11c2ed84119d1bd1ebf8b7317c20e0f79da24ca754a02cb6775c00a17ae1" 2024-08-07T17:35:06.7712915Z }, 2024-08-07T17:35:06.7713327Z { 2024-08-07T17:35:06.7713867Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7714469Z "size": 32, 2024-08-07T17:35:06.7714998Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:06.7715598Z }, 2024-08-07T17:35:06.7715864Z { 2024-08-07T17:35:06.7716319Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7716860Z "size": 106, 2024-08-07T17:35:06.7717341Z "digest": "sha256:5c6f2f3fe7c213d4b989fac53b07197b9f9ab99fe04360200826bcb2ca7ac66e" 2024-08-07T17:35:06.7717941Z }, 2024-08-07T17:35:06.7718250Z { 2024-08-07T17:35:06.7718643Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7719333Z "size": 424, 2024-08-07T17:35:06.7719961Z "digest": "sha256:3886e8f57dc887a87bb710fbda080ef4fd521f1ed2e560a895b000c05f6e9b0a" 2024-08-07T17:35:06.7720525Z }, 2024-08-07T17:35:06.7720845Z { 2024-08-07T17:35:06.7721287Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7721812Z "size": 19308828, 2024-08-07T17:35:06.7722384Z "digest": "sha256:cc822658f193feb19cb990795b3114bca9c649bd221c0b1aab684e652069c295" 2024-08-07T17:35:06.7722996Z }, 2024-08-07T17:35:06.7723257Z { 2024-08-07T17:35:06.7723717Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7724246Z "size": 108, 2024-08-07T17:35:06.7724761Z "digest": "sha256:cca044783521353ee93922f8ea943e74964e2a9815b43751866ff494b8d4d8d9" 2024-08-07T17:35:06.7725434Z }, 2024-08-07T17:35:06.7725740Z { 2024-08-07T17:35:06.7726201Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7726808Z "size": 438, 2024-08-07T17:35:06.7727328Z "digest": "sha256:0eb80a0ac0006a45aa6b4581263cc1613ddf9545bd4a1d5a544310efe898ad81" 2024-08-07T17:35:06.7727876Z }, 2024-08-07T17:35:06.7728206Z { 2024-08-07T17:35:06.7728700Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7729199Z "size": 699, 2024-08-07T17:35:06.7729844Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-07T17:35:06.7730477Z }, 2024-08-07T17:35:06.7730772Z { 2024-08-07T17:35:06.7731232Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7731827Z "size": 141, 2024-08-07T17:35:06.7732328Z "digest": "sha256:a9f11b2b3622f5530c5cfefe973e96e028323fab1ff02e7a6337eb86fadebeb5" 2024-08-07T17:35:06.7732987Z }, 2024-08-07T17:35:06.7733297Z { 2024-08-07T17:35:06.7733689Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7734243Z "size": 135, 2024-08-07T17:35:06.7734761Z "digest": "sha256:d7e94279f8b2fe549e3f23637275d9dca8c831563ce84f1d44117eb297ddc871" 2024-08-07T17:35:06.7735297Z }, 2024-08-07T17:35:06.7735613Z { 2024-08-07T17:35:06.7736102Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7736588Z "size": 32, 2024-08-07T17:35:06.7737137Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:06.7737740Z }, 2024-08-07T17:35:06.7738002Z { 2024-08-07T17:35:06.7738458Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7738993Z "size": 190, 2024-08-07T17:35:06.7739474Z "digest": "sha256:073f669362431eb57b23c427bcb2898ba4bb050cf4f9d37c09e270ceb847ccd6" 2024-08-07T17:35:06.7740316Z }, 2024-08-07T17:35:06.7740634Z { 2024-08-07T17:35:06.7741046Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7741600Z "size": 1210, 2024-08-07T17:35:06.7742173Z "digest": "sha256:719a1c2abcfd7e4113aa2f03adeb54350a5e381c720688ea53bc0e18401ad33d" 2024-08-07T17:35:06.7742734Z }, 2024-08-07T17:35:06.7743067Z { 2024-08-07T17:35:06.7743511Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7744003Z "size": 699, 2024-08-07T17:35:06.7744549Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-07T17:35:06.7745285Z }, 2024-08-07T17:35:06.7745563Z { 2024-08-07T17:35:06.7746033Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7746557Z "size": 138, 2024-08-07T17:35:06.7747034Z "digest": "sha256:8e1355d516c5e0527c25402c664fc648a8a2b814090e6ee0fdd60f49a8826ac1" 2024-08-07T17:35:06.7747634Z }, 2024-08-07T17:35:06.7747982Z { 2024-08-07T17:35:06.7748403Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7748954Z "size": 1842834361, 2024-08-07T17:35:06.7749499Z "digest": "sha256:4d5ca8eafd7982e3b256257abe6dfce2dc314e52d34defe80c8298de227a5f19" 2024-08-07T17:35:06.7750071Z }, 2024-08-07T17:35:06.7750391Z { 2024-08-07T17:35:06.7750826Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7751321Z "size": 153, 2024-08-07T17:35:06.7751851Z "digest": "sha256:73831405501a21694a29eaa05a64c05dbc545d9c5ba676a5b5f204fa2300291a" 2024-08-07T17:35:06.7752436Z }, 2024-08-07T17:35:06.7752699Z { 2024-08-07T17:35:06.7753214Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7753755Z "size": 846, 2024-08-07T17:35:06.7754226Z "digest": "sha256:0b48a1547ae65f413b4be264aad75996efd1ab0fac4107687531843d1ee54e5c" 2024-08-07T17:35:06.7754830Z }, 2024-08-07T17:35:06.7755141Z { 2024-08-07T17:35:06.7755530Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7756072Z "size": 699, 2024-08-07T17:35:06.7756603Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-07T17:35:06.7769091Z }, 2024-08-07T17:35:06.7769482Z { 2024-08-07T17:35:06.7769907Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7770342Z "size": 139, 2024-08-07T17:35:06.7770776Z "digest": "sha256:794615be0ac3201bd2845c4cd2e2f9e9651f1fba0142b0c74dccfda79fee82b1" 2024-08-07T17:35:06.7771263Z }, 2024-08-07T17:35:06.7771470Z { 2024-08-07T17:35:06.7771988Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7772425Z "size": 32, 2024-08-07T17:35:06.7772834Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:06.7773323Z }, 2024-08-07T17:35:06.7773533Z { 2024-08-07T17:35:06.7773858Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7774295Z "size": 157, 2024-08-07T17:35:06.7774710Z "digest": "sha256:ad492c23051a703625b3e779faf07bba8b9bea291956c20e744bba89898af6be" 2024-08-07T17:35:06.7775176Z }, 2024-08-07T17:35:06.7775417Z { 2024-08-07T17:35:06.7775755Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7776170Z "size": 905, 2024-08-07T17:35:06.7776622Z "digest": "sha256:fc956bf4e5b49119cab7cc7bda867c61bed76c76c1e113dfa7c7b184251f8173" 2024-08-07T17:35:06.7777114Z }, 2024-08-07T17:35:06.7777300Z { 2024-08-07T17:35:06.7777641Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7778070Z "size": 699, 2024-08-07T17:35:06.7778476Z "digest": "sha256:03fb78f207e3cbcfedfc7619a1d60fa576d3293d508a7e97dc09a5e7a9c65ab0" 2024-08-07T17:35:06.7778962Z }, 2024-08-07T17:35:06.7779164Z { 2024-08-07T17:35:06.7779496Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7779938Z "size": 134, 2024-08-07T17:35:06.7780348Z "digest": "sha256:874491333fb5b81d9cbbd71520163a6baf07e6f011dc148b1634b94e301972b8" 2024-08-07T17:35:06.7780812Z }, 2024-08-07T17:35:06.7781015Z { 2024-08-07T17:35:06.7781353Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7781769Z "size": 32, 2024-08-07T17:35:06.7782191Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:06.7782681Z }, 2024-08-07T17:35:06.7782873Z { 2024-08-07T17:35:06.7783208Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7783721Z "size": 156, 2024-08-07T17:35:06.7784134Z "digest": "sha256:62fd235ebb62f15f85f93abd31bdfcd2445df1b40f97d0b474a4879e05a47b0e" 2024-08-07T17:35:06.7784626Z }, 2024-08-07T17:35:06.7784816Z { 2024-08-07T17:35:06.7785161Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7785590Z "size": 1577, 2024-08-07T17:35:06.7785987Z "digest": "sha256:42646dcb0889a48f164507344b56ed0254f1c64ca68b2f974c7fa9f2569f1925" 2024-08-07T17:35:06.7786463Z }, 2024-08-07T17:35:06.7786667Z { 2024-08-07T17:35:06.7786987Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7787415Z "size": 32, 2024-08-07T17:35:06.7787833Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:06.7788301Z }, 2024-08-07T17:35:06.7788504Z { 2024-08-07T17:35:06.7788874Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7789291Z "size": 128, 2024-08-07T17:35:06.7789712Z "digest": "sha256:5df8c6bd8177de4d452857fef1396a5c9574f50e37b342b1a192f89ec36fb4e5" 2024-08-07T17:35:06.7790194Z }, 2024-08-07T17:35:06.7790382Z { 2024-08-07T17:35:06.7790735Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7791167Z "size": 377, 2024-08-07T17:35:06.7791573Z "digest": "sha256:96381dfde5a802fe7b27ba3ae3b5f8589f88c170c5bae6863be97a37fa86688b" 2024-08-07T17:35:06.7792059Z }, 2024-08-07T17:35:06.7792259Z { 2024-08-07T17:35:06.7792581Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7793066Z "size": 32, 2024-08-07T17:35:06.7793482Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:06.7793949Z }, 2024-08-07T17:35:06.7794151Z { 2024-08-07T17:35:06.7794484Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7794893Z "size": 104, 2024-08-07T17:35:06.7795307Z "digest": "sha256:d0dba839d4c041ba853621262a7e730c3699058d43cfe7d6f31ec311708acc73" 2024-08-07T17:35:06.7795866Z }, 2024-08-07T17:35:06.7796056Z { 2024-08-07T17:35:06.7796398Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7796826Z "size": 1839, 2024-08-07T17:35:06.7797232Z "digest": "sha256:c1d4cac78640fc7d364011da807a363d6b248a2c46ccf99a6e1488c4974c17e3" 2024-08-07T17:35:06.7797713Z }, 2024-08-07T17:35:06.7797901Z { 2024-08-07T17:35:06.7798240Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7798670Z "size": 7529628, 2024-08-07T17:35:06.7799190Z "digest": "sha256:67cb12cd081ccde6047c7cbbdd4d99c4499d4b12d3db421c307ef6d901ff0c81" 2024-08-07T17:35:06.7799680Z }, 2024-08-07T17:35:06.7799887Z { 2024-08-07T17:35:06.7800224Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7800685Z "size": 106, 2024-08-07T17:35:06.7801105Z "digest": "sha256:b431724b7afe431951242ade99ee881ce85e51dc44dd1f8c0c1ed1710de62f68" 2024-08-07T17:35:06.7801579Z }, 2024-08-07T17:35:06.7801814Z { 2024-08-07T17:35:06.7802153Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7802567Z "size": 161, 2024-08-07T17:35:06.7802978Z "digest": "sha256:190dccfa687dc1e698f27965d8065a183168d7c66361505f368765456ff9f4bc" 2024-08-07T17:35:06.7803454Z }, 2024-08-07T17:35:06.7803647Z { 2024-08-07T17:35:06.7803984Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7804411Z "size": 7942, 2024-08-07T17:35:06.7804850Z "digest": "sha256:b01bea4b6634f47c51ede2b4a95ceade4e61356ca29551168b493ae6d4492f44" 2024-08-07T17:35:06.7805339Z }, 2024-08-07T17:35:06.7805544Z { 2024-08-07T17:35:06.7805869Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7806294Z "size": 8065, 2024-08-07T17:35:06.7806723Z "digest": "sha256:77fc3d1eca2c044fa3497f2e035f9ca2e5ffcd8f16b810f1173ae52af23b71a7" 2024-08-07T17:35:06.7807289Z }, 2024-08-07T17:35:06.7807495Z { 2024-08-07T17:35:06.7807841Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7808259Z "size": 302, 2024-08-07T17:35:06.7808676Z "digest": "sha256:96d6dcd01b9f6c5557ba405d028442d808dd329f4f93e75f2c3ba0554e703a52" 2024-08-07T17:35:06.7809154Z }, 2024-08-07T17:35:06.7809340Z { 2024-08-07T17:35:06.7809673Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7810100Z "size": 32, 2024-08-07T17:35:06.7810504Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:06.7810984Z }, 2024-08-07T17:35:06.7811186Z { 2024-08-07T17:35:06.7811506Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7811928Z "size": 108, 2024-08-07T17:35:06.7812327Z "digest": "sha256:79ef14b93271d6a0315533761cb955f79fba8ee77b9a73ba823923e7dd1613f7" 2024-08-07T17:35:06.7812804Z }, 2024-08-07T17:35:06.7813002Z { 2024-08-07T17:35:06.7813327Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7813752Z "size": 54145774, 2024-08-07T17:35:06.7814176Z "digest": "sha256:45103ff2548064a8f9e94bdb31d8bc89e396f9880773cc06e0540dac48d20e10" 2024-08-07T17:35:06.7814636Z }, 2024-08-07T17:35:06.7814840Z { 2024-08-07T17:35:06.7815175Z "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip", 2024-08-07T17:35:06.7815585Z "size": 32, 2024-08-07T17:35:06.7816002Z "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1" 2024-08-07T17:35:06.7816490Z } 2024-08-07T17:35:06.7816678Z ] 2024-08-07T17:35:06.7816882Z } 2024-08-07T17:35:06.7913777Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/} 2024-08-07T17:35:06.7914172Z tag=${ECR_DOCKER_IMAGE##*/} 2024-08-07T17:35:06.7914586Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}" 2024-08-07T17:35:06.7921486Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:06.7921879Z env: 2024-08-07T17:35:06.7922108Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:06.7922829Z ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:06.7923580Z ##[endgroup] 2024-08-07T17:35:06.7949975Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-jammy-py3.8-gcc11-02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:06.8025155Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main 2024-08-07T17:35:06.8025622Z with: 2024-08-07T17:35:06.8026281Z docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:06.8027134Z docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:06.8027554Z env: 2024-08-07T17:35:06.8027768Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:06.8028047Z ##[endgroup] 2024-08-07T17:35:06.8074783Z ##[group]Run set -x 2024-08-07T17:35:06.8075094Z set -x 2024-08-07T17:35:06.8075324Z set +e 2024-08-07T17:35:06.8075562Z  2024-08-07T17:35:06.8075789Z login() { 2024-08-07T17:35:06.8076270Z  aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1" 2024-08-07T17:35:06.8076810Z } 2024-08-07T17:35:06.8077031Z  2024-08-07T17:35:06.8077299Z retry () { 2024-08-07T17:35:06.8077574Z  $* || (sleep 1 && $*) || (sleep 2 && $*) 2024-08-07T17:35:06.8077905Z } 2024-08-07T17:35:06.8078129Z  2024-08-07T17:35:06.8078363Z retry login "${DOCKER_REGISTRY}" 2024-08-07T17:35:06.8078771Z  2024-08-07T17:35:06.8079001Z set -e 2024-08-07T17:35:06.8079346Z # ignore output since only exit code is used for conditional 2024-08-07T17:35:06.8079860Z # only pull docker image if it's not available locally 2024-08-07T17:35:06.8080581Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then 2024-08-07T17:35:06.8081084Z  retry docker pull "${DOCKER_IMAGE}" 2024-08-07T17:35:06.8081419Z fi 2024-08-07T17:35:06.8087441Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:35:06.8087830Z env: 2024-08-07T17:35:06.8088066Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:35:06.8088787Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:06.8089681Z DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:06.8090082Z ##[endgroup] 2024-08-07T17:35:06.8113607Z + set +e 2024-08-07T17:35:06.8113966Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:06.8114400Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:06.8117183Z + aws ecr get-login-password --region us-east-1 2024-08-07T17:35:06.8118158Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com 2024-08-07T17:35:07.3891111Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json. 2024-08-07T17:35:07.3891795Z Configure a credential helper to remove this warning. See 2024-08-07T17:35:07.3892528Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store 2024-08-07T17:35:07.3892933Z 2024-08-07T17:35:07.3893040Z Login Succeeded 2024-08-07T17:35:07.3905392Z + set -e 2024-08-07T17:35:07.3906338Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:07.4039854Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:07.4041250Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:35:07.6905065Z 02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9: Pulling from pytorch/pytorch-linux-jammy-py3.8-gcc11 2024-08-07T17:35:07.6911464Z 9b857f539cb1: Pulling fs layer 2024-08-07T17:35:07.6912462Z ceceb076946e: Pulling fs layer 2024-08-07T17:35:07.6912886Z 042461771f00: Pulling fs layer 2024-08-07T17:35:07.6913329Z bac001ac2fee: Pulling fs layer 2024-08-07T17:35:07.6913783Z 4a587220513d: Pulling fs layer 2024-08-07T17:35:07.6914559Z af8081143f00: Pulling fs layer 2024-08-07T17:35:07.6915004Z e7a423dcab01: Pulling fs layer 2024-08-07T17:35:07.6915378Z ad26542e0d01: Pulling fs layer 2024-08-07T17:35:07.6915830Z 9e1d4f8784a0: Pulling fs layer 2024-08-07T17:35:07.6916205Z bac001ac2fee: Waiting 2024-08-07T17:35:07.6916567Z b1f815d16274: Pulling fs layer 2024-08-07T17:35:07.6916980Z 4a587220513d: Waiting 2024-08-07T17:35:07.6917365Z e7a423dcab01: Waiting 2024-08-07T17:35:07.6917731Z af8081143f00: Waiting 2024-08-07T17:35:07.6918093Z 91f9dddffbee: Pulling fs layer 2024-08-07T17:35:07.6918409Z ad26542e0d01: Waiting 2024-08-07T17:35:07.6918847Z 03fb78f207e3: Pulling fs layer 2024-08-07T17:35:07.6919281Z 9e1d4f8784a0: Waiting 2024-08-07T17:35:07.6919679Z b1f815d16274: Waiting 2024-08-07T17:35:07.6920066Z 91f9dddffbee: Waiting 2024-08-07T17:35:07.6920473Z 8385f3c14a81: Pulling fs layer 2024-08-07T17:35:07.6920986Z 4f4fb700ef54: Pulling fs layer 2024-08-07T17:35:07.6921476Z 03fb78f207e3: Waiting 2024-08-07T17:35:07.6921797Z 8385f3c14a81: Waiting 2024-08-07T17:35:07.6922077Z 202c71391ade: Pulling fs layer 2024-08-07T17:35:07.6922375Z 240ee294d88b: Pulling fs layer 2024-08-07T17:35:07.6922644Z 4f4fb700ef54: Waiting 2024-08-07T17:35:07.6922903Z 7a823d3d7147: Pulling fs layer 2024-08-07T17:35:07.6923194Z a1b9e8856530: Pulling fs layer 2024-08-07T17:35:07.6923464Z 202c71391ade: Waiting 2024-08-07T17:35:07.6923760Z 9c5b585a8116: Pulling fs layer 2024-08-07T17:35:07.6924036Z 9cd1bcd9f117: Pulling fs layer 2024-08-07T17:35:07.6924334Z 2b9731d295d6: Pulling fs layer 2024-08-07T17:35:07.6924612Z 7a823d3d7147: Waiting 2024-08-07T17:35:07.6925020Z a1b9e8856530: Waiting 2024-08-07T17:35:07.6925273Z 9c5b585a8116: Waiting 2024-08-07T17:35:07.6925534Z 8d223c48f542: Pulling fs layer 2024-08-07T17:35:07.6925813Z 49a0e418a175: Pulling fs layer 2024-08-07T17:35:07.6926094Z 2b9731d295d6: Waiting 2024-08-07T17:35:07.6926348Z 8d223c48f542: Waiting 2024-08-07T17:35:07.6926594Z 51ed474f6c32: Pulling fs layer 2024-08-07T17:35:07.6926887Z 1c01ecb575c6: Pulling fs layer 2024-08-07T17:35:07.6927170Z 49a0e418a175: Waiting 2024-08-07T17:35:07.6927412Z a2630e5fe166: Pulling fs layer 2024-08-07T17:35:07.6927700Z 554a42b2a2e6: Pulling fs layer 2024-08-07T17:35:07.6927978Z 51ed474f6c32: Waiting 2024-08-07T17:35:07.6928209Z a2630e5fe166: Waiting 2024-08-07T17:35:07.6928466Z c9a20a1fda16: Pulling fs layer 2024-08-07T17:35:07.6928749Z 1c01ecb575c6: Waiting 2024-08-07T17:35:07.6928993Z 2c2e35b979f5: Pulling fs layer 2024-08-07T17:35:07.6929273Z 554a42b2a2e6: Waiting 2024-08-07T17:35:07.6929506Z c9a20a1fda16: Waiting 2024-08-07T17:35:07.6929769Z 0e3c02dff025: Pulling fs layer 2024-08-07T17:35:07.6930061Z 2c2cb9893903: Pulling fs layer 2024-08-07T17:35:07.6930337Z ec0a42784a49: Pulling fs layer 2024-08-07T17:35:07.6930627Z a20886f4e537: Pulling fs layer 2024-08-07T17:35:07.6930906Z 0e3c02dff025: Waiting 2024-08-07T17:35:07.6931140Z 2c2cb9893903: Waiting 2024-08-07T17:35:07.6931384Z ec0a42784a49: Waiting 2024-08-07T17:35:07.6931646Z af1fb4fa4eed: Pulling fs layer 2024-08-07T17:35:07.6931920Z a20886f4e537: Waiting 2024-08-07T17:35:07.6932173Z af1fb4fa4eed: Waiting 2024-08-07T17:35:07.6932438Z de13eba39cb5: Pulling fs layer 2024-08-07T17:35:07.6932721Z ac851722c4ad: Pulling fs layer 2024-08-07T17:35:07.6933012Z 225d8e1484f0: Pulling fs layer 2024-08-07T17:35:07.6933301Z 0b92275d2037: Pulling fs layer 2024-08-07T17:35:07.6933565Z ac851722c4ad: Waiting 2024-08-07T17:35:07.6933823Z 86d521d46977: Pulling fs layer 2024-08-07T17:35:07.6934107Z de13eba39cb5: Waiting 2024-08-07T17:35:07.6934358Z 3914906be0cb: Pulling fs layer 2024-08-07T17:35:07.6934643Z 225d8e1484f0: Waiting 2024-08-07T17:35:07.6934898Z 0b92275d2037: Waiting 2024-08-07T17:35:07.6935141Z 193407c5262c: Pulling fs layer 2024-08-07T17:35:07.6935421Z 3914906be0cb: Waiting 2024-08-07T17:35:07.6935652Z 86d521d46977: Waiting 2024-08-07T17:35:07.6935914Z adf9d181cfbc: Pulling fs layer 2024-08-07T17:35:07.6936199Z 193407c5262c: Waiting 2024-08-07T17:35:07.6936441Z 736c4af6f527: Pulling fs layer 2024-08-07T17:35:07.6936732Z 801521a42094: Pulling fs layer 2024-08-07T17:35:07.6937134Z adf9d181cfbc: Waiting 2024-08-07T17:35:07.6937409Z ed7b11c2ed84: Pulling fs layer 2024-08-07T17:35:07.6937712Z 5c6f2f3fe7c2: Pulling fs layer 2024-08-07T17:35:07.6938012Z 3886e8f57dc8: Pulling fs layer 2024-08-07T17:35:07.6938294Z cc822658f193: Pulling fs layer 2024-08-07T17:35:07.6938592Z cca044783521: Pulling fs layer 2024-08-07T17:35:07.6938878Z 736c4af6f527: Waiting 2024-08-07T17:35:07.6939132Z 0eb80a0ac000: Pulling fs layer 2024-08-07T17:35:07.6939506Z ed7b11c2ed84: Waiting 2024-08-07T17:35:07.6939865Z 5c6f2f3fe7c2: Waiting 2024-08-07T17:35:07.6940412Z 801521a42094: Waiting 2024-08-07T17:35:07.6940689Z a9f11b2b3622: Pulling fs layer 2024-08-07T17:35:07.6940985Z 3886e8f57dc8: Waiting 2024-08-07T17:35:07.6941222Z cc822658f193: Waiting 2024-08-07T17:35:07.6941642Z d7e94279f8b2: Pulling fs layer 2024-08-07T17:35:07.6941916Z 0eb80a0ac000: Waiting 2024-08-07T17:35:07.6942175Z a9f11b2b3622: Waiting 2024-08-07T17:35:07.6942438Z 073f66936243: Pulling fs layer 2024-08-07T17:35:07.6942728Z 719a1c2abcfd: Pulling fs layer 2024-08-07T17:35:07.6943028Z 8e1355d516c5: Pulling fs layer 2024-08-07T17:35:07.6943319Z d7e94279f8b2: Waiting 2024-08-07T17:35:07.6943565Z 4d5ca8eafd79: Pulling fs layer 2024-08-07T17:35:07.6943842Z 073f66936243: Waiting 2024-08-07T17:35:07.6944095Z 73831405501a: Pulling fs layer 2024-08-07T17:35:07.6944368Z 0b48a1547ae6: Pulling fs layer 2024-08-07T17:35:07.6944643Z 8e1355d516c5: Waiting 2024-08-07T17:35:07.6944893Z 4d5ca8eafd79: Waiting 2024-08-07T17:35:07.6945141Z 794615be0ac3: Pulling fs layer 2024-08-07T17:35:07.6945589Z ad492c23051a: Pulling fs layer 2024-08-07T17:35:07.6945885Z fc956bf4e5b4: Pulling fs layer 2024-08-07T17:35:07.6946149Z 794615be0ac3: Waiting 2024-08-07T17:35:07.6946406Z 874491333fb5: Pulling fs layer 2024-08-07T17:35:07.6946685Z ad492c23051a: Waiting 2024-08-07T17:35:07.6946916Z 874491333fb5: Waiting 2024-08-07T17:35:07.6947173Z 62fd235ebb62: Pulling fs layer 2024-08-07T17:35:07.6947463Z 42646dcb0889: Pulling fs layer 2024-08-07T17:35:07.6947731Z 73831405501a: Waiting 2024-08-07T17:35:07.6947992Z 5df8c6bd8177: Pulling fs layer 2024-08-07T17:35:07.6948270Z 96381dfde5a8: Pulling fs layer 2024-08-07T17:35:07.6948558Z 62fd235ebb62: Waiting 2024-08-07T17:35:07.6948823Z d0dba839d4c0: Pulling fs layer 2024-08-07T17:35:07.6949095Z 42646dcb0889: Waiting 2024-08-07T17:35:07.6949354Z c1d4cac78640: Pulling fs layer 2024-08-07T17:35:07.6949668Z 67cb12cd081c: Pulling fs layer 2024-08-07T17:35:07.6949951Z d0dba839d4c0: Waiting 2024-08-07T17:35:07.6950202Z b431724b7afe: Pulling fs layer 2024-08-07T17:35:07.6950488Z c1d4cac78640: Waiting 2024-08-07T17:35:07.6950742Z 190dccfa687d: Pulling fs layer 2024-08-07T17:35:07.6951025Z 96381dfde5a8: Waiting 2024-08-07T17:35:07.6951276Z 67cb12cd081c: Waiting 2024-08-07T17:35:07.6951522Z b01bea4b6634: Pulling fs layer 2024-08-07T17:35:07.6951818Z 77fc3d1eca2c: Pulling fs layer 2024-08-07T17:35:07.6952205Z 96d6dcd01b9f: Pulling fs layer 2024-08-07T17:35:07.6952502Z 0b48a1547ae6: Waiting 2024-08-07T17:35:07.6952752Z fc956bf4e5b4: Waiting 2024-08-07T17:35:07.6953018Z 79ef14b93271: Pulling fs layer 2024-08-07T17:35:07.6953291Z 45103ff25480: Pulling fs layer 2024-08-07T17:35:07.6953569Z 96d6dcd01b9f: Waiting 2024-08-07T17:35:07.6953823Z 77fc3d1eca2c: Waiting 2024-08-07T17:35:07.6954057Z 45103ff25480: Waiting 2024-08-07T17:35:07.6954641Z 79ef14b93271: Waiting 2024-08-07T17:35:07.6954941Z b01bea4b6634: Waiting 2024-08-07T17:35:07.7712651Z ceceb076946e: Verifying Checksum 2024-08-07T17:35:07.7713137Z ceceb076946e: Download complete 2024-08-07T17:35:07.8412104Z bac001ac2fee: Download complete 2024-08-07T17:35:07.9203101Z 4a587220513d: Verifying Checksum 2024-08-07T17:35:07.9203499Z 4a587220513d: Download complete 2024-08-07T17:35:07.9878122Z af8081143f00: Download complete 2024-08-07T17:35:08.0625444Z 9b857f539cb1: Download complete 2024-08-07T17:35:08.0743351Z e7a423dcab01: Verifying Checksum 2024-08-07T17:35:08.0743963Z e7a423dcab01: Download complete 2024-08-07T17:35:08.1384690Z ad26542e0d01: Verifying Checksum 2024-08-07T17:35:08.1385473Z ad26542e0d01: Download complete 2024-08-07T17:35:08.2146220Z b1f815d16274: Download complete 2024-08-07T17:35:08.2965761Z 91f9dddffbee: Verifying Checksum 2024-08-07T17:35:08.2966272Z 91f9dddffbee: Download complete 2024-08-07T17:35:08.3680783Z 03fb78f207e3: Download complete 2024-08-07T17:35:09.0401591Z 9b857f539cb1: Pull complete 2024-08-07T17:35:09.0603648Z ceceb076946e: Pull complete 2024-08-07T17:35:09.2349369Z 9e1d4f8784a0: Verifying Checksum 2024-08-07T17:35:09.2349936Z 9e1d4f8784a0: Download complete 2024-08-07T17:35:09.2450148Z 4f4fb700ef54: Download complete 2024-08-07T17:35:09.3162604Z 202c71391ade: Verifying Checksum 2024-08-07T17:35:09.3163088Z 202c71391ade: Download complete 2024-08-07T17:35:09.4268879Z 240ee294d88b: Verifying Checksum 2024-08-07T17:35:09.4269340Z 240ee294d88b: Download complete 2024-08-07T17:35:09.4993545Z 7a823d3d7147: Verifying Checksum 2024-08-07T17:35:09.4994075Z 7a823d3d7147: Download complete 2024-08-07T17:35:09.5993694Z a1b9e8856530: Verifying Checksum 2024-08-07T17:35:09.5994177Z a1b9e8856530: Download complete 2024-08-07T17:35:09.6797162Z 9c5b585a8116: Download complete 2024-08-07T17:35:09.7476535Z 9cd1bcd9f117: Verifying Checksum 2024-08-07T17:35:09.7477139Z 9cd1bcd9f117: Download complete 2024-08-07T17:35:09.8163989Z 2b9731d295d6: Download complete 2024-08-07T17:35:09.8940989Z 8d223c48f542: Verifying Checksum 2024-08-07T17:35:09.8941370Z 8d223c48f542: Download complete 2024-08-07T17:35:09.9795028Z 49a0e418a175: Verifying Checksum 2024-08-07T17:35:09.9795539Z 49a0e418a175: Download complete 2024-08-07T17:35:10.8838132Z 042461771f00: Verifying Checksum 2024-08-07T17:35:10.8839046Z 042461771f00: Download complete 2024-08-07T17:35:10.9622960Z 1c01ecb575c6: Verifying Checksum 2024-08-07T17:35:10.9623414Z 1c01ecb575c6: Download complete 2024-08-07T17:35:11.0326664Z a2630e5fe166: Verifying Checksum 2024-08-07T17:35:11.0327157Z a2630e5fe166: Download complete 2024-08-07T17:35:11.0442558Z 51ed474f6c32: Verifying Checksum 2024-08-07T17:35:11.0443173Z 51ed474f6c32: Download complete 2024-08-07T17:35:11.1099444Z 554a42b2a2e6: Verifying Checksum 2024-08-07T17:35:11.1100052Z 554a42b2a2e6: Download complete 2024-08-07T17:35:11.1197210Z c9a20a1fda16: Verifying Checksum 2024-08-07T17:35:11.1197884Z c9a20a1fda16: Download complete 2024-08-07T17:35:11.2120717Z 2c2e35b979f5: Verifying Checksum 2024-08-07T17:35:11.2121320Z 2c2e35b979f5: Download complete 2024-08-07T17:35:11.2942000Z 2c2cb9893903: Download complete 2024-08-07T17:35:11.3690136Z ec0a42784a49: Verifying Checksum 2024-08-07T17:35:11.3690697Z ec0a42784a49: Download complete 2024-08-07T17:35:11.4650564Z a20886f4e537: Verifying Checksum 2024-08-07T17:35:11.4651173Z a20886f4e537: Download complete 2024-08-07T17:35:11.5486738Z af1fb4fa4eed: Download complete 2024-08-07T17:35:11.6281869Z de13eba39cb5: Verifying Checksum 2024-08-07T17:35:11.6282452Z de13eba39cb5: Download complete 2024-08-07T17:35:11.6970703Z ac851722c4ad: Download complete 2024-08-07T17:35:11.7625224Z 225d8e1484f0: Verifying Checksum 2024-08-07T17:35:11.7625809Z 225d8e1484f0: Download complete 2024-08-07T17:35:11.8438193Z 0b92275d2037: Verifying Checksum 2024-08-07T17:35:11.8438830Z 0b92275d2037: Download complete 2024-08-07T17:35:11.9185633Z 86d521d46977: Verifying Checksum 2024-08-07T17:35:11.9186292Z 86d521d46977: Download complete 2024-08-07T17:35:11.9954649Z 3914906be0cb: Verifying Checksum 2024-08-07T17:35:11.9955250Z 3914906be0cb: Download complete 2024-08-07T17:35:12.0835524Z 193407c5262c: Verifying Checksum 2024-08-07T17:35:12.0836061Z 193407c5262c: Download complete 2024-08-07T17:35:12.1644182Z adf9d181cfbc: Verifying Checksum 2024-08-07T17:35:12.1644630Z adf9d181cfbc: Download complete 2024-08-07T17:35:12.6479287Z 736c4af6f527: Verifying Checksum 2024-08-07T17:35:12.6479910Z 736c4af6f527: Download complete 2024-08-07T17:35:12.7188389Z 801521a42094: Verifying Checksum 2024-08-07T17:35:12.7188966Z 801521a42094: Download complete 2024-08-07T17:35:12.8061650Z ed7b11c2ed84: Verifying Checksum 2024-08-07T17:35:12.8064088Z ed7b11c2ed84: Download complete 2024-08-07T17:35:12.8901637Z 5c6f2f3fe7c2: Download complete 2024-08-07T17:35:12.9790340Z 3886e8f57dc8: Verifying Checksum 2024-08-07T17:35:12.9790979Z 3886e8f57dc8: Download complete 2024-08-07T17:35:13.2247667Z cc822658f193: Verifying Checksum 2024-08-07T17:35:13.2248131Z cc822658f193: Download complete 2024-08-07T17:35:13.3144645Z cca044783521: Verifying Checksum 2024-08-07T17:35:13.3145051Z cca044783521: Download complete 2024-08-07T17:35:13.3927572Z 0eb80a0ac000: Verifying Checksum 2024-08-07T17:35:13.3928202Z 0eb80a0ac000: Download complete 2024-08-07T17:35:13.4647829Z a9f11b2b3622: Download complete 2024-08-07T17:35:13.5257574Z d7e94279f8b2: Verifying Checksum 2024-08-07T17:35:13.5258152Z d7e94279f8b2: Download complete 2024-08-07T17:35:13.6023592Z 073f66936243: Download complete 2024-08-07T17:35:13.6727742Z 719a1c2abcfd: Verifying Checksum 2024-08-07T17:35:13.6728347Z 719a1c2abcfd: Download complete 2024-08-07T17:35:13.7595193Z 8e1355d516c5: Download complete 2024-08-07T17:35:17.5865897Z 0e3c02dff025: Verifying Checksum 2024-08-07T17:35:17.5866490Z 0e3c02dff025: Download complete 2024-08-07T17:35:17.6662281Z 73831405501a: Download complete 2024-08-07T17:35:17.7502639Z 0b48a1547ae6: Download complete 2024-08-07T17:35:17.8318762Z 794615be0ac3: Verifying Checksum 2024-08-07T17:35:17.8319455Z 794615be0ac3: Download complete 2024-08-07T17:35:17.9009958Z ad492c23051a: Verifying Checksum 2024-08-07T17:35:17.9010550Z ad492c23051a: Download complete 2024-08-07T17:35:17.9675268Z fc956bf4e5b4: Verifying Checksum 2024-08-07T17:35:17.9675937Z fc956bf4e5b4: Download complete 2024-08-07T17:35:18.0477626Z 874491333fb5: Download complete 2024-08-07T17:35:18.1233301Z 62fd235ebb62: Verifying Checksum 2024-08-07T17:35:18.1233916Z 62fd235ebb62: Download complete 2024-08-07T17:35:18.2110194Z 42646dcb0889: Download complete 2024-08-07T17:35:18.2891941Z 5df8c6bd8177: Verifying Checksum 2024-08-07T17:35:18.2892392Z 5df8c6bd8177: Download complete 2024-08-07T17:35:18.3670763Z 96381dfde5a8: Download complete 2024-08-07T17:35:18.4517513Z d0dba839d4c0: Verifying Checksum 2024-08-07T17:35:18.4518063Z d0dba839d4c0: Download complete 2024-08-07T17:35:18.5319862Z c1d4cac78640: Download complete 2024-08-07T17:35:18.6596971Z 67cb12cd081c: Download complete 2024-08-07T17:35:18.7285285Z b431724b7afe: Verifying Checksum 2024-08-07T17:35:18.7285674Z b431724b7afe: Download complete 2024-08-07T17:35:18.8346300Z 190dccfa687d: Verifying Checksum 2024-08-07T17:35:18.8346767Z 190dccfa687d: Download complete 2024-08-07T17:35:18.9136758Z b01bea4b6634: Verifying Checksum 2024-08-07T17:35:18.9137229Z b01bea4b6634: Download complete 2024-08-07T17:35:18.9999406Z 77fc3d1eca2c: Verifying Checksum 2024-08-07T17:35:18.9999883Z 77fc3d1eca2c: Download complete 2024-08-07T17:35:19.0882894Z 96d6dcd01b9f: Verifying Checksum 2024-08-07T17:35:19.0883344Z 96d6dcd01b9f: Download complete 2024-08-07T17:35:19.1737306Z 79ef14b93271: Download complete 2024-08-07T17:35:21.0754614Z 45103ff25480: Verifying Checksum 2024-08-07T17:35:21.0755053Z 45103ff25480: Download complete 2024-08-07T17:35:22.8515799Z 042461771f00: Pull complete 2024-08-07T17:35:23.0334571Z bac001ac2fee: Pull complete 2024-08-07T17:35:23.2228920Z 4a587220513d: Pull complete 2024-08-07T17:35:23.4454210Z af8081143f00: Pull complete 2024-08-07T17:35:23.6268095Z e7a423dcab01: Pull complete 2024-08-07T17:35:23.8070282Z ad26542e0d01: Pull complete 2024-08-07T17:35:44.7218485Z 9e1d4f8784a0: Pull complete 2024-08-07T17:35:44.8550194Z 4d5ca8eafd79: Verifying Checksum 2024-08-07T17:35:44.8551744Z 4d5ca8eafd79: Download complete 2024-08-07T17:35:44.9098032Z b1f815d16274: Pull complete 2024-08-07T17:35:45.0914851Z 91f9dddffbee: Pull complete 2024-08-07T17:35:45.1986599Z 03fb78f207e3: Pull complete 2024-08-07T17:35:46.2204556Z 8385f3c14a81: Download complete 2024-08-07T17:36:34.7290368Z 8385f3c14a81: Pull complete 2024-08-07T17:36:34.7459004Z 4f4fb700ef54: Pull complete 2024-08-07T17:36:34.7624610Z 202c71391ade: Pull complete 2024-08-07T17:36:34.7966960Z 240ee294d88b: Pull complete 2024-08-07T17:36:34.8131095Z 7a823d3d7147: Pull complete 2024-08-07T17:36:34.8674234Z a1b9e8856530: Pull complete 2024-08-07T17:36:34.8839180Z 9c5b585a8116: Pull complete 2024-08-07T17:36:34.9031075Z 9cd1bcd9f117: Pull complete 2024-08-07T17:36:34.9186639Z 2b9731d295d6: Pull complete 2024-08-07T17:36:34.9482197Z 8d223c48f542: Pull complete 2024-08-07T17:36:34.9638454Z 49a0e418a175: Pull complete 2024-08-07T17:36:36.9769326Z 51ed474f6c32: Pull complete 2024-08-07T17:36:37.1610350Z 1c01ecb575c6: Pull complete 2024-08-07T17:36:37.3612040Z a2630e5fe166: Pull complete 2024-08-07T17:36:37.5380757Z 554a42b2a2e6: Pull complete 2024-08-07T17:36:37.7236422Z c9a20a1fda16: Pull complete 2024-08-07T17:36:37.9150029Z 2c2e35b979f5: Pull complete 2024-08-07T17:36:46.8129480Z 0e3c02dff025: Pull complete 2024-08-07T17:36:47.0104288Z 2c2cb9893903: Pull complete 2024-08-07T17:36:47.1920396Z ec0a42784a49: Pull complete 2024-08-07T17:36:47.3406476Z a20886f4e537: Pull complete 2024-08-07T17:36:47.3610458Z af1fb4fa4eed: Pull complete 2024-08-07T17:36:47.4023440Z de13eba39cb5: Pull complete 2024-08-07T17:36:47.4259110Z ac851722c4ad: Pull complete 2024-08-07T17:36:47.4710627Z 225d8e1484f0: Pull complete 2024-08-07T17:36:47.5338147Z 0b92275d2037: Pull complete 2024-08-07T17:36:47.6425157Z 86d521d46977: Pull complete 2024-08-07T17:36:47.6670307Z 3914906be0cb: Pull complete 2024-08-07T17:36:47.7184948Z 193407c5262c: Pull complete 2024-08-07T17:36:47.7465328Z adf9d181cfbc: Pull complete 2024-08-07T17:36:49.3457429Z 736c4af6f527: Pull complete 2024-08-07T17:36:49.3630784Z 801521a42094: Pull complete 2024-08-07T17:36:49.3805314Z ed7b11c2ed84: Pull complete 2024-08-07T17:36:49.4207259Z 5c6f2f3fe7c2: Pull complete 2024-08-07T17:36:49.4392162Z 3886e8f57dc8: Pull complete 2024-08-07T17:36:49.7030762Z cc822658f193: Pull complete 2024-08-07T17:36:49.7283058Z cca044783521: Pull complete 2024-08-07T17:36:49.7464987Z 0eb80a0ac000: Pull complete 2024-08-07T17:36:49.7834767Z a9f11b2b3622: Pull complete 2024-08-07T17:36:49.8018547Z d7e94279f8b2: Pull complete 2024-08-07T17:36:49.8388124Z 073f66936243: Pull complete 2024-08-07T17:36:49.8575974Z 719a1c2abcfd: Pull complete 2024-08-07T17:36:49.8932697Z 8e1355d516c5: Pull complete 2024-08-07T17:37:22.1433312Z 4d5ca8eafd79: Pull complete 2024-08-07T17:37:22.2980322Z 73831405501a: Pull complete 2024-08-07T17:37:22.4130073Z 0b48a1547ae6: Pull complete 2024-08-07T17:37:22.5432213Z 794615be0ac3: Pull complete 2024-08-07T17:37:22.6162210Z ad492c23051a: Pull complete 2024-08-07T17:37:22.6710768Z fc956bf4e5b4: Pull complete 2024-08-07T17:37:22.8279904Z 874491333fb5: Pull complete 2024-08-07T17:37:22.9180465Z 62fd235ebb62: Pull complete 2024-08-07T17:37:22.9623561Z 42646dcb0889: Pull complete 2024-08-07T17:37:23.1386369Z 5df8c6bd8177: Pull complete 2024-08-07T17:37:23.2198941Z 96381dfde5a8: Pull complete 2024-08-07T17:37:23.5918785Z d0dba839d4c0: Pull complete 2024-08-07T17:37:23.7721097Z c1d4cac78640: Pull complete 2024-08-07T17:37:24.2870312Z 67cb12cd081c: Pull complete 2024-08-07T17:37:24.4712497Z b431724b7afe: Pull complete 2024-08-07T17:37:24.6464233Z 190dccfa687d: Pull complete 2024-08-07T17:37:24.7149171Z b01bea4b6634: Pull complete 2024-08-07T17:37:24.8017677Z 77fc3d1eca2c: Pull complete 2024-08-07T17:37:24.9881331Z 96d6dcd01b9f: Pull complete 2024-08-07T17:37:25.3584162Z 79ef14b93271: Pull complete 2024-08-07T17:37:27.6551439Z 45103ff25480: Pull complete 2024-08-07T17:37:27.8410911Z Digest: sha256:c75221497aa22766802555fccb76014708414eebd47578b213b43b3f8f4cddb9 2024-08-07T17:37:27.8548907Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:37:27.8624883Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:37:27.8669416Z ##[group]Run echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-07T17:37:27.8670128Z echo "IN_ARC_RUNNER=$([ -f /.inarc ] && echo true || echo false)" >> "$GITHUB_OUTPUT" 2024-08-07T17:37:27.8678432Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:27.8678841Z env: 2024-08-07T17:37:27.8679063Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:27.8679341Z ##[endgroup] 2024-08-07T17:37:27.8894418Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-07T17:37:27.8895041Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 2024-08-07T17:37:27.8895570Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 & 2024-08-07T17:37:27.8896048Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}" 2024-08-07T17:37:27.8901831Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:27.8902227Z env: 2024-08-07T17:37:27.8902445Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:27.8902722Z ##[endgroup] 2024-08-07T17:37:28.6124826Z Defaulting to user installation because normal site-packages is not writeable 2024-08-07T17:37:28.6371407Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.9/site-packages (5.9.1) 2024-08-07T17:37:28.6377280Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.9/site-packages (11.525.84) 2024-08-07T17:37:28.7924428Z Prepare all required actions 2024-08-07T17:37:28.7925261Z Getting action download info 2024-08-07T17:37:28.9451228Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6) 2024-08-07T17:37:29.2958363Z Download action repository 'actions/download-artifact@v3' (SHA:9bc31d5ccc31df68ecc42ccf4149144866c47d8a) 2024-08-07T17:37:29.5137441Z ##[group]Run ./.github/actions/download-build-artifacts 2024-08-07T17:37:29.5137843Z with: 2024-08-07T17:37:29.5138071Z name: linux-jammy-py3.8-gcc11 2024-08-07T17:37:29.5138376Z s3-bucket: gha-artifacts 2024-08-07T17:37:29.5138644Z env: 2024-08-07T17:37:29.5138864Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:29.5139140Z ##[endgroup] 2024-08-07T17:37:29.5269120Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-07T17:37:29.5269474Z with: 2024-08-07T17:37:29.5269724Z name: linux-jammy-py3.8-gcc11 2024-08-07T17:37:29.5270023Z s3-bucket: gha-artifacts 2024-08-07T17:37:29.5270366Z region: us-east-1 2024-08-07T17:37:29.5270614Z env: 2024-08-07T17:37:29.5270844Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:29.5271103Z ##[endgroup] 2024-08-07T17:37:30.2328609Z (node:268787) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-07T17:37:30.2329138Z 2024-08-07T17:37:30.2329365Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-07T17:37:30.2329940Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-07T17:37:30.2330523Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-07T17:37:30.3050137Z Found 1 objects with prefix pytorch/pytorch/10288745067/linux-jammy-py3.8-gcc11/ 2024-08-07T17:37:30.3050876Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-07T17:37:34.0287056Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip 2024-08-07T17:37:34.0294792Z Artifact download has finished successfully 2024-08-07T17:37:34.0460446Z ##[group]Run unzip -o artifacts.zip 2024-08-07T17:37:34.0460812Z unzip -o artifacts.zip 2024-08-07T17:37:34.0466599Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:34.0466984Z env: 2024-08-07T17:37:34.0467219Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:34.0467496Z ##[endgroup] 2024-08-07T17:37:34.0739421Z Archive: artifacts.zip 2024-08-07T17:37:34.0740138Z creating: dist/ 2024-08-07T17:37:35.0161530Z inflating: dist/torch-2.5.0a0+git016588f-cp38-cp38-linux_x86_64.whl 2024-08-07T17:37:35.0162380Z creating: build/custom_test_artifacts/ 2024-08-07T17:37:35.0163065Z creating: build/custom_test_artifacts/custom-op-build/ 2024-08-07T17:37:35.0163914Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/ 2024-08-07T17:37:35.0164916Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/ 2024-08-07T17:37:35.0166183Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-07T17:37:35.0167423Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/ 2024-08-07T17:37:35.0168641Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-07T17:37:35.0169944Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-07T17:37:35.0171257Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-07T17:37:35.0172728Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-07T17:37:35.0174268Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-07T17:37:35.0175576Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-07T17:37:35.0176906Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-07T17:37:35.0178698Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-07T17:37:35.0180281Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-07T17:37:35.0181819Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-07T17:37:35.0183577Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-07T17:37:35.0185168Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-07T17:37:35.0186830Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-07T17:37:35.0188221Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/ 2024-08-07T17:37:35.0189482Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache 2024-08-07T17:37:35.0190684Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/ 2024-08-07T17:37:35.0192063Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts 2024-08-07T17:37:35.0193655Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make 2024-08-07T17:37:35.0195094Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make 2024-08-07T17:37:35.0196484Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt 2024-08-07T17:37:35.0197884Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake 2024-08-07T17:37:35.0199387Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make 2024-08-07T17:37:35.0200815Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake 2024-08-07T17:37:35.0202198Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make 2024-08-07T17:37:35.0203602Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make 2024-08-07T17:37:35.0214332Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d 2024-08-07T17:37:35.0345058Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o 2024-08-07T17:37:35.0346513Z creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/ 2024-08-07T17:37:35.0348039Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts 2024-08-07T17:37:35.0349752Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make 2024-08-07T17:37:35.0351254Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make 2024-08-07T17:37:35.0352776Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt 2024-08-07T17:37:35.0354263Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake 2024-08-07T17:37:35.0355725Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make 2024-08-07T17:37:35.0357228Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake 2024-08-07T17:37:35.0358769Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make 2024-08-07T17:37:35.0360235Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make 2024-08-07T17:37:35.0373534Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d 2024-08-07T17:37:35.0450238Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o 2024-08-07T17:37:35.0451972Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-07T17:37:35.0453460Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt 2024-08-07T17:37:35.0454737Z extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks 2024-08-07T17:37:35.0456125Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2 2024-08-07T17:37:35.0457317Z inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake 2024-08-07T17:37:35.0475165Z inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt 2024-08-07T17:37:35.0476223Z inflating: build/custom_test_artifacts/custom-op-build/Makefile 2024-08-07T17:37:35.0477227Z inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake 2024-08-07T17:37:35.0574401Z inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so 2024-08-07T17:37:35.0633824Z inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops 2024-08-07T17:37:35.0634776Z creating: build/custom_test_artifacts/jit-hook-build/ 2024-08-07T17:37:35.0635601Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/ 2024-08-07T17:37:35.0636619Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/ 2024-08-07T17:37:35.0639216Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-07T17:37:35.0640615Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/ 2024-08-07T17:37:35.0641814Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-07T17:37:35.0643069Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-07T17:37:35.0644344Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-07T17:37:35.0645732Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-07T17:37:35.0647250Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-07T17:37:35.0648536Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-07T17:37:35.0649834Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-07T17:37:35.0651367Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-07T17:37:35.0652910Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-07T17:37:35.0654437Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-07T17:37:35.0655888Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-07T17:37:35.0657439Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-07T17:37:35.0659072Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-07T17:37:35.0660415Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/ 2024-08-07T17:37:35.0661632Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache 2024-08-07T17:37:35.0662942Z creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/ 2024-08-07T17:37:35.0664358Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts 2024-08-07T17:37:35.0665959Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make 2024-08-07T17:37:35.0667431Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make 2024-08-07T17:37:35.0669038Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt 2024-08-07T17:37:35.0670449Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake 2024-08-07T17:37:35.0671880Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make 2024-08-07T17:37:35.0673447Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake 2024-08-07T17:37:35.0675094Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make 2024-08-07T17:37:35.0676463Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make 2024-08-07T17:37:35.0686335Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d 2024-08-07T17:37:35.0746715Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o 2024-08-07T17:37:35.0748331Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-07T17:37:35.0749782Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt 2024-08-07T17:37:35.0751060Z extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks 2024-08-07T17:37:35.0752246Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2 2024-08-07T17:37:35.0753425Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake 2024-08-07T17:37:35.0754618Z inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt 2024-08-07T17:37:35.0755605Z inflating: build/custom_test_artifacts/jit-hook-build/Makefile 2024-08-07T17:37:35.0756671Z inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake 2024-08-07T17:37:35.0803161Z inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks 2024-08-07T17:37:35.0804179Z creating: build/custom_test_artifacts/custom-backend-build/ 2024-08-07T17:37:35.0805127Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/ 2024-08-07T17:37:35.0806213Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/ 2024-08-07T17:37:35.0807769Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml 2024-08-07T17:37:35.0809124Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/ 2024-08-07T17:37:35.0810433Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeSystem.cmake 2024-08-07T17:37:35.0811808Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/ 2024-08-07T17:37:35.0813189Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/tmp/ 2024-08-07T17:37:35.0814742Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/CMakeCCompilerId.c 2024-08-07T17:37:35.0816337Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdC/a.out 2024-08-07T17:37:35.0817745Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/ 2024-08-07T17:37:35.0819149Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/tmp/ 2024-08-07T17:37:35.0820822Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/CMakeCXXCompilerId.cpp 2024-08-07T17:37:35.0822515Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CompilerIdCXX/a.out 2024-08-07T17:37:35.0824079Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_C.bin 2024-08-07T17:37:35.0825709Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCCompiler.cmake 2024-08-07T17:37:35.0827620Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeDetermineCompilerABI_CXX.bin 2024-08-07T17:37:35.0829315Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.26.4/CMakeCXXCompiler.cmake 2024-08-07T17:37:35.0830774Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/ 2024-08-07T17:37:35.0832142Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache 2024-08-07T17:37:35.0833655Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/ 2024-08-07T17:37:35.0835181Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts 2024-08-07T17:37:35.0836853Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make 2024-08-07T17:37:35.0838596Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make 2024-08-07T17:37:35.0840460Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt 2024-08-07T17:37:35.0842123Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake 2024-08-07T17:37:35.0843706Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make 2024-08-07T17:37:35.0845271Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake 2024-08-07T17:37:35.0846944Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make 2024-08-07T17:37:35.0848534Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make 2024-08-07T17:37:35.0850192Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d 2024-08-07T17:37:35.0957921Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o 2024-08-07T17:37:35.0959587Z creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/ 2024-08-07T17:37:35.0961293Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts 2024-08-07T17:37:35.0963168Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make 2024-08-07T17:37:35.0964957Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make 2024-08-07T17:37:35.0966609Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt 2024-08-07T17:37:35.0968252Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake 2024-08-07T17:37:35.0969915Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make 2024-08-07T17:37:35.0971561Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake 2024-08-07T17:37:35.0973218Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make 2024-08-07T17:37:35.0974854Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make 2024-08-07T17:37:35.0986530Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d 2024-08-07T17:37:35.1038198Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o 2024-08-07T17:37:35.1040298Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-07T17:37:35.1041847Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt 2024-08-07T17:37:35.1043467Z extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks 2024-08-07T17:37:35.1044725Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2 2024-08-07T17:37:35.1046022Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake 2024-08-07T17:37:35.1047285Z inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt 2024-08-07T17:37:35.1048535Z inflating: build/custom_test_artifacts/custom-backend-build/Makefile 2024-08-07T17:37:35.1049712Z inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake 2024-08-07T17:37:35.1146594Z inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so 2024-08-07T17:37:35.1189535Z inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend 2024-08-07T17:37:35.1190684Z creating: build/lib/ 2024-08-07T17:37:35.1191183Z inflating: build/lib/libclog.a 2024-08-07T17:37:35.1200891Z inflating: build/lib/libpthreadpool.a 2024-08-07T17:37:35.1208936Z inflating: build/lib/libcpuinfo_internals.a 2024-08-07T17:37:35.1217322Z inflating: build/lib/libcpuinfo.a 2024-08-07T17:37:35.1281271Z inflating: build/lib/libgtest.a 2024-08-07T17:37:35.1361114Z inflating: build/lib/libprotobuf-lite.a 2024-08-07T17:37:35.1368630Z inflating: build/lib/libittnotify.a 2024-08-07T17:37:35.1438839Z inflating: build/lib/libbenchmark.a 2024-08-07T17:37:35.1466112Z inflating: build/lib/libtensorpipe_uv.a 2024-08-07T17:37:35.1527689Z inflating: build/lib/libasmjit.a 2024-08-07T17:37:35.1950973Z inflating: build/lib/libprotobuf.a 2024-08-07T17:37:35.2059681Z inflating: build/lib/libgloo.a 2024-08-07T17:37:35.2080020Z inflating: build/lib/libfmt.a 2024-08-07T17:37:35.2080792Z inflating: build/lib/libfoxi_loader.a 2024-08-07T17:37:35.2082728Z inflating: build/lib/libtorch_global_deps.so 2024-08-07T17:37:35.2175107Z inflating: build/lib/libc10.so 2024-08-07T17:37:35.2177606Z inflating: build/lib/libnnpack_reference_layers.a 2024-08-07T17:37:35.2196387Z inflating: build/lib/libpytorch_qnnpack.a 2024-08-07T17:37:35.2670124Z inflating: build/lib/libprotoc.a 2024-08-07T17:37:35.2687638Z inflating: build/lib/libgmock.a 2024-08-07T17:37:35.2688312Z inflating: build/lib/libbenchmark_main.a 2024-08-07T17:37:35.2689273Z inflating: build/lib/libgtest_main.a 2024-08-07T17:37:35.2707194Z inflating: build/lib/libnnpack.a 2024-08-07T17:37:36.1854012Z inflating: build/lib/libdnnl.a 2024-08-07T17:37:36.2379385Z inflating: build/lib/libtensorpipe.a 2024-08-07T17:37:36.2379876Z inflating: build/lib/libgmock_main.a 2024-08-07T17:37:36.3372035Z inflating: build/lib/libfbgemm.a 2024-08-07T17:37:36.3645003Z inflating: build/lib/libkineto.a 2024-08-07T17:37:36.3683387Z inflating: build/lib/libonnx_proto.a 2024-08-07T17:37:36.3858069Z inflating: build/lib/libXNNPACK.a 2024-08-07T17:37:36.4518911Z inflating: build/lib/libonnx.a 2024-08-07T17:37:38.6792933Z inflating: build/lib/libtorch_cpu.so 2024-08-07T17:37:38.6793563Z inflating: build/lib/libtorch.so 2024-08-07T17:37:38.6798123Z inflating: build/lib/libunbox_lib.a 2024-08-07T17:37:38.6801842Z inflating: build/lib/libshm.so 2024-08-07T17:37:38.6872662Z inflating: build/lib/libtorchbind_test.so 2024-08-07T17:37:38.6891823Z inflating: build/lib/libjitbackend_test.so 2024-08-07T17:37:38.6915817Z inflating: build/lib/libbackend_with_compiler.so 2024-08-07T17:37:38.6940031Z inflating: build/lib/libaoti_custom_ops.so 2024-08-07T17:37:38.8757081Z inflating: build/lib/libtorch_python.so 2024-08-07T17:37:38.8790278Z inflating: build/lib/libnnapi_backend.so 2024-08-07T17:37:38.8790670Z creating: build/bin/ 2024-08-07T17:37:38.8791027Z creating: build/bin/CMakeFiles/ 2024-08-07T17:37:38.8791679Z inflating: build/bin/CMakeFiles/CMakeDirectoryInformation.cmake 2024-08-07T17:37:38.8792553Z extracting: build/bin/CMakeFiles/progress.marks 2024-08-07T17:37:38.8793016Z inflating: build/bin/Makefile 2024-08-07T17:37:38.8793368Z inflating: build/bin/cmake_install.cmake 2024-08-07T17:37:38.8793726Z inflating: build/bin/CTestTestfile.cmake 2024-08-07T17:37:38.8844512Z inflating: build/bin/c10_CompileTimeFunctionPointer_test 2024-08-07T17:37:38.8896416Z inflating: build/bin/c10_DeviceGuard_test 2024-08-07T17:37:38.8947579Z inflating: build/bin/c10_Device_test 2024-08-07T17:37:38.9007206Z inflating: build/bin/c10_DispatchKeySet_test 2024-08-07T17:37:38.9061348Z inflating: build/bin/c10_Scalar_test 2024-08-07T17:37:38.9110537Z inflating: build/bin/c10_StreamGuard_test 2024-08-07T17:37:38.9162065Z inflating: build/bin/c10_SymInt_test 2024-08-07T17:37:38.9216621Z inflating: build/bin/c10_InlineDeviceGuard_test 2024-08-07T17:37:38.9272351Z inflating: build/bin/c10_SizesAndStrides_test 2024-08-07T17:37:38.9328373Z inflating: build/bin/c10_InlineStreamGuard_test 2024-08-07T17:37:38.9377248Z inflating: build/bin/c10_ConstexprCrc_test 2024-08-07T17:37:38.9430337Z inflating: build/bin/c10_Bitset_test 2024-08-07T17:37:38.9500681Z inflating: build/bin/c10_cow_test 2024-08-07T17:37:38.9551034Z inflating: build/bin/c10_DeadlockDetection_test 2024-08-07T17:37:38.9601929Z inflating: build/bin/c10_Half_test 2024-08-07T17:37:38.9657776Z inflating: build/bin/c10_LeftRight_test 2024-08-07T17:37:38.9713135Z inflating: build/bin/c10_Metaprogramming_test 2024-08-07T17:37:38.9765092Z inflating: build/bin/c10_TypeIndex_test 2024-08-07T17:37:38.9816597Z inflating: build/bin/c10_TypeList_test 2024-08-07T17:37:38.9867416Z inflating: build/bin/c10_Synchronized_test 2024-08-07T17:37:38.9922911Z inflating: build/bin/c10_ThreadLocal_test 2024-08-07T17:37:38.9972154Z inflating: build/bin/c10_TypeTraits_test 2024-08-07T17:37:39.0024798Z inflating: build/bin/c10_accumulate_test 2024-08-07T17:37:39.0080623Z inflating: build/bin/c10_bfloat16_test 2024-08-07T17:37:39.0131986Z inflating: build/bin/c10_bit_cast_test 2024-08-07T17:37:39.0189610Z inflating: build/bin/c10_complex_math_test 2024-08-07T17:37:39.0242969Z inflating: build/bin/c10_exception_test 2024-08-07T17:37:39.0299020Z inflating: build/bin/c10_complex_test 2024-08-07T17:37:39.0350051Z inflating: build/bin/c10_flags_test 2024-08-07T17:37:39.0401197Z inflating: build/bin/c10_generic_math_test 2024-08-07T17:37:39.0453262Z inflating: build/bin/c10_irange_test 2024-08-07T17:37:39.0612916Z inflating: build/bin/c10_intrusive_ptr_test 2024-08-07T17:37:39.0667271Z inflating: build/bin/c10_lazy_test 2024-08-07T17:37:39.0725171Z inflating: build/bin/c10_logging_test 2024-08-07T17:37:39.0800624Z inflating: build/bin/c10_optional_test 2024-08-07T17:37:39.0863545Z inflating: build/bin/c10_ordered_preserving_dict_test 2024-08-07T17:37:39.0918006Z inflating: build/bin/c10_registry_test 2024-08-07T17:37:39.1066442Z inflating: build/bin/c10_small_vector_test 2024-08-07T17:37:39.1119530Z inflating: build/bin/c10_string_util_test 2024-08-07T17:37:39.1172603Z inflating: build/bin/c10_ssize_test 2024-08-07T17:37:39.1232773Z inflating: build/bin/c10_string_view_test 2024-08-07T17:37:39.1284620Z inflating: build/bin/c10_tempfile_test 2024-08-07T17:37:39.1332362Z inflating: build/bin/c10_intrusive_ptr_benchmark 2024-08-07T17:37:39.1389640Z inflating: build/bin/c10_typeid_test 2024-08-07T17:37:39.1831410Z inflating: build/bin/protoc-3.13.0.0 2024-08-07T17:37:39.2272728Z inflating: build/bin/protoc 2024-08-07T17:37:39.2595539Z inflating: build/bin/vec_test_all_types_DEFAULT 2024-08-07T17:37:39.2936460Z inflating: build/bin/vec_test_all_types_AVX512 2024-08-07T17:37:39.3289883Z inflating: build/bin/vec_test_all_types_AVX2 2024-08-07T17:37:39.3343427Z inflating: build/bin/BackoffTest 2024-08-07T17:37:39.3397736Z inflating: build/bin/FileStoreTest 2024-08-07T17:37:39.3455407Z inflating: build/bin/TCPStoreTest 2024-08-07T17:37:39.3509683Z inflating: build/bin/HashStoreTest 2024-08-07T17:37:39.3512611Z inflating: build/bin/example_allreduce 2024-08-07T17:37:39.3579026Z inflating: build/bin/ProcessGroupGlooTest 2024-08-07T17:37:39.3634004Z inflating: build/bin/test_dist_autograd 2024-08-07T17:37:39.3636775Z inflating: build/bin/parallel_benchmark 2024-08-07T17:37:39.3705546Z inflating: build/bin/test_cpp_rpc 2024-08-07T17:37:39.3772205Z inflating: build/bin/test_mobile_nnc 2024-08-07T17:37:39.3781297Z inflating: build/bin/aot_model_compiler_test 2024-08-07T17:37:39.3839632Z inflating: build/bin/static_runtime_bench 2024-08-07T17:37:39.4171391Z inflating: build/bin/test_lazy 2024-08-07T17:37:39.4415931Z inflating: build/bin/static_runtime_test 2024-08-07T17:37:39.4489869Z inflating: build/bin/Dict_test 2024-08-07T17:37:39.4543442Z inflating: build/bin/Dimname_test 2024-08-07T17:37:39.4609245Z inflating: build/bin/MaybeOwned_test 2024-08-07T17:37:39.4667380Z inflating: build/bin/NamedTensor_test 2024-08-07T17:37:39.4727047Z inflating: build/bin/apply_utils_test 2024-08-07T17:37:39.4785970Z inflating: build/bin/atest 2024-08-07T17:37:39.4849951Z inflating: build/bin/basic 2024-08-07T17:37:39.4905512Z inflating: build/bin/broadcast_test 2024-08-07T17:37:39.4957436Z inflating: build/bin/cpu_allocator_test 2024-08-07T17:37:39.5016671Z inflating: build/bin/cpu_generator_test 2024-08-07T17:37:39.5070480Z inflating: build/bin/cpu_profiling_allocator_test 2024-08-07T17:37:39.5163088Z inflating: build/bin/cpu_rng_test 2024-08-07T17:37:39.5214394Z inflating: build/bin/dispatch_key_set_test 2024-08-07T17:37:39.5265824Z inflating: build/bin/dlconvertor_test 2024-08-07T17:37:39.5324702Z inflating: build/bin/extension_backend_test 2024-08-07T17:37:39.5380652Z inflating: build/bin/half_test 2024-08-07T17:37:39.5434764Z inflating: build/bin/math_kernel_test 2024-08-07T17:37:39.5486170Z inflating: build/bin/lazy_tensor_test 2024-08-07T17:37:39.5580415Z inflating: build/bin/ivalue_test 2024-08-07T17:37:39.5634792Z inflating: build/bin/memory_format_test 2024-08-07T17:37:39.5688763Z inflating: build/bin/memory_overlapping_test 2024-08-07T17:37:39.5742922Z inflating: build/bin/mobile_memory_cleanup 2024-08-07T17:37:39.5799820Z inflating: build/bin/native_test 2024-08-07T17:37:39.5851766Z inflating: build/bin/operator_name_test 2024-08-07T17:37:39.5903465Z inflating: build/bin/operators_test 2024-08-07T17:37:39.5956487Z inflating: build/bin/packedtensoraccessor_test 2024-08-07T17:37:39.6023595Z inflating: build/bin/pow_test 2024-08-07T17:37:39.6081664Z inflating: build/bin/quantized_test 2024-08-07T17:37:39.7176259Z inflating: build/bin/test_api 2024-08-07T17:37:39.7226672Z inflating: build/bin/reduce_ops_test 2024-08-07T17:37:39.7279508Z inflating: build/bin/reportMemoryUsage_test 2024-08-07T17:37:39.7337256Z inflating: build/bin/scalar_tensor_test 2024-08-07T17:37:39.7389523Z inflating: build/bin/StorageUtils_test 2024-08-07T17:37:39.7448593Z inflating: build/bin/scalar_test 2024-08-07T17:37:39.7501875Z inflating: build/bin/stride_properties_test 2024-08-07T17:37:39.7557706Z inflating: build/bin/type_ptr_test 2024-08-07T17:37:39.7636155Z inflating: build/bin/tensor_iterator_test 2024-08-07T17:37:39.7691417Z inflating: build/bin/test_parallel 2024-08-07T17:37:39.7694414Z inflating: build/bin/thread_init_test 2024-08-07T17:37:39.7755169Z inflating: build/bin/type_test 2024-08-07T17:37:39.7756919Z inflating: build/bin/verify_api_visibility 2024-08-07T17:37:39.7810581Z inflating: build/bin/undefined_tensor_test 2024-08-07T17:37:39.7880238Z inflating: build/bin/legacy_vmap_test 2024-08-07T17:37:39.7932512Z inflating: build/bin/weakref_test 2024-08-07T17:37:39.7984904Z inflating: build/bin/wrapdim_test 2024-08-07T17:37:39.8089878Z inflating: build/bin/List_test 2024-08-07T17:37:39.8150777Z inflating: build/bin/IListRef_test 2024-08-07T17:37:39.8202932Z inflating: build/bin/xla_tensor_test 2024-08-07T17:37:39.8269415Z inflating: build/bin/KernelFunction_test 2024-08-07T17:37:39.8389075Z inflating: build/bin/kernel_function_legacy_test 2024-08-07T17:37:39.8484178Z inflating: build/bin/kernel_function_test 2024-08-07T17:37:39.8608517Z inflating: build/bin/kernel_lambda_legacy_test 2024-08-07T17:37:39.8709268Z inflating: build/bin/kernel_lambda_test 2024-08-07T17:37:39.8761592Z inflating: build/bin/CppSignature_test 2024-08-07T17:37:39.8856369Z inflating: build/bin/make_boxed_from_unboxed_functor_test 2024-08-07T17:37:39.8917473Z inflating: build/bin/kernel_stackbased_test 2024-08-07T17:37:39.8967413Z inflating: build/bin/op_allowlist_test 2024-08-07T17:37:39.9031549Z inflating: build/bin/inline_container_test 2024-08-07T17:37:39.9087896Z inflating: build/bin/backend_fallback_test 2024-08-07T17:37:39.9390227Z inflating: build/bin/op_registration_test 2024-08-07T17:37:39.9444580Z inflating: build/bin/test_edge_op_registration 2024-08-07T17:37:39.9449228Z inflating: build/bin/torch_shm_manager 2024-08-07T17:37:39.9462057Z inflating: build/bin/tutorial_tensorexpr 2024-08-07T17:37:40.0229975Z inflating: build/bin/test_tensorexpr 2024-08-07T17:37:40.0790145Z inflating: build/bin/test_jit 2024-08-07T17:37:40.0790532Z creating: .additional_ci_files/ 2024-08-07T17:37:40.0845063Z inflating: .additional_ci_files/test-times.json 2024-08-07T17:37:40.1066555Z inflating: .additional_ci_files/test-class-times.json 2024-08-07T17:37:40.1096302Z ##[group]Run rm artifacts.zip 2024-08-07T17:37:40.1096641Z rm artifacts.zip 2024-08-07T17:37:40.1102498Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:40.1102898Z env: 2024-08-07T17:37:40.1103132Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:40.1103397Z ##[endgroup] 2024-08-07T17:37:40.1481519Z ##[group]Run df -H 2024-08-07T17:37:40.1481812Z df -H 2024-08-07T17:37:40.1487481Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:40.1487880Z env: 2024-08-07T17:37:40.1488096Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:40.1488372Z ##[endgroup] 2024-08-07T17:37:40.1615576Z Filesystem Size Used Avail Use% Mounted on 2024-08-07T17:37:40.1616324Z devtmpfs 4.2M 0 4.2M 0% /dev 2024-08-07T17:37:40.1616693Z tmpfs 8.2G 0 8.2G 0% /dev/shm 2024-08-07T17:37:40.1617050Z tmpfs 3.3G 488k 3.3G 1% /run 2024-08-07T17:37:40.1617391Z /dev/nvme0n1p1 161G 26G 136G 17% / 2024-08-07T17:37:40.1617717Z tmpfs 8.2G 21k 8.2G 1% /tmp 2024-08-07T17:37:40.1618251Z /dev/nvme0n1p128 11M 1.4M 9.2M 13% /boot/efi 2024-08-07T17:37:40.1702436Z Prepare all required actions 2024-08-07T17:37:40.1702840Z Getting action download info 2024-08-07T17:37:40.3150271Z ##[group]Run ./.github/actions/download-td-artifacts 2024-08-07T17:37:40.3150654Z with: 2024-08-07T17:37:40.3150860Z env: 2024-08-07T17:37:40.3151091Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:40.3151367Z ##[endgroup] 2024-08-07T17:37:40.3258483Z ##[group]Run seemethere/download-artifact-s3@v4 2024-08-07T17:37:40.3258860Z with: 2024-08-07T17:37:40.3259101Z name: td_results 2024-08-07T17:37:40.3259352Z s3-bucket: gha-artifacts 2024-08-07T17:37:40.3259645Z region: us-east-1 2024-08-07T17:37:40.3259895Z env: 2024-08-07T17:37:40.3260130Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:40.3260449Z ##[endgroup] 2024-08-07T17:37:40.8141359Z (node:268820) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023. 2024-08-07T17:37:40.8141879Z 2024-08-07T17:37:40.8142160Z Please migrate your code to use AWS SDK for JavaScript (v3). 2024-08-07T17:37:40.8142729Z For more information, check the migration guide at https://a.co/7PzMCcy 2024-08-07T17:37:40.8143306Z (Use `node --trace-warnings ...` to show where the warning was created) 2024-08-07T17:37:40.8848803Z Found 1 objects with prefix pytorch/pytorch/10288745067/td_results/ 2024-08-07T17:37:40.8849498Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-07T17:37:40.9555806Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json 2024-08-07T17:37:40.9563422Z Artifact download has finished successfully 2024-08-07T17:37:40.9771929Z ##[group]Run mkdir -p .additional_ci_files 2024-08-07T17:37:40.9772472Z mkdir -p .additional_ci_files 2024-08-07T17:37:40.9772880Z mv td_results.json .additional_ci_files/td_results.json 2024-08-07T17:37:40.9778780Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:40.9779174Z env: 2024-08-07T17:37:40.9779412Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:40.9779687Z ##[endgroup] 2024-08-07T17:37:41.0155342Z ##[group]Run .github/scripts/parse_ref.py 2024-08-07T17:37:41.0155759Z .github/scripts/parse_ref.py 2024-08-07T17:37:41.0161764Z shell: /usr/bin/bash -e {0} 2024-08-07T17:37:41.0162038Z env: 2024-08-07T17:37:41.0162269Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:41.0162546Z ##[endgroup] 2024-08-07T17:37:41.0577105Z Prepare all required actions 2024-08-07T17:37:41.0680040Z ##[group]Run ./.github/actions/get-workflow-job-id 2024-08-07T17:37:41.0680435Z with: 2024-08-07T17:37:41.0680876Z github-token: *** 2024-08-07T17:37:41.0681128Z env: 2024-08-07T17:37:41.0681353Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:41.0681631Z ##[endgroup] 2024-08-07T17:37:41.0766494Z ##[group]Run set -eux 2024-08-07T17:37:41.0766798Z set -eux 2024-08-07T17:37:41.0767247Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}" 2024-08-07T17:37:41.0773312Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:41.0773708Z env: 2024-08-07T17:37:41.0773923Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:41.0774528Z GITHUB_TOKEN: *** 2024-08-07T17:37:41.0774901Z ##[endgroup] 2024-08-07T17:37:41.0799882Z + python3 .github/scripts/get_workflow_job_id.py 10288745067 i-0efccfdf668c75c22 2024-08-07T17:37:43.9670087Z setting job-id=28475634164 2024-08-07T17:37:43.9671257Z setting job-name=linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-07T17:37:43.9892752Z Prepare all required actions 2024-08-07T17:37:43.9893183Z Getting action download info 2024-08-07T17:37:44.1016718Z ##[group]Run ./.github/actions/filter-test-configs 2024-08-07T17:37:44.1017096Z with: 2024-08-07T17:37:44.1017540Z github-token: *** 2024-08-07T17:37:44.1020277Z test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]} 2024-08-07T17:37:44.1023162Z job-name: linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-07T17:37:44.1023653Z env: 2024-08-07T17:37:44.1023870Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:44.1024144Z ##[endgroup] 2024-08-07T17:37:44.1123469Z ##[group]Run nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482 2024-08-07T17:37:44.1123900Z with: 2024-08-07T17:37:44.1124134Z shell: bash 2024-08-07T17:37:44.1124384Z timeout_minutes: 10 2024-08-07T17:37:44.1124639Z max_attempts: 5 2024-08-07T17:37:44.1124904Z retry_wait_seconds: 30 2024-08-07T17:37:44.1125731Z command: set -eux # PyYAML 6.0 doesn't work with MacOS x86 anymore # This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2 python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-07T17:37:44.1126601Z polling_interval_seconds: 1 2024-08-07T17:37:44.1126907Z warning_on_retry: true 2024-08-07T17:37:44.1127188Z continue_on_error: false 2024-08-07T17:37:44.1127452Z env: 2024-08-07T17:37:44.1127684Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:44.1128436Z GITHUB_TOKEN: *** 2024-08-07T17:37:44.1128696Z ##[endgroup] 2024-08-07T17:37:44.1965717Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1 2024-08-07T17:37:44.4421184Z Defaulting to user installation because normal site-packages is not writeable 2024-08-07T17:37:44.4588612Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.9/site-packages (2.27.1) 2024-08-07T17:37:44.4592548Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.9/site-packages (6.0.1) 2024-08-07T17:37:44.4717801Z Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2.0.12) 2024-08-07T17:37:44.4722056Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2024.7.4) 2024-08-07T17:37:44.4726429Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10) 2024-08-07T17:37:44.4737100Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10) 2024-08-07T17:37:45.1665797Z Command completed after 1 attempt(s). 2024-08-07T17:37:45.1765919Z ##[group]Run set -x 2024-08-07T17:37:45.1766199Z set -x 2024-08-07T17:37:45.1766441Z  2024-08-07T17:37:45.1766842Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-07T17:37:45.1767329Z # in runner workspace 2024-08-07T17:37:45.1767731Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py" 2024-08-07T17:37:45.1773721Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:45.1774100Z env: 2024-08-07T17:37:45.1774343Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:45.1774824Z ##[endgroup] 2024-08-07T17:37:45.1798753Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py 2024-08-07T17:37:45.2134415Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-07T17:37:45.2134851Z echo "Workflow: ${GITHUB_WORKFLOW}" 2024-08-07T17:37:45.2135216Z echo "Job name: ${JOB_NAME}" 2024-08-07T17:37:45.2135514Z  2024-08-07T17:37:45.2135910Z # Use relative path here as this could be checked out anywhere, not necessarily 2024-08-07T17:37:45.2136406Z # in runner workspace 2024-08-07T17:37:45.2136836Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \ 2024-08-07T17:37:45.2137335Z  --workflow "${GITHUB_WORKFLOW}" \ 2024-08-07T17:37:45.2137687Z  --job-name "${JOB_NAME}" \ 2024-08-07T17:37:45.2140928Z  --test-matrix "{"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]}" \ 2024-08-07T17:37:45.2143759Z  --selected-test-configs "" \ 2024-08-07T17:37:45.2144170Z  --pr-number "${PR_NUMBER}" \ 2024-08-07T17:37:45.2144483Z  --tag "${TAG}" \ 2024-08-07T17:37:45.2144795Z  --event-name "${EVENT_NAME}" \ 2024-08-07T17:37:45.2145146Z  --schedule "${SCHEDULE}" \ 2024-08-07T17:37:45.2145462Z  --branch "${HEAD_BRANCH}" 2024-08-07T17:37:45.2151666Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:45.2152220Z env: 2024-08-07T17:37:45.2152439Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:45.2152950Z GITHUB_TOKEN: *** 2024-08-07T17:37:45.2153373Z JOB_NAME: linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-07T17:37:45.2153856Z PR_NUMBER: 131248 2024-08-07T17:37:45.2154103Z TAG: 2024-08-07T17:37:45.2154333Z EVENT_NAME: pull_request 2024-08-07T17:37:45.2154593Z SCHEDULE: 2024-08-07T17:37:45.2154827Z HEAD_BRANCH: 2024-08-07T17:37:45.2155066Z ##[endgroup] 2024-08-07T17:37:45.2178560Z Workflow: pull 2024-08-07T17:37:45.2179023Z Job name: linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-07T17:37:45.5000069Z INFO:root:Found no test-config label on the PR, so all test configs are included 2024-08-07T17:37:45.6982408Z ##[group]Run echo "Filtered matrix:" 2024-08-07T17:37:45.6982781Z echo "Filtered matrix:" 2024-08-07T17:37:45.6985597Z echo "{"include": [{"config": "default", "shard": 1, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 4, "runner": "amz2023.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "amz2023.linux.2xlarge"}]}" 2024-08-07T17:37:45.6988334Z  2024-08-07T17:37:45.6988731Z echo 2024-08-07T17:37:45.6989027Z echo "Is the current job unstable? False" 2024-08-07T17:37:45.6989365Z  2024-08-07T17:37:45.6989587Z echo 2024-08-07T17:37:45.6989871Z echo "Is keep-going label set? False" 2024-08-07T17:37:45.6990193Z  2024-08-07T17:37:45.6990410Z echo 2024-08-07T17:37:45.6990660Z echo "Renabled issues? " 2024-08-07T17:37:45.6996530Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:45.6996940Z env: 2024-08-07T17:37:45.6997160Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:45.6997436Z ##[endgroup] 2024-08-07T17:37:45.7021125Z Filtered matrix: 2024-08-07T17:37:45.7024767Z {include: [{config: default, shard: 1, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: default, shard: 2, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: default, shard: 3, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: default, shard: 4, num_shards: 4, runner: amz2023.linux.2xlarge}, {config: docs_test, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: jit_legacy, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: backwards_compat, shard: 1, num_shards: 1, runner: amz2023.linux.2xlarge}, {config: distributed, shard: 1, num_shards: 2, runner: amz2023.linux.2xlarge}, {config: distributed, shard: 2, num_shards: 2, runner: amz2023.linux.2xlarge}]} 2024-08-07T17:37:45.7027342Z 2024-08-07T17:37:45.7027466Z Is the current job unstable? False 2024-08-07T17:37:45.7027673Z 2024-08-07T17:37:45.7027800Z Is keep-going label set? False 2024-08-07T17:37:45.7027993Z 2024-08-07T17:37:45.7028087Z Renabled issues? 2024-08-07T17:37:45.7210891Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-07T17:37:45.7211453Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}" 2024-08-07T17:37:45.7217251Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T17:37:45.7217648Z env: 2024-08-07T17:37:45.7217867Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:45.7218147Z JOB_TIMEOUT: 240 2024-08-07T17:37:45.7218389Z ##[endgroup] 2024-08-07T17:37:45.7368255Z ##[group]Run set -x 2024-08-07T17:37:45.7368607Z set -x 2024-08-07T17:37:45.7368976Z  2024-08-07T17:37:45.7369246Z if [[ $TEST_CONFIG == 'multigpu' ]]; then 2024-08-07T17:37:45.7369659Z  TEST_COMMAND=.ci/pytorch/multigpu-test.sh 2024-08-07T17:37:45.7370054Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then 2024-08-07T17:37:45.7370434Z  TEST_COMMAND=.ci/onnx/test.sh 2024-08-07T17:37:45.7370754Z else 2024-08-07T17:37:45.7371006Z  TEST_COMMAND=.ci/pytorch/test.sh 2024-08-07T17:37:45.7371331Z fi 2024-08-07T17:37:45.7371551Z  2024-08-07T17:37:45.7371889Z # detached container should get cleaned up by teardown_ec2_linux 2024-08-07T17:37:45.7372443Z # TODO: Stop building test binaries as part of the build phase 2024-08-07T17:37:45.7372927Z # Used for GPU_FLAG since that doesn't play nice 2024-08-07T17:37:45.7373368Z # shellcheck disable=SC2086,SC2090 2024-08-07T17:37:45.7373709Z container_name=$(docker run \ 2024-08-07T17:37:45.7374040Z  ${GPU_FLAG:-} \ 2024-08-07T17:37:45.7374331Z  -e BUILD_ENVIRONMENT \ 2024-08-07T17:37:45.7374632Z  -e PR_NUMBER \ 2024-08-07T17:37:45.7374919Z  -e GITHUB_ACTIONS \ 2024-08-07T17:37:45.7375224Z  -e GITHUB_REPOSITORY \ 2024-08-07T17:37:45.7375525Z  -e GITHUB_WORKFLOW \ 2024-08-07T17:37:45.7375826Z  -e GITHUB_JOB \ 2024-08-07T17:37:45.7376110Z  -e GITHUB_RUN_ID \ 2024-08-07T17:37:45.7376387Z  -e GITHUB_RUN_NUMBER \ 2024-08-07T17:37:45.7376698Z  -e GITHUB_RUN_ATTEMPT \ 2024-08-07T17:37:45.7377003Z  -e JOB_ID \ 2024-08-07T17:37:45.7377255Z  -e JOB_NAME \ 2024-08-07T17:37:45.7377527Z  -e BASE_SHA \ 2024-08-07T17:37:45.7377794Z  -e BRANCH \ 2024-08-07T17:37:45.7378040Z  -e SHA1 \ 2024-08-07T17:37:45.7378310Z  -e AWS_DEFAULT_REGION \ 2024-08-07T17:37:45.7378617Z  -e IN_WHEEL_TEST \ 2024-08-07T17:37:45.7378898Z  -e SHARD_NUMBER \ 2024-08-07T17:37:45.7379204Z  -e TEST_CONFIG \ 2024-08-07T17:37:45.7379495Z  -e NUM_TEST_SHARDS \ 2024-08-07T17:37:45.7379789Z  -e REENABLED_ISSUES \ 2024-08-07T17:37:45.7380111Z  -e CONTINUE_THROUGH_ERROR \ 2024-08-07T17:37:45.7380446Z  -e VERBOSE_TEST_LOGS \ 2024-08-07T17:37:45.7380745Z  -e TEST_SHOWLOCALS \ 2024-08-07T17:37:45.7381053Z  -e NO_TEST_TIMEOUT \ 2024-08-07T17:37:45.7381348Z  -e NO_TD \ 2024-08-07T17:37:45.7381607Z  -e TD_DISTRIBUTED \ 2024-08-07T17:37:45.7381908Z  -e PR_LABELS \ 2024-08-07T17:37:45.7382206Z  -e MAX_JOBS="$(nproc --ignore=2)" \ 2024-08-07T17:37:45.7382564Z  -e SCCACHE_BUCKET \ 2024-08-07T17:37:45.7382873Z  -e SCCACHE_S3_KEY_PREFIX \ 2024-08-07T17:37:45.7383356Z  -e XLA_CUDA \ 2024-08-07T17:37:45.7383675Z  -e XLA_CLANG_CACHE_S3_BUCKET_NAME \ 2024-08-07T17:37:45.7384062Z  -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \ 2024-08-07T17:37:45.7384453Z  -e PYTORCH_TEST_RERUN_DISABLED_TESTS \ 2024-08-07T17:37:45.7384821Z  -e SKIP_SCCACHE_INITIALIZATION=1 \ 2024-08-07T17:37:45.7385185Z  -e HUGGING_FACE_HUB_TOKEN \ 2024-08-07T17:37:45.7385511Z  -e DASHBOARD_TAG \ 2024-08-07T17:37:45.7385843Z  --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \ 2024-08-07T17:37:45.7386261Z  --security-opt seccomp=unconfined \ 2024-08-07T17:37:45.7386619Z  --cap-add=SYS_PTRACE \ 2024-08-07T17:37:45.7386909Z  --ipc=host \ 2024-08-07T17:37:45.7387186Z  --shm-size="${SHM_SIZE}" \ 2024-08-07T17:37:45.7387491Z  --tty \ 2024-08-07T17:37:45.7387725Z  --detach \ 2024-08-07T17:37:45.7388001Z  --name="${container_name}" \ 2024-08-07T17:37:45.7388958Z  --user jenkins \ 2024-08-07T17:37:45.7389310Z  -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \ 2024-08-07T17:37:45.7389812Z  -w /var/lib/jenkins/workspace \ 2024-08-07T17:37:45.7390148Z  "${DOCKER_IMAGE}" 2024-08-07T17:37:45.7390409Z ) 2024-08-07T17:37:45.7390716Z # Propagate download.pytorch.org IP to container 2024-08-07T17:37:45.7391409Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts" 2024-08-07T17:37:45.7392129Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}" 2024-08-07T17:37:45.7392814Z docker exec -t "${container_name}" sh -c "pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}" 2024-08-07T17:37:45.7398957Z shell: /usr/bin/bash -e {0} 2024-08-07T17:37:45.7399325Z env: 2024-08-07T17:37:45.7399570Z GIT_DEFAULT_BRANCH: main 2024-08-07T17:37:45.7399889Z BUILD_ENVIRONMENT: linux-jammy-py3.8-gcc11 2024-08-07T17:37:45.7400230Z PR_NUMBER: 131248 2024-08-07T17:37:45.7400503Z GITHUB_REPOSITORY: pytorch/pytorch 2024-08-07T17:37:45.7400832Z GITHUB_WORKFLOW: pull 2024-08-07T17:37:45.7401084Z GITHUB_JOB: test 2024-08-07T17:37:45.7401332Z GITHUB_RUN_ID: 10288745067 2024-08-07T17:37:45.7401615Z GITHUB_RUN_NUMBER: 234358 2024-08-07T17:37:45.7401881Z GITHUB_RUN_ATTEMPT: 1 2024-08-07T17:37:45.7402147Z JOB_ID: 28475634164 2024-08-07T17:37:45.7402574Z JOB_NAME: linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-07T17:37:45.7403052Z BRANCH: pull/131248 2024-08-07T17:37:45.7403340Z SHA1: 016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:37:45.7403824Z BASE_SHA: 6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T17:37:45.7404174Z TEST_CONFIG: distributed 2024-08-07T17:37:45.7404452Z SHARD_NUMBER: 1 2024-08-07T17:37:45.7404700Z NUM_TEST_SHARDS: 2 2024-08-07T17:37:45.7404941Z REENABLED_ISSUES: 2024-08-07T17:37:45.7405209Z CONTINUE_THROUGH_ERROR: False 2024-08-07T17:37:45.7405512Z VERBOSE_TEST_LOGS: False 2024-08-07T17:37:45.7405775Z TEST_SHOWLOCALS: False 2024-08-07T17:37:45.7406052Z NO_TEST_TIMEOUT: False 2024-08-07T17:37:45.7406313Z NO_TD: False 2024-08-07T17:37:45.7406536Z TD_DISTRIBUTED: False 2024-08-07T17:37:45.7406859Z SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2 2024-08-07T17:37:45.7407240Z SCCACHE_S3_KEY_PREFIX: pull 2024-08-07T17:37:45.7407507Z SHM_SIZE: 1g 2024-08-07T17:37:45.7408176Z DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:37:45.7408984Z XLA_CUDA: 2024-08-07T17:37:45.7409341Z XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla 2024-08-07T17:37:45.7409816Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0 2024-08-07T17:37:45.7410157Z PYTORCH_TEST_RERUN_DISABLED_TESTS: 0 2024-08-07T17:37:45.7410457Z DASHBOARD_TAG: 2024-08-07T17:37:45.7410706Z HUGGING_FACE_HUB_TOKEN: 2024-08-07T17:37:45.7410976Z ##[endgroup] 2024-08-07T17:37:45.7435077Z + [[ distributed == \m\u\l\t\i\g\p\u ]] 2024-08-07T17:37:45.7435700Z + [[ linux-jammy-py3.8-gcc11 == *onnx* ]] 2024-08-07T17:37:45.7436054Z + TEST_COMMAND=.ci/pytorch/test.sh 2024-08-07T17:37:45.7443422Z +++ nproc --ignore=2 2024-08-07T17:37:45.7475823Z ++ docker run -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=6 -e SCCACHE_BUCKET -e SCCACHE_S3_KEY_PREFIX -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e DASHBOARD_TAG --env-file=/tmp/github_env_10288745067 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=1g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T17:37:55.6084809Z + container_name=595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T17:37:55.6086996Z + grep download.pytorch.org /etc/hosts 2024-08-07T17:37:55.6089077Z + docker exec -i 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c sudo bash -c '/bin/cat >> /etc/hosts' 2024-08-07T17:37:55.8437139Z + echo DOCKER_CONTAINER_ID=595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T17:37:55.8440365Z ++ echo dist/torch-2.5.0a0+git016588f-cp38-cp38-linux_x86_64.whl 2024-08-07T17:37:55.8442861Z + docker exec -t 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c sh -c 'pip install dist/torch-2.5.0a0+git016588f-cp38-cp38-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh' 2024-08-07T17:37:56.8736115Z Processing ./dist/torch-2.5.0a0+git016588f-cp38-cp38-linux_x86_64.whl (from torch==2.5.0a0+git016588f) 2024-08-07T17:37:57.3162919Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (3.13.1) 2024-08-07T17:37:57.3168800Z Requirement already satisfied: typing-extensions>=4.8.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (4.12.2) 2024-08-07T17:37:57.3172308Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (2.8.8) 2024-08-07T17:37:57.3175896Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (3.1.4) 2024-08-07T17:37:57.3179164Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (2024.6.1) 2024-08-07T17:37:57.3188772Z Requirement already satisfied: sympy==1.12.1 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (1.12.1) 2024-08-07T17:37:57.3211601Z Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from sympy==1.12.1->torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (1.3.0) 2024-08-07T17:37:57.3237565Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (3.3.0) 2024-08-07T17:37:57.3302085Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from opt-einsum>=3.3->torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (1.21.2) 2024-08-07T17:37:57.4166221Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.8/lib/python3.8/site-packages (from jinja2->torch==2.5.0a0+git016588f->torch==2.5.0a0+git016588f) (2.1.5) 2024-08-07T17:37:58.7032117Z Installing collected packages: torch 2024-08-07T17:38:06.8409251Z Successfully installed torch-2.5.0a0+git016588f 2024-08-07T17:38:06.9303372Z ++ dirname .ci/pytorch/test.sh 2024-08-07T17:38:06.9320295Z + source .ci/pytorch/common.sh 2024-08-07T17:38:06.9326772Z +++ dirname .ci/pytorch/common.sh 2024-08-07T17:38:06.9332731Z ++ source .ci/pytorch/common_utils.sh 2024-08-07T17:38:06.9338343Z +++ declare -f -t trap_add 2024-08-07T17:38:06.9342813Z ++ set -ex 2024-08-07T17:38:06.9343233Z ++ [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-07T17:38:06.9343801Z ++ BUILD_TEST_LIBTORCH=0 2024-08-07T17:38:06.9344290Z + [[ linux-jammy-py3.8-gcc11 != *rocm* ]] 2024-08-07T17:38:06.9347236Z ++ stat -c %u /var/lib/jenkins/workspace 2024-08-07T17:38:06.9395777Z + WORKSPACE_ORIGINAL_OWNER_ID=1000 2024-08-07T17:38:06.9396806Z + trap_add cleanup_workspace EXIT 2024-08-07T17:38:06.9397150Z + trap_add_cmd=cleanup_workspace 2024-08-07T17:38:06.9397606Z + shift 2024-08-07T17:38:06.9397828Z + for trap_add_name in "$@" 2024-08-07T17:38:06.9402926Z +++ trap -p EXIT 2024-08-07T17:38:06.9405026Z ++ eval 'extract_trap_cmd ' 2024-08-07T17:38:06.9405407Z +++ extract_trap_cmd 2024-08-07T17:38:06.9405708Z +++ printf '%s\n' '' 2024-08-07T17:38:06.9405979Z ++ printf '%s\n' cleanup_workspace 2024-08-07T17:38:06.9407946Z + trap -- ' 2024-08-07T17:38:06.9408193Z cleanup_workspace' EXIT 2024-08-07T17:38:06.9408515Z + sudo chown -R jenkins /var/lib/jenkins/workspace 2024-08-07T17:38:07.4359945Z + git config --global --add safe.directory /var/lib/jenkins/workspace 2024-08-07T17:38:07.4652774Z + echo 'Environment variables:' 2024-08-07T17:38:07.4653135Z Environment variables: 2024-08-07T17:38:07.4653385Z + env 2024-08-07T17:38:07.4674781Z INSTALLED_DB=yes 2024-08-07T17:38:07.4675453Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:38:07.4676123Z CONTINUE_THROUGH_ERROR=False 2024-08-07T17:38:07.4676479Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11 2024-08-07T17:38:07.4676837Z HOSTNAME=595390c2978c 2024-08-07T17:38:07.4677436Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_d37addda-08b6-4fa4-bb97-472ff48746f1 2024-08-07T17:38:07.4678077Z GITHUB_ACTION=__self 2024-08-07T17:38:07.4678430Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-07T17:38:07.4678754Z GITHUB_RUN_NUMBER=234358 2024-08-07T17:38:07.4679017Z TEST_CONFIG=distributed 2024-08-07T17:38:07.4679304Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-07T17:38:07.4679644Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-07T17:38:07.4680100Z GITHUB_TRIGGERING_ACTOR=zdevito 2024-08-07T17:38:07.4680492Z GITHUB_REF_TYPE=branch 2024-08-07T17:38:07.4680805Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-07T17:38:07.4681238Z BASE_SHA=6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T17:38:07.4681636Z XLA_CUDA= 2024-08-07T17:38:07.4681866Z HUGGING_FACE_HUB_TOKEN= 2024-08-07T17:38:07.4682342Z *** 2024-08-07T17:38:07.4682576Z GITHUB_REPOSITORY_ID=65600975 2024-08-07T17:38:07.4682873Z GITHUB_ACTIONS=true 2024-08-07T17:38:07.4683163Z SHA1=016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:38:07.4683546Z GITHUB_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T17:38:07.4684089Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/131248/merge 2024-08-07T17:38:07.4684598Z UCC_HOME=/usr 2024-08-07T17:38:07.4684839Z VERBOSE_TEST_LOGS=False 2024-08-07T17:38:07.4685104Z GITHUB_REF=refs/pull/131248/merge 2024-08-07T17:38:07.4685402Z SHARD_NUMBER=1 2024-08-07T17:38:07.4685653Z GITHUB_REF_PROTECTED=false 2024-08-07T17:38:07.4685917Z HOME=/var/lib/jenkins 2024-08-07T17:38:07.4686211Z GITHUB_API_URL=https://api.github.com 2024-08-07T17:38:07.4686553Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-07T17:38:07.4686846Z UCX_COMMIT= 2024-08-07T17:38:07.4687119Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-07T17:38:07.4687399Z NUM_TEST_SHARDS=2 2024-08-07T17:38:07.4687632Z UCX_HOME=/usr 2024-08-07T17:38:07.4688219Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_d37addda-08b6-4fa4-bb97-472ff48746f1 2024-08-07T17:38:07.4689059Z JOB_NAME=linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-07T17:38:07.4689852Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_d37addda-08b6-4fa4-bb97-472ff48746f1 2024-08-07T17:38:07.4690688Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-07T17:38:07.4691304Z GITHUB_EVENT_NAME=pull_request 2024-08-07T17:38:07.4691604Z DASHBOARD_TAG= 2024-08-07T17:38:07.4691836Z GITHUB_RUN_ID=10288745067 2024-08-07T17:38:07.4692495Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_d37addda-08b6-4fa4-bb97-472ff48746f1 2024-08-07T17:38:07.4693197Z GITHUB_ACTOR=zdevito 2024-08-07T17:38:07.4693441Z PR_NUMBER=131248 2024-08-07T17:38:07.4693679Z DESIRED_CUDA= 2024-08-07T17:38:07.4694179Z GITHUB_RUN_ATTEMPT=1 2024-08-07T17:38:07.4694452Z ANACONDA_PYTHON_VERSION=3.8 2024-08-07T17:38:07.4694909Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-07T17:38:07.4695258Z TERM=xterm 2024-08-07T17:38:07.4695490Z INSTALLED_VISION=yes 2024-08-07T17:38:07.4695754Z BRANCH=pull/131248 2024-08-07T17:38:07.4696001Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-07T17:38:07.4696296Z CUDA_PATH=/usr/local/cuda 2024-08-07T17:38:07.4696832Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-07T17:38:07.4697418Z GITHUB_SERVER_URL=https://github.com 2024-08-07T17:38:07.4697736Z UCC_COMMIT= 2024-08-07T17:38:07.4697964Z REENABLED_ISSUES= 2024-08-07T17:38:07.4698193Z DOCS=yes 2024-08-07T17:38:07.4698422Z INSTALLED_ANDROID= 2024-08-07T17:38:07.4698663Z SHLVL=1 2024-08-07T17:38:07.4698862Z MAX_JOBS=6 2024-08-07T17:38:07.4699093Z GITHUB_ACTOR_ID=370202 2024-08-07T17:38:07.4699439Z GITHUB_WORKFLOW_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T17:38:07.4699828Z GITHUB_REF_NAME=131248/merge 2024-08-07T17:38:07.4700244Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-07T17:38:07.4700695Z GITHUB_JOB=test 2024-08-07T17:38:07.4700927Z NO_TEST_TIMEOUT=False 2024-08-07T17:38:07.4701190Z TD_DISTRIBUTED=False 2024-08-07T17:38:07.4701471Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-07T17:38:07.4701774Z GITHUB_RETENTION_DAYS=90 2024-08-07T17:38:07.4702056Z OPENSSL_DIR=/opt/openssl 2024-08-07T17:38:07.4702324Z GITHUB_ACTION_REPOSITORY= 2024-08-07T17:38:07.4703120Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T17:38:07.4703958Z GITHUB_BASE_REF=gh/zdevito/267/base 2024-08-07T17:38:07.4704275Z INSTALLED_ACL= 2024-08-07T17:38:07.4704500Z CI=true 2024-08-07T17:38:07.4704740Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-07T17:38:07.4705022Z JOB_ID=28475634164 2024-08-07T17:38:07.4705284Z INSTALLED_PROTOBUF=yes 2024-08-07T17:38:07.4705569Z GITHUB_HEAD_REF=gh/zdevito/267/head 2024-08-07T17:38:07.4705870Z GITHUB_ACTION_REF= 2024-08-07T17:38:07.4706180Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-07T17:38:07.4706555Z TEST_SHOWLOCALS=False 2024-08-07T17:38:07.4706809Z GITHUB_WORKFLOW=pull 2024-08-07T17:38:07.4707082Z DEBIAN_FRONTEND=noninteractive 2024-08-07T17:38:07.4707736Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_d37addda-08b6-4fa4-bb97-472ff48746f1 2024-08-07T17:38:07.4708391Z NO_TD=False 2024-08-07T17:38:07.4708638Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-07T17:38:07.4708928Z _=/usr/bin/env 2024-08-07T17:38:07.4709244Z ++ python -c 'import site; print(site.getsitepackages()[0])' 2024-08-07T17:38:07.5272105Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch 2024-08-07T17:38:07.5272911Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-08-07T17:38:07.5273837Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib 2024-08-07T17:38:07.5274583Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/test 2024-08-07T17:38:07.5275073Z + BUILD_DIR=build 2024-08-07T17:38:07.5275323Z + BUILD_RENAMED_DIR=build_renamed 2024-08-07T17:38:07.5275637Z + BUILD_BIN_DIR=build/bin 2024-08-07T17:38:07.5275906Z + SHARD_NUMBER=1 2024-08-07T17:38:07.5276178Z + NUM_TEST_SHARDS=2 2024-08-07T17:38:07.5276472Z + export VALGRIND=ON 2024-08-07T17:38:07.5276826Z + VALGRIND=ON 2024-08-07T17:38:07.5277167Z + [[ linux-jammy-py3.8-gcc11 == *clang9* ]] 2024-08-07T17:38:07.5277496Z + [[ 0 == \1 ]] 2024-08-07T17:38:07.5277736Z + [[ False == \1 ]] 2024-08-07T17:38:07.5277990Z + [[ linux-jammy-py3.8-gcc11 != *bazel* ]] 2024-08-07T17:38:07.5278405Z ++ realpath build/custom_test_artifacts 2024-08-07T17:38:07.5310942Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts 2024-08-07T17:38:07.5311634Z + [[ -n '' ]] 2024-08-07T17:38:07.5312144Z + echo 'Environment variables' 2024-08-07T17:38:07.5312460Z Environment variables 2024-08-07T17:38:07.5312703Z + env 2024-08-07T17:38:07.5318435Z INSTALLED_DB=yes 2024-08-07T17:38:07.5319216Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T17:38:07.5319752Z CONTINUE_THROUGH_ERROR=False 2024-08-07T17:38:07.5320196Z BUILD_ENVIRONMENT=linux-jammy-py3.8-gcc11 2024-08-07T17:38:07.5320709Z HOSTNAME=595390c2978c 2024-08-07T17:38:07.5321325Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_d37addda-08b6-4fa4-bb97-472ff48746f1 2024-08-07T17:38:07.5322014Z GITHUB_ACTION=__self 2024-08-07T17:38:07.5322293Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0 2024-08-07T17:38:07.5322608Z GITHUB_RUN_NUMBER=234358 2024-08-07T17:38:07.5322870Z TEST_CONFIG=distributed 2024-08-07T17:38:07.5323158Z GITHUB_REPOSITORY_OWNER_ID=21003710 2024-08-07T17:38:07.5323499Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all 2024-08-07T17:38:07.5323823Z GITHUB_TRIGGERING_ACTOR=zdevito 2024-08-07T17:38:07.5324135Z GITHUB_REF_TYPE=branch 2024-08-07T17:38:07.5324403Z TORCH_CUDA_ARCH_LIST=Maxwell 2024-08-07T17:38:07.5324721Z BASE_SHA=6ce09a9bb33e4011761558032e2165ad7b49fb68 2024-08-07T17:38:07.5325062Z XLA_CUDA= 2024-08-07T17:38:07.5325294Z HUGGING_FACE_HUB_TOKEN= 2024-08-07T17:38:07.5325692Z *** 2024-08-07T17:38:07.5325922Z GITHUB_REPOSITORY_ID=65600975 2024-08-07T17:38:07.5326218Z GITHUB_ACTIONS=true 2024-08-07T17:38:07.5326483Z SHA1=016588f53c6904b840aa56aa86f95460b4d9c996 2024-08-07T17:38:07.5326872Z GITHUB_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T17:38:07.5327612Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/pull/131248/merge 2024-08-07T17:38:07.5328111Z UCC_HOME=/usr 2024-08-07T17:38:07.5328353Z VERBOSE_TEST_LOGS=False 2024-08-07T17:38:07.5328631Z GITHUB_REF=refs/pull/131248/merge 2024-08-07T17:38:07.5328914Z SHARD_NUMBER=1 2024-08-07T17:38:07.5329166Z GITHUB_REF_PROTECTED=false 2024-08-07T17:38:07.5329445Z HOME=/var/lib/jenkins 2024-08-07T17:38:07.5329728Z GITHUB_API_URL=https://api.github.com 2024-08-07T17:38:07.5330071Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0 2024-08-07T17:38:07.5330367Z UCX_COMMIT= 2024-08-07T17:38:07.5330604Z SCCACHE_S3_KEY_PREFIX=pull 2024-08-07T17:38:07.5330882Z NUM_TEST_SHARDS=2 2024-08-07T17:38:07.5331108Z UCX_HOME=/usr 2024-08-07T17:38:07.5331696Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_d37addda-08b6-4fa4-bb97-472ff48746f1 2024-08-07T17:38:07.5332528Z JOB_NAME=linux-jammy-py3.8-gcc11 / test (distributed, 1, 2, amz2023.linux.2xlarge) 2024-08-07T17:38:07.5333322Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_d37addda-08b6-4fa4-bb97-472ff48746f1 2024-08-07T17:38:07.5334162Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json 2024-08-07T17:38:07.5334695Z GITHUB_EVENT_NAME=pull_request 2024-08-07T17:38:07.5334992Z DASHBOARD_TAG= 2024-08-07T17:38:07.5335228Z GITHUB_RUN_ID=10288745067 2024-08-07T17:38:07.5335888Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_d37addda-08b6-4fa4-bb97-472ff48746f1 2024-08-07T17:38:07.5336673Z GITHUB_ACTOR=zdevito 2024-08-07T17:38:07.5337008Z PR_NUMBER=131248 2024-08-07T17:38:07.5337253Z DESIRED_CUDA= 2024-08-07T17:38:07.5337477Z GITHUB_RUN_ATTEMPT=1 2024-08-07T17:38:07.5337756Z VALGRIND=ON 2024-08-07T17:38:07.5338003Z ANACONDA_PYTHON_VERSION=3.8 2024-08-07T17:38:07.5338347Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql 2024-08-07T17:38:07.5338714Z TERM=xterm 2024-08-07T17:38:07.5338975Z INSTALLED_VISION=yes 2024-08-07T17:38:07.5339228Z BRANCH=pull/131248 2024-08-07T17:38:07.5339494Z OPENSSL_ROOT_DIR=/opt/openssl 2024-08-07T17:38:07.5339790Z CUDA_PATH=/usr/local/cuda 2024-08-07T17:38:07.5340858Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux 2024-08-07T17:38:07.5341467Z GITHUB_SERVER_URL=https://github.com 2024-08-07T17:38:07.5341793Z UCC_COMMIT= 2024-08-07T17:38:07.5342011Z REENABLED_ISSUES= 2024-08-07T17:38:07.5342441Z DOCS=yes 2024-08-07T17:38:07.5342670Z INSTALLED_ANDROID= 2024-08-07T17:38:07.5342905Z SHLVL=1 2024-08-07T17:38:07.5343216Z MAX_JOBS=6 2024-08-07T17:38:07.5343455Z GITHUB_ACTOR_ID=370202 2024-08-07T17:38:07.5343788Z GITHUB_WORKFLOW_SHA=f779f6b7738020e244184bded4026b37de3f9f24 2024-08-07T17:38:07.5344194Z GITHUB_REF_NAME=131248/merge 2024-08-07T17:38:07.5344618Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla 2024-08-07T17:38:07.5345052Z GITHUB_JOB=test 2024-08-07T17:38:07.5345300Z NO_TEST_TIMEOUT=False 2024-08-07T17:38:07.5345549Z TD_DISTRIBUTED=False 2024-08-07T17:38:07.5345828Z GITHUB_REPOSITORY=pytorch/pytorch 2024-08-07T17:38:07.5346149Z GITHUB_RETENTION_DAYS=90 2024-08-07T17:38:07.5346413Z OPENSSL_DIR=/opt/openssl 2024-08-07T17:38:07.5346693Z GITHUB_ACTION_REPOSITORY= 2024-08-07T17:38:07.5347476Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T17:38:07.5348298Z GITHUB_BASE_REF=gh/zdevito/267/base 2024-08-07T17:38:07.5348610Z INSTALLED_ACL= 2024-08-07T17:38:07.5348841Z CI=true 2024-08-07T17:38:07.5349061Z GITHUB_REPOSITORY_OWNER=pytorch 2024-08-07T17:38:07.5349354Z JOB_ID=28475634164 2024-08-07T17:38:07.5349604Z INSTALLED_PROTOBUF=yes 2024-08-07T17:38:07.5349872Z GITHUB_HEAD_REF=gh/zdevito/267/head 2024-08-07T17:38:07.5350179Z GITHUB_ACTION_REF= 2024-08-07T17:38:07.5350482Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2 2024-08-07T17:38:07.5350856Z TEST_SHOWLOCALS=False 2024-08-07T17:38:07.5351121Z GITHUB_WORKFLOW=pull 2024-08-07T17:38:07.5351400Z DEBIAN_FRONTEND=noninteractive 2024-08-07T17:38:07.5352043Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_d37addda-08b6-4fa4-bb97-472ff48746f1 2024-08-07T17:38:07.5352710Z NO_TD=False 2024-08-07T17:38:07.5352958Z SKIP_SCCACHE_INITIALIZATION=1 2024-08-07T17:38:07.5353235Z _=/usr/bin/env 2024-08-07T17:38:07.5353477Z + echo 'Testing pytorch' 2024-08-07T17:38:07.5353752Z Testing pytorch 2024-08-07T17:38:07.5354007Z + export LANG=C.UTF-8 2024-08-07T17:38:07.5354265Z + LANG=C.UTF-8 2024-08-07T17:38:07.5408335Z + PR_NUMBER=131248 2024-08-07T17:38:07.5408756Z + [[ distributed == \d\e\f\a\u\l\t ]] 2024-08-07T17:38:07.5409278Z + [[ distributed == \d\i\s\t\r\i\b\u\t\e\d ]] 2024-08-07T17:38:07.5409743Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-07T17:38:07.5410256Z + [[ distributed == \s\l\o\w ]] 2024-08-07T17:38:07.5410625Z + [[ linux-jammy-py3.8-gcc11 == *slow-gradcheck* ]] 2024-08-07T17:38:07.5411016Z + [[ linux-jammy-py3.8-gcc11 == *cuda* ]] 2024-08-07T17:38:07.5411350Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-07T17:38:07.5411695Z + [[ linux-jammy-py3.8-gcc11 == *xpu* ]] 2024-08-07T17:38:07.5412032Z + [[ distributed == *crossref* ]] 2024-08-07T17:38:07.5412341Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-07T17:38:07.5412677Z + [[ linux-jammy-py3.8-gcc11 == *xpu* ]] 2024-08-07T17:38:07.5414597Z + [[ linux-jammy-py3.8-gcc11 != *-bazel-* ]] 2024-08-07T17:38:07.5414975Z + pip_install --user ninja==1.10.2 2024-08-07T17:38:07.5415355Z + pip install --progress-bar off --user ninja==1.10.2 2024-08-07T17:38:09.0165488Z Collecting ninja==1.10.2 2024-08-07T17:38:09.0352339Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB) 2024-08-07T17:38:09.0725065Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB) 2024-08-07T17:38:10.3413964Z Installing collected packages: ninja 2024-08-07T17:38:10.3508464Z  WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH. 2024-08-07T17:38:10.3509493Z Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. 2024-08-07T17:38:10.3561277Z Successfully installed ninja-1.10.2 2024-08-07T17:38:10.4526618Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T17:38:10.4528264Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T17:38:10.4529411Z + [[ linux-jammy-py3.8-gcc11 == *aarch64* ]] 2024-08-07T17:38:10.4529756Z + install_tlparse 2024-08-07T17:38:10.4530021Z + pip_install --user tlparse==0.3.7 2024-08-07T17:38:10.4530415Z + pip install --progress-bar off --user tlparse==0.3.7 2024-08-07T17:38:10.8817217Z Collecting tlparse==0.3.7 2024-08-07T17:38:10.9028101Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (346 bytes) 2024-08-07T17:38:10.9143410Z Downloading tlparse-0.3.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB) 2024-08-07T17:38:12.1469845Z Installing collected packages: tlparse 2024-08-07T17:38:12.1899871Z Successfully installed tlparse-0.3.7 2024-08-07T17:38:12.2808748Z ++ python -m site --user-base 2024-08-07T17:38:12.3040208Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.8/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2024-08-07T17:38:12.3041367Z + [[ linux-jammy-py3.8-gcc11 == *asan* ]] 2024-08-07T17:38:12.3041725Z + [[ linux-jammy-py3.8-gcc11 == *-debug* ]] 2024-08-07T17:38:12.3042070Z + [[ linux-jammy-py3.8-gcc11 != *-bazel-* ]] 2024-08-07T17:38:12.3042572Z + echo 'We are not in debug mode: linux-jammy-py3.8-gcc11. Expect the assertion to pass' 2024-08-07T17:38:12.3043269Z We are not in debug mode: linux-jammy-py3.8-gcc11. Expect the assertion to pass 2024-08-07T17:38:12.3044241Z + cd test 2024-08-07T17:38:12.3045224Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)' 2024-08-07T17:38:13.9442385Z + [[ distributed == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]] 2024-08-07T17:38:13.9443131Z + [[ distributed == \n\o\g\p\u\_\A\V\X\5\1\2 ]] 2024-08-07T17:38:13.9445244Z + DYNAMO_BENCHMARK_FLAGS=() 2024-08-07T17:38:13.9445741Z + [[ distributed == *dynamo_eager* ]] 2024-08-07T17:38:13.9446157Z + [[ distributed == *aot_eager* ]] 2024-08-07T17:38:13.9446470Z + [[ distributed == *aot_inductor* ]] 2024-08-07T17:38:13.9446813Z + [[ distributed == *inductor* ]] 2024-08-07T17:38:13.9447202Z + [[ distributed == *dynamic* ]] 2024-08-07T17:38:13.9447501Z + [[ distributed == *cpu* ]] 2024-08-07T17:38:13.9447879Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda) 2024-08-07T17:38:13.9467108Z + [[ linux-jammy-py3.8-gcc11 == *libtorch* ]] 2024-08-07T17:38:13.9467486Z + [[ linux-jammy-py3.8-gcc11 == *-bazel-* ]] 2024-08-07T17:38:13.9470448Z + cd test 2024-08-07T17:38:13.9471000Z + python -c 'import torch; print(torch.__config__.show())' 2024-08-07T17:38:15.1807940Z PyTorch built with: 2024-08-07T17:38:15.1808256Z - GCC 11.4 2024-08-07T17:38:15.1808504Z - C++ Version: 201703 2024-08-07T17:38:15.1809130Z - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-07T17:38:15.1809862Z - Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-07T17:38:15.1810336Z - OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-07T17:38:15.1810697Z - LAPACK is enabled (usually provided by MKL) 2024-08-07T17:38:15.1811035Z - NNPACK is enabled 2024-08-07T17:38:15.1811314Z - CPU capability usage: AVX512 2024-08-07T17:38:15.1816700Z - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Werror -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.5.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 2024-08-07T17:38:15.1822137Z 2024-08-07T17:38:15.4222464Z + cd test 2024-08-07T17:38:15.4222869Z + python -c 'import torch; print(torch.__config__.parallel_info())' 2024-08-07T17:38:16.6602600Z ATen/Parallel: 2024-08-07T17:38:16.6602963Z at::get_num_threads() : 4 2024-08-07T17:38:16.6603276Z at::get_num_interop_threads() : 4 2024-08-07T17:38:16.6603648Z OpenMP 201511 (a.k.a. OpenMP 4.5) 2024-08-07T17:38:16.6603966Z omp_get_max_threads() : 4 2024-08-07T17:38:16.6604550Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications 2024-08-07T17:38:16.6605167Z mkl_get_max_threads() : 4 2024-08-07T17:38:16.6605568Z Intel(R) MKL-DNN v3.4.2 (Git Hash 1137e04ec0b5251ca2b4400a4fd3c667ce843d67) 2024-08-07T17:38:16.6606013Z std::thread::hardware_concurrency() : 8 2024-08-07T17:38:16.6606348Z Environment variables: 2024-08-07T17:38:16.6606622Z OMP_NUM_THREADS : [not set] 2024-08-07T17:38:16.6606898Z MKL_NUM_THREADS : [not set] 2024-08-07T17:38:16.6607198Z ATen parallel backend: OpenMP 2024-08-07T17:38:16.6607389Z 2024-08-07T17:38:16.9250575Z + [[ linux-jammy-py3.8-gcc11 == *aarch64* ]] 2024-08-07T17:38:16.9251109Z + [[ distributed == *backward* ]] 2024-08-07T17:38:16.9251433Z + [[ distributed == *xla* ]] 2024-08-07T17:38:16.9251803Z + [[ distributed == *executorch* ]] 2024-08-07T17:38:16.9252148Z + [[ distributed == \j\i\t\_\l\e\g\a\c\y ]] 2024-08-07T17:38:16.9252564Z + [[ linux-jammy-py3.8-gcc11 == *libtorch* ]] 2024-08-07T17:38:16.9252941Z + [[ distributed == distributed ]] 2024-08-07T17:38:16.9253288Z + test_distributed 2024-08-07T17:38:16.9253575Z + echo 'Testing distributed python tests' 2024-08-07T17:38:16.9253979Z Testing distributed python tests 2024-08-07T17:38:16.9254397Z + python test/run_test.py --distributed-tests --shard 1 2 --verbose 2024-08-07T17:38:17.0470399Z test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-07T17:38:17.0471162Z import pkg_resources 2024-08-07T17:38:20.6158123Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-07T17:38:20.6712641Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-07T17:38:20.7243558Z Ignoring disabled issues: [''] 2024-08-07T17:38:20.7355593Z Found test times from artifacts 2024-08-07T17:38:20.7782511Z Found test times from artifacts 2024-08-07T17:38:20.7799451Z Running all tests 2024-08-07T17:38:20.7932507Z Running parallel tests on 3 processes 2024-08-07T17:38:20.7938031Z Name: tests to run (est. time: 59.97min) 2024-08-07T17:38:20.7938582Z Serial tests (96): 2024-08-07T17:38:20.7939065Z distributed/_tensor/test_dtensor_compile 1/1 2024-08-07T17:38:20.7939458Z distributed/pipelining/test_backward 1/1 2024-08-07T17:38:20.7939835Z distributed/checkpoint/test_nested_dict 1/1 2024-08-07T17:38:20.7940393Z distributed/_shard/sharded_tensor/test_logger 1/1 2024-08-07T17:38:20.7940802Z distributed/pipelining/test_transformer 1/1 2024-08-07T17:38:20.7941178Z distributed/test_symmetric_memory 1/1 2024-08-07T17:38:20.7941573Z distributed/tensor/parallel/test_micro_pipeline_tp 1/1 2024-08-07T17:38:20.7941994Z distributed/test_dynamo_distributed 1/1 2024-08-07T17:38:20.7942610Z distributed/fsdp/test_fsdp_core 1/1 2024-08-07T17:38:20.7942989Z distributed/pipelining/test_schedule_multiproc 1/1 2024-08-07T17:38:20.7943601Z distributed/_composable/test_contract 1/1 2024-08-07T17:38:20.7943963Z distributed/test_device_mesh 1/1 2024-08-07T17:38:20.7944290Z distributed/_tensor/test_tensor_ops 1/1 2024-08-07T17:38:20.7944649Z distributed/_tensor/test_random_ops 1/1 2024-08-07T17:38:20.7945106Z distributed/_tensor/experimental/test_register_sharding 1/1 2024-08-07T17:38:20.7945562Z distributed/tensor/parallel/test_tp_examples 1/1 2024-08-07T17:38:20.7946008Z distributed/_tensor/test_math_ops 1/1 2024-08-07T17:38:20.7946372Z distributed/optim/test_named_optimizer 1/1 2024-08-07T17:38:20.7946731Z distributed/_tensor/test_redistribute 1/1 2024-08-07T17:38:20.7947115Z distributed/checkpoint/test_format_utils 1/1 2024-08-07T17:38:20.7947541Z distributed/_tensor/experimental/test_tp_transform 1/1 2024-08-07T17:38:20.7947980Z distributed/_tensor/experimental/test_local_map 1/1 2024-08-07T17:38:20.7948410Z distributed/checkpoint/test_fsdp_optim_state 1/1 2024-08-07T17:38:20.7948846Z distributed/checkpoint/test_dtensor_resharding 1/1 2024-08-07T17:38:20.7949266Z distributed/checkpoint/test_tp_checkpoint 1/1 2024-08-07T17:38:20.7949705Z distributed/_tensor/test_pointwise_ops 1/1 2024-08-07T17:38:20.7950061Z distributed/checkpoint/test_fsspec 1/1 2024-08-07T17:38:20.7950421Z distributed/test_collective_utils 1/1 2024-08-07T17:38:20.7950791Z distributed/fsdp/test_fsdp_optim_state 1/1 2024-08-07T17:38:20.7951232Z distributed/checkpoint/test_fsdp_tp_checkpoint_conversion 1/1 2024-08-07T17:38:20.7951681Z distributed/test_fake_pg 1/1 2024-08-07T17:38:20.7952024Z distributed/_tensor/debug/test_comm_mode 1/1 2024-08-07T17:38:20.7952384Z distributed/checkpoint/test_utils 1/1 2024-08-07T17:38:20.7952796Z distributed/_composable/fsdp/test_fully_shard_init 1/1 2024-08-07T17:38:20.7953208Z distributed/test_c10d_logger 1/1 2024-08-07T17:38:20.7953532Z distributed/fsdp/test_shard_utils 1/1 2024-08-07T17:38:20.7953881Z distributed/test_nccl 1/1 2024-08-07T17:38:20.7954199Z distributed/rpc/test_share_memory 1/1 2024-08-07T17:38:20.7954572Z distributed/checkpoint/test_fsdp_model_state 1/1 2024-08-07T17:38:20.7955004Z distributed/tensor/parallel/test_parallelize_api 1/1 2024-08-07T17:38:20.7955414Z distributed/test_control_collectives 1/1 2024-08-07T17:38:20.7955854Z distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_ 1/1 2024-08-07T17:38:20.7956317Z distributed/test_distributed_spawn 1/3 2024-08-07T17:38:20.7956678Z distributed/test_distributed_spawn 3/3 2024-08-07T17:38:20.7957053Z distributed/fsdp/test_fsdp_backward_prefetch 1/1 2024-08-07T17:38:20.7957508Z distributed/_composable/fsdp/test_fully_shard_autograd 1/1 2024-08-07T17:38:20.7958001Z distributed/_composable/fsdp/test_fully_shard_training 1/1 2024-08-07T17:38:20.7958525Z distributed/fsdp/test_fsdp_freezing_weights 1/1 2024-08-07T17:38:20.7958957Z distributed/optim/test_zero_redundancy_optimizer 1/1 2024-08-07T17:38:20.7959440Z distributed/algorithms/quantization/test_quantization 1/1 2024-08-07T17:38:20.7959854Z distributed/test_data_parallel 1/1 2024-08-07T17:38:20.7960196Z distributed/algorithms/test_join 1/1 2024-08-07T17:38:20.7960563Z distributed/fsdp/test_checkpoint_wrapper 1/1 2024-08-07T17:38:20.7960972Z distributed/_composable/fsdp/test_fully_shard_comm 1/1 2024-08-07T17:38:20.7961377Z distributed/fsdp/test_fsdp_input 1/1 2024-08-07T17:38:20.7961725Z distributed/fsdp/test_fsdp_uneven 1/1 2024-08-07T17:38:20.7962095Z distributed/fsdp/test_fsdp_multiple_wrapping 1/1 2024-08-07T17:38:20.7962478Z distributed/fsdp/test_utils 1/1 2024-08-07T17:38:20.7962811Z distributed/fsdp/test_fsdp_misc 1/1 2024-08-07T17:38:20.7963152Z distributed/fsdp/test_fsdp_checkpoint 1/1 2024-08-07T17:38:20.7963525Z distributed/fsdp/test_fsdp_exec_order 1/1 2024-08-07T17:38:20.7964007Z distributed/fsdp/test_fsdp_fine_tune 1/1 2024-08-07T17:38:20.7964348Z distributed/test_c10d_common 1/1 2024-08-07T17:38:20.7964819Z distributed/_shard/sharding_plan/test_sharding_plan 1/1 2024-08-07T17:38:20.7965245Z distributed/checkpoint/test_state_dict 1/1 2024-08-07T17:38:20.7965589Z distributed/_shard/test_sharder 1/1 2024-08-07T17:38:20.7965927Z distributed/fsdp/test_fsdp_meta 1/1 2024-08-07T17:38:20.7966275Z distributed/rpc/test_faulty_agent 1/1 2024-08-07T17:38:20.7966624Z distributed/rpc/test_tensorpipe_agent 2/4 2024-08-07T17:38:20.7967000Z distributed/rpc/test_tensorpipe_agent 4/4 2024-08-07T17:38:20.7967398Z distributed/_shard/sharded_tensor/ops/test_init 1/1 2024-08-07T17:38:20.7967832Z distributed/_shard/sharded_optim/test_sharded_optim 1/1 2024-08-07T17:38:20.7968240Z distributed/test_c10d_spawn_gloo 1/1 2024-08-07T17:38:20.7968582Z distributed/fsdp/test_fsdp_comm 1/1 2024-08-07T17:38:20.7968972Z distributed/_shard/sharded_tensor/ops/test_embedding 1/1 2024-08-07T17:38:20.7969468Z distributed/_shard/sharded_tensor/ops/test_embedding_bag 1/1 2024-08-07T17:38:20.7969999Z distributed/_shard/sharded_tensor/test_sharded_tensor_reshard 1/1 2024-08-07T17:38:20.7970428Z distributed/test_store 1/1 2024-08-07T17:38:20.7970740Z distributed/test_c10d_spawn_ucc 1/1 2024-08-07T17:38:20.7971077Z distributed/test_c10d_spawn_nccl 1/1 2024-08-07T17:38:20.7971400Z distributed/test_c10d_nccl 1/1 2024-08-07T17:38:20.7971787Z distributed/_shard/sharded_tensor/test_sharded_tensor 1/1 2024-08-07T17:38:20.7972197Z distributed/test_launcher 1/1 2024-08-07T17:38:20.7972506Z distributed/test_c10d_ops_nccl 1/1 2024-08-07T17:38:20.7972847Z distributed/elastic/events/lib_test 1/1 2024-08-07T17:38:20.7973209Z distributed/elastic/metrics/api_test 1/1 2024-08-07T17:38:20.7973589Z distributed/elastic/multiprocessing/api_test 1/1 2024-08-07T17:38:20.7973980Z distributed/elastic/timer/api_test 1/1 2024-08-07T17:38:20.7974375Z distributed/elastic/timer/local_timer_example 1/1 2024-08-07T17:38:20.7974775Z distributed/elastic/timer/local_timer_test 1/1 2024-08-07T17:38:20.7975178Z distributed/elastic/utils/distributed_test 1/1 2024-08-07T17:38:20.7975568Z distributed/elastic/utils/logging_test 1/1 2024-08-07T17:38:20.7975918Z distributed/elastic/utils/util_test 1/1 2024-08-07T17:38:20.7976307Z distributed/pipelining/test_composability 1/1 2024-08-07T17:38:20.7976733Z distributed/tensor/parallel/test_ddp_2d_parallel 1/1 2024-08-07T17:38:20.7977171Z distributed/tensor/parallel/test_fsdp_2d_parallel 1/1 2024-08-07T17:38:20.7977558Z Parallel tests (0): 2024-08-07T17:38:20.7977835Z Name: excluded (est. time: 0.0min) 2024-08-07T17:38:20.7978127Z Serial tests (0): 2024-08-07T17:38:20.7978383Z Parallel tests (0): 2024-08-07T17:38:20.7978767Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-07T17:38:20.8187553Z Running distributed/_tensor/test_dtensor_compile 1/1 ... [2024-08-07 17:38:20.818273] 2024-08-07T17:38:20.8193538Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_dtensor_compile.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:38:20.818923] 2024-08-07T17:38:49.4595878Z 2024-08-07T17:38:49.4597378Z distributed/_tensor/test_dtensor_compile 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_dtensor_compile_1.1_27403634dda4d062_.log 2024-08-07T17:38:49.4612255Z Running 26 items in this shard: test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_device_mesh_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_attribute_access_on_intermediate, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_constructor_w_dynamo_disable, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_constructor_w_graph_break, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_contiguous_dtensor_noncontiguous_local_as_tangent, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_different_gradient_placement, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_dont_recompile_on_same_placement_devicemesh, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_dynamo_device_mesh_attrs, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_noncontiguous_output, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_partial_placement_graph_output, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dtensor_partial_placement_redistribute_unbalanced_correct_strides, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor_from_local, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor_from_local_redistribute, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_dtensor_recompile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_to_local_kwargs, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_dynamo_to_local_kwargs_forward_hook, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_fakify_dtensor, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_graph_input_is_async, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_placement_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompile::test_tp_compile_comm_reordering, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_2d_fsdp_tp_ac_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_2d_fsdp_tp_compile, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_compile_dtensor_redistribute_backward, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_tp_compile_fullgraph_is_seq_parallel_False, test/distributed/_tensor/test_dtensor_compile.py::TestDTensorCompileE2E::test_tp_compile_fullgraph_is_seq_parallel_True 2024-08-07T17:38:49.4624620Z 2024-08-07T17:38:49.4624925Z Running distributed/pipelining/test_backward 1/1 ... [2024-08-07 17:38:49.460324] 2024-08-07T17:38:49.4626263Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_backward.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:38:49.460761] 2024-08-07T17:38:52.5359591Z 2024-08-07T17:38:52.5361667Z distributed/pipelining/test_backward 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_backward_1.1_723b03cf90bd92c4_.log 2024-08-07T17:38:52.5364111Z Running 1 items in this shard: test/distributed/pipelining/test_backward.py::StageBackwardTests::test_stage_backward 2024-08-07T17:38:52.5365130Z 2024-08-07T17:38:52.5368699Z Running distributed/checkpoint/test_nested_dict 1/1 ... [2024-08-07 17:38:52.536656] 2024-08-07T17:38:52.5374246Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_nested_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:38:52.537149] 2024-08-07T17:38:55.6122030Z 2024-08-07T17:38:55.6123361Z distributed/checkpoint/test_nested_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_nested_dict_1.1_9deabd4db0d22dd1_.log 2024-08-07T17:38:55.6125375Z Running 2 items in this shard: test/distributed/checkpoint/test_nested_dict.py::TestFlattening::test_flattening_round_trip, test/distributed/checkpoint/test_nested_dict.py::TestFlattening::test_mapping 2024-08-07T17:38:55.6126397Z 2024-08-07T17:38:55.6128147Z Running distributed/_shard/sharded_tensor/test_logger 1/1 ... [2024-08-07 17:38:55.612621] 2024-08-07T17:38:55.6132557Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/test_logger.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:38:55.613048] 2024-08-07T17:38:58.6879327Z 2024-08-07T17:38:58.6880549Z distributed/_shard/sharded_tensor/test_logger 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.test_logger_1.1_654f904899cf6944_.log 2024-08-07T17:38:58.6882123Z Running 1 items in this shard: test/distributed/_shard/sharded_tensor/test_logger.py::ShardingSpecLoggerTest::test_get_or_create_logger 2024-08-07T17:38:58.6882793Z 2024-08-07T17:38:58.6886009Z Running distributed/pipelining/test_transformer 1/1 ... [2024-08-07 17:38:58.688430] 2024-08-07T17:38:58.6891545Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_transformer.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:38:58.688898] 2024-08-07T17:39:02.4152302Z 2024-08-07T17:39:02.4153531Z distributed/pipelining/test_transformer 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_transformer_1.1_bef753c7e8696009_.log 2024-08-07T17:39:02.4155088Z Running 1 items in this shard: test/distributed/pipelining/test_transformer.py::TransformerTests::test_ir 2024-08-07T17:39:02.4155688Z 2024-08-07T17:39:02.4158701Z Running distributed/test_symmetric_memory 1/1 ... [2024-08-07 17:39:02.415690] 2024-08-07T17:39:02.4174378Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_symmetric_memory.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:39:02.416169] 2024-08-07T17:39:05.1410098Z 2024-08-07T17:39:05.1411327Z distributed/test_symmetric_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_symmetric_memory_1.1_891cba56055c2a0e_.log 2024-08-07T17:39:05.1422252Z Running 20 items in this shard: test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_cuda_nvlink_connectivity_detection, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_empty_strided_p2p, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_empty_strided_p2p_persistent, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_matmul_gather_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_matmul_gather_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_scaled_matmul_gather_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_all_gather_scaled_matmul_gather_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_matmul_reduce_scatter_scatter_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_matmul_reduce_scatter_scatter_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_scaled_matmul_reduce_scatter_scatter_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_fused_scaled_matmul_reduce_scatter_scatter_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_all_gather_symm_mem_input_False, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_all_gather_symm_mem_input_True, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_avg_symm_mem_input_False, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_avg_symm_mem_input_True, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_sum_symm_mem_input_False, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_low_contention_reduce_scatter_reduce_op_sum_symm_mem_input_True, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_optimal_layout_dim_0, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_optimal_layout_dim_1, test/distributed/test_symmetric_memory.py::SymmetricMemoryTest::test_optimal_layout_dim_2 2024-08-07T17:39:05.1432765Z 2024-08-07T17:39:05.1433170Z Running distributed/tensor/parallel/test_micro_pipeline_tp 1/1 ... [2024-08-07 17:39:05.141421] 2024-08-07T17:39:05.1434734Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_micro_pipeline_tp.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:39:05.141878] 2024-08-07T17:39:09.0186004Z 2024-08-07T17:39:09.0187243Z distributed/tensor/parallel/test_micro_pipeline_tp 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_micro_pipeline_tp_1.1_d842afa1d972c0bb_.log 2024-08-07T17:39:09.0206436Z Running 29 items in this shard: test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_dtensor_seq_par_shard_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_dtensor_seq_par_shard_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_find_all_gather_patterns, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_find_reduce_scatter_patterns, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_2_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_2_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_2_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_3_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_3_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_matmul_A_dims_3_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_2_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_2_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_2_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_3_gather_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_3_gather_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_all_gather_scaled_matmul_A_dims_3_gather_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_2_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_2_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_2_scatter_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_3_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_3_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_matmul_reduce_scatter_A_dims_3_scatter_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_2_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_2_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_2_scatter_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_3_scatter_dim_0, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_3_scatter_dim_1, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_fuse_scaled_matmul_reduce_scatter_A_dims_3_scatter_dim_2, test/distributed/tensor/parallel/test_micro_pipeline_tp.py::MicroPipelineTPTest::test_get_unexposed_collectives 2024-08-07T17:39:09.0224664Z 2024-08-07T17:39:09.0225001Z Running distributed/test_dynamo_distributed 1/1 ... [2024-08-07 17:39:09.019069] 2024-08-07T17:39:09.0226412Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_dynamo_distributed.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:39:09.019533] 2024-08-07T17:39:39.2446716Z 2024-08-07T17:39:39.2448038Z distributed/test_dynamo_distributed 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_dynamo_distributed_1.1_d5a7cdfd0c4bf804_.log 2024-08-07T17:39:39.2469496Z Running 46 items in this shard: test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_call_method_forward, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_hf_bert_ddp_aot_eager, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_hf_bert_ddp_inductor, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_issue90375, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_symbol_splitting, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_unbacked_symbol_splitting_direct, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_unbacked_symbol_splitting_indirect, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_unbacked_symbol_splitting_no_binding, test/distributed/test_dynamo_distributed.py::TestFakeDistributedSingleProc::test_unbacked_symbol_splitting_torture_multi, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_compiler_collectives_automatic_dynamic_scalar, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_compiler_collectives_automatic_dynamic_speculation_divergence, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_compiler_collectives_automatic_dynamic_tensor, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_compiler_collectives_graph_break_empty_graph_still_collective, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_ddp_activation_checkpointing, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_ddp_baseline_aot_eager_multiprocess, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_activation_checkpointing, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_aot_eager, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_inductor, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_fsdp_setattr, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_aot_eager, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_aot_eager_static_graph, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_inductor, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_ddp_inductor_static_graph, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_fsdp, test/distributed/test_dynamo_distributed.py::TestMultiProc::test_hf_bert_fsdp_activation_checkpointing, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_aot_autograd, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_async_subclass_no_specialize, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_custom_layer, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_ddp_baseline_aot_eager, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_ddp_baseline_inductor, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_empty_graph_inductor, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_dup_tensors_diff_source, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_dup_tensors_same_source, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_orig_params_assert, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_skip_guards, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_skip_register_attr_or_module, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_fsdp_staticmethod, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_ctx_manager, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor_layout_optimizations_inference, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor_layout_optimizations_training, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_graph_split_inductor_transpose, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_higher_order_op, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_ignored_parameters, test/distributed/test_dynamo_distributed.py::TestSingleProc::test_no_split 2024-08-07T17:39:39.2488790Z 2024-08-07T17:39:39.2489043Z Running distributed/fsdp/test_fsdp_core 1/1 ... [2024-08-07 17:39:39.245627] 2024-08-07T17:39:39.2490311Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_core.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:39:39.246251] 2024-08-07T17:42:36.7511047Z 2024-08-07T17:42:36.7512146Z distributed/fsdp/test_fsdp_core 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_core_1.1_7a0f40f279bdb086_.log 2024-08-07T17:42:36.7542832Z Running 60 items in this shard: test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_optim_step_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_optim_step_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_optim_step_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_optim_step_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_optim_step_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_optim_step_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_reduce_scatter_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_reduce_scatter_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_reduce_scatter_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_reduce_scatter_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_reduce_scatter_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_delayed_reduce_scatter_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_with_delay_before_free_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_with_delay_before_free_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_with_delay_before_free_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_with_delay_before_free_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_with_delay_before_free_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_mixture_of_experts_with_delay_before_free_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_always_wrap_model_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_always_wrap_model_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_always_wrap_model_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_always_wrap_model_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_always_wrap_model_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_always_wrap_model_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_single_iteration_mixed_precision_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_single_iteration_mixed_precision_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_single_iteration_mixed_precision_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_single_iteration_mixed_precision_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_single_iteration_mixed_precision_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_nested_wrapped_model_single_iteration_mixed_precision_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_transformer_offload_false_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_transformer_offload_false_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_transformer_offload_false_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_transformer_offload_true_no_shard, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_transformer_offload_true_none, test/distributed/fsdp/test_fsdp_core.py::TestParityWithDDP::test_transformer_offload_true_shard_grad_op, test/distributed/fsdp/test_fsdp_core.py::TestParamInit::test_param_change_after_init_mixed_precision_False, test/distributed/fsdp/test_fsdp_core.py::TestParamInit::test_param_change_after_init_mixed_precision_True, test/distributed/fsdp/test_fsdp_core.py::TestHooks::test_pre_backward_hook_registration_after_state_dict, test/distributed/fsdp/test_fsdp_core.py::TestHooks::test_pre_backward_hook_registration_cuda_first_False, test/distributed/fsdp/test_fsdp_core.py::TestHooks::test_pre_backward_hook_registration_cuda_first_True, test/distributed/fsdp/test_fsdp_core.py::TestHooks::test_register_functions_called_cuda_first_False_mixed_precision_False, test/distributed/fsdp/test_fsdp_core.py::TestHooks::test_register_functions_called_cuda_first_False_mixed_precision_True, test/distributed/fsdp/test_fsdp_core.py::TestHooks::test_register_functions_called_cuda_first_True_mixed_precision_False, test/distributed/fsdp/test_fsdp_core.py::TestHooks::test_register_functions_called_cuda_first_True_mixed_precision_True, test/distributed/fsdp/test_fsdp_core.py::TestNoGrad::test_transformer_no_grad_mixed_precision_False, test/distributed/fsdp/test_fsdp_core.py::TestNoGrad::test_transformer_no_grad_mixed_precision_True, test/distributed/fsdp/test_fsdp_core.py::TestAutograd::test_unshard_params_as_tensors 2024-08-07T17:42:36.7571981Z 2024-08-07T17:42:36.7572323Z Running distributed/pipelining/test_schedule_multiproc 1/1 ... [2024-08-07 17:42:36.751361] 2024-08-07T17:42:36.7573746Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_schedule_multiproc.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:42:36.751860] 2024-08-07T17:42:39.5423901Z 2024-08-07T17:42:39.5425704Z distributed/pipelining/test_schedule_multiproc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_schedule_multiproc_1.1_5c816662d6e371a8_.log 2024-08-07T17:42:39.5427298Z 2024-08-07T17:42:39.5430478Z Running distributed/_composable/test_contract 1/1 ... [2024-08-07 17:42:39.542832] 2024-08-07T17:42:39.5436770Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/test_contract.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:42:39.543271] 2024-08-07T17:42:42.3169005Z 2024-08-07T17:42:42.3170069Z distributed/_composable/test_contract 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_contract_1.1_280d7e24038acd5b_.log 2024-08-07T17:42:42.3172595Z Running 5 items in this shard: test/distributed/_composable/test_contract.py::TestContract::test_add_hooks, test/distributed/_composable/test_contract.py::TestContract::test_modify_fqn, test/distributed/_composable/test_contract.py::TestContract::test_multi_module_api, test/distributed/_composable/test_contract.py::TestContract::test_registry, test/distributed/_composable/test_contract.py::TestContract::test_state 2024-08-07T17:42:42.3174406Z 2024-08-07T17:42:42.3175599Z Running distributed/test_device_mesh 1/1 ... [2024-08-07 17:42:42.317381] 2024-08-07T17:42:42.3180923Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_device_mesh.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:42:42.317825] 2024-08-07T17:45:08.0441795Z 2024-08-07T17:45:08.0442802Z distributed/test_device_mesh 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_device_mesh_1.1_887754677d247eaf_.log 2024-08-07T17:45:08.0458640Z Running 39 items in this shard: test/distributed/test_device_mesh.py::DeviceMeshTest::test_assert_invalid_mesh_tensor, test/distributed/test_device_mesh.py::DeviceMeshTest::test_device_mesh_2d, test/distributed/test_device_mesh.py::DeviceMeshTest::test_device_mesh_init_backend, test/distributed/test_device_mesh.py::DeviceMeshTest::test_fake_pg_device_mesh, test/distributed/test_device_mesh.py::DeviceMeshTest::test_from_group_with_global_pg, test/distributed/test_device_mesh.py::DeviceMeshTest::test_from_group_with_invalid_mesh, test/distributed/test_device_mesh.py::DeviceMeshTest::test_get_group_and_get_all_groups, test/distributed/test_device_mesh.py::DeviceMeshTest::test_get_local_rank, test/distributed/test_device_mesh.py::DeviceMeshTest::test_get_local_rank_raises_exception, test/distributed/test_device_mesh.py::DeviceMeshTest::test_init_process_group, test/distributed/test_device_mesh.py::DeviceMeshTest::test_raises_invalid_device_type, test/distributed/test_device_mesh.py::DeviceMeshTest::test_set_mesh_dim_group_options, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_device_mesh_hash, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_device_mesh_nd, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_device_mesh_parent_child_hash, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_from_group_with_mesh_shape, test/distributed/test_device_mesh.py::DeviceMeshTestNDim::test_get_local_rank_3d, test/distributed/test_device_mesh.py::InitDeviceMeshTest::test_init_device_mesh, test/distributed/test_device_mesh.py::InitDeviceMeshTest::test_raises_duplicate_mesh_dim_names, test/distributed/test_device_mesh.py::InitDeviceMeshTest::test_raises_mesh_shape_mesh_dim_names_mismatch, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_cache_and_reuse_submesh_slice_result, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_1d, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_2d, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_3d, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_get_item_3d_noncontinuous_slicing, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_raises_invalid_mesh_dim_name, test/distributed/test_device_mesh.py::TestDeviceMeshGetItem::test_raises_no_mesh_dim_found, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_mesh_dim_by_name, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_parent_mesh, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_parent_mesh_dim_exist, test/distributed/test_device_mesh.py::TestMeshEnv::test_get_parent_mesh_dim_not_exist, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_all_gather_uneven, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_broadcast_1d, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_broadcast_nd, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_reduce_scatter_contiguous, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_reduce_scatter_uneven, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_scatter_1d, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_scatter_nd, test/distributed/test_device_mesh.py::DeviceMeshCollectiveTest::test_scatter_uneven 2024-08-07T17:45:08.0472780Z 2024-08-07T17:45:08.0473064Z Running distributed/_tensor/test_tensor_ops 1/1 ... [2024-08-07 17:45:08.044594] 2024-08-07T17:45:08.0474697Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_tensor_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:45:08.045047] 2024-08-07T17:46:30.9430863Z 2024-08-07T17:46:30.9432081Z distributed/_tensor/test_tensor_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_tensor_ops_1.1_25043569cb7d2846_.log 2024-08-07T17:46:30.9442780Z Running 25 items in this shard: test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_aten_contiguous, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_clone, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_contiguous, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_detach, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_dtensor_dtype_conversion, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_empty_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_equal, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_fill_inplace, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_fill_inplace_partial_sum, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_full_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_gather, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_index, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_inplace_op, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_new_empty_strided, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_new_full, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_ones_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_ones_like_partial_sum, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_op_out_variant, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_scatter, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_slice, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_stack, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_where_type_promotion, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_zero_inplace, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_zeros_like, test/distributed/_tensor/test_tensor_ops.py::DistTensorOpsTest::test_zeros_like_partial_sum 2024-08-07T17:46:30.9452433Z 2024-08-07T17:46:30.9452761Z Running distributed/_tensor/test_random_ops 1/1 ... [2024-08-07 17:46:30.944084] 2024-08-07T17:46:30.9454139Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_random_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:46:30.944500] 2024-08-07T17:46:54.4131800Z 2024-08-07T17:46:54.4133294Z distributed/_tensor/test_random_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_random_ops_1.1_d09b38c08bcf30fc_.log 2024-08-07T17:46:54.4137054Z Running 7 items in this shard: test/distributed/_tensor/test_random_ops.py::DistTensorRandomInitTest::test_init_ops, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_dropout_1d, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_rand_1d, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_uniform_2d, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_manual_seed, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_meta_tensor_init, test/distributed/_tensor/test_random_ops.py::DistTensorRandomOpTest::test_rng_tracker_init 2024-08-07T17:46:54.4140654Z 2024-08-07T17:46:54.4141437Z Running distributed/_tensor/experimental/test_register_sharding 1/1 ... [2024-08-07 17:46:54.413589] 2024-08-07T17:46:54.4143782Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/experimental/test_register_sharding.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:46:54.414063] 2024-08-07T17:47:03.5027450Z 2024-08-07T17:47:03.5029585Z distributed/_tensor/experimental/test_register_sharding 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.experimental.test_register_sharding_1.1_2a8ab8905cc47b76_.log 2024-08-07T17:47:03.5033274Z Running 2 items in this shard: test/distributed/_tensor/experimental/test_register_sharding.py::TestRegisterSharding::test_argmax, test/distributed/_tensor/experimental/test_register_sharding.py::TestRegisterSharding::test_softmax_fwd 2024-08-07T17:47:03.5035053Z 2024-08-07T17:47:03.5036162Z Running distributed/tensor/parallel/test_tp_examples 1/1 ... [2024-08-07 17:47:03.503399] 2024-08-07T17:47:03.5043547Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_tp_examples.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:47:03.503958] 2024-08-07T17:47:33.3404583Z 2024-08-07T17:47:33.3405762Z distributed/tensor/parallel/test_tp_examples 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_tp_examples_1.1_eca9453f07e52aaf_.log 2024-08-07T17:47:33.3412185Z Running 9 items in this shard: test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_loss_parallel, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_mlp_inference, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_mlp_training_is_seq_parallel_False_recompute_activation_False, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_mlp_training_is_seq_parallel_True_recompute_activation_False, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_False_float32, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_False_float64, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_True_float32, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_True_float64, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_weight_tying 2024-08-07T17:47:33.3417948Z 2024-08-07T17:47:33.3418204Z Running distributed/_tensor/test_math_ops 1/1 ... [2024-08-07 17:47:33.340921] 2024-08-07T17:47:33.3419473Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_math_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:47:33.341388] 2024-08-07T17:48:11.2483025Z 2024-08-07T17:48:11.2484162Z distributed/_tensor/test_math_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_math_ops_1.1_de716607eb414837_.log 2024-08-07T17:48:11.2488966Z Running 11 items in this shard: test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_foreach_norm, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_layer_norm_bwd, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_layer_norm_fwd, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_linear_op_reductions, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_mean, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_nll_loss_and_cross_entropy, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_shard0_svd, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_shard_math_ops, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_softmax_fwd, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_softmax_with_bwd, test/distributed/_tensor/test_math_ops.py::DistMathOpsTest::test_topk 2024-08-07T17:48:11.2492902Z 2024-08-07T17:48:11.2493461Z Running distributed/optim/test_named_optimizer 1/1 ... [2024-08-07 17:48:11.248777] 2024-08-07T17:48:11.2494854Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/optim/test_named_optimizer.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:48:11.249210] 2024-08-07T17:48:13.3923576Z 2024-08-07T17:48:13.3924827Z distributed/optim/test_named_optimizer 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.optim.test_named_optimizer_1.1_cd04f844150c9dc3_.log 2024-08-07T17:48:13.3925703Z 2024-08-07T17:48:13.3929600Z Running distributed/_tensor/test_redistribute 1/1 ... [2024-08-07 17:48:13.392784] 2024-08-07T17:48:13.3935023Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_redistribute.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:48:13.393260] 2024-08-07T17:48:54.4503617Z 2024-08-07T17:48:54.4506590Z distributed/_tensor/test_redistribute 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_redistribute_1.1_483d96b06839d388_.log 2024-08-07T17:48:54.4512908Z Running 12 items in this shard: test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_partial_to_replicate_forward_backward, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_partial_to_shard, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_redistribute_negative_shard_dim, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_redistribute_shard_dim_change, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_redistribute_uneven_sharding, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_replicate_to_local_partial_grad, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_replicate_to_partial, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_replicate_to_replicate_forward_backward, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_replicate_to_shard_forward_backward, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_shard_dim_alltoall, test/distributed/_tensor/test_redistribute.py::RedistributeTest::test_shard_to_replicate_forward_backward, test/distributed/_tensor/test_redistribute.py::MultiDimRedistributeTest::test_multi_dim_mesh 2024-08-07T17:48:54.4518802Z 2024-08-07T17:48:54.4519101Z Running distributed/checkpoint/test_format_utils 1/1 ... [2024-08-07 17:48:54.450965] 2024-08-07T17:48:54.4520457Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_format_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:48:54.451441] 2024-08-07T17:49:06.4458891Z 2024-08-07T17:49:06.4460053Z distributed/checkpoint/test_format_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_format_utils_1.1_54e74efdf0f6c1ec_.log 2024-08-07T17:49:06.4462543Z Running 3 items in this shard: test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_dcp_to_torch_save, test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_online_torch_save_to_dcp, test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_torch_save_to_dcp 2024-08-07T17:49:06.4464026Z 2024-08-07T17:49:06.4465830Z Running distributed/_tensor/experimental/test_tp_transform 1/1 ... [2024-08-07 17:49:06.446383] 2024-08-07T17:49:06.4470739Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/experimental/test_tp_transform.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:49:06.446833] 2024-08-07T17:49:20.2961455Z 2024-08-07T17:49:20.2962668Z distributed/_tensor/experimental/test_tp_transform 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.experimental.test_tp_transform_1.1_5dea6052b71ee941_.log 2024-08-07T17:49:20.2965109Z Running 3 items in this shard: test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_e2e, test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_no_bias, test/distributed/_tensor/experimental/test_tp_transform.py::TensorParallelTest::test_tp_transform_with_uncovered_op 2024-08-07T17:49:20.2966873Z 2024-08-07T17:49:20.2967472Z Running distributed/_tensor/experimental/test_local_map 1/1 ... [2024-08-07 17:49:20.296572] 2024-08-07T17:49:20.2972539Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/experimental/test_local_map.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:49:20.297016] 2024-08-07T17:49:34.6434661Z 2024-08-07T17:49:34.6436222Z distributed/_tensor/experimental/test_local_map 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.experimental.test_local_map_1.1_4d11346e47505839_.log 2024-08-07T17:49:34.6439016Z Running 4 items in this shard: test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_correctness, test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_in_placements, test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_out_placements, test/distributed/_tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_redistribute 2024-08-07T17:49:34.6441598Z 2024-08-07T17:49:34.6442073Z Running distributed/checkpoint/test_fsdp_optim_state 1/1 ... [2024-08-07 17:49:34.643937] 2024-08-07T17:49:34.6446840Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_fsdp_optim_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:49:34.644461] 2024-08-07T17:49:43.3319518Z 2024-08-07T17:49:43.3320932Z distributed/checkpoint/test_fsdp_optim_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_fsdp_optim_state_1.1_6f380acf678099bb_.log 2024-08-07T17:49:43.3323145Z Running 2 items in this shard: test/distributed/checkpoint/test_fsdp_optim_state.py::FsdpOptimStateCheckpoint::test_load_sharded_optimizer_state_dict_pass_planner_False, test/distributed/checkpoint/test_fsdp_optim_state.py::FsdpOptimStateCheckpoint::test_load_sharded_optimizer_state_dict_pass_planner_True 2024-08-07T17:49:43.3324564Z 2024-08-07T17:49:43.3325750Z Running distributed/checkpoint/test_dtensor_resharding 1/1 ... [2024-08-07 17:49:43.332380] 2024-08-07T17:49:43.3330989Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_dtensor_resharding.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:49:43.332860] 2024-08-07T17:50:00.8394312Z 2024-08-07T17:50:00.8395794Z distributed/checkpoint/test_dtensor_resharding 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_dtensor_resharding_1.1_72c25f9e5b4d5dd9_.log 2024-08-07T17:50:00.8399829Z Running 5 items in this shard: test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardPlacementChange::test_1d_to_1d_reshard_placement_change, test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardPlacementChange::test_2d_to_2d_reshard_placement_change, test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardMeshChange::test_1d_to_2d_reshard_mesh_change, test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardMeshChange::test_2d_to_1d_reshard_mesh_change, test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardMeshChange::test_dtensor_checkpoint_resharding_with_empty_shard 2024-08-07T17:50:00.8402998Z 2024-08-07T17:50:00.8403303Z Running distributed/checkpoint/test_tp_checkpoint 1/1 ... [2024-08-07 17:50:00.839954] 2024-08-07T17:50:00.8406900Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_tp_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:50:00.840438] 2024-08-07T17:50:09.8791311Z 2024-08-07T17:50:09.8792504Z distributed/checkpoint/test_tp_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_tp_checkpoint_1.1_597ac79be7499ade_.log 2024-08-07T17:50:09.8794342Z Running 2 items in this shard: test/distributed/checkpoint/test_tp_checkpoint.py::TestTpCheckpoint::test_tp_checkpoint, test/distributed/checkpoint/test_tp_checkpoint.py::TestTpCheckpoint::test_tp_checkpoint_load_on_meta_device 2024-08-07T17:50:09.8797087Z 2024-08-07T17:50:09.8797445Z Running distributed/_tensor/test_pointwise_ops 1/1 ... [2024-08-07 17:50:09.879555] 2024-08-07T17:50:09.8802599Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/test_pointwise_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:50:09.880008] 2024-08-07T17:50:13.0059656Z 2024-08-07T17:50:13.0060918Z distributed/_tensor/test_pointwise_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.test_pointwise_ops_1.1_fe12ae9a6e8b8056_.log 2024-08-07T17:50:13.0064586Z Running 7 items in this shard: test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_activations, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_dropout, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_dropout_backward, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_dropout_errors, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_mul_out, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_partial_add, test/distributed/_tensor/test_pointwise_ops.py::DistElementwiseOpsTest::test_partial_mul 2024-08-07T17:50:13.0067569Z 2024-08-07T17:50:13.0067841Z Running distributed/checkpoint/test_fsspec 1/1 ... [2024-08-07 17:50:13.006401] 2024-08-07T17:50:13.0070644Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_fsspec.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:50:13.006840] 2024-08-07T17:50:15.6818821Z 2024-08-07T17:50:15.6819912Z distributed/checkpoint/test_fsspec 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_fsspec_1.1_a928051542ebbc5e_.log 2024-08-07T17:50:15.6821729Z Running 2 items in this shard: test/distributed/checkpoint/test_fsspec.py::TestFSSpec::test_fsspec, test/distributed/checkpoint/test_fsspec.py::TestFSSpec::test_overwrite 2024-08-07T17:50:15.6822620Z 2024-08-07T17:50:15.6824767Z Running distributed/test_collective_utils 1/1 ... [2024-08-07 17:50:15.682319] 2024-08-07T17:50:15.6829851Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_collective_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:50:15.682770] 2024-08-07T17:50:18.2991960Z 2024-08-07T17:50:18.2993693Z distributed/test_collective_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_collective_utils_1.1_075eb1f3e483964d_.log 2024-08-07T17:50:18.2995174Z 2024-08-07T17:50:18.2998382Z Running distributed/fsdp/test_fsdp_optim_state 1/1 ... [2024-08-07 17:50:18.299620] 2024-08-07T17:50:18.3005016Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_optim_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:50:18.300158] 2024-08-07T17:53:29.5822355Z 2024-08-07T17:53:29.5823694Z distributed/fsdp/test_fsdp_optim_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_optim_state_1.1_55cb4daf6c7c2cce_.log 2024-08-07T17:53:29.5863704Z Running 60 items in this shard: test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_compatible_with_trec, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_flatten_sharded_optim_state_dict_nested, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_flatten_sharded_optim_state_dict_transformer, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_full_optim_state_dict_keys, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_full_optim_state_dict_nested_invalid, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_interface_arguments, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_no_grad, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_input_warning, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_without_param_groups, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type0_use_multiple_param_groups_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type0_use_multiple_param_groups_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type1_use_multiple_param_groups_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type1_use_multiple_param_groups_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_names, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_save_load_without_0th_param_state_state_dict_type0, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_save_load_without_0th_param_state_state_dict_type1, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_halve_world_size, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_transformer, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_halve_world_size, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_transformer, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type0_add_to_fsdp_module_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type0_add_to_fsdp_module_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type1_add_to_fsdp_module_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type1_add_to_fsdp_module_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_state_dict_with_none_tensor_state, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_use_orig_params, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_with_empty_optimizer_state, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_with_no_shard 2024-08-07T17:53:29.5901731Z 2024-08-07T17:53:29.5902100Z Running distributed/checkpoint/test_fsdp_tp_checkpoint_conversion 1/1 ... [2024-08-07 17:53:29.583169] 2024-08-07T17:53:29.5903603Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_fsdp_tp_checkpoint_conversion.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:53:29.583576] 2024-08-07T17:53:35.5652406Z 2024-08-07T17:53:35.5653468Z distributed/checkpoint/test_fsdp_tp_checkpoint_conversion 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_fsdp_tp_checkpoint_conversion_1.1_8c88faaa58245cf6_.log 2024-08-07T17:53:35.5656607Z Running 1 items in this shard: test/distributed/checkpoint/test_fsdp_tp_checkpoint_conversion.py::TestFsdpTpCheckpointConversion::test_fsdp_to_tp 2024-08-07T17:53:35.5657293Z 2024-08-07T17:53:35.5659275Z Running distributed/test_fake_pg 1/1 ... [2024-08-07 17:53:35.565697] 2024-08-07T17:53:35.5663823Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_fake_pg.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:53:35.566150] 2024-08-07T17:53:38.5910507Z 2024-08-07T17:53:38.5911785Z distributed/test_fake_pg 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_fake_pg_1.1_af9b709a23444167_.log 2024-08-07T17:53:38.5915867Z Running 13 items in this shard: test/distributed/test_fake_pg.py::TestFakePG::test_all_reduce, test/distributed/test_fake_pg.py::TestFakePG::test_allgather, test/distributed/test_fake_pg.py::TestFakePG::test_alltoall, test/distributed/test_fake_pg.py::TestFakePG::test_alltoall_base, test/distributed/test_fake_pg.py::TestFakePG::test_broadcast, test/distributed/test_fake_pg.py::TestFakePG::test_construct_fsdp, test/distributed/test_fake_pg.py::TestFakePG::test_fake_pg_tracing, test/distributed/test_fake_pg.py::TestFakePG::test_fsdp_fake_e2e, test/distributed/test_fake_pg.py::TestFakePG::test_fsdp_tp_fake_e2e, test/distributed/test_fake_pg.py::TestFakePG::test_recv, test/distributed/test_fake_pg.py::TestFakePG::test_reduce_scatter, test/distributed/test_fake_pg.py::TestFakePG::test_scatter, test/distributed/test_fake_pg.py::TestFakePG::test_send 2024-08-07T17:53:38.5919490Z 2024-08-07T17:53:38.5919773Z Running distributed/_tensor/debug/test_comm_mode 1/1 ... [2024-08-07 17:53:38.591503] 2024-08-07T17:53:38.5921730Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_tensor/debug/test_comm_mode.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:53:38.591949] 2024-08-07T17:53:41.5163890Z 2024-08-07T17:53:41.5165948Z distributed/_tensor/debug/test_comm_mode 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tensor.debug.test_comm_mode_1.1_84ccf848e8d8ba3f_.log 2024-08-07T17:53:41.5170457Z Running 4 items in this shard: test/distributed/_tensor/debug/test_comm_mode.py::TestCommMode::test_comm_mode, test/distributed/_tensor/debug/test_comm_mode.py::TestCommMode::test_comm_mode_coalesced, test/distributed/_tensor/debug/test_comm_mode.py::TestCommMode::test_comm_mode_with_c10d, test/distributed/_tensor/debug/test_comm_mode.py::TestCommMode::test_comm_mode_with_dtensor 2024-08-07T17:53:41.5173256Z 2024-08-07T17:53:41.5173714Z Running distributed/checkpoint/test_utils 1/1 ... [2024-08-07 17:53:41.517104] 2024-08-07T17:53:41.5179866Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:53:41.517651] 2024-08-07T17:53:44.3918183Z 2024-08-07T17:53:44.3919387Z distributed/checkpoint/test_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_utils_1.1_83acbd0140a20a79_.log 2024-08-07T17:53:44.3922579Z Running 6 items in this shard: test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_dcp_logger, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_flat_data, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_index_hint_ignored_on_equals, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_index_hint_ignored_on_hash, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_init_convert_offset, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_sharded_tensor_lookup 2024-08-07T17:53:44.3925136Z 2024-08-07T17:53:44.3925473Z Running distributed/_composable/fsdp/test_fully_shard_init 1/1 ... [2024-08-07 17:53:44.392302] 2024-08-07T17:53:44.3929643Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_init.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:53:44.392739] 2024-08-07T17:53:47.3668063Z 2024-08-07T17:53:47.3670407Z distributed/_composable/fsdp/test_fully_shard_init 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_init_1.1_8516b3de56b3a2b1_.log 2024-08-07T17:53:47.3698032Z Running 28 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardDeviceTensor::test_move_states_to_device_tensor, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardDeviceDTensor::test_move_states_to_device_dtensor_invalid, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardDeviceDTensor::test_move_states_to_device_dtensor_valid, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMeshArg::test_invalid_mesh_ndim, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_duplicate, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_list_of_mlps, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_nested, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_nested_fully_shard_and_replicate, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_single, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_states_list_of_mlps, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_states_nested_fully_shard, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_states_shared_params_and_buffers, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardParamModuleInfos::test_get_param_module_infos_duplicates, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardParamModuleInfos::test_get_param_module_infos_list_of_mlps, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardParamModuleInfos::test_get_param_module_infos_shared_params, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardedParameterTensor::test_shard_tensor_parameters, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardedParameterDTensor::test_shard_dtensor_parameters, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_double_lazy_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_is_root, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_module_and_param_fqns, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_multi_module_root, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_invalid_meta_device_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_meta_device_1d_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_meta_device_2d_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_rank0_broadcast_meta_device_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardProcessGroupInit::test_1d_process_group_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardProcessGroupInit::test_2d_process_group_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardHSDPBroadcast::test_hsdp_broadcast_across_replicas 2024-08-07T17:53:47.3722722Z 2024-08-07T17:53:47.3723210Z Running distributed/test_c10d_logger 1/1 ... [2024-08-07 17:53:47.367579] 2024-08-07T17:53:47.3725459Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_logger.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:53:47.368148] 2024-08-07T17:53:58.2067909Z 2024-08-07T17:53:58.2069528Z distributed/test_c10d_logger 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_logger_1.1_d2c36dbc8d014b65_.log 2024-08-07T17:53:58.2072982Z Running 3 items in this shard: test/distributed/test_c10d_logger.py::C10dErrorLoggerTest::test_exception_logger, test/distributed/test_c10d_logger.py::C10dErrorLoggerTest::test_get_or_create_logger, test/distributed/test_c10d_logger.py::C10dErrorLoggerTest::test_time_logger 2024-08-07T17:53:58.2075131Z 2024-08-07T17:53:58.2076892Z Running distributed/fsdp/test_shard_utils 1/1 ... [2024-08-07 17:53:58.207482] 2024-08-07T17:53:58.2083633Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_shard_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:53:58.208026] 2024-08-07T17:54:06.9931651Z 2024-08-07T17:54:06.9932764Z distributed/fsdp/test_shard_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_shard_utils_1.1_91e612eaea63a7b3_.log 2024-08-07T17:54:06.9934551Z Running 2 items in this shard: test/distributed/fsdp/test_shard_utils.py::TestShardUtilsDistributed::test_create_chunk_sharded_tensor, test/distributed/fsdp/test_shard_utils.py::TestShardUtilsDistributedDTensor::test_create_chunk_dtensor 2024-08-07T17:54:06.9935626Z 2024-08-07T17:54:06.9937890Z Running distributed/test_nccl 1/1 ... [2024-08-07 17:54:06.993590] 2024-08-07T17:54:06.9942987Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_nccl.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:54:06.994071] 2024-08-07T17:54:10.0942711Z 2024-08-07T17:54:10.0943993Z distributed/test_nccl 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_nccl_1.1_fad9c73172d8063a_.log 2024-08-07T17:54:10.0944792Z Running 0 items in this shard: 2024-08-07T17:54:10.0945002Z 2024-08-07T17:54:10.0948662Z Running distributed/rpc/test_share_memory 1/1 ... [2024-08-07 17:54:10.094677] 2024-08-07T17:54:10.0953874Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_share_memory.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:54:10.095132] 2024-08-07T17:54:19.3730831Z 2024-08-07T17:54:19.3732223Z distributed/rpc/test_share_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_share_memory_1.1_829428c73b4beb81_.log 2024-08-07T17:54:19.3733406Z Running 1 items in this shard: test/distributed/rpc/test_share_memory.py::TestRPCPickler::test_case 2024-08-07T17:54:19.3734222Z Running 1 items in this shard: test/distributed/rpc/test_share_memory.py::TestRPCPickler::test_case 2024-08-07T17:54:19.3734688Z 2024-08-07T17:54:19.3737615Z Running distributed/checkpoint/test_fsdp_model_state 1/1 ... [2024-08-07 17:54:19.373544] 2024-08-07T17:54:19.3742773Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_fsdp_model_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:54:19.374014] 2024-08-07T17:54:28.4629100Z 2024-08-07T17:54:28.4630699Z distributed/checkpoint/test_fsdp_model_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_fsdp_model_state_1.1_ecb5c83af14f216b_.log 2024-08-07T17:54:28.4632714Z Running 2 items in this shard: test/distributed/checkpoint/test_fsdp_model_state.py::FsdpModelStateCheckpoint::test_fsdp_model_state_no_resharding, test/distributed/checkpoint/test_fsdp_model_state.py::FsdpModelStateCheckpoint::test_fsdp_model_state_with_resharding 2024-08-07T17:54:28.4634044Z 2024-08-07T17:54:28.4635294Z Running distributed/tensor/parallel/test_parallelize_api 1/1 ... [2024-08-07 17:54:28.463323] 2024-08-07T17:54:28.4640395Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_parallelize_api.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:54:28.463755] 2024-08-07T17:55:01.4544178Z 2024-08-07T17:55:01.4547263Z distributed/tensor/parallel/test_parallelize_api 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_parallelize_api_1.1_a840e28441e9632e_.log 2024-08-07T17:55:01.4553724Z Running 10 items in this shard: test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_linear_col_wise_parallel, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_linear_row_wise_parallel, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_parallelize_mlp_with_module_api, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_parallelize_mlp_with_module_api_nested, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_parallelize_module_multi_wildcard, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_parallelize_module_with_digit, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_parallelize_module_with_question, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_parallelize_module_with_star, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_prepare_module_input, test/distributed/tensor/parallel/test_parallelize_api.py::TensorParallelAPITests::test_prepare_module_output 2024-08-07T17:55:01.4559674Z 2024-08-07T17:55:01.4559962Z Running distributed/test_control_collectives 1/1 ... [2024-08-07 17:55:01.454464] 2024-08-07T17:55:01.4561281Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_control_collectives.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:55:01.454912] 2024-08-07T17:55:04.6306856Z 2024-08-07T17:55:04.6308417Z distributed/test_control_collectives 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_control_collectives_1.1_f2425810dab4c571_.log 2024-08-07T17:55:04.6315043Z Running 13 items in this shard: test/distributed/test_control_collectives.py::TestCollectives::test_all_gather_timeout, test/distributed/test_control_collectives.py::TestCollectives::test_all_sum, test/distributed/test_control_collectives.py::TestCollectives::test_all_sum_timeout, test/distributed/test_control_collectives.py::TestCollectives::test_barrier, test/distributed/test_control_collectives.py::TestCollectives::test_barrier_timeout, test/distributed/test_control_collectives.py::TestCollectives::test_broadcast, test/distributed/test_control_collectives.py::TestCollectives::test_broadcast_timeout, test/distributed/test_control_collectives.py::TestCollectives::test_gather, test/distributed/test_control_collectives.py::TestCollectives::test_gather_timeout, test/distributed/test_control_collectives.py::TestCollectives::test_scatter, test/distributed/test_control_collectives.py::TestCollectives::test_scatter_timeout, test/distributed/test_control_collectives.py::TestCollectives::test_simple_user_func, test/distributed/test_control_collectives.py::TestCollectives::test_unique 2024-08-07T17:55:04.6320745Z 2024-08-07T17:55:04.6321127Z Running distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_ 1/1 ... [2024-08-07 17:55:04.631147] 2024-08-07T17:55:04.6322791Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:55:04.631591] 2024-08-07T17:55:07.7073809Z 2024-08-07T17:55:07.7075399Z distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_ 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_clip_grad_norm__1.1_550162c2b93d941a_.log 2024-08-07T17:55:07.7077580Z Running 2 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_.py::TestClipGradNormWorldSize2::test_clip_grad_norm_1d, test/distributed/_composable/fsdp/test_fully_shard_clip_grad_norm_.py::TestClipGradNormWorldSize4::test_clip_grad_norm_2d 2024-08-07T17:55:07.7078838Z 2024-08-07T17:55:07.7080177Z Running distributed/test_distributed_spawn 1/3 ... [2024-08-07 17:55:07.707824] 2024-08-07T17:55:07.7148495Z MPI not available -- MPI backend tests will be skipped 2024-08-07T17:55:07.7149835Z Running distributed tests for the test backend with env init_method 2024-08-07T17:55:07.7158655Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=1', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:55:07.715619] 2024-08-07T17:55:10.6849087Z 2024-08-07T17:55:10.6850153Z distributed/test_distributed_spawn 1/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_1.3_e8697588080e6458_.log 2024-08-07T17:55:10.6851120Z Running 0 items in this shard: 2024-08-07T17:55:10.6851317Z 2024-08-07T17:55:10.6858881Z Running distributed tests for the test backend with file init_method 2024-08-07T17:55:10.6864240Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=1', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:55:10.686185] 2024-08-07T17:55:13.6633438Z 2024-08-07T17:55:13.6634496Z distributed/test_distributed_spawn 1/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_1.3_56ed6e9c8f7c449c_.log 2024-08-07T17:55:13.6635427Z Running 0 items in this shard: 2024-08-07T17:55:13.6635624Z 2024-08-07T17:55:13.6642799Z Running distributed tests for the gloo backend with env init_method 2024-08-07T17:55:13.6648680Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=1', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 17:55:13.664618] 2024-08-07T18:04:46.9361692Z 2024-08-07T18:04:46.9362767Z distributed/test_distributed_spawn 1/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_1.3_24c56f5018b6d515_.log 2024-08-07T18:04:46.9426622Z Running 102 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_1_level_hierarchical_model_averager_equivalent_to_periodic_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_3_level_hierarchical_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_Backend_enum_class, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_2D_Input, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Channels_Last, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_Running_Value, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_No_Affine, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_requires_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_cat_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_default_pg, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max_complex_unsupported, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo_tags, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_mixed_backend_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_list_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_object_list, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_with_logger, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping_requires_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_compile_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_create_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_forward_backward_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_pickling_powerSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_ignore_params_arg, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_cpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_ignored_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_execution_trace, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_python_error_logged, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_shared_grad_acc_unused_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_module_states, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_join_disable, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs_stop_iteration_sync_bn, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_default_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_no_find_unused, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_data_parallel_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_future, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_invalid_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_irecv, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_failure_order, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_init_abort, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_world_size_not_divisible_by_group_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_tuple_module, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_api_cpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_without_logger 2024-08-07T18:04:46.9474370Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_1_level_hierarchical_model_averager_equivalent_to_periodic_model_averager 2024-08-07T18:04:46.9475726Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_3_level_hierarchical_model_averager 2024-08-07T18:04:46.9476817Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_Backend_enum_class 2024-08-07T18:04:46.9477861Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU 2024-08-07T18:04:46.9479094Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_2D_Input 2024-08-07T18:04:46.9480395Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Channels_Last 2024-08-07T18:04:46.9481797Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_Running_Value 2024-08-07T18:04:46.9483161Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_No_Affine 2024-08-07T18:04:46.9484403Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_requires_grad 2024-08-07T18:04:46.9485552Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync 2024-08-07T18:04:46.9486738Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_hook 2024-08-07T18:04:46.9488112Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_grad_is_view 2024-08-07T18:04:46.9489219Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda 2024-08-07T18:04:46.9490452Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_cat_tensor_cuda 2024-08-07T18:04:46.9491567Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_default_pg 2024-08-07T18:04:46.9492653Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_subgroup 2024-08-07T18:04:46.9495491Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_max 2024-08-07T18:04:46.9496682Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_min 2024-08-07T18:04:46.9497861Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_product 2024-08-07T18:04:46.9499058Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_sum 2024-08-07T18:04:46.9500205Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_max 2024-08-07T18:04:46.9501333Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_min 2024-08-07T18:04:46.9502413Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max 2024-08-07T18:04:46.9503579Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max_complex_unsupported 2024-08-07T18:04:46.9504740Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_min 2024-08-07T18:04:46.9505900Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_max 2024-08-07T18:04:46.9506957Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_min 2024-08-07T18:04:46.9507998Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_max 2024-08-07T18:04:46.9508996Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_max 2024-08-07T18:04:46.9509994Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_product 2024-08-07T18:04:46.9510971Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum 2024-08-07T18:04:46.9511977Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_async 2024-08-07T18:04:46.9513024Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_async 2024-08-07T18:04:46.9514060Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group 2024-08-07T18:04:46.9515220Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group_cuda 2024-08-07T18:04:46.9516261Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group 2024-08-07T18:04:46.9517420Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split 2024-08-07T18:04:46.9518633Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda_complex 2024-08-07T18:04:46.9519933Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group 2024-08-07T18:04:46.9521156Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group_cuda 2024-08-07T18:04:46.9522388Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group 2024-08-07T18:04:46.9523596Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group_cuda 2024-08-07T18:04:46.9524725Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_full_group 2024-08-07T18:04:46.9525698Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier 2024-08-07T18:04:46.9526621Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_cuda 2024-08-07T18:04:46.9527625Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group_cuda 2024-08-07T18:04:46.9528674Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_group 2024-08-07T18:04:46.9529734Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo_tags 2024-08-07T18:04:46.9530849Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_mixed_backend_err 2024-08-07T18:04:46.9531985Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_list_err 2024-08-07T18:04:46.9533020Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast 2024-08-07T18:04:46.9534008Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_object_list 2024-08-07T18:04:46.9535020Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager 2024-08-07T18:04:46.9536056Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager_async 2024-08-07T18:04:46.9537271Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_with_logger 2024-08-07T18:04:46.9538536Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping 2024-08-07T18:04:46.9539779Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping_requires_grad 2024-08-07T18:04:46.9541247Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_compile_static_graph 2024-08-07T18:04:46.9542608Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_create_graph 2024-08-07T18:04:46.9543638Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_forward_backward_hook 2024-08-07T18:04:46.9544730Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_pickling_powerSGD 2024-08-07T18:04:46.9545952Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_ignore_params_arg 2024-08-07T18:04:46.9546980Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_cpu 2024-08-07T18:04:46.9548198Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_ignored_params 2024-08-07T18:04:46.9549384Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_execution_trace 2024-08-07T18:04:46.9550481Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_python_error_logged 2024-08-07T18:04:46.9551582Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_shared_grad_acc_unused_params 2024-08-07T18:04:46.9552695Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_module_states 2024-08-07T18:04:46.9553782Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_join_disable 2024-08-07T18:04:46.9554851Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs 2024-08-07T18:04:46.9556129Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs_stop_iteration_sync_bn 2024-08-07T18:04:46.9557338Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_default_group 2024-08-07T18:04:46.9558604Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_no_find_unused 2024-08-07T18:04:46.9559708Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_full_group 2024-08-07T18:04:46.9560684Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_group 2024-08-07T18:04:46.9561701Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_data_parallel_params 2024-08-07T18:04:46.9562704Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_future 2024-08-07T18:04:46.9563621Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank 2024-08-07T18:04:46.9564596Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_invalid_static_graph 2024-08-07T18:04:46.9565559Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_irecv 2024-08-07T18:04:46.9566535Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_autograd_profiler 2024-08-07T18:04:46.9567613Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_failure_order 2024-08-07T18:04:46.9568711Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo 2024-08-07T18:04:46.9569796Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_subgroup 2024-08-07T18:04:46.9570896Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allreduce 2024-08-07T18:04:46.9571928Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_init_abort 2024-08-07T18:04:46.9572899Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups 2024-08-07T18:04:46.9574118Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_world_size_not_divisible_by_group_size 2024-08-07T18:04:46.9575407Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_tuple_module 2024-08-07T18:04:46.9576527Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_product 2024-08-07T18:04:46.9577558Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_min 2024-08-07T18:04:46.9578539Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_product 2024-08-07T18:04:46.9579483Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum 2024-08-07T18:04:46.9580449Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda 2024-08-07T18:04:46.9581417Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda 2024-08-07T18:04:46.9582370Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl 2024-08-07T18:04:46.9583431Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_autograd_profiler 2024-08-07T18:04:46.9584556Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_torch_profiler 2024-08-07T18:04:46.9585644Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_torch_profiler 2024-08-07T18:04:46.9586744Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_autograd_profiler 2024-08-07T18:04:46.9587845Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_api_cpu 2024-08-07T18:04:46.9588964Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_without_logger 2024-08-07T18:04:46.9589622Z 2024-08-07T18:04:46.9589863Z Running distributed tests for the gloo backend with file init_method 2024-08-07T18:04:46.9591151Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=1', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:04:46.941338] 2024-08-07T18:14:17.8855023Z 2024-08-07T18:14:17.8856729Z distributed/test_distributed_spawn 1/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_1.3_f715dc2d6b2e8796_.log 2024-08-07T18:14:17.8946381Z Running 102 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_1_level_hierarchical_model_averager_equivalent_to_periodic_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_3_level_hierarchical_model_averager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_Backend_enum_class, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_2D_Input, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Channels_Last, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_Running_Value, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_No_Affine, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_requires_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_cat_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_default_pg, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max_complex_unsupported, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo_tags, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_mixed_backend_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_list_err, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_object_list, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager_async, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_with_logger, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping_requires_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_compile_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_create_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_forward_backward_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_pickling_powerSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_ignore_params_arg, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_cpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_ignored_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_execution_trace, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_python_error_logged, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_shared_grad_acc_unused_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_module_states, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_join_disable, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs_stop_iteration_sync_bn, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_default_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_no_find_unused, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_data_parallel_params, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_future, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_invalid_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_irecv, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_failure_order, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_subgroup, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_init_abort, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_world_size_not_divisible_by_group_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_tuple_module, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_api_cpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_without_logger 2024-08-07T18:14:17.9004193Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_1_level_hierarchical_model_averager_equivalent_to_periodic_model_averager 2024-08-07T18:14:17.9006687Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_3_level_hierarchical_model_averager 2024-08-07T18:14:17.9008290Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_Backend_enum_class 2024-08-07T18:14:17.9010015Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU 2024-08-07T18:14:17.9012240Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_2D_Input 2024-08-07T18:14:17.9014577Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Channels_Last 2024-08-07T18:14:17.9016773Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_Running_Value 2024-08-07T18:14:17.9019021Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_No_Affine 2024-08-07T18:14:17.9021154Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_requires_grad 2024-08-07T18:14:17.9023231Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync 2024-08-07T18:14:17.9025269Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_allreduce_hook 2024-08-07T18:14:17.9027396Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_accumulate_gradients_no_sync_grad_is_view 2024-08-07T18:14:17.9029193Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_cuda 2024-08-07T18:14:17.9030908Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_cat_tensor_cuda 2024-08-07T18:14:17.9032934Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_default_pg 2024-08-07T18:14:17.9034725Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_object_subgroup 2024-08-07T18:14:17.9036874Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_max 2024-08-07T18:14:17.9038495Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_min 2024-08-07T18:14:17.9039669Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_product 2024-08-07T18:14:17.9041210Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_full_group_sum 2024-08-07T18:14:17.9042371Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_max 2024-08-07T18:14:17.9043484Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_min 2024-08-07T18:14:17.9044565Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max 2024-08-07T18:14:17.9045716Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_max_complex_unsupported 2024-08-07T18:14:17.9046863Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_min 2024-08-07T18:14:17.9047925Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_max 2024-08-07T18:14:17.9048981Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_min 2024-08-07T18:14:17.9050030Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_max 2024-08-07T18:14:17.9051030Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_max 2024-08-07T18:14:17.9052018Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_product 2024-08-07T18:14:17.9052985Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum 2024-08-07T18:14:17.9054000Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_async 2024-08-07T18:14:17.9055041Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_async 2024-08-07T18:14:17.9056084Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group 2024-08-07T18:14:17.9057135Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_full_group_cuda 2024-08-07T18:14:17.9058174Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group 2024-08-07T18:14:17.9059226Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split 2024-08-07T18:14:17.9060376Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_cuda_complex 2024-08-07T18:14:17.9062007Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group 2024-08-07T18:14:17.9063388Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_full_group_cuda 2024-08-07T18:14:17.9064623Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_full_group 2024-08-07T18:14:17.9065928Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_group_cuda 2024-08-07T18:14:17.9067048Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_backend_full_group 2024-08-07T18:14:17.9068012Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier 2024-08-07T18:14:17.9068941Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_cuda 2024-08-07T18:14:17.9069955Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group_cuda 2024-08-07T18:14:17.9070998Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_group 2024-08-07T18:14:17.9072060Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo_tags 2024-08-07T18:14:17.9073172Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_mixed_backend_err 2024-08-07T18:14:17.9074314Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_op_list_err 2024-08-07T18:14:17.9075338Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast 2024-08-07T18:14:17.9076325Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_object_list 2024-08-07T18:14:17.9077335Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager 2024-08-07T18:14:17.9078450Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_coalescing_manager_async 2024-08-07T18:14:17.9079672Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_with_logger 2024-08-07T18:14:17.9080937Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping 2024-08-07T18:14:17.9082183Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_build_debug_param_to_name_mapping_requires_grad 2024-08-07T18:14:17.9083422Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_compile_static_graph 2024-08-07T18:14:17.9084452Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_create_graph 2024-08-07T18:14:17.9085467Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_forward_backward_hook 2024-08-07T18:14:17.9086555Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_pickling_powerSGD 2024-08-07T18:14:17.9087615Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_ignore_params_arg 2024-08-07T18:14:17.9088651Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_cpu 2024-08-07T18:14:17.9089758Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_ignored_params 2024-08-07T18:14:17.9090990Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_profiling_execution_trace 2024-08-07T18:14:17.9092072Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_python_error_logged 2024-08-07T18:14:17.9093229Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_shared_grad_acc_unused_params 2024-08-07T18:14:17.9094309Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_sync_module_states 2024-08-07T18:14:17.9095384Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_input_join_disable 2024-08-07T18:14:17.9096444Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs 2024-08-07T18:14:17.9097534Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_uneven_inputs_stop_iteration_sync_bn 2024-08-07T18:14:17.9098755Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_default_group 2024-08-07T18:14:17.9099944Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_no_find_unused 2024-08-07T18:14:17.9101043Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_full_group 2024-08-07T18:14:17.9102011Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_group 2024-08-07T18:14:17.9103015Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_data_parallel_params 2024-08-07T18:14:17.9104014Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_future 2024-08-07T18:14:17.9104942Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank 2024-08-07T18:14:17.9105899Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_invalid_static_graph 2024-08-07T18:14:17.9106862Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_irecv 2024-08-07T18:14:17.9107841Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_isend_autograd_profiler 2024-08-07T18:14:17.9108919Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_failure_order 2024-08-07T18:14:17.9110006Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo 2024-08-07T18:14:17.9111095Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_gloo_subgroup 2024-08-07T18:14:17.9112213Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allreduce 2024-08-07T18:14:17.9113238Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_init_abort 2024-08-07T18:14:17.9114211Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups 2024-08-07T18:14:17.9115349Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_world_size_not_divisible_by_group_size 2024-08-07T18:14:17.9116583Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_output_unused_in_loss_tuple_module 2024-08-07T18:14:17.9117689Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_product 2024-08-07T18:14:17.9118930Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_min 2024-08-07T18:14:17.9119969Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_product 2024-08-07T18:14:17.9120905Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum 2024-08-07T18:14:17.9121860Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda 2024-08-07T18:14:17.9122825Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda 2024-08-07T18:14:17.9123786Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl 2024-08-07T18:14:17.9124820Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_autograd_profiler 2024-08-07T18:14:17.9125948Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_nccl_torch_profiler 2024-08-07T18:14:17.9127038Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_torch_profiler 2024-08-07T18:14:17.9128136Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_autograd_profiler 2024-08-07T18:14:17.9129235Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_api_cpu 2024-08-07T18:14:17.9130347Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_without_logger 2024-08-07T18:14:17.9131000Z 2024-08-07T18:14:17.9131276Z Running distributed/test_distributed_spawn 3/3 ... [2024-08-07 18:14:17.891585] 2024-08-07T18:14:17.9131813Z MPI not available -- MPI backend tests will be skipped 2024-08-07T18:14:17.9132316Z Running distributed tests for the test backend with env init_method 2024-08-07T18:14:17.9133643Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=3', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:14:17.900258] 2024-08-07T18:14:20.7687392Z 2024-08-07T18:14:20.7688581Z distributed/test_distributed_spawn 3/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_3.3_f9a093cd0ad5201f_.log 2024-08-07T18:14:20.7689519Z Running 0 items in this shard: 2024-08-07T18:14:20.7689717Z 2024-08-07T18:14:20.7696475Z Running distributed tests for the test backend with file init_method 2024-08-07T18:14:20.7702662Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=3', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:14:20.770030] 2024-08-07T18:14:23.6308524Z 2024-08-07T18:14:23.6310624Z distributed/test_distributed_spawn 3/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_3.3_5e816eef6d3cec2d_.log 2024-08-07T18:14:23.6312372Z Running 0 items in this shard: 2024-08-07T18:14:23.6312743Z 2024-08-07T18:14:23.6320085Z Running distributed tests for the gloo backend with env init_method 2024-08-07T18:14:23.6327327Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=3', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:14:23.632393] 2024-08-07T18:23:12.3456051Z 2024-08-07T18:23:12.3458308Z distributed/test_distributed_spawn 3/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_3.3_3da988a817a08221_.log 2024-08-07T18:23:12.3537711Z Running 91 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_gradient, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_non_default_stream, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_with_amp_and_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_stack_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_average_parameters, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_no_rank_zero_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_ring_exchange_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_self_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_without_logger, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_grad_as_bucket_view_false, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer_via_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_different_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_same_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce_process_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_post_localSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_gpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_num_params_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_shape_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_error, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_namedtuple, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_set_grad_to_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_no_set_grad_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_remove_autograd_hooks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_returns_tensor_with_no_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_static_graph_nested_types, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_grad_undefined, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_new_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_different_graph_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_checks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_wait_all_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allgather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_reduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_high_priority_stream, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_negative_input_rank, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_group_size_exceeds_world_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager_param_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda_twice, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_checks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_object_list, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_multi_forward, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_undefined_grad_parity_unused_parameters, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_with_logger 2024-08-07T18:23:12.3588353Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU_grad_is_view 2024-08-07T18:23:12.3590162Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_gradient 2024-08-07T18:23:12.3591528Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_non_default_stream 2024-08-07T18:23:12.3592801Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_with_amp_and_grad_is_view 2024-08-07T18:23:12.3593918Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather 2024-08-07T18:23:12.3594942Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_complex 2024-08-07T18:23:12.3596036Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_group 2024-08-07T18:23:12.3597077Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_full_group 2024-08-07T18:23:12.3598094Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_group 2024-08-07T18:23:12.3599230Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_stack_tensor_cuda 2024-08-07T18:23:12.3600387Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_product 2024-08-07T18:23:12.3601509Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_sum 2024-08-07T18:23:12.3602595Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_sum 2024-08-07T18:23:12.3603679Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_product 2024-08-07T18:23:12.3604990Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_product 2024-08-07T18:23:12.3606033Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_sum 2024-08-07T18:23:12.3607163Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_min 2024-08-07T18:23:12.3608193Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_complex 2024-08-07T18:23:12.3609246Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda_complex 2024-08-07T18:23:12.3610291Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group_cuda 2024-08-07T18:23:12.3611392Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group 2024-08-07T18:23:12.3612591Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group_cuda 2024-08-07T18:23:12.3613773Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda 2024-08-07T18:23:12.3614864Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_average_parameters 2024-08-07T18:23:12.3615869Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group 2024-08-07T18:23:12.3616858Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group_cuda 2024-08-07T18:23:12.3617902Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_full_group 2024-08-07T18:23:12.3618969Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo 2024-08-07T18:23:12.3620073Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_no_rank_zero_nccl 2024-08-07T18:23:12.3621227Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_ring_exchange_nccl 2024-08-07T18:23:12.3622358Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_self_nccl 2024-08-07T18:23:12.3623413Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_full_group 2024-08-07T18:23:12.3624617Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_without_logger 2024-08-07T18:23:12.3625954Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_grad_as_bucket_view_false 2024-08-07T18:23:12.3627167Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer_via_hook 2024-08-07T18:23:12.3628256Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce 2024-08-07T18:23:12.3629395Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_different_across_ranks 2024-08-07T18:23:12.3630559Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_same_across_ranks 2024-08-07T18:23:12.3631609Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device 2024-08-07T18:23:12.3632742Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce_process_group 2024-08-07T18:23:12.3633945Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_post_localSGD 2024-08-07T18:23:12.3635011Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_gpu 2024-08-07T18:23:12.3636115Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_num_params_across_ranks 2024-08-07T18:23:12.3637274Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_shape_across_ranks 2024-08-07T18:23:12.3638495Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_error 2024-08-07T18:23:12.3639602Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_namedtuple 2024-08-07T18:23:12.3641151Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_set_grad_to_none 2024-08-07T18:23:12.3642567Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_no_set_grad_none 2024-08-07T18:23:12.3643783Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd 2024-08-07T18:23:12.3644887Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd_static_graph 2024-08-07T18:23:12.3645999Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_remove_autograd_hooks 2024-08-07T18:23:12.3647108Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_returns_tensor_with_no_grad 2024-08-07T18:23:12.3648219Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_static_graph_nested_types 2024-08-07T18:23:12.3649369Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_grad_undefined 2024-08-07T18:23:12.3650553Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_new_group 2024-08-07T18:23:12.3651674Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_different_graph_across_ranks 2024-08-07T18:23:12.3652765Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather 2024-08-07T18:23:12.3653711Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_checks 2024-08-07T18:23:12.3654676Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object 2024-08-07T18:23:12.3655667Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_group 2024-08-07T18:23:12.3656750Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang 2024-08-07T18:23:12.3657890Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_wait_all_ranks 2024-08-07T18:23:12.3659009Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allgather 2024-08-07T18:23:12.3660072Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_reduce 2024-08-07T18:23:12.3661279Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_high_priority_stream 2024-08-07T18:23:12.3662549Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_negative_input_rank 2024-08-07T18:23:12.3663791Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_group_size_exceeds_world_size 2024-08-07T18:23:12.3665001Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager_param_group 2024-08-07T18:23:12.3666153Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity 2024-08-07T18:23:12.3667331Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_grad_is_view 2024-08-07T18:23:12.3668643Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd_grad_is_view 2024-08-07T18:23:12.3669874Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_min 2024-08-07T18:23:12.3670912Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_sum 2024-08-07T18:23:12.3671928Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_max 2024-08-07T18:23:12.3672877Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_min 2024-08-07T18:23:12.3673889Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_tensor_cuda 2024-08-07T18:23:12.3674946Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda_twice 2024-08-07T18:23:12.3675928Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter 2024-08-07T18:23:12.3676866Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_checks 2024-08-07T18:23:12.3677843Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_complex 2024-08-07T18:23:12.3678925Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda_complex 2024-08-07T18:23:12.3679926Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_full_group 2024-08-07T18:23:12.3680940Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_object_list 2024-08-07T18:23:12.3681921Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv 2024-08-07T18:23:12.3682984Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_autograd_profiler 2024-08-07T18:23:12.3684072Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag 2024-08-07T18:23:12.3685148Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_torch_profiler 2024-08-07T18:23:12.3686265Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum_cuda 2024-08-07T18:23:12.3687334Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_multi_forward 2024-08-07T18:23:12.3688544Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_undefined_grad_parity_unused_parameters 2024-08-07T18:23:12.3689797Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_with_logger 2024-08-07T18:23:12.3690432Z 2024-08-07T18:23:12.3690673Z Running distributed tests for the gloo backend with file init_method 2024-08-07T18:23:12.3691974Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_distributed_spawn.py', '--shard-id=3', '--num-shards=3', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:23:12.351278] 2024-08-07T18:32:04.0413665Z 2024-08-07T18:32:04.0416961Z distributed/test_distributed_spawn 3/3 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_distributed_spawn_3.3_f9d042a87186e77f_.log 2024-08-07T18:32:04.0470934Z Running 91 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_gradient, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_non_default_stream, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_with_amp_and_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_stack_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_product, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_average_parameters, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_no_rank_zero_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_ring_exchange_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_self_nccl, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_without_logger, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_grad_as_bucket_view_false, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer_via_hook, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_different_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_same_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce_process_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_post_localSGD, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_gpu, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_num_params_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_shape_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_error, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_namedtuple, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_set_grad_to_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_no_set_grad_none, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd_static_graph, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_remove_autograd_hooks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_returns_tensor_with_no_grad, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_static_graph_nested_types, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_grad_undefined, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_new_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_different_graph_across_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_checks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_wait_all_ranks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allgather, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_reduce, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_high_priority_stream, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_negative_input_rank, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_group_size_exceeds_world_size, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager_param_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd_grad_is_view, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_sum, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_max, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_min, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_tensor_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda_twice, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_checks, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda_complex, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_full_group, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_object_list, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_autograd_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_torch_profiler, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum_cuda, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_multi_forward, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_undefined_grad_parity_unused_parameters, test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_with_logger 2024-08-07T18:32:04.0513225Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallelCPU_grad_is_view 2024-08-07T18:32:04.0514567Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_SyncBatchNorm_Diff_Input_Sizes_gradient 2024-08-07T18:32:04.0515898Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_non_default_stream 2024-08-07T18:32:04.0517224Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_DistributedDataParallel_with_amp_and_grad_is_view 2024-08-07T18:32:04.0518352Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather 2024-08-07T18:32:04.0519431Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_complex 2024-08-07T18:32:04.0520535Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_coalesced_group 2024-08-07T18:32:04.0521587Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_full_group 2024-08-07T18:32:04.0522604Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_group 2024-08-07T18:32:04.0523759Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_gather_into_stack_tensor_cuda 2024-08-07T18:32:04.0524921Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_product 2024-08-07T18:32:04.0526109Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_group_sum 2024-08-07T18:32:04.0527200Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_coalesced_sum 2024-08-07T18:32:04.0528286Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_full_group_product 2024-08-07T18:32:04.0529357Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_product 2024-08-07T18:32:04.0530404Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_group_sum 2024-08-07T18:32:04.0531401Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_min 2024-08-07T18:32:04.0532435Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_reduce_sum_cuda_complex 2024-08-07T18:32:04.0533488Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_cuda_complex 2024-08-07T18:32:04.0534529Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_group_cuda 2024-08-07T18:32:04.0535625Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group 2024-08-07T18:32:04.0536809Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_equal_split_group_cuda 2024-08-07T18:32:04.0537991Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_all_to_all_single_unequal_split_cuda 2024-08-07T18:32:04.0539085Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_average_parameters 2024-08-07T18:32:04.0540267Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_full_group 2024-08-07T18:32:04.0541272Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_group_cuda 2024-08-07T18:32:04.0542316Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_barrier_timeout_full_group 2024-08-07T18:32:04.0543378Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_gloo 2024-08-07T18:32:04.0544487Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_no_rank_zero_nccl 2024-08-07T18:32:04.0545638Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_ring_exchange_nccl 2024-08-07T18:32:04.0546769Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_batch_isend_irecv_self_nccl 2024-08-07T18:32:04.0547826Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_broadcast_full_group 2024-08-07T18:32:04.0549032Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_compute_bucket_assignment_by_size_sparse_error_without_logger 2024-08-07T18:32:04.0550360Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_apply_optim_in_backward_grad_as_bucket_view_false 2024-08-07T18:32:04.0551705Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_broadcast_buffer_via_hook 2024-08-07T18:32:04.0552904Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_buffer_hook_allreduce 2024-08-07T18:32:04.0554038Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_different_across_ranks 2024-08-07T18:32:04.0555198Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_control_flow_same_across_ranks 2024-08-07T18:32:04.0556247Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_device 2024-08-07T18:32:04.0557324Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_allreduce_process_group 2024-08-07T18:32:04.0558535Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_hook_parity_post_localSGD 2024-08-07T18:32:04.0559609Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_logging_data_gpu 2024-08-07T18:32:04.0560724Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_num_params_across_ranks 2024-08-07T18:32:04.0561895Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_model_diff_shape_across_ranks 2024-08-07T18:32:04.0563064Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_multiple_nested_unused_params_error 2024-08-07T18:32:04.0564161Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_namedtuple 2024-08-07T18:32:04.0565342Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_grad_as_bucket_view_set_grad_to_none 2024-08-07T18:32:04.0566741Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_native_mixed_precision_no_grad_as_bucket_view_no_set_grad_none 2024-08-07T18:32:04.0567955Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd 2024-08-07T18:32:04.0569050Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_new_tensor_in_fwd_static_graph 2024-08-07T18:32:04.0570167Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_remove_autograd_hooks 2024-08-07T18:32:04.0571274Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_returns_tensor_with_no_grad 2024-08-07T18:32:04.0572380Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_static_graph_nested_types 2024-08-07T18:32:04.0573627Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_grad_undefined 2024-08-07T18:32:04.0574808Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_ddp_update_process_group_new_group 2024-08-07T18:32:04.0575924Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_different_graph_across_ranks 2024-08-07T18:32:04.0576931Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather 2024-08-07T18:32:04.0577868Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_checks 2024-08-07T18:32:04.0578837Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_gather_object 2024-08-07T18:32:04.0579901Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_get_rank_size_group 2024-08-07T18:32:04.0581068Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_allreduce_hang 2024-08-07T18:32:04.0582211Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_monitored_barrier_wait_all_ranks 2024-08-07T18:32:04.0583333Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_allgather 2024-08-07T18:32:04.0584401Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_backend_bool_reduce 2024-08-07T18:32:04.0585469Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_nccl_high_priority_stream 2024-08-07T18:32:04.0586643Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_by_enumeration_negative_input_rank 2024-08-07T18:32:04.0587881Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_new_subgroups_group_size_exceeds_world_size 2024-08-07T18:32:04.0589089Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_periodic_model_averager_param_group 2024-08-07T18:32:04.0590241Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity 2024-08-07T18:32:04.0591423Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_grad_is_view 2024-08-07T18:32:04.0592732Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_post_localSGD_optimizer_parity_with_hierarchical_sgd_grad_is_view 2024-08-07T18:32:04.0593959Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_min 2024-08-07T18:32:04.0594995Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_full_group_sum 2024-08-07T18:32:04.0596006Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_group_max 2024-08-07T18:32:04.0596962Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_min 2024-08-07T18:32:04.0597970Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_scatter_tensor_cuda 2024-08-07T18:32:04.0599093Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_reduce_sum_cuda_twice 2024-08-07T18:32:04.0600072Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter 2024-08-07T18:32:04.0601007Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_checks 2024-08-07T18:32:04.0601990Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_complex 2024-08-07T18:32:04.0602990Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_cuda_complex 2024-08-07T18:32:04.0603996Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_full_group 2024-08-07T18:32:04.0605011Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_scatter_object_list 2024-08-07T18:32:04.0605983Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv 2024-08-07T18:32:04.0607115Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_any_source_autograd_profiler 2024-08-07T18:32:04.0608256Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag 2024-08-07T18:32:04.0609338Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_send_recv_with_tag_torch_profiler 2024-08-07T18:32:04.0610457Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_sparse_all_reduce_sum_cuda 2024-08-07T18:32:04.0611526Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_static_graph_multi_forward 2024-08-07T18:32:04.0612675Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_undefined_grad_parity_unused_parameters 2024-08-07T18:32:04.0613871Z Running 1 items in this shard: test/distributed/test_distributed_spawn.py::TestDistBackendWithSpawn::test_verify_model_across_rank_with_logger 2024-08-07T18:32:04.0614516Z 2024-08-07T18:32:04.0614827Z Running distributed/fsdp/test_fsdp_backward_prefetch 1/1 ... [2024-08-07 18:32:04.045753] 2024-08-07T18:32:04.0616216Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_backward_prefetch.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:32:04.046154] 2024-08-07T18:32:09.6743260Z 2024-08-07T18:32:09.6744548Z distributed/fsdp/test_fsdp_backward_prefetch 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_backward_prefetch_1.1_351dec4e2a650c5f_.log 2024-08-07T18:32:09.6745946Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_backward_prefetch.py::TestBackwardPrefetch::test_backward_prefetch 2024-08-07T18:32:09.6746531Z 2024-08-07T18:32:09.6749681Z Running distributed/_composable/fsdp/test_fully_shard_autograd 1/1 ... [2024-08-07 18:32:09.674780] 2024-08-07T18:32:09.6754694Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_autograd.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:32:09.675223] 2024-08-07T18:32:12.4490041Z 2024-08-07T18:32:12.4491835Z distributed/_composable/fsdp/test_fully_shard_autograd 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_autograd_1.1_d4932b7054b1f879_.log 2024-08-07T18:32:12.4498292Z Running 5 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_autograd.py::TestFullyShardAutograd::test_nontensor_activations, test/distributed/_composable/fsdp/test_fully_shard_autograd.py::TestFullyShardAutograd::test_unused_forward_module, test/distributed/_composable/fsdp/test_fully_shard_autograd.py::TestFullyShardAutograd::test_unused_forward_output, test/distributed/_composable/fsdp/test_fully_shard_autograd.py::TestFullyShardPostAccGradHookMultiThread::test_post_acc_grad_hook_runs, test/distributed/_composable/fsdp/test_fully_shard_autograd.py::TestFullyShardPostAccGradHookMultiProcess::test_post_acc_grad_hook_optim_parity 2024-08-07T18:32:12.4501724Z 2024-08-07T18:32:12.4502097Z Running distributed/_composable/fsdp/test_fully_shard_training 1/1 ... [2024-08-07 18:32:12.449513] 2024-08-07T18:32:12.4503554Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_training.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:32:12.450001] 2024-08-07T18:32:15.4241710Z 2024-08-07T18:32:15.4243165Z distributed/_composable/fsdp/test_fully_shard_training 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_training_1.1_e45121bd932f0b1b_.log 2024-08-07T18:32:15.4255336Z Running 19 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_training.py::test_compiled_fsdp, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardForwardInputs::test_root_move_forward_input_to_device, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardRegisteredParams::test_param_registration_after_backward, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardRegisteredParams::test_param_registration_after_forward, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardCastAfterInit::test_to_float64_after_init, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_explicit_prefetching, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_multi_forward_module, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_non_root_forward_backward, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_post_optim_event, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_multi_group, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_multi_group_cpu_offload_eager, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_single_group, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCompose::test_train_parity_with_activation_checkpointing, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardSharedParams::test_train_parity_with_shared_params, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardGradientAccumulation::test_1f1b_microbatching, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardGradientAccumulation::test_gradient_accumulation, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardNDTraining::test_2d_mlp_with_nd_mesh, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardHSDPTraining::test_train_parity_hsdp, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardCustomForwardMethod::test_register_fsdp_forward_method 2024-08-07T18:32:15.4265858Z 2024-08-07T18:32:15.4266173Z Running distributed/fsdp/test_fsdp_freezing_weights 1/1 ... [2024-08-07 18:32:15.424640] 2024-08-07T18:32:15.4267519Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_freezing_weights.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:32:15.425085] 2024-08-07T18:33:50.7404965Z 2024-08-07T18:33:50.7408562Z distributed/fsdp/test_fsdp_freezing_weights 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_freezing_weights_1.1_f1c3af21d7e916ae_.log 2024-08-07T18:33:50.7462326Z Running 32 items in this shard: test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_False_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_GradToNone_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_False_disable_autograd_True_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_False_forward_prefetch_True, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_False, test/distributed/fsdp/test_fsdp_freezing_weights.py::TestFreezingWeights::test_freezing_weights_with_nested_trunk_True_freezing_method_FreezingMethod_RequiresGrad_freeze_after_wrap_fsdp_True_disable_autograd_True_forward_prefetch_True 2024-08-07T18:33:50.7496330Z 2024-08-07T18:33:50.7496691Z Running distributed/optim/test_zero_redundancy_optimizer 1/1 ... [2024-08-07 18:33:50.741473] 2024-08-07T18:33:50.7498114Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/optim/test_zero_redundancy_optimizer.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:33:50.742045] 2024-08-07T18:35:03.5763897Z 2024-08-07T18:35:03.5765487Z distributed/optim/test_zero_redundancy_optimizer 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.optim.test_zero_redundancy_optimizer_1.1_e046fda781b36268_.log 2024-08-07T18:35:03.5797570Z Running 42 items in this shard: test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_constructor, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_lr_scheduler, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_same_dense_param_type, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_state_dict, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_step_with_extra_inner_key, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_step_with_kwargs, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_step_without_closure, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerSingleRank::test_zero_grad, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_add_param_group, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_collect_shards, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_False_static_graph_False_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_False_static_graph_False_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_False_static_graph_True_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_False_static_graph_True_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_True_static_graph_False_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_True_static_graph_False_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_True_static_graph_True_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_False_gradient_as_bucket_view_True_static_graph_True_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_False_static_graph_False_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_False_static_graph_False_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_False_static_graph_True_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_False_static_graph_True_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_True_static_graph_False_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_True_static_graph_False_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_True_static_graph_True_shard_buckets_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_ddp_zero_overlap_use_gpu_True_use_interleaved_hook_True_gradient_as_bucket_view_True_static_graph_True_shard_buckets_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_AdamW_maximize_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_AdamW_maximize_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_Adam_maximize_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_Adam_maximize_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_SGD_maximize_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_local_optimizer_parity_optimizer_class_str_SGD_maximize_True, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_lr_scheduler, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_multiple_param_groups, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_nondefault_process_group, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_sharding, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_step, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_step_with_closure, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_zero_join_cpu, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_zero_join_gpu, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_zero_model_parallel_parameters_as_bucket_view_False, test/distributed/optim/test_zero_redundancy_optimizer.py::TestZeroRedundancyOptimizerDistributed::test_zero_model_parallel_parameters_as_bucket_view_True 2024-08-07T18:35:03.5828414Z 2024-08-07T18:35:03.5828784Z Running distributed/algorithms/quantization/test_quantization 1/1 ... [2024-08-07 18:35:03.577053] 2024-08-07T18:35:03.5834450Z MPI not available -- MPI backend tests will be skipped 2024-08-07T18:35:03.5836333Z Running distributed tests for the test backend with env init_method 2024-08-07T18:35:03.5844634Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/algorithms/quantization/test_quantization.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:35:03.584218] 2024-08-07T18:35:06.0695897Z 2024-08-07T18:35:06.0697218Z distributed/algorithms/quantization/test_quantization 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_f0cf8fcd71b4beea_.log 2024-08-07T18:35:06.0698477Z Running 0 items in this shard: 2024-08-07T18:35:06.0698691Z 2024-08-07T18:35:06.0704722Z Running distributed tests for the test backend with file init_method 2024-08-07T18:35:06.0710944Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/algorithms/quantization/test_quantization.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:35:06.070840] 2024-08-07T18:35:08.6059066Z 2024-08-07T18:35:08.6060642Z distributed/algorithms/quantization/test_quantization 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_c3c4165a9ebc2baf_.log 2024-08-07T18:35:08.6061918Z Running 0 items in this shard: 2024-08-07T18:35:08.6062124Z 2024-08-07T18:35:08.6068146Z Running distributed tests for the gloo backend with env init_method 2024-08-07T18:35:08.6074155Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/algorithms/quantization/test_quantization.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:35:08.607174] 2024-08-07T18:35:32.0175216Z 2024-08-07T18:35:32.0176531Z distributed/algorithms/quantization/test_quantization 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_955157289cde159c_.log 2024-08-07T18:35:32.0190186Z Running 6 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_bfp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_fp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_bfp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_fp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_bfp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_fp16 2024-08-07T18:35:32.0194113Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_bfp16 2024-08-07T18:35:32.0195304Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_fp16 2024-08-07T18:35:32.0196495Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_bfp16 2024-08-07T18:35:32.0197679Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_fp16 2024-08-07T18:35:32.0198937Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_bfp16 2024-08-07T18:35:32.0200178Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_fp16 2024-08-07T18:35:32.0200847Z 2024-08-07T18:35:32.0201098Z Running distributed tests for the gloo backend with file init_method 2024-08-07T18:35:32.0202515Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/algorithms/quantization/test_quantization.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:35:32.018958] 2024-08-07T18:35:55.2919453Z 2024-08-07T18:35:55.2921338Z distributed/algorithms/quantization/test_quantization 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_c79bcd2b698e8e88_.log 2024-08-07T18:35:55.2929426Z Running 6 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_bfp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_fp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_bfp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_fp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_bfp16, test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_fp16 2024-08-07T18:35:55.2936727Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_bfp16 2024-08-07T18:35:55.2938948Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_gather_fp16 2024-08-07T18:35:55.2941247Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_bfp16 2024-08-07T18:35:55.2943160Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_fp16 2024-08-07T18:35:55.2945119Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_bfp16 2024-08-07T18:35:55.2947375Z Running 1 items in this shard: test/distributed/algorithms/quantization/test_quantization.py::DistQuantizationTests::test_all_to_all_single_fp16 2024-08-07T18:35:55.2948481Z 2024-08-07T18:35:55.2948854Z Running distributed/test_data_parallel 1/1 ... [2024-08-07 18:35:55.293922] 2024-08-07T18:35:55.2950551Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_data_parallel.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:35:55.294451] 2024-08-07T18:35:58.3186754Z 2024-08-07T18:35:58.3188325Z distributed/test_data_parallel 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_data_parallel_1.1_f6fb3bf1d57e6261_.log 2024-08-07T18:35:58.3221962Z Running 46 items in this shard: test/distributed/test_data_parallel.py::TestDataParallel::test_autocast, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_buffers_requiring_grad, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_complex, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_device_args, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_function_deletion, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_lazy_linear, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_model_device, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_model_no_refcycles, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_module_zero_inputs, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_multiple_input, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_nested_input, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_nested_output, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_no_grad, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_rnn, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_small_back, test/distributed/test_data_parallel.py::TestDataParallel::test_data_parallel_sparse, test/distributed/test_data_parallel.py::TestDataParallel::test_gather_cpu, test/distributed/test_data_parallel.py::TestDataParallel::test_gather_different_len_dicts, test/distributed/test_data_parallel.py::TestDataParallel::test_gather_gpu, test/distributed/test_data_parallel.py::TestDataParallel::test_parallel_apply, test/distributed/test_data_parallel.py::TestDataParallel::test_parallel_apply_autocast, test/distributed/test_data_parallel.py::TestDataParallel::test_parallel_apply_passes_exception, test/distributed/test_data_parallel.py::TestDataParallel::test_parameter_list_dict_replica, test/distributed/test_data_parallel.py::TestDataParallel::test_replicate, test/distributed/test_data_parallel.py::TestDataParallel::test_replicate_buffers, test/distributed/test_data_parallel.py::TestDataParallel::test_save_replica_module, test/distributed/test_data_parallel.py::TestDataParallel::test_scatter_cpu, test/distributed/test_data_parallel.py::TestDataParallel::test_scatter_gpu, test/distributed/test_data_parallel.py::TestDataParallel::test_strided_grad_layout, test/distributed/test_data_parallel.py::TestDataParallel::test_zero_grad, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_cpu_float16, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_cpu_float32, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_cpu_float64, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_cpu_float16, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_cpu_float32, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_cpu_float64, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_dict_cpu_float16, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_dict_cpu_float32, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_dict_cpu_float64, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_list_cpu_float16, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_list_cpu_float32, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_list_cpu_float64, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_tuple_cpu_float16, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_tuple_cpu_float32, test/distributed/test_data_parallel.py::TestDataParallelDeviceTypeCPU::test_data_parallel_module_kwargs_only_empty_tuple_cpu_float64 2024-08-07T18:35:58.3251258Z 2024-08-07T18:35:58.3251571Z Running distributed/algorithms/test_join 1/1 ... [2024-08-07 18:35:58.319552] 2024-08-07T18:35:58.3252992Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/algorithms/test_join.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:35:58.320153] 2024-08-07T18:36:26.1887490Z 2024-08-07T18:36:26.1889068Z distributed/algorithms/test_join 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.algorithms.test_join_1.1_4542232bb5dc16ee_.log 2024-08-07T18:36:26.1896325Z Running 9 items in this shard: test/distributed/algorithms/test_join.py::TestJoin::test_join_kwargs, test/distributed/algorithms/test_join.py::TestJoin::test_multiple_joinable_disable, test/distributed/algorithms/test_join.py::TestJoin::test_multiple_joinables, test/distributed/algorithms/test_join.py::TestJoin::test_multiple_joinables_throw, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_disable, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_main_hooks, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_post_hooks, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_throw 2024-08-07T18:36:26.1901853Z 2024-08-07T18:36:26.1902402Z Running distributed/fsdp/test_checkpoint_wrapper 1/1 ... [2024-08-07 18:36:26.189473] 2024-08-07T18:36:26.1904995Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_checkpoint_wrapper.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:36:26.190051] 2024-08-07T18:36:29.2150132Z 2024-08-07T18:36:29.2151326Z distributed/fsdp/test_checkpoint_wrapper 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_checkpoint_wrapper_1.1_ef2a42f6809847b1_.log 2024-08-07T18:36:29.2156346Z Running 8 items in this shard: test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_apply_activation_checkpointing, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_checkpoint_wrapper_args_kwargs, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_checkpoint_wrapper_cpu_offload, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_checkpoint_wrapper_kwarg_support, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_checkpoint_wrapper_parity, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_forward_missing_attributes, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_fqn, test/distributed/fsdp/test_checkpoint_wrapper.py::CheckpointWrapperTest::test_load_activation_checkpointed_module 2024-08-07T18:36:29.2160418Z 2024-08-07T18:36:29.2160751Z Running distributed/_composable/fsdp/test_fully_shard_comm 1/1 ... [2024-08-07 18:36:29.215489] 2024-08-07T18:36:29.2162215Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_comm.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:36:29.215941] 2024-08-07T18:36:32.2409369Z 2024-08-07T18:36:32.2410585Z distributed/_composable/fsdp/test_fully_shard_comm 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_comm_1.1_b5e7376268557eb1_.log 2024-08-07T18:36:32.2419290Z Running 14 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCollectiveOps::test_all_gather_fp32, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCollectiveOps::test_reduce_scatter_fp16, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCollectiveOps::test_reduce_scatter_fp32, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCommunication::test_fully_shard_communication_count, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCommunication::test_manual_reshard_with_reshard_after_forward_false, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardCommunication::test_set_reduce_scatter_divide_factor, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardPrefetch::test_fully_shard_backward_prefetch, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardPrefetch::test_fully_shard_multi_module_backward_prefetch, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardPrefetch::test_fully_shard_multi_module_unused_module, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardPrefetch::test_set_modules_to_backward_prefetch, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardPrefetch::test_set_modules_to_forward_prefetch, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardUnshardMultiProcess::test_unshard_async, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardUnshardMultiThread::test_unshard_no_param_group, test/distributed/_composable/fsdp/test_fully_shard_comm.py::TestFullyShardUnshardMultiThread::test_unshard_without_lazy_init 2024-08-07T18:36:32.2427158Z 2024-08-07T18:36:32.2427420Z Running distributed/fsdp/test_fsdp_input 1/1 ... [2024-08-07 18:36:32.241435] 2024-08-07T18:36:32.2429664Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_input.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:36:32.241874] 2024-08-07T18:36:40.5280103Z 2024-08-07T18:36:40.5281227Z distributed/fsdp/test_fsdp_input 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_input_1.1_45b5f72a1e855bb5_.log 2024-08-07T18:36:40.5282703Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_input.py::TestInput::test_input_type_dict, test/distributed/fsdp/test_fsdp_input.py::TestInput::test_input_type_list 2024-08-07T18:36:40.5283486Z 2024-08-07T18:36:40.5286703Z Running distributed/fsdp/test_fsdp_uneven 1/1 ... [2024-08-07 18:36:40.528491] 2024-08-07T18:36:40.5292095Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_uneven.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:36:40.528976] 2024-08-07T18:36:46.5114625Z 2024-08-07T18:36:46.5115849Z distributed/fsdp/test_fsdp_uneven 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_uneven_1.1_6069062bff1932ca_.log 2024-08-07T18:36:46.5117091Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_uneven.py::TestUnevenParamShard::test_one_iteration 2024-08-07T18:36:46.5117632Z 2024-08-07T18:36:46.5120930Z Running distributed/fsdp/test_fsdp_multiple_wrapping 1/1 ... [2024-08-07 18:36:46.511906] 2024-08-07T18:36:46.5126001Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_multiple_wrapping.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:36:46.512368] 2024-08-07T18:36:52.5937979Z 2024-08-07T18:36:52.5939273Z distributed/fsdp/test_fsdp_multiple_wrapping 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_multiple_wrapping_1.1_527c483bd4c903ca_.log 2024-08-07T18:36:52.5940978Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_multiple_wrapping.py::TestMultipleWrapping::test_multiple_wrapping 2024-08-07T18:36:52.5941640Z 2024-08-07T18:36:52.5944739Z Running distributed/fsdp/test_utils 1/1 ... [2024-08-07 18:36:52.594250] 2024-08-07T18:36:52.5949534Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:36:52.594723] 2024-08-07T18:36:55.5693606Z 2024-08-07T18:36:55.5694612Z distributed/fsdp/test_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_utils_1.1_a669c6dbcfa7fcad_.log 2024-08-07T18:36:55.5697018Z Running 5 items in this shard: test/distributed/fsdp/test_utils.py::TestUtils::test_apply_to_tensors_cpu_cuda, test/distributed/fsdp/test_utils.py::TestUtils::test_apply_to_tensors_devices0, test/distributed/fsdp/test_utils.py::TestUtils::test_apply_to_tensors_devices1, test/distributed/fsdp/test_utils.py::TestUtils::test_packed_sequence, test/distributed/fsdp/test_utils.py::TestUtils::test_replace_by_prefix 2024-08-07T18:36:55.5698775Z 2024-08-07T18:36:55.5699769Z Running distributed/fsdp/test_fsdp_misc 1/1 ... [2024-08-07 18:36:55.569813] 2024-08-07T18:36:55.5705240Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_misc.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:36:55.570304] 2024-08-07T18:37:37.6114000Z 2024-08-07T18:37:37.6115605Z distributed/fsdp/test_fsdp_misc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_misc_1.1_ca9e7465b2abbdf1_.log 2024-08-07T18:37:37.6130873Z Running 28 items in this shard: test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_cpu_init_with_sync_module_states, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_cpu_init_stays_on_cpu, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_cpu_training, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_device_id_use_index_False, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_device_id_use_index_True, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_not_all_outputs_used_in_loss, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_optim_overlap_no_use_orig_params_error, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_optimizer_overlap, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_zero2_eval_with_prefetch, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_cpu_gpu_module, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_device_id_auto_wrap, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_device_id_cpu_offload, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_device_id_no_move_ignored_params_and_bufs, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_ignored_module_meta, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_namedtuple, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_same_model_across_ranks, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_unsupported_module_cls, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_homogeneous_attributes, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_module_device_mismatches_device_id, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_multigpu_module, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_no_params, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscWorldSize1::test_training_device_mismatch_errors, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscWorldSize1::test_unsafe_setattr, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscWorldSize1::test_world_size_1_sharding_strategy_warning 2024-08-07T18:37:37.6144337Z 2024-08-07T18:37:37.6144652Z Running distributed/fsdp/test_fsdp_checkpoint 1/1 ... [2024-08-07 18:37:37.611991] 2024-08-07T18:37:37.6145983Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:37:37.612450] 2024-08-07T18:38:30.8944724Z 2024-08-07T18:38:30.8947556Z distributed/fsdp/test_fsdp_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_checkpoint_1.1_cd17babaaa9c57fe_.log 2024-08-07T18:38:30.8961109Z Running 17 items in this shard: test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpointSubmodule::test_checkpoint_submodule_use_reentrant_False 2024-08-07T18:38:30.8973661Z 2024-08-07T18:38:30.8973971Z Running distributed/fsdp/test_fsdp_exec_order 1/1 ... [2024-08-07 18:38:30.895006] 2024-08-07T18:38:30.8975382Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_exec_order.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:38:30.895483] 2024-08-07T18:38:57.7328664Z 2024-08-07T18:38:57.7330064Z distributed/fsdp/test_fsdp_exec_order 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_exec_order_1.1_697efbc3812aa66b_.log 2024-08-07T18:38:57.7336399Z Running 8 items in this shard: test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_invalid_first_iter_order_sharding_strategy0, test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_invalid_first_iter_order_sharding_strategy1, test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_invalid_later_iter_order_sharding_strategy0_iters_before_path_change_1, test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_invalid_later_iter_order_sharding_strategy0_iters_before_path_change_3, test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_invalid_later_iter_order_sharding_strategy1_iters_before_path_change_1, test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_invalid_later_iter_order_sharding_strategy1_iters_before_path_change_3, test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_train_eval_sharding_strategy0, test/distributed/fsdp/test_fsdp_exec_order.py::TestFSDPExecOrder::test_train_eval_sharding_strategy1 2024-08-07T18:38:57.7341588Z 2024-08-07T18:38:57.7341872Z Running distributed/fsdp/test_fsdp_fine_tune 1/1 ... [2024-08-07 18:38:57.733332] 2024-08-07T18:38:57.7343171Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_fine_tune.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:38:57.733815] 2024-08-07T18:39:00.7592484Z 2024-08-07T18:39:00.7593663Z distributed/fsdp/test_fsdp_fine_tune 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_fine_tune_1.1_2a4d3f932e1f1fa5_.log 2024-08-07T18:39:00.7596269Z Running 4 items in this shard: test/distributed/fsdp/test_fsdp_fine_tune.py::TestFSDPFineTune::test_backward_reshard_hooks, test/distributed/fsdp/test_fsdp_fine_tune.py::TestFSDPFineTune::test_hooks_multi_traversal, test/distributed/fsdp/test_fsdp_fine_tune.py::TestFSDPFineTune::test_parity_with_ddp, test/distributed/fsdp/test_fsdp_fine_tune.py::TestFSDPFineTune::test_parity_with_non_frozen_fsdp 2024-08-07T18:39:00.7598332Z 2024-08-07T18:39:00.7598884Z Running distributed/test_c10d_common 1/1 ... [2024-08-07 18:39:00.759705] 2024-08-07T18:39:00.7603889Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_common.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:39:00.760143] 2024-08-07T18:41:26.9533820Z 2024-08-07T18:41:26.9535228Z distributed/test_c10d_common 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_common_1.1_cb7635dfc3ee7d07_.log 2024-08-07T18:41:26.9547195Z Running 23 items in this shard: test/distributed/test_c10d_common.py::TimeoutTest::test_store_based_barrier, test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_multi_limit_multi_dtype, test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_multi_limit_single_dtype, test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_single_limit_multi_dtype, test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_single_limit_single_dtype, test/distributed/test_c10d_common.py::CommTest::test_debug_level, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_backend_class_attr, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_backend_config, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_collectives, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_get_backend_name, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_init_process_group_with_multiple_backends, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_is_backend_available, test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_send_recv, test/distributed/test_c10d_common.py::ProcessGroupWithDispatchedCollectivesTests::test_init_process_group_for_all_backends, test/distributed/test_c10d_common.py::ProcessGroupWithDispatchedCollectivesTests::test_init_process_group_optional_backend, test/distributed/test_c10d_common.py::ReduceOpTest::test_op_isinstance_of_reduceop, test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_copyable, test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_equal, test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_pickle, test/distributed/test_c10d_common.py::LocalRankTest::testNodeLocalRank, test/distributed/test_c10d_common.py::LocalRankTest::testNodeLocalRankOverridesFallback, test/distributed/test_c10d_common.py::LocalRankTest::testWithoutEnv, test/distributed/test_c10d_common.py::LocalRankTest::testWithoutEnvWithFallback 2024-08-07T18:41:26.9557611Z Running 1 items in this shard: test/distributed/test_c10d_common.py::TimeoutTest::test_store_based_barrier 2024-08-07T18:41:26.9558764Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_multi_limit_multi_dtype 2024-08-07T18:41:26.9559798Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_multi_limit_single_dtype 2024-08-07T18:41:26.9560825Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_single_limit_multi_dtype 2024-08-07T18:41:26.9561843Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ComputeBucketAssignmentTest::test_single_limit_single_dtype 2024-08-07T18:41:26.9562733Z Running 1 items in this shard: test/distributed/test_c10d_common.py::CommTest::test_debug_level 2024-08-07T18:41:26.9563623Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_backend_class_attr 2024-08-07T18:41:26.9564626Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_backend_config 2024-08-07T18:41:26.9565587Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_collectives 2024-08-07T18:41:26.9566570Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_get_backend_name 2024-08-07T18:41:26.9567700Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_init_process_group_with_multiple_backends 2024-08-07T18:41:26.9568826Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_is_backend_available 2024-08-07T18:41:26.9569813Z Running 1 items in this shard: test/distributed/test_c10d_common.py::PythonProcessGroupExtensionTest::test_send_recv 2024-08-07T18:41:26.9570934Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ProcessGroupWithDispatchedCollectivesTests::test_init_process_group_for_all_backends 2024-08-07T18:41:26.9572270Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ProcessGroupWithDispatchedCollectivesTests::test_init_process_group_optional_backend 2024-08-07T18:41:26.9573440Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ReduceOpTest::test_op_isinstance_of_reduceop 2024-08-07T18:41:26.9574308Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_copyable 2024-08-07T18:41:26.9575131Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_equal 2024-08-07T18:41:26.9575945Z Running 1 items in this shard: test/distributed/test_c10d_common.py::ReduceOpTest::test_reduceop_pickle 2024-08-07T18:41:26.9576867Z Running 1 items in this shard: test/distributed/test_c10d_common.py::LocalRankTest::testNodeLocalRank 2024-08-07T18:41:26.9577753Z Running 1 items in this shard: test/distributed/test_c10d_common.py::LocalRankTest::testNodeLocalRankOverridesFallback 2024-08-07T18:41:26.9578628Z Running 1 items in this shard: test/distributed/test_c10d_common.py::LocalRankTest::testWithoutEnv 2024-08-07T18:41:26.9579444Z Running 1 items in this shard: test/distributed/test_c10d_common.py::LocalRankTest::testWithoutEnvWithFallback 2024-08-07T18:41:26.9579956Z 2024-08-07T18:41:26.9580277Z Running distributed/_shard/sharding_plan/test_sharding_plan 1/1 ... [2024-08-07 18:41:26.954442] 2024-08-07T18:41:26.9581698Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharding_plan/test_sharding_plan.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:41:26.954904] 2024-08-07T18:41:29.6294225Z 2024-08-07T18:41:29.6295809Z distributed/_shard/sharding_plan/test_sharding_plan 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharding_plan.test_sharding_plan_1.1_406e68116fb5644f_.log 2024-08-07T18:41:29.6298373Z Running 3 items in this shard: test/distributed/_shard/sharding_plan/test_sharding_plan.py::TestShardingPlan::test_custom_sharding_planner, test/distributed/_shard/sharding_plan/test_sharding_plan.py::TestShardingPlan::test_shard_module_sub_process_group, test/distributed/_shard/sharding_plan/test_sharding_plan.py::TestShardingPlan::test_sharding_plan_errors 2024-08-07T18:41:29.6300059Z 2024-08-07T18:41:29.6300626Z Running distributed/checkpoint/test_state_dict 1/1 ... [2024-08-07 18:41:29.629901] 2024-08-07T18:41:29.6305710Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/checkpoint/test_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:41:29.630353] 2024-08-07T18:41:36.0127537Z 2024-08-07T18:41:36.0128871Z distributed/checkpoint/test_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_state_dict_1.1_36f056619d6de226_.log 2024-08-07T18:41:36.0139334Z Running 25 items in this shard: test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_activation_ckpt_fqns_ddp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_activation_ckpt_fqns_fsdp1, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_broadcast_from_rank0, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_broadcast_from_rank0_hsdp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_compiled_fsdp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_cpu_offload_full_state_dict, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_ddp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_deprecate_fsdp_api, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_deprecate_partial, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_extra_state, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_flattened_osd, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_frozen_parameters, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_fsdp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_fsdp2, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_fsdp_ddp, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_fsdp_root_not_initialized, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_non_persistent_buffers, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_optim_state_dict_param_matching, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_optim_state_dict_tensor_matching, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_setting_meta_device_model, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_setting_meta_device_model_broadcasting, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_shared_weight, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_single_gpu, test/distributed/checkpoint/test_state_dict.py::TestStateDict::test_strict, test/distributed/checkpoint/test_state_dict.py::TestNoComm::test_no_dist 2024-08-07T18:41:36.0149130Z 2024-08-07T18:41:36.0149389Z Running distributed/_shard/test_sharder 1/1 ... [2024-08-07 18:41:36.013269] 2024-08-07T18:41:36.0150630Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/test_sharder.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:41:36.013732] 2024-08-07T18:41:38.8383782Z 2024-08-07T18:41:38.8385350Z distributed/_shard/test_sharder 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.test_sharder_1.1_e61db70520c65da5_.log 2024-08-07T18:41:38.8386935Z Running 2 items in this shard: test/distributed/_shard/test_sharder.py::TestCustomSharder::test_custom_sharder, test/distributed/_shard/test_sharder.py::TestCustomSharder::test_custom_sharder_errors 2024-08-07T18:41:38.8387933Z 2024-08-07T18:41:38.8390359Z Running distributed/fsdp/test_fsdp_meta 1/1 ... [2024-08-07 18:41:38.838836] 2024-08-07T18:41:38.8395326Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_meta.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:41:38.839280] 2024-08-07T18:42:03.7521387Z 2024-08-07T18:42:03.7524959Z distributed/fsdp/test_fsdp_meta 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_meta_1.1_f8ea1dae9bdc541e_.log 2024-08-07T18:42:03.7539097Z Running 15 items in this shard: test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_bad_arg_meta, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_bad_arg_torchdistx, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_meta_device_with_mixed_precision, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_meta_device_default_init_auto_wrap_False, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_meta_device_default_init_auto_wrap_True, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_meta_device_reset_params_auto_wrap_False, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_meta_device_reset_params_auto_wrap_True, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_torchdistX_default_init_auto_wrap_False, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_torchdistX_default_init_auto_wrap_True, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_torchdistX_init_fn_auto_wrap_False, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_nested_model_with_torchdistX_init_fn_auto_wrap_True, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_simple_model_with_meta_device_default_init, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_simple_model_with_meta_device_reset_params, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_simple_model_with_torchdistX_default_init, test/distributed/fsdp/test_fsdp_meta.py::TestFSDPWithMetaDevice::test_simple_model_with_torchdistX_init_fn 2024-08-07T18:42:03.7551325Z 2024-08-07T18:42:03.7551754Z Running distributed/rpc/test_faulty_agent 1/1 ... [2024-08-07 18:42:03.752828] 2024-08-07T18:42:03.7553687Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_faulty_agent.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:42:03.753432] 2024-08-07T18:47:52.1112411Z 2024-08-07T18:47:52.1114106Z distributed/rpc/test_faulty_agent 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_faulty_agent_1.1_b0889ba0258f7816_.log 2024-08-07T18:47:52.1140609Z Running 28 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_check_failed_messages, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_faulty_messages, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_messages_to_delay, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_no_faulty_messages, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_builtin_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_script_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rref_to_here_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_verify_backend_options, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_context_cleanup_tensor_with_grad, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_verify_backend_options, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_remote_timeout_to_here_in_jit, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_in_jit, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_script_func, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_to_here_timeout_in_jit, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_python, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_torchscript_function 2024-08-07T18:47:52.1162487Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout 2024-08-07T18:47:52.1164328Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout_to_self 2024-08-07T18:47:52.1166090Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_check_failed_messages 2024-08-07T18:47:52.1167840Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_faulty_messages 2024-08-07T18:47:52.1169465Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_messages_to_delay 2024-08-07T18:47:52.1170952Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_no_faulty_messages 2024-08-07T18:47:52.1172669Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout 2024-08-07T18:47:52.1174538Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout_to_self 2024-08-07T18:47:52.1176265Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle 2024-08-07T18:47:52.1177940Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle_to_self 2024-08-07T18:47:52.1179873Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout 2024-08-07T18:47:52.1181755Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout_to_self 2024-08-07T18:47:52.1183571Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_builtin_timeout 2024-08-07T18:47:52.1185210Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_script_timeout 2024-08-07T18:47:52.1186388Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rref_to_here_timeout 2024-08-07T18:47:52.1187459Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout 2024-08-07T18:47:52.1188617Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout_to_self 2024-08-07T18:47:52.1189799Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout 2024-08-07T18:47:52.1190982Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout_to_self 2024-08-07T18:47:52.1192101Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_verify_backend_options 2024-08-07T18:47:52.1193219Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_context_cleanup_tensor_with_grad 2024-08-07T18:47:52.1194392Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_verify_backend_options 2024-08-07T18:47:52.1195522Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_remote_timeout_to_here_in_jit 2024-08-07T18:47:52.1196641Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_in_jit 2024-08-07T18:47:52.1197754Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_script_func 2024-08-07T18:47:52.1198967Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_to_here_timeout_in_jit 2024-08-07T18:47:52.1200031Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_python 2024-08-07T18:47:52.1201115Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_torchscript_function 2024-08-07T18:47:52.1201746Z 2024-08-07T18:47:52.1202017Z Running distributed/rpc/test_tensorpipe_agent 2/4 ... [2024-08-07 18:47:52.113369] 2024-08-07T18:47:52.1203394Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_tensorpipe_agent.py', '--shard-id=2', '--num-shards=4', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 18:47:52.113949] 2024-08-07T19:02:41.6584129Z 2024-08-07T19:02:41.6585857Z distributed/rpc/test_tensorpipe_agent 2/4 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_tensorpipe_agent_2.4_3c73dc1b9a6663fe_.log 2024-08-07T19:02:41.6669804Z Running 122 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_add, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_chained_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_chained_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_static_method, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_static_method_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_builtin_remote_self, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_cannot_infer_backend_from_options, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_default_timeout_used, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_duplicate_name, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_expected_src, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_get_worker_infos, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_pg_then_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_invalid_names, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_shutdown, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_builtin_remote_ret, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rref_stress, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_owner_rref_backward, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_pg_init_no_rpc_init, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_pickle_future, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_export_trace, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_remote_events_profiled, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_rpc_memory, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_rpc_record_shapes, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_autograd_context_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_udf_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_built_in, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_method, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_function_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_nested_pickle, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_no_return_result, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_raise_in_user_func, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rref_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors_in_container, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors_multi_async_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_reinit, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_local_rrefs, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_async_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_async_function_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_timeouts, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_py_pickle_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_owner_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_owner_non_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_scalar_add, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_add, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_self_remote_arg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_send_to_rank, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_server_process_global_profiler, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_set_and_get_default_rpc_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_stress_heavy_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_stress_light_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rref_backward, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_raise_in_user_func, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_dense, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_complex_python_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_output_tensors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_unused_tensors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_debug_info, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_dist_autograd_profiling, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_builtin_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_python_remote_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_multiple_backward_with_errors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_nested_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_graph_with_tensors_not_require_grad_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_remote_complex_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_trainer_ps_torchscript_functions, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_wrong_return_type_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_script_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_no_kwargs_are_populated_by_defaults, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_record_function_jit_end_callbacks_with_fork, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_record_function_on_caller_rpc_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rpc_async_jit_profiled, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_as_arg_and_return, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_is_owner, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_jit_pickle_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_list_mutate, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_local_value, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_functions_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_user_rrefs_confirmed, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_dist_backward, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_jit_fork_within_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_sync, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_get_module_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_remote_module_py_pickle_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire_script_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_outside, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_outside_uneven_inputs, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_no_ddp, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_and_static_init_rpc_together, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc_without_rank, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_op_with_invalid_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rpc_rref_args_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_sparse_tensors_in_container, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_rpc_arg_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_set_and_get_num_worker_threads, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_nested_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_self_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_with_grad_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_remote_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_python_remote_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_no_graph_with_tensors_not_require_grad_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_rpc_complex_args_sparse 2024-08-07T19:02:41.6723801Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_add 2024-08-07T19:02:41.6724864Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy_remote 2024-08-07T19:02:41.6726019Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_chained_async 2024-08-07T19:02:41.6727125Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_chained_remote 2024-08-07T19:02:41.6728198Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise_async 2024-08-07T19:02:41.6729245Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise_remote 2024-08-07T19:02:41.6730344Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_async 2024-08-07T19:02:41.6731391Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_static_method 2024-08-07T19:02:41.6732395Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_static_method_remote 2024-08-07T19:02:41.6733403Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_builtin_remote_self 2024-08-07T19:02:41.6734346Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_simple 2024-08-07T19:02:41.6735370Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_cannot_infer_backend_from_options 2024-08-07T19:02:41.6736418Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_default_timeout_used 2024-08-07T19:02:41.6737380Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_duplicate_name 2024-08-07T19:02:41.6738308Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_expected_src 2024-08-07T19:02:41.6739243Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done 2024-08-07T19:02:41.6740368Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_done_exception 2024-08-07T19:02:41.6741345Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_get_worker_infos 2024-08-07T19:02:41.6742281Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_pg_then_rpc 2024-08-07T19:02:41.6743225Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_invalid_names 2024-08-07T19:02:41.6744162Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_shutdown 2024-08-07T19:02:41.6745133Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_builtin_remote_ret 2024-08-07T19:02:41.6746105Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_rpc 2024-08-07T19:02:41.6747045Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rref_stress 2024-08-07T19:02:41.6748015Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_owner_rref_backward 2024-08-07T19:02:41.6748966Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_pg_init_no_rpc_init 2024-08-07T19:02:41.6749918Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_pickle_future 2024-08-07T19:02:41.6750883Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_export_trace 2024-08-07T19:02:41.6752053Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_remote_events_profiled 2024-08-07T19:02:41.6753175Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_rpc_memory 2024-08-07T19:02:41.6754191Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_rpc_record_shapes 2024-08-07T19:02:41.6755322Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_autograd_context_single_threaded 2024-08-07T19:02:41.6756526Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin_single_threaded 2024-08-07T19:02:41.6757665Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_udf 2024-08-07T19:02:41.6758838Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_udf_single_threaded 2024-08-07T19:02:41.6759902Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_built_in 2024-08-07T19:02:41.6760822Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_method 2024-08-07T19:02:41.6761797Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_function_exception 2024-08-07T19:02:41.6762778Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_nested_pickle 2024-08-07T19:02:41.6763805Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_no_return_result 2024-08-07T19:02:41.6764787Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_raise_in_user_func 2024-08-07T19:02:41.6765752Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rref_args 2024-08-07T19:02:41.6766721Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors_in_container 2024-08-07T19:02:41.6767730Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors_multi_async_call 2024-08-07T19:02:41.6768692Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_reinit 2024-08-07T19:02:41.6769596Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future 2024-08-07T19:02:41.6770529Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future_async 2024-08-07T19:02:41.6771503Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future_remote 2024-08-07T19:02:41.6772479Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_local_rrefs 2024-08-07T19:02:41.6773490Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_async_function 2024-08-07T19:02:41.6774603Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_async_function_single_threaded 2024-08-07T19:02:41.6775665Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_timeouts 2024-08-07T19:02:41.6776658Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_py_pickle_not_supported 2024-08-07T19:02:41.6777693Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_owner_blocking 2024-08-07T19:02:41.6778796Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_owner_non_blocking 2024-08-07T19:02:41.6779830Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_scalar_add 2024-08-07T19:02:41.6780728Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_add 2024-08-07T19:02:41.6781720Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_self_remote_arg 2024-08-07T19:02:41.6782741Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_send_to_rank 2024-08-07T19:02:41.6783736Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_server_process_global_profiler 2024-08-07T19:02:41.6784811Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_set_and_get_default_rpc_timeout 2024-08-07T19:02:41.6785808Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_stress_heavy_rpc 2024-08-07T19:02:41.6786765Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_stress_light_rpc 2024-08-07T19:02:41.6787726Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rref_backward 2024-08-07T19:02:41.6788716Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_raise_in_user_func 2024-08-07T19:02:41.6789708Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_timeout 2024-08-07T19:02:41.6790686Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_dense 2024-08-07T19:02:41.6791710Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_workers_timeout 2024-08-07T19:02:41.6792779Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_complex_python_udf 2024-08-07T19:02:41.6793959Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_output_tensors 2024-08-07T19:02:41.6795068Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref 2024-08-07T19:02:41.6796130Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_unused_tensors 2024-08-07T19:02:41.6797162Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_debug_info 2024-08-07T19:02:41.6798209Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_dist_autograd_profiling 2024-08-07T19:02:41.6799392Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_builtin_call 2024-08-07T19:02:41.6800505Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_python_remote_call 2024-08-07T19:02:41.6801707Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_multiple_backward_with_errors 2024-08-07T19:02:41.6802792Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_nested_context 2024-08-07T19:02:41.6803959Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_graph_with_tensors_not_require_grad_remote 2024-08-07T19:02:41.6805151Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_remote_complex_args 2024-08-07T19:02:41.6806363Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_trainer_ps_torchscript_functions 2024-08-07T19:02:41.6807510Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim 2024-08-07T19:02:41.6808628Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_wrong_return_type_remote 2024-08-07T19:02:41.6810165Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_script_udf 2024-08-07T19:02:41.6811242Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_no_kwargs_are_populated_by_defaults 2024-08-07T19:02:41.6812413Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_record_function_jit_end_callbacks_with_fork 2024-08-07T19:02:41.6813611Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_record_function_on_caller_rpc_async 2024-08-07T19:02:41.6814687Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_udf 2024-08-07T19:02:41.6815690Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rpc_async_jit_profiled 2024-08-07T19:02:41.6816730Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_as_arg_and_return 2024-08-07T19:02:41.6817725Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_is_owner 2024-08-07T19:02:41.6818758Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_jit_pickle_not_supported 2024-08-07T19:02:41.6819779Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_list_mutate 2024-08-07T19:02:41.6820765Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_local_value 2024-08-07T19:02:41.6821839Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_functions_not_supported 2024-08-07T19:02:41.6822931Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_user_rrefs_confirmed 2024-08-07T19:02:41.6823948Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_dist_backward 2024-08-07T19:02:41.6825038Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_jit_fork_within_context 2024-08-07T19:02:41.6826113Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_sync 2024-08-07T19:02:41.6827124Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_get_module_rref 2024-08-07T19:02:41.6828265Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_remote_module_py_pickle_not_supported 2024-08-07T19:02:41.6829632Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire_script_not_supported 2024-08-07T19:02:41.6830964Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_outside 2024-08-07T19:02:41.6832188Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_outside_uneven_inputs 2024-08-07T19:02:41.6833400Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_no_ddp 2024-08-07T19:02:41.6834697Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_and_static_init_rpc_together 2024-08-07T19:02:41.6835981Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc 2024-08-07T19:02:41.6837175Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_init_rpc_without_rank 2024-08-07T19:02:41.6838440Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_op_with_invalid_args 2024-08-07T19:02:41.6839605Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rpc_rref_args_sparse 2024-08-07T19:02:41.6840928Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_sparse 2024-08-07T19:02:41.6842110Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_sparse_tensors_in_container 2024-08-07T19:02:41.6843380Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_rpc_arg_sparse 2024-08-07T19:02:41.6844642Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_set_and_get_num_worker_threads 2024-08-07T19:02:41.6845928Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_nested_sparse 2024-08-07T19:02:41.6847231Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_self_sparse 2024-08-07T19:02:41.6848598Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_with_grad_sparse 2024-08-07T19:02:41.6849979Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_call_sparse 2024-08-07T19:02:41.6851358Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_remote_call_sparse 2024-08-07T19:02:41.6852743Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_python_remote_call_sparse 2024-08-07T19:02:41.6854166Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_no_graph_with_tensors_not_require_grad_sparse 2024-08-07T19:02:41.6855532Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_rpc_complex_args_sparse 2024-08-07T19:02:41.6856227Z 2024-08-07T19:02:41.6856516Z Running distributed/rpc/test_tensorpipe_agent 4/4 ... [2024-08-07 19:02:41.664793] 2024-08-07T19:02:41.6857879Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/rpc/test_tensorpipe_agent.py', '--shard-id=4', '--num-shards=4', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:02:41.665239] 2024-08-07T19:15:51.2556861Z 2024-08-07T19:15:51.2557946Z distributed/rpc/test_tensorpipe_agent 4/4 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_tensorpipe_agent_4.4_753ab7ef8e7087b4_.log 2024-08-07T19:15:51.2639046Z Running 112 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_all_gather, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_all_gather_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_method, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_method_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_fanout_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_nested, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_nested_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_with_future_ctor_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_double_end_callbacks, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_legacy, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_wrong_arg_num, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_wrong_arg_type, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_function_not_on_callee, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_graceful_shutdown_with_uneven_workload, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_handle_send_exceptions, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_rpc_twice, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_value_not_on_owner, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_mark_future_twice, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_non_cont_tensors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_owner_equality, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_pass_local_rrefs, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_udf_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_builtin, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_async_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_sync_rpc_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_raise_in_user_func_escaped_str, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_user_defined, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_remote_throw, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_remote_with_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_all, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_multithreaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_subset, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_context_debug_info, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_leak, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_class_self, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_reuse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_tensor, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_tensor_self, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_str, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_remote_arg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_self_rpc_arg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_shutdown_errors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_shutdown_followed_by_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rpc_pickler, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rref_after_shutdown, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_exit_early_python, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_multiple_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_worker_id, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_worker_info_pickle, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeParameterServerTest::test_batch_updating_parameter_server, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_async_dist_autograd, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_multi, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_self, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_without_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_no_tensors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_only_on_return_value, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_call_itself, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_multiple_backward, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_worker_ids_recorded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_none_grads, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_wrong_decorator_order, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_script_throw, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_python_function_remotely_from_script_not_supported, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_script_function_that_not_exists_remotely_from_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_script_function_that_raises_remotely_from_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_chain, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_create_script_module_on_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_future_passed_between_python_and_jit, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_less_than_needed_args_are_specified, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_load_script_module_with_pickled_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rpc_torchscript_record_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_function_exception, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_get_gradients, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_sync_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_with_kwargs, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_remote_module_py_pickle_not_supported_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_remote_parameters, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_inside, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_comparison_uneven_inputs, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_dist_autograd_sparse_grads, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_existing_rank_can_communicate_with_new_rank_cuda, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_new_rank_can_communicated_with_existing_rank, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_my_parameter_server_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_remote_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rref_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rref_stress_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_user_share_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_proxy_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_tensorpipe_set_default_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_wait_all_workers_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_multiple_round_trips_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_no_grad_on_tensor_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_python_udf_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_script_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_remote_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_remote_call_itself_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_python_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_mixed_requires_grad_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_nested_backward_accumulate_grads_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_trainer_ps_sparse 2024-08-07T19:15:51.2691054Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_all_gather 2024-08-07T19:15:51.2692008Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_all_gather_timeout 2024-08-07T19:15:51.2692980Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_method 2024-08-07T19:15:51.2693970Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_method_remote 2024-08-07T19:15:51.2695053Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_fanout_remote 2024-08-07T19:15:51.2696211Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_nested 2024-08-07T19:15:51.2697333Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_nested_remote 2024-08-07T19:15:51.2698341Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_raise 2024-08-07T19:15:51.2699409Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_with_future_ctor_remote 2024-08-07T19:15:51.2700566Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_remote 2024-08-07T19:15:51.2701718Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_double_end_callbacks 2024-08-07T19:15:51.2702839Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_legacy 2024-08-07T19:15:51.2703867Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_wrong_arg_num 2024-08-07T19:15:51.2704880Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_wrong_arg_type 2024-08-07T19:15:51.2705876Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_function_not_on_callee 2024-08-07T19:15:51.2706948Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_graceful_shutdown_with_uneven_workload 2024-08-07T19:15:51.2708019Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_handle_send_exceptions 2024-08-07T19:15:51.2708994Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_rpc_twice 2024-08-07T19:15:51.2709967Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_value_not_on_owner 2024-08-07T19:15:51.2710957Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_mark_future_twice 2024-08-07T19:15:51.2711894Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rpc 2024-08-07T19:15:51.2712798Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rref 2024-08-07T19:15:51.2713732Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_non_cont_tensors 2024-08-07T19:15:51.2714670Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_owner_equality 2024-08-07T19:15:51.2715610Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_pass_local_rrefs 2024-08-07T19:15:51.2716666Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_udf_single_threaded 2024-08-07T19:15:51.2717792Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_builtin 2024-08-07T19:15:51.2718918Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_async_rpc 2024-08-07T19:15:51.2720048Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_sync_rpc_single_threaded 2024-08-07T19:15:51.2721209Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_raise_in_user_func_escaped_str 2024-08-07T19:15:51.2722240Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_user_defined 2024-08-07T19:15:51.2723240Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_remote_throw 2024-08-07T19:15:51.2724250Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_remote_with_exception 2024-08-07T19:15:51.2725230Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_all 2024-08-07T19:15:51.2726219Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_multithreaded 2024-08-07T19:15:51.2727222Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_subset 2024-08-07T19:15:51.2728195Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_context_debug_info 2024-08-07T19:15:51.2729160Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_leak 2024-08-07T19:15:51.2730116Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_class_self 2024-08-07T19:15:51.2731079Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_reuse 2024-08-07T19:15:51.2732029Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_tensor 2024-08-07T19:15:51.2733003Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_tensor_self 2024-08-07T19:15:51.2733948Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_str 2024-08-07T19:15:51.2734843Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_timeout 2024-08-07T19:15:51.2735783Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_blocking 2024-08-07T19:15:51.2736802Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_remote_arg 2024-08-07T19:15:51.2737875Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_self_rpc_arg 2024-08-07T19:15:51.2738896Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_shutdown_errors 2024-08-07T19:15:51.2739876Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_shutdown_followed_by_rpc 2024-08-07T19:15:51.2741082Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rpc_pickler 2024-08-07T19:15:51.2742047Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rref_after_shutdown 2024-08-07T19:15:51.2743076Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_exit_early_python 2024-08-07T19:15:51.2744086Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_multiple_call 2024-08-07T19:15:51.2745036Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_worker_id 2024-08-07T19:15:51.2745954Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_worker_info_pickle 2024-08-07T19:15:51.2747052Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeParameterServerTest::test_batch_updating_parameter_server 2024-08-07T19:15:51.2748195Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_async_dist_autograd 2024-08-07T19:15:51.2749243Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_multi 2024-08-07T19:15:51.2750419Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple 2024-08-07T19:15:51.2751562Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_self 2024-08-07T19:15:51.2752641Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_without_rpc 2024-08-07T19:15:51.2753730Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_no_tensors 2024-08-07T19:15:51.2754861Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_only_on_return_value 2024-08-07T19:15:51.2756015Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_call_itself 2024-08-07T19:15:51.2757135Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_multiple_backward 2024-08-07T19:15:51.2758186Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_worker_ids_recorded 2024-08-07T19:15:51.2759350Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_none_grads 2024-08-07T19:15:51.2760417Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_remote 2024-08-07T19:15:51.2761499Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_wrong_decorator_order 2024-08-07T19:15:51.2762578Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_script_throw 2024-08-07T19:15:51.2763823Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_python_function_remotely_from_script_not_supported 2024-08-07T19:15:51.2765159Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_script_function_that_not_exists_remotely_from_script 2024-08-07T19:15:51.2766457Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_script_function_that_raises_remotely_from_script 2024-08-07T19:15:51.2767599Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_chain 2024-08-07T19:15:51.2768635Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_create_script_module_on_remote 2024-08-07T19:15:51.2769778Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_future_passed_between_python_and_jit 2024-08-07T19:15:51.2770919Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_less_than_needed_args_are_specified 2024-08-07T19:15:51.2772079Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_load_script_module_with_pickled_rref 2024-08-07T19:15:51.2773223Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rpc_torchscript_record_function 2024-08-07T19:15:51.2774380Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_torchscript_function_exception 2024-08-07T19:15:51.2775501Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_get_gradients 2024-08-07T19:15:51.2776557Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_sync_script 2024-08-07T19:15:51.2777699Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_forward_with_kwargs 2024-08-07T19:15:51.2778868Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_remote_module_py_pickle_not_supported_script 2024-08-07T19:15:51.2780108Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_remote_parameters 2024-08-07T19:15:51.2781303Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire 2024-08-07T19:15:51.2782535Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_inside 2024-08-07T19:15:51.2783703Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_comparison_uneven_inputs 2024-08-07T19:15:51.2784898Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_dist_autograd_local_vs_remote 2024-08-07T19:15:51.2786079Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_dist_autograd_sparse_grads 2024-08-07T19:15:51.2787412Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_existing_rank_can_communicate_with_new_rank_cuda 2024-08-07T19:15:51.2788891Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_dynamic_rpc_new_rank_can_communicated_with_existing_rank 2024-08-07T19:15:51.2790225Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_my_parameter_server_sparse 2024-08-07T19:15:51.2791389Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_remote_sparse 2024-08-07T19:15:51.2792523Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rref_sparse 2024-08-07T19:15:51.2793683Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rref_stress_sparse 2024-08-07T19:15:51.2794901Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_user_share_sparse 2024-08-07T19:15:51.2796070Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_rref_proxy_timeout 2024-08-07T19:15:51.2797262Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_tensorpipe_set_default_timeout 2024-08-07T19:15:51.2798542Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_wait_all_workers_sparse 2024-08-07T19:15:51.2799832Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_multiple_round_trips_sparse 2024-08-07T19:15:51.2801205Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_no_grad_on_tensor_sparse 2024-08-07T19:15:51.2802520Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_sparse 2024-08-07T19:15:51.2803833Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_python_udf_sparse 2024-08-07T19:15:51.2805217Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_script_call_sparse 2024-08-07T19:15:51.2806671Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_remote_call_sparse 2024-08-07T19:15:51.2808041Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_call_sparse 2024-08-07T19:15:51.2809510Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_remote_call_itself_sparse 2024-08-07T19:15:51.2810916Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_python_call_sparse 2024-08-07T19:15:51.2812215Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_mixed_requires_grad_sparse 2024-08-07T19:15:51.2813582Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_nested_backward_accumulate_grads_sparse 2024-08-07T19:15:51.2814917Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_trainer_ps_sparse 2024-08-07T19:15:51.2815587Z 2024-08-07T19:15:51.2815907Z Running distributed/_shard/sharded_tensor/ops/test_init 1/1 ... [2024-08-07 19:15:51.261586] 2024-08-07T19:15:51.2817278Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_init.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:15:51.262017] 2024-08-07T19:15:53.7351850Z 2024-08-07T19:15:53.7353090Z distributed/_shard/sharded_tensor/ops/test_init 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_init_1.1_5ac4b6c2113a40c7_.log 2024-08-07T19:15:53.7355651Z Running 3 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_init.py::TestShardedTensorNNInit::test_init_sharded_tensor_with_kaiming_uniform, test/distributed/_shard/sharded_tensor/ops/test_init.py::TestShardedTensorNNInit::test_init_sharded_tensor_with_normal, test/distributed/_shard/sharded_tensor/ops/test_init.py::TestShardedTensorNNInit::test_init_sharded_tensor_with_uniform 2024-08-07T19:15:53.7357561Z 2024-08-07T19:15:53.7358555Z Running distributed/_shard/sharded_optim/test_sharded_optim 1/1 ... [2024-08-07 19:15:53.735676] 2024-08-07T19:15:53.7363301Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_optim/test_sharded_optim.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:15:53.736111] 2024-08-07T19:15:56.2092466Z 2024-08-07T19:15:56.2093677Z distributed/_shard/sharded_optim/test_sharded_optim 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_optim.test_sharded_optim_1.1_938a488d10343518_.log 2024-08-07T19:15:56.2095624Z Running 2 items in this shard: test/distributed/_shard/sharded_optim/test_sharded_optim.py::TestShardedOptimizer::test_named_params_with_sharded_tensor, test/distributed/_shard/sharded_optim/test_sharded_optim.py::TestShardedOptimizer::test_sharded_optim 2024-08-07T19:15:56.2096753Z 2024-08-07T19:15:56.2099082Z Running distributed/test_c10d_spawn_gloo 1/1 ... [2024-08-07 19:15:56.209738] 2024-08-07T19:15:56.2103921Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_spawn_gloo.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:15:56.210179] 2024-08-07T19:17:03.9685881Z 2024-08-07T19:17:03.9687848Z distributed/test_c10d_spawn_gloo 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_spawn_gloo_1.1_69483467310e45eb_.log 2024-08-07T19:17:03.9696100Z Running 15 items in this shard: test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allgather_chunk_gloo, test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allgather_gloo, test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allreduce_gloo, test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_broadcast_gloo, test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cpu, test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cuda, test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_rnn, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_gather, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all_single, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_allreduce, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_broadcast, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_gather, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_reduce, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_scatter 2024-08-07T19:17:03.9703723Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allgather_chunk_gloo 2024-08-07T19:17:03.9704791Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allgather_gloo 2024-08-07T19:17:03.9705808Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_allreduce_gloo 2024-08-07T19:17:03.9706842Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::ProcessGroupShareTensorTest::test_shared_broadcast_gloo 2024-08-07T19:17:03.9707859Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cpu 2024-08-07T19:17:03.9708876Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cuda 2024-08-07T19:17:03.9709866Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_rnn 2024-08-07T19:17:03.9710848Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_gather 2024-08-07T19:17:03.9711805Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all 2024-08-07T19:17:03.9712782Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all_single 2024-08-07T19:17:03.9713768Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_allreduce 2024-08-07T19:17:03.9714723Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_broadcast 2024-08-07T19:17:03.9715657Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_gather 2024-08-07T19:17:03.9716567Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_reduce 2024-08-07T19:17:03.9717488Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_scatter 2024-08-07T19:17:03.9718009Z 2024-08-07T19:17:03.9718274Z Running distributed/fsdp/test_fsdp_comm 1/1 ... [2024-08-07 19:17:03.969377] 2024-08-07T19:17:03.9719693Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/fsdp/test_fsdp_comm.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:17:03.969814] 2024-08-07T19:17:29.5891263Z 2024-08-07T19:17:29.5892273Z distributed/fsdp/test_fsdp_comm 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_comm_1.1_9ae0a98a1384c4f7_.log 2024-08-07T19:17:29.5898642Z Running 10 items in this shard: test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_False_use_no_sync_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_comm.py::TestCommunication::test_communication_nested_model_True_use_no_sync_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_comm.py::TestExplicitUnshard::test_unshard_async_use_orig_params_False, test/distributed/fsdp/test_fsdp_comm.py::TestExplicitUnshard::test_unshard_async_use_orig_params_True 2024-08-07T19:17:29.5904453Z 2024-08-07T19:17:29.5904790Z Running distributed/_shard/sharded_tensor/ops/test_embedding 1/1 ... [2024-08-07 19:17:29.589596] 2024-08-07T19:17:29.5906234Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_embedding.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:17:29.590066] 2024-08-07T19:17:32.0640695Z 2024-08-07T19:17:32.0642161Z distributed/_shard/sharded_tensor/ops/test_embedding 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_1.1_dd71ca186388b9df_.log 2024-08-07T19:17:32.0644186Z Running 2 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_embedding.py::TestShardedEmbedding::test_sharded_embedding_colwise, test/distributed/_shard/sharded_tensor/ops/test_embedding.py::TestShardedEmbedding::test_sharded_embedding_rowwise 2024-08-07T19:17:32.0645367Z 2024-08-07T19:17:32.0647404Z Running distributed/_shard/sharded_tensor/ops/test_embedding_bag 1/1 ... [2024-08-07 19:17:32.064518] 2024-08-07T19:17:32.0652326Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_embedding_bag.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:17:32.064968] 2024-08-07T19:17:34.5381069Z 2024-08-07T19:17:34.5382615Z distributed/_shard/sharded_tensor/ops/test_embedding_bag 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_bag_1.1_8b9deb8faf643d86_.log 2024-08-07T19:17:34.5384736Z Running 2 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_embedding_bag.py::TestShardedEmbeddingBag::test_sharded_embedding_bag_colwise, test/distributed/_shard/sharded_tensor/ops/test_embedding_bag.py::TestShardedEmbeddingBag::test_sharded_embedding_bag_rowwise 2024-08-07T19:17:34.5386046Z 2024-08-07T19:17:34.5387745Z Running distributed/_shard/sharded_tensor/test_sharded_tensor_reshard 1/1 ... [2024-08-07 19:17:34.538560] 2024-08-07T19:17:34.5392852Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/test_sharded_tensor_reshard.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:17:34.538996] 2024-08-07T19:17:37.0121967Z 2024-08-07T19:17:37.0123507Z distributed/_shard/sharded_tensor/test_sharded_tensor_reshard 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_reshard_1.1_faf5fb214e871656_.log 2024-08-07T19:17:37.0125584Z Running 2 items in this shard: test/distributed/_shard/sharded_tensor/test_sharded_tensor_reshard.py::TestReshard::test_sharded_tensor_reshard, test/distributed/_shard/sharded_tensor/test_sharded_tensor_reshard.py::TestReshard::test_sharded_tensor_reshard_errors 2024-08-07T19:17:37.0126722Z 2024-08-07T19:17:37.0128669Z Running distributed/test_store 1/1 ... [2024-08-07 19:17:37.012682] 2024-08-07T19:17:37.0134488Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_store.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:17:37.013188] 2024-08-07T19:22:00.4578118Z 2024-08-07T19:22:00.4579619Z distributed/test_store 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_store_1.1_3d473ec3bcb95972_.log 2024-08-07T19:22:00.4629780Z Running 86 items in this shard: test/distributed/test_store.py::FileStoreTest::test_append, test/distributed/test_store.py::FileStoreTest::test_compare_set, test/distributed/test_store.py::FileStoreTest::test_init_pg_and_rpc_with_same_file, test/distributed/test_store.py::FileStoreTest::test_multi_get, test/distributed/test_store.py::FileStoreTest::test_multi_set, test/distributed/test_store.py::FileStoreTest::test_refcount, test/distributed/test_store.py::FileStoreTest::test_set_get_check, test/distributed/test_store.py::FileStoreTest::test_simple_wait, test/distributed/test_store.py::HashStoreTest::test_append, test/distributed/test_store.py::HashStoreTest::test_compare_set, test/distributed/test_store.py::HashStoreTest::test_multi_get, test/distributed/test_store.py::HashStoreTest::test_multi_set, test/distributed/test_store.py::HashStoreTest::test_set_get_check, test/distributed/test_store.py::HashStoreTest::test_simple_wait, test/distributed/test_store.py::PrefixStoreTest::test_get_underlying_store, test/distributed/test_store.py::PrefixFileStoreTest::test_append, test/distributed/test_store.py::PrefixFileStoreTest::test_compare_set, test/distributed/test_store.py::PrefixFileStoreTest::test_multi_get, test/distributed/test_store.py::PrefixFileStoreTest::test_multi_set, test/distributed/test_store.py::PrefixFileStoreTest::test_set_get_check, test/distributed/test_store.py::PrefixFileStoreTest::test_simple_wait, test/distributed/test_store.py::TCPStoreTest::test_address_already_in_use, test/distributed/test_store.py::TCPStoreTest::test_append, test/distributed/test_store.py::TCPStoreTest::test_compare_set, test/distributed/test_store.py::TCPStoreTest::test_init_pg_and_rpc_with_same_socket, test/distributed/test_store.py::TCPStoreTest::test_multi_get, test/distributed/test_store.py::TCPStoreTest::test_multi_set, test/distributed/test_store.py::TCPStoreTest::test_multi_worker_with_fixed_world_size, test/distributed/test_store.py::TCPStoreTest::test_multi_worker_with_nonfixed_world_size, test/distributed/test_store.py::TCPStoreTest::test_multitenancy, test/distributed/test_store.py::TCPStoreTest::test_numkeys_delkeys, test/distributed/test_store.py::TCPStoreTest::test_repr, test/distributed/test_store.py::TCPStoreTest::test_set_get_check, test/distributed/test_store.py::TCPStoreTest::test_simple_wait, test/distributed/test_store.py::TCPStoreTest::test_store_timeout_on_missing_clients, test/distributed/test_store.py::TCPStoreTest::test_take_over_listen_socket, test/distributed/test_store.py::LibUvTCPStoreTest::test_address_already_in_use, test/distributed/test_store.py::LibUvTCPStoreTest::test_append, test/distributed/test_store.py::LibUvTCPStoreTest::test_compare_set, test/distributed/test_store.py::LibUvTCPStoreTest::test_init_pg_and_rpc_with_same_socket, test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_get, test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_set, test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_worker_with_fixed_world_size, test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_worker_with_nonfixed_world_size, test/distributed/test_store.py::LibUvTCPStoreTest::test_multitenancy, test/distributed/test_store.py::LibUvTCPStoreTest::test_numkeys_delkeys, test/distributed/test_store.py::LibUvTCPStoreTest::test_repr, test/distributed/test_store.py::LibUvTCPStoreTest::test_set_get_check, test/distributed/test_store.py::LibUvTCPStoreTest::test_simple_wait, test/distributed/test_store.py::LibUvTCPStoreTest::test_store_timeout_on_missing_clients, test/distributed/test_store.py::LibUvTCPStoreTest::test_take_over_listen_socket, test/distributed/test_store.py::PrefixTCPStoreTest::test_append, test/distributed/test_store.py::PrefixTCPStoreTest::test_compare_set, test/distributed/test_store.py::PrefixTCPStoreTest::test_multi_get, test/distributed/test_store.py::PrefixTCPStoreTest::test_multi_set, test/distributed/test_store.py::PrefixTCPStoreTest::test_set_get_check, test/distributed/test_store.py::PrefixTCPStoreTest::test_simple_wait, test/distributed/test_store.py::PrefixTCPStoreTest::test_underlying_non_prefix_store, test/distributed/test_store.py::PythonStoreTest::test_set_get, test/distributed/test_store.py::RendezvousTest::test_unknown_handler, test/distributed/test_store.py::RendezvousTest::test_url_with_node_params, test/distributed/test_store.py::RendezvousEnvTest::test_nominal, test/distributed/test_store.py::RendezvousFileTest::test_common_errors, test/distributed/test_store.py::RendezvousFileTest::test_nominal, test/distributed/test_store.py::RendezvousTCPTest::test_common_errors, test/distributed/test_store.py::RendezvousTCPTest::test_dns_timeout, test/distributed/test_store.py::RendezvousTCPTest::test_nominal, test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_timeout_doest_break_client, test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_timeout_set, test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_url_with_libuv, test/distributed/test_store.py::TestPythonStore::test_append_roundtrip, test/distributed/test_store.py::TestPythonStore::test_extended_methods_fallbacks, test/distributed/test_store.py::TestPythonStore::test_has_extended_api_passthrough, test/distributed/test_store.py::TestPythonStore::test_has_extended_api_roundtrip, test/distributed/test_store.py::TestPythonStore::test_multi_get_roundtrip, test/distributed/test_store.py::TestPythonStore::test_multi_set_roundtrip, test/distributed/test_store.py::TestPythonStore::test_optional_methods_fail, test/distributed/test_store.py::TestMultiThreadedWait::test_wait_file_store, test/distributed/test_store.py::TestMultiThreadedWait::test_wait_hash_store, test/distributed/test_store.py::TestMultiThreadedWait::test_wait_prefix_file_store, test/distributed/test_store.py::TestMultiThreadedWait::test_wait_tcp_store, test/distributed/test_store.py::TestMultiThreadedWait::test_wait_tcp_store_uv, test/distributed/test_store.py::TimeoutTest::test_interrupt_doesnt_break_wait, test/distributed/test_store.py::InitPgWithNonUvStore::test_with_env_var, test/distributed/test_store.py::InitPgWithNonUvStore::test_with_url_param, test/distributed/test_store.py::TestClientProtocol::test_client_connect 2024-08-07T19:22:00.4668251Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_append 2024-08-07T19:22:00.4668993Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_compare_set 2024-08-07T19:22:00.4669815Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_init_pg_and_rpc_with_same_file 2024-08-07T19:22:00.4670852Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_multi_get 2024-08-07T19:22:00.4671694Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_multi_set 2024-08-07T19:22:00.4672422Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_refcount 2024-08-07T19:22:00.4673147Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_set_get_check 2024-08-07T19:22:00.4673901Z Running 1 items in this shard: test/distributed/test_store.py::FileStoreTest::test_simple_wait 2024-08-07T19:22:00.4674624Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_append 2024-08-07T19:22:00.4675346Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_compare_set 2024-08-07T19:22:00.4676062Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_multi_get 2024-08-07T19:22:00.4676787Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_multi_set 2024-08-07T19:22:00.4677526Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_set_get_check 2024-08-07T19:22:00.4678265Z Running 1 items in this shard: test/distributed/test_store.py::HashStoreTest::test_simple_wait 2024-08-07T19:22:00.4679130Z Running 1 items in this shard: test/distributed/test_store.py::PrefixStoreTest::test_get_underlying_store 2024-08-07T19:22:00.4679924Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_append 2024-08-07T19:22:00.4680693Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_compare_set 2024-08-07T19:22:00.4681494Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_multi_get 2024-08-07T19:22:00.4682282Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_multi_set 2024-08-07T19:22:00.4683081Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_set_get_check 2024-08-07T19:22:00.4683874Z Running 1 items in this shard: test/distributed/test_store.py::PrefixFileStoreTest::test_simple_wait 2024-08-07T19:22:00.4684691Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_address_already_in_use 2024-08-07T19:22:00.4685443Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_append 2024-08-07T19:22:00.4686149Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_compare_set 2024-08-07T19:22:00.4686968Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_init_pg_and_rpc_with_same_socket 2024-08-07T19:22:00.4687772Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_multi_get 2024-08-07T19:22:00.4688475Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_multi_set 2024-08-07T19:22:00.4689529Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_multi_worker_with_fixed_world_size 2024-08-07T19:22:00.4690612Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_multi_worker_with_nonfixed_world_size 2024-08-07T19:22:00.4691458Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_multitenancy 2024-08-07T19:22:00.4692203Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_numkeys_delkeys 2024-08-07T19:22:00.4692919Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_repr 2024-08-07T19:22:00.4693630Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_set_get_check 2024-08-07T19:22:00.4694361Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_simple_wait 2024-08-07T19:22:00.4695176Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_store_timeout_on_missing_clients 2024-08-07T19:22:00.4696126Z Running 1 items in this shard: test/distributed/test_store.py::TCPStoreTest::test_take_over_listen_socket 2024-08-07T19:22:00.4697051Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_address_already_in_use 2024-08-07T19:22:00.4697846Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_append 2024-08-07T19:22:00.4698612Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_compare_set 2024-08-07T19:22:00.4699480Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_init_pg_and_rpc_with_same_socket 2024-08-07T19:22:00.4700325Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_get 2024-08-07T19:22:00.4701090Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_set 2024-08-07T19:22:00.4701958Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_worker_with_fixed_world_size 2024-08-07T19:22:00.4702917Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_multi_worker_with_nonfixed_world_size 2024-08-07T19:22:00.4703811Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_multitenancy 2024-08-07T19:22:00.4704610Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_numkeys_delkeys 2024-08-07T19:22:00.4705374Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_repr 2024-08-07T19:22:00.4706121Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_set_get_check 2024-08-07T19:22:00.4706908Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_simple_wait 2024-08-07T19:22:00.4707770Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_store_timeout_on_missing_clients 2024-08-07T19:22:00.4708664Z Running 1 items in this shard: test/distributed/test_store.py::LibUvTCPStoreTest::test_take_over_listen_socket 2024-08-07T19:22:00.4709480Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_append 2024-08-07T19:22:00.4710252Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_compare_set 2024-08-07T19:22:00.4711034Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_multi_get 2024-08-07T19:22:00.4711789Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_multi_set 2024-08-07T19:22:00.4712575Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_set_get_check 2024-08-07T19:22:00.4713373Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_simple_wait 2024-08-07T19:22:00.4714218Z Running 1 items in this shard: test/distributed/test_store.py::PrefixTCPStoreTest::test_underlying_non_prefix_store 2024-08-07T19:22:00.4715046Z Running 1 items in this shard: test/distributed/test_store.py::PythonStoreTest::test_set_get 2024-08-07T19:22:00.4715803Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTest::test_unknown_handler 2024-08-07T19:22:00.4716603Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTest::test_url_with_node_params 2024-08-07T19:22:00.4717369Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousEnvTest::test_nominal 2024-08-07T19:22:00.4718137Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousFileTest::test_common_errors 2024-08-07T19:22:00.4718990Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousFileTest::test_nominal 2024-08-07T19:22:00.4719754Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_common_errors 2024-08-07T19:22:00.4720539Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_dns_timeout 2024-08-07T19:22:00.4721380Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_nominal 2024-08-07T19:22:00.4722312Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_timeout_doest_break_client 2024-08-07T19:22:00.4723242Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_timeout_set 2024-08-07T19:22:00.4724118Z Running 1 items in this shard: test/distributed/test_store.py::RendezvousTCPTest::test_tcp_store_url_with_libuv 2024-08-07T19:22:00.4724966Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_append_roundtrip 2024-08-07T19:22:00.4725799Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_extended_methods_fallbacks 2024-08-07T19:22:00.4726685Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_has_extended_api_passthrough 2024-08-07T19:22:00.4727575Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_has_extended_api_roundtrip 2024-08-07T19:22:00.4728405Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_multi_get_roundtrip 2024-08-07T19:22:00.4729226Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_multi_set_roundtrip 2024-08-07T19:22:00.4730054Z Running 1 items in this shard: test/distributed/test_store.py::TestPythonStore::test_optional_methods_fail 2024-08-07T19:22:00.4730893Z Running 1 items in this shard: test/distributed/test_store.py::TestMultiThreadedWait::test_wait_file_store 2024-08-07T19:22:00.4731710Z Running 1 items in this shard: test/distributed/test_store.py::TestMultiThreadedWait::test_wait_hash_store 2024-08-07T19:22:00.4732579Z Running 1 items in this shard: test/distributed/test_store.py::TestMultiThreadedWait::test_wait_prefix_file_store 2024-08-07T19:22:00.4733447Z Running 1 items in this shard: test/distributed/test_store.py::TestMultiThreadedWait::test_wait_tcp_store 2024-08-07T19:22:00.4734282Z Running 1 items in this shard: test/distributed/test_store.py::TestMultiThreadedWait::test_wait_tcp_store_uv 2024-08-07T19:22:00.4735142Z Running 1 items in this shard: test/distributed/test_store.py::TimeoutTest::test_interrupt_doesnt_break_wait 2024-08-07T19:22:00.4735975Z Running 1 items in this shard: test/distributed/test_store.py::InitPgWithNonUvStore::test_with_env_var 2024-08-07T19:22:00.4736790Z Running 1 items in this shard: test/distributed/test_store.py::InitPgWithNonUvStore::test_with_url_param 2024-08-07T19:22:00.4737592Z Running 1 items in this shard: test/distributed/test_store.py::TestClientProtocol::test_client_connect 2024-08-07T19:22:00.4738066Z 2024-08-07T19:22:00.4738302Z Running distributed/test_c10d_spawn_ucc 1/1 ... [2024-08-07 19:22:00.461355] 2024-08-07T19:22:00.4739598Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_spawn_ucc.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:22:00.461922] 2024-08-07T19:22:27.9332286Z 2024-08-07T19:22:27.9333375Z distributed/test_c10d_spawn_ucc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_spawn_ucc_1.1_acd3d13d6c908e53_.log 2024-08-07T19:22:27.9338258Z Running 9 items in this shard: test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_allgather_ucc, test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_allreduce_ucc, test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_broadcast_ucc, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_gather, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_to_all, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_to_all_single, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_allreduce, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_broadcast, test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_reduce 2024-08-07T19:22:27.9342882Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_allgather_ucc 2024-08-07T19:22:27.9343891Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_allreduce_ucc 2024-08-07T19:22:27.9344909Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::ProcessGroupShareTensorTest::test_shared_broadcast_ucc 2024-08-07T19:22:27.9345883Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_gather 2024-08-07T19:22:27.9346827Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_to_all 2024-08-07T19:22:27.9347791Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_all_to_all_single 2024-08-07T19:22:27.9348762Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_allreduce 2024-08-07T19:22:27.9349689Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_broadcast 2024-08-07T19:22:27.9350586Z Running 1 items in this shard: test/distributed/test_c10d_spawn_ucc.py::TestDistributedNNFunctionsUcc::test_reduce 2024-08-07T19:22:27.9351105Z 2024-08-07T19:22:27.9351363Z Running distributed/test_c10d_spawn_nccl 1/1 ... [2024-08-07 19:22:27.933485] 2024-08-07T19:22:27.9352674Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_spawn_nccl.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:22:27.933922] 2024-08-07T19:23:06.6344303Z 2024-08-07T19:23:06.6346532Z distributed/test_c10d_spawn_nccl 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_spawn_nccl_1.1_bab5ea31e545e423_.log 2024-08-07T19:23:06.6353175Z Running 13 items in this shard: test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_allgather_nccl, test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_allreduce_nccl, test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_broadcast_nccl, test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_reduce_nccl, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather_base, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all_single, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_allreduce, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_broadcast, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter_non_contiguous 2024-08-07T19:23:06.6360190Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_allgather_nccl 2024-08-07T19:23:06.6361351Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_allreduce_nccl 2024-08-07T19:23:06.6362452Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_broadcast_nccl 2024-08-07T19:23:06.6363810Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::ProcessGroupShareTensorTest::test_shared_reduce_nccl 2024-08-07T19:23:06.6365040Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather 2024-08-07T19:23:06.6366090Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather_base 2024-08-07T19:23:06.6367124Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all 2024-08-07T19:23:06.6368221Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all_single 2024-08-07T19:23:06.6369262Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_allreduce 2024-08-07T19:23:06.6370320Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_broadcast 2024-08-07T19:23:06.6371306Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce 2024-08-07T19:23:06.6372325Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter 2024-08-07T19:23:06.6373506Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter_non_contiguous 2024-08-07T19:23:06.6374191Z 2024-08-07T19:23:06.6374427Z Running distributed/test_c10d_nccl 1/1 ... [2024-08-07 19:23:06.635175] 2024-08-07T19:23:06.6375772Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_nccl.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:23:06.635637] 2024-08-07T19:23:08.0366377Z 2024-08-07T19:23:08.0367590Z distributed/test_c10d_nccl 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_nccl_1.1_353930d0561f4846_.log 2024-08-07T19:23:08.0368332Z 2024-08-07T19:23:08.0372804Z Running distributed/_shard/sharded_tensor/test_sharded_tensor 1/1 ... [2024-08-07 19:23:08.037107] 2024-08-07T19:23:08.0378080Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/_shard/sharded_tensor/test_sharded_tensor.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:23:08.037589] 2024-08-07T19:23:16.6226786Z 2024-08-07T19:23:16.6228050Z distributed/_shard/sharded_tensor/test_sharded_tensor 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_1.1_6a8da622b7bb0a90_.log 2024-08-07T19:23:16.6265708Z Running 69 items in this shard: test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorMetadata::test_serialize_and_deserialize, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestCreateTensorFromParams::test_empty, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardParameter::test_shard_parameter, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardParameter::test_shard_parameter_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardTensor::test_shard_tensor, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardTensor::test_shard_tensor_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardTensor::test_shard_tensor_with_empty_shard, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestModuleHookApi::test_collect_local_shard, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestModuleHookApi::test_reshard_output, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestLocalTensor::test_local_tensor, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestLocalTensor::test_local_tensor_error, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_cleanup, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_complete_world_size, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_like, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_full, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_ones, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_rand, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_zeros, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_gather_even, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_gather_uneven, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_insufficient_sharding_dims, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_invalid_pg_rpc_ranks, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_invalid_sharding, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_load_state_dict_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_multiple_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_partial_world_size, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_sharded_tensor_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_sharded_tensor_sizes, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_sharding_columns, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_state_dict, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_state_dict_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_state_dict_no_sharded_tensors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_create_sharded_tensor_with_ones, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_gather_even, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_gather_uneven, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_grid_sharding, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_multiple_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_partial_world_size, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_device, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_to_cpu, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_to_cuda, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_to_test, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_uneven_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_with_rpc_names, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalTensor::test_init_from_local_tensor, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalTensor::test_init_from_local_tensor_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_and_global_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_and_global_metadata_invalid_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_pin_memory, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_property_cross_ranks, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_shards_gaps, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_shards_overlap, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_st_base_init_from_local_shards_and_global_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorCustomOps::test_custom_op, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorCustomOps::test_custom_op_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorCustomOps::test_custom_op_override, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardMetadata::test_create_shard_with_no_placement, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardMetadata::test_shard_metadata_init, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorSubGroupInit::test_sub_process_group_placement_validation, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorSubGroupInit::test_sub_process_group_sharded_tensor_init, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestCreateTensorNoProcessGroupMode::test_init_from_local_shards_and_global_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestCreateTensorNoProcessGroupMode::test_non_contiguous_local_shards 2024-08-07T19:23:16.6301387Z 2024-08-07T19:23:16.6301629Z Running distributed/test_launcher 1/1 ... [2024-08-07 19:23:16.623267] 2024-08-07T19:23:16.6302844Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_launcher.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:23:16.623710] 2024-08-07T19:23:20.4988292Z 2024-08-07T19:23:20.4989219Z distributed/test_launcher 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_launcher_1.1_59596ed3b15e7495_.log 2024-08-07T19:23:20.4990368Z Running 1 items in this shard: test/distributed/test_launcher.py::TestDistributedLaunch::test_launch_user_script 2024-08-07T19:23:20.4990880Z 2024-08-07T19:23:20.4995139Z Running distributed/test_c10d_ops_nccl 1/1 ... [2024-08-07 19:23:20.499334] 2024-08-07T19:23:20.5001177Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/test_c10d_ops_nccl.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:23:20.499797] 2024-08-07T19:23:21.8702745Z 2024-08-07T19:23:21.8704369Z distributed/test_c10d_ops_nccl 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_ops_nccl_1.1_ce98a0c8fa7ee0c5_.log 2024-08-07T19:23:21.8705782Z 2024-08-07T19:23:21.8710838Z Running distributed/elastic/events/lib_test 1/1 ... [2024-08-07 19:23:21.870876] 2024-08-07T19:23:21.8715757Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/events/lib_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:23:21.871343] 2024-08-07T19:23:24.7452425Z 2024-08-07T19:23:24.7453493Z distributed/elastic/events/lib_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.events.lib_test_1.1_508233a75c193c46_.log 2024-08-07T19:23:24.7457531Z Running 8 items in this shard: test/distributed/elastic/events/lib_test.py::EventLibTest::test_event_created, test/distributed/elastic/events/lib_test.py::EventLibTest::test_event_deser, test/distributed/elastic/events/lib_test.py::EventLibTest::test_get_or_create_logger, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_construct_and_record_rdzv_event, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_construct_and_record_rdzv_event_does_not_run_if_invalid_dest, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_rdzv_event_created, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_rdzv_event_deserialize, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_rdzv_event_str 2024-08-07T19:23:24.7460802Z 2024-08-07T19:23:24.7461085Z Running distributed/elastic/metrics/api_test 1/1 ... [2024-08-07 19:23:24.745637] 2024-08-07T19:23:24.7462960Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/metrics/api_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:23:24.746090] 2024-08-07T19:23:27.5197510Z 2024-08-07T19:23:27.5198858Z distributed/elastic/metrics/api_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.metrics.api_test_1.1_7cd4a2b37b4bca4e_.log 2024-08-07T19:23:27.5200999Z Running 3 items in this shard: test/distributed/elastic/metrics/api_test.py::MetricsApiTest::test_get_metric_name, test/distributed/elastic/metrics/api_test.py::MetricsApiTest::test_inheritance, test/distributed/elastic/metrics/api_test.py::MetricsApiTest::test_profile 2024-08-07T19:23:27.5202343Z 2024-08-07T19:23:27.5204957Z Running distributed/elastic/multiprocessing/api_test 1/1 ... [2024-08-07 19:23:27.520304] 2024-08-07T19:23:27.5210179Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/multiprocessing/api_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:23:27.520778] 2024-08-07T19:23:57.9870219Z 2024-08-07T19:23:57.9872110Z distributed/elastic/multiprocessing/api_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.multiprocessing.api_test_1.1_6485352c97c63668_.log 2024-08-07T19:23:57.9905515Z Running 40 items in this shard: test/distributed/elastic/multiprocessing/api_test.py::RunProcResultsTest::test_get_failures, test/distributed/elastic/multiprocessing/api_test.py::RunProcResultsTest::test_is_failed, test/distributed/elastic/multiprocessing/api_test.py::StdTest::test_from_str_bad_input, test/distributed/elastic/multiprocessing/api_test.py::StdTest::test_from_value, test/distributed/elastic/multiprocessing/api_test.py::StdTest::test_from_value_map, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_args_env_len_mismatch, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_binary_exit, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_binary_incorrect_entrypoint, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_binary_raises, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_function_large_ret_val, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_function_raise, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_function_with_tensor, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_invalid_log_dir, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_multiprocess_context_close, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_multiprocessing_context_poll_raises_exception, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_pcontext_wait, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_subprocess_context_close, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_to_map, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_validate_full_rank, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_void_function, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesTest::test_wait_for_all_child_procs_to_exit, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_args_env_len_mismatch, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_binary, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_binary_exit, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_binary_incorrect_entrypoint, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_binary_raises, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_binary_redirect_and_tee, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_function, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_function_large_ret_val, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_function_raise, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_function_with_tensor, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_invalid_log_dir, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_multiprocess_context_close, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_multiprocessing_context_poll_raises_exception, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_pcontext_wait, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_subprocess_context_close, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_to_map, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_validate_full_rank, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_void_function, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListTest::test_wait_for_all_child_procs_to_exit 2024-08-07T19:23:57.9933105Z 2024-08-07T19:23:57.9933407Z Running distributed/elastic/timer/api_test 1/1 ... [2024-08-07 19:23:57.987982] 2024-08-07T19:23:57.9934853Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/timer/api_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:23:57.988542] 2024-08-07T19:23:59.3634201Z 2024-08-07T19:23:59.3635274Z distributed/elastic/timer/api_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.timer.api_test_1.1_354d8207446e8e82_.log 2024-08-07T19:23:59.3636080Z 2024-08-07T19:23:59.3640660Z Running distributed/elastic/timer/local_timer_example 1/1 ... [2024-08-07 19:23:59.363881] 2024-08-07T19:23:59.3645981Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/timer/local_timer_example.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:23:59.364368] 2024-08-07T19:24:16.1145855Z 2024-08-07T19:24:16.1147179Z distributed/elastic/timer/local_timer_example 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.timer.local_timer_example_1.1_bfdbafa1fbffceec_.log 2024-08-07T19:24:16.1149235Z Running 2 items in this shard: test/distributed/elastic/timer/local_timer_example.py::LocalTimerExample::test_example_start_method_spawn, test/distributed/elastic/timer/local_timer_example.py::LocalTimerExample::test_torch_mp_example 2024-08-07T19:24:16.1150392Z 2024-08-07T19:24:16.1152100Z Running distributed/elastic/timer/local_timer_test 1/1 ... [2024-08-07 19:24:16.115027] 2024-08-07T19:24:16.1156911Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/timer/local_timer_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:24:16.115471] 2024-08-07T19:24:23.1948028Z 2024-08-07T19:24:23.1949947Z distributed/elastic/timer/local_timer_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.timer.local_timer_test_1.1_18df99314526bda1_.log 2024-08-07T19:24:23.1962443Z Running 14 items in this shard: test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_client_interaction, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_exception_propagation, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_get_timer_recursive, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_happy_path, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_no_client, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_timer, test/distributed/elastic/timer/local_timer_test.py::MultiprocessingRequestQueueTest::test_get, test/distributed/elastic/timer/local_timer_test.py::MultiprocessingRequestQueueTest::test_get_less_than_size, test/distributed/elastic/timer/local_timer_test.py::MultiprocessingRequestQueueTest::test_get_size, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_acquire_release, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_expired_timers, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_valid_timers, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_watchdog_call_count, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_watchdog_empty_queue 2024-08-07T19:24:23.1973827Z 2024-08-07T19:24:23.1974365Z Running distributed/elastic/utils/distributed_test 1/1 ... [2024-08-07 19:24:23.195286] 2024-08-07T19:24:23.1976592Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/utils/distributed_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:24:23.195760] 2024-08-07T19:24:33.4790302Z 2024-08-07T19:24:33.4791841Z distributed/elastic/utils/distributed_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.utils.distributed_test_1.1_4e94509edd919cc8_.log 2024-08-07T19:24:33.4797139Z Running 8 items in this shard: test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_multi, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_no_port_multi, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_single_server, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_timeout_on_server, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_timeout_on_worker, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_with_libuv_support, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_port_already_in_use_on_server, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_port_already_in_use_on_worker 2024-08-07T19:24:33.4801689Z 2024-08-07T19:24:33.4802070Z Running distributed/elastic/utils/logging_test 1/1 ... [2024-08-07 19:24:33.479684] 2024-08-07T19:24:33.4803887Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/utils/logging_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:24:33.480130] 2024-08-07T19:24:36.3036562Z 2024-08-07T19:24:36.3037707Z distributed/elastic/utils/logging_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.utils.logging_test_1.1_791637ad493fbd81_.log 2024-08-07T19:24:36.3039381Z Running 2 items in this shard: test/distributed/elastic/utils/logging_test.py::LoggingTest::test_derive_module_name, test/distributed/elastic/utils/logging_test.py::LoggingTest::test_logger_name 2024-08-07T19:24:36.3040473Z 2024-08-07T19:24:36.3042793Z Running distributed/elastic/utils/util_test 1/1 ... [2024-08-07 19:24:36.304115] 2024-08-07T19:24:36.3047727Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/elastic/utils/util_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:24:36.304551] 2024-08-07T19:24:39.1281063Z 2024-08-07T19:24:39.1282442Z distributed/elastic/utils/util_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.utils.util_test_1.1_7f784de2e35773ac_.log 2024-08-07T19:24:39.1287091Z Running 10 items in this shard: test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_barrier, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_barrier_hash_store, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_get_all_rank_0, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_get_all_rank_n, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_synchronize, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_synchronize_hash_store, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger_custom_name, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger_different, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger_none 2024-08-07T19:24:39.1291203Z 2024-08-07T19:24:39.1291721Z Running distributed/pipelining/test_composability 1/1 ... [2024-08-07 19:24:39.128570] 2024-08-07T19:24:39.1293292Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/pipelining/test_composability.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:24:39.129007] 2024-08-07T19:24:41.8255997Z 2024-08-07T19:24:41.8257443Z distributed/pipelining/test_composability 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_composability_1.1_aaca4c3bcb2d8d7d_.log 2024-08-07T19:24:41.8258476Z 2024-08-07T19:24:41.8261993Z Running distributed/tensor/parallel/test_ddp_2d_parallel 1/1 ... [2024-08-07 19:24:41.826028] 2024-08-07T19:24:41.8267344Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_ddp_2d_parallel.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:24:41.826486] 2024-08-07T19:24:47.5057112Z 2024-08-07T19:24:47.5058331Z distributed/tensor/parallel/test_ddp_2d_parallel 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_ddp_2d_parallel_1.1_c5cf0db5ffa1de43_.log 2024-08-07T19:24:47.5059925Z Running 1 items in this shard: test/distributed/tensor/parallel/test_ddp_2d_parallel.py::Test2dParallelIntegration::test_2d_ddp_integration_functionality 2024-08-07T19:24:47.5060648Z 2024-08-07T19:24:47.5063418Z Running distributed/tensor/parallel/test_fsdp_2d_parallel 1/1 ... [2024-08-07 19:24:47.506138] 2024-08-07T19:24:47.5068364Z Executing ['/opt/conda/envs/py_3.8/bin/python', '-bb', 'distributed/tensor/parallel/test_fsdp_2d_parallel.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2024-08-07 19:24:47.506597] 2024-08-07T19:25:23.6981141Z 2024-08-07T19:25:23.6982330Z distributed/tensor/parallel/test_fsdp_2d_parallel 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_fsdp_2d_parallel_1.1_9f3565ab995c86f6_.log 2024-08-07T19:25:23.6990714Z Running 12 items in this shard: test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelTraining::test_2d_e2e_training_default, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelTraining::test_2d_e2e_training_not_use_orig_params, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelTraining::test_2d_e2e_training_use_orig_params, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelTraining::test_2d_fsdp_state_enable_extension, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelTraining::test_raise_invalid_tp_composition, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelStateDict::test_2d_load_state_dict_is_even_sharded_model_False, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelStateDict::test_2d_load_state_dict_is_even_sharded_model_True, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelStateDict::test_2d_optim_state_dict_is_even_sharded_model_False, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelStateDict::test_2d_optim_state_dict_is_even_sharded_model_True, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelStateDict::test_2d_state_dict_is_even_sharded_model_False, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelStateDict::test_2d_state_dict_is_even_sharded_model_True, test/distributed/tensor/parallel/test_fsdp_2d_parallel.py::TestNew2dParallelStateDict::test_fsdp_2d_extension 2024-08-07T19:25:23.6997707Z 2024-08-07T19:25:24.5621039Z 2024-08-07T19:25:24.5621527Z real 107m7.637s 2024-08-07T19:25:24.5621821Z user 222m23.518s 2024-08-07T19:25:24.5622052Z sys 45m3.376s 2024-08-07T19:25:24.5622290Z + assert_git_not_dirty 2024-08-07T19:25:24.5622590Z + [[ linux-jammy-py3.8-gcc11 != *rocm* ]] 2024-08-07T19:25:24.5622949Z + [[ linux-jammy-py3.8-gcc11 != *xla* ]] 2024-08-07T19:25:24.5628235Z ++ git status --porcelain 2024-08-07T19:25:24.5629617Z ++ grep -v '?? third_party' 2024-08-07T19:25:37.1213072Z ++ true 2024-08-07T19:25:37.1214110Z + git_status= 2024-08-07T19:25:37.1214471Z + [[ -n '' ]] 2024-08-07T19:25:37.1215426Z + [[ linux-jammy-py3.8-gcc11 == *cuda* ]] 2024-08-07T19:25:37.1215823Z + [[ linux-jammy-py3.8-gcc11 == *rocm* ]] 2024-08-07T19:25:37.1216302Z + [[ 1 == 1 ]] 2024-08-07T19:25:37.1216539Z + test_rpc 2024-08-07T19:25:37.1216783Z + echo 'Testing RPC C++ tests' 2024-08-07T19:25:37.1217069Z Testing RPC C++ tests 2024-08-07T19:25:37.1219201Z + ln -sf /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libtorch.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libtorch_global_deps.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libtorch_python.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libtorchbind_test.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-08-07T19:25:37.1248466Z + ln -sf /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/lib/libc10.so /opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-08-07T19:25:37.1257343Z + CPP_TESTS_DIR=/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin 2024-08-07T19:25:37.1257977Z + python test/run_test.py --cpp --verbose -i cpp/test_cpp_rpc 2024-08-07T19:25:37.2206699Z test/run_test.py:21: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html 2024-08-07T19:25:37.2207428Z import pkg_resources 2024-08-07T19:25:40.2651841Z Downloading https://ossci-metrics.s3.amazonaws.com/slow-tests.json to /var/lib/jenkins/workspace/test/.pytorch-slow-tests.json 2024-08-07T19:25:40.2654299Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json 2024-08-07T19:25:40.2760107Z Found test times from artifacts 2024-08-07T19:25:40.3179855Z Found test times from artifacts 2024-08-07T19:25:40.3194309Z Running all tests 2024-08-07T19:25:40.3197678Z Running parallel tests on 3 processes 2024-08-07T19:25:40.3198291Z Name: tests to run (est. time: 0.0min) 2024-08-07T19:25:40.3198817Z Serial tests (1): 2024-08-07T19:25:40.3199249Z cpp/test_cpp_rpc 1/1 2024-08-07T19:25:40.3199668Z Parallel tests (0): 2024-08-07T19:25:40.3200087Z Name: excluded (est. time: 0.0min) 2024-08-07T19:25:40.3200526Z Serial tests (0): 2024-08-07T19:25:40.3200880Z Parallel tests (0): 2024-08-07T19:25:40.3201470Z Starting test batch 'tests to run' 0.0 seconds after initiating testing 2024-08-07T19:25:40.3396212Z Running cpp/test_cpp_rpc 1/1 ... [2024-08-07 19:25:40.339252] 2024-08-07T19:25:40.3413075Z Executing ['pytest', '/opt/conda/envs/py_3.8/lib/python3.8/site-packages/torch/bin/test_cpp_rpc', '-v', '-vv', '-rfEX', '-n', '3', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-f35993719ca94866.xml', '-x', '--reruns=2'] ... [2024-08-07 19:25:40.340881] 2024-08-07T19:25:45.2224464Z 2024-08-07T19:25:45.2225564Z cpp/test_cpp_rpc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.test_cpp_rpc_1.1_2eaa93faec77983f_.log 2024-08-07T19:25:45.2226309Z 2024-08-07T19:25:46.0492623Z + cleanup_workspace 2024-08-07T19:25:46.0493293Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.' 2024-08-07T19:25:46.0494128Z sudo may print the following warning message that can be ignored. The chown command will still run. 2024-08-07T19:25:46.0494774Z + echo ' sudo: setrlimit(RLIMIT_STACK): Operation not permitted' 2024-08-07T19:25:46.0495244Z sudo: setrlimit(RLIMIT_STACK): Operation not permitted 2024-08-07T19:25:46.0495802Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42' 2024-08-07T19:25:46.0496414Z For more details refer to https://github.com/sudo-project/sudo/issues/42 2024-08-07T19:25:46.0496889Z + sudo chown -R 1000 /var/lib/jenkins/workspace 2024-08-07T19:25:46.6545533Z ##[group]Run cat test/**/*_toprint.log || true 2024-08-07T19:25:46.6545930Z cat test/**/*_toprint.log || true 2024-08-07T19:25:46.6565828Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:25:46.6566234Z env: 2024-08-07T19:25:46.6566460Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:25:46.6567104Z DOCKER_CONTAINER_ID: 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:25:46.6567618Z ##[endgroup] 2024-08-07T19:25:46.6640762Z cat: 'test/**/*_toprint.log': No such file or directory 2024-08-07T19:25:46.6680878Z ##[group]Run kill "$MONITOR_SCRIPT_PID" 2024-08-07T19:25:46.6681258Z kill "$MONITOR_SCRIPT_PID" 2024-08-07T19:25:46.6686479Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:25:46.6686876Z env: 2024-08-07T19:25:46.6687104Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:25:46.6687559Z DOCKER_CONTAINER_ID: 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:25:46.6688083Z MONITOR_SCRIPT_PID: 268780 2024-08-07T19:25:46.6688371Z ##[endgroup] 2024-08-07T19:25:46.6834987Z Prepare all required actions 2024-08-07T19:25:46.6835404Z Getting action download info 2024-08-07T19:25:46.8835563Z Download action repository 'actions/upload-artifact@v3' (SHA:a8a3f3ad30e3422c9c7b888a15615d19a852ae32) 2024-08-07T19:25:47.0724803Z ##[group]Run ./.github/actions/upload-test-artifacts 2024-08-07T19:25:47.0725176Z with: 2024-08-07T19:25:47.0725525Z file-suffix: test-distributed-1-2-amz2023.linux.2xlarge_28475634164 2024-08-07T19:25:47.0725974Z s3-bucket: gha-artifacts 2024-08-07T19:25:47.0726248Z env: 2024-08-07T19:25:47.0726456Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:25:47.0726932Z DOCKER_CONTAINER_ID: 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:25:47.0727433Z ##[endgroup] 2024-08-07T19:25:47.0760918Z ##[group]Run # Remove any previous test jsons if they exist 2024-08-07T19:25:47.0761399Z # Remove any previous test jsons if they exist 2024-08-07T19:25:47.0761796Z rm -f test-jsons-*.zip 2024-08-07T19:25:47.0762230Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test -i '*.json' 2024-08-07T19:25:47.0767764Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:25:47.0768157Z env: 2024-08-07T19:25:47.0768388Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:25:47.0768863Z DOCKER_CONTAINER_ID: 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:25:47.0769495Z FILE_SUFFIX: test-distributed-1-2-amz2023.linux.2xlarge_28475634164 2024-08-07T19:25:47.0769929Z ##[endgroup] 2024-08-07T19:25:47.1012375Z adding: test/allowlist_for_publicAPI.json (deflated 79%) 2024-08-07T19:25:47.1039787Z adding: test/benchmark_utils/callgrind_artifacts.json (deflated 92%) 2024-08-07T19:25:47.1040519Z adding: test/minioptest_failures_dict.json (deflated 70%) 2024-08-07T19:25:47.1046752Z adding: test/profiler/profiler_utils_mock_events.json (deflated 87%) 2024-08-07T19:25:47.1048410Z adding: test/test-reports/td_exclusions-23c2ea13fe8fed2dacea.json (deflated 86%) 2024-08-07T19:25:47.1049638Z adding: test/test-reports/td_exclusions-72abd2b7c66722ffaef9.json (deflated 18%) 2024-08-07T19:25:47.1053774Z adding: test/.pytorch-slow-tests.json (deflated 81%) 2024-08-07T19:25:47.1066220Z adding: test/.pytorch-disabled-tests.json (deflated 89%) 2024-08-07T19:25:47.1096129Z ##[group]Run # Remove any previous test reports if they exist 2024-08-07T19:25:47.1096605Z # Remove any previous test reports if they exist 2024-08-07T19:25:47.1097065Z rm -f test-reports-*.zip 2024-08-07T19:25:47.1097514Z zip -r "test-reports-${FILE_SUFFIX}.zip" test -i '*.xml' -i '*.csv' 2024-08-07T19:25:47.1103015Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:25:47.1103411Z env: 2024-08-07T19:25:47.1103642Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:25:47.1104096Z DOCKER_CONTAINER_ID: 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:25:47.1104729Z FILE_SUFFIX: test-distributed-1-2-amz2023.linux.2xlarge_28475634164 2024-08-07T19:25:47.1105172Z ##[endgroup] 2024-08-07T19:25:47.1372880Z adding: test/test-reports/python-pytest/distributed._tensor.test_dtensor_compile/distributed._tensor.test_dtensor_compile-3578dc51304b484a.xml (deflated 84%) 2024-08-07T19:25:47.1374475Z adding: test/test-reports/python-pytest/distributed.pipelining.test_backward/distributed.pipelining.test_backward-126051168fbc6fb6.xml (deflated 37%) 2024-08-07T19:25:47.1376091Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_nested_dict/distributed.checkpoint.test_nested_dict-351545d1d9735668.xml (deflated 51%) 2024-08-07T19:25:47.1377648Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.test_logger/distributed._shard.sharded_tensor.test_logger-3a7ed58cb13ca7e5.xml (deflated 37%) 2024-08-07T19:25:47.1379195Z adding: test/test-reports/python-pytest/distributed.pipelining.test_transformer/distributed.pipelining.test_transformer-560ee5a1a053df81.xml (deflated 39%) 2024-08-07T19:25:47.1380797Z adding: test/test-reports/python-pytest/distributed.test_symmetric_memory/distributed.test_symmetric_memory-570e062d53c50483.xml (deflated 93%) 2024-08-07T19:25:47.1382332Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_micro_pipeline_tp/distributed.tensor.parallel.test_micro_pipeline_tp-40763b7b61e5fb51.xml (deflated 95%) 2024-08-07T19:25:47.1383864Z adding: test/test-reports/python-pytest/distributed.test_dynamo_distributed/distributed.test_dynamo_distributed-4d275e3fad078bf3.xml (deflated 92%) 2024-08-07T19:25:47.1385253Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_core/distributed.fsdp.test_fsdp_core-a37d7cd1fccd1507.xml (deflated 95%) 2024-08-07T19:25:47.1386658Z adding: test/test-reports/python-pytest/distributed._composable.test_contract/distributed._composable.test_contract-30d874ded02a8454.xml (deflated 69%) 2024-08-07T19:25:47.1387902Z adding: test/test-reports/python-pytest/distributed.test_device_mesh/distributed.test_device_mesh-e7764be3f3e6c99d.xml (deflated 85%) 2024-08-07T19:25:47.1389250Z adding: test/test-reports/python-pytest/distributed._tensor.test_tensor_ops/distributed._tensor.test_tensor_ops-e77a4e190e234214.xml (deflated 85%) 2024-08-07T19:25:47.1390577Z adding: test/test-reports/python-pytest/distributed._tensor.test_random_ops/distributed._tensor.test_random_ops-905f5c22ac5e4e32.xml (deflated 82%) 2024-08-07T19:25:47.1392171Z adding: test/test-reports/python-pytest/distributed._tensor.experimental.test_register_sharding/distributed._tensor.experimental.test_register_sharding-2b6c099412bf99bc.xml (deflated 52%) 2024-08-07T19:25:47.1393849Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_tp_examples/distributed.tensor.parallel.test_tp_examples-37281068fe16c3e0.xml (deflated 83%) 2024-08-07T19:25:47.1395320Z adding: test/test-reports/python-pytest/distributed._tensor.test_math_ops/distributed._tensor.test_math_ops-0d1d0d2b9bcfe242.xml (deflated 79%) 2024-08-07T19:25:47.1396641Z adding: test/test-reports/python-pytest/distributed._tensor.test_redistribute/distributed._tensor.test_redistribute-e57e6f56c6269431.xml (deflated 81%) 2024-08-07T19:25:47.1398058Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_format_utils/distributed.checkpoint.test_format_utils-2a4f9398aa8e17ee.xml (deflated 59%) 2024-08-07T19:25:47.1399674Z adding: test/test-reports/python-pytest/distributed._tensor.experimental.test_tp_transform/distributed._tensor.experimental.test_tp_transform-cc8f005bb24c6836.xml (deflated 62%) 2024-08-07T19:25:47.1401345Z adding: test/test-reports/python-pytest/distributed._tensor.experimental.test_local_map/distributed._tensor.experimental.test_local_map-2a45ceb35028c1b5.xml (deflated 67%) 2024-08-07T19:25:47.1402958Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_fsdp_optim_state/distributed.checkpoint.test_fsdp_optim_state-e64805958991ed6f.xml (deflated 65%) 2024-08-07T19:25:47.1404495Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_dtensor_resharding/distributed.checkpoint.test_dtensor_resharding-395c7b6148e219f4.xml (deflated 80%) 2024-08-07T19:25:47.1406104Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_tp_checkpoint/distributed.checkpoint.test_tp_checkpoint-83e04cc16b3e607e.xml (deflated 65%) 2024-08-07T19:25:47.1407733Z adding: test/test-reports/python-pytest/distributed._tensor.test_pointwise_ops/distributed._tensor.test_pointwise_ops-e788266b4cf32400.xml (deflated 70%) 2024-08-07T19:25:47.1409213Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_fsspec/distributed.checkpoint.test_fsspec-1d359d0931168075.xml (deflated 65%) 2024-08-07T19:25:47.1410723Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_optim_state/distributed.fsdp.test_fsdp_optim_state-fc0735030b12a680.xml (deflated 95%) 2024-08-07T19:25:47.1412487Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_fsdp_tp_checkpoint_conversion/distributed.checkpoint.test_fsdp_tp_checkpoint_conversion-30296511b5dd9972.xml (deflated 48%) 2024-08-07T19:25:47.1414014Z adding: test/test-reports/python-pytest/distributed.test_fake_pg/distributed.test_fake_pg-334367f3b3e63379.xml (deflated 81%) 2024-08-07T19:25:47.1415319Z adding: test/test-reports/python-pytest/distributed._tensor.debug.test_comm_mode/distributed._tensor.debug.test_comm_mode-19a49edb020091aa.xml (deflated 60%) 2024-08-07T19:25:47.1416774Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_utils/distributed.checkpoint.test_utils-732689123f405b7d.xml (deflated 71%) 2024-08-07T19:25:47.1418344Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_init/distributed._composable.fsdp.test_fully_shard_init-218dc7a22ba034ce.xml (deflated 92%) 2024-08-07T19:25:47.1419837Z adding: test/test-reports/python-pytest/distributed.test_c10d_logger/distributed.test_c10d_logger-00be65e21b44519d.xml (deflated 66%) 2024-08-07T19:25:47.1421177Z adding: test/test-reports/python-pytest/distributed.fsdp.test_shard_utils/distributed.fsdp.test_shard_utils-daba11c4f888d18d.xml (deflated 63%) 2024-08-07T19:25:47.1422460Z adding: test/test-reports/python-pytest/distributed.test_nccl/distributed.test_nccl-362c25ceaeb143d9.xml (deflated 28%) 2024-08-07T19:25:47.1423731Z adding: test/test-reports/python-pytest/distributed.rpc.test_share_memory/distributed.rpc.test_share_memory-88963b2fe892c5c5.xml (deflated 47%) 2024-08-07T19:25:47.1425230Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_fsdp_model_state/distributed.checkpoint.test_fsdp_model_state-22b4723db098fd76.xml (deflated 65%) 2024-08-07T19:25:47.1426905Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_parallelize_api/distributed.tensor.parallel.test_parallelize_api-55dad7d70332cf01.xml (deflated 81%) 2024-08-07T19:25:47.1428538Z adding: test/test-reports/python-pytest/distributed.test_control_collectives/distributed.test_control_collectives-830c3b4067b6089e.xml (deflated 84%) 2024-08-07T19:25:47.1430207Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_clip_grad_norm_/distributed._composable.fsdp.test_fully_shard_clip_grad_norm_-f35513997d483475.xml (deflated 53%) 2024-08-07T19:25:47.1431954Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_backward_prefetch/distributed.fsdp.test_fsdp_backward_prefetch-64a9d30809d472de.xml (deflated 47%) 2024-08-07T19:25:47.1433642Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_autograd/distributed._composable.fsdp.test_fully_shard_autograd-cf8399a11724d66e.xml (deflated 68%) 2024-08-07T19:25:47.1435441Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_training/distributed._composable.fsdp.test_fully_shard_training-2856a823ca59c6b5.xml (deflated 83%) 2024-08-07T19:25:47.1437141Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_freezing_weights/distributed.fsdp.test_fsdp_freezing_weights-1289e292d112b9d5.xml (deflated 96%) 2024-08-07T19:25:47.1438853Z adding: test/test-reports/python-pytest/distributed.optim.test_zero_redundancy_optimizer/distributed.optim.test_zero_redundancy_optimizer-2625e37078665f1d.xml (deflated 94%) 2024-08-07T19:25:47.1457723Z adding: test/test-reports/python-pytest/distributed.test_data_parallel/distributed.test_data_parallel-bb897cf011b645ce.xml (deflated 94%) 2024-08-07T19:25:47.1459278Z adding: test/test-reports/python-pytest/distributed.algorithms.test_join/distributed.algorithms.test_join-7ce43bbb4c4c7863.xml (deflated 79%) 2024-08-07T19:25:47.1460776Z adding: test/test-reports/python-pytest/distributed.fsdp.test_checkpoint_wrapper/distributed.fsdp.test_checkpoint_wrapper-bee77ecbfb98771a.xml (deflated 75%) 2024-08-07T19:25:47.1462378Z adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_comm/distributed._composable.fsdp.test_fully_shard_comm-076b209f87af1108.xml (deflated 84%) 2024-08-07T19:25:47.1464006Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_input/distributed.fsdp.test_fsdp_input-7b33648632e213da.xml (deflated 64%) 2024-08-07T19:25:47.1465390Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_uneven/distributed.fsdp.test_fsdp_uneven-4fd03c56f2da8274.xml (deflated 43%) 2024-08-07T19:25:47.1466907Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_multiple_wrapping/distributed.fsdp.test_fsdp_multiple_wrapping-9c820b19c1ed5e24.xml (deflated 47%) 2024-08-07T19:25:47.1468447Z adding: test/test-reports/python-pytest/distributed.fsdp.test_utils/distributed.fsdp.test_utils-54f2852191167d76.xml (deflated 70%) 2024-08-07T19:25:47.1469785Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_misc/distributed.fsdp.test_fsdp_misc-244b2c4caf324397.xml (deflated 91%) 2024-08-07T19:25:47.1471168Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_checkpoint/distributed.fsdp.test_fsdp_checkpoint-a807257ac23ed12e.xml (deflated 92%) 2024-08-07T19:25:47.1472644Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_exec_order/distributed.fsdp.test_fsdp_exec_order-254aa4361a24481c.xml (deflated 87%) 2024-08-07T19:25:47.1474055Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_fine_tune/distributed.fsdp.test_fsdp_fine_tune-22a30c48277f43c0.xml (deflated 64%) 2024-08-07T19:25:47.1475420Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-5a566e9d82f93001.xml (deflated 36%) 2024-08-07T19:25:47.1476679Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-1458e81397f226c2.xml (deflated 35%) 2024-08-07T19:25:47.1477878Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-dcae2d99ff323f87.xml (deflated 34%) 2024-08-07T19:25:47.1479267Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-990747c1d1dd9e2e.xml (deflated 35%) 2024-08-07T19:25:47.1480540Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-6374c9e817d8b1ef.xml (deflated 36%) 2024-08-07T19:25:47.1481819Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-9daf50a807ee0d37.xml (deflated 36%) 2024-08-07T19:25:47.1483102Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-64c60dd9f0bdd743.xml (deflated 35%) 2024-08-07T19:25:47.1484366Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-004c716f1f6d40c3.xml (deflated 34%) 2024-08-07T19:25:47.1485618Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-401fea2c0d9cd2e3.xml (deflated 35%) 2024-08-07T19:25:47.1486891Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-7f3ea48f8eff3380.xml (deflated 34%) 2024-08-07T19:25:47.1488175Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-4c12dedabc896ba4.xml (deflated 35%) 2024-08-07T19:25:47.1489459Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-7ca850ecc277b3c6.xml (deflated 34%) 2024-08-07T19:25:47.1490793Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-5f3541f62f7ce977.xml (deflated 34%) 2024-08-07T19:25:47.1492032Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-7314f0aad2ef1a66.xml (deflated 34%) 2024-08-07T19:25:47.1493425Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-f95de1809f1dadf6.xml (deflated 34%) 2024-08-07T19:25:47.1494691Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-e1f4291d240a6324.xml (deflated 35%) 2024-08-07T19:25:47.1496039Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-b4ba8c713e9e48fb.xml (deflated 36%) 2024-08-07T19:25:47.1497317Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-6957d04c4ba6a42c.xml (deflated 35%) 2024-08-07T19:25:47.1498552Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-675fea093fe5ae32.xml (deflated 36%) 2024-08-07T19:25:47.1499818Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-ca621d9499aab0c1.xml (deflated 35%) 2024-08-07T19:25:47.1501057Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-7adf23ecf111be92.xml (deflated 34%) 2024-08-07T19:25:47.1502341Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-1c8216dbf6771612.xml (deflated 33%) 2024-08-07T19:25:47.1503579Z adding: test/test-reports/python-pytest/distributed.test_c10d_common/distributed.test_c10d_common-42a7133cc3ae1140.xml (deflated 33%) 2024-08-07T19:25:47.1505062Z adding: test/test-reports/python-pytest/distributed._shard.sharding_plan.test_sharding_plan/distributed._shard.sharding_plan.test_sharding_plan-df9d1ef4e644702b.xml (deflated 75%) 2024-08-07T19:25:47.1506643Z adding: test/test-reports/python-pytest/distributed.checkpoint.test_state_dict/distributed.checkpoint.test_state_dict-9bcd9345c9e84880.xml (deflated 85%) 2024-08-07T19:25:47.1508062Z adding: test/test-reports/python-pytest/distributed._shard.test_sharder/distributed._shard.test_sharder-cae1e33a6948262a.xml (deflated 66%) 2024-08-07T19:25:47.1509405Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_meta/distributed.fsdp.test_fsdp_meta-5d7bb93453a179cb.xml (deflated 90%) 2024-08-07T19:25:47.1510770Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-1d49c6041a5506d0.xml (deflated 36%) 2024-08-07T19:25:47.1512122Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-121a29e37b211025.xml (deflated 36%) 2024-08-07T19:25:47.1513471Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-8632002aa899f799.xml (deflated 36%) 2024-08-07T19:25:47.1514819Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-8f3da7883db65ca1.xml (deflated 37%) 2024-08-07T19:25:47.1516202Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-0aee23dbaad0225c.xml (deflated 36%) 2024-08-07T19:25:47.1517553Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-62658f11023e3e3a.xml (deflated 37%) 2024-08-07T19:25:47.1518968Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-40863eb876cd93e6.xml (deflated 36%) 2024-08-07T19:25:47.1520333Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-818bd227c3eb6cec.xml (deflated 36%) 2024-08-07T19:25:47.1521707Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-19d326c41e471cf0.xml (deflated 36%) 2024-08-07T19:25:47.1523159Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-07a72abd474ce6de.xml (deflated 36%) 2024-08-07T19:25:47.1524524Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-5dd3bf9dcc840864.xml (deflated 36%) 2024-08-07T19:25:47.1525922Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-3993f012c1bea39f.xml (deflated 36%) 2024-08-07T19:25:47.1527280Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-72eceff6203ac4e8.xml (deflated 36%) 2024-08-07T19:25:47.1528732Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-3ece28c1143a7cb4.xml (deflated 36%) 2024-08-07T19:25:47.1530114Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-48d5f2ed01965239.xml (deflated 36%) 2024-08-07T19:25:47.1531513Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-ce8201b09ac497ce.xml (deflated 36%) 2024-08-07T19:25:47.1532901Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-8f8e8e8e223103c3.xml (deflated 36%) 2024-08-07T19:25:47.1534259Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-f9d79e069e12b4e5.xml (deflated 36%) 2024-08-07T19:25:47.1535609Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-64040a17ceeb373f.xml (deflated 36%) 2024-08-07T19:25:47.1536978Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-5e18bce7c6bb52b5.xml (deflated 35%) 2024-08-07T19:25:47.1538345Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-3840d252acbcf30f.xml (deflated 36%) 2024-08-07T19:25:47.1539715Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-c11ceae5b7824f5a.xml (deflated 35%) 2024-08-07T19:25:47.1541220Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-4da142d1fe72a1c7.xml (deflated 35%) 2024-08-07T19:25:47.1542492Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-7c15b122b425cd37.xml (deflated 35%) 2024-08-07T19:25:47.1543784Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-c0b0bb660817dcde.xml (deflated 35%) 2024-08-07T19:25:47.1545123Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-24d35c1dcba6065e.xml (deflated 36%) 2024-08-07T19:25:47.1546446Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-52a9bd39e1cb25a5.xml (deflated 35%) 2024-08-07T19:25:47.1547818Z adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-d6b47bc34a42bd80.xml (deflated 36%) 2024-08-07T19:25:47.1549209Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7316aef6d5e47fdc.xml (deflated 36%) 2024-08-07T19:25:47.1550641Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8fd063fa754f3333.xml (deflated 36%) 2024-08-07T19:25:47.1552075Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9b9346b9ae1fd075.xml (deflated 36%) 2024-08-07T19:25:47.1553571Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cb43fb81eeb44d2e.xml (deflated 36%) 2024-08-07T19:25:47.1555129Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5675ba0c0573882f.xml (deflated 36%) 2024-08-07T19:25:47.1556617Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d37e33ca689c82ed.xml (deflated 35%) 2024-08-07T19:25:47.1558077Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3edeee838bf308cf.xml (deflated 36%) 2024-08-07T19:25:47.1559620Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b93c74b1f45a71d5.xml (deflated 35%) 2024-08-07T19:25:47.1561282Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fd1cc27c18545237.xml (deflated 36%) 2024-08-07T19:25:47.1562709Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-544050d7640f2ab8.xml (deflated 36%) 2024-08-07T19:25:47.1564156Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-11c65a3e14147495.xml (deflated 35%) 2024-08-07T19:25:47.1565625Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b919cac08f63b9ea.xml (deflated 36%) 2024-08-07T19:25:47.1567063Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ba521839b2b1ee06.xml (deflated 36%) 2024-08-07T19:25:47.1568483Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-527577859f875926.xml (deflated 36%) 2024-08-07T19:25:47.1569925Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b873d9b0cd15ea55.xml (deflated 37%) 2024-08-07T19:25:47.1571468Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-bcfe9d2db87de01b.xml (deflated 35%) 2024-08-07T19:25:47.1573000Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e83220185f57b756.xml (deflated 35%) 2024-08-07T19:25:47.1574482Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e28ef1796764eea9.xml (deflated 36%) 2024-08-07T19:25:47.1575993Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fc2792ce91ad947b.xml (deflated 36%) 2024-08-07T19:25:47.1577457Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f6a61890bd311d5e.xml (deflated 36%) 2024-08-07T19:25:47.1578930Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e89b3dcaad839e38.xml (deflated 35%) 2024-08-07T19:25:47.1580444Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-123b394dcb857ded.xml (deflated 36%) 2024-08-07T19:25:47.1582073Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-271eeb2ceab21017.xml (deflated 35%) 2024-08-07T19:25:47.1583562Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4cf9d61f9830188a.xml (deflated 36%) 2024-08-07T19:25:47.1585029Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-bebddc4a21af8906.xml (deflated 35%) 2024-08-07T19:25:47.1586479Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1e8480b272e3b8c7.xml (deflated 36%) 2024-08-07T19:25:47.1587927Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3d8c003008da0cbd.xml (deflated 35%) 2024-08-07T19:25:47.1589458Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e70745e7834d68c6.xml (deflated 35%) 2024-08-07T19:25:47.1590895Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-225aa3fd65d8a959.xml (deflated 36%) 2024-08-07T19:25:47.1592385Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4d07ad6a0637d2b2.xml (deflated 36%) 2024-08-07T19:25:47.1593912Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d4e1824c0eae2b76.xml (deflated 36%) 2024-08-07T19:25:47.1595414Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-00daf27ea67fbf21.xml (deflated 35%) 2024-08-07T19:25:47.1596869Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-01cfbfd6faf9e22e.xml (deflated 35%) 2024-08-07T19:25:47.1598317Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9934e1adef18227d.xml (deflated 35%) 2024-08-07T19:25:47.1599810Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ec42494b3ddcd4b6.xml (deflated 35%) 2024-08-07T19:25:47.1601268Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d1d50a7add4998b4.xml (deflated 35%) 2024-08-07T19:25:47.1602729Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-de84fbd079cd464f.xml (deflated 36%) 2024-08-07T19:25:47.1604167Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f012284077daa223.xml (deflated 36%) 2024-08-07T19:25:47.1605594Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-94816b95d8bf2336.xml (deflated 36%) 2024-08-07T19:25:47.1607054Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-bc4b195b0ff17843.xml (deflated 36%) 2024-08-07T19:25:47.1608491Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0529d5dae4c04e9e.xml (deflated 36%) 2024-08-07T19:25:47.1609961Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5beda15ad7832f43.xml (deflated 36%) 2024-08-07T19:25:47.1611412Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4f32cdda9b25df9c.xml (deflated 37%) 2024-08-07T19:25:47.1612861Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c782452d3202067d.xml (deflated 36%) 2024-08-07T19:25:47.1614358Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-15aadcc9c637cf14.xml (deflated 36%) 2024-08-07T19:25:47.1615799Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2150a9e83ea01457.xml (deflated 36%) 2024-08-07T19:25:47.1617240Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d083204061969cb2.xml (deflated 36%) 2024-08-07T19:25:47.1618696Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f11c7635c70bebf4.xml (deflated 36%) 2024-08-07T19:25:47.1620114Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2d216127e248af10.xml (deflated 36%) 2024-08-07T19:25:47.1621692Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b899b1aeb7eb346d.xml (deflated 36%) 2024-08-07T19:25:47.1623137Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7b62452b23cc4e03.xml (deflated 36%) 2024-08-07T19:25:47.1624593Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8d8fac1a7edb03f4.xml (deflated 37%) 2024-08-07T19:25:47.1626056Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-88d235adcd36ee23.xml (deflated 36%) 2024-08-07T19:25:47.1627556Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5152a915de65c6a9.xml (deflated 35%) 2024-08-07T19:25:47.1629013Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-680b76a21a3cbd9d.xml (deflated 35%) 2024-08-07T19:25:47.1630474Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c79fc4b42dbbb6ae.xml (deflated 35%) 2024-08-07T19:25:47.1631926Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6d251d130538cee1.xml (deflated 36%) 2024-08-07T19:25:47.1633369Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5af0e42f95640caf.xml (deflated 37%) 2024-08-07T19:25:47.1634818Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e67c043446da026a.xml (deflated 36%) 2024-08-07T19:25:47.1636270Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b7c44d868a99f394.xml (deflated 36%) 2024-08-07T19:25:47.1637717Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4b6b4c62487ca54b.xml (deflated 36%) 2024-08-07T19:25:47.1639226Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-72da6ef7bc6fd55a.xml (deflated 36%) 2024-08-07T19:25:47.1641969Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-923bed5cfcd45cd0.xml (deflated 36%) 2024-08-07T19:25:47.1643408Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d1750560be6e02dd.xml (deflated 36%) 2024-08-07T19:25:47.1644874Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-542f933c70473d9d.xml (deflated 35%) 2024-08-07T19:25:47.1646332Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ecb01fa665561630.xml (deflated 35%) 2024-08-07T19:25:47.1647751Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-bca1f778ae88d69b.xml (deflated 36%) 2024-08-07T19:25:47.1649213Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-afbd699f8160a3fb.xml (deflated 35%) 2024-08-07T19:25:47.1650655Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-130ae7379f7188ba.xml (deflated 34%) 2024-08-07T19:25:47.1652117Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e4c2c62bfa149878.xml (deflated 35%) 2024-08-07T19:25:47.1653616Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9cc983980ccae47b.xml (deflated 34%) 2024-08-07T19:25:47.1655048Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2a1a5f9cfa8639c7.xml (deflated 35%) 2024-08-07T19:25:47.1656602Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2c4c88d4308f7226.xml (deflated 34%) 2024-08-07T19:25:47.1658045Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7e80ea39237a4d09.xml (deflated 36%) 2024-08-07T19:25:47.1659495Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f650773296779764.xml (deflated 34%) 2024-08-07T19:25:47.1661023Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d4fe05c038a53341.xml (deflated 35%) 2024-08-07T19:25:47.1662486Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-50b97f5eabbca49a.xml (deflated 35%) 2024-08-07T19:25:47.1663917Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0cfcf09b91e47d7b.xml (deflated 35%) 2024-08-07T19:25:47.1665366Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c64d76679a11391b.xml (deflated 36%) 2024-08-07T19:25:47.1666762Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d2e3137818602e71.xml (deflated 34%) 2024-08-07T19:25:47.1668213Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-93d6bbac001e805d.xml (deflated 35%) 2024-08-07T19:25:47.1669641Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-663378d254eeea9e.xml (deflated 36%) 2024-08-07T19:25:47.1671087Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-96cb0e704123d373.xml (deflated 35%) 2024-08-07T19:25:47.1672486Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e3404f3184ce641a.xml (deflated 35%) 2024-08-07T19:25:47.1673930Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0eb7314b9cd0942f.xml (deflated 35%) 2024-08-07T19:25:47.1675413Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-551de9b0dd951ae5.xml (deflated 34%) 2024-08-07T19:25:47.1676857Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d9646d2c7e348271.xml (deflated 35%) 2024-08-07T19:25:47.1678302Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-87971b65a12f5ebd.xml (deflated 35%) 2024-08-07T19:25:47.1679828Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-efee79ec0e97b3a1.xml (deflated 35%) 2024-08-07T19:25:47.1681335Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-96ec47cce602459d.xml (deflated 35%) 2024-08-07T19:25:47.1683001Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-075e17fe3055edc4.xml (deflated 35%) 2024-08-07T19:25:47.1684626Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1c6779f974e788cd.xml (deflated 35%) 2024-08-07T19:25:47.1686164Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8ffdd1fe87408694.xml (deflated 35%) 2024-08-07T19:25:47.1687597Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-afca5bc00162364e.xml (deflated 35%) 2024-08-07T19:25:47.1689138Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f6955395362d36e6.xml (deflated 36%) 2024-08-07T19:25:47.1690652Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a033ff871ec0654f.xml (deflated 36%) 2024-08-07T19:25:47.1692099Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0aa2daff5e0157a6.xml (deflated 34%) 2024-08-07T19:25:47.1693609Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a058104c28c3c8b5.xml (deflated 34%) 2024-08-07T19:25:47.1695011Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f27bf82fef461178.xml (deflated 35%) 2024-08-07T19:25:47.1696298Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-57dd0a9f8f842cb3.xml (deflated 35%) 2024-08-07T19:25:47.1697600Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-459d2fa8ea23c96d.xml (deflated 36%) 2024-08-07T19:25:47.1698885Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0624fffc2549bc82.xml (deflated 36%) 2024-08-07T19:25:47.1700248Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-72ad3deeb2a8cc2e.xml (deflated 55%) 2024-08-07T19:25:47.1702393Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c51643371b78d6c3.xml (deflated 55%) 2024-08-07T19:25:47.1704522Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3c61d65b160c1e24.xml (deflated 34%) 2024-08-07T19:25:47.1706485Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4ebeeb58e0d4d88e.xml (deflated 37%) 2024-08-07T19:25:47.1708554Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cfa07235a28af7fb.xml (deflated 37%) 2024-08-07T19:25:47.1710796Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9c70bdc90d238632.xml (deflated 37%) 2024-08-07T19:25:47.1712978Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-18f898bfe3b32038.xml (deflated 36%) 2024-08-07T19:25:47.1715154Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-17f352e37324c487.xml (deflated 37%) 2024-08-07T19:25:47.1717360Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e127d82467a9f908.xml (deflated 37%) 2024-08-07T19:25:47.1719620Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1ae41ebffccc77c5.xml (deflated 38%) 2024-08-07T19:25:47.1721865Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e430a99558d3a2c9.xml (deflated 37%) 2024-08-07T19:25:47.1724106Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2e107000b763c87a.xml (deflated 36%) 2024-08-07T19:25:47.1726402Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a2f1e5edf0bda939.xml (deflated 37%) 2024-08-07T19:25:47.1728700Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-59edd22aaf5a2f8f.xml (deflated 36%) 2024-08-07T19:25:47.1731238Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4a1addce0775dfa9.xml (deflated 37%) 2024-08-07T19:25:47.1733499Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7b5db5df528eee57.xml (deflated 37%) 2024-08-07T19:25:47.1735779Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c103ef75b703ad7d.xml (deflated 36%) 2024-08-07T19:25:47.1738054Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-72d9f482417b89ae.xml (deflated 36%) 2024-08-07T19:25:47.1740793Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ba717abdf8a27cbc.xml (deflated 37%) 2024-08-07T19:25:47.1743097Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-64e7b505b3f68637.xml (deflated 36%) 2024-08-07T19:25:47.1745354Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e730c0d9dbd747ad.xml (deflated 35%) 2024-08-07T19:25:47.1747621Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-be261dd72d7227ab.xml (deflated 36%) 2024-08-07T19:25:47.1749928Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4bd5786ff57eb11e.xml (deflated 36%) 2024-08-07T19:25:47.1752253Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f56be684ed6a047c.xml (deflated 36%) 2024-08-07T19:25:47.1754586Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fbe1ebba15d4d1f5.xml (deflated 36%) 2024-08-07T19:25:47.1756785Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-17cda021d2a613d9.xml (deflated 36%) 2024-08-07T19:25:47.1759106Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c5f56808f26c54b9.xml (deflated 36%) 2024-08-07T19:25:47.1761555Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8f6ff8f965b7d7fa.xml (deflated 35%) 2024-08-07T19:25:47.1763843Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-17dfde9471648e27.xml (deflated 36%) 2024-08-07T19:25:47.1766298Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3a6c68088bec71d7.xml (deflated 35%) 2024-08-07T19:25:47.1768748Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d35ba59d132d2ceb.xml (deflated 35%) 2024-08-07T19:25:47.1771219Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6fc03f2fb4d7ff76.xml (deflated 35%) 2024-08-07T19:25:47.1773584Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0fab0b8fd2aa25a8.xml (deflated 35%) 2024-08-07T19:25:47.1776063Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b59dca551806c0d5.xml (deflated 35%) 2024-08-07T19:25:47.1778410Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8bd6ff5be7ceb669.xml (deflated 35%) 2024-08-07T19:25:47.1780827Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3541f5cdfca49fab.xml (deflated 35%) 2024-08-07T19:25:47.1783011Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-72f52643aeb834ab.xml (deflated 35%) 2024-08-07T19:25:47.1785694Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-51dbaa262aea298f.xml (deflated 35%) 2024-08-07T19:25:47.1788063Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-38d2db9ab2bf33b0.xml (deflated 35%) 2024-08-07T19:25:47.1790483Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8a29af7bfb532954.xml (deflated 35%) 2024-08-07T19:25:47.1793027Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4f4a8bafd4265c52.xml (deflated 36%) 2024-08-07T19:25:47.1795459Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c843e9a50d7eef76.xml (deflated 35%) 2024-08-07T19:25:47.1797948Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-62acc61aea601ac0.xml (deflated 36%) 2024-08-07T19:25:47.1800486Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-99c05c0d6adb7fe8.xml (deflated 35%) 2024-08-07T19:25:47.1802869Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c5780a79f2734f4b.xml (deflated 36%) 2024-08-07T19:25:47.1805284Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-25aab4a57c46f856.xml (deflated 35%) 2024-08-07T19:25:47.1807727Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6eecebce53666c5a.xml (deflated 35%) 2024-08-07T19:25:47.1810196Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6901a63fe2cfb993.xml (deflated 35%) 2024-08-07T19:25:47.1812721Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4a09d0021d077f68.xml (deflated 35%) 2024-08-07T19:25:47.1815253Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-df6219a64c8d7fc6.xml (deflated 35%) 2024-08-07T19:25:47.1817692Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-195db8b5bc855bcd.xml (deflated 35%) 2024-08-07T19:25:47.1820147Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-11eb177506be4884.xml (deflated 35%) 2024-08-07T19:25:47.1822674Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-597d246a68386a89.xml (deflated 35%) 2024-08-07T19:25:47.1825196Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4dded4bdf482b380.xml (deflated 35%) 2024-08-07T19:25:47.1828178Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5859a247858d63c7.xml (deflated 35%) 2024-08-07T19:25:47.1830820Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-bbb2ce37f73d3022.xml (deflated 35%) 2024-08-07T19:25:47.1833342Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7fbaa71b94afe9e0.xml (deflated 35%) 2024-08-07T19:25:47.1836092Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8cbef3d820785945.xml (deflated 35%) 2024-08-07T19:25:47.1838746Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9284e4f53222f6b0.xml (deflated 36%) 2024-08-07T19:25:47.1841538Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-21f2485c70d84e7e.xml (deflated 35%) 2024-08-07T19:25:47.1844258Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a5ab47a4ad922872.xml (deflated 35%) 2024-08-07T19:25:47.1846800Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-85950292ed577c20.xml (deflated 36%) 2024-08-07T19:25:47.1849664Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2fd3caa0d98bffcd.xml (deflated 36%) 2024-08-07T19:25:47.1852036Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-14f3a98671b2fe58.xml (deflated 36%) 2024-08-07T19:25:47.1854504Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8402a2577696997b.xml (deflated 35%) 2024-08-07T19:25:47.1857063Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1857fdce86b9d539.xml (deflated 36%) 2024-08-07T19:25:47.1859695Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-66570e5782459973.xml (deflated 36%) 2024-08-07T19:25:47.1862332Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0e0b22d55f29e9e6.xml (deflated 36%) 2024-08-07T19:25:47.1864993Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-44e927026ea8264e.xml (deflated 35%) 2024-08-07T19:25:47.1867573Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d832181778e411bf.xml (deflated 36%) 2024-08-07T19:25:47.1870311Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-dcae36f5c59138ec.xml (deflated 35%) 2024-08-07T19:25:47.1872795Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4dc05fb3c47167f4.xml (deflated 35%) 2024-08-07T19:25:47.1875307Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-939408bac8f6b28e.xml (deflated 36%) 2024-08-07T19:25:47.1877966Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e85629a0a39b24d6.xml (deflated 35%) 2024-08-07T19:25:47.1880669Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a25be976089b0be7.xml (deflated 35%) 2024-08-07T19:25:47.1883221Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1fdf40c1b780e5a5.xml (deflated 37%) 2024-08-07T19:25:47.1885910Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5d40ba6b814eaf1f.xml (deflated 36%) 2024-08-07T19:25:47.1888507Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-52ab7fb2d665b8e9.xml (deflated 34%) 2024-08-07T19:25:47.1891088Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-128eb1fc59219c63.xml (deflated 34%) 2024-08-07T19:25:47.1893822Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-17cd6a0f5f337a94.xml (deflated 34%) 2024-08-07T19:25:47.1896421Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-46d9ca7b8d376041.xml (deflated 34%) 2024-08-07T19:25:47.1899106Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c02d1ede3688c4d6.xml (deflated 35%) 2024-08-07T19:25:47.1901980Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e9166338269410d3.xml (deflated 34%) 2024-08-07T19:25:47.1904599Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-11957cd72ad386e5.xml (deflated 35%) 2024-08-07T19:25:47.1907107Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-40984def90ad71f0.xml (deflated 34%) 2024-08-07T19:25:47.1909970Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-dcd2eba8cb5f7883.xml (deflated 34%) 2024-08-07T19:25:47.1912639Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-af835b0f71c2b794.xml (deflated 36%) 2024-08-07T19:25:47.1915365Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8e0eaf74d4a19631.xml (deflated 35%) 2024-08-07T19:25:47.1917940Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-32585d5a6b883ebd.xml (deflated 35%) 2024-08-07T19:25:47.1920602Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a701b0b742a44733.xml (deflated 35%) 2024-08-07T19:25:47.1923273Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0a1d072ddb6f4918.xml (deflated 35%) 2024-08-07T19:25:47.1925946Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f761696b4305f3e0.xml (deflated 35%) 2024-08-07T19:25:47.1928411Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c303c54a2d214c31.xml (deflated 36%) 2024-08-07T19:25:47.1931094Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ffcf313a00ed75b9.xml (deflated 34%) 2024-08-07T19:25:47.1933827Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b5b1dce9d8fcd3f9.xml (deflated 35%) 2024-08-07T19:25:47.1936515Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-46afd9dfb08c2783.xml (deflated 34%) 2024-08-07T19:25:47.1939137Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8c6c2af0a483a85c.xml (deflated 35%) 2024-08-07T19:25:47.1941986Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5e5f38a4d4aacfe7.xml (deflated 35%) 2024-08-07T19:25:47.1943418Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-265b86bd4b6e2e6a.xml (deflated 35%) 2024-08-07T19:25:47.1944910Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-27f18c35bfc278dd.xml (deflated 35%) 2024-08-07T19:25:47.1946267Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-53267f90cadef2c2.xml (deflated 34%) 2024-08-07T19:25:47.1947660Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9e2f17d657dc257c.xml (deflated 35%) 2024-08-07T19:25:47.1949102Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3124f6618e6ffd60.xml (deflated 34%) 2024-08-07T19:25:47.1950498Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c982ef03eebfa9e4.xml (deflated 36%) 2024-08-07T19:25:47.1951995Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fedc0019ea28ff4b.xml (deflated 36%) 2024-08-07T19:25:47.1953435Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b04b9d17706c07b1.xml (deflated 35%) 2024-08-07T19:25:47.1954837Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b9e6beda0c973410.xml (deflated 55%) 2024-08-07T19:25:47.1956307Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5bc8a2559e490bdb.xml (deflated 36%) 2024-08-07T19:25:47.1957715Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c22e908850db2165.xml (deflated 34%) 2024-08-07T19:25:47.1959192Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7e1a499044f1efc7.xml (deflated 36%) 2024-08-07T19:25:47.1960708Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-28896f3d8d01028e.xml (deflated 41%) 2024-08-07T19:25:47.1962157Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-098f4960c5db1bc7.xml (deflated 37%) 2024-08-07T19:25:47.1963521Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e0c7913190f0e7a2.xml (deflated 37%) 2024-08-07T19:25:47.1964901Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ccf34c97084eb445.xml (deflated 37%) 2024-08-07T19:25:47.1966340Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-437bd447ed9233df.xml (deflated 37%) 2024-08-07T19:25:47.1967948Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-73d330469a38c9d7.xml (deflated 37%) 2024-08-07T19:25:47.1969393Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ba174b88189d6115.xml (deflated 37%) 2024-08-07T19:25:47.1970750Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-18621fcf01d90f3b.xml (deflated 36%) 2024-08-07T19:25:47.1972157Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c7c962438badb944.xml (deflated 38%) 2024-08-07T19:25:47.1973446Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-42dc245516c87325.xml (deflated 36%) 2024-08-07T19:25:47.1974779Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f30997b879436271.xml (deflated 36%) 2024-08-07T19:25:47.1976097Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a1417f4dec066c4d.xml (deflated 37%) 2024-08-07T19:25:47.1977411Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e726bbe9f340cdd3.xml (deflated 35%) 2024-08-07T19:25:47.1978711Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-470161dbc2bf8fba.xml (deflated 36%) 2024-08-07T19:25:47.1980031Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5ed110bcf1c47c91.xml (deflated 36%) 2024-08-07T19:25:47.1981347Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5fce45a98bf0fb63.xml (deflated 36%) 2024-08-07T19:25:47.1982729Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6bbe49b88ef40832.xml (deflated 36%) 2024-08-07T19:25:47.1984019Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-464cea69745b0666.xml (deflated 37%) 2024-08-07T19:25:47.1985531Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e65dac1b7afb93fe.xml (deflated 36%) 2024-08-07T19:25:47.1986887Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fe39ae53d78b46fc.xml (deflated 36%) 2024-08-07T19:25:47.1988298Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-73a1f6e1faa681cb.xml (deflated 37%) 2024-08-07T19:25:47.1989705Z adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-079837f17742de4c.xml (deflated 36%) 2024-08-07T19:25:47.1991076Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_init/distributed._shard.sharded_tensor.ops.test_init-ec98e4622acf7462.xml (deflated 76%) 2024-08-07T19:25:47.1992580Z adding: test/test-reports/python-pytest/distributed._shard.sharded_optim.test_sharded_optim/distributed._shard.sharded_optim.test_sharded_optim-6e600cad988c213c.xml (deflated 67%) 2024-08-07T19:25:47.1993950Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-a1f0f0ab75501b75.xml (deflated 43%) 2024-08-07T19:25:47.1995151Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-702525090144157a.xml (deflated 43%) 2024-08-07T19:25:47.1996336Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-b86943fcd244ee1d.xml (deflated 43%) 2024-08-07T19:25:47.1997541Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-d1628b44856ca31b.xml (deflated 43%) 2024-08-07T19:25:47.1998799Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-01da30ec5d527054.xml (deflated 36%) 2024-08-07T19:25:47.1999996Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-6b4eef68b929b4d2.xml (deflated 43%) 2024-08-07T19:25:47.2001175Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-f117de03e5d0e9e2.xml (deflated 43%) 2024-08-07T19:25:47.2002376Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-ff6e7eca523f2831.xml (deflated 44%) 2024-08-07T19:25:47.2003577Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-d7ffed9d2da105fa.xml (deflated 44%) 2024-08-07T19:25:47.2004788Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-b013e3f93ecba00b.xml (deflated 44%) 2024-08-07T19:25:47.2005971Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-401095f1698c809a.xml (deflated 43%) 2024-08-07T19:25:47.2007181Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-3f4240464b3c8c04.xml (deflated 44%) 2024-08-07T19:25:47.2008370Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-452885640a2ae19b.xml (deflated 44%) 2024-08-07T19:25:47.2009568Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-d0345734735befee.xml (deflated 44%) 2024-08-07T19:25:47.2010752Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-ddc0dc618c1128ca.xml (deflated 44%) 2024-08-07T19:25:47.2012025Z adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_comm/distributed.fsdp.test_fsdp_comm-ddd058b15d5c2813.xml (deflated 87%) 2024-08-07T19:25:47.2013400Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_embedding/distributed._shard.sharded_tensor.ops.test_embedding-1ac4a13dd7a0555d.xml (deflated 68%) 2024-08-07T19:25:47.2014967Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_embedding_bag/distributed._shard.sharded_tensor.ops.test_embedding_bag-f057d7f208644e99.xml (deflated 68%) 2024-08-07T19:25:47.2016698Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.test_sharded_tensor_reshard/distributed._shard.sharded_tensor.test_sharded_tensor_reshard-36e4c85c2bc3b023.xml (deflated 70%) 2024-08-07T19:25:47.2018080Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-efda420c34809e4d.xml (deflated 44%) 2024-08-07T19:25:47.2019140Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-cff05249f5bf5999.xml (deflated 36%) 2024-08-07T19:25:47.2020178Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-139d7818fe4c3ce4.xml (deflated 38%) 2024-08-07T19:25:47.2021224Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-57623137b64a97da.xml (deflated 43%) 2024-08-07T19:25:47.2022264Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-024b3db28fe4a196.xml (deflated 43%) 2024-08-07T19:25:47.2023309Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-fa430e5eadd221bb.xml (deflated 36%) 2024-08-07T19:25:47.2024351Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-bb6f404dd49e7ee4.xml (deflated 36%) 2024-08-07T19:25:47.2025407Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-4df92f31f4d0d5dc.xml (deflated 37%) 2024-08-07T19:25:47.2026456Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-950bb74eb76ec089.xml (deflated 37%) 2024-08-07T19:25:47.2027499Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-1071daa4e5730a3b.xml (deflated 37%) 2024-08-07T19:25:47.2028525Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-16795e10566b8b7a.xml (deflated 37%) 2024-08-07T19:25:47.2029567Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-71781fadf9d89926.xml (deflated 37%) 2024-08-07T19:25:47.2030619Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-0cd9f5a91b148f6d.xml (deflated 36%) 2024-08-07T19:25:47.2031668Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-bdb036b69a13308d.xml (deflated 36%) 2024-08-07T19:25:47.2032711Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-99273b9396cfde77.xml (deflated 36%) 2024-08-07T19:25:47.2033751Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-8d42c17fbe5d20ed.xml (deflated 43%) 2024-08-07T19:25:47.2034792Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-c55663f234a097f2.xml (deflated 36%) 2024-08-07T19:25:47.2035818Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-a66727efc9c9fece.xml (deflated 43%) 2024-08-07T19:25:47.2036866Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-a281a7b88ce37183.xml (deflated 43%) 2024-08-07T19:25:47.2037907Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-35a14add6c48e269.xml (deflated 35%) 2024-08-07T19:25:47.2039002Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-9a1db8b8901332f6.xml (deflated 36%) 2024-08-07T19:25:47.2040298Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-d7e6eaf017c55d19.xml (deflated 36%) 2024-08-07T19:25:47.2041381Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-cf82cc935915c8f8.xml (deflated 36%) 2024-08-07T19:25:47.2042424Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-d657a64eaa676cf2.xml (deflated 36%) 2024-08-07T19:25:47.2043465Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-fac418f7fddab20e.xml (deflated 38%) 2024-08-07T19:25:47.2044516Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-2fa2bf1778c9476a.xml (deflated 36%) 2024-08-07T19:25:47.2045661Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-694edfd5c77b81ea.xml (deflated 36%) 2024-08-07T19:25:47.2046706Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-08ada4a4440170ef.xml (deflated 34%) 2024-08-07T19:25:47.2047741Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-048cbf650e04f5e0.xml (deflated 34%) 2024-08-07T19:25:47.2048787Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-df95bcebeef795c6.xml (deflated 36%) 2024-08-07T19:25:47.2049859Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-a02356cfeb5d0c39.xml (deflated 37%) 2024-08-07T19:25:47.2050893Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-cf750f422a617049.xml (deflated 37%) 2024-08-07T19:25:47.2051934Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-75184a8e3ee77633.xml (deflated 37%) 2024-08-07T19:25:47.2052971Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-faf146aad896a788.xml (deflated 36%) 2024-08-07T19:25:47.2054018Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-46b1a4eea8c72efa.xml (deflated 36%) 2024-08-07T19:25:47.2055056Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-cd6a5c40d2223f29.xml (deflated 35%) 2024-08-07T19:25:47.2056097Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-26762a5f185f9658.xml (deflated 34%) 2024-08-07T19:25:47.2057131Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-dd581cd1cc68b884.xml (deflated 35%) 2024-08-07T19:25:47.2058162Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-038e493a4dc72841.xml (deflated 35%) 2024-08-07T19:25:47.2059199Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-37c28e56d4853042.xml (deflated 38%) 2024-08-07T19:25:47.2060239Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-6b561cff93a0521b.xml (deflated 34%) 2024-08-07T19:25:47.2061283Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-02647b6a67e27e28.xml (deflated 34%) 2024-08-07T19:25:47.2062314Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-2ca37a98789ea5b0.xml (deflated 33%) 2024-08-07T19:25:47.2063364Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-7a1112df4088d0c6.xml (deflated 34%) 2024-08-07T19:25:47.2064411Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-19502c3923736dd7.xml (deflated 35%) 2024-08-07T19:25:47.2065458Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-4c6de17aeef467e6.xml (deflated 35%) 2024-08-07T19:25:47.2066516Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-0f1c50b1e259596a.xml (deflated 35%) 2024-08-07T19:25:47.2067568Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-6f58e8d285b6987c.xml (deflated 35%) 2024-08-07T19:25:47.2068689Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-a763e63c2f78c32f.xml (deflated 34%) 2024-08-07T19:25:47.2069725Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-29f2f005bf63aedb.xml (deflated 35%) 2024-08-07T19:25:47.2070772Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-85a1daf7489a3d15.xml (deflated 34%) 2024-08-07T19:25:47.2071825Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-7921b4362b00aad0.xml (deflated 35%) 2024-08-07T19:25:47.2072924Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-9d9de016816165d4.xml (deflated 36%) 2024-08-07T19:25:47.2073954Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-33015b552c9f3d10.xml (deflated 35%) 2024-08-07T19:25:47.2075000Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-d3823b0434aef41e.xml (deflated 36%) 2024-08-07T19:25:47.2076055Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-afd76baab0790a51.xml (deflated 35%) 2024-08-07T19:25:47.2077096Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-4fc6adea2dd4bc02.xml (deflated 35%) 2024-08-07T19:25:47.2078153Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-9aa4d763655ce939.xml (deflated 35%) 2024-08-07T19:25:47.2079278Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-e2b98b0d4926ea8d.xml (deflated 37%) 2024-08-07T19:25:47.2080339Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-eee1f75ba6b1a3df.xml (deflated 36%) 2024-08-07T19:25:47.2081379Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-6522df66ba86ca03.xml (deflated 36%) 2024-08-07T19:25:47.2082431Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-fa1159725bc81e89.xml (deflated 35%) 2024-08-07T19:25:47.2083493Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-ebf0e25ca39d9aa7.xml (deflated 35%) 2024-08-07T19:25:47.2084553Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-3aa5f05ba9a7cbda.xml (deflated 35%) 2024-08-07T19:25:47.2085593Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-887da17301e4a10f.xml (deflated 35%) 2024-08-07T19:25:47.2086643Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-6c115597cccec58c.xml (deflated 35%) 2024-08-07T19:25:47.2087700Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-1e469a6ab1bc7f41.xml (deflated 35%) 2024-08-07T19:25:47.2088738Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-d94f451369a1df4f.xml (deflated 35%) 2024-08-07T19:25:47.2089792Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-d96381f532632a90.xml (deflated 35%) 2024-08-07T19:25:47.2090839Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-879cfa73b74676a1.xml (deflated 34%) 2024-08-07T19:25:47.2091886Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-67b164179f2d21a6.xml (deflated 36%) 2024-08-07T19:25:47.2092917Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-4789b27bfc0f3546.xml (deflated 35%) 2024-08-07T19:25:47.2093967Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-0f6ea97cb12689a2.xml (deflated 35%) 2024-08-07T19:25:47.2095014Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-52030f3f671bf594.xml (deflated 35%) 2024-08-07T19:25:47.2096041Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-12a1402608ca1e72.xml (deflated 35%) 2024-08-07T19:25:47.2097146Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-bbe5565aaf455425.xml (deflated 36%) 2024-08-07T19:25:47.2098197Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-b58ff5b3495e5abf.xml (deflated 36%) 2024-08-07T19:25:47.2099244Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-1d80c6380f3c5b6c.xml (deflated 35%) 2024-08-07T19:25:47.2100278Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-751d5be3938244ab.xml (deflated 36%) 2024-08-07T19:25:47.2101366Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-723250fc728151cb.xml (deflated 35%) 2024-08-07T19:25:47.2102412Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-9c9b6363f0e9478a.xml (deflated 35%) 2024-08-07T19:25:47.2103460Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-aff4096d07e7d183.xml (deflated 35%) 2024-08-07T19:25:47.2104493Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-8c37fb41db980301.xml (deflated 36%) 2024-08-07T19:25:47.2105539Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-3a6442a1e1868f3e.xml (deflated 33%) 2024-08-07T19:25:47.2106582Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-784650627ce38c29.xml (deflated 33%) 2024-08-07T19:25:47.2107610Z adding: test/test-reports/python-pytest/distributed.test_store/distributed.test_store-65b7f6637ef03f28.xml (deflated 35%) 2024-08-07T19:25:47.2108724Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-9d5bc4762d2213a9.xml (deflated 43%) 2024-08-07T19:25:47.2109905Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-143fd63dbf0f8bb6.xml (deflated 43%) 2024-08-07T19:25:47.2111087Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-71bef987771cf20f.xml (deflated 43%) 2024-08-07T19:25:47.2112245Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-3e3a50027fc55f33.xml (deflated 45%) 2024-08-07T19:25:47.2113420Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-20fb17228e0c4fdf.xml (deflated 46%) 2024-08-07T19:25:47.2114598Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-bf56d681db201200.xml (deflated 46%) 2024-08-07T19:25:47.2115774Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-d42708c97d39cf70.xml (deflated 45%) 2024-08-07T19:25:47.2116930Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-8e89420865e04e63.xml (deflated 46%) 2024-08-07T19:25:47.2118115Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_ucc/distributed.test_c10d_spawn_ucc-8996c2be1dff9d9b.xml (deflated 46%) 2024-08-07T19:25:47.2119364Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-46b25e073e8b435b.xml (deflated 43%) 2024-08-07T19:25:47.2120548Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-f71742c16baef4e9.xml (deflated 43%) 2024-08-07T19:25:47.2121744Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-a9c85991cd9ca105.xml (deflated 43%) 2024-08-07T19:25:47.2122944Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-c02cff269f5fadc8.xml (deflated 43%) 2024-08-07T19:25:47.2124146Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-9c664307609cd2db.xml (deflated 45%) 2024-08-07T19:25:47.2125391Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-d95f72669bbbe09c.xml (deflated 45%) 2024-08-07T19:25:47.2126587Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-d0086f03909c9652.xml (deflated 46%) 2024-08-07T19:25:47.2127784Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-d3431e042875f7ff.xml (deflated 46%) 2024-08-07T19:25:47.2128986Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-7b354b5f074d3449.xml (deflated 45%) 2024-08-07T19:25:47.2130240Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-da7e1e5be2f4028a.xml (deflated 46%) 2024-08-07T19:25:47.2131435Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-31375ec495b43d05.xml (deflated 46%) 2024-08-07T19:25:47.2132629Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-1532a1e4b841ddda.xml (deflated 45%) 2024-08-07T19:25:47.2133842Z adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-d0bab4de686290fb.xml (deflated 45%) 2024-08-07T19:25:47.2135211Z adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.test_sharded_tensor/distributed._shard.sharded_tensor.test_sharded_tensor-f16d826c855a883d.xml (deflated 95%) 2024-08-07T19:25:47.2136554Z adding: test/test-reports/python-pytest/distributed.test_launcher/distributed.test_launcher-a43bf37a76349888.xml (deflated 40%) 2024-08-07T19:25:47.2137757Z adding: test/test-reports/python-pytest/distributed.elastic.events.lib_test/distributed.elastic.events.lib_test-d243551f1cf4d083.xml (deflated 76%) 2024-08-07T19:25:47.2139069Z adding: test/test-reports/python-pytest/distributed.elastic.metrics.api_test/distributed.elastic.metrics.api_test-6838dff90b26a7dd.xml (deflated 60%) 2024-08-07T19:25:47.2140619Z adding: test/test-reports/python-pytest/distributed.elastic.multiprocessing.api_test/distributed.elastic.multiprocessing.api_test-48be8c5bd6c20fb7.xml (deflated 88%) 2024-08-07T19:25:47.2142119Z adding: test/test-reports/python-pytest/distributed.elastic.timer.local_timer_example/distributed.elastic.timer.local_timer_example-e43d49fac16e8e0e.xml (deflated 52%) 2024-08-07T19:25:47.2143568Z adding: test/test-reports/python-pytest/distributed.elastic.timer.local_timer_test/distributed.elastic.timer.local_timer_test-f61ca2d872209e6a.xml (deflated 80%) 2024-08-07T19:25:47.2145006Z adding: test/test-reports/python-pytest/distributed.elastic.utils.distributed_test/distributed.elastic.utils.distributed_test-42f2bb11736ad5f6.xml (deflated 78%) 2024-08-07T19:25:47.2146393Z adding: test/test-reports/python-pytest/distributed.elastic.utils.logging_test/distributed.elastic.utils.logging_test-d4f0211e14d6400a.xml (deflated 51%) 2024-08-07T19:25:47.2147726Z adding: test/test-reports/python-pytest/distributed.elastic.utils.util_test/distributed.elastic.utils.util_test-306dcbe4640326ce.xml (deflated 79%) 2024-08-07T19:25:47.2149130Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_ddp_2d_parallel/distributed.tensor.parallel.test_ddp_2d_parallel-de836eb0a5e38324.xml (deflated 46%) 2024-08-07T19:25:47.2150650Z adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_fsdp_2d_parallel/distributed.tensor.parallel.test_fsdp_2d_parallel-04a793d825f40a1b.xml (deflated 89%) 2024-08-07T19:25:47.2151823Z adding: test/test-reports/python-pytest/test.run_test/test.run_test-f35993719ca94866.xml (deflated 64%) 2024-08-07T19:25:47.2152858Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d8eb85a232f1619a.xml (deflated 42%) 2024-08-07T19:25:47.2154066Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9a58c5089bc782ad.xml (deflated 43%) 2024-08-07T19:25:47.2155357Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-985cdecf094f8d13.xml (deflated 38%) 2024-08-07T19:25:47.2156567Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d386dde8ffcd326f.xml (deflated 36%) 2024-08-07T19:25:47.2157776Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f680100d11391c9f.xml (deflated 41%) 2024-08-07T19:25:47.2159094Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8694507e55005b14.xml (deflated 41%) 2024-08-07T19:25:47.2160283Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-95d6b973601c6f02.xml (deflated 40%) 2024-08-07T19:25:47.2161489Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ca03a13aeafec168.xml (deflated 41%) 2024-08-07T19:25:47.2162710Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e0e74c129ea1aae1.xml (deflated 36%) 2024-08-07T19:25:47.2163919Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-788a6e35e1ceb5eb.xml (deflated 35%) 2024-08-07T19:25:47.2165120Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-38eb7d56ba1f5eff.xml (deflated 35%) 2024-08-07T19:25:47.2166335Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b550d656d6458967.xml (deflated 36%) 2024-08-07T19:25:47.2167542Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-914d05f045ffde5c.xml (deflated 43%) 2024-08-07T19:25:47.2168751Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-84ca6f0453e468ea.xml (deflated 45%) 2024-08-07T19:25:47.2169955Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9daa44a71e38edef.xml (deflated 35%) 2024-08-07T19:25:47.2171168Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b03d22c7cd751eab.xml (deflated 35%) 2024-08-07T19:25:47.2172381Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f579cd8bf1dc91b4.xml (deflated 35%) 2024-08-07T19:25:47.2173598Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0a5bac279585edb9.xml (deflated 36%) 2024-08-07T19:25:47.2174797Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2d1f5a3b757625ee.xml (deflated 35%) 2024-08-07T19:25:47.2176016Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-75ee2ceba6b7df0a.xml (deflated 35%) 2024-08-07T19:25:47.2177237Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e3725ca31337dc23.xml (deflated 35%) 2024-08-07T19:25:47.2178437Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3103b144d87d06b1.xml (deflated 35%) 2024-08-07T19:25:47.2179644Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fe6a2262c872b37a.xml (deflated 35%) 2024-08-07T19:25:47.2180857Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0999ee86be246a4e.xml (deflated 36%) 2024-08-07T19:25:47.2182071Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-18c126139d12aca0.xml (deflated 35%) 2024-08-07T19:25:47.2183323Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-788d95964a0bd40a.xml (deflated 35%) 2024-08-07T19:25:47.2184532Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b7afa4363eaabc56.xml (deflated 36%) 2024-08-07T19:25:47.2185750Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fe6947f8bffe1853.xml (deflated 35%) 2024-08-07T19:25:47.2186976Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-40d169ffe01b687f.xml (deflated 35%) 2024-08-07T19:25:47.2188232Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-914cea7472bbe0ab.xml (deflated 35%) 2024-08-07T19:25:47.2189454Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ce851fbbd096e975.xml (deflated 36%) 2024-08-07T19:25:47.2190672Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-50d9f9671442ebd0.xml (deflated 35%) 2024-08-07T19:25:47.2191893Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-efcc686bda495eef.xml (deflated 42%) 2024-08-07T19:25:47.2193106Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fb7dc85545cacba6.xml (deflated 43%) 2024-08-07T19:25:47.2194328Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-78aa9c4a016ecc3f.xml (deflated 43%) 2024-08-07T19:25:47.2195553Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-806286b99b3fafee.xml (deflated 43%) 2024-08-07T19:25:47.2196769Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-470a993fa8e7cf44.xml (deflated 44%) 2024-08-07T19:25:47.2197979Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ec39d68ca717cdf2.xml (deflated 44%) 2024-08-07T19:25:47.2199253Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d5a73332327d5059.xml (deflated 44%) 2024-08-07T19:25:47.2200468Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b8d409732a5e3e58.xml (deflated 44%) 2024-08-07T19:25:47.2201669Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-818df733fa91a083.xml (deflated 44%) 2024-08-07T19:25:47.2202892Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-56ba240954fbcdab.xml (deflated 44%) 2024-08-07T19:25:47.2204119Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f6d6b1514360b4cc.xml (deflated 42%) 2024-08-07T19:25:47.2205344Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0491910c3ac83b0c.xml (deflated 35%) 2024-08-07T19:25:47.2206544Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2035ef15af422236.xml (deflated 42%) 2024-08-07T19:25:47.2207752Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9ad9034b8b7a3ff4.xml (deflated 42%) 2024-08-07T19:25:47.2208964Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3dedfb3e13c41ff1.xml (deflated 35%) 2024-08-07T19:25:47.2210182Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f316453141a2f1b8.xml (deflated 35%) 2024-08-07T19:25:47.2211382Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d2bdaa6f8e7219f9.xml (deflated 42%) 2024-08-07T19:25:47.2212658Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4f6696e0f4fd2539.xml (deflated 42%) 2024-08-07T19:25:47.2213864Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8ea7038e738fef53.xml (deflated 35%) 2024-08-07T19:25:47.2215074Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c58a5fcc2161d330.xml (deflated 47%) 2024-08-07T19:25:47.2216271Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f170d8e1cc488be9.xml (deflated 47%) 2024-08-07T19:25:47.2217535Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fdb2b63ab2f02126.xml (deflated 47%) 2024-08-07T19:25:47.2218741Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-635d5658b496d356.xml (deflated 41%) 2024-08-07T19:25:47.2219950Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-402b63c30d67aa97.xml (deflated 42%) 2024-08-07T19:25:47.2221147Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c1eec0259310780a.xml (deflated 42%) 2024-08-07T19:25:47.2222357Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-af07032e0ba1186a.xml (deflated 42%) 2024-08-07T19:25:47.2223570Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-124cf034cf453a65.xml (deflated 35%) 2024-08-07T19:25:47.2224777Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-22ebcd6c935ac4a7.xml (deflated 43%) 2024-08-07T19:25:47.2225995Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b1e2d4af85d8d1f7.xml (deflated 42%) 2024-08-07T19:25:47.2227210Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e0209183cd04cd55.xml (deflated 42%) 2024-08-07T19:25:47.2228416Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1c59529462029014.xml (deflated 35%) 2024-08-07T19:25:47.2229605Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e4ace651388e6bec.xml (deflated 42%) 2024-08-07T19:25:47.2230820Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a6931bb02ae944c6.xml (deflated 43%) 2024-08-07T19:25:47.2232037Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-51cc1fdf8f3589d1.xml (deflated 43%) 2024-08-07T19:25:47.2233250Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-90b411d801061e2f.xml (deflated 42%) 2024-08-07T19:25:47.2234451Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-867d7c8ea69817f2.xml (deflated 42%) 2024-08-07T19:25:47.2235656Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-320e408d92640636.xml (deflated 42%) 2024-08-07T19:25:47.2236856Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-65a3a09d3c28b390.xml (deflated 44%) 2024-08-07T19:25:47.2238072Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0dd9e3f1e2572a3d.xml (deflated 42%) 2024-08-07T19:25:47.2239403Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f937f9af72d0c4a8.xml (deflated 43%) 2024-08-07T19:25:47.2240787Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8b913f06ff598761.xml (deflated 42%) 2024-08-07T19:25:47.2242110Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cbb54eb5a1767b44.xml (deflated 36%) 2024-08-07T19:25:47.2243330Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a722f53e4a449d0e.xml (deflated 36%) 2024-08-07T19:25:47.2244534Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-058a26e3bfc797dd.xml (deflated 43%) 2024-08-07T19:25:47.2245843Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f47fcb97d57edd27.xml (deflated 35%) 2024-08-07T19:25:47.2247061Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-69e752d09b584f2b.xml (deflated 37%) 2024-08-07T19:25:47.2248264Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-25ee5e1b2c778e23.xml (deflated 42%) 2024-08-07T19:25:47.2249489Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-69684e21e565bc8b.xml (deflated 36%) 2024-08-07T19:25:47.2250703Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d94ab3067aaf51d0.xml (deflated 36%) 2024-08-07T19:25:47.2251920Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fb329e11611a711e.xml (deflated 36%) 2024-08-07T19:25:47.2253128Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f604c2caf469479a.xml (deflated 35%) 2024-08-07T19:25:47.2254344Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-19755b73cb80a65f.xml (deflated 35%) 2024-08-07T19:25:47.2255554Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-718d466df37b08b9.xml (deflated 44%) 2024-08-07T19:25:47.2256763Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8f95a0caa8508834.xml (deflated 42%) 2024-08-07T19:25:47.2257956Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8c1e33ed00c539c9.xml (deflated 43%) 2024-08-07T19:25:47.2259163Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b47d2477004c01ed.xml (deflated 43%) 2024-08-07T19:25:47.2260382Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c1ec5e8afd95cfd7.xml (deflated 42%) 2024-08-07T19:25:47.2261596Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fda7925299889aea.xml (deflated 36%) 2024-08-07T19:25:47.2262794Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b0fd66d1012c7ff3.xml (deflated 35%) 2024-08-07T19:25:47.2264006Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c02238e19dd10f47.xml (deflated 36%) 2024-08-07T19:25:47.2265217Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c45ec73a3479d822.xml (deflated 36%) 2024-08-07T19:25:47.2266427Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c3112a439bb00b06.xml (deflated 43%) 2024-08-07T19:25:47.2267629Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-035a4f289eeb9fea.xml (deflated 42%) 2024-08-07T19:25:47.2268847Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a4e16e5cd5bc298e.xml (deflated 42%) 2024-08-07T19:25:47.2270105Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-408ded7079e433e8.xml (deflated 41%) 2024-08-07T19:25:47.2271300Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c8f9065d4d4cb10e.xml (deflated 42%) 2024-08-07T19:25:47.2272516Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a892ea108a8ac5b6.xml (deflated 36%) 2024-08-07T19:25:47.2273729Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-265e9da37c8cabb5.xml (deflated 35%) 2024-08-07T19:25:47.2274988Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-612d0d9687070cfc.xml (deflated 35%) 2024-08-07T19:25:47.2276176Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fed065f48185f1d8.xml (deflated 43%) 2024-08-07T19:25:47.2277393Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d74eae5143566280.xml (deflated 42%) 2024-08-07T19:25:47.2278658Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-91793670d1f5c889.xml (deflated 43%) 2024-08-07T19:25:47.2279869Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ef8a135c50acec04.xml (deflated 37%) 2024-08-07T19:25:47.2281066Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e86424efe58e10b7.xml (deflated 36%) 2024-08-07T19:25:47.2282284Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8f8bacca3087f841.xml (deflated 41%) 2024-08-07T19:25:47.2283502Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a59bd9a229de0b9e.xml (deflated 41%) 2024-08-07T19:25:47.2284720Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-229774cba3c1fba2.xml (deflated 40%) 2024-08-07T19:25:47.2285913Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-19eb207fd8376455.xml (deflated 42%) 2024-08-07T19:25:47.2287121Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6755087f5bd15219.xml (deflated 36%) 2024-08-07T19:25:47.2288338Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f0f76d283ce2750e.xml (deflated 36%) 2024-08-07T19:25:47.2289562Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2c639eab63c77eef.xml (deflated 35%) 2024-08-07T19:25:47.2290767Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bcf355f4253892e4.xml (deflated 36%) 2024-08-07T19:25:47.2291984Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-62a48e2f889c8cf7.xml (deflated 43%) 2024-08-07T19:25:47.2293201Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fa7baf1b7535f5e7.xml (deflated 45%) 2024-08-07T19:25:47.2294404Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cdad098f2f0d8d49.xml (deflated 35%) 2024-08-07T19:25:47.2295620Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9036f60d45d28677.xml (deflated 35%) 2024-08-07T19:25:47.2296835Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a7da1fa8e2802bd5.xml (deflated 35%) 2024-08-07T19:25:47.2298055Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-dadf6b2d541df09c.xml (deflated 35%) 2024-08-07T19:25:47.2299339Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-644dc52a7b69a507.xml (deflated 36%) 2024-08-07T19:25:47.2300547Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-54177f76033b7d07.xml (deflated 35%) 2024-08-07T19:25:47.2301763Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6515c50811ddba18.xml (deflated 35%) 2024-08-07T19:25:47.2302982Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fa8de00d1d0aacba.xml (deflated 35%) 2024-08-07T19:25:47.2304241Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1f90686763c6c397.xml (deflated 35%) 2024-08-07T19:25:47.2305453Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-46e5c71f06d266ac.xml (deflated 35%) 2024-08-07T19:25:47.2306678Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-339dbc9a1b866cf3.xml (deflated 35%) 2024-08-07T19:25:47.2307895Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9a1292157a06c204.xml (deflated 35%) 2024-08-07T19:25:47.2309101Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1fbaa7eefc85b845.xml (deflated 35%) 2024-08-07T19:25:47.2310326Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4d34ad3e8740fbd9.xml (deflated 35%) 2024-08-07T19:25:47.2311551Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-37b77e8dfaffbe24.xml (deflated 35%) 2024-08-07T19:25:47.2312770Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-508e7e2ebd5b641c.xml (deflated 36%) 2024-08-07T19:25:47.2313978Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fd540ffcc3eef279.xml (deflated 35%) 2024-08-07T19:25:47.2315202Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7d0affc0c4372163.xml (deflated 36%) 2024-08-07T19:25:47.2316424Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a934c52ab073d35d.xml (deflated 42%) 2024-08-07T19:25:47.2317629Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d313fae70ca4ef20.xml (deflated 43%) 2024-08-07T19:25:47.2318906Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9285bab9a1bb6fa1.xml (deflated 43%) 2024-08-07T19:25:47.2320130Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7882659caae3ee27.xml (deflated 43%) 2024-08-07T19:25:47.2321356Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-92f715be45fa726c.xml (deflated 44%) 2024-08-07T19:25:47.2322558Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-01f1d4228588fefe.xml (deflated 44%) 2024-08-07T19:25:47.2323779Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-20f6d53388d0cb25.xml (deflated 44%) 2024-08-07T19:25:47.2324997Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7fa43b7783945dd2.xml (deflated 44%) 2024-08-07T19:25:47.2326210Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1b1f666d80c79bc7.xml (deflated 44%) 2024-08-07T19:25:47.2327403Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3bf723e33555f011.xml (deflated 44%) 2024-08-07T19:25:47.2328660Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5e77167a63e81547.xml (deflated 43%) 2024-08-07T19:25:47.2329870Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ee652925726f68c5.xml (deflated 36%) 2024-08-07T19:25:47.2331076Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-931315ccf9408dde.xml (deflated 42%) 2024-08-07T19:25:47.2332326Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-23a4c309dddbea25.xml (deflated 42%) 2024-08-07T19:25:47.2333543Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-90942a030d4b4dae.xml (deflated 36%) 2024-08-07T19:25:47.2334760Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d0c9b66a896206be.xml (deflated 35%) 2024-08-07T19:25:47.2335973Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e21816140bbf3578.xml (deflated 42%) 2024-08-07T19:25:47.2337164Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-000aed0e6c406130.xml (deflated 42%) 2024-08-07T19:25:47.2338376Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d47a29094c33f15d.xml (deflated 36%) 2024-08-07T19:25:47.2339588Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2cca93dc77945311.xml (deflated 47%) 2024-08-07T19:25:47.2341103Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-985bd2561c5f1dd9.xml (deflated 47%) 2024-08-07T19:25:47.2342317Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-95cb1c012f070b22.xml (deflated 47%) 2024-08-07T19:25:47.2343539Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-569e4f332d6ebdb4.xml (deflated 42%) 2024-08-07T19:25:47.2344751Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ae0d1f162b69d937.xml (deflated 42%) 2024-08-07T19:25:47.2345953Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f62f296547605c87.xml (deflated 42%) 2024-08-07T19:25:47.2347169Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2ea09311b9731aa0.xml (deflated 42%) 2024-08-07T19:25:47.2348382Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9c3d5d684782f7fb.xml (deflated 35%) 2024-08-07T19:25:47.2349594Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8febbc9621c52734.xml (deflated 42%) 2024-08-07T19:25:47.2350796Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c2f210bd9d022f40.xml (deflated 42%) 2024-08-07T19:25:47.2352012Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8e90e96d6aa711b5.xml (deflated 42%) 2024-08-07T19:25:47.2353223Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0fd762ab094392c6.xml (deflated 35%) 2024-08-07T19:25:47.2354429Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5f52b04de47e6fd4.xml (deflated 42%) 2024-08-07T19:25:47.2355627Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0e40473eee50f676.xml (deflated 43%) 2024-08-07T19:25:47.2356835Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1165dda95c46fe41.xml (deflated 43%) 2024-08-07T19:25:47.2358170Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a3074847b02bdac0.xml (deflated 42%) 2024-08-07T19:25:47.2359462Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d6450eb1dd5dfc17.xml (deflated 42%) 2024-08-07T19:25:47.2360664Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-82fe846263e2f968.xml (deflated 42%) 2024-08-07T19:25:47.2361952Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d9caf857c8b08397.xml (deflated 44%) 2024-08-07T19:25:47.2363170Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4e066ff1424267de.xml (deflated 42%) 2024-08-07T19:25:47.2364378Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-dccf4bdb1ea04994.xml (deflated 43%) 2024-08-07T19:25:47.2365592Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-45faf8652b06cdf0.xml (deflated 43%) 2024-08-07T19:25:47.2366811Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b620dbe36ce66849.xml (deflated 36%) 2024-08-07T19:25:47.2368030Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4a653e5389df18b5.xml (deflated 36%) 2024-08-07T19:25:47.2369229Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7bee39868647cf79.xml (deflated 42%) 2024-08-07T19:25:47.2370438Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2bc1a74be5e06d23.xml (deflated 36%) 2024-08-07T19:25:47.2371653Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c9888cb37e8661a7.xml (deflated 37%) 2024-08-07T19:25:47.2372860Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ea7d1a4523574434.xml (deflated 42%) 2024-08-07T19:25:47.2374055Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a3f28e7ca33eaa0b.xml (deflated 36%) 2024-08-07T19:25:47.2375273Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-84158f71b91b5f2d.xml (deflated 35%) 2024-08-07T19:25:47.2376491Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fbdce612a9be6724.xml (deflated 36%) 2024-08-07T19:25:47.2377705Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9ba209e271ea21fe.xml (deflated 35%) 2024-08-07T19:25:47.2378899Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-576bf462ff47a386.xml (deflated 35%) 2024-08-07T19:25:47.2380097Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-451243425c997643.xml (deflated 44%) 2024-08-07T19:25:47.2381294Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-46f883dbc958f429.xml (deflated 43%) 2024-08-07T19:25:47.2382505Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fbf8c2ac422e90e8.xml (deflated 43%) 2024-08-07T19:25:47.2383710Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a0ad164b3c88923f.xml (deflated 43%) 2024-08-07T19:25:47.2384923Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a588b8edb0b56fc1.xml (deflated 42%) 2024-08-07T19:25:47.2386197Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d364160872cb52b7.xml (deflated 35%) 2024-08-07T19:25:47.2387387Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f56b3cb4b46250d0.xml (deflated 36%) 2024-08-07T19:25:47.2388595Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-013e9498024d6d88.xml (deflated 36%) 2024-08-07T19:25:47.2389800Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d467edbecb1aaf8b.xml (deflated 35%) 2024-08-07T19:25:47.2391063Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6bc84e8346330eba.xml (deflated 43%) 2024-08-07T19:25:47.2392259Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-14d956b03ab0c098.xml (deflated 42%) 2024-08-07T19:25:47.2393472Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d9dd0183b5408381.xml (deflated 41%) 2024-08-07T19:25:47.2394683Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b3b2fdb80d6a6343.xml (deflated 41%) 2024-08-07T19:25:47.2395895Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a6781959e6fe7b94.xml (deflated 42%) 2024-08-07T19:25:47.2397092Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a4c2f71b1edf4d63.xml (deflated 36%) 2024-08-07T19:25:47.2398311Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a99d83f743ca3940.xml (deflated 35%) 2024-08-07T19:25:47.2399575Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cfae411965759232.xml (deflated 35%) 2024-08-07T19:25:47.2400795Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3d444737fede3e73.xml (deflated 43%) 2024-08-07T19:25:47.2401988Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-124d35350ea27a0e.xml (deflated 36%) 2024-08-07T19:25:47.2403200Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0ad66e49cf03d561.xml (deflated 41%) 2024-08-07T19:25:47.2404418Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a094976eef8de388.xml (deflated 42%) 2024-08-07T19:25:47.2405629Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-529c70b2dd835791.xml (deflated 42%) 2024-08-07T19:25:47.2406832Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fa9eb53d9256bcd3.xml (deflated 35%) 2024-08-07T19:25:47.2408049Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1e4c98ed3bbfe9d8.xml (deflated 35%) 2024-08-07T19:25:47.2409265Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fd5ef9e0d50fe610.xml (deflated 35%) 2024-08-07T19:25:47.2410462Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cbdf27866e6ef942.xml (deflated 35%) 2024-08-07T19:25:47.2411669Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a081ffa8242960bb.xml (deflated 35%) 2024-08-07T19:25:47.2412880Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7eec0b9080b59426.xml (deflated 45%) 2024-08-07T19:25:47.2414088Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1bd831d07ea10020.xml (deflated 35%) 2024-08-07T19:25:47.2415362Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a2ac50bc5f98923e.xml (deflated 35%) 2024-08-07T19:25:47.2416578Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fb5cf9e6609a853a.xml (deflated 35%) 2024-08-07T19:25:47.2417799Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-dbdd168f18ab4d8e.xml (deflated 35%) 2024-08-07T19:25:47.2419070Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-456dc02daad652a1.xml (deflated 36%) 2024-08-07T19:25:47.2420273Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-35e4d2f744fbdb98.xml (deflated 35%) 2024-08-07T19:25:47.2421491Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-aa3130fcfcc7caf2.xml (deflated 35%) 2024-08-07T19:25:47.2422722Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-94e8cc29f7f4932d.xml (deflated 42%) 2024-08-07T19:25:47.2423944Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-98dd4d10a0442db8.xml (deflated 43%) 2024-08-07T19:25:47.2425152Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c9ac32d3aa3bc2dd.xml (deflated 44%) 2024-08-07T19:25:47.2426380Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-aadd197741a6256e.xml (deflated 44%) 2024-08-07T19:25:47.2427605Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2fc0bced878da7de.xml (deflated 44%) 2024-08-07T19:25:47.2428888Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ad38eb4bfbc3cd1e.xml (deflated 44%) 2024-08-07T19:25:47.2430093Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-33895122b91825c7.xml (deflated 42%) 2024-08-07T19:25:47.2431308Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3ccbfad0c302115c.xml (deflated 35%) 2024-08-07T19:25:47.2432518Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6927bc131b30c4ec.xml (deflated 42%) 2024-08-07T19:25:47.2433726Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f3309efd6fcd0e79.xml (deflated 35%) 2024-08-07T19:25:47.2434945Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c8ca3977f05edf06.xml (deflated 35%) 2024-08-07T19:25:47.2436160Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8a16a50d2225aa45.xml (deflated 41%) 2024-08-07T19:25:47.2437378Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f472be5b620c405e.xml (deflated 42%) 2024-08-07T19:25:47.2438622Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b492f673eb96b065.xml (deflated 41%) 2024-08-07T19:25:47.2439840Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-84753845ac3dc001.xml (deflated 35%) 2024-08-07T19:25:47.2441313Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-385522ccae4eec5c.xml (deflated 41%) 2024-08-07T19:25:47.2442526Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5e03245698bff4d5.xml (deflated 42%) 2024-08-07T19:25:47.2443728Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c77cb78bcbe7c2b7.xml (deflated 42%) 2024-08-07T19:25:47.2445054Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-675b49e710e1453f.xml (deflated 42%) 2024-08-07T19:25:47.2446254Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-80357cc864a83d12.xml (deflated 42%) 2024-08-07T19:25:47.2447469Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-af0dcf92ce78b41f.xml (deflated 42%) 2024-08-07T19:25:47.2448730Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c793af0a9fc08218.xml (deflated 43%) 2024-08-07T19:25:47.2449949Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b183ea83ab9e3ed2.xml (deflated 42%) 2024-08-07T19:25:47.2451168Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-32f2caf342f4bbcc.xml (deflated 42%) 2024-08-07T19:25:47.2452391Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-17273bbcf1bc4fc9.xml (deflated 42%) 2024-08-07T19:25:47.2453600Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b819d51e1f6f1a5e.xml (deflated 42%) 2024-08-07T19:25:47.2454814Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0ef6b2635e849d6f.xml (deflated 43%) 2024-08-07T19:25:47.2456041Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f07ca9e1110eeabd.xml (deflated 42%) 2024-08-07T19:25:47.2457241Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3710bf780e136e7c.xml (deflated 42%) 2024-08-07T19:25:47.2458458Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-414c2ca2471ae251.xml (deflated 41%) 2024-08-07T19:25:47.2459673Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bea497df330fef52.xml (deflated 42%) 2024-08-07T19:25:47.2460888Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1e46baa48660bf6b.xml (deflated 42%) 2024-08-07T19:25:47.2462087Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8f2866b05afedbb2.xml (deflated 42%) 2024-08-07T19:25:47.2463304Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-60db088bad340e23.xml (deflated 42%) 2024-08-07T19:25:47.2464512Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2eced31ac4ae533b.xml (deflated 42%) 2024-08-07T19:25:47.2465725Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-98a02e2432249ba4.xml (deflated 42%) 2024-08-07T19:25:47.2466920Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-76c7bb57c6efc116.xml (deflated 42%) 2024-08-07T19:25:47.2468133Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9df189085daf4319.xml (deflated 43%) 2024-08-07T19:25:47.2469347Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-00e01812fa79fcf9.xml (deflated 42%) 2024-08-07T19:25:47.2470567Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-aac13bc291f4281d.xml (deflated 35%) 2024-08-07T19:25:47.2471763Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-397682adf9ffc515.xml (deflated 35%) 2024-08-07T19:25:47.2473047Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6e02d7b64b0b499c.xml (deflated 35%) 2024-08-07T19:25:47.2474264Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-facd90dedb8519d0.xml (deflated 35%) 2024-08-07T19:25:47.2475490Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f07b8d6eb34fc8ec.xml (deflated 42%) 2024-08-07T19:25:47.2476702Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-be4694ed5f1ebc0e.xml (deflated 35%) 2024-08-07T19:25:47.2477991Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ce033b87bab968a3.xml (deflated 44%) 2024-08-07T19:25:47.2479280Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fab6d1db391a7c44.xml (deflated 45%) 2024-08-07T19:25:47.2480489Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e27dfd5065b4a5a2.xml (deflated 45%) 2024-08-07T19:25:47.2481711Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-230e01648720df27.xml (deflated 41%) 2024-08-07T19:25:47.2482929Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f67c07cf91102fd5.xml (deflated 41%) 2024-08-07T19:25:47.2484147Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ef86e25f49e2941d.xml (deflated 42%) 2024-08-07T19:25:47.2485352Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c6e4ae61fbe9f31f.xml (deflated 41%) 2024-08-07T19:25:47.2486567Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b938a38b51d50c66.xml (deflated 41%) 2024-08-07T19:25:47.2487784Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-eb9e0d56367f7387.xml (deflated 41%) 2024-08-07T19:25:47.2488993Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6e371a1840c12c62.xml (deflated 35%) 2024-08-07T19:25:47.2490187Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-054ab3b0bbb7421f.xml (deflated 35%) 2024-08-07T19:25:47.2491395Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-155393b8588e241b.xml (deflated 35%) 2024-08-07T19:25:47.2492606Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-93c515f25a0277e7.xml (deflated 35%) 2024-08-07T19:25:47.2493812Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a37db20306f1a3b9.xml (deflated 45%) 2024-08-07T19:25:47.2495020Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-47746ab7f85cca2c.xml (deflated 42%) 2024-08-07T19:25:47.2496233Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-e2ec7fe063e6348a.xml (deflated 37%) 2024-08-07T19:25:47.2497439Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-766294c18f47653d.xml (deflated 36%) 2024-08-07T19:25:47.2498645Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-794f1fa61ea8a6b3.xml (deflated 35%) 2024-08-07T19:25:47.2499849Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4f787173dbd7ed01.xml (deflated 42%) 2024-08-07T19:25:47.2501064Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-32ecd5a63bd45cf3.xml (deflated 35%) 2024-08-07T19:25:47.2502340Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7ef8de9ab07a6bff.xml (deflated 35%) 2024-08-07T19:25:47.2503540Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-186b953729e3507d.xml (deflated 35%) 2024-08-07T19:25:47.2504741Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0169d7ec8b73ff55.xml (deflated 35%) 2024-08-07T19:25:47.2505997Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-70a23b9a91744dcb.xml (deflated 35%) 2024-08-07T19:25:47.2507212Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5a73de862fcea2bf.xml (deflated 35%) 2024-08-07T19:25:47.2508414Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f82d06d774a0d0b8.xml (deflated 42%) 2024-08-07T19:25:47.2509630Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ffdf47ff44e246e6.xml (deflated 42%) 2024-08-07T19:25:47.2510845Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-97cde5a3156c1aa7.xml (deflated 42%) 2024-08-07T19:25:47.2512068Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bc2d7aff4dd47dab.xml (deflated 43%) 2024-08-07T19:25:47.2513277Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-78de290df7de9a2b.xml (deflated 36%) 2024-08-07T19:25:47.2514487Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6432732c30590c27.xml (deflated 41%) 2024-08-07T19:25:47.2515697Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-68b41241ea28a384.xml (deflated 42%) 2024-08-07T19:25:47.2516914Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1d5354241c004829.xml (deflated 42%) 2024-08-07T19:25:47.2518110Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-45043a6d803650e5.xml (deflated 35%) 2024-08-07T19:25:47.2519382Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6ee913b90469c900.xml (deflated 36%) 2024-08-07T19:25:47.2520598Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8da0ac45d4848ca1.xml (deflated 35%) 2024-08-07T19:25:47.2521813Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b691da4397bf6cbd.xml (deflated 35%) 2024-08-07T19:25:47.2523014Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4fd93af965043c72.xml (deflated 36%) 2024-08-07T19:25:47.2524228Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6a3cf21926580491.xml (deflated 45%) 2024-08-07T19:25:47.2525442Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-819906cdd1aee82f.xml (deflated 36%) 2024-08-07T19:25:47.2526638Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-714b393b727e440f.xml (deflated 35%) 2024-08-07T19:25:47.2527849Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5447c6c967d16f9c.xml (deflated 36%) 2024-08-07T19:25:47.2529067Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2e4db63de2c44ba5.xml (deflated 35%) 2024-08-07T19:25:47.2530284Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2ccd467dd8cacabc.xml (deflated 36%) 2024-08-07T19:25:47.2531549Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f5aff22b219d8e0e.xml (deflated 35%) 2024-08-07T19:25:47.2532766Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-341da451b964dab6.xml (deflated 35%) 2024-08-07T19:25:47.2533976Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a07612d163a272a9.xml (deflated 41%) 2024-08-07T19:25:47.2535241Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-26f44cdb61ed927c.xml (deflated 43%) 2024-08-07T19:25:47.2536442Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d8ae823135750afb.xml (deflated 44%) 2024-08-07T19:25:47.2537659Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a6771eea59454119.xml (deflated 44%) 2024-08-07T19:25:47.2538867Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-2a7a88ece9d0c429.xml (deflated 44%) 2024-08-07T19:25:47.2540229Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b66dd2720bcccf72.xml (deflated 45%) 2024-08-07T19:25:47.2541434Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ed18aee49614598a.xml (deflated 42%) 2024-08-07T19:25:47.2542653Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3ec671ea2be60aa6.xml (deflated 36%) 2024-08-07T19:25:47.2543866Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-d83c8c273911172b.xml (deflated 42%) 2024-08-07T19:25:47.2545084Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3abd1c7f0826eadb.xml (deflated 36%) 2024-08-07T19:25:47.2546282Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bce8f21f5fb8e442.xml (deflated 35%) 2024-08-07T19:25:47.2547495Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-14453962118a7275.xml (deflated 41%) 2024-08-07T19:25:47.2548695Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-13d0fe68d601b186.xml (deflated 42%) 2024-08-07T19:25:47.2549887Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-39ea5c0e7d003617.xml (deflated 41%) 2024-08-07T19:25:47.2551096Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-49706ef089ceff76.xml (deflated 35%) 2024-08-07T19:25:47.2552305Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-37c6144e295df06c.xml (deflated 41%) 2024-08-07T19:25:47.2553517Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-424ede4f6ba78418.xml (deflated 42%) 2024-08-07T19:25:47.2554714Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b85cd67cd5f422f2.xml (deflated 42%) 2024-08-07T19:25:47.2555927Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ca3e6a406f66babe.xml (deflated 42%) 2024-08-07T19:25:47.2557143Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-32bc4e519ca30115.xml (deflated 42%) 2024-08-07T19:25:47.2558397Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a6c5e354c89bb167.xml (deflated 42%) 2024-08-07T19:25:47.2559699Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-cdd28b75c88fe62a.xml (deflated 43%) 2024-08-07T19:25:47.2560915Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-209dbbc1c33f5289.xml (deflated 42%) 2024-08-07T19:25:47.2562119Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-5f1d381681303b92.xml (deflated 42%) 2024-08-07T19:25:47.2563321Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-428a57671531a7d8.xml (deflated 41%) 2024-08-07T19:25:47.2564594Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c57a51c1e869d9cb.xml (deflated 42%) 2024-08-07T19:25:47.2565817Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-f9308ea5c83c7f50.xml (deflated 43%) 2024-08-07T19:25:47.2567031Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9b0917203ace2ac6.xml (deflated 42%) 2024-08-07T19:25:47.2568241Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3d01ccaa8a67233b.xml (deflated 43%) 2024-08-07T19:25:47.2569445Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b4af85ed537b4699.xml (deflated 41%) 2024-08-07T19:25:47.2570660Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-3e73fd4fa2640df2.xml (deflated 42%) 2024-08-07T19:25:47.2571882Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-788c08dba91df706.xml (deflated 42%) 2024-08-07T19:25:47.2573086Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b80247d6fc21464d.xml (deflated 42%) 2024-08-07T19:25:47.2574307Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ccc323ea9ef50f82.xml (deflated 42%) 2024-08-07T19:25:47.2575526Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-62d00410d5537bfe.xml (deflated 42%) 2024-08-07T19:25:47.2576730Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-70ea492de6478a9d.xml (deflated 42%) 2024-08-07T19:25:47.2595069Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-796d5daff8842887.xml (deflated 43%) 2024-08-07T19:25:47.2596392Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-beb52ca84fd92d2e.xml (deflated 43%) 2024-08-07T19:25:47.2597653Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ae4164d0248ead98.xml (deflated 42%) 2024-08-07T19:25:47.2599023Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-0c3bf18a59583f0b.xml (deflated 36%) 2024-08-07T19:25:47.2600258Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fc73355603b95fd7.xml (deflated 35%) 2024-08-07T19:25:47.2601455Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6694fd2e04fa9e6b.xml (deflated 35%) 2024-08-07T19:25:47.2602660Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c4faac8a3bd41d31.xml (deflated 35%) 2024-08-07T19:25:47.2603864Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ac132c29df984d83.xml (deflated 42%) 2024-08-07T19:25:47.2605046Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-372045e3f1798854.xml (deflated 35%) 2024-08-07T19:25:47.2606366Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fdee94878b221654.xml (deflated 44%) 2024-08-07T19:25:47.2607568Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-566a88c0da70ee57.xml (deflated 45%) 2024-08-07T19:25:47.2608756Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-634a6f0751b3acfd.xml (deflated 45%) 2024-08-07T19:25:47.2609959Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b9a0c91851e620be.xml (deflated 41%) 2024-08-07T19:25:47.2611226Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-294fd171c90b9a7e.xml (deflated 42%) 2024-08-07T19:25:47.2612420Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-19c8d150fcf03053.xml (deflated 42%) 2024-08-07T19:25:47.2613608Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1c296ceb0702ad09.xml (deflated 42%) 2024-08-07T19:25:47.2614809Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-38c2e0b7597f4061.xml (deflated 41%) 2024-08-07T19:25:47.2616000Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7535869cd8928382.xml (deflated 41%) 2024-08-07T19:25:47.2617186Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-453c4e1c5232ad02.xml (deflated 35%) 2024-08-07T19:25:47.2618383Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-4bf6581dc0f05d1f.xml (deflated 35%) 2024-08-07T19:25:47.2619577Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-979f4f92b41d2841.xml (deflated 35%) 2024-08-07T19:25:47.2620762Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fc426bd06c17f6a1.xml (deflated 35%) 2024-08-07T19:25:47.2621946Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-c29e162b532fd9f3.xml (deflated 45%) 2024-08-07T19:25:47.2623141Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-a0d877ef4848e1e8.xml (deflated 43%) 2024-08-07T19:25:47.2624335Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-41f27f50747bc600.xml (deflated 37%) 2024-08-07T19:25:47.2625524Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-20eda8612433cd7c.xml (deflated 36%) 2024-08-07T19:25:47.2626714Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-6f4cb18d8f6f1635.xml (deflated 35%) 2024-08-07T19:25:47.2627913Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-7db564dc26834950.xml (deflated 42%) 2024-08-07T19:25:47.2629110Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-41cf9f68e6cb81ad.xml (deflated 35%) 2024-08-07T19:25:47.2630304Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-b66511ac7633d692.xml (deflated 35%) 2024-08-07T19:25:47.2631495Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-bbe0d3f882a33756.xml (deflated 36%) 2024-08-07T19:25:47.2632686Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-ccb166133bc627f0.xml (deflated 35%) 2024-08-07T19:25:47.2633888Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-fe074d5703f7c499.xml (deflated 35%) 2024-08-07T19:25:47.2635148Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-8cbafd9243c33db0.xml (deflated 35%) 2024-08-07T19:25:47.2636340Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-1e824ebe3b9b0f82.xml (deflated 42%) 2024-08-07T19:25:47.2637544Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-9a727cc5cd3aac2d.xml (deflated 42%) 2024-08-07T19:25:47.2638860Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-04ef8ce3c3b0f0b7.xml (deflated 43%) 2024-08-07T19:25:47.2640294Z adding: test/test-reports/dist-gloo/distributed.test_distributed_spawn/distributed.test_distributed_spawn-610561beb7366322.xml (deflated 43%) 2024-08-07T19:25:47.2641710Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-20df4b439acff5c9.xml (deflated 38%) 2024-08-07T19:25:47.2643351Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-b1d722100df50c14.xml (deflated 38%) 2024-08-07T19:25:47.2644983Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-b87e5e5cfe03fb45.xml (deflated 48%) 2024-08-07T19:25:47.2646595Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-1f2df4b61d50db03.xml (deflated 48%) 2024-08-07T19:25:47.2648220Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-ec5c681387b0bf6b.xml (deflated 48%) 2024-08-07T19:25:47.2649847Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-a21def2499449f04.xml (deflated 48%) 2024-08-07T19:25:47.2651474Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-a6caacca9c5f312c.xml (deflated 38%) 2024-08-07T19:25:47.2653100Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-f2565a38c1202ca0.xml (deflated 37%) 2024-08-07T19:25:47.2654712Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-d20a19f292ecb8c7.xml (deflated 48%) 2024-08-07T19:25:47.2656325Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-d6f9424ed227114f.xml (deflated 48%) 2024-08-07T19:25:47.2657943Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-578695bb3a90776d.xml (deflated 48%) 2024-08-07T19:25:47.2659562Z adding: test/test-reports/dist-gloo/distributed.algorithms.quantization.test_quantization/distributed.algorithms.quantization.test_quantization-ad08f7d152aa6965.xml (deflated 48%) 2024-08-07T19:25:47.2688557Z ##[group]Run # Remove any previous usage logs if they exist 2024-08-07T19:25:47.2689026Z # Remove any previous usage logs if they exist 2024-08-07T19:25:47.2689396Z rm -f logs-*.zip 2024-08-07T19:25:47.2689864Z # this workflow is also run in bazel build test, but we dont generate usage reports for it 2024-08-07T19:25:47.2690408Z # so check to see if the file exists first 2024-08-07T19:25:47.2690766Z if [ -f 'usage_log.txt' ]; then 2024-08-07T19:25:47.2691127Z  zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt' 2024-08-07T19:25:47.2691576Z fi 2024-08-07T19:25:47.2691840Z if ls test/**/*.log 1> /dev/null 2>&1; then 2024-08-07T19:25:47.2692226Z  zip -r "logs-${FILE_SUFFIX}.zip" test -i '*.log' 2024-08-07T19:25:47.2692571Z fi 2024-08-07T19:25:47.2698079Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:25:47.2698443Z env: 2024-08-07T19:25:47.2698667Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:25:47.2699229Z DOCKER_CONTAINER_ID: 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:25:47.2699840Z FILE_SUFFIX: test-distributed-1-2-amz2023.linux.2xlarge_28475634164 2024-08-07T19:25:47.2700290Z ##[endgroup] 2024-08-07T19:25:47.2793288Z adding: usage_log.txt (deflated 92%) 2024-08-07T19:25:47.3046691Z adding: test/test-reports/distributed._tensor.test_dtensor_compile_1.1_27403634dda4d062_.log (deflated 85%) 2024-08-07T19:25:47.3047715Z adding: test/test-reports/distributed.pipelining.test_backward_1.1_723b03cf90bd92c4_.log (deflated 54%) 2024-08-07T19:25:47.3048651Z adding: test/test-reports/distributed.checkpoint.test_nested_dict_1.1_9deabd4db0d22dd1_.log (deflated 59%) 2024-08-07T19:25:47.3049637Z adding: test/test-reports/distributed._shard.sharded_tensor.test_logger_1.1_654f904899cf6944_.log (deflated 55%) 2024-08-07T19:25:47.3050639Z adding: test/test-reports/distributed.pipelining.test_transformer_1.1_bef753c7e8696009_.log (deflated 54%) 2024-08-07T19:25:47.3051507Z adding: test/test-reports/distributed.test_symmetric_memory_1.1_891cba56055c2a0e_.log (deflated 86%) 2024-08-07T19:25:47.3052471Z adding: test/test-reports/distributed.tensor.parallel.test_micro_pipeline_tp_1.1_d842afa1d972c0bb_.log (deflated 91%) 2024-08-07T19:25:47.3053407Z adding: test/test-reports/distributed.test_dynamo_distributed_1.1_d5a7cdfd0c4bf804_.log (deflated 88%) 2024-08-07T19:25:47.3054799Z adding: test/test-reports/distributed.fsdp.test_fsdp_core_1.1_7a0f40f279bdb086_.log (deflated 92%) 2024-08-07T19:25:47.3055745Z adding: test/test-reports/distributed.pipelining.test_schedule_multiproc_1.1_5c816662d6e371a8_.log (deflated 2%) 2024-08-07T19:25:47.3056641Z adding: test/test-reports/distributed._composable.test_contract_1.1_280d7e24038acd5b_.log (deflated 67%) 2024-08-07T19:25:47.3057544Z adding: test/test-reports/distributed.test_device_mesh_1.1_887754677d247eaf_.log (deflated 83%) 2024-08-07T19:25:47.3064736Z adding: test/test-reports/distributed._tensor.test_tensor_ops_1.1_25043569cb7d2846_.log (deflated 95%) 2024-08-07T19:25:47.3065671Z adding: test/test-reports/distributed._tensor.test_random_ops_1.1_d09b38c08bcf30fc_.log (deflated 75%) 2024-08-07T19:25:47.3066728Z adding: test/test-reports/distributed.elastic.timer.local_timer_example_1.1_bfdbafa1fbffceec_.log (deflated 67%) 2024-08-07T19:25:47.3067818Z adding: test/test-reports/distributed._tensor.experimental.test_register_sharding_1.1_2a8ab8905cc47b76_.log (deflated 77%) 2024-08-07T19:25:47.3068919Z adding: test/test-reports/distributed.tensor.parallel.test_tp_examples_1.1_eca9453f07e52aaf_.log (deflated 87%) 2024-08-07T19:25:47.3069805Z adding: test/test-reports/distributed._tensor.test_math_ops_1.1_de716607eb414837_.log (deflated 89%) 2024-08-07T19:25:47.3070725Z adding: test/test-reports/distributed.optim.test_named_optimizer_1.1_cd04f844150c9dc3_.log (deflated 34%) 2024-08-07T19:25:47.3071594Z adding: test/test-reports/distributed._tensor.test_redistribute_1.1_483d96b06839d388_.log (deflated 92%) 2024-08-07T19:25:47.3072614Z adding: test/test-reports/distributed.checkpoint.test_format_utils_1.1_54e74efdf0f6c1ec_.log (deflated 86%) 2024-08-07T19:25:47.3073657Z adding: test/test-reports/distributed._tensor.experimental.test_tp_transform_1.1_5dea6052b71ee941_.log (deflated 65%) 2024-08-07T19:25:47.3074732Z adding: test/test-reports/distributed._tensor.experimental.test_local_map_1.1_4d11346e47505839_.log (deflated 78%) 2024-08-07T19:25:47.3075715Z adding: test/test-reports/distributed.checkpoint.test_fsdp_optim_state_1.1_6f380acf678099bb_.log (deflated 62%) 2024-08-07T19:25:47.3076961Z adding: test/test-reports/distributed.checkpoint.test_dtensor_resharding_1.1_72c25f9e5b4d5dd9_.log (deflated 72%) 2024-08-07T19:25:47.3077889Z adding: test/test-reports/distributed.checkpoint.test_tp_checkpoint_1.1_597ac79be7499ade_.log (deflated 60%) 2024-08-07T19:25:47.3078846Z adding: test/test-reports/distributed._tensor.test_pointwise_ops_1.1_fe12ae9a6e8b8056_.log (deflated 69%) 2024-08-07T19:25:47.3079709Z adding: test/test-reports/distributed.checkpoint.test_fsspec_1.1_a928051542ebbc5e_.log (deflated 58%) 2024-08-07T19:25:47.3080543Z adding: test/test-reports/distributed.test_collective_utils_1.1_075eb1f3e483964d_.log (stored 0%) 2024-08-07T19:25:47.3083838Z adding: test/test-reports/distributed.fsdp.test_fsdp_optim_state_1.1_55cb4daf6c7c2cce_.log (deflated 97%) 2024-08-07T19:25:47.3084894Z adding: test/test-reports/distributed.elastic.timer.local_timer_test_1.1_18df99314526bda1_.log (deflated 78%) 2024-08-07T19:25:47.3085945Z adding: test/test-reports/distributed.checkpoint.test_fsdp_tp_checkpoint_conversion_1.1_8c88faaa58245cf6_.log (deflated 57%) 2024-08-07T19:25:47.3086991Z adding: test/test-reports/distributed.test_fake_pg_1.1_af9b709a23444167_.log (deflated 74%) 2024-08-07T19:25:47.3087798Z adding: test/test-reports/distributed._tensor.debug.test_comm_mode_1.1_84ccf848e8d8ba3f_.log (deflated 62%) 2024-08-07T19:25:47.3088801Z adding: test/test-reports/distributed.checkpoint.test_utils_1.1_83acbd0140a20a79_.log (deflated 66%) 2024-08-07T19:25:47.3089825Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_init_1.1_8516b3de56b3a2b1_.log (deflated 87%) 2024-08-07T19:25:47.3090771Z adding: test/test-reports/distributed.test_c10d_logger_1.1_d2c36dbc8d014b65_.log (deflated 60%) 2024-08-07T19:25:47.3091587Z adding: test/test-reports/distributed.fsdp.test_shard_utils_1.1_91e612eaea63a7b3_.log (deflated 59%) 2024-08-07T19:25:47.3092428Z adding: test/test-reports/distributed.test_nccl_1.1_fad9c73172d8063a_.log (deflated 49%) 2024-08-07T19:25:47.3093296Z adding: test/test-reports/distributed.rpc.test_share_memory_1.1_829428c73b4beb81_.log (deflated 56%) 2024-08-07T19:25:47.3094164Z adding: test/test-reports/distributed.checkpoint.test_fsdp_model_state_1.1_ecb5c83af14f216b_.log (deflated 61%) 2024-08-07T19:25:47.3095181Z adding: test/test-reports/distributed.tensor.parallel.test_parallelize_api_1.1_a840e28441e9632e_.log (deflated 91%) 2024-08-07T19:25:47.3096193Z adding: test/test-reports/distributed.test_control_collectives_1.1_f2425810dab4c571_.log (deflated 78%) 2024-08-07T19:25:47.3097130Z adding: test/test-reports/distributed.elastic.multiprocessing.api_test_1.1_6485352c97c63668_.log (deflated 88%) 2024-08-07T19:25:47.3098148Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_clip_grad_norm__1.1_550162c2b93d941a_.log (deflated 63%) 2024-08-07T19:25:47.3099104Z adding: test/test-reports/distributed.test_distributed_spawn_1.3_e8697588080e6458_.log (deflated 37%) 2024-08-07T19:25:47.3099952Z adding: test/test-reports/distributed.test_distributed_spawn_1.3_56ed6e9c8f7c449c_.log (deflated 37%) 2024-08-07T19:25:47.3116049Z adding: test/test-reports/distributed.test_distributed_spawn_1.3_24c56f5018b6d515_.log (deflated 97%) 2024-08-07T19:25:47.3135266Z adding: test/test-reports/distributed.test_distributed_spawn_1.3_f715dc2d6b2e8796_.log (deflated 97%) 2024-08-07T19:25:47.3136159Z adding: test/test-reports/distributed.test_distributed_spawn_3.3_f9a093cd0ad5201f_.log (deflated 37%) 2024-08-07T19:25:47.3137003Z adding: test/test-reports/distributed.test_distributed_spawn_3.3_5e816eef6d3cec2d_.log (deflated 37%) 2024-08-07T19:25:47.3153659Z adding: test/test-reports/distributed.test_distributed_spawn_3.3_3da988a817a08221_.log (deflated 97%) 2024-08-07T19:25:47.3170063Z adding: test/test-reports/distributed.test_distributed_spawn_3.3_f9d042a87186e77f_.log (deflated 97%) 2024-08-07T19:25:47.3171150Z adding: test/test-reports/distributed.fsdp.test_fsdp_backward_prefetch_1.1_351dec4e2a650c5f_.log (deflated 55%) 2024-08-07T19:25:47.3172293Z adding: test/test-reports/distributed.elastic.utils.logging_test_1.1_791637ad493fbd81_.log (deflated 58%) 2024-08-07T19:25:47.3173227Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_autograd_1.1_d4932b7054b1f879_.log (deflated 72%) 2024-08-07T19:25:47.3174520Z adding: test/test-reports/distributed.elastic.utils.distributed_test_1.1_4e94509edd919cc8_.log (deflated 93%) 2024-08-07T19:25:47.3175808Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_training_1.1_e45121bd932f0b1b_.log (deflated 83%) 2024-08-07T19:25:47.3177777Z adding: test/test-reports/distributed.fsdp.test_fsdp_freezing_weights_1.1_f1c3af21d7e916ae_.log (deflated 93%) 2024-08-07T19:25:47.3180493Z adding: test/test-reports/distributed.optim.test_zero_redundancy_optimizer_1.1_e046fda781b36268_.log (deflated 94%) 2024-08-07T19:25:47.3181629Z adding: test/test-reports/distributed.elastic.utils.util_test_1.1_7f784de2e35773ac_.log (deflated 75%) 2024-08-07T19:25:47.3182649Z adding: test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_f0cf8fcd71b4beea_.log (deflated 14%) 2024-08-07T19:25:47.3183684Z adding: test/test-reports/distributed.pipelining.test_composability_1.1_aaca4c3bcb2d8d7d_.log (deflated 3%) 2024-08-07T19:25:47.3184730Z adding: test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_c3c4165a9ebc2baf_.log (deflated 14%) 2024-08-07T19:25:47.3185833Z adding: test/test-reports/distributed.tensor.parallel.test_ddp_2d_parallel_1.1_c5cf0db5ffa1de43_.log (deflated 56%) 2024-08-07T19:25:47.3186906Z adding: test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_955157289cde159c_.log (deflated 90%) 2024-08-07T19:25:47.3188014Z adding: test/test-reports/distributed.tensor.parallel.test_fsdp_2d_parallel_1.1_9f3565ab995c86f6_.log (deflated 82%) 2024-08-07T19:25:47.3189120Z adding: test/test-reports/distributed.algorithms.quantization.test_quantization_1.1_c79bcd2b698e8e88_.log (deflated 90%) 2024-08-07T19:25:47.3190143Z adding: test/test-reports/distributed.test_data_parallel_1.1_f6fb3bf1d57e6261_.log (deflated 89%) 2024-08-07T19:25:47.3191012Z adding: test/test-reports/distributed.algorithms.test_join_1.1_4542232bb5dc16ee_.log (deflated 81%) 2024-08-07T19:25:47.3191928Z adding: test/test-reports/distributed.fsdp.test_checkpoint_wrapper_1.1_ef2a42f6809847b1_.log (deflated 74%) 2024-08-07T19:25:47.3192942Z adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_comm_1.1_b5e7376268557eb1_.log (deflated 82%) 2024-08-07T19:25:47.3193846Z adding: test/test-reports/distributed.fsdp.test_fsdp_input_1.1_45b5f72a1e855bb5_.log (deflated 58%) 2024-08-07T19:25:47.3194731Z adding: test/test-reports/distributed.fsdp.test_fsdp_uneven_1.1_6069062bff1932ca_.log (deflated 54%) 2024-08-07T19:25:47.3195691Z adding: test/test-reports/distributed.fsdp.test_fsdp_multiple_wrapping_1.1_527c483bd4c903ca_.log (deflated 56%) 2024-08-07T19:25:47.3196548Z adding: test/test-reports/distributed.fsdp.test_utils_1.1_a669c6dbcfa7fcad_.log (deflated 64%) 2024-08-07T19:25:47.3197344Z adding: test/test-reports/distributed.fsdp.test_fsdp_misc_1.1_ca9e7465b2abbdf1_.log (deflated 91%) 2024-08-07T19:25:47.3198168Z adding: test/test-reports/distributed.fsdp.test_fsdp_checkpoint_1.1_cd17babaaa9c57fe_.log (deflated 87%) 2024-08-07T19:25:47.3199242Z adding: test/test-reports/distributed.fsdp.test_fsdp_exec_order_1.1_697efbc3812aa66b_.log (deflated 80%) 2024-08-07T19:25:47.3200100Z adding: test/test-reports/distributed.fsdp.test_fsdp_fine_tune_1.1_2a4d3f932e1f1fa5_.log (deflated 68%) 2024-08-07T19:25:47.3200957Z adding: test/test-reports/distributed.test_c10d_common_1.1_cb7635dfc3ee7d07_.log (deflated 94%) 2024-08-07T19:25:47.3201892Z adding: test/test-reports/distributed._shard.sharding_plan.test_sharding_plan_1.1_406e68116fb5644f_.log (deflated 66%) 2024-08-07T19:25:47.3202875Z adding: test/test-reports/distributed.checkpoint.test_state_dict_1.1_36f056619d6de226_.log (deflated 87%) 2024-08-07T19:25:47.3203821Z adding: test/test-reports/distributed._shard.test_sharder_1.1_e61db70520c65da5_.log (deflated 59%) 2024-08-07T19:25:47.3204614Z adding: test/test-reports/distributed.fsdp.test_fsdp_meta_1.1_f8ea1dae9bdc541e_.log (deflated 84%) 2024-08-07T19:25:47.3209913Z adding: test/test-reports/distributed.rpc.test_faulty_agent_1.1_b0889ba0258f7816_.log (deflated 96%) 2024-08-07T19:25:47.3245450Z adding: test/test-reports/distributed.rpc.test_tensorpipe_agent_2.4_3c73dc1b9a6663fe_.log (deflated 97%) 2024-08-07T19:25:47.3279003Z adding: test/test-reports/distributed.rpc.test_tensorpipe_agent_4.4_753ab7ef8e7087b4_.log (deflated 97%) 2024-08-07T19:25:47.3280224Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_init_1.1_5ac4b6c2113a40c7_.log (deflated 67%) 2024-08-07T19:25:47.3281179Z adding: test/test-reports/distributed._shard.sharded_optim.test_sharded_optim_1.1_938a488d10343518_.log (deflated 62%) 2024-08-07T19:25:47.3282164Z adding: test/test-reports/distributed.test_c10d_spawn_gloo_1.1_69483467310e45eb_.log (deflated 94%) 2024-08-07T19:25:47.3283069Z adding: test/test-reports/distributed.fsdp.test_fsdp_comm_1.1_9ae0a98a1384c4f7_.log (deflated 81%) 2024-08-07T19:25:47.3284130Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_1.1_dd71ca186388b9df_.log (deflated 62%) 2024-08-07T19:25:47.3285178Z adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_bag_1.1_8b9deb8faf643d86_.log (deflated 63%) 2024-08-07T19:25:47.3286099Z adding: test/test-reports/distributed.elastic.timer.api_test_1.1_354d8207446e8e82_.log (stored 0%) 2024-08-07T19:25:47.3287043Z adding: test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_reshard_1.1_faf5fb214e871656_.log (deflated 64%) 2024-08-07T19:25:47.3294904Z adding: test/test-reports/distributed.test_store_1.1_3d473ec3bcb95972_.log (deflated 95%) 2024-08-07T19:25:47.3296108Z adding: test/test-reports/distributed.test_c10d_spawn_ucc_1.1_acd3d13d6c908e53_.log (deflated 92%) 2024-08-07T19:25:47.3297824Z adding: test/test-reports/distributed.test_c10d_spawn_nccl_1.1_bab5ea31e545e423_.log (deflated 93%) 2024-08-07T19:25:47.3298648Z adding: test/test-reports/distributed.test_c10d_nccl_1.1_353930d0561f4846_.log (stored 0%) 2024-08-07T19:25:47.3299334Z adding: test/test-reports/cpp.test_cpp_rpc_1.1_2eaa93faec77983f_.log (deflated 78%) 2024-08-07T19:25:47.3301092Z adding: test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_1.1_6a8da622b7bb0a90_.log (deflated 91%) 2024-08-07T19:25:47.3302115Z adding: test/test-reports/distributed.test_launcher_1.1_59596ed3b15e7495_.log (deflated 58%) 2024-08-07T19:25:47.3302965Z adding: test/test-reports/distributed.test_c10d_ops_nccl_1.1_ce98a0c8fa7ee0c5_.log (stored 0%) 2024-08-07T19:25:47.3303764Z adding: test/test-reports/distributed.elastic.events.lib_test_1.1_508233a75c193c46_.log (deflated 73%) 2024-08-07T19:25:47.3304624Z adding: test/test-reports/distributed.elastic.metrics.api_test_1.1_7cd4a2b37b4bca4e_.log (deflated 62%) 2024-08-07T19:25:47.3334452Z ##[group]Run # Remove any previous debugging artifacts if they exist 2024-08-07T19:25:47.3334997Z # Remove any previous debugging artifacts if they exist 2024-08-07T19:25:47.3335403Z rm -f debug-*.zip 2024-08-07T19:25:47.3335701Z if [ -d 'test/debug' ]; then 2024-08-07T19:25:47.3336068Z  zip -r "debug-${FILE_SUFFIX}.zip" test/debug 2024-08-07T19:25:47.3336406Z fi 2024-08-07T19:25:47.3342032Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:25:47.3342426Z env: 2024-08-07T19:25:47.3342642Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:25:47.3343126Z DOCKER_CONTAINER_ID: 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:25:47.3343759Z FILE_SUFFIX: test-distributed-1-2-amz2023.linux.2xlarge_28475634164 2024-08-07T19:25:47.3344175Z ##[endgroup] 2024-08-07T19:25:47.3424015Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T19:25:47.3424491Z with: 2024-08-07T19:25:47.3424708Z s3-bucket: gha-artifacts 2024-08-07T19:25:47.3425042Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T19:25:47.3425407Z retention-days: 14 2024-08-07T19:25:47.3425659Z if-no-files-found: warn 2024-08-07T19:25:47.3425942Z path: test-jsons-*.zip 2024-08-07T19:25:47.3426212Z name: artifact 2024-08-07T19:25:47.3426441Z region: us-east-1 2024-08-07T19:25:47.3426682Z env: 2024-08-07T19:25:47.3426912Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:25:47.3427362Z DOCKER_CONTAINER_ID: 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:25:47.3427868Z ##[endgroup] 2024-08-07T19:25:47.6979680Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-07T19:25:47.6980416Z With the provided path, there will be 1 file uploaded 2024-08-07T19:25:47.6980916Z Uploading to s3 prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T19:25:47.7020288Z Starting upload of test-jsons-test-distributed-1-2-amz2023.linux.2xlarge_28475634164.zip 2024-08-07T19:25:47.8413847Z Finished upload of test-jsons-test-distributed-1-2-amz2023.linux.2xlarge_28475634164.zip 2024-08-07T19:25:47.8566699Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T19:25:47.8567067Z with: 2024-08-07T19:25:47.8567290Z s3-bucket: gha-artifacts 2024-08-07T19:25:47.8567629Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T19:25:47.8567992Z retention-days: 14 2024-08-07T19:25:47.8568246Z if-no-files-found: error 2024-08-07T19:25:47.8568538Z path: test-reports-*.zip 2024-08-07T19:25:47.8568812Z name: artifact 2024-08-07T19:25:47.8569037Z region: us-east-1 2024-08-07T19:25:47.8569276Z env: 2024-08-07T19:25:47.8569496Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:25:47.8569959Z DOCKER_CONTAINER_ID: 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:25:47.8570461Z ##[endgroup] 2024-08-07T19:25:48.1967758Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-07T19:25:48.1968430Z With the provided path, there will be 1 file uploaded 2024-08-07T19:25:48.1968928Z Uploading to s3 prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T19:25:48.2008042Z Starting upload of test-reports-test-distributed-1-2-amz2023.linux.2xlarge_28475634164.zip 2024-08-07T19:25:48.3836308Z Finished upload of test-reports-test-distributed-1-2-amz2023.linux.2xlarge_28475634164.zip 2024-08-07T19:25:48.3995229Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T19:25:48.3995606Z with: 2024-08-07T19:25:48.3995841Z s3-bucket: gha-artifacts 2024-08-07T19:25:48.3996183Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T19:25:48.3996561Z retention-days: 14 2024-08-07T19:25:48.3996821Z if-no-files-found: ignore 2024-08-07T19:25:48.3997124Z path: logs-*.zip 2024-08-07T19:25:48.3997371Z name: artifact 2024-08-07T19:25:48.3997599Z region: us-east-1 2024-08-07T19:25:48.3997841Z env: 2024-08-07T19:25:48.3998068Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:25:48.3998613Z DOCKER_CONTAINER_ID: 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:25:48.3999132Z ##[endgroup] 2024-08-07T19:25:48.7536228Z NOTE: s3-prefix specified, ignoring name parameter 2024-08-07T19:25:48.7536861Z With the provided path, there will be 1 file uploaded 2024-08-07T19:25:48.7537338Z Uploading to s3 prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T19:25:48.7577988Z Starting upload of logs-test-distributed-1-2-amz2023.linux.2xlarge_28475634164.zip 2024-08-07T19:25:48.9193806Z Finished upload of logs-test-distributed-1-2-amz2023.linux.2xlarge_28475634164.zip 2024-08-07T19:25:48.9347884Z ##[group]Run seemethere/upload-artifact-s3@v5 2024-08-07T19:25:48.9348242Z with: 2024-08-07T19:25:48.9348467Z s3-bucket: gha-artifacts 2024-08-07T19:25:48.9348831Z s3-prefix: pytorch/pytorch/10288745067/1/artifact 2024-08-07T19:25:48.9349201Z retention-days: 14 2024-08-07T19:25:48.9349453Z if-no-files-found: ignore 2024-08-07T19:25:48.9349738Z path: debug-*.zip 2024-08-07T19:25:48.9349984Z name: artifact 2024-08-07T19:25:48.9350341Z region: us-east-1 2024-08-07T19:25:48.9350584Z env: 2024-08-07T19:25:48.9350809Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:25:48.9351261Z DOCKER_CONTAINER_ID: 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:25:48.9351761Z ##[endgroup] 2024-08-07T19:25:49.2707070Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded. 2024-08-07T19:25:49.2862955Z ##[group]Run # shellcheck disable=SC2156 2024-08-07T19:25:49.2863336Z # shellcheck disable=SC2156 2024-08-07T19:25:49.2863943Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \; 2024-08-07T19:25:49.2869792Z shell: /usr/bin/bash -e {0} 2024-08-07T19:25:49.2870065Z env: 2024-08-07T19:25:49.2870295Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:25:49.2870765Z DOCKER_CONTAINER_ID: 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:25:49.2871257Z ##[endgroup] 2024-08-07T19:25:49.5094792Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main 2024-08-07T19:25:49.5095232Z with: 2024-08-07T19:25:49.5095457Z env: 2024-08-07T19:25:49.5095691Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:25:49.5096154Z DOCKER_CONTAINER_ID: 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:25:49.5096666Z ##[endgroup] 2024-08-07T19:25:49.5121807Z ##[group]Run set -eou pipefail 2024-08-07T19:25:49.5122140Z set -eou pipefail 2024-08-07T19:25:49.5122405Z  2024-08-07T19:25:49.5122794Z echo "Holding runner for 2 hours until all ssh sessions have logged out" 2024-08-07T19:25:49.5123278Z for _ in $(seq 1440); do 2024-08-07T19:25:49.5123612Z  # Break if no ssh session exists anymore 2024-08-07T19:25:49.5123976Z  if [ "$(who)" = "" ]; then 2024-08-07T19:25:49.5124287Z  break 2024-08-07T19:25:49.5124518Z  fi 2024-08-07T19:25:49.5124804Z  echo "." 2024-08-07T19:25:49.5125061Z  sleep 5 2024-08-07T19:25:49.5125310Z done 2024-08-07T19:25:49.5130765Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:25:49.5131142Z env: 2024-08-07T19:25:49.5131370Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:25:49.5131840Z DOCKER_CONTAINER_ID: 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:25:49.5132329Z ##[endgroup] 2024-08-07T19:25:49.5154261Z Holding runner for 2 hours until all ssh sessions have logged out 2024-08-07T19:25:49.5227942Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T19:25:49.5228521Z # ignore expansion of "docker ps -q" since it could be empty 2024-08-07T19:25:49.5228952Z # shellcheck disable=SC2046 2024-08-07T19:25:49.5229305Z docker stop $(docker ps -q) || true 2024-08-07T19:25:49.5229666Z # Prune all of the docker images 2024-08-07T19:25:49.5229996Z docker system prune -af 2024-08-07T19:25:49.5235582Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:25:49.5235977Z env: 2024-08-07T19:25:49.5236197Z GIT_DEFAULT_BRANCH: main 2024-08-07T19:25:49.5236668Z DOCKER_CONTAINER_ID: 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:25:49.5237172Z ##[endgroup] 2024-08-07T19:26:00.0422370Z 595390c2978c 2024-08-07T19:26:00.3799914Z Deleted Containers: 2024-08-07T19:26:00.3800406Z 595390c2978c095a01090de14a4b073eb7c3c89af1527c6a8bf0909321ce528c 2024-08-07T19:26:00.3800759Z 2024-08-07T19:26:03.4385044Z Deleted Images: 2024-08-07T19:26:03.4386236Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11:02ec4fbd5adcb3fb91cf5ce431dec18b633de7d9 2024-08-07T19:26:03.4387521Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.8-gcc11@sha256:c75221497aa22766802555fccb76014708414eebd47578b213b43b3f8f4cddb9 2024-08-07T19:26:03.4388571Z deleted: sha256:eb2e8636b63e804e38c5e56190d8b860c4bf8a9e0794f88825eff958ed6b1e76 2024-08-07T19:26:03.4389456Z deleted: sha256:3a75aa289a82638e31ac4b969360c49c8013f3546a449692cb458fcde3935f22 2024-08-07T19:26:03.4390116Z deleted: sha256:bdfa3e02d65d776f3bb70b6bf71d852ddca4c1dfbdca2356544996f07fef2ed8 2024-08-07T19:26:03.4390781Z deleted: sha256:6a85078ea3791ab81c962cdfc07c05a8b7eb4c5576721c3b56b45f4adf3fd0d1 2024-08-07T19:26:03.4391440Z deleted: sha256:935ccaf7b02ef4b0aabe934f8ed13eaf6bbc24da0954bdc3eea688aafcf5aea6 2024-08-07T19:26:03.4392114Z deleted: sha256:eef572047e96af9ee6bb247ba9acc9c558d7468ff89e2297aa4ad37d1f6043bb 2024-08-07T19:26:03.4392780Z deleted: sha256:4cde019d0b2c20c0af3f1631adfd033c260533dfeefc25dad09f2900dd488b69 2024-08-07T19:26:03.4393430Z deleted: sha256:d2e3c7156b77f135b6bfcbd686f30b96a9fdd1cf7c08b62815572e2d2f7c68be 2024-08-07T19:26:03.4394097Z deleted: sha256:9cd733cd617c46bdc2cb513ead975cec6d449f22a08e36d803cbcef61fa1ef01 2024-08-07T19:26:03.4394890Z deleted: sha256:64744261caf2f507a0f6160868b1d4f0a4a435fb546701053735b889023799eb 2024-08-07T19:26:03.4395541Z deleted: sha256:a2047a4af9745168d52ba90b39a780bc22dc15918425324774e959c0755645a5 2024-08-07T19:26:03.4396178Z deleted: sha256:7d8b4203c0f94a0eccdf0a3ee5819d3be23e8acb9412c3330f66669f03d23e4a 2024-08-07T19:26:03.4396836Z deleted: sha256:7c66ee01d3a6002bf7c53c998d9320f1fb14268af96da395dc28488d5e28001e 2024-08-07T19:26:03.4397493Z deleted: sha256:c1821ba8bdf74b922560615bc28d42e36b63152f7f8e7b07fead450cf78c03b3 2024-08-07T19:26:03.4398143Z deleted: sha256:738da9c7930e061cb698fdaff5ab4d9019fd00adeeacbc24c07eeef3b9a1e8f3 2024-08-07T19:26:03.4398879Z deleted: sha256:b035793979cb8d15291d75f7ceb86be4c7e50b0b55c3ad2de59633310f8eb2ad 2024-08-07T19:26:03.4399527Z deleted: sha256:8caa23c592f4ec71096f15d741c7f74b3674d14f201069b87058d486ee469d80 2024-08-07T19:26:03.4400148Z deleted: sha256:5ec471cacf0cf9317535522156613c2b647715813f96552a7756767caa05b5bc 2024-08-07T19:26:03.4400790Z deleted: sha256:3ddc946b392989b8e7e8086bc23fdab9391ec27042559856bb40fd155a0626b0 2024-08-07T19:26:03.4401446Z deleted: sha256:8bb9b390febb36e87a303534a0672e799f14263107dccecb8fb816711b19b197 2024-08-07T19:26:03.4402092Z deleted: sha256:7636da8a45668f13fc8bdd44f4bcbdfe08506e1222f2e6bf0b6173ff1ebc9d73 2024-08-07T19:26:03.4402755Z deleted: sha256:db811dce25acbc1220f0b64f6b6378af1a8715aadf36172b548ccf95a8a10f5c 2024-08-07T19:26:03.4403409Z deleted: sha256:ea9f1876d9bef2b9737a8084aa575ae2ccf54320762796b1a25b2fd7a1f9a842 2024-08-07T19:26:03.4404046Z deleted: sha256:ee9308661047f85a0cb1e065673e3e34a223a0fabc0d758ca482d1caf72bc156 2024-08-07T19:26:03.4404708Z deleted: sha256:83af056623fdf4bdd8eb6689dd830c618fc706f07f5745771bd23dee1a9df53c 2024-08-07T19:26:03.4405359Z deleted: sha256:1697cd38e32034c99390abaf80e0ef262175ceb38304c2a33a2391ea089625ce 2024-08-07T19:26:03.4406008Z deleted: sha256:2c941f6435258652fe85b0aa3fe7dfd6446f8ad4b90ff57b75cff5173ca901d1 2024-08-07T19:26:03.4406636Z deleted: sha256:de31781f10299152d97a453b662468e0fe812bf2c80e40d7e87d21f1aeebe0de 2024-08-07T19:26:03.4407283Z deleted: sha256:df51025351b2d7b823ae2da119ef42f7a8bcb65067852d58b58793b87a4020cd 2024-08-07T19:26:03.4407940Z deleted: sha256:dea53edb79ae100c832ff7d8e52aa46514a05f85b18820e9798e14cfca8d2b0a 2024-08-07T19:26:03.4408587Z deleted: sha256:30a0b14b371bbf3393cc59e86ebddb2c2bf5ec6e14a2623357e0e8522004373a 2024-08-07T19:26:03.4409240Z deleted: sha256:86275d2a7f4ab08c4db777a2d58601c8e9a33f085073ee419cecc0f80706ba0a 2024-08-07T19:26:03.4409884Z deleted: sha256:be527c207999e42b06dd2b66329fee5905a775d5a617576a8822937689e181b2 2024-08-07T19:26:03.4410512Z deleted: sha256:b726bfe9f806f36a474aff38d139841ad1f3062ad40b2644b38a070efa891fc4 2024-08-07T19:26:03.4411159Z deleted: sha256:79621bf70363b2e574b56de08889a123c0aafdf331b6d403b1227d601196878f 2024-08-07T19:26:03.4411803Z deleted: sha256:23da9a238173789a5e64303a1d5b580bdc2ff4ee4adb1726a937e74fccdb3304 2024-08-07T19:26:03.4412438Z deleted: sha256:6ae72960efb5326803e5d131f2b788bea7d945094d6b4154fbb0d23a1724bc9d 2024-08-07T19:26:03.4413090Z deleted: sha256:4b522ca122bee088c81b99d32935d0c7f798acd919e4787f71f60860f47039c8 2024-08-07T19:26:03.4413811Z deleted: sha256:8099c7b0342e10a71c3957854138df7a86715582a05dfeb507862407dda38fc3 2024-08-07T19:26:03.4414456Z deleted: sha256:9a3099fdec71a88416fee7bf46f4a7692b473fb9684dd93aaa0370840bb1ec2a 2024-08-07T19:26:03.4415093Z deleted: sha256:c7ffa0248132fad86167206fd29b1b8f49bf20a42406063d47df97a13d8f198a 2024-08-07T19:26:03.4415742Z deleted: sha256:496001e36192545049eca8ecc830251c2fa1eb6daf21421c8dafe0acb1f81538 2024-08-07T19:26:03.4416397Z deleted: sha256:9813a6d93cac15977f4540b3e5b3fe849269f35658ed780bdda563ee74b15b3f 2024-08-07T19:26:03.4417023Z deleted: sha256:e29ab5e9037a9838843863a372ef1748406bc95406369a70955134a7cae06d63 2024-08-07T19:26:03.4417672Z deleted: sha256:27d3f1ba6f4aaf25f597a8dcf5d90168631cbbaa2283d5d82528bb267dbc9aa4 2024-08-07T19:26:03.4418320Z deleted: sha256:e7b742f0c6815a00a368e84b648d3d3707bb9f50fc1111d2c99506f5e446552f 2024-08-07T19:26:03.4418951Z deleted: sha256:fd8caf0a273887946a30471b6d8565942faac80f5294f4b3f60779ba4fb1589c 2024-08-07T19:26:03.4419679Z deleted: sha256:6b4ae34804843731029058254b2a78ec9bf655c143179356562fb6792a6d387e 2024-08-07T19:26:03.4420327Z deleted: sha256:5eb13a2c08fccfe8a68cb2eedd5374ca8533971837acdecf1a6241b970be84d4 2024-08-07T19:26:03.4420978Z deleted: sha256:d782a8ac99055b7a7b36960eadce2aa26f0702a2e9de910d9a282b4ad4cf0409 2024-08-07T19:26:03.4421639Z deleted: sha256:32b90a12e0e9cbec20f720d2dd8dc1fe94d05f62a3106ccbe443c7674bcc4ce0 2024-08-07T19:26:03.4422303Z deleted: sha256:ec714b229d3a0dff80ffbf136428ab5a5271e5df7b61198f3fefcda6287ccb6c 2024-08-07T19:26:03.4422946Z deleted: sha256:2f5754399f29badb5803e0b433f6704280c4b79475b44de6728af1dfbcbb2323 2024-08-07T19:26:03.4423603Z deleted: sha256:7bf42df28ece50b780ec31bab2ecea6382f1fe26a88816cc6b9718265cf3465e 2024-08-07T19:26:03.4424264Z deleted: sha256:9a6aa7079f47ded223c38219eed1824b2ffb668af29fa9bcb485254f7e83ab21 2024-08-07T19:26:03.4424923Z deleted: sha256:e9724bcc7ec56c5acb95e352940c94b1c2a9ee52109ad8aab536930bc9728176 2024-08-07T19:26:03.4425572Z deleted: sha256:8f8ef3805ce4283573b6bf58f161a9b3331f50a60ce8d81d4ac659f1fff055c0 2024-08-07T19:26:03.4426229Z deleted: sha256:0729580f4cbe15e55af6aad9352c76c40a739f10bb9b43550680d486bd5c076b 2024-08-07T19:26:03.4426888Z deleted: sha256:4a0b4eb049da56b51a7fdfa4d8206378bdbb1b016dfc6e0bca982791a12fd5eb 2024-08-07T19:26:03.4427545Z deleted: sha256:cf4eeeecbbf6221e1ef64a38d6ba85c73e0bbf10869125ebf081e18f5ef4e9e0 2024-08-07T19:26:03.4428205Z deleted: sha256:f3f2f0324ca26243445f66f17828f234c7a77640dc5d9c44b7a90746c31a2c69 2024-08-07T19:26:03.4428858Z deleted: sha256:014362212fc6cec487fa443cf40098dfadb1562fbd6c17eaa1d987bcac2005d5 2024-08-07T19:26:03.4429515Z deleted: sha256:9d8bd07fc065e2f3faa82c6be619acdbe2a0ac5c84fa4d34564dd9ca50c1abb4 2024-08-07T19:26:03.4430184Z deleted: sha256:be515201f04cb68521fbdaf149a170e1cfee851eb7cf0d0e21cf44788d2d2738 2024-08-07T19:26:03.4430835Z deleted: sha256:a80708e23e0af5c08681444d907936033a760a3707ff65145db0d08946092e6c 2024-08-07T19:26:03.4431470Z deleted: sha256:a12ed841d341f49b467e30c6f384e0cba615d2dc38e7c23f7a3e926f8218fee0 2024-08-07T19:26:03.4432138Z deleted: sha256:508edf28b5543e1a9d07f1caeb9dddf6e77d34ef0d416dc734399f3f49335923 2024-08-07T19:26:03.4432798Z deleted: sha256:0e101de6946d43b7137acf4e038ab8d10e32dbf0c2480641c24be8ed6086abc1 2024-08-07T19:26:03.4433471Z deleted: sha256:c1c19d0e8afe1ecbbde109ac71c1d3f17a00bfcf3d50834770b8d5464ee9c8b2 2024-08-07T19:26:03.4434119Z deleted: sha256:9dc9d6055307261f20c9b3b25bff6eba44e06a05a5b285d733599015160c7dbc 2024-08-07T19:26:03.4434774Z deleted: sha256:7b94babd6975875afe9642a2b9bcf59e402dc13a48cc8be6fd65c2a5fef3ca7c 2024-08-07T19:26:03.4435436Z deleted: sha256:3873bca3dc5064ab463ae26e0a2ab4e9372bea50dd21aa9d60f740676dad7486 2024-08-07T19:26:03.4436080Z deleted: sha256:cbf23c83ce2d413d3afd8790f223a461b296a546765135e62cd690dad638d411 2024-08-07T19:26:03.4436735Z deleted: sha256:bf83fffdacd881f54312039756eae0f29ee91313edb99240a21b46a7a8a52112 2024-08-07T19:26:03.4437392Z deleted: sha256:f8a51c6f429112858d9fa585e8865b12aef6f984426a6b249a9c2405bd6bc617 2024-08-07T19:26:03.4438113Z deleted: sha256:a5f3bdddd25f3ff04aeee3425b589034eea5abb8f3356a23da937bdfae388f96 2024-08-07T19:26:03.4438861Z deleted: sha256:a63fb4f23633e92cf452b4d42c11fabe08bc5941a7b3ccfdb514d9f5df78ace9 2024-08-07T19:26:03.4439529Z deleted: sha256:55935042e44f44818cd67bfb9dbce2a9b57e383adb3caee10c52a60bb4ca2907 2024-08-07T19:26:03.4440561Z deleted: sha256:4e77f30e7a210eadab33be7d5a8e30b207fe0595cc887f660034793669ad4882 2024-08-07T19:26:03.4441625Z deleted: sha256:ff95c9e3cc791feac6467bb1a86bd910894d5aa807dbc7ef8d0656f4c313fb55 2024-08-07T19:26:03.4442698Z deleted: sha256:2d609ec5d6f0f513dc06ab2d069d1f3705dbb888c1c33b153d4d7b402a3a8445 2024-08-07T19:26:03.4443674Z deleted: sha256:6871dd9601fad505a94c55343d42d89a15ba2f3a15352e4264e4aef36be2d3b2 2024-08-07T19:26:03.4444624Z deleted: sha256:e7aab1aad704a9457fcc4018383183080c494fdd601a6d38dc2a06489e3819a6 2024-08-07T19:26:03.4445586Z deleted: sha256:77b1d18f5457391c44c5bf9bfbbdac615becb4cf2149d82856165b77ad19132e 2024-08-07T19:26:03.4446734Z deleted: sha256:5ad805ac6b7297997e85c12ead27e5b34775f55d66c96af2c9c643752c231060 2024-08-07T19:26:03.4447681Z deleted: sha256:f169358745a88fef9997e00330b991a45ceb3553651bbc391a938a896ddeaebf 2024-08-07T19:26:03.4448643Z deleted: sha256:51b3d62ea293d971aac7f5c6e7a8cac80e68e53419b75c0f225b3e14c1b78b1b 2024-08-07T19:26:03.4449610Z deleted: sha256:380d79122c2fdff1088471ef238b65b189372fe3d073caf4500c53935a53a6f5 2024-08-07T19:26:03.4450589Z deleted: sha256:cf4619277942bc0846de485114cca961fa146acbd85711f3a6ee5160681d0a8a 2024-08-07T19:26:03.4451565Z deleted: sha256:331c0a0477e8aa63e33aba884616ecd6cf5dafa7fe512374b8e8d01fb50bdf6f 2024-08-07T19:26:03.4452550Z deleted: sha256:052665ffd1620d190087d835e6fae9b128aaea33ff929767ce2a7d6b5c9c77f2 2024-08-07T19:26:03.4453496Z deleted: sha256:4534c8036770494ac61772b0ffe2379aaa62dbeb0d1307fff0164be698430f2c 2024-08-07T19:26:03.4454531Z deleted: sha256:931b7ff0cb6f494b27d31a4cbec3efe62ac54676add9c7469560302f1541ecaf 2024-08-07T19:26:03.4455121Z 2024-08-07T19:26:03.4455616Z Total reclaimed space: 15.25GB 2024-08-07T19:26:03.4533049Z Post job cleanup. 2024-08-07T19:26:03.4583146Z Post job cleanup. 2024-08-07T19:26:03.5444452Z [command]/usr/bin/git version 2024-08-07T19:26:03.5485148Z git version 2.40.1 2024-08-07T19:26:03.5520536Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/2f7b0a43-2d32-48b5-880b-830ab1f30cd0' before making global git config changes 2024-08-07T19:26:03.5521523Z Adding repository directory to the temporary git global config as a safe directory 2024-08-07T19:26:03.5525359Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch 2024-08-07T19:26:03.5561686Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand 2024-08-07T19:26:03.5593104Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2024-08-07T19:26:03.5874529Z Entering 'android/libs/fbjni' 2024-08-07T19:26:03.5924873Z Entering 'third_party/FP16' 2024-08-07T19:26:03.5975251Z Entering 'third_party/FXdiv' 2024-08-07T19:26:03.6024236Z Entering 'third_party/NNPACK' 2024-08-07T19:26:03.6073455Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T19:26:03.6122480Z Entering 'third_party/XNNPACK' 2024-08-07T19:26:03.6189374Z Entering 'third_party/benchmark' 2024-08-07T19:26:03.6239122Z Entering 'third_party/cpp-httplib' 2024-08-07T19:26:03.6287431Z Entering 'third_party/cpuinfo' 2024-08-07T19:26:03.6337564Z Entering 'third_party/cudnn_frontend' 2024-08-07T19:26:03.6387006Z Entering 'third_party/cutlass' 2024-08-07T19:26:03.6443920Z Entering 'third_party/eigen' 2024-08-07T19:26:03.6495596Z Entering 'third_party/fbgemm' 2024-08-07T19:26:03.6544749Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T19:26:03.6591951Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T19:26:03.6642296Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T19:26:03.6697299Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T19:26:03.6745647Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T19:26:03.6795498Z Entering 'third_party/flatbuffers' 2024-08-07T19:26:03.6847804Z Entering 'third_party/fmt' 2024-08-07T19:26:03.6896811Z Entering 'third_party/foxi' 2024-08-07T19:26:03.6947398Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T19:26:03.6997029Z Entering 'third_party/gloo' 2024-08-07T19:26:03.7046396Z Entering 'third_party/googletest' 2024-08-07T19:26:03.7095635Z Entering 'third_party/ideep' 2024-08-07T19:26:03.7164902Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T19:26:03.7220631Z Entering 'third_party/ittapi' 2024-08-07T19:26:03.7269781Z Entering 'third_party/kineto' 2024-08-07T19:26:03.7317733Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T19:26:03.7366358Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T19:26:03.7416703Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T19:26:03.7465304Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T19:26:03.7513702Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T19:26:03.7561718Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T19:26:03.7613514Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T19:26:03.7663189Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T19:26:03.7711816Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T19:26:03.7761632Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T19:26:03.7811005Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T19:26:03.7860649Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T19:26:03.7909974Z Entering 'third_party/mimalloc' 2024-08-07T19:26:03.7961425Z Entering 'third_party/nccl/nccl' 2024-08-07T19:26:03.8009821Z Entering 'third_party/nlohmann' 2024-08-07T19:26:03.8063682Z Entering 'third_party/onnx' 2024-08-07T19:26:03.8128525Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T19:26:03.8180142Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T19:26:03.8231459Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T19:26:03.8282184Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T19:26:03.8331451Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T19:26:03.8380095Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T19:26:03.8428084Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T19:26:03.8478965Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T19:26:03.8527337Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T19:26:03.8575085Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T19:26:03.8622156Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T19:26:03.8672836Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T19:26:03.8722897Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T19:26:03.8791092Z Entering 'third_party/pocketfft' 2024-08-07T19:26:03.8840606Z Entering 'third_party/protobuf' 2024-08-07T19:26:03.8891804Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T19:26:03.8939798Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T19:26:03.8989712Z Entering 'third_party/psimd' 2024-08-07T19:26:03.9038747Z Entering 'third_party/pthreadpool' 2024-08-07T19:26:03.9088177Z Entering 'third_party/pybind11' 2024-08-07T19:26:03.9137368Z Entering 'third_party/python-peachpy' 2024-08-07T19:26:03.9186960Z Entering 'third_party/sleef' 2024-08-07T19:26:03.9235639Z Entering 'third_party/tensorpipe' 2024-08-07T19:26:03.9284077Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T19:26:03.9331443Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T19:26:03.9379644Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T19:26:03.9427310Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T19:26:03.9474807Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T19:26:03.9540539Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2024-08-07T19:26:03.9567171Z http.https://github.com/.extraheader 2024-08-07T19:26:03.9575257Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader 2024-08-07T19:26:03.9608679Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2024-08-07T19:26:03.9876800Z Entering 'android/libs/fbjni' 2024-08-07T19:26:03.9910237Z http.https://github.com/.extraheader 2024-08-07T19:26:03.9941532Z Entering 'third_party/FP16' 2024-08-07T19:26:03.9974688Z http.https://github.com/.extraheader 2024-08-07T19:26:04.0004736Z Entering 'third_party/FXdiv' 2024-08-07T19:26:04.0037716Z http.https://github.com/.extraheader 2024-08-07T19:26:04.0069585Z Entering 'third_party/NNPACK' 2024-08-07T19:26:04.0103642Z http.https://github.com/.extraheader 2024-08-07T19:26:04.0133879Z Entering 'third_party/VulkanMemoryAllocator' 2024-08-07T19:26:04.0167078Z http.https://github.com/.extraheader 2024-08-07T19:26:04.0197145Z Entering 'third_party/XNNPACK' 2024-08-07T19:26:04.0229768Z http.https://github.com/.extraheader 2024-08-07T19:26:04.0278152Z Entering 'third_party/benchmark' 2024-08-07T19:26:04.0312519Z http.https://github.com/.extraheader 2024-08-07T19:26:04.0343685Z Entering 'third_party/cpp-httplib' 2024-08-07T19:26:04.0376107Z http.https://github.com/.extraheader 2024-08-07T19:26:04.0406329Z Entering 'third_party/cpuinfo' 2024-08-07T19:26:04.0438837Z http.https://github.com/.extraheader 2024-08-07T19:26:04.0468815Z Entering 'third_party/cudnn_frontend' 2024-08-07T19:26:04.0501169Z http.https://github.com/.extraheader 2024-08-07T19:26:04.0531253Z Entering 'third_party/cutlass' 2024-08-07T19:26:04.0564297Z http.https://github.com/.extraheader 2024-08-07T19:26:04.0602556Z Entering 'third_party/eigen' 2024-08-07T19:26:04.0634586Z http.https://github.com/.extraheader 2024-08-07T19:26:04.0668828Z Entering 'third_party/fbgemm' 2024-08-07T19:26:04.0701259Z http.https://github.com/.extraheader 2024-08-07T19:26:04.0731483Z Entering 'third_party/fbgemm/third_party/asmjit' 2024-08-07T19:26:04.0763817Z http.https://github.com/.extraheader 2024-08-07T19:26:04.0793148Z Entering 'third_party/fbgemm/third_party/cpuinfo' 2024-08-07T19:26:04.0826603Z http.https://github.com/.extraheader 2024-08-07T19:26:04.0856928Z Entering 'third_party/fbgemm/third_party/cutlass' 2024-08-07T19:26:04.0888454Z http.https://github.com/.extraheader 2024-08-07T19:26:04.0924732Z Entering 'third_party/fbgemm/third_party/googletest' 2024-08-07T19:26:04.0957422Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1004739Z Entering 'third_party/fbgemm/third_party/hipify_torch' 2024-08-07T19:26:04.1018987Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1050084Z Entering 'third_party/flatbuffers' 2024-08-07T19:26:04.1082712Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1115409Z Entering 'third_party/fmt' 2024-08-07T19:26:04.1148813Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1179445Z Entering 'third_party/foxi' 2024-08-07T19:26:04.1212357Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1243248Z Entering 'third_party/gemmlowp/gemmlowp' 2024-08-07T19:26:04.1276521Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1306822Z Entering 'third_party/gloo' 2024-08-07T19:26:04.1340464Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1370539Z Entering 'third_party/googletest' 2024-08-07T19:26:04.1403229Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1432982Z Entering 'third_party/ideep' 2024-08-07T19:26:04.1466363Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1496000Z Entering 'third_party/ideep/mkl-dnn' 2024-08-07T19:26:04.1527581Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1566485Z Entering 'third_party/ittapi' 2024-08-07T19:26:04.1599807Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1629308Z Entering 'third_party/kineto' 2024-08-07T19:26:04.1663466Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1693791Z Entering 'third_party/kineto/libkineto/third_party/dynolog' 2024-08-07T19:26:04.1725863Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1756640Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM' 2024-08-07T19:26:04.1788929Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1821209Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr' 2024-08-07T19:26:04.1853409Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1884036Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt' 2024-08-07T19:26:04.1915870Z http.https://github.com/.extraheader 2024-08-07T19:26:04.1947168Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags' 2024-08-07T19:26:04.1979249Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2009175Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc' 2024-08-07T19:26:04.2041988Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2074581Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog' 2024-08-07T19:26:04.2106661Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2137398Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest' 2024-08-07T19:26:04.2170224Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2200803Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json' 2024-08-07T19:26:04.2232669Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2264026Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs' 2024-08-07T19:26:04.2295938Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2327929Z Entering 'third_party/kineto/libkineto/third_party/fmt' 2024-08-07T19:26:04.2360260Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2390483Z Entering 'third_party/kineto/libkineto/third_party/googletest' 2024-08-07T19:26:04.2422750Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2454677Z Entering 'third_party/mimalloc' 2024-08-07T19:26:04.2487704Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2518224Z Entering 'third_party/nccl/nccl' 2024-08-07T19:26:04.2553123Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2583616Z Entering 'third_party/nlohmann' 2024-08-07T19:26:04.2616623Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2647970Z Entering 'third_party/onnx' 2024-08-07T19:26:04.2681061Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2726367Z Entering 'third_party/onnx/third_party/benchmark' 2024-08-07T19:26:04.2759246Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2788987Z Entering 'third_party/onnx/third_party/pybind11' 2024-08-07T19:26:04.2822095Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2854248Z Entering 'third_party/opentelemetry-cpp' 2024-08-07T19:26:04.2887891Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2920435Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark' 2024-08-07T19:26:04.2952597Z http.https://github.com/.extraheader 2024-08-07T19:26:04.2982352Z Entering 'third_party/opentelemetry-cpp/third_party/googletest' 2024-08-07T19:26:04.3014187Z http.https://github.com/.extraheader 2024-08-07T19:26:04.3045185Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl' 2024-08-07T19:26:04.3077070Z http.https://github.com/.extraheader 2024-08-07T19:26:04.3106656Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json' 2024-08-07T19:26:04.3139437Z http.https://github.com/.extraheader 2024-08-07T19:26:04.3170682Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto' 2024-08-07T19:26:04.3202803Z http.https://github.com/.extraheader 2024-08-07T19:26:04.3232057Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp' 2024-08-07T19:26:04.3264721Z http.https://github.com/.extraheader 2024-08-07T19:26:04.3294448Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp' 2024-08-07T19:26:04.3326294Z http.https://github.com/.extraheader 2024-08-07T19:26:04.3358010Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb' 2024-08-07T19:26:04.3390163Z http.https://github.com/.extraheader 2024-08-07T19:26:04.3422461Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest' 2024-08-07T19:26:04.3456360Z http.https://github.com/.extraheader 2024-08-07T19:26:04.3487509Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg' 2024-08-07T19:26:04.3519728Z http.https://github.com/.extraheader 2024-08-07T19:26:04.3571046Z Entering 'third_party/pocketfft' 2024-08-07T19:26:04.3604402Z http.https://github.com/.extraheader 2024-08-07T19:26:04.3634548Z Entering 'third_party/protobuf' 2024-08-07T19:26:04.3668315Z http.https://github.com/.extraheader 2024-08-07T19:26:04.3702510Z Entering 'third_party/protobuf/third_party/benchmark' 2024-08-07T19:26:04.3735028Z http.https://github.com/.extraheader 2024-08-07T19:26:04.3765973Z Entering 'third_party/protobuf/third_party/googletest' 2024-08-07T19:26:04.3798143Z http.https://github.com/.extraheader 2024-08-07T19:26:04.3829605Z Entering 'third_party/psimd' 2024-08-07T19:26:04.3862596Z http.https://github.com/.extraheader 2024-08-07T19:26:04.3892695Z Entering 'third_party/pthreadpool' 2024-08-07T19:26:04.3925993Z http.https://github.com/.extraheader 2024-08-07T19:26:04.3956081Z Entering 'third_party/pybind11' 2024-08-07T19:26:04.3989178Z http.https://github.com/.extraheader 2024-08-07T19:26:04.4020310Z Entering 'third_party/python-peachpy' 2024-08-07T19:26:04.4053898Z http.https://github.com/.extraheader 2024-08-07T19:26:04.4084357Z Entering 'third_party/sleef' 2024-08-07T19:26:04.4117884Z http.https://github.com/.extraheader 2024-08-07T19:26:04.4148182Z Entering 'third_party/tensorpipe' 2024-08-07T19:26:04.4181664Z http.https://github.com/.extraheader 2024-08-07T19:26:04.4211923Z Entering 'third_party/tensorpipe/third_party/googletest' 2024-08-07T19:26:04.4244317Z http.https://github.com/.extraheader 2024-08-07T19:26:04.4274226Z Entering 'third_party/tensorpipe/third_party/libnop' 2024-08-07T19:26:04.4307380Z http.https://github.com/.extraheader 2024-08-07T19:26:04.4336636Z Entering 'third_party/tensorpipe/third_party/libuv' 2024-08-07T19:26:04.4369282Z http.https://github.com/.extraheader 2024-08-07T19:26:04.4399208Z Entering 'third_party/tensorpipe/third_party/pybind11' 2024-08-07T19:26:04.4432839Z http.https://github.com/.extraheader 2024-08-07T19:26:04.4462271Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang' 2024-08-07T19:26:04.4494857Z http.https://github.com/.extraheader 2024-08-07T19:26:04.4605278Z A job completed hook has been configured by the self-hosted runner administrator 2024-08-07T19:26:04.4639469Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh' 2024-08-07T19:26:04.4644870Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0} 2024-08-07T19:26:04.4645259Z ##[endgroup] 2024-08-07T19:26:11.0036865Z Cleaning up orphan processes