2025-03-14T04:08:07.5351529Z Current runner version: '2.322.0'
2025-03-14T04:08:07.5358907Z Runner name: 'i-09d49014f68772654'
2025-03-14T04:08:07.5359841Z Runner group name: 'Default'
2025-03-14T04:08:07.5360895Z Machine name: 'ip-10-0-57-87'
2025-03-14T04:08:07.5365503Z ##[group]GITHUB_TOKEN Permissions
2025-03-14T04:08:07.5368318Z Actions: read
2025-03-14T04:08:07.5369076Z Attestations: read
2025-03-14T04:08:07.5369727Z Checks: read
2025-03-14T04:08:07.5370367Z Contents: read
2025-03-14T04:08:07.5371018Z Deployments: read
2025-03-14T04:08:07.5371674Z Discussions: read
2025-03-14T04:08:07.5372397Z Issues: read
2025-03-14T04:08:07.5373015Z Metadata: read
2025-03-14T04:08:07.5373645Z Packages: read
2025-03-14T04:08:07.5374319Z Pages: read
2025-03-14T04:08:07.5374955Z PullRequests: read
2025-03-14T04:08:07.5375627Z RepositoryProjects: read
2025-03-14T04:08:07.5376354Z SecurityEvents: read
2025-03-14T04:08:07.5376984Z Statuses: read
2025-03-14T04:08:07.5377618Z ##[endgroup]
2025-03-14T04:08:07.5381082Z Secret source: Actions
2025-03-14T04:08:07.5382215Z Prepare workflow directory
2025-03-14T04:08:07.8529551Z Prepare all required actions
2025-03-14T04:08:07.8573301Z Getting action download info
2025-03-14T04:08:08.0554990Z Download action repository 'pytorch/test-infra@main' (SHA:de00dac6adc071cb2f9861380a0ed3947b93e5cc)
2025-03-14T04:08:09.4502200Z Download action repository 'pytorch/pytorch@main' (SHA:aed0b7a742a2d7b7901790622829cbd2135049a4)
2025-03-14T04:08:22.6734167Z Download action repository 'aws-actions/configure-aws-credentials@v3' (SHA:50ac8dd1e1b10d09dac7b8727528b91bed831ac0)
2025-03-14T04:08:22.8649781Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a)
2025-03-14T04:08:23.1163327Z Getting action download info
2025-03-14T04:08:23.2134959Z Download action repository 'actions/checkout@v4' (SHA:11bd71901bbe5b1630ceea73d27597364c9af683)
2025-03-14T04:08:23.4393156Z Getting action download info
2025-03-14T04:08:23.5358262Z Download action repository 'nick-fields/retry@v3.0.0' (SHA:7152eba30c6575329ac0576536151aca5a72780e)
2025-03-14T04:08:23.7084986Z Getting action download info
2025-03-14T04:08:23.7963883Z Download action repository 'nick-fields/retry@3e91a01664abd3c5cd539100d10d33b9c5b68482' (SHA:3e91a01664abd3c5cd539100d10d33b9c5b68482)
2025-03-14T04:08:23.9548642Z Getting action download info
2025-03-14T04:08:24.0812239Z Uses: pytorch/pytorch/.github/workflows/_linux-test.yml@refs/heads/main (aed0b7a742a2d7b7901790622829cbd2135049a4)
2025-03-14T04:08:24.0814362Z ##[group] Inputs
2025-03-14T04:08:24.0814739Z   build-environment: linux-jammy-py3.9-gcc11
2025-03-14T04:08:24.0817927Z   test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "lf.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "lf.linux.2xlarge"}, {"config": "numpy_2_x", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}]}
2025-03-14T04:08:24.0821388Z   docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:24.0822166Z   sync-tag: 
2025-03-14T04:08:24.0822940Z   timeout-minutes: 240
2025-03-14T04:08:24.0823228Z   use-gha: 
2025-03-14T04:08:24.0823473Z   dashboard-tag: 
2025-03-14T04:08:24.0823742Z   s3-bucket: gha-artifacts
2025-03-14T04:08:24.0824300Z   aws-role-to-assume: 
2025-03-14T04:08:24.0825042Z   disable-monitor: false
2025-03-14T04:08:24.0825352Z ##[endgroup]
2025-03-14T04:08:24.0825889Z Complete job name: linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)
2025-03-14T04:08:24.1313926Z A job started hook has been configured by the self-hosted runner administrator
2025-03-14T04:08:24.1412606Z ##[group]Run '/home/ec2-user/runner-scripts/before_job.sh'
2025-03-14T04:08:24.1421531Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:08:24.1422249Z ##[endgroup]
2025-03-14T04:08:25.9108193Z Runner Type: lf.linux.2xlarge
2025-03-14T04:08:25.9108760Z Instance Type: c5.2xlarge
2025-03-14T04:08:25.9109082Z AMI Name: unknown
2025-03-14T04:08:25.9134633Z AMI ID: ami-05b10e08d247fb927
2025-03-14T04:08:31.2553808Z ##[group]Run pytorch/test-infra/.github/actions/setup-ssh@main
2025-03-14T04:08:31.2554277Z with:
2025-03-14T04:08:31.2554964Z   github-secret: ***
2025-03-14T04:08:31.2555710Z   instructions: All testing is done inside the container, to start an interactive session run:
  docker exec -it $(docker container ps --format '{{.ID}}') bash

2025-03-14T04:08:31.2556527Z   activate-with-label: false
2025-03-14T04:08:31.2556810Z   label: with-ssh
2025-03-14T04:08:31.2557061Z   remove-existing-keys: true
2025-03-14T04:08:31.2557342Z   fail-silently: true
2025-03-14T04:08:31.2557583Z env:
2025-03-14T04:08:31.2557795Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:31.2558064Z ##[endgroup]
2025-03-14T04:08:31.3702252Z Please see https://github.com/pytorch/pytorch/wiki/Debugging-using-with-ssh-for-Github-Actions for more info.
2025-03-14T04:08:31.3704554Z Not on pull request and ciflow reference could not be extracted, skipping adding ssh keys
2025-03-14T04:08:31.3869741Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main
2025-03-14T04:08:31.3870217Z with:
2025-03-14T04:08:31.3870462Z   no-sudo: true
2025-03-14T04:08:31.3870733Z   submodules: recursive
2025-03-14T04:08:31.3871025Z   fetch-depth: 0
2025-03-14T04:08:31.3871284Z env:
2025-03-14T04:08:31.3871547Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:31.3871821Z ##[endgroup]
2025-03-14T04:08:31.3954416Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT"
2025-03-14T04:08:31.3955439Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT"
2025-03-14T04:08:31.3963199Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:08:31.3963628Z env:
2025-03-14T04:08:31.3963878Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:31.3964173Z ##[endgroup]
2025-03-14T04:08:31.4050772Z ##[group]Run # Use all available CPUs for fetching
2025-03-14T04:08:31.4051241Z # Use all available CPUs for fetching
2025-03-14T04:08:31.4051637Z cd "${GITHUB_WORKSPACE}"
2025-03-14T04:08:31.4052058Z git config --global fetch.parallel 0
2025-03-14T04:08:31.4052484Z git config --global submodule.fetchJobs 0
2025-03-14T04:08:31.4052878Z 
2025-03-14T04:08:31.4053271Z # Clean workspace. The default checkout action should also do this, but
2025-03-14T04:08:31.4053778Z # do it here as well just in case
2025-03-14T04:08:31.4054112Z if [[ -d .git ]]; then
2025-03-14T04:08:31.4054430Z   if [ -z "${NO_SUDO}" ]; then
2025-03-14T04:08:31.4054767Z     sudo git clean -ffdx
2025-03-14T04:08:31.4055074Z   else
2025-03-14T04:08:31.4055331Z     git clean -ffdx
2025-03-14T04:08:31.4055620Z   fi
2025-03-14T04:08:31.4055856Z fi
2025-03-14T04:08:31.4061285Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:08:31.4061700Z env:
2025-03-14T04:08:31.4062008Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:31.4062322Z   NO_SUDO: true
2025-03-14T04:08:31.4062571Z ##[endgroup]
2025-03-14T04:08:32.0446539Z Removing .additional_ci_files/
2025-03-14T04:08:32.0447132Z Removing .github/scripts/__pycache__/
2025-03-14T04:08:32.0448027Z Removing artifacts.zip
2025-03-14T04:08:32.0448656Z Removing aten/src/ATen/Config.h
2025-03-14T04:08:32.0449219Z Removing build/
2025-03-14T04:08:32.0449707Z Removing compile_commands.json
2025-03-14T04:08:32.0450164Z Removing dist/
2025-03-14T04:08:32.0450474Z Removing functorch/functorch.so
2025-03-14T04:08:32.0450948Z Removing sccache-stats-linux-jammy-py3.9-gcc11-no-ops-38754274268.json
2025-03-14T04:08:32.0451454Z Removing test/test-reports/
2025-03-14T04:08:32.0451770Z Removing third_party/__pycache__/
2025-03-14T04:08:32.0452154Z Removing third_party/nccl/
2025-03-14T04:08:32.0452636Z Removing tools/__pycache__/
2025-03-14T04:08:32.0452974Z Removing tools/autograd/__pycache__/
2025-03-14T04:08:32.0453320Z Removing tools/pyi/__pycache__/
2025-03-14T04:08:32.0453658Z Removing tools/setup_helpers/__pycache__/
2025-03-14T04:08:32.0454030Z Removing tools/shared/_utils_internal.py
2025-03-14T04:08:32.0454388Z Removing tools/stats/__pycache__/
2025-03-14T04:08:32.0454701Z Removing torch.egg-info/
2025-03-14T04:08:32.0455030Z Removing torch/_C/_VariableFunctions.pyi
2025-03-14T04:08:32.0455399Z Removing torch/_C/__init__.pyi
2025-03-14T04:08:32.0455711Z Removing torch/_C/_nn.pyi
2025-03-14T04:08:32.0456005Z Removing torch/_VF.pyi
2025-03-14T04:08:32.0456292Z Removing torch/__pycache__/
2025-03-14T04:08:32.0456592Z Removing torch/bin/
2025-03-14T04:08:32.0456908Z Removing torch/csrc/api/include/torch/version.h
2025-03-14T04:08:32.0457304Z Removing torch/csrc/autograd/generated/
2025-03-14T04:08:32.0457786Z Removing torch/csrc/inductor/aoti_torch/generated/c_shim_cpu.cpp
2025-03-14T04:08:32.0458355Z Removing torch/csrc/inductor/aoti_torch/generated/c_shim_cuda.cpp
2025-03-14T04:08:32.0459028Z Removing torch/csrc/lazy/generated/LazyIr.h
2025-03-14T04:08:32.0459489Z Removing torch/csrc/lazy/generated/LazyNativeFunctions.cpp
2025-03-14T04:08:32.0459997Z Removing torch/csrc/lazy/generated/LazyNativeFunctions.h
2025-03-14T04:08:32.0460496Z Removing torch/csrc/lazy/generated/LazyNonNativeIr.h
2025-03-14T04:08:32.0460999Z Removing torch/csrc/lazy/generated/RegisterAutogradLazy.cpp
2025-03-14T04:08:32.0461487Z Removing torch/csrc/lazy/generated/RegisterLazy.cpp
2025-03-14T04:08:32.0461883Z Removing torch/include/
2025-03-14T04:08:32.0462175Z Removing torch/lib/cmake/
2025-03-14T04:08:32.0462480Z Removing torch/lib/libXNNPACK.a
2025-03-14T04:08:32.0462808Z Removing torch/lib/libaoti_custom_ops.so
2025-03-14T04:08:32.0463173Z Removing torch/lib/libasmjit.a
2025-03-14T04:08:32.0463529Z Removing torch/lib/libbackend_with_compiler.so
2025-03-14T04:08:32.0463911Z Removing torch/lib/libc10.so
2025-03-14T04:08:32.0464229Z Removing torch/lib/libclog.a
2025-03-14T04:08:32.0464554Z Removing torch/lib/libcpuinfo.a
2025-03-14T04:08:32.0464878Z Removing torch/lib/libdnnl.a
2025-03-14T04:08:32.0465195Z Removing torch/lib/libfbgemm.a
2025-03-14T04:08:32.0465602Z Removing torch/lib/libfmt.a
2025-03-14T04:08:32.0465930Z Removing torch/lib/libjitbackend_test.so
2025-03-14T04:08:32.0466288Z Removing torch/lib/libkineto.a
2025-03-14T04:08:32.0466634Z Removing torch/lib/libmicrokernels-prod.a
2025-03-14T04:08:32.0466996Z Removing torch/lib/libnnpack.a
2025-03-14T04:08:32.0467326Z Removing torch/lib/libprotobuf-lite.a
2025-03-14T04:08:32.0467674Z Removing torch/lib/libprotobuf.a
2025-03-14T04:08:32.0467998Z Removing torch/lib/libprotoc.a
2025-03-14T04:08:32.0468306Z Removing torch/lib/libpthreadpool.a
2025-03-14T04:08:32.0468654Z Removing torch/lib/libpytorch_qnnpack.a
2025-03-14T04:08:32.0468999Z Removing torch/lib/libshm.so
2025-03-14T04:08:32.0469307Z Removing torch/lib/libsleef.a
2025-03-14T04:08:32.0469627Z Removing torch/lib/libtensorpipe.a
2025-03-14T04:08:32.0469970Z Removing torch/lib/libtensorpipe_uv.a
2025-03-14T04:08:32.0470321Z Removing torch/lib/libtorch.so
2025-03-14T04:08:32.0470640Z Removing torch/lib/libtorch_cpu.so
2025-03-14T04:08:32.0470987Z Removing torch/lib/libtorch_global_deps.so
2025-03-14T04:08:32.0471356Z Removing torch/lib/libtorch_python.so
2025-03-14T04:08:32.0471712Z Removing torch/lib/libtorchbind_test.so
2025-03-14T04:08:32.0472184Z Removing torch/lib/pkgconfig/
2025-03-14T04:08:32.0472516Z Removing torch/nn/functional.pyi
2025-03-14T04:08:32.0472846Z Removing torch/return_types.pyi
2025-03-14T04:08:32.0473155Z Removing torch/share/
2025-03-14T04:08:32.0473429Z Removing torch/test/
2025-03-14T04:08:32.0473813Z Removing torch/testing/_internal/generated/annotated_fn_args.py
2025-03-14T04:08:32.0474376Z Removing torch/utils/benchmark/utils/valgrind_wrapper/callgrind.h
2025-03-14T04:08:32.0474942Z Removing torch/utils/benchmark/utils/valgrind_wrapper/valgrind.h
2025-03-14T04:08:32.0475437Z Removing torch/utils/data/datapipes/datapipe.pyi
2025-03-14T04:08:32.0475824Z Removing torch/version.py
2025-03-14T04:08:32.0476113Z Removing torchgen/__pycache__/
2025-03-14T04:08:32.0476437Z Removing torchgen/aoti/__pycache__/
2025-03-14T04:08:32.0476786Z Removing torchgen/api/__pycache__/
2025-03-14T04:08:32.0477135Z Removing torchgen/api/types/__pycache__/
2025-03-14T04:08:32.0477534Z Removing torchgen/dest/__pycache__/
2025-03-14T04:08:32.0477897Z Removing torchgen/executorch/__pycache__/
2025-03-14T04:08:32.0478316Z Removing torchgen/executorch/api/__pycache__/
2025-03-14T04:08:32.0478742Z Removing torchgen/executorch/api/types/__pycache__/
2025-03-14T04:08:32.0479138Z Removing torchgen/packaged/
2025-03-14T04:08:32.0479477Z Removing torchgen/selective_build/__pycache__/
2025-03-14T04:08:32.0552568Z ##[group]Run actions/checkout@v4
2025-03-14T04:08:32.0552888Z with:
2025-03-14T04:08:32.0553158Z   ref: aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:08:32.0553517Z   fetch-depth: 0
2025-03-14T04:08:32.0553780Z   submodules: recursive
2025-03-14T04:08:32.0554063Z   show-progress: false
2025-03-14T04:08:32.0554515Z   repository: pytorch/pytorch
2025-03-14T04:08:32.0554941Z   token: ***
2025-03-14T04:08:32.0555186Z   ssh-strict: true
2025-03-14T04:08:32.0555440Z   ssh-user: git
2025-03-14T04:08:32.0555703Z   persist-credentials: true
2025-03-14T04:08:32.0555993Z   clean: true
2025-03-14T04:08:32.0556259Z   sparse-checkout-cone-mode: true
2025-03-14T04:08:32.0556573Z   fetch-tags: false
2025-03-14T04:08:32.0556824Z   lfs: false
2025-03-14T04:08:32.0557071Z   set-safe-directory: true
2025-03-14T04:08:32.0557354Z env:
2025-03-14T04:08:32.0557587Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:32.0557872Z ##[endgroup]
2025-03-14T04:08:32.1651787Z Syncing repository: pytorch/pytorch
2025-03-14T04:08:32.1653124Z ##[group]Getting Git version info
2025-03-14T04:08:32.1653638Z Working directory is '/home/ec2-user/actions-runner/_work/pytorch/pytorch'
2025-03-14T04:08:32.1654357Z [command]/usr/bin/git version
2025-03-14T04:08:32.1654669Z git version 2.47.1
2025-03-14T04:08:32.1665909Z ##[endgroup]
2025-03-14T04:08:32.1674319Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/96178061-2ba3-46b3-a58c-704721376c41/.gitconfig'
2025-03-14T04:08:32.1692634Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/96178061-2ba3-46b3-a58c-704721376c41' before making global git config changes
2025-03-14T04:08:32.1693677Z Adding repository directory to the temporary git global config as a safe directory
2025-03-14T04:08:32.1697083Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch
2025-03-14T04:08:32.1738479Z [command]/usr/bin/git config --local --get remote.origin.url
2025-03-14T04:08:32.1755749Z https://github.com/pytorch/pytorch
2025-03-14T04:08:32.1768937Z ##[group]Removing previously created refs, to avoid conflicts
2025-03-14T04:08:32.1771988Z [command]/usr/bin/git rev-parse --symbolic-full-name --verify --quiet HEAD
2025-03-14T04:08:32.1790355Z HEAD
2025-03-14T04:08:32.1821598Z ##[endgroup]
2025-03-14T04:08:32.1824458Z [command]/usr/bin/git submodule status
2025-03-14T04:08:32.2125300Z  7e1e1fe3858c63c251c637ae41a20de425dde96f android/libs/fbjni (v0.1.0-12-g7e1e1fe)
2025-03-14T04:08:32.2194262Z  4dfe081cf6bcd15db339cf2680b9281b8451eeb3 third_party/FP16 (4dfe081)
2025-03-14T04:08:32.2260885Z  b408327ac2a15ec3e43352421954f5b1967701d1 third_party/FXdiv (b408327)
2025-03-14T04:08:32.2349949Z  c07e3a0400713d546e0dea2d5466dd22ea389c73 third_party/NNPACK (c07e3a0)
2025-03-14T04:08:32.2371878Z  e170594ac7cf1dac584da473d4ca9301087090c1 third_party/NVTX (v3.1.0)
2025-03-14T04:08:32.2448295Z  a6bfc237255a6bac1513f7c1ebde6d8aed6b5191 third_party/VulkanMemoryAllocator (v2.1.0-705-ga6bfc23)
2025-03-14T04:08:32.3060258Z  51a0103656eff6fc9bfd39a4597923c4b542c883 third_party/XNNPACK (remotes/origin/ds/ndk-1243-g51a010365)
2025-03-14T04:08:32.3083180Z  0d98dba29d66e93259db7daa53a9327df767a415 third_party/benchmark (v1.6.1)
2025-03-14T04:08:32.3116127Z  8086bbe3a78d931eb96fe12fdc014082e18d18d3 third_party/composable_kernel (mock-tag-test-6-g8086bbe3a)
2025-03-14T04:08:32.3254410Z  3b6597bba913d51161383657829b7e644e59c006 third_party/cpp-httplib (v0.15.3-20-g3b6597b)
2025-03-14T04:08:32.3364931Z  1e83a2fdd3102f65c6f1fb602c1b320486218a99 third_party/cpuinfo (1e83a2f)
2025-03-14T04:08:32.3398589Z  91b7532f3386768bba4f444ee7672b497f34da8a third_party/cudnn_frontend (v0.5-44-g91b7532)
2025-03-14T04:08:32.3498675Z  afa1772203677c5118fcd82537a9c8fefbcc7008 third_party/cutlass (v3.8.0)
2025-03-14T04:08:32.4285759Z  3147391d946bb4b6c68edd901f2add6ac1f31f8c third_party/eigen (3.4.0)
2025-03-14T04:08:32.4664517Z  dbc3157bf256f1339b3fa1fef2be89ac4078be0e third_party/fbgemm (v0.4.1-446-gdbc3157b)
2025-03-14T04:08:32.4749191Z  979702c87a8713a8e0a5e9fee122b90d2ef13be5 third_party/flash-attention (v2.7.4)
2025-03-14T04:08:32.4772388Z  01834de25e4bf3975a9a00e816292b1ad0fe184b third_party/flatbuffers (v23.3.3)
2025-03-14T04:08:32.5200128Z  123913715afeb8a437e6388b4473fcc4753e1c9a third_party/fmt (11.1.4)
2025-03-14T04:08:32.5313100Z  3fb5c176c17c765a3492cd2f0321b0dab712f350 third_party/gemmlowp/gemmlowp (remotes/origin/revert-87-master-135-g3fb5c17)
2025-03-14T04:08:32.5435369Z  5354032ea08eadd7fc4456477f7f7c6308818509 third_party/gloo (5354032)
2025-03-14T04:08:32.5697391Z  b514bdc898e2951020cbdca1304b75f5950d1f59 third_party/googletest (release-1.8.0-3484-gb514bdc8)
2025-03-14T04:08:32.5775183Z  719d8e6cd7f7a0e01b155657526d693acf97c2b3 third_party/ideep (pytorch-rls-v3.7.1)
2025-03-14T04:08:32.5831231Z  5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42 third_party/ittapi (v3.23.0-14-g5b8a7d7)
2025-03-14T04:08:32.6093598Z  2859721fd9e73d3ca1c56f827dbc64e6d68f78a2 third_party/kineto (remotes/origin/sraikund/test-53-g2859721)
2025-03-14T04:08:32.6115871Z  ef685a13cfbe8d418aa2ed34350e21e4938358b6 third_party/kleidiai (v1.3.0)
2025-03-14T04:08:32.6137507Z  b66e3214d8a104669c2ec05ae91ebc26a8f5ab78 third_party/mimalloc (v1.8.2)
2025-03-14T04:08:32.6629009Z  87cda1d6646592ac5866dc703c8e1839046a6806 third_party/nlohmann (v3.10.1-113-g87cda1d6)
2025-03-14T04:08:32.6993453Z  b8baa8446686496da4cc8fda09f2b6fe65c2a02c third_party/onnx (v1.17.0)
2025-03-14T04:08:32.7017796Z  a799f4aed9c94b765dcdaabaeab7d5e7e2310878 third_party/opentelemetry-cpp (v1.14.2)
2025-03-14T04:08:32.7041586Z  9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe third_party/pocketfft (release_for_eigen-11-g9d3ab05)
2025-03-14T04:08:32.7620353Z  d1eca4e4b421cd2997495c4b4e65cea6be4e9b8a third_party/protobuf (v3.7.0-rc.2-1279-gd1eca4e4b)
2025-03-14T04:08:32.7689886Z  072586a71b55b7f8c584153d223e95687148a900 third_party/psimd (heads/master)
2025-03-14T04:08:32.7748647Z  4fe0e1e183925bf8cfa6aae24237e724a96479b8 third_party/pthreadpool (0.1-144-g4fe0e1e)
2025-03-14T04:08:32.7841985Z  a2e59f0e7065404b44dfe92a28aca47ba1378dc4 third_party/pybind11 (v2.11.0-182-ga2e59f0e)
2025-03-14T04:08:32.7909682Z  f45429b087dd7d5bc78bb40dc7cf06425c252d67 third_party/python-peachpy (remotes/origin/pre-generated)
2025-03-14T04:08:32.8045497Z  56e1f79cb140fb9326d612d0be06b5250565cade third_party/sleef (3.7-33-g56e1f79)
2025-03-14T04:08:32.8108643Z  52791a2fd214b2a9dc5759d36725909c1daa7f2e third_party/tensorpipe (remotes/origin/master)
2025-03-14T04:08:32.8120642Z ##[group]Cleaning the repository
2025-03-14T04:08:32.8125376Z [command]/usr/bin/git clean -ffdx
2025-03-14T04:08:32.8314723Z [command]/usr/bin/git reset --hard HEAD
2025-03-14T04:08:33.8787515Z Updating files:  78% (13029/16540)
2025-03-14T04:08:33.9004287Z Updating files:  79% (13067/16540)
2025-03-14T04:08:33.9254262Z Updating files:  80% (13232/16540)
2025-03-14T04:08:33.9377069Z Updating files:  81% (13398/16540)
2025-03-14T04:08:33.9522304Z Updating files:  82% (13563/16540)
2025-03-14T04:08:33.9669544Z Updating files:  83% (13729/16540)
2025-03-14T04:08:33.9808405Z Updating files:  84% (13894/16540)
2025-03-14T04:08:33.9924059Z Updating files:  85% (14059/16540)
2025-03-14T04:08:34.0051892Z Updating files:  86% (14225/16540)
2025-03-14T04:08:34.0175306Z Updating files:  87% (14390/16540)
2025-03-14T04:08:34.0342995Z Updating files:  88% (14556/16540)
2025-03-14T04:08:34.0527839Z Updating files:  89% (14721/16540)
2025-03-14T04:08:34.0654122Z Updating files:  90% (14886/16540)
2025-03-14T04:08:34.0778149Z Updating files:  91% (15052/16540)
2025-03-14T04:08:34.0941928Z Updating files:  92% (15217/16540)
2025-03-14T04:08:34.1123296Z Updating files:  93% (15383/16540)
2025-03-14T04:08:34.1327370Z Updating files:  94% (15548/16540)
2025-03-14T04:08:34.1495045Z Updating files:  95% (15713/16540)
2025-03-14T04:08:34.1631989Z Updating files:  96% (15879/16540)
2025-03-14T04:08:34.1952654Z Updating files:  97% (16044/16540)
2025-03-14T04:08:34.2124032Z Updating files:  98% (16210/16540)
2025-03-14T04:08:34.2295435Z Updating files:  99% (16375/16540)
2025-03-14T04:08:34.2295927Z Updating files: 100% (16540/16540)
2025-03-14T04:08:34.2296265Z Updating files: 100% (16540/16540), done.
2025-03-14T04:08:34.2472501Z HEAD is now at aed0b7a742a [c10d] Add param recording for uniqueID broadcasting and allgather (#149166)
2025-03-14T04:08:34.2548964Z ##[endgroup]
2025-03-14T04:08:34.2551735Z ##[group]Disabling automatic garbage collection
2025-03-14T04:08:34.2557721Z [command]/usr/bin/git config --local gc.auto 0
2025-03-14T04:08:34.2585618Z ##[endgroup]
2025-03-14T04:08:34.2586374Z ##[group]Setting up auth
2025-03-14T04:08:34.2592384Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2025-03-14T04:08:34.2620161Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2025-03-14T04:08:34.2882931Z Entering 'android/libs/fbjni'
2025-03-14T04:08:34.2933134Z Entering 'third_party/FP16'
2025-03-14T04:08:34.2981205Z Entering 'third_party/FXdiv'
2025-03-14T04:08:34.3029725Z Entering 'third_party/NNPACK'
2025-03-14T04:08:34.3079393Z Entering 'third_party/NVTX'
2025-03-14T04:08:34.3130618Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:08:34.3180411Z Entering 'third_party/XNNPACK'
2025-03-14T04:08:34.3246719Z Entering 'third_party/benchmark'
2025-03-14T04:08:34.3297552Z Entering 'third_party/composable_kernel'
2025-03-14T04:08:34.3357125Z Entering 'third_party/cpp-httplib'
2025-03-14T04:08:34.3406789Z Entering 'third_party/cpuinfo'
2025-03-14T04:08:34.3457638Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:08:34.3506186Z Entering 'third_party/cutlass'
2025-03-14T04:08:34.3563382Z Entering 'third_party/eigen'
2025-03-14T04:08:34.3614765Z Entering 'third_party/fbgemm'
2025-03-14T04:08:34.3663389Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:08:34.3712912Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:08:34.3760802Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:08:34.3815172Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:08:34.3862914Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:08:34.3913576Z Entering 'third_party/flash-attention'
2025-03-14T04:08:34.3964283Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:08:34.4018736Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:08:34.4075763Z Entering 'third_party/flatbuffers'
2025-03-14T04:08:34.4128357Z Entering 'third_party/fmt'
2025-03-14T04:08:34.4179522Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:08:34.4229555Z Entering 'third_party/gloo'
2025-03-14T04:08:34.4283713Z Entering 'third_party/googletest'
2025-03-14T04:08:34.4339197Z Entering 'third_party/ideep'
2025-03-14T04:08:34.4389137Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:08:34.4448859Z Entering 'third_party/ittapi'
2025-03-14T04:08:34.4498115Z Entering 'third_party/kineto'
2025-03-14T04:08:34.4547200Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:08:34.4596438Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:08:34.4646850Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:08:34.4695568Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:08:34.4747944Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:08:34.4795381Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:08:34.4844785Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:08:34.4892577Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:08:34.4941313Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:08:34.4990281Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:08:34.5040491Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:08:34.5088141Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:08:34.5137820Z Entering 'third_party/kleidiai'
2025-03-14T04:08:34.5186940Z Entering 'third_party/mimalloc'
2025-03-14T04:08:34.5237407Z Entering 'third_party/nlohmann'
2025-03-14T04:08:34.5287240Z Entering 'third_party/onnx'
2025-03-14T04:08:34.5351040Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:08:34.5402439Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:08:34.5452223Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:08:34.5498898Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:08:34.5546526Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:08:34.5593784Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:08:34.5642905Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:08:34.5690276Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:08:34.5738677Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:08:34.5784864Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:08:34.5834700Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:08:34.5884162Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:08:34.5951346Z Entering 'third_party/pocketfft'
2025-03-14T04:08:34.6000170Z Entering 'third_party/protobuf'
2025-03-14T04:08:34.6050449Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:08:34.6097198Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:08:34.6146723Z Entering 'third_party/psimd'
2025-03-14T04:08:34.6194497Z Entering 'third_party/pthreadpool'
2025-03-14T04:08:34.6242911Z Entering 'third_party/pybind11'
2025-03-14T04:08:34.6291320Z Entering 'third_party/python-peachpy'
2025-03-14T04:08:34.6339881Z Entering 'third_party/sleef'
2025-03-14T04:08:34.6388700Z Entering 'third_party/tensorpipe'
2025-03-14T04:08:34.6438681Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:08:34.6486501Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:08:34.6534124Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:08:34.6581591Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:08:34.6627782Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:08:34.6693610Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2025-03-14T04:08:34.6718472Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2025-03-14T04:08:34.6981459Z Entering 'android/libs/fbjni'
2025-03-14T04:08:34.7032116Z Entering 'third_party/FP16'
2025-03-14T04:08:34.7080420Z Entering 'third_party/FXdiv'
2025-03-14T04:08:34.7129668Z Entering 'third_party/NNPACK'
2025-03-14T04:08:34.7178681Z Entering 'third_party/NVTX'
2025-03-14T04:08:34.7228585Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:08:34.7277468Z Entering 'third_party/XNNPACK'
2025-03-14T04:08:34.7341790Z Entering 'third_party/benchmark'
2025-03-14T04:08:34.7390723Z Entering 'third_party/composable_kernel'
2025-03-14T04:08:34.7445653Z Entering 'third_party/cpp-httplib'
2025-03-14T04:08:34.7494621Z Entering 'third_party/cpuinfo'
2025-03-14T04:08:34.7544329Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:08:34.7593423Z Entering 'third_party/cutlass'
2025-03-14T04:08:34.7651589Z Entering 'third_party/eigen'
2025-03-14T04:08:34.7705909Z Entering 'third_party/fbgemm'
2025-03-14T04:08:34.7755128Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:08:34.7803027Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:08:34.7851672Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:08:34.7906896Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:08:34.7954821Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:08:34.8004292Z Entering 'third_party/flash-attention'
2025-03-14T04:08:34.8055344Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:08:34.8110063Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:08:34.8166631Z Entering 'third_party/flatbuffers'
2025-03-14T04:08:34.8219945Z Entering 'third_party/fmt'
2025-03-14T04:08:34.8270079Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:08:34.8319826Z Entering 'third_party/gloo'
2025-03-14T04:08:34.8369102Z Entering 'third_party/googletest'
2025-03-14T04:08:34.8419161Z Entering 'third_party/ideep'
2025-03-14T04:08:34.8466910Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:08:34.8520431Z Entering 'third_party/ittapi'
2025-03-14T04:08:34.8569460Z Entering 'third_party/kineto'
2025-03-14T04:08:34.8618201Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:08:34.8666686Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:08:34.8716808Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:08:34.8765757Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:08:34.8814779Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:08:34.8861809Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:08:34.8914179Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:08:34.8961989Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:08:34.9012331Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:08:34.9061497Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:08:34.9110865Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:08:34.9158280Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:08:34.9207480Z Entering 'third_party/kleidiai'
2025-03-14T04:08:34.9256128Z Entering 'third_party/mimalloc'
2025-03-14T04:08:34.9306424Z Entering 'third_party/nlohmann'
2025-03-14T04:08:34.9355786Z Entering 'third_party/onnx'
2025-03-14T04:08:34.9418133Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:08:34.9468814Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:08:34.9520057Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:08:34.9566739Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:08:34.9614561Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:08:34.9661340Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:08:34.9709549Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:08:34.9756003Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:08:34.9803543Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:08:34.9849828Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:08:34.9899351Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:08:34.9948972Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:08:35.0017120Z Entering 'third_party/pocketfft'
2025-03-14T04:08:35.0066219Z Entering 'third_party/protobuf'
2025-03-14T04:08:35.0118862Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:08:35.0166519Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:08:35.0216432Z Entering 'third_party/psimd'
2025-03-14T04:08:35.0265902Z Entering 'third_party/pthreadpool'
2025-03-14T04:08:35.0314406Z Entering 'third_party/pybind11'
2025-03-14T04:08:35.0363043Z Entering 'third_party/python-peachpy'
2025-03-14T04:08:35.0411632Z Entering 'third_party/sleef'
2025-03-14T04:08:35.0460660Z Entering 'third_party/tensorpipe'
2025-03-14T04:08:35.0510035Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:08:35.0557510Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:08:35.0605791Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:08:35.0652970Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:08:35.0699046Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:08:35.0767252Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***
2025-03-14T04:08:35.0808612Z ##[endgroup]
2025-03-14T04:08:35.0809070Z ##[group]Fetching the repository
2025-03-14T04:08:35.0816122Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --no-recurse-submodules origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/*
2025-03-14T04:08:35.5049366Z From https://github.com/pytorch/pytorch
2025-03-14T04:08:35.5049866Z  * [new tag]                 ciflow/trunk/149064 -> ciflow/trunk/149064
2025-03-14T04:08:35.5608588Z [command]/usr/bin/git rev-parse --verify --quiet aed0b7a742a2d7b7901790622829cbd2135049a4^{object}
2025-03-14T04:08:35.5630991Z aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:08:35.5634845Z ##[endgroup]
2025-03-14T04:08:35.5635457Z ##[group]Determining the checkout info
2025-03-14T04:08:35.5636194Z ##[endgroup]
2025-03-14T04:08:35.5639210Z [command]/usr/bin/git sparse-checkout disable
2025-03-14T04:08:35.6772736Z [command]/usr/bin/git config --local --unset-all extensions.worktreeConfig
2025-03-14T04:08:35.6797994Z ##[group]Checking out the ref
2025-03-14T04:08:35.6801615Z [command]/usr/bin/git checkout --progress --force aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:08:35.7139615Z HEAD is now at aed0b7a742a [c10d] Add param recording for uniqueID broadcasting and allgather (#149166)
2025-03-14T04:08:35.7154837Z ##[endgroup]
2025-03-14T04:08:35.7155594Z ##[group]Setting up auth for fetching submodules
2025-03-14T04:08:35.7161591Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic ***
2025-03-14T04:08:35.7202231Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf
2025-03-14T04:08:35.7227861Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com:
2025-03-14T04:08:35.7252980Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com:
2025-03-14T04:08:35.7279312Z ##[endgroup]
2025-03-14T04:08:35.7280159Z ##[group]Fetching submodules
2025-03-14T04:08:35.7281053Z [command]/usr/bin/git submodule sync --recursive
2025-03-14T04:08:35.7558804Z Synchronizing submodule url for 'android/libs/fbjni'
2025-03-14T04:08:35.7578575Z Synchronizing submodule url for 'third_party/FP16'
2025-03-14T04:08:35.7598101Z Synchronizing submodule url for 'third_party/FXdiv'
2025-03-14T04:08:35.7618032Z Synchronizing submodule url for 'third_party/NNPACK'
2025-03-14T04:08:35.7638474Z Synchronizing submodule url for 'third_party/NVTX'
2025-03-14T04:08:35.7658610Z Synchronizing submodule url for 'third_party/VulkanMemoryAllocator'
2025-03-14T04:08:35.7678319Z Synchronizing submodule url for 'third_party/XNNPACK'
2025-03-14T04:08:35.7713945Z Synchronizing submodule url for 'third_party/benchmark'
2025-03-14T04:08:35.7733216Z Synchronizing submodule url for 'third_party/composable_kernel'
2025-03-14T04:08:35.7758507Z Synchronizing submodule url for 'third_party/cpp-httplib'
2025-03-14T04:08:35.7777846Z Synchronizing submodule url for 'third_party/cpuinfo'
2025-03-14T04:08:35.7797856Z Synchronizing submodule url for 'third_party/cudnn_frontend'
2025-03-14T04:08:35.7818186Z Synchronizing submodule url for 'third_party/cutlass'
2025-03-14T04:08:35.7847202Z Synchronizing submodule url for 'third_party/eigen'
2025-03-14T04:08:35.7869493Z Synchronizing submodule url for 'third_party/fbgemm'
2025-03-14T04:08:35.7888176Z Synchronizing submodule url for 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:08:35.7907078Z Synchronizing submodule url for 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:08:35.7925205Z Synchronizing submodule url for 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:08:35.7950192Z Synchronizing submodule url for 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:08:35.7967747Z Synchronizing submodule url for 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:08:35.7988680Z Synchronizing submodule url for 'third_party/flash-attention'
2025-03-14T04:08:35.8008234Z Synchronizing submodule url for 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:08:35.8031577Z Synchronizing submodule url for 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:08:35.8062661Z Synchronizing submodule url for 'third_party/flatbuffers'
2025-03-14T04:08:35.8085356Z Synchronizing submodule url for 'third_party/fmt'
2025-03-14T04:08:35.8106471Z Synchronizing submodule url for 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:08:35.8126565Z Synchronizing submodule url for 'third_party/gloo'
2025-03-14T04:08:35.8146107Z Synchronizing submodule url for 'third_party/googletest'
2025-03-14T04:08:35.8165937Z Synchronizing submodule url for 'third_party/ideep'
2025-03-14T04:08:35.8182716Z Synchronizing submodule url for 'third_party/ideep/mkl-dnn'
2025-03-14T04:08:35.8209670Z Synchronizing submodule url for 'third_party/ittapi'
2025-03-14T04:08:35.8230427Z Synchronizing submodule url for 'third_party/kineto'
2025-03-14T04:08:35.8249424Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:08:35.8267815Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:08:35.8288095Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:08:35.8307104Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:08:35.8326528Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:08:35.8344286Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:08:35.8364879Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:08:35.8383765Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:08:35.8404123Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:08:35.8424343Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:08:35.8444769Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:08:35.8462845Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:08:35.8483624Z Synchronizing submodule url for 'third_party/kleidiai'
2025-03-14T04:08:35.8505325Z Synchronizing submodule url for 'third_party/mimalloc'
2025-03-14T04:08:35.8525908Z Synchronizing submodule url for 'third_party/nlohmann'
2025-03-14T04:08:35.8547609Z Synchronizing submodule url for 'third_party/onnx'
2025-03-14T04:08:35.8581306Z Synchronizing submodule url for 'third_party/onnx/third_party/pybind11'
2025-03-14T04:08:35.8606473Z Synchronizing submodule url for 'third_party/opentelemetry-cpp'
2025-03-14T04:08:35.8626967Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:08:35.8645330Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:08:35.8663219Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:08:35.8681069Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:08:35.8700450Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:08:35.8718481Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:08:35.8736658Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:08:35.8753934Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:08:35.8774108Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:08:35.8794691Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:08:35.8834051Z Synchronizing submodule url for 'third_party/pocketfft'
2025-03-14T04:08:35.8854547Z Synchronizing submodule url for 'third_party/protobuf'
2025-03-14T04:08:35.8875524Z Synchronizing submodule url for 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:08:35.8893501Z Synchronizing submodule url for 'third_party/protobuf/third_party/googletest'
2025-03-14T04:08:35.8915348Z Synchronizing submodule url for 'third_party/psimd'
2025-03-14T04:08:35.8935621Z Synchronizing submodule url for 'third_party/pthreadpool'
2025-03-14T04:08:35.8955116Z Synchronizing submodule url for 'third_party/pybind11'
2025-03-14T04:08:35.8974843Z Synchronizing submodule url for 'third_party/python-peachpy'
2025-03-14T04:08:35.8994441Z Synchronizing submodule url for 'third_party/sleef'
2025-03-14T04:08:35.9018350Z Synchronizing submodule url for 'third_party/tensorpipe'
2025-03-14T04:08:35.9036739Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:08:35.9054784Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:08:35.9072465Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:08:35.9090527Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:08:35.9110014Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:08:35.9139861Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive
2025-03-14T04:08:35.9588130Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f'
2025-03-14T04:08:35.9712197Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3'
2025-03-14T04:08:35.9806676Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1'
2025-03-14T04:08:36.0081221Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73'
2025-03-14T04:08:36.0582043Z Submodule path 'third_party/NVTX': checked out 'e170594ac7cf1dac584da473d4ca9301087090c1'
2025-03-14T04:08:36.1136282Z Submodule path 'third_party/VulkanMemoryAllocator': checked out 'a6bfc237255a6bac1513f7c1ebde6d8aed6b5191'
2025-03-14T04:08:37.0746444Z Submodule path 'third_party/XNNPACK': checked out '51a0103656eff6fc9bfd39a4597923c4b542c883'
2025-03-14T04:08:37.1053544Z Submodule path 'third_party/benchmark': checked out '0d98dba29d66e93259db7daa53a9327df767a415'
2025-03-14T04:08:37.4277630Z Submodule path 'third_party/composable_kernel': checked out '8086bbe3a78d931eb96fe12fdc014082e18d18d3'
2025-03-14T04:08:37.4786083Z Submodule path 'third_party/cpp-httplib': checked out '3b6597bba913d51161383657829b7e644e59c006'
2025-03-14T04:08:37.5968130Z Submodule path 'third_party/cpuinfo': checked out '1e83a2fdd3102f65c6f1fb602c1b320486218a99'
2025-03-14T04:08:37.6397864Z Submodule path 'third_party/cudnn_frontend': checked out '91b7532f3386768bba4f444ee7672b497f34da8a'
2025-03-14T04:08:38.4591276Z Submodule path 'third_party/cutlass': checked out 'afa1772203677c5118fcd82537a9c8fefbcc7008'
2025-03-14T04:08:38.8899248Z Submodule path 'third_party/eigen': checked out '3147391d946bb4b6c68edd901f2add6ac1f31f8c'
2025-03-14T04:08:38.9804035Z Submodule path 'third_party/fbgemm': checked out 'dbc3157bf256f1339b3fa1fef2be89ac4078be0e'
2025-03-14T04:08:39.0397130Z Submodule path 'third_party/fbgemm/third_party/asmjit': checked out 'd3fbf7c9bc7c1d1365a94a45614b91c5a3706b81'
2025-03-14T04:08:39.1575410Z Submodule path 'third_party/fbgemm/third_party/cpuinfo': checked out 'ed8b86a253800bafdb7b25c5c399f91bff9cb1f3'
2025-03-14T04:08:39.7248710Z Submodule path 'third_party/fbgemm/third_party/cutlass': checked out 'fc9ebc645b63f3a6bc80aaefde5c063fb72110d6'
2025-03-14T04:08:39.7909383Z Submodule path 'third_party/fbgemm/third_party/googletest': checked out 'cbf019de22c8dd37b2108da35b2748fd702d1796'
2025-03-14T04:08:39.8034203Z Submodule path 'third_party/fbgemm/third_party/hipify_torch': checked out '23f53b025b466d8ec3c45d52290d3442f7fbe6b1'
2025-03-14T04:08:39.9107928Z Submodule path 'third_party/flash-attention': checked out '979702c87a8713a8e0a5e9fee122b90d2ef13be5'
2025-03-14T04:08:40.2192461Z Submodule path 'third_party/flash-attention/csrc/composable_kernel': checked out '888317e698e9803c62bd38568abc9e05d7709f33'
2025-03-14T04:08:40.9340165Z Submodule path 'third_party/flash-attention/csrc/cutlass': checked out 'c506e16788cb08416a4a57e11a9067beeee29420'
2025-03-14T04:08:41.0911756Z Submodule path 'third_party/flatbuffers': checked out '01834de25e4bf3975a9a00e816292b1ad0fe184b'
2025-03-14T04:08:41.1320040Z Submodule path 'third_party/fmt': checked out '123913715afeb8a437e6388b4473fcc4753e1c9a'
2025-03-14T04:08:41.1768885Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350'
2025-03-14T04:08:41.2092499Z Submodule path 'third_party/gloo': checked out '5354032ea08eadd7fc4456477f7f7c6308818509'
2025-03-14T04:08:41.2604410Z Submodule path 'third_party/googletest': checked out 'b514bdc898e2951020cbdca1304b75f5950d1f59'
2025-03-14T04:08:41.2754848Z Submodule path 'third_party/ideep': checked out '719d8e6cd7f7a0e01b155657526d693acf97c2b3'
2025-03-14T04:08:41.9971300Z Submodule path 'third_party/ideep/mkl-dnn': checked out '8d263e693366ef8db40acc569cc7d8edf644556d'
2025-03-14T04:08:42.0157492Z Submodule path 'third_party/ittapi': checked out '5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42'
2025-03-14T04:08:42.1169619Z Submodule path 'third_party/kineto': checked out '2859721fd9e73d3ca1c56f827dbc64e6d68f78a2'
2025-03-14T04:08:42.2152035Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e'
2025-03-14T04:08:42.4411686Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9'
2025-03-14T04:08:42.4641461Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400'
2025-03-14T04:08:42.5269205Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05'
2025-03-14T04:08:42.5418142Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067'
2025-03-14T04:08:42.5522526Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4'
2025-03-14T04:08:42.5759648Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446'
2025-03-14T04:08:42.6310597Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850'
2025-03-14T04:08:42.7833108Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5'
2025-03-14T04:08:42.8022515Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150'
2025-03-14T04:08:42.8464646Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164'
2025-03-14T04:08:42.9121217Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347'
2025-03-14T04:08:42.9548052Z Submodule path 'third_party/kleidiai': checked out 'ef685a13cfbe8d418aa2ed34350e21e4938358b6'
2025-03-14T04:08:43.0060511Z Submodule path 'third_party/mimalloc': checked out 'b66e3214d8a104669c2ec05ae91ebc26a8f5ab78'
2025-03-14T04:08:43.1623646Z Submodule path 'third_party/nlohmann': checked out '87cda1d6646592ac5866dc703c8e1839046a6806'
2025-03-14T04:08:43.6255899Z Submodule path 'third_party/onnx': checked out 'b8baa8446686496da4cc8fda09f2b6fe65c2a02c'
2025-03-14T04:08:43.6768023Z Submodule path 'third_party/onnx/third_party/pybind11': checked out '3e9dfa2866941655c56877882565e7577de6fc7b'
2025-03-14T04:08:43.7826231Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878'
2025-03-14T04:08:43.8095530Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2'
2025-03-14T04:08:43.8654967Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1'
2025-03-14T04:08:43.8826670Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa'
2025-03-14T04:08:44.0176141Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d'
2025-03-14T04:08:44.0316239Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce'
2025-03-14T04:08:44.0488271Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5'
2025-03-14T04:08:44.0703550Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d'
2025-03-14T04:08:44.3670586Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4'
2025-03-14T04:08:44.4346646Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929'
2025-03-14T04:08:45.1342039Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50'
2025-03-14T04:08:45.1457377Z Submodule path 'third_party/pocketfft': checked out '9d3ab05a7fffbc71a492bc6a17be034e83e8f0fe'
2025-03-14T04:08:45.5506372Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a'
2025-03-14T04:08:45.5737543Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8'
2025-03-14T04:08:45.6451168Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081'
2025-03-14T04:08:45.6546597Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900'
2025-03-14T04:08:45.6670593Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8'
2025-03-14T04:08:45.7113600Z Submodule path 'third_party/pybind11': checked out 'a2e59f0e7065404b44dfe92a28aca47ba1378dc4'
2025-03-14T04:08:45.7453191Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67'
2025-03-14T04:08:45.8064962Z Submodule path 'third_party/sleef': checked out '56e1f79cb140fb9326d612d0be06b5250565cade'
2025-03-14T04:08:45.8437923Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e'
2025-03-14T04:08:45.9108289Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e'
2025-03-14T04:08:45.9304784Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281'
2025-03-14T04:08:46.0249239Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242'
2025-03-14T04:08:46.0686421Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef'
2025-03-14T04:08:46.0782668Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5'
2025-03-14T04:08:46.0819003Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0
2025-03-14T04:08:46.1088551Z Entering 'android/libs/fbjni'
2025-03-14T04:08:46.1127613Z Entering 'third_party/FP16'
2025-03-14T04:08:46.1165960Z Entering 'third_party/FXdiv'
2025-03-14T04:08:46.1203414Z Entering 'third_party/NNPACK'
2025-03-14T04:08:46.1240351Z Entering 'third_party/NVTX'
2025-03-14T04:08:46.1277617Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:08:46.1314661Z Entering 'third_party/XNNPACK'
2025-03-14T04:08:46.1367635Z Entering 'third_party/benchmark'
2025-03-14T04:08:46.1405506Z Entering 'third_party/composable_kernel'
2025-03-14T04:08:46.1449837Z Entering 'third_party/cpp-httplib'
2025-03-14T04:08:46.1487062Z Entering 'third_party/cpuinfo'
2025-03-14T04:08:46.1525539Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:08:46.1562190Z Entering 'third_party/cutlass'
2025-03-14T04:08:46.1607279Z Entering 'third_party/eigen'
2025-03-14T04:08:46.1645904Z Entering 'third_party/fbgemm'
2025-03-14T04:08:46.1683046Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:08:46.1720947Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:08:46.1756347Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:08:46.1799210Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:08:46.1835229Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:08:46.1872609Z Entering 'third_party/flash-attention'
2025-03-14T04:08:46.1910763Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:08:46.1952268Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:08:46.1997308Z Entering 'third_party/flatbuffers'
2025-03-14T04:08:46.2036926Z Entering 'third_party/fmt'
2025-03-14T04:08:46.2073797Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:08:46.2110628Z Entering 'third_party/gloo'
2025-03-14T04:08:46.2147589Z Entering 'third_party/googletest'
2025-03-14T04:08:46.2184462Z Entering 'third_party/ideep'
2025-03-14T04:08:46.2220855Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:08:46.2263820Z Entering 'third_party/ittapi'
2025-03-14T04:08:46.2301653Z Entering 'third_party/kineto'
2025-03-14T04:08:46.2339440Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:08:46.2375777Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:08:46.2414008Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:08:46.2451234Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:08:46.2490334Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:08:46.2526422Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:08:46.2564843Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:08:46.2601539Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:08:46.2638977Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:08:46.2676275Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:08:46.2714282Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:08:46.2750602Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:08:46.2788544Z Entering 'third_party/kleidiai'
2025-03-14T04:08:46.2827694Z Entering 'third_party/mimalloc'
2025-03-14T04:08:46.2865042Z Entering 'third_party/nlohmann'
2025-03-14T04:08:46.2903577Z Entering 'third_party/onnx'
2025-03-14T04:08:46.2956104Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:08:46.2995175Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:08:46.3034037Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:08:46.3070138Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:08:46.3107237Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:08:46.3142621Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:08:46.3179440Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:08:46.3217457Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:08:46.3253304Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:08:46.3289109Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:08:46.3327447Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:08:46.3365727Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:08:46.3422841Z Entering 'third_party/pocketfft'
2025-03-14T04:08:46.3460756Z Entering 'third_party/protobuf'
2025-03-14T04:08:46.3503832Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:08:46.3540274Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:08:46.3578945Z Entering 'third_party/psimd'
2025-03-14T04:08:46.3619611Z Entering 'third_party/pthreadpool'
2025-03-14T04:08:46.3656409Z Entering 'third_party/pybind11'
2025-03-14T04:08:46.3693762Z Entering 'third_party/python-peachpy'
2025-03-14T04:08:46.3731338Z Entering 'third_party/sleef'
2025-03-14T04:08:46.3769832Z Entering 'third_party/tensorpipe'
2025-03-14T04:08:46.3807833Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:08:46.3844129Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:08:46.3880490Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:08:46.3917353Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:08:46.3952821Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:08:46.4004828Z ##[endgroup]
2025-03-14T04:08:46.4005299Z ##[group]Persisting credentials for submodules
2025-03-14T04:08:46.4009614Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :"
2025-03-14T04:08:46.4270692Z Entering 'android/libs/fbjni'
2025-03-14T04:08:46.4303637Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4304218Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4333700Z Entering 'third_party/FP16'
2025-03-14T04:08:46.4367160Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4367567Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4396848Z Entering 'third_party/FXdiv'
2025-03-14T04:08:46.4429157Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4429558Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4458563Z Entering 'third_party/NNPACK'
2025-03-14T04:08:46.4490834Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4491236Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4520965Z Entering 'third_party/NVTX'
2025-03-14T04:08:46.4553858Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4554352Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4585363Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:08:46.4621273Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4621664Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4651340Z Entering 'third_party/XNNPACK'
2025-03-14T04:08:46.4683781Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4684174Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4729723Z Entering 'third_party/benchmark'
2025-03-14T04:08:46.4762324Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4762715Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4792180Z Entering 'third_party/composable_kernel'
2025-03-14T04:08:46.4824857Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4825329Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4862097Z Entering 'third_party/cpp-httplib'
2025-03-14T04:08:46.4894410Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4894980Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4925917Z Entering 'third_party/cpuinfo'
2025-03-14T04:08:46.4959019Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4959407Z url.https://github.com/.insteadof
2025-03-14T04:08:46.4989515Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:08:46.5022678Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5023100Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5052731Z Entering 'third_party/cutlass'
2025-03-14T04:08:46.5085028Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5085426Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5124090Z Entering 'third_party/eigen'
2025-03-14T04:08:46.5156760Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5157141Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5188444Z Entering 'third_party/fbgemm'
2025-03-14T04:08:46.5220811Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5221191Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5251575Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:08:46.5283136Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5283527Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5312567Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:08:46.5343705Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5344088Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5373420Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:08:46.5405141Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5405772Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5442043Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:08:46.5473631Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5474024Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5505295Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:08:46.5536621Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5537008Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5567207Z Entering 'third_party/flash-attention'
2025-03-14T04:08:46.5599347Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5599737Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5630950Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:08:46.5662964Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5663529Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5722012Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:08:46.5731065Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5731443Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5769254Z Entering 'third_party/flatbuffers'
2025-03-14T04:08:46.5803146Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5803798Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5835988Z Entering 'third_party/fmt'
2025-03-14T04:08:46.5868566Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5869204Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5898214Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:08:46.5930224Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5960019Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5960542Z Entering 'third_party/gloo'
2025-03-14T04:08:46.5992553Z url.https://github.com/.insteadof
2025-03-14T04:08:46.5993201Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6022443Z Entering 'third_party/googletest'
2025-03-14T04:08:46.6054554Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6055172Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6084372Z Entering 'third_party/ideep'
2025-03-14T04:08:46.6117003Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6117629Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6146579Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:08:46.6178179Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6178611Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6216229Z Entering 'third_party/ittapi'
2025-03-14T04:08:46.6248819Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6278293Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6278679Z Entering 'third_party/kineto'
2025-03-14T04:08:46.6312104Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6312481Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6342023Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:08:46.6373698Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6374130Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6403887Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:08:46.6435878Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6436263Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6467487Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:08:46.6499733Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6500108Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6530750Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:08:46.6562794Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6563157Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6593445Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:08:46.6625688Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6626053Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6656295Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:08:46.6689589Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6690254Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6723113Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:08:46.6754807Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6755407Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6784750Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:08:46.6816724Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6817364Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6847641Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:08:46.6879252Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6879657Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6910710Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:08:46.6942751Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6943324Z url.https://github.com/.insteadof
2025-03-14T04:08:46.6974983Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:08:46.7007608Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7008259Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7037517Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:08:46.7069368Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7069920Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7101765Z Entering 'third_party/kleidiai'
2025-03-14T04:08:46.7134362Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7134982Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7164749Z Entering 'third_party/mimalloc'
2025-03-14T04:08:46.7197110Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7197734Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7227517Z Entering 'third_party/nlohmann'
2025-03-14T04:08:46.7259892Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7260517Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7290703Z Entering 'third_party/onnx'
2025-03-14T04:08:46.7322955Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7323563Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7369180Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:08:46.7402326Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7402898Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7435140Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:08:46.7468943Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7469528Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7501377Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:08:46.7533471Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7533843Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7562851Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:08:46.7595603Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7595986Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7625334Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:08:46.7657068Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7657429Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7686521Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:08:46.7718349Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7718722Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7749628Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:08:46.7781902Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7782275Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7811878Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:08:46.7843665Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7844042Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7873234Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:08:46.7905439Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7905825Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7935195Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:08:46.7968241Z url.https://github.com/.insteadof
2025-03-14T04:08:46.7968645Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8000761Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:08:46.8034198Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8034875Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8067200Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:08:46.8099402Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8099791Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8149018Z Entering 'third_party/pocketfft'
2025-03-14T04:08:46.8182575Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8183132Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8213317Z Entering 'third_party/protobuf'
2025-03-14T04:08:46.8246191Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8246578Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8279428Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:08:46.8312090Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8312746Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8343131Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:08:46.8374450Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8374839Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8406647Z Entering 'third_party/psimd'
2025-03-14T04:08:46.8439776Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8440344Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8469542Z Entering 'third_party/pthreadpool'
2025-03-14T04:08:46.8502448Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8503025Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8531971Z Entering 'third_party/pybind11'
2025-03-14T04:08:46.8564986Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8565554Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8594387Z Entering 'third_party/python-peachpy'
2025-03-14T04:08:46.8628096Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8628469Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8657998Z Entering 'third_party/sleef'
2025-03-14T04:08:46.8690331Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8690882Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8720349Z Entering 'third_party/tensorpipe'
2025-03-14T04:08:46.8753765Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8754214Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8783098Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:08:46.8814698Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8815345Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8844531Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:08:46.8876358Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8877005Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8905788Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:08:46.8937147Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8937818Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8966712Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:08:46.8998363Z url.https://github.com/.insteadof
2025-03-14T04:08:46.8998770Z url.https://github.com/.insteadof
2025-03-14T04:08:46.9027178Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:08:46.9059468Z url.https://github.com/.insteadof
2025-03-14T04:08:46.9059832Z url.https://github.com/.insteadof
2025-03-14T04:08:46.9106117Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url"
2025-03-14T04:08:46.9367665Z Entering 'android/libs/fbjni'
2025-03-14T04:08:46.9412957Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config	remote.origin.url
2025-03-14T04:08:46.9427583Z Entering 'third_party/FP16'
2025-03-14T04:08:46.9472408Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config	remote.origin.url
2025-03-14T04:08:46.9488040Z Entering 'third_party/FXdiv'
2025-03-14T04:08:46.9533736Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config	remote.origin.url
2025-03-14T04:08:46.9548220Z Entering 'third_party/NNPACK'
2025-03-14T04:08:46.9592350Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config	remote.origin.url
2025-03-14T04:08:46.9607923Z Entering 'third_party/NVTX'
2025-03-14T04:08:46.9652364Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NVTX/config	remote.origin.url
2025-03-14T04:08:46.9667678Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:08:46.9712887Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config	remote.origin.url
2025-03-14T04:08:46.9727829Z Entering 'third_party/XNNPACK'
2025-03-14T04:08:46.9772621Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config	remote.origin.url
2025-03-14T04:08:46.9803112Z Entering 'third_party/benchmark'
2025-03-14T04:08:46.9847929Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config	remote.origin.url
2025-03-14T04:08:46.9862820Z Entering 'third_party/composable_kernel'
2025-03-14T04:08:46.9907800Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/composable_kernel/config	remote.origin.url
2025-03-14T04:08:46.9928375Z Entering 'third_party/cpp-httplib'
2025-03-14T04:08:46.9973695Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config	remote.origin.url
2025-03-14T04:08:46.9988474Z Entering 'third_party/cpuinfo'
2025-03-14T04:08:47.0033240Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config	remote.origin.url
2025-03-14T04:08:47.0049468Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:08:47.0095705Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config	remote.origin.url
2025-03-14T04:08:47.0111396Z Entering 'third_party/cutlass'
2025-03-14T04:08:47.0156970Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config	remote.origin.url
2025-03-14T04:08:47.0180033Z Entering 'third_party/eigen'
2025-03-14T04:08:47.0225886Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/eigen/config	remote.origin.url
2025-03-14T04:08:47.0242689Z Entering 'third_party/fbgemm'
2025-03-14T04:08:47.0288564Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config	remote.origin.url
2025-03-14T04:08:47.0303356Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:08:47.0349520Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/asmjit/config	remote.origin.url
2025-03-14T04:08:47.0363661Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:08:47.0408401Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cpuinfo/config	remote.origin.url
2025-03-14T04:08:47.0422998Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:08:47.0467550Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/cutlass/config	remote.origin.url
2025-03-14T04:08:47.0488035Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:08:47.0532800Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/googletest/config	remote.origin.url
2025-03-14T04:08:47.0547284Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:08:47.0591795Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/third_party/hipify_torch/config	remote.origin.url
2025-03-14T04:08:47.0607569Z Entering 'third_party/flash-attention'
2025-03-14T04:08:47.0654566Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/config	remote.origin.url
2025-03-14T04:08:47.0669701Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:08:47.0715543Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/composable_kernel/config	remote.origin.url
2025-03-14T04:08:47.0735195Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:08:47.0779463Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/cutlass/config	remote.origin.url
2025-03-14T04:08:47.0803152Z Entering 'third_party/flatbuffers'
2025-03-14T04:08:47.0849543Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config	remote.origin.url
2025-03-14T04:08:47.0866542Z Entering 'third_party/fmt'
2025-03-14T04:08:47.0913607Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config	remote.origin.url
2025-03-14T04:08:47.0928247Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:08:47.0973062Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config	remote.origin.url
2025-03-14T04:08:47.0987874Z Entering 'third_party/gloo'
2025-03-14T04:08:47.1032608Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config	remote.origin.url
2025-03-14T04:08:47.1047601Z Entering 'third_party/googletest'
2025-03-14T04:08:47.1091977Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config	remote.origin.url
2025-03-14T04:08:47.1107319Z Entering 'third_party/ideep'
2025-03-14T04:08:47.1152842Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config	remote.origin.url
2025-03-14T04:08:47.1166579Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:08:47.1213032Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config	remote.origin.url
2025-03-14T04:08:47.1235760Z Entering 'third_party/ittapi'
2025-03-14T04:08:47.1281989Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config	remote.origin.url
2025-03-14T04:08:47.1296479Z Entering 'third_party/kineto'
2025-03-14T04:08:47.1342783Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config	remote.origin.url
2025-03-14T04:08:47.1357169Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:08:47.1403088Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config	remote.origin.url
2025-03-14T04:08:47.1417206Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:08:47.1463094Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config	remote.origin.url
2025-03-14T04:08:47.1477921Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:08:47.1524559Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config	remote.origin.url
2025-03-14T04:08:47.1538115Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:08:47.1583333Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config	remote.origin.url
2025-03-14T04:08:47.1597971Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:08:47.1644059Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config	remote.origin.url
2025-03-14T04:08:47.1657094Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:08:47.1703662Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config	remote.origin.url
2025-03-14T04:08:47.1719855Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:08:47.1765268Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config	remote.origin.url
2025-03-14T04:08:47.1779137Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:08:47.1825329Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config	remote.origin.url
2025-03-14T04:08:47.1839736Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:08:47.1884797Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config	remote.origin.url
2025-03-14T04:08:47.1902174Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:08:47.1947883Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config	remote.origin.url
2025-03-14T04:08:47.1963833Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:08:47.2008987Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config	remote.origin.url
2025-03-14T04:08:47.2023069Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:08:47.2067634Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config	remote.origin.url
2025-03-14T04:08:47.2083320Z Entering 'third_party/kleidiai'
2025-03-14T04:08:47.2129163Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kleidiai/config	remote.origin.url
2025-03-14T04:08:47.2145200Z Entering 'third_party/mimalloc'
2025-03-14T04:08:47.2189182Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config	remote.origin.url
2025-03-14T04:08:47.2203574Z Entering 'third_party/nlohmann'
2025-03-14T04:08:47.2247885Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config	remote.origin.url
2025-03-14T04:08:47.2263676Z Entering 'third_party/onnx'
2025-03-14T04:08:47.2308592Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config	remote.origin.url
2025-03-14T04:08:47.2338131Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:08:47.2383653Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config	remote.origin.url
2025-03-14T04:08:47.2401794Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:08:47.2448303Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config	remote.origin.url
2025-03-14T04:08:47.2463946Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:08:47.2509821Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config	remote.origin.url
2025-03-14T04:08:47.2523652Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:08:47.2569823Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config	remote.origin.url
2025-03-14T04:08:47.2584063Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:08:47.2629298Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config	remote.origin.url
2025-03-14T04:08:47.2643424Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:08:47.2688055Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config	remote.origin.url
2025-03-14T04:08:47.2703304Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:08:47.2747827Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config	remote.origin.url
2025-03-14T04:08:47.2761609Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:08:47.2806814Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config	remote.origin.url
2025-03-14T04:08:47.2820486Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:08:47.2865737Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config	remote.origin.url
2025-03-14T04:08:47.2878600Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:08:47.2924129Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config	remote.origin.url
2025-03-14T04:08:47.2940211Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:08:47.2987178Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config	remote.origin.url
2025-03-14T04:08:47.3003039Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:08:47.3047752Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config	remote.origin.url
2025-03-14T04:08:47.3081716Z Entering 'third_party/pocketfft'
2025-03-14T04:08:47.3127897Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config	remote.origin.url
2025-03-14T04:08:47.3141726Z Entering 'third_party/protobuf'
2025-03-14T04:08:47.3187485Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config	remote.origin.url
2025-03-14T04:08:47.3205476Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:08:47.3249963Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config	remote.origin.url
2025-03-14T04:08:47.3263643Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:08:47.3308750Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config	remote.origin.url
2025-03-14T04:08:47.3324707Z Entering 'third_party/psimd'
2025-03-14T04:08:47.3370792Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config	remote.origin.url
2025-03-14T04:08:47.3385060Z Entering 'third_party/pthreadpool'
2025-03-14T04:08:47.3429841Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config	remote.origin.url
2025-03-14T04:08:47.3444588Z Entering 'third_party/pybind11'
2025-03-14T04:08:47.3489621Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config	remote.origin.url
2025-03-14T04:08:47.3504653Z Entering 'third_party/python-peachpy'
2025-03-14T04:08:47.3549752Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config	remote.origin.url
2025-03-14T04:08:47.3564422Z Entering 'third_party/sleef'
2025-03-14T04:08:47.3610214Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config	remote.origin.url
2025-03-14T04:08:47.3625030Z Entering 'third_party/tensorpipe'
2025-03-14T04:08:47.3670450Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config	remote.origin.url
2025-03-14T04:08:47.3685511Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:08:47.3733631Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config	remote.origin.url
2025-03-14T04:08:47.3747736Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:08:47.3791991Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config	remote.origin.url
2025-03-14T04:08:47.3807174Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:08:47.3851237Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config	remote.origin.url
2025-03-14T04:08:47.3866634Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:08:47.3910572Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config	remote.origin.url
2025-03-14T04:08:47.3923711Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:08:47.3969830Z file:/home/ec2-user/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config	remote.origin.url
2025-03-14T04:08:47.4885048Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:'
2025-03-14T04:08:47.5156360Z Entering 'android/libs/fbjni'
2025-03-14T04:08:47.5194561Z Entering 'third_party/FP16'
2025-03-14T04:08:47.5232387Z Entering 'third_party/FXdiv'
2025-03-14T04:08:47.5270176Z Entering 'third_party/NNPACK'
2025-03-14T04:08:47.5308644Z Entering 'third_party/NVTX'
2025-03-14T04:08:47.5347876Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:08:47.5385328Z Entering 'third_party/XNNPACK'
2025-03-14T04:08:47.5438702Z Entering 'third_party/benchmark'
2025-03-14T04:08:47.5477062Z Entering 'third_party/composable_kernel'
2025-03-14T04:08:47.5520517Z Entering 'third_party/cpp-httplib'
2025-03-14T04:08:47.5557642Z Entering 'third_party/cpuinfo'
2025-03-14T04:08:47.5595489Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:08:47.5633753Z Entering 'third_party/cutlass'
2025-03-14T04:08:47.5677961Z Entering 'third_party/eigen'
2025-03-14T04:08:47.5717835Z Entering 'third_party/fbgemm'
2025-03-14T04:08:47.5756018Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:08:47.5793827Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:08:47.5830805Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:08:47.5873535Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:08:47.5909521Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:08:47.5946731Z Entering 'third_party/flash-attention'
2025-03-14T04:08:47.5984851Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:08:47.6026707Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:08:47.6071489Z Entering 'third_party/flatbuffers'
2025-03-14T04:08:47.6112691Z Entering 'third_party/fmt'
2025-03-14T04:08:47.6150004Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:08:47.6187229Z Entering 'third_party/gloo'
2025-03-14T04:08:47.6225267Z Entering 'third_party/googletest'
2025-03-14T04:08:47.6262919Z Entering 'third_party/ideep'
2025-03-14T04:08:47.6300213Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:08:47.6343844Z Entering 'third_party/ittapi'
2025-03-14T04:08:47.6380576Z Entering 'third_party/kineto'
2025-03-14T04:08:47.6418719Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:08:47.6455530Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:08:47.6494105Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:08:47.6530926Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:08:47.6567499Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:08:47.6604323Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:08:47.6642843Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:08:47.6679573Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:08:47.6718908Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:08:47.6757166Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:08:47.6795210Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:08:47.6833259Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:08:47.6870786Z Entering 'third_party/kleidiai'
2025-03-14T04:08:47.6909831Z Entering 'third_party/mimalloc'
2025-03-14T04:08:47.6947005Z Entering 'third_party/nlohmann'
2025-03-14T04:08:47.6985530Z Entering 'third_party/onnx'
2025-03-14T04:08:47.7038998Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:08:47.7077919Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:08:47.7118569Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:08:47.7153958Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:08:47.7189328Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:08:47.7224880Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:08:47.7262301Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:08:47.7297753Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:08:47.7333557Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:08:47.7369052Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:08:47.7407839Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:08:47.7446995Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:08:47.7503286Z Entering 'third_party/pocketfft'
2025-03-14T04:08:47.7540724Z Entering 'third_party/protobuf'
2025-03-14T04:08:47.7581655Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:08:47.7618407Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:08:47.7656649Z Entering 'third_party/psimd'
2025-03-14T04:08:47.7693834Z Entering 'third_party/pthreadpool'
2025-03-14T04:08:47.7731162Z Entering 'third_party/pybind11'
2025-03-14T04:08:47.7768376Z Entering 'third_party/python-peachpy'
2025-03-14T04:08:47.7805880Z Entering 'third_party/sleef'
2025-03-14T04:08:47.7843259Z Entering 'third_party/tensorpipe'
2025-03-14T04:08:47.7881038Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:08:47.7917292Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:08:47.7953229Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:08:47.7989357Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:08:47.8025261Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:08:47.8079244Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:'
2025-03-14T04:08:47.8341530Z Entering 'android/libs/fbjni'
2025-03-14T04:08:47.8380474Z Entering 'third_party/FP16'
2025-03-14T04:08:47.8418199Z Entering 'third_party/FXdiv'
2025-03-14T04:08:47.8455068Z Entering 'third_party/NNPACK'
2025-03-14T04:08:47.8492523Z Entering 'third_party/NVTX'
2025-03-14T04:08:47.8531813Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T04:08:47.8569100Z Entering 'third_party/XNNPACK'
2025-03-14T04:08:47.8622038Z Entering 'third_party/benchmark'
2025-03-14T04:08:47.8659572Z Entering 'third_party/composable_kernel'
2025-03-14T04:08:47.8702893Z Entering 'third_party/cpp-httplib'
2025-03-14T04:08:47.8740102Z Entering 'third_party/cpuinfo'
2025-03-14T04:08:47.8777756Z Entering 'third_party/cudnn_frontend'
2025-03-14T04:08:47.8815284Z Entering 'third_party/cutlass'
2025-03-14T04:08:47.8859504Z Entering 'third_party/eigen'
2025-03-14T04:08:47.8898499Z Entering 'third_party/fbgemm'
2025-03-14T04:08:47.8936893Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T04:08:47.8972332Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T04:08:47.9009107Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T04:08:47.9050191Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T04:08:47.9086058Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T04:08:47.9123691Z Entering 'third_party/flash-attention'
2025-03-14T04:08:47.9163069Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T04:08:47.9207145Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T04:08:47.9253864Z Entering 'third_party/flatbuffers'
2025-03-14T04:08:47.9293010Z Entering 'third_party/fmt'
2025-03-14T04:08:47.9330560Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T04:08:47.9368167Z Entering 'third_party/gloo'
2025-03-14T04:08:47.9405672Z Entering 'third_party/googletest'
2025-03-14T04:08:47.9444001Z Entering 'third_party/ideep'
2025-03-14T04:08:47.9481571Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T04:08:47.9526449Z Entering 'third_party/ittapi'
2025-03-14T04:08:47.9563504Z Entering 'third_party/kineto'
2025-03-14T04:08:47.9601863Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T04:08:47.9638174Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T04:08:47.9676281Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T04:08:47.9712513Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T04:08:47.9749139Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T04:08:47.9785359Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T04:08:47.9824733Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T04:08:47.9860934Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T04:08:47.9898041Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T04:08:47.9936117Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T04:08:47.9975974Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T04:08:48.0012430Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T04:08:48.0051349Z Entering 'third_party/kleidiai'
2025-03-14T04:08:48.0089344Z Entering 'third_party/mimalloc'
2025-03-14T04:08:48.0128741Z Entering 'third_party/nlohmann'
2025-03-14T04:08:48.0167234Z Entering 'third_party/onnx'
2025-03-14T04:08:48.0220781Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T04:08:48.0260189Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T04:08:48.0300368Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T04:08:48.0337193Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T04:08:48.0373675Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T04:08:48.0409717Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T04:08:48.0447200Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T04:08:48.0483114Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T04:08:48.0519376Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T04:08:48.0555529Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T04:08:48.0594647Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T04:08:48.0633786Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T04:08:48.0689866Z Entering 'third_party/pocketfft'
2025-03-14T04:08:48.0728052Z Entering 'third_party/protobuf'
2025-03-14T04:08:48.0768963Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T04:08:48.0805836Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T04:08:48.0845180Z Entering 'third_party/psimd'
2025-03-14T04:08:48.0883347Z Entering 'third_party/pthreadpool'
2025-03-14T04:08:48.0923359Z Entering 'third_party/pybind11'
2025-03-14T04:08:48.0960808Z Entering 'third_party/python-peachpy'
2025-03-14T04:08:48.0999068Z Entering 'third_party/sleef'
2025-03-14T04:08:48.1036924Z Entering 'third_party/tensorpipe'
2025-03-14T04:08:48.1076403Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T04:08:48.1112171Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T04:08:48.1148209Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T04:08:48.1185278Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T04:08:48.1226150Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T04:08:48.1275754Z ##[endgroup]
2025-03-14T04:08:48.1308834Z [command]/usr/bin/git log -1 --format=%H
2025-03-14T04:08:48.1327996Z aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:08:48.1501388Z Prepare all required actions
2025-03-14T04:08:48.1502014Z Getting action download info
2025-03-14T04:08:48.2727582Z ##[group]Run ./.github/actions/setup-linux
2025-03-14T04:08:48.2727949Z env:
2025-03-14T04:08:48.2728197Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:48.2728484Z ##[endgroup]
2025-03-14T04:08:48.2775492Z ##[group]Run set -euo pipefail
2025-03-14T04:08:48.2775879Z set -euo pipefail
2025-03-14T04:08:48.2776205Z function get_ec2_metadata() {
2025-03-14T04:08:48.2776623Z   # Pulled from instance metadata endpoint for EC2
2025-03-14T04:08:48.2777292Z   # see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html
2025-03-14T04:08:48.2777888Z   category=$1
2025-03-14T04:08:48.2778275Z   # If it is GCP runner (runner name contains gcp), do not run this
2025-03-14T04:08:48.2778740Z   runner_name_str=i-09d49014f68772654
2025-03-14T04:08:48.2779149Z   if [[ -f /.inarc ]]; then
2025-03-14T04:08:48.2779524Z     echo "ARC Runner, no info on ec2 metadata"
2025-03-14T04:08:48.2779944Z   elif [[ $runner_name_str == *"gcp"* ]]; then
2025-03-14T04:08:48.2780457Z     echo "Runner is from Google Cloud Platform, No info on ec2 metadata"
2025-03-14T04:08:48.2780915Z   else
2025-03-14T04:08:48.2781805Z     curl -H "X-aws-ec2-metadata-token: $(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 30")" -fsSL "http://169.254.169.254/latest/meta-data/${category}"
2025-03-14T04:08:48.2782757Z   fi
2025-03-14T04:08:48.2782995Z }
2025-03-14T04:08:48.2783282Z echo "ami-id: $(get_ec2_metadata ami-id)"
2025-03-14T04:08:48.2783744Z echo "instance-id: $(get_ec2_metadata instance-id)"
2025-03-14T04:08:48.2784256Z echo "instance-type: $(get_ec2_metadata instance-type)"
2025-03-14T04:08:48.2784705Z echo "system info $(uname -a)"
2025-03-14T04:08:48.2791451Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:08:48.2791861Z env:
2025-03-14T04:08:48.2792108Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:48.2792409Z ##[endgroup]
2025-03-14T04:08:48.2916905Z ami-id: ami-05b10e08d247fb927
2025-03-14T04:08:48.3003523Z instance-id: i-09d49014f68772654
2025-03-14T04:08:48.3089045Z instance-type: c5.2xlarge
2025-03-14T04:08:48.3099302Z system info Linux ip-10-0-57-87.ec2.internal 6.1.128-136.201.amzn2023.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Feb 10 16:18:01 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
2025-03-14T04:08:48.3127738Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT"
2025-03-14T04:08:48.3128725Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT"
2025-03-14T04:08:48.3135148Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:08:48.3135574Z env:
2025-03-14T04:08:48.3135825Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:48.3136110Z ##[endgroup]
2025-03-14T04:08:48.3194488Z ##[group]Run if systemctl is-active --quiet docker; then
2025-03-14T04:08:48.3195065Z if systemctl is-active --quiet docker; then
2025-03-14T04:08:48.3195477Z     echo "Docker daemon is running...";
2025-03-14T04:08:48.3195831Z else
2025-03-14T04:08:48.3196212Z     echo "Starting docker deamon..." && sudo systemctl start docker;
2025-03-14T04:08:48.3196664Z fi
2025-03-14T04:08:48.3202024Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:08:48.3202430Z env:
2025-03-14T04:08:48.3202672Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:48.3202961Z ##[endgroup]
2025-03-14T04:08:48.3267631Z Docker daemon is running...
2025-03-14T04:08:48.3313008Z ##[group]Run nick-fields/retry@v3.0.0
2025-03-14T04:08:48.3313327Z with:
2025-03-14T04:08:48.3313557Z   shell: bash
2025-03-14T04:08:48.3313965Z   timeout_minutes: 5
2025-03-14T04:08:48.3314234Z   max_attempts: 3
2025-03-14T04:08:48.3314497Z   retry_wait_seconds: 30
2025-03-14T04:08:48.3316881Z   command: AWS_ACCOUNT_ID=$(aws sts get-caller-identity|grep Account|cut -f4 -d\")
aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \
    --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com"

# For LF Runners we need to make sure we also login to Meta's ECR docker registry too.
META_AWS_ACCOUNT_ID=308535385114
if [ "$AWS_ACCOUNT_ID" != "$META_AWS_ACCOUNT_ID" ] ; then
    aws ecr get-login-password --region "$AWS_DEFAULT_REGION" | docker login --username AWS \
        --password-stdin "$META_AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com"
fi

2025-03-14T04:08:48.3319304Z   polling_interval_seconds: 1
2025-03-14T04:08:48.3319612Z   warning_on_retry: true
2025-03-14T04:08:48.3319901Z   continue_on_error: false
2025-03-14T04:08:48.3320183Z env:
2025-03-14T04:08:48.3320418Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:48.3320706Z   AWS_RETRY_MODE: standard
2025-03-14T04:08:48.3320978Z   AWS_MAX_ATTEMPTS: 5
2025-03-14T04:08:48.3321384Z   AWS_DEFAULT_REGION: us-east-1
2025-03-14T04:08:48.3321684Z ##[endgroup]
2025-03-14T04:08:49.4762947Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json.
2025-03-14T04:08:49.4764182Z Configure a credential helper to remove this warning. See
2025-03-14T04:08:49.4764812Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store
2025-03-14T04:08:49.4765235Z 
2025-03-14T04:08:49.4765336Z Login Succeeded
2025-03-14T04:08:49.9883643Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json.
2025-03-14T04:08:49.9884609Z Configure a credential helper to remove this warning. See
2025-03-14T04:08:49.9885477Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store
2025-03-14T04:08:49.9886183Z 
2025-03-14T04:08:49.9886360Z Login Succeeded
2025-03-14T04:08:50.4142531Z Command completed after 1 attempt(s).
2025-03-14T04:08:50.4201733Z ##[group]Run env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}"
2025-03-14T04:08:50.4202309Z env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}"
2025-03-14T04:08:50.4202821Z env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}"
2025-03-14T04:08:50.4209363Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:08:50.4209773Z env:
2025-03-14T04:08:50.4210018Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:50.4210307Z ##[endgroup]
2025-03-14T04:08:50.4286912Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty
2025-03-14T04:08:50.4287509Z # ignore expansion of "docker ps -q" since it could be empty
2025-03-14T04:08:50.4288113Z # shellcheck disable=SC2046
2025-03-14T04:08:50.4288565Z docker stop $(docker ps -q) || true
2025-03-14T04:08:50.4289003Z # Prune all of the docker images
2025-03-14T04:08:50.4289465Z docker system prune -af
2025-03-14T04:08:50.4294962Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:08:50.4295544Z env:
2025-03-14T04:08:50.4295929Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:50.4296465Z ##[endgroup]
2025-03-14T04:08:50.4519407Z "docker stop" requires at least 1 argument.
2025-03-14T04:08:50.4519910Z See 'docker stop --help'.
2025-03-14T04:08:50.4520127Z 
2025-03-14T04:08:50.4520386Z Usage:  docker stop [OPTIONS] CONTAINER [CONTAINER...]
2025-03-14T04:08:50.4520687Z 
2025-03-14T04:08:50.4520921Z Stop one or more running containers
2025-03-14T04:08:50.4644053Z Total reclaimed space: 0B
2025-03-14T04:08:50.4679615Z ##[group]Run set +e
2025-03-14T04:08:50.4679928Z set +e
2025-03-14T04:08:50.4680188Z set -x
2025-03-14T04:08:50.4680436Z 
2025-03-14T04:08:50.4680703Z PT_DOMAIN=download.pytorch.org
2025-03-14T04:08:50.4681319Z # TODO: Flaky access to download.pytorch.org https://github.com/pytorch/pytorch/issues/100400,
2025-03-14T04:08:50.4682270Z # cleaning this up once the issue is fixed. There are more than one resolved IP here, the last
2025-03-14T04:08:50.4682839Z # one is returned at random
2025-03-14T04:08:50.4683265Z RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" | tail -n1)
2025-03-14T04:08:50.4683672Z 
2025-03-14T04:08:50.4683922Z if [ -z "${RESOLVED_IP}" ]; then
2025-03-14T04:08:50.4684380Z   echo "Couldn't resolve ${PT_DOMAIN}, retrying with Google DNS..."
2025-03-14T04:08:50.4684936Z   RESOLVED_IP=$(dig -4 +short "${PT_DOMAIN}" @8.8.8.8 | tail -n1)
2025-03-14T04:08:50.4685360Z 
2025-03-14T04:08:50.4685619Z   if [ -z "${RESOLVED_IP}" ]; then
2025-03-14T04:08:50.4686031Z     echo "Couldn't resolve ${PT_DOMAIN}, exiting..."
2025-03-14T04:08:50.4686419Z     exit 1
2025-03-14T04:08:50.4686669Z   fi
2025-03-14T04:08:50.4686904Z fi
2025-03-14T04:08:50.4687131Z 
2025-03-14T04:08:50.4687411Z if grep -r "${PT_DOMAIN}" /etc/hosts; then
2025-03-14T04:08:50.4687801Z   # Clean up any old records first
2025-03-14T04:08:50.4688186Z   sudo sed -i "/${PT_DOMAIN}/d" /etc/hosts
2025-03-14T04:08:50.4688540Z fi
2025-03-14T04:08:50.4688774Z 
2025-03-14T04:08:50.4689106Z echo "${RESOLVED_IP} ${PT_DOMAIN}" | sudo tee -a /etc/hosts
2025-03-14T04:08:50.4689530Z cat /etc/hosts
2025-03-14T04:08:50.4694956Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:08:50.4695359Z env:
2025-03-14T04:08:50.4695601Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:50.4695887Z ##[endgroup]
2025-03-14T04:08:50.4717645Z + PT_DOMAIN=download.pytorch.org
2025-03-14T04:08:50.4722873Z ++ dig -4 +short download.pytorch.org
2025-03-14T04:08:50.4723297Z ++ tail -n1
2025-03-14T04:08:50.5333344Z + RESOLVED_IP=18.160.10.22
2025-03-14T04:08:50.5333897Z + '[' -z 18.160.10.22 ']'
2025-03-14T04:08:50.5334425Z + grep -r download.pytorch.org /etc/hosts
2025-03-14T04:08:50.5343898Z 18.160.10.22 download.pytorch.org
2025-03-14T04:08:50.5345358Z + sudo sed -i /download.pytorch.org/d /etc/hosts
2025-03-14T04:08:50.8594102Z + echo '18.160.10.22 download.pytorch.org'
2025-03-14T04:08:50.8594560Z + sudo tee -a /etc/hosts
2025-03-14T04:08:51.0083155Z 18.160.10.22 download.pytorch.org
2025-03-14T04:08:51.0097644Z + cat /etc/hosts
2025-03-14T04:08:51.0105990Z 127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
2025-03-14T04:08:51.0111652Z ::1         localhost6 localhost6.localdomain6
2025-03-14T04:08:51.0112116Z 18.160.10.22 download.pytorch.org
2025-03-14T04:08:51.0272333Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main
2025-03-14T04:08:51.0272842Z with:
2025-03-14T04:08:51.0273536Z   docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:51.0274333Z   docker-build-dir: .ci/docker
2025-03-14T04:08:51.0274640Z   working-directory: .
2025-03-14T04:08:51.0275015Z   docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:08:51.0275443Z   force-push: false
2025-03-14T04:08:51.0275698Z env:
2025-03-14T04:08:51.0276059Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:51.0276328Z ##[endgroup]
2025-03-14T04:08:51.0302707Z ##[group]Run set -ex
2025-03-14T04:08:51.0303039Z set -ex
2025-03-14T04:08:51.0303273Z 
2025-03-14T04:08:51.0303695Z # If the docker build directory or the build script doesn't exist, the action will
2025-03-14T04:08:51.0304452Z # gracefully return the docker image name as it is.  Pulling docker image in Linux
2025-03-14T04:08:51.0305055Z # job could then download the pre-built image as usual
2025-03-14T04:08:51.0305658Z if [[ ! -d "${DOCKER_BUILD_DIR}" ]] || [[ ! -f "${DOCKER_BUILD_DIR}/build.sh" ]]; then
2025-03-14T04:08:51.0306168Z   echo "skip=true" >> "${GITHUB_OUTPUT}"
2025-03-14T04:08:51.0306652Z   echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}"
2025-03-14T04:08:51.0307091Z 
2025-03-14T04:08:51.0307472Z   echo "There is no Docker build script in ${REPO_NAME} repo, skipping..."
2025-03-14T04:08:51.0307949Z   exit 0
2025-03-14T04:08:51.0308193Z else
2025-03-14T04:08:51.0308466Z   echo "skip=false" >> "${GITHUB_OUTPUT}"
2025-03-14T04:08:51.0308813Z fi
2025-03-14T04:08:51.0309043Z 
2025-03-14T04:08:51.0309406Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then
2025-03-14T04:08:51.0310044Z   # The docker image name already includes the ECR prefix and tag, so we can just
2025-03-14T04:08:51.0310610Z   # use it as it is, but first let's extract the tag
2025-03-14T04:08:51.0311122Z   DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}')
2025-03-14T04:08:51.0311662Z   echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}"
2025-03-14T04:08:51.0312182Z   echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}"
2025-03-14T04:08:51.0312619Z else
2025-03-14T04:08:51.0313052Z   DOCKER_TAG=$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}")
2025-03-14T04:08:51.0313554Z   echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}"
2025-03-14T04:08:51.0314232Z   echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}"
2025-03-14T04:08:51.0314823Z fi
2025-03-14T04:08:51.0321871Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:08:51.0322282Z env:
2025-03-14T04:08:51.0322520Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:51.0322816Z   REPO_NAME: pytorch
2025-03-14T04:08:51.0323532Z   DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:51.0324315Z   DOCKER_BUILD_DIR: .ci/docker
2025-03-14T04:08:51.0324712Z   DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:08:51.0325124Z ##[endgroup]
2025-03-14T04:08:51.0347991Z + [[ ! -d .ci/docker ]]
2025-03-14T04:08:51.0348291Z + [[ ! -f .ci/docker/build.sh ]]
2025-03-14T04:08:51.0348608Z + echo skip=false
2025-03-14T04:08:51.0349861Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849 == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]]
2025-03-14T04:08:51.0354810Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:51.0355638Z ++ awk -F '[:,]' '{print $2}'
2025-03-14T04:08:51.0374158Z + DOCKER_TAG=aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:51.0374645Z + echo docker-tag=aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:51.0375500Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:51.0405946Z ##[group]Run set +e
2025-03-14T04:08:51.0406276Z set +e
2025-03-14T04:08:51.0406639Z set -x
2025-03-14T04:08:51.0406884Z 
2025-03-14T04:08:51.0407115Z login() {
2025-03-14T04:08:51.0407619Z   aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1"
2025-03-14T04:08:51.0408168Z }
2025-03-14T04:08:51.0408399Z 
2025-03-14T04:08:51.0408628Z retry () {
2025-03-14T04:08:51.0408930Z   $*  || (sleep 1 && $*) || (sleep 2 && $*)
2025-03-14T04:08:51.0409269Z }
2025-03-14T04:08:51.0409499Z 
2025-03-14T04:08:51.0409741Z retry login "${DOCKER_REGISTRY}"
2025-03-14T04:08:51.0410068Z 
2025-03-14T04:08:51.0410306Z START_TIME=$(date +%s)
2025-03-14T04:08:51.0410621Z # Wait up to 120 minutes
2025-03-14T04:08:51.0411008Z while [[ $(( $(date +%s) - 7200 )) -lt $START_TIME ]]; do
2025-03-14T04:08:51.0411529Z   # Check if image already exists, if it does then skip building it
2025-03-14T04:08:51.0412053Z   if docker manifest inspect "${DOCKER_IMAGE}"; then
2025-03-14T04:08:51.0412441Z     exit 0
2025-03-14T04:08:51.0412693Z   fi
2025-03-14T04:08:51.0412925Z 
2025-03-14T04:08:51.0413332Z   # NB: This flag is used by Docker build workflow to push the image to ECR, so we can
2025-03-14T04:08:51.0414030Z   # use this to differentiate between the Docker build and regular build jobs. For the
2025-03-14T04:08:51.0414725Z   # latter, it will wait for the Docker images to become available before continuing
2025-03-14T04:08:51.0415273Z   if [ "${DOCKER_PUSH:-false}" == "true" ]; then
2025-03-14T04:08:51.0415698Z     # It's a Docker build job, let's build the image
2025-03-14T04:08:51.0416069Z     break
2025-03-14T04:08:51.0416318Z   else
2025-03-14T04:08:51.0416678Z     # It's a regular build job, wait for the image to become available
2025-03-14T04:08:51.0417119Z     sleep 300
2025-03-14T04:08:51.0417384Z   fi
2025-03-14T04:08:51.0417616Z done
2025-03-14T04:08:51.0417845Z 
2025-03-14T04:08:51.0418216Z # NB: This part requires a full checkout. Otherwise, the merge base will
2025-03-14T04:08:51.0418818Z # be empty.  The default action would be to continue rebuild the image
2025-03-14T04:08:51.0419352Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then
2025-03-14T04:08:51.0419834Z   # if we're on the base branch then use the parent commit
2025-03-14T04:08:51.0420264Z   MERGE_BASE=$(git rev-parse HEAD~)
2025-03-14T04:08:51.0420605Z else
2025-03-14T04:08:51.0420954Z   # otherwise we're on a PR, so use the most recent base commit
2025-03-14T04:08:51.0421461Z   MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION")
2025-03-14T04:08:51.0421850Z fi
2025-03-14T04:08:51.0422079Z 
2025-03-14T04:08:51.0422335Z if [[ -z "${MERGE_BASE}" ]]; then
2025-03-14T04:08:51.0422722Z   echo "rebuild=true" >> "${GITHUB_OUTPUT}"
2025-03-14T04:08:51.0423075Z 
2025-03-14T04:08:51.0423567Z   echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..."
2025-03-14T04:08:51.0424148Z   exit 0
2025-03-14T04:08:51.0424389Z fi
2025-03-14T04:08:51.0424619Z 
2025-03-14T04:08:51.0425066Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then
2025-03-14T04:08:51.0425888Z   echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit"
2025-03-14T04:08:51.0426511Z   exit 1
2025-03-14T04:08:51.0426761Z fi
2025-03-14T04:08:51.0426993Z 
2025-03-14T04:08:51.0427385Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}")
2025-03-14T04:08:51.0428080Z # If no image exists but the hash is the same as the previous hash then we should error out here
2025-03-14T04:08:51.0428779Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then
2025-03-14T04:08:51.0429497Z   echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch"
2025-03-14T04:08:51.0430307Z   echo "         Will re-build docker image to store in local cache, TTS may be longer"
2025-03-14T04:08:51.0430799Z fi
2025-03-14T04:08:51.0431027Z 
2025-03-14T04:08:51.0431296Z echo "rebuild=true" >> "${GITHUB_OUTPUT}"
2025-03-14T04:08:51.0436704Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:08:51.0437107Z env:
2025-03-14T04:08:51.0437343Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:51.0437632Z   DOCKER_BUILD_DIR: .ci/docker
2025-03-14T04:08:51.0438033Z   BASE_REVISION: aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:08:51.0438850Z   DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:51.0439661Z   DOCKER_TAG: aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:51.0440126Z   DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:08:51.0440542Z   DOCKER_PUSH: 
2025-03-14T04:08:51.0440788Z ##[endgroup]
2025-03-14T04:08:51.0462246Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:08:51.0462728Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:08:51.0465098Z + aws ecr get-login-password --region us-east-1
2025-03-14T04:08:51.0465932Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:08:51.5696942Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json.
2025-03-14T04:08:51.5698039Z Configure a credential helper to remove this warning. See
2025-03-14T04:08:51.5699321Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store
2025-03-14T04:08:51.5699865Z 
2025-03-14T04:08:51.5699980Z Login Succeeded
2025-03-14T04:08:51.5714471Z ++ date +%s
2025-03-14T04:08:51.5722901Z + START_TIME=1741925331
2025-03-14T04:08:51.5726500Z ++ date +%s
2025-03-14T04:08:51.5734757Z + [[ 1741918131 -lt 1741925331 ]]
2025-03-14T04:08:51.5735627Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:51.8794239Z {
2025-03-14T04:08:51.8794649Z 	"schemaVersion": 2,
2025-03-14T04:08:51.8795333Z 	"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
2025-03-14T04:08:51.8796058Z 	"config": {
2025-03-14T04:08:51.8796592Z 		"mediaType": "application/vnd.docker.container.image.v1+json",
2025-03-14T04:08:51.8797117Z 		"size": 41958,
2025-03-14T04:08:51.8797564Z 		"digest": "sha256:106fb2ab21a44c9d34c6968e6196c73d5cf7eaa50499fe5f98d7dd73b69b1298"
2025-03-14T04:08:51.8798066Z 	},
2025-03-14T04:08:51.8798437Z 	"layers": [
2025-03-14T04:08:51.8798676Z 		{
2025-03-14T04:08:51.8799171Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8799624Z 			"size": 30440118,
2025-03-14T04:08:51.8800074Z 			"digest": "sha256:8f84a9f2102e97a4a6bf673b150fc9894df5acc9618ad3484c6c36f768c1caa0"
2025-03-14T04:08:51.8800575Z 		},
2025-03-14T04:08:51.8800789Z 		{
2025-03-14T04:08:51.8801141Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8801828Z 			"size": 1894,
2025-03-14T04:08:51.8802271Z 			"digest": "sha256:d6ee0b7aac7a248d2611054390e644c9e333ad45a00e8b4d9c0d468ca843d34d"
2025-03-14T04:08:51.8802772Z 		},
2025-03-14T04:08:51.8802988Z 		{
2025-03-14T04:08:51.8803340Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8803782Z 			"size": 319404616,
2025-03-14T04:08:51.8804236Z 			"digest": "sha256:747c5ff5a7b725a3577fd3b0f0efa685f4c5c0f422567fd86bde651e8921254a"
2025-03-14T04:08:51.8805003Z 		},
2025-03-14T04:08:51.8805331Z 		{
2025-03-14T04:08:51.8805793Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8806475Z 			"size": 864,
2025-03-14T04:08:51.8807019Z 			"digest": "sha256:87b8b8bf9a960751234a2383c4090d4fd2aaa748e9eeb13451fae72a83ab75bb"
2025-03-14T04:08:51.8807588Z 		},
2025-03-14T04:08:51.8807946Z 		{
2025-03-14T04:08:51.8808351Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8809467Z 			"size": 104,
2025-03-14T04:08:51.8810070Z 			"digest": "sha256:3cebdd3ecb5155dce0596c1ed52edef16b240b9ba313d511a009320bffe358d1"
2025-03-14T04:08:51.8810849Z 		},
2025-03-14T04:08:51.8811174Z 		{
2025-03-14T04:08:51.8811644Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8812539Z 			"size": 704,
2025-03-14T04:08:51.8813397Z 			"digest": "sha256:9875190f952116d672ea745fea3032465e9448a0e5e27b1df20d3d676a166cda"
2025-03-14T04:08:51.8814142Z 		},
2025-03-14T04:08:51.8814465Z 		{
2025-03-14T04:08:51.8814920Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8815540Z 			"size": 1217,
2025-03-14T04:08:51.8816081Z 			"digest": "sha256:68592b961c78d24e578c3b9f47b7c0f90b3810da59c75c552e5fa1504b27fdd0"
2025-03-14T04:08:51.8816645Z 		},
2025-03-14T04:08:51.8816992Z 		{
2025-03-14T04:08:51.8817447Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8817990Z 			"size": 484,
2025-03-14T04:08:51.8818549Z 			"digest": "sha256:cada203b76efa095b7b5298e41177ebaf9c74ed67d4e345e05c5dfe9d17fb68f"
2025-03-14T04:08:51.8819147Z 		},
2025-03-14T04:08:51.8819474Z 		{
2025-03-14T04:08:51.8819922Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8820427Z 			"size": 110341890,
2025-03-14T04:08:51.8821040Z 			"digest": "sha256:ee39a89b29f4499500181864cbef20e56cb2a346c4508507817654b865a8fac8"
2025-03-14T04:08:51.8821576Z 		},
2025-03-14T04:08:51.8821864Z 		{
2025-03-14T04:08:51.8822358Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8822895Z 			"size": 4163,
2025-03-14T04:08:51.8823430Z 			"digest": "sha256:636df2e6d85485c19388304f97b49a69e21341af7856e246fc0b8c4111fd901d"
2025-03-14T04:08:51.8824037Z 		},
2025-03-14T04:08:51.8824338Z 		{
2025-03-14T04:08:51.8824814Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8825340Z 			"size": 1860,
2025-03-14T04:08:51.8825961Z 			"digest": "sha256:1d4ea74fe29a0aceb5d0a9b6b09cd16a65f3fbccde220253f5baa7a35b07315b"
2025-03-14T04:08:51.8826656Z 		},
2025-03-14T04:08:51.8826957Z 		{
2025-03-14T04:08:51.8827418Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8827965Z 			"size": 700,
2025-03-14T04:08:51.8828486Z 			"digest": "sha256:5c57d945e77401a617a527ef203cf58a433d446c540a73ef7be6c55c495d2d27"
2025-03-14T04:08:51.8829100Z 		},
2025-03-14T04:08:51.8829401Z 		{
2025-03-14T04:08:51.8829827Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8830414Z 			"size": 477,
2025-03-14T04:08:51.8830934Z 			"digest": "sha256:cc3d1769377187b6db7c70b8444a87ddb477bf4fb22dd1233b578b302be6e3ac"
2025-03-14T04:08:51.8831572Z 		},
2025-03-14T04:08:51.8831848Z 		{
2025-03-14T04:08:51.8832295Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8832827Z 			"size": 2831721438,
2025-03-14T04:08:51.8833370Z 			"digest": "sha256:b97e880ec263a76365a8498ddb3112e1c91b0b6265c64626a887a3d1d6624745"
2025-03-14T04:08:51.8834074Z 		},
2025-03-14T04:08:51.8834402Z 		{
2025-03-14T04:08:51.8834857Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8835365Z 			"size": 32,
2025-03-14T04:08:51.8835933Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:08:51.8836535Z 		},
2025-03-14T04:08:51.8836877Z 		{
2025-03-14T04:08:51.8837346Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8837886Z 			"size": 380,
2025-03-14T04:08:51.8838443Z 			"digest": "sha256:eceeac2a64b3ade86c0073f74b4444846360846100ba945d54da0d7a33cb2011"
2025-03-14T04:08:51.8839111Z 		},
2025-03-14T04:08:51.8839389Z 		{
2025-03-14T04:08:51.8839892Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8840423Z 			"size": 232984,
2025-03-14T04:08:51.8840973Z 			"digest": "sha256:e0ce9fa68ff6e8a1b8f54194466e96c694186db76ed79a32009e6c4682614924"
2025-03-14T04:08:51.8841535Z 		},
2025-03-14T04:08:51.8841883Z 		{
2025-03-14T04:08:51.8842344Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8842899Z 			"size": 231,
2025-03-14T04:08:51.8843424Z 			"digest": "sha256:af0b38d0861b24f1c7e37b3d9664265d69c857db276aa19b2480fa3f2100f2cd"
2025-03-14T04:08:51.8844045Z 		},
2025-03-14T04:08:51.8844348Z 		{
2025-03-14T04:08:51.8844779Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8845375Z 			"size": 3779998,
2025-03-14T04:08:51.8845902Z 			"digest": "sha256:83e5b1b4a717e144c97db8652ee1528500f5c9c11cb5293870b63ce027b1f00e"
2025-03-14T04:08:51.8846516Z 		},
2025-03-14T04:08:51.8864893Z 		{
2025-03-14T04:08:51.8865298Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8865909Z 			"size": 1946,
2025-03-14T04:08:51.8866354Z 			"digest": "sha256:d306a6a6b642437342bc86c6d0d55753684fed1dda7117b6f8fdd57173d17a15"
2025-03-14T04:08:51.8866859Z 		},
2025-03-14T04:08:51.8867075Z 		{
2025-03-14T04:08:51.8867445Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8867887Z 			"size": 105,
2025-03-14T04:08:51.8868326Z 			"digest": "sha256:8bf6bdf7bb7bb91b0207c6bd708e9199c045b1620fd906577de1506a1cf9ccb8"
2025-03-14T04:08:51.8868829Z 		},
2025-03-14T04:08:51.8869045Z 		{
2025-03-14T04:08:51.8869394Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8869839Z 			"size": 802,
2025-03-14T04:08:51.8870242Z 			"digest": "sha256:17779673764c26c4622773ab631e86e95cf576117ff76286806968a40cd991e3"
2025-03-14T04:08:51.8870724Z 		},
2025-03-14T04:08:51.8870947Z 		{
2025-03-14T04:08:51.8871295Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8871735Z 			"size": 32,
2025-03-14T04:08:51.8872169Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:08:51.8872672Z 		},
2025-03-14T04:08:51.8872885Z 		{
2025-03-14T04:08:51.8873239Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8873679Z 			"size": 104,
2025-03-14T04:08:51.8874115Z 			"digest": "sha256:2e8ffa6248b345caa55699e776b4a1972ec552a47e5f00a4dee806b77b36267b"
2025-03-14T04:08:51.8874611Z 		},
2025-03-14T04:08:51.8874823Z 		{
2025-03-14T04:08:51.8875170Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8875607Z 			"size": 504,
2025-03-14T04:08:51.8876036Z 			"digest": "sha256:95b9c737ea76db1de6d7ba727d0f7a10389be7d5618a549226c457d2e80046bf"
2025-03-14T04:08:51.8876523Z 		},
2025-03-14T04:08:51.8876734Z 		{
2025-03-14T04:08:51.8877086Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8877531Z 			"size": 101109951,
2025-03-14T04:08:51.8877989Z 			"digest": "sha256:ad432fda02cd00d37cc14bf9b5c94ea051bad685e8f588aed5bf5a0a375c1121"
2025-03-14T04:08:51.8878500Z 		},
2025-03-14T04:08:51.8878713Z 		{
2025-03-14T04:08:51.8879061Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8879626Z 			"size": 109,
2025-03-14T04:08:51.8880061Z 			"digest": "sha256:5d3bc42935c325d35554c2c26a2036aac9d361eeb250eb44ccaa57b59d301740"
2025-03-14T04:08:51.8880556Z 		},
2025-03-14T04:08:51.8880764Z 		{
2025-03-14T04:08:51.8881107Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8881546Z 			"size": 489,
2025-03-14T04:08:51.8881973Z 			"digest": "sha256:d8c07316b7ca8031bb31783977108a6d795faef073e177c59dc9558aeb253e4b"
2025-03-14T04:08:51.8882466Z 		},
2025-03-14T04:08:51.8882661Z 		{
2025-03-14T04:08:51.8883007Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8883519Z 			"size": 340,
2025-03-14T04:08:51.8883958Z 			"digest": "sha256:6824395fbc6f8e95841ebab1d1aa68b43778e48b729830d2e46daaa9e2849c79"
2025-03-14T04:08:51.8884453Z 		},
2025-03-14T04:08:51.8884664Z 		{
2025-03-14T04:08:51.8885013Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8885451Z 			"size": 103,
2025-03-14T04:08:51.8885889Z 			"digest": "sha256:119cd85655c13ec6ee30ffa675f3245cb7d569606371dfa2731d61a15f72318d"
2025-03-14T04:08:51.8886381Z 		},
2025-03-14T04:08:51.8886594Z 		{
2025-03-14T04:08:51.8886942Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8887381Z 			"size": 1474,
2025-03-14T04:08:51.8887818Z 			"digest": "sha256:dbd96d29d52e23dc9eb20a5b76f2ca4290f3672fe6600073f344e7ef19502335"
2025-03-14T04:08:51.8888314Z 		},
2025-03-14T04:08:51.8888523Z 		{
2025-03-14T04:08:51.8888859Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8889308Z 			"size": 446377705,
2025-03-14T04:08:51.8889764Z 			"digest": "sha256:cc77928dad7d6dafde2ac3215a0b33d862df084c1c4bd021b57cbd153c6f7811"
2025-03-14T04:08:51.8890273Z 		},
2025-03-14T04:08:51.8890483Z 		{
2025-03-14T04:08:51.8890830Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8891271Z 			"size": 163,
2025-03-14T04:08:51.8891700Z 			"digest": "sha256:c0d38f09f6f4fbae8018498813f88165537e9df831885f3c489fe5bed64ed666"
2025-03-14T04:08:51.8892196Z 		},
2025-03-14T04:08:51.8892407Z 		{
2025-03-14T04:08:51.8892753Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8893197Z 			"size": 423,
2025-03-14T04:08:51.8893625Z 			"digest": "sha256:08e926bccef670c4e8197d3cb2f1112a60816ca70da062e2b62069077bd65693"
2025-03-14T04:08:51.8894128Z 		},
2025-03-14T04:08:51.8894346Z 		{
2025-03-14T04:08:51.8894700Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8895149Z 			"size": 32,
2025-03-14T04:08:51.8895575Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:08:51.8896083Z 		},
2025-03-14T04:08:51.8896301Z 		{
2025-03-14T04:08:51.8896654Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8897098Z 			"size": 111,
2025-03-14T04:08:51.8897536Z 			"digest": "sha256:f937a51b99861f82bc10d3dc736613fc0fcb2086d2fde0650fec6d5c7174f06d"
2025-03-14T04:08:51.8898031Z 		},
2025-03-14T04:08:51.8898239Z 		{
2025-03-14T04:08:51.8898788Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8899229Z 			"size": 473,
2025-03-14T04:08:51.8899659Z 			"digest": "sha256:b81a345fe0318a77ff62d0781599022cad659fe3f2a82dfab546828c605c5224"
2025-03-14T04:08:51.8900150Z 		},
2025-03-14T04:08:51.8900359Z 		{
2025-03-14T04:08:51.8900709Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8901146Z 			"size": 32,
2025-03-14T04:08:51.8901577Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:08:51.8902087Z 		},
2025-03-14T04:08:51.8902285Z 		{
2025-03-14T04:08:51.8902633Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8903072Z 			"size": 112,
2025-03-14T04:08:51.8903505Z 			"digest": "sha256:5ee2483891c55c734a220c50debb57ed7cd708b57b5fb155584ac47ff4834647"
2025-03-14T04:08:51.8904004Z 		},
2025-03-14T04:08:51.8904367Z 		{
2025-03-14T04:08:51.8904721Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8905164Z 			"size": 565,
2025-03-14T04:08:51.8905665Z 			"digest": "sha256:ca4fb7ee460a82a8bca7963d650826afb2244f674bffb61b3f346a5b04479e60"
2025-03-14T04:08:51.8906177Z 		},
2025-03-14T04:08:51.8906396Z 		{
2025-03-14T04:08:51.8906749Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8907196Z 			"size": 43164125,
2025-03-14T04:08:51.8907651Z 			"digest": "sha256:6d7937c4d3ea40a5e1fefef1ee63c190a0ac0841f9641ccd977053d6596ede67"
2025-03-14T04:08:51.8908241Z 		},
2025-03-14T04:08:51.8908459Z 		{
2025-03-14T04:08:51.8908798Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8909242Z 			"size": 106,
2025-03-14T04:08:51.8909676Z 			"digest": "sha256:3ddc0e8251be4c6c1b7a232108ec8694895f2c160f4414745a68e6b2c17a60db"
2025-03-14T04:08:51.8910173Z 		},
2025-03-14T04:08:51.8910384Z 		{
2025-03-14T04:08:51.8910737Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8911181Z 			"size": 346,
2025-03-14T04:08:51.8911627Z 			"digest": "sha256:ee39eb7c5421cb7fda7e719bb4de6bf12339dfee3acba9792eff1cbbb0af9b79"
2025-03-14T04:08:51.8912144Z 		},
2025-03-14T04:08:51.8912351Z 		{
2025-03-14T04:08:51.8912698Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8913139Z 			"size": 32,
2025-03-14T04:08:51.8913575Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:08:51.8914081Z 		},
2025-03-14T04:08:51.8914288Z 		{
2025-03-14T04:08:51.8914632Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8915056Z 			"size": 106,
2025-03-14T04:08:51.8915475Z 			"digest": "sha256:14258b78822885f7504f3919763f33f2c71a149224f4b8aee08099d64a18fcc1"
2025-03-14T04:08:51.8915958Z 		},
2025-03-14T04:08:51.8916164Z 		{
2025-03-14T04:08:51.8916514Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8916954Z 			"size": 425,
2025-03-14T04:08:51.8917396Z 			"digest": "sha256:60b8a611752bb34ef1ee770e1e6bda5ce6cdb55f7e6efcda6bd8b8ab02c4f7f0"
2025-03-14T04:08:51.8917906Z 		},
2025-03-14T04:08:51.8918118Z 		{
2025-03-14T04:08:51.8918464Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8918907Z 			"size": 19308694,
2025-03-14T04:08:51.8919355Z 			"digest": "sha256:9b3a3723fc19e194359cd3e9438f9fa0d6fb17c1a2cdca8386e5632b88b6ddb3"
2025-03-14T04:08:51.8919854Z 		},
2025-03-14T04:08:51.8920065Z 		{
2025-03-14T04:08:51.8920415Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8920854Z 			"size": 108,
2025-03-14T04:08:51.8921277Z 			"digest": "sha256:6ac5fca3a6751643e255f27e8f74e6a3a33f37a1aed5839c3c47c897c07ea165"
2025-03-14T04:08:51.8921778Z 		},
2025-03-14T04:08:51.8921988Z 		{
2025-03-14T04:08:51.8922336Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8922776Z 			"size": 644,
2025-03-14T04:08:51.8923204Z 			"digest": "sha256:aa265769b086942607060bb2d82fdf084ee875d4f2c15d662e6dd1501e2fe511"
2025-03-14T04:08:51.8923694Z 		},
2025-03-14T04:08:51.8923907Z 		{
2025-03-14T04:08:51.8924256Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8924697Z 			"size": 700,
2025-03-14T04:08:51.8925167Z 			"digest": "sha256:5c57d945e77401a617a527ef203cf58a433d446c540a73ef7be6c55c495d2d27"
2025-03-14T04:08:51.8925662Z 		},
2025-03-14T04:08:51.8925868Z 		{
2025-03-14T04:08:51.8926215Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8926662Z 			"size": 141,
2025-03-14T04:08:51.8927101Z 			"digest": "sha256:9b5bf41436cc5e36cde4116df4d07ba5f54606c69a2fdc085cd1b84aa9586235"
2025-03-14T04:08:51.8927607Z 		},
2025-03-14T04:08:51.8927815Z 		{
2025-03-14T04:08:51.8928165Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8928736Z 			"size": 136,
2025-03-14T04:08:51.8929178Z 			"digest": "sha256:e0440b1e75c34de256bfba55087340b0c8203b47d5b98e5502ce0a22c2581d41"
2025-03-14T04:08:51.8929679Z 		},
2025-03-14T04:08:51.8929896Z 		{
2025-03-14T04:08:51.8930247Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8930689Z 			"size": 32,
2025-03-14T04:08:51.8931126Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:08:51.8931631Z 		},
2025-03-14T04:08:51.8931848Z 		{
2025-03-14T04:08:51.8932189Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8932714Z 			"size": 193,
2025-03-14T04:08:51.8933153Z 			"digest": "sha256:7b0f902826bad190df088883d0df612721b5a5dbc6e61d62be42ef087e8158bb"
2025-03-14T04:08:51.8933654Z 		},
2025-03-14T04:08:51.8933869Z 		{
2025-03-14T04:08:51.8934221Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8934665Z 			"size": 1401,
2025-03-14T04:08:51.8935106Z 			"digest": "sha256:082b0db9631d225d87ddb930ea166afd3c0219c8ba1ea8e306670ed3809f1f4e"
2025-03-14T04:08:51.8935604Z 		},
2025-03-14T04:08:51.8935815Z 		{
2025-03-14T04:08:51.8936162Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8936600Z 			"size": 700,
2025-03-14T04:08:51.8937027Z 			"digest": "sha256:5c57d945e77401a617a527ef203cf58a433d446c540a73ef7be6c55c495d2d27"
2025-03-14T04:08:51.8937515Z 		},
2025-03-14T04:08:51.8937722Z 		{
2025-03-14T04:08:51.8938069Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8938493Z 			"size": 140,
2025-03-14T04:08:51.8938922Z 			"digest": "sha256:03e517f304e985802bf68fa90be336961be9da4b914f66cae73037314904c81f"
2025-03-14T04:08:51.8939413Z 		},
2025-03-14T04:08:51.8939623Z 		{
2025-03-14T04:08:51.8939975Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8940418Z 			"size": 5175245601,
2025-03-14T04:08:51.8940881Z 			"digest": "sha256:cac24658ea3005985cead5d2585aec9414beabf42d51f6239a4b6db86c1ecbb0"
2025-03-14T04:08:51.8941387Z 		},
2025-03-14T04:08:51.8941599Z 		{
2025-03-14T04:08:51.8941948Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8942385Z 			"size": 156,
2025-03-14T04:08:51.8942818Z 			"digest": "sha256:f587b2ee91c1d6d7c4205a82b962b10f57ed4af1018e5cb530fff6942dc98841"
2025-03-14T04:08:51.8943318Z 		},
2025-03-14T04:08:51.8943531Z 		{
2025-03-14T04:08:51.8943878Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8944316Z 			"size": 1401,
2025-03-14T04:08:51.8944736Z 			"digest": "sha256:082b0db9631d225d87ddb930ea166afd3c0219c8ba1ea8e306670ed3809f1f4e"
2025-03-14T04:08:51.8945240Z 		},
2025-03-14T04:08:51.8945525Z 		{
2025-03-14T04:08:51.8945880Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8946326Z 			"size": 700,
2025-03-14T04:08:51.8946756Z 			"digest": "sha256:5c57d945e77401a617a527ef203cf58a433d446c540a73ef7be6c55c495d2d27"
2025-03-14T04:08:51.8947258Z 		},
2025-03-14T04:08:51.8947467Z 		{
2025-03-14T04:08:51.8947814Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8948255Z 			"size": 140,
2025-03-14T04:08:51.8948670Z 			"digest": "sha256:0715583772232529847ce8b0ccc1500e0283c70db7953939b2236e76864732b2"
2025-03-14T04:08:51.8949151Z 		},
2025-03-14T04:08:51.8949361Z 		{
2025-03-14T04:08:51.8949713Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8950155Z 			"size": 32,
2025-03-14T04:08:51.8950587Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:08:51.8951089Z 		},
2025-03-14T04:08:51.8951288Z 		{
2025-03-14T04:08:51.8951637Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8952080Z 			"size": 161,
2025-03-14T04:08:51.8952500Z 			"digest": "sha256:141eb03e8120142060761597d1a4a5940a3969eefd78f2c7aa01497ab8c2b7ae"
2025-03-14T04:08:51.8952989Z 		},
2025-03-14T04:08:51.8953286Z 		{
2025-03-14T04:08:51.8953638Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8954078Z 			"size": 765,
2025-03-14T04:08:51.8954500Z 			"digest": "sha256:9008a6e05a8e8214b0d67aa98a0c421ca86681003a1be4ca9b59963f295e8f44"
2025-03-14T04:08:51.8954990Z 		},
2025-03-14T04:08:51.8955199Z 		{
2025-03-14T04:08:51.8955546Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8955987Z 			"size": 700,
2025-03-14T04:08:51.8956411Z 			"digest": "sha256:5c57d945e77401a617a527ef203cf58a433d446c540a73ef7be6c55c495d2d27"
2025-03-14T04:08:51.8956967Z 		},
2025-03-14T04:08:51.8957180Z 		{
2025-03-14T04:08:51.8957519Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8957962Z 			"size": 138,
2025-03-14T04:08:51.8958396Z 			"digest": "sha256:59d79792a1c902266b1cd478484e0c82f06c804abc826f5fa7f5b455fb3e2504"
2025-03-14T04:08:51.8958890Z 		},
2025-03-14T04:08:51.8959101Z 		{
2025-03-14T04:08:51.8959447Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8959886Z 			"size": 32,
2025-03-14T04:08:51.8960320Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:08:51.8960819Z 		},
2025-03-14T04:08:51.8961030Z 		{
2025-03-14T04:08:51.8961378Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8961819Z 			"size": 160,
2025-03-14T04:08:51.8962255Z 			"digest": "sha256:fbe892ee1d5dfa13631cd4b16c692e3ba05e277ecc6d9cf161085978e1e586d6"
2025-03-14T04:08:51.8962764Z 		},
2025-03-14T04:08:51.8962980Z 		{
2025-03-14T04:08:51.8963334Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8963765Z 			"size": 908,
2025-03-14T04:08:51.8964199Z 			"digest": "sha256:2117add440a2075a3f768b7e231b44d932b3b247c16ca0c1b98657edff974d62"
2025-03-14T04:08:51.8964707Z 		},
2025-03-14T04:08:51.8964924Z 		{
2025-03-14T04:08:51.8965283Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8965728Z 			"size": 700,
2025-03-14T04:08:51.8966159Z 			"digest": "sha256:5c57d945e77401a617a527ef203cf58a433d446c540a73ef7be6c55c495d2d27"
2025-03-14T04:08:51.8966741Z 		},
2025-03-14T04:08:51.8966956Z 		{
2025-03-14T04:08:51.8967311Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8967756Z 			"size": 135,
2025-03-14T04:08:51.8968187Z 			"digest": "sha256:b59fff3e88c48f58960674750445315384bcd773d317474607f1ecdc4d5d9878"
2025-03-14T04:08:51.8968679Z 		},
2025-03-14T04:08:51.8968890Z 		{
2025-03-14T04:08:51.8969245Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8969685Z 			"size": 32,
2025-03-14T04:08:51.8970105Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:08:51.8970604Z 		},
2025-03-14T04:08:51.8970814Z 		{
2025-03-14T04:08:51.8971160Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8971603Z 			"size": 158,
2025-03-14T04:08:51.8972034Z 			"digest": "sha256:6e1d1e5ed2fc5344ac3048fac4773e7e588d1be2f3b22680651add9875e81cf0"
2025-03-14T04:08:51.8972532Z 		},
2025-03-14T04:08:51.8972744Z 		{
2025-03-14T04:08:51.8973089Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8973528Z 			"size": 1484,
2025-03-14T04:08:51.8973970Z 			"digest": "sha256:e66c39c0da12a5b14c4c74cfc598e67e378eec4610d7262d04f20a875b560fcd"
2025-03-14T04:08:51.8974472Z 		},
2025-03-14T04:08:51.8974690Z 		{
2025-03-14T04:08:51.8975039Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8975489Z 			"size": 32,
2025-03-14T04:08:51.8975923Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:08:51.8976424Z 		},
2025-03-14T04:08:51.8976624Z 		{
2025-03-14T04:08:51.8976970Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8977408Z 			"size": 137,
2025-03-14T04:08:51.8977920Z 			"digest": "sha256:43dd24f0c7b5c89ba70fa8b555738447704919322a8fe141e8a043f03922deba"
2025-03-14T04:08:51.8978420Z 		},
2025-03-14T04:08:51.8978637Z 		{
2025-03-14T04:08:51.8978990Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8979439Z 			"size": 381,
2025-03-14T04:08:51.8979891Z 			"digest": "sha256:f0ff235daf3eb5ff6245c8c4fadb753d6d6fdc8a2a5580f8ad92be1d615b35da"
2025-03-14T04:08:51.8980410Z 		},
2025-03-14T04:08:51.8980624Z 		{
2025-03-14T04:08:51.8980970Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8981466Z 			"size": 32,
2025-03-14T04:08:51.8981902Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:08:51.8982406Z 		},
2025-03-14T04:08:51.8982618Z 		{
2025-03-14T04:08:51.8982953Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8983393Z 			"size": 104,
2025-03-14T04:08:51.8983825Z 			"digest": "sha256:14c7e3cf113cfe5f0430ec98bdaa99c2d0e46017df9450ebe08c4fa7372b83ef"
2025-03-14T04:08:51.8984327Z 		},
2025-03-14T04:08:51.8984536Z 		{
2025-03-14T04:08:51.8984882Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8985307Z 			"size": 1899,
2025-03-14T04:08:51.8985809Z 			"digest": "sha256:fafd68c92609757bd2b730aebe76c52e1b20cb911cf85d4178231892ebd782e3"
2025-03-14T04:08:51.8986291Z 		},
2025-03-14T04:08:51.8986483Z 		{
2025-03-14T04:08:51.8986816Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8987238Z 			"size": 196997029,
2025-03-14T04:08:51.8987684Z 			"digest": "sha256:5ec7e9d96465a200843a6cb21ae6169ec59ccdd57c8cbb911eb1a482fab133f7"
2025-03-14T04:08:51.8988169Z 		},
2025-03-14T04:08:51.8988370Z 		{
2025-03-14T04:08:51.8988706Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8989126Z 			"size": 106,
2025-03-14T04:08:51.8989546Z 			"digest": "sha256:023489ff1b0b198a2d6e35d9e2fe902abfe45943d81c699b45f523a6829be3b5"
2025-03-14T04:08:51.8990028Z 		},
2025-03-14T04:08:51.8990226Z 		{
2025-03-14T04:08:51.8990562Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8990991Z 			"size": 165,
2025-03-14T04:08:51.8991415Z 			"digest": "sha256:5f15c6ed2fe960aad91dbbe9af42567cef1587162dc55719e3d4a39391be0b86"
2025-03-14T04:08:51.8991902Z 		},
2025-03-14T04:08:51.8992105Z 		{
2025-03-14T04:08:51.8992442Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8992869Z 			"size": 7943,
2025-03-14T04:08:51.8993280Z 			"digest": "sha256:339b86aa93616123716fc31174917cd56e98634ad69c9a81d2ed2b6ccbd708b8"
2025-03-14T04:08:51.8993764Z 		},
2025-03-14T04:08:51.8993958Z 		{
2025-03-14T04:08:51.8994293Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8994719Z 			"size": 8067,
2025-03-14T04:08:51.8995133Z 			"digest": "sha256:fd2517474e93a2e174d5354b25b45246dfd05b8b48d3b180151a2a1897862eb7"
2025-03-14T04:08:51.8995617Z 		},
2025-03-14T04:08:51.8995838Z 		{
2025-03-14T04:08:51.8996196Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8996641Z 			"size": 303,
2025-03-14T04:08:51.8997077Z 			"digest": "sha256:d28088afca533f1a2ab5293f31fde55d4923ade72dd81850d141883472d168c7"
2025-03-14T04:08:51.8997576Z 		},
2025-03-14T04:08:51.8997792Z 		{
2025-03-14T04:08:51.8998148Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.8998762Z 			"size": 32,
2025-03-14T04:08:51.8999203Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:08:51.8999715Z 		},
2025-03-14T04:08:51.8999931Z 		{
2025-03-14T04:08:51.9000286Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.9000732Z 			"size": 108,
2025-03-14T04:08:51.9001170Z 			"digest": "sha256:b57c57cc548b47ae388dd971e885c1f6b4ceccf129609e6360abb6035bafc1a6"
2025-03-14T04:08:51.9001674Z 		},
2025-03-14T04:08:51.9001877Z 		{
2025-03-14T04:08:51.9002366Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.9002811Z 			"size": 54145664,
2025-03-14T04:08:51.9003263Z 			"digest": "sha256:94dba42bfd0c59d7c9f6737103c1a996b96638ffc5eb3c81627d65a4584e7ab3"
2025-03-14T04:08:51.9003767Z 		},
2025-03-14T04:08:51.9003977Z 		{
2025-03-14T04:08:51.9004330Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-03-14T04:08:51.9004768Z 			"size": 32,
2025-03-14T04:08:51.9005198Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-03-14T04:08:51.9005701Z 		}
2025-03-14T04:08:51.9005985Z 	]
2025-03-14T04:08:51.9006195Z }
2025-03-14T04:08:51.9006422Z + exit 0
2025-03-14T04:08:51.9040798Z ##[group]Run set -eux
2025-03-14T04:08:51.9041105Z set -eux
2025-03-14T04:08:51.9042050Z aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token | jq --raw-output '.SecretString' | jq -r .docker_hub_readonly_token | docker login --username pytorchbot --password-stdin
2025-03-14T04:08:51.9049427Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:08:51.9049845Z env:
2025-03-14T04:08:51.9050090Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:51.9050375Z ##[endgroup]
2025-03-14T04:08:51.9078332Z + aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token
2025-03-14T04:08:51.9079126Z + jq --raw-output .SecretString
2025-03-14T04:08:51.9080924Z + jq -r .docker_hub_readonly_token
2025-03-14T04:08:51.9082254Z + docker login --username pytorchbot --password-stdin
2025-03-14T04:08:52.4879951Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json.
2025-03-14T04:08:52.4883370Z Configure a credential helper to remove this warning. See
2025-03-14T04:08:52.4884022Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store
2025-03-14T04:08:52.4884432Z 
2025-03-14T04:08:52.4884546Z Login Succeeded
2025-03-14T04:08:52.4967633Z ##[group]Run tag=${ECR_DOCKER_IMAGE##*/}
2025-03-14T04:08:52.4968045Z tag=${ECR_DOCKER_IMAGE##*/}
2025-03-14T04:08:52.4968462Z echo "docker pull ghcr.io/pytorch/ci-image:${tag/:/-}"
2025-03-14T04:08:52.4975060Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:08:52.4975472Z env:
2025-03-14T04:08:52.4975714Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:52.4976452Z   ECR_DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:52.4977214Z ##[endgroup]
2025-03-14T04:08:52.5002693Z docker pull ghcr.io/pytorch/ci-image:pytorch-linux-jammy-py3.9-gcc11-aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:52.5054471Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main
2025-03-14T04:08:52.5054938Z with:
2025-03-14T04:08:52.5055615Z   docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:52.5056477Z   docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:08:52.5056890Z env:
2025-03-14T04:08:52.5057127Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:52.5057410Z ##[endgroup]
2025-03-14T04:08:52.5079373Z ##[group]Run set -x
2025-03-14T04:08:52.5079670Z set -x
2025-03-14T04:08:52.5079922Z set +e
2025-03-14T04:08:52.5080170Z 
2025-03-14T04:08:52.5080404Z login() {
2025-03-14T04:08:52.5080913Z   aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1"
2025-03-14T04:08:52.5081464Z }
2025-03-14T04:08:52.5081706Z 
2025-03-14T04:08:52.5081974Z retry () {
2025-03-14T04:08:52.5082272Z   $*  || (sleep 1 && $*) || (sleep 2 && $*)
2025-03-14T04:08:52.5082615Z }
2025-03-14T04:08:52.5082850Z 
2025-03-14T04:08:52.5083110Z retry login "${DOCKER_REGISTRY}"
2025-03-14T04:08:52.5083446Z 
2025-03-14T04:08:52.5083675Z set -e
2025-03-14T04:08:52.5084032Z # ignore output since only exit code is used for conditional
2025-03-14T04:08:52.5084555Z # only pull docker image if it's not available locally
2025-03-14T04:08:52.5085132Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then
2025-03-14T04:08:52.5085665Z   retry docker pull "${DOCKER_IMAGE}"
2025-03-14T04:08:52.5086012Z fi
2025-03-14T04:08:52.5091112Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:08:52.5091510Z env:
2025-03-14T04:08:52.5091760Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:08:52.5092630Z   DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:52.5093491Z   DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:08:52.5093912Z ##[endgroup]
2025-03-14T04:08:52.5115863Z + set +e
2025-03-14T04:08:52.5116472Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:08:52.5116962Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:08:52.5119592Z + aws ecr get-login-password --region us-east-1
2025-03-14T04:08:52.5120764Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-03-14T04:08:53.0232904Z WARNING! Your password will be stored unencrypted in /home/ec2-user/.docker/config.json.
2025-03-14T04:08:53.0233599Z Configure a credential helper to remove this warning. See
2025-03-14T04:08:53.0234221Z https://docs.docker.com/engine/reference/commandline/login/#credentials-store
2025-03-14T04:08:53.0234796Z 
2025-03-14T04:08:53.0235325Z Login Succeeded
2025-03-14T04:08:53.0245982Z + set -e
2025-03-14T04:08:53.0246744Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:53.0352957Z + retry docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:53.0354183Z + docker pull 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:08:53.2563524Z aa89d6e739080d90fa18625d57297c6734465849: Pulling from pytorch/pytorch-linux-jammy-py3.9-gcc11
2025-03-14T04:08:53.2573255Z 8f84a9f2102e: Pulling fs layer
2025-03-14T04:08:53.2574140Z d6ee0b7aac7a: Pulling fs layer
2025-03-14T04:08:53.2575071Z 747c5ff5a7b7: Pulling fs layer
2025-03-14T04:08:53.2575612Z 87b8b8bf9a96: Pulling fs layer
2025-03-14T04:08:53.2576128Z 3cebdd3ecb51: Pulling fs layer
2025-03-14T04:08:53.2576875Z 9875190f9521: Pulling fs layer
2025-03-14T04:08:53.2577247Z 68592b961c78: Pulling fs layer
2025-03-14T04:08:53.2577923Z cada203b76ef: Pulling fs layer
2025-03-14T04:08:53.2578483Z ee39a89b29f4: Pulling fs layer
2025-03-14T04:08:53.2579040Z 636df2e6d854: Pulling fs layer
2025-03-14T04:08:53.2579557Z 1d4ea74fe29a: Pulling fs layer
2025-03-14T04:08:53.2580036Z 5c57d945e774: Pulling fs layer
2025-03-14T04:08:53.2580469Z cc3d17693771: Pulling fs layer
2025-03-14T04:08:53.2580833Z b97e880ec263: Pulling fs layer
2025-03-14T04:08:53.2581137Z 4f4fb700ef54: Pulling fs layer
2025-03-14T04:08:53.2581490Z 3cebdd3ecb51: Waiting
2025-03-14T04:08:53.2581833Z eceeac2a64b3: Pulling fs layer
2025-03-14T04:08:53.2582199Z e0ce9fa68ff6: Pulling fs layer
2025-03-14T04:08:53.2582492Z ee39a89b29f4: Waiting
2025-03-14T04:08:53.2582770Z af0b38d0861b: Pulling fs layer
2025-03-14T04:08:53.2583201Z 83e5b1b4a717: Pulling fs layer
2025-03-14T04:08:53.2583677Z cada203b76ef: Waiting
2025-03-14T04:08:53.2584172Z 636df2e6d854: Waiting
2025-03-14T04:08:53.2584672Z d306a6a6b642: Pulling fs layer
2025-03-14T04:08:53.2585289Z 68592b961c78: Waiting
2025-03-14T04:08:53.2586073Z 87b8b8bf9a96: Waiting
2025-03-14T04:08:53.2586503Z 9875190f9521: Waiting
2025-03-14T04:08:53.2586790Z 8bf6bdf7bb7b: Pulling fs layer
2025-03-14T04:08:53.2587130Z 5c57d945e774: Waiting
2025-03-14T04:08:53.2587398Z 17779673764c: Pulling fs layer
2025-03-14T04:08:53.2587687Z cc3d17693771: Waiting
2025-03-14T04:08:53.2587945Z b97e880ec263: Waiting
2025-03-14T04:08:53.2588209Z eceeac2a64b3: Waiting
2025-03-14T04:08:53.2588483Z 2e8ffa6248b3: Pulling fs layer
2025-03-14T04:08:53.2588778Z 4f4fb700ef54: Waiting
2025-03-14T04:08:53.2589034Z e0ce9fa68ff6: Waiting
2025-03-14T04:08:53.2589303Z 95b9c737ea76: Pulling fs layer
2025-03-14T04:08:53.2589598Z af0b38d0861b: Waiting
2025-03-14T04:08:53.2589870Z ad432fda02cd: Pulling fs layer
2025-03-14T04:08:53.2590152Z 8bf6bdf7bb7b: Waiting
2025-03-14T04:08:53.2590414Z d306a6a6b642: Waiting
2025-03-14T04:08:53.2590845Z 17779673764c: Waiting
2025-03-14T04:08:53.2591142Z 83e5b1b4a717: Waiting
2025-03-14T04:08:53.2591413Z 5d3bc42935c3: Pulling fs layer
2025-03-14T04:08:53.2591730Z 95b9c737ea76: Waiting
2025-03-14T04:08:53.2592004Z d8c07316b7ca: Pulling fs layer
2025-03-14T04:08:53.2592322Z ad432fda02cd: Waiting
2025-03-14T04:08:53.2592581Z 5d3bc42935c3: Waiting
2025-03-14T04:08:53.2592954Z 6824395fbc6f: Pulling fs layer
2025-03-14T04:08:53.2593248Z d8c07316b7ca: Waiting
2025-03-14T04:08:53.2593518Z 119cd85655c1: Pulling fs layer
2025-03-14T04:08:53.2593826Z dbd96d29d52e: Pulling fs layer
2025-03-14T04:08:53.2594117Z 6824395fbc6f: Waiting
2025-03-14T04:08:53.2594388Z cc77928dad7d: Pulling fs layer
2025-03-14T04:08:53.2594728Z 119cd85655c1: Waiting
2025-03-14T04:08:53.2595000Z c0d38f09f6f4: Pulling fs layer
2025-03-14T04:08:53.2595295Z dbd96d29d52e: Waiting
2025-03-14T04:08:53.2595624Z 08e926bccef6: Pulling fs layer
2025-03-14T04:08:53.2596082Z cc77928dad7d: Waiting
2025-03-14T04:08:53.2596360Z f937a51b9986: Pulling fs layer
2025-03-14T04:08:53.2596718Z b81a345fe031: Pulling fs layer
2025-03-14T04:08:53.2597010Z 08e926bccef6: Waiting
2025-03-14T04:08:53.2597272Z f937a51b9986: Waiting
2025-03-14T04:08:53.2597541Z 5ee2483891c5: Pulling fs layer
2025-03-14T04:08:53.2597852Z ca4fb7ee460a: Pulling fs layer
2025-03-14T04:08:53.2598144Z 5ee2483891c5: Waiting
2025-03-14T04:08:53.2598589Z 6d7937c4d3ea: Pulling fs layer
2025-03-14T04:08:53.2598903Z 3ddc0e8251be: Pulling fs layer
2025-03-14T04:08:53.2599196Z ee39eb7c5421: Pulling fs layer
2025-03-14T04:08:53.2599490Z 6d7937c4d3ea: Waiting
2025-03-14T04:08:53.2599781Z 3ddc0e8251be: Waiting
2025-03-14T04:08:53.2600042Z ee39eb7c5421: Waiting
2025-03-14T04:08:53.2600307Z 14258b788228: Pulling fs layer
2025-03-14T04:08:53.2600607Z 60b8a611752b: Pulling fs layer
2025-03-14T04:08:53.2600931Z 9b3a3723fc19: Pulling fs layer
2025-03-14T04:08:53.2601237Z 6ac5fca3a675: Pulling fs layer
2025-03-14T04:08:53.2601540Z aa265769b086: Pulling fs layer
2025-03-14T04:08:53.2601841Z 9b5bf41436cc: Pulling fs layer
2025-03-14T04:08:53.2602297Z 14258b788228: Waiting
2025-03-14T04:08:53.2602571Z e0440b1e75c3: Pulling fs layer
2025-03-14T04:08:53.2602954Z 60b8a611752b: Waiting
2025-03-14T04:08:53.2603260Z aa265769b086: Waiting
2025-03-14T04:08:53.2603524Z 6ac5fca3a675: Waiting
2025-03-14T04:08:53.2603771Z 9b5bf41436cc: Waiting
2025-03-14T04:08:53.2604080Z 7b0f902826ba: Pulling fs layer
2025-03-14T04:08:53.2604371Z e0440b1e75c3: Waiting
2025-03-14T04:08:53.2604626Z 9b3a3723fc19: Waiting
2025-03-14T04:08:53.2604892Z 082b0db9631d: Pulling fs layer
2025-03-14T04:08:53.2605255Z 7b0f902826ba: Waiting
2025-03-14T04:08:53.2605617Z 03e517f304e9: Pulling fs layer
2025-03-14T04:08:53.2606071Z cac24658ea30: Pulling fs layer
2025-03-14T04:08:53.2606500Z f587b2ee91c1: Pulling fs layer
2025-03-14T04:08:53.2606975Z 082b0db9631d: Waiting
2025-03-14T04:08:53.2607391Z cac24658ea30: Waiting
2025-03-14T04:08:53.2607809Z 03e517f304e9: Waiting
2025-03-14T04:08:53.2608212Z 071558377223: Pulling fs layer
2025-03-14T04:08:53.2608716Z 141eb03e8120: Pulling fs layer
2025-03-14T04:08:53.2609057Z 9008a6e05a8e: Pulling fs layer
2025-03-14T04:08:53.2609425Z f587b2ee91c1: Waiting
2025-03-14T04:08:53.2609688Z 071558377223: Waiting
2025-03-14T04:08:53.2609954Z 9008a6e05a8e: Waiting
2025-03-14T04:08:53.2610232Z 59d79792a1c9: Pulling fs layer
2025-03-14T04:08:53.2610534Z 141eb03e8120: Waiting
2025-03-14T04:08:53.2611041Z 59d79792a1c9: Waiting
2025-03-14T04:08:53.2611348Z fbe892ee1d5d: Pulling fs layer
2025-03-14T04:08:53.2611657Z 2117add440a2: Pulling fs layer
2025-03-14T04:08:53.2611955Z fbe892ee1d5d: Waiting
2025-03-14T04:08:53.2612223Z 2117add440a2: Waiting
2025-03-14T04:08:53.2612502Z b59fff3e88c4: Pulling fs layer
2025-03-14T04:08:53.2612863Z 6e1d1e5ed2fc: Pulling fs layer
2025-03-14T04:08:53.2613176Z e66c39c0da12: Pulling fs layer
2025-03-14T04:08:53.2613482Z b59fff3e88c4: Waiting
2025-03-14T04:08:53.2613750Z 6e1d1e5ed2fc: Waiting
2025-03-14T04:08:53.2614029Z 43dd24f0c7b5: Pulling fs layer
2025-03-14T04:08:53.2614325Z e66c39c0da12: Waiting
2025-03-14T04:08:53.2614740Z f0ff235daf3e: Pulling fs layer
2025-03-14T04:08:53.2615058Z 14c7e3cf113c: Pulling fs layer
2025-03-14T04:08:53.2615355Z 43dd24f0c7b5: Waiting
2025-03-14T04:08:53.2615633Z fafd68c92609: Pulling fs layer
2025-03-14T04:08:53.2615931Z f0ff235daf3e: Waiting
2025-03-14T04:08:53.2616207Z 5ec7e9d96465: Pulling fs layer
2025-03-14T04:08:53.2616500Z 14c7e3cf113c: Waiting
2025-03-14T04:08:53.2616766Z 023489ff1b0b: Pulling fs layer
2025-03-14T04:08:53.2617056Z fafd68c92609: Waiting
2025-03-14T04:08:53.2617325Z 5f15c6ed2fe9: Pulling fs layer
2025-03-14T04:08:53.2617615Z 5ec7e9d96465: Waiting
2025-03-14T04:08:53.2617882Z 339b86aa9361: Pulling fs layer
2025-03-14T04:08:53.2618169Z 023489ff1b0b: Waiting
2025-03-14T04:08:53.2618436Z fd2517474e93: Pulling fs layer
2025-03-14T04:08:53.2618725Z 5f15c6ed2fe9: Waiting
2025-03-14T04:08:53.2618984Z d28088afca53: Pulling fs layer
2025-03-14T04:08:53.2619279Z fd2517474e93: Waiting
2025-03-14T04:08:53.2619553Z b57c57cc548b: Pulling fs layer
2025-03-14T04:08:53.2619853Z d28088afca53: Waiting
2025-03-14T04:08:53.2620134Z 94dba42bfd0c: Pulling fs layer
2025-03-14T04:08:53.2620431Z b57c57cc548b: Waiting
2025-03-14T04:08:53.2620690Z 94dba42bfd0c: Waiting
2025-03-14T04:08:53.3392217Z d6ee0b7aac7a: Verifying Checksum
2025-03-14T04:08:53.3392600Z d6ee0b7aac7a: Download complete
2025-03-14T04:08:53.4207824Z 87b8b8bf9a96: Download complete
2025-03-14T04:08:53.4943474Z 3cebdd3ecb51: Verifying Checksum
2025-03-14T04:08:53.4943875Z 3cebdd3ecb51: Download complete
2025-03-14T04:08:53.5948301Z 9875190f9521: Verifying Checksum
2025-03-14T04:08:53.5948742Z 9875190f9521: Download complete
2025-03-14T04:08:53.6255980Z 8f84a9f2102e: Verifying Checksum
2025-03-14T04:08:53.6256364Z 8f84a9f2102e: Download complete
2025-03-14T04:08:53.6801564Z 68592b961c78: Verifying Checksum
2025-03-14T04:08:53.6802205Z 68592b961c78: Download complete
2025-03-14T04:08:53.6999646Z cada203b76ef: Download complete
2025-03-14T04:08:53.7932225Z 636df2e6d854: Download complete
2025-03-14T04:08:53.8806298Z 1d4ea74fe29a: Verifying Checksum
2025-03-14T04:08:53.8806930Z 1d4ea74fe29a: Download complete
2025-03-14T04:08:53.9707005Z 5c57d945e774: Verifying Checksum
2025-03-14T04:08:53.9707569Z 5c57d945e774: Download complete
2025-03-14T04:08:54.0607056Z cc3d17693771: Download complete
2025-03-14T04:08:54.6949665Z 8f84a9f2102e: Pull complete
2025-03-14T04:08:54.8485586Z d6ee0b7aac7a: Pull complete
2025-03-14T04:08:54.8497451Z ee39a89b29f4: Verifying Checksum
2025-03-14T04:08:54.8497795Z ee39a89b29f4: Download complete
2025-03-14T04:08:54.8576656Z 4f4fb700ef54: Verifying Checksum
2025-03-14T04:08:54.8577098Z 4f4fb700ef54: Download complete
2025-03-14T04:08:54.9552726Z eceeac2a64b3: Verifying Checksum
2025-03-14T04:08:54.9553298Z eceeac2a64b3: Download complete
2025-03-14T04:08:55.0508698Z e0ce9fa68ff6: Download complete
2025-03-14T04:08:55.1387476Z af0b38d0861b: Download complete
2025-03-14T04:08:55.2404147Z 83e5b1b4a717: Verifying Checksum
2025-03-14T04:08:55.2404664Z 83e5b1b4a717: Download complete
2025-03-14T04:08:55.3052991Z d306a6a6b642: Download complete
2025-03-14T04:08:55.3737004Z 8bf6bdf7bb7b: Download complete
2025-03-14T04:08:55.4430355Z 17779673764c: Verifying Checksum
2025-03-14T04:08:55.4430749Z 17779673764c: Download complete
2025-03-14T04:08:55.5386075Z 2e8ffa6248b3: Verifying Checksum
2025-03-14T04:08:55.5386661Z 2e8ffa6248b3: Download complete
2025-03-14T04:08:55.6296707Z 95b9c737ea76: Verifying Checksum
2025-03-14T04:08:55.6297279Z 95b9c737ea76: Download complete
2025-03-14T04:08:56.5155793Z 747c5ff5a7b7: Verifying Checksum
2025-03-14T04:08:56.5156365Z 747c5ff5a7b7: Download complete
2025-03-14T04:08:56.6186621Z 5d3bc42935c3: Verifying Checksum
2025-03-14T04:08:56.6187131Z 5d3bc42935c3: Download complete
2025-03-14T04:08:56.6989143Z d8c07316b7ca: Download complete
2025-03-14T04:08:56.7747014Z 6824395fbc6f: Verifying Checksum
2025-03-14T04:08:56.7747604Z 6824395fbc6f: Download complete
2025-03-14T04:08:56.8344149Z 119cd85655c1: Verifying Checksum
2025-03-14T04:08:56.8344774Z 119cd85655c1: Download complete
2025-03-14T04:08:56.9075952Z dbd96d29d52e: Verifying Checksum
2025-03-14T04:08:56.9076614Z dbd96d29d52e: Download complete
2025-03-14T04:08:56.9350075Z ad432fda02cd: Verifying Checksum
2025-03-14T04:08:56.9350731Z ad432fda02cd: Download complete
2025-03-14T04:08:57.0038139Z c0d38f09f6f4: Verifying Checksum
2025-03-14T04:08:57.0038804Z c0d38f09f6f4: Download complete
2025-03-14T04:08:57.0966492Z 08e926bccef6: Download complete
2025-03-14T04:08:57.1681905Z f937a51b9986: Download complete
2025-03-14T04:08:57.2613949Z b81a345fe031: Verifying Checksum
2025-03-14T04:08:57.2614494Z b81a345fe031: Download complete
2025-03-14T04:08:57.3486240Z 5ee2483891c5: Verifying Checksum
2025-03-14T04:08:57.3487712Z 5ee2483891c5: Download complete
2025-03-14T04:08:57.4356408Z ca4fb7ee460a: Download complete
2025-03-14T04:08:58.0002902Z 6d7937c4d3ea: Verifying Checksum
2025-03-14T04:08:58.0003604Z 6d7937c4d3ea: Download complete
2025-03-14T04:08:58.0848018Z 3ddc0e8251be: Download complete
2025-03-14T04:08:58.1636120Z ee39eb7c5421: Verifying Checksum
2025-03-14T04:08:58.1637233Z ee39eb7c5421: Download complete
2025-03-14T04:08:58.2391145Z 14258b788228: Verifying Checksum
2025-03-14T04:08:58.2392355Z 14258b788228: Download complete
2025-03-14T04:08:58.3333359Z 60b8a611752b: Verifying Checksum
2025-03-14T04:08:58.3333881Z 60b8a611752b: Download complete
2025-03-14T04:08:58.5866078Z 9b3a3723fc19: Verifying Checksum
2025-03-14T04:08:58.5866756Z 9b3a3723fc19: Download complete
2025-03-14T04:08:58.6883300Z 6ac5fca3a675: Verifying Checksum
2025-03-14T04:08:58.6884290Z 6ac5fca3a675: Download complete
2025-03-14T04:08:58.7974421Z aa265769b086: Download complete
2025-03-14T04:08:58.8910959Z 9b5bf41436cc: Verifying Checksum
2025-03-14T04:08:58.8911599Z 9b5bf41436cc: Download complete
2025-03-14T04:08:58.9750360Z e0440b1e75c3: Verifying Checksum
2025-03-14T04:08:58.9751502Z e0440b1e75c3: Download complete
2025-03-14T04:08:59.0711850Z 7b0f902826ba: Verifying Checksum
2025-03-14T04:08:59.0712772Z 7b0f902826ba: Download complete
2025-03-14T04:08:59.1895332Z 082b0db9631d: Download complete
2025-03-14T04:08:59.3244062Z 03e517f304e9: Verifying Checksum
2025-03-14T04:08:59.3244709Z 03e517f304e9: Download complete
2025-03-14T04:09:04.5276339Z cc77928dad7d: Verifying Checksum
2025-03-14T04:09:04.5276916Z cc77928dad7d: Download complete
2025-03-14T04:09:04.6419516Z f587b2ee91c1: Verifying Checksum
2025-03-14T04:09:04.6420312Z f587b2ee91c1: Download complete
2025-03-14T04:09:04.7253390Z 071558377223: Download complete
2025-03-14T04:09:04.8128570Z 141eb03e8120: Verifying Checksum
2025-03-14T04:09:04.8129067Z 141eb03e8120: Download complete
2025-03-14T04:09:04.8977945Z 9008a6e05a8e: Download complete
2025-03-14T04:09:04.9906505Z 59d79792a1c9: Verifying Checksum
2025-03-14T04:09:04.9907362Z 59d79792a1c9: Download complete
2025-03-14T04:09:05.0807275Z fbe892ee1d5d: Download complete
2025-03-14T04:09:05.1761935Z 2117add440a2: Verifying Checksum
2025-03-14T04:09:05.1762547Z 2117add440a2: Download complete
2025-03-14T04:09:05.2439590Z b59fff3e88c4: Verifying Checksum
2025-03-14T04:09:05.2440317Z b59fff3e88c4: Download complete
2025-03-14T04:09:05.3307339Z 6e1d1e5ed2fc: Download complete
2025-03-14T04:09:05.3830056Z 747c5ff5a7b7: Pull complete
2025-03-14T04:09:05.4237619Z e66c39c0da12: Verifying Checksum
2025-03-14T04:09:05.4238254Z e66c39c0da12: Download complete
2025-03-14T04:09:05.4991924Z 87b8b8bf9a96: Pull complete
2025-03-14T04:09:05.5057742Z 43dd24f0c7b5: Verifying Checksum
2025-03-14T04:09:05.5058575Z 43dd24f0c7b5: Download complete
2025-03-14T04:09:05.5977533Z f0ff235daf3e: Download complete
2025-03-14T04:09:05.6192803Z 3cebdd3ecb51: Pull complete
2025-03-14T04:09:05.6822386Z 14c7e3cf113c: Download complete
2025-03-14T04:09:05.7079058Z 9875190f9521: Pull complete
2025-03-14T04:09:05.7608761Z fafd68c92609: Download complete
2025-03-14T04:09:05.7980112Z 68592b961c78: Pull complete
2025-03-14T04:09:05.8344181Z cada203b76ef: Pull complete
2025-03-14T04:09:11.1553333Z 5ec7e9d96465: Verifying Checksum
2025-03-14T04:09:11.1553857Z 5ec7e9d96465: Download complete
2025-03-14T04:09:11.2525891Z 023489ff1b0b: Verifying Checksum
2025-03-14T04:09:11.2526594Z 023489ff1b0b: Download complete
2025-03-14T04:09:11.3447924Z 5f15c6ed2fe9: Verifying Checksum
2025-03-14T04:09:11.3448402Z 5f15c6ed2fe9: Download complete
2025-03-14T04:09:11.4348434Z 339b86aa9361: Verifying Checksum
2025-03-14T04:09:11.4348963Z 339b86aa9361: Download complete
2025-03-14T04:09:11.5056307Z fd2517474e93: Download complete
2025-03-14T04:09:11.5936444Z d28088afca53: Verifying Checksum
2025-03-14T04:09:11.5937298Z d28088afca53: Download complete
2025-03-14T04:09:11.6823524Z b57c57cc548b: Verifying Checksum
2025-03-14T04:09:11.6823980Z b57c57cc548b: Download complete
2025-03-14T04:09:13.0812333Z 94dba42bfd0c: Verifying Checksum
2025-03-14T04:09:13.0812853Z 94dba42bfd0c: Download complete
2025-03-14T04:09:28.9906956Z ee39a89b29f4: Pull complete
2025-03-14T04:09:29.1127175Z 636df2e6d854: Pull complete
2025-03-14T04:09:29.1748522Z 1d4ea74fe29a: Pull complete
2025-03-14T04:09:29.2205926Z 5c57d945e774: Pull complete
2025-03-14T04:09:29.4196442Z cc3d17693771: Pull complete
2025-03-14T04:09:45.5556869Z b97e880ec263: Verifying Checksum
2025-03-14T04:09:45.5557282Z b97e880ec263: Download complete
2025-03-14T04:10:25.4702203Z cac24658ea30: Verifying Checksum
2025-03-14T04:10:25.4702777Z cac24658ea30: Download complete
2025-03-14T04:10:59.7181720Z b97e880ec263: Pull complete
2025-03-14T04:10:59.8854985Z 4f4fb700ef54: Pull complete
2025-03-14T04:10:59.9666656Z eceeac2a64b3: Pull complete
2025-03-14T04:11:00.0870161Z e0ce9fa68ff6: Pull complete
2025-03-14T04:11:00.1753394Z af0b38d0861b: Pull complete
2025-03-14T04:11:00.2698051Z 83e5b1b4a717: Pull complete
2025-03-14T04:11:00.4117669Z d306a6a6b642: Pull complete
2025-03-14T04:11:00.5551005Z 8bf6bdf7bb7b: Pull complete
2025-03-14T04:11:00.6173410Z 17779673764c: Pull complete
2025-03-14T04:11:00.6736562Z 2e8ffa6248b3: Pull complete
2025-03-14T04:11:00.7048148Z 95b9c737ea76: Pull complete
2025-03-14T04:11:03.6796241Z ad432fda02cd: Pull complete
2025-03-14T04:11:03.7143132Z 5d3bc42935c3: Pull complete
2025-03-14T04:11:03.7445070Z d8c07316b7ca: Pull complete
2025-03-14T04:11:03.7762862Z 6824395fbc6f: Pull complete
2025-03-14T04:11:03.8189449Z 119cd85655c1: Pull complete
2025-03-14T04:11:03.9682035Z dbd96d29d52e: Pull complete
2025-03-14T04:11:15.0672408Z cc77928dad7d: Pull complete
2025-03-14T04:11:15.2428995Z c0d38f09f6f4: Pull complete
2025-03-14T04:11:15.3296011Z 08e926bccef6: Pull complete
2025-03-14T04:11:15.5042349Z f937a51b9986: Pull complete
2025-03-14T04:11:15.5704313Z b81a345fe031: Pull complete
2025-03-14T04:11:15.6301247Z 5ee2483891c5: Pull complete
2025-03-14T04:11:15.6598394Z ca4fb7ee460a: Pull complete
2025-03-14T04:11:17.8349434Z 6d7937c4d3ea: Pull complete
2025-03-14T04:11:17.9457983Z 3ddc0e8251be: Pull complete
2025-03-14T04:11:18.0268998Z ee39eb7c5421: Pull complete
2025-03-14T04:11:18.1952583Z 14258b788228: Pull complete
2025-03-14T04:11:18.2238496Z 60b8a611752b: Pull complete
2025-03-14T04:11:18.7382754Z 9b3a3723fc19: Pull complete
2025-03-14T04:11:18.7672903Z 6ac5fca3a675: Pull complete
2025-03-14T04:11:18.7965752Z aa265769b086: Pull complete
2025-03-14T04:11:18.8576301Z 9b5bf41436cc: Pull complete
2025-03-14T04:11:18.9580869Z e0440b1e75c3: Pull complete
2025-03-14T04:11:19.1200674Z 7b0f902826ba: Pull complete
2025-03-14T04:11:19.2337519Z 082b0db9631d: Pull complete
2025-03-14T04:11:19.3227503Z 03e517f304e9: Pull complete
2025-03-14T04:13:50.8519245Z cac24658ea30: Pull complete
2025-03-14T04:13:50.9328511Z f587b2ee91c1: Pull complete
2025-03-14T04:13:51.2793805Z 071558377223: Pull complete
2025-03-14T04:13:51.3632581Z 141eb03e8120: Pull complete
2025-03-14T04:13:51.4562559Z 9008a6e05a8e: Pull complete
2025-03-14T04:13:51.5596931Z 59d79792a1c9: Pull complete
2025-03-14T04:13:51.6276385Z fbe892ee1d5d: Pull complete
2025-03-14T04:13:51.6566745Z 2117add440a2: Pull complete
2025-03-14T04:13:51.8256389Z b59fff3e88c4: Pull complete
2025-03-14T04:13:51.9086324Z 6e1d1e5ed2fc: Pull complete
2025-03-14T04:13:51.9384777Z e66c39c0da12: Pull complete
2025-03-14T04:13:51.9967252Z 43dd24f0c7b5: Pull complete
2025-03-14T04:13:52.0663437Z f0ff235daf3e: Pull complete
2025-03-14T04:13:52.2264713Z 14c7e3cf113c: Pull complete
2025-03-14T04:13:52.2586653Z fafd68c92609: Pull complete
2025-03-14T04:13:59.1026955Z 5ec7e9d96465: Pull complete
2025-03-14T04:13:59.2579500Z 023489ff1b0b: Pull complete
2025-03-14T04:13:59.3469964Z 5f15c6ed2fe9: Pull complete
2025-03-14T04:13:59.4727593Z 339b86aa9361: Pull complete
2025-03-14T04:13:59.5476462Z fd2517474e93: Pull complete
2025-03-14T04:13:59.6389178Z d28088afca53: Pull complete
2025-03-14T04:13:59.8599844Z b57c57cc548b: Pull complete
2025-03-14T04:14:01.6996513Z 94dba42bfd0c: Pull complete
2025-03-14T04:14:01.8528268Z Digest: sha256:af98f5b94e2a6aa569662f00ae8e7d849202075bbcf06a2f7dd51e480c3adbef
2025-03-14T04:14:01.8690163Z Status: Downloaded newer image for 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:14:01.8992152Z 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:14:01.9038927Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT"
2025-03-14T04:14:01.9039941Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT"
2025-03-14T04:14:01.9047279Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:14:01.9047715Z env:
2025-03-14T04:14:01.9047964Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:01.9048247Z ##[endgroup]
2025-03-14T04:14:01.9175892Z Prepare all required actions
2025-03-14T04:14:01.9247147Z ##[group]Run ./.github/actions/get-workflow-job-id
2025-03-14T04:14:01.9247526Z with:
2025-03-14T04:14:01.9247949Z   github-token: ***
2025-03-14T04:14:01.9248204Z env:
2025-03-14T04:14:01.9248448Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:01.9248738Z ##[endgroup]
2025-03-14T04:14:01.9299923Z ##[group]Run set -eux
2025-03-14T04:14:01.9300230Z set -eux
2025-03-14T04:14:01.9300707Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}"
2025-03-14T04:14:01.9306619Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:14:01.9307031Z env:
2025-03-14T04:14:01.9307273Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:01.9307737Z   GITHUB_TOKEN: ***
2025-03-14T04:14:01.9307997Z ##[endgroup]
2025-03-14T04:14:01.9330520Z + python3 .github/scripts/get_workflow_job_id.py 13849515290 i-09d49014f68772654
2025-03-14T04:14:03.6675660Z setting job-id=38754724585
2025-03-14T04:14:03.6676307Z setting job-name=linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)
2025-03-14T04:14:03.6840911Z ##[group]Run python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 dataclasses_json==0.6.7
2025-03-14T04:14:03.6841691Z python3 -m pip install psutil==5.9.1 nvidia-ml-py==11.525.84 dataclasses_json==0.6.7
2025-03-14T04:14:03.6842325Z python3 -m tools.stats.monitor > usage_log.txt 2>&1 &
2025-03-14T04:14:03.6842835Z echo "monitor-script-pid=${!}" >> "${GITHUB_OUTPUT}"
2025-03-14T04:14:03.6849256Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:14:03.6849662Z env:
2025-03-14T04:14:03.6849889Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:03.6850173Z   JOB_ID: 38754724585
2025-03-14T04:14:03.6850596Z   JOB_NAME: linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)
2025-03-14T04:14:03.6851083Z   WORKFLOW_NAME: pull
2025-03-14T04:14:03.6851361Z   WORKFLOW_RUN_ID: 13849515290
2025-03-14T04:14:03.6851652Z ##[endgroup]
2025-03-14T04:14:03.9547675Z Defaulting to user installation because normal site-packages is not writeable
2025-03-14T04:14:03.9758391Z Requirement already satisfied: psutil==5.9.1 in /home/ec2-user/.local/lib/python3.9/site-packages (5.9.1)
2025-03-14T04:14:03.9764246Z Requirement already satisfied: nvidia-ml-py==11.525.84 in /home/ec2-user/.local/lib/python3.9/site-packages (11.525.84)
2025-03-14T04:14:03.9769256Z Requirement already satisfied: dataclasses_json==0.6.7 in /home/ec2-user/.local/lib/python3.9/site-packages (0.6.7)
2025-03-14T04:14:03.9880225Z Requirement already satisfied: typing-inspect<1,>=0.4.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from dataclasses_json==0.6.7) (0.9.0)
2025-03-14T04:14:03.9885675Z Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from dataclasses_json==0.6.7) (3.26.1)
2025-03-14T04:14:03.9977297Z Requirement already satisfied: packaging>=17.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from marshmallow<4.0.0,>=3.18.0->dataclasses_json==0.6.7) (24.2)
2025-03-14T04:14:04.0006278Z Requirement already satisfied: mypy-extensions>=0.3.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from typing-inspect<1,>=0.4.0->dataclasses_json==0.6.7) (1.0.0)
2025-03-14T04:14:04.0010337Z Requirement already satisfied: typing-extensions>=3.7.4 in /home/ec2-user/.local/lib/python3.9/site-packages (from typing-inspect<1,>=0.4.0->dataclasses_json==0.6.7) (4.12.2)
2025-03-14T04:14:04.1937659Z Prepare all required actions
2025-03-14T04:14:04.1938085Z Getting action download info
2025-03-14T04:14:04.2974772Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6)
2025-03-14T04:14:04.6040759Z Download action repository 'actions/download-artifact@v4' (SHA:cc203385981b70ca67e1cc392babf9cc229d5806)
2025-03-14T04:14:04.8263124Z ##[group]Run ./.github/actions/download-build-artifacts
2025-03-14T04:14:04.8263547Z with:
2025-03-14T04:14:04.8263812Z   name: linux-jammy-py3.9-gcc11
2025-03-14T04:14:04.8264152Z   s3-bucket: gha-artifacts
2025-03-14T04:14:04.8264454Z env:
2025-03-14T04:14:04.8264704Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:04.8264991Z ##[endgroup]
2025-03-14T04:14:04.8333016Z ##[group]Run seemethere/download-artifact-s3@v4
2025-03-14T04:14:04.8333399Z with:
2025-03-14T04:14:04.8333660Z   name: linux-jammy-py3.9-gcc11
2025-03-14T04:14:04.8334004Z   s3-bucket: gha-artifacts
2025-03-14T04:14:04.8334339Z   region: us-east-1
2025-03-14T04:14:04.8334596Z env:
2025-03-14T04:14:04.8334833Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:04.8335113Z ##[endgroup]
2025-03-14T04:14:05.4986965Z (node:626510) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.
2025-03-14T04:14:05.4987514Z 
2025-03-14T04:14:05.4987716Z Please migrate your code to use AWS SDK for JavaScript (v3).
2025-03-14T04:14:05.4988280Z For more information, check the migration guide at https://a.co/7PzMCcy
2025-03-14T04:14:05.4988882Z (Use `node --trace-warnings ...` to show where the warning was created)
2025-03-14T04:14:05.6063158Z Found 1 objects with prefix pytorch/pytorch/13849515290/linux-jammy-py3.9-gcc11/
2025-03-14T04:14:05.6063897Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip
2025-03-14T04:14:09.5839187Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/artifacts.zip
2025-03-14T04:14:09.5846049Z Artifact download has finished successfully
2025-03-14T04:14:09.6013806Z ##[group]Run unzip -o artifacts.zip
2025-03-14T04:14:09.6014185Z unzip -o artifacts.zip
2025-03-14T04:14:09.6021403Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:14:09.6021820Z env:
2025-03-14T04:14:09.6022062Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:09.6022348Z ##[endgroup]
2025-03-14T04:14:09.6292460Z Archive:  artifacts.zip
2025-03-14T04:14:09.6293294Z    creating: dist/
2025-03-14T04:14:10.6100977Z   inflating: dist/torch-2.8.0a0+gitaed0b7a-cp39-cp39-linux_x86_64.whl  
2025-03-14T04:14:10.6218274Z   inflating: dist/.ninja_log         
2025-03-14T04:14:10.6218736Z    creating: build/custom_test_artifacts/
2025-03-14T04:14:10.6219172Z    creating: build/custom_test_artifacts/custom-op-build/
2025-03-14T04:14:10.6219706Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/
2025-03-14T04:14:10.6220331Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/
2025-03-14T04:14:10.6223572Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml  
2025-03-14T04:14:10.6224288Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/
2025-03-14T04:14:10.6224998Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeSystem.cmake  
2025-03-14T04:14:10.6225841Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdC/
2025-03-14T04:14:10.6226579Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdC/tmp/
2025-03-14T04:14:10.6227526Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdC/CMakeCCompilerId.c  
2025-03-14T04:14:10.6228887Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdC/a.out  
2025-03-14T04:14:10.6229696Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeCCompiler.cmake  
2025-03-14T04:14:10.6230630Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCXX/
2025-03-14T04:14:10.6231387Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCXX/tmp/
2025-03-14T04:14:10.6232640Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCXX/CMakeCXXCompilerId.cpp  
2025-03-14T04:14:10.6233902Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CompilerIdCXX/a.out  
2025-03-14T04:14:10.6234785Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeCXXCompiler.cmake  
2025-03-14T04:14:10.6236159Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_C.bin  
2025-03-14T04:14:10.6237972Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_CXX.bin  
2025-03-14T04:14:10.6238793Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/
2025-03-14T04:14:10.6239503Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache  
2025-03-14T04:14:10.6240214Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/
2025-03-14T04:14:10.6240998Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts  
2025-03-14T04:14:10.6241896Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make  
2025-03-14T04:14:10.6242756Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make  
2025-03-14T04:14:10.6243553Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt  
2025-03-14T04:14:10.6244384Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake  
2025-03-14T04:14:10.6245210Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make  
2025-03-14T04:14:10.6246049Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake  
2025-03-14T04:14:10.6246878Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make  
2025-03-14T04:14:10.6247697Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make  
2025-03-14T04:14:10.6263268Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d  
2025-03-14T04:14:10.6441336Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o  
2025-03-14T04:14:10.6442462Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/
2025-03-14T04:14:10.6444040Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts  
2025-03-14T04:14:10.6445775Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make  
2025-03-14T04:14:10.6447660Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make  
2025-03-14T04:14:10.6448759Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt  
2025-03-14T04:14:10.6449720Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake  
2025-03-14T04:14:10.6450607Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make  
2025-03-14T04:14:10.6451493Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake  
2025-03-14T04:14:10.6452370Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make  
2025-03-14T04:14:10.6453232Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make  
2025-03-14T04:14:10.6467985Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d  
2025-03-14T04:14:10.6543048Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o  
2025-03-14T04:14:10.6544848Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake  
2025-03-14T04:14:10.6546508Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt  
2025-03-14T04:14:10.6547858Z  extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks  
2025-03-14T04:14:10.6549099Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2  
2025-03-14T04:14:10.6550368Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake  
2025-03-14T04:14:10.6551593Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt  
2025-03-14T04:14:10.6552709Z   inflating: build/custom_test_artifacts/custom-op-build/Makefile  
2025-03-14T04:14:10.6553720Z   inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake  
2025-03-14T04:14:10.6705417Z   inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so  
2025-03-14T04:14:10.6765509Z   inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops  
2025-03-14T04:14:10.6766512Z    creating: build/custom_test_artifacts/jit-hook-build/
2025-03-14T04:14:10.6767406Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/
2025-03-14T04:14:10.6768444Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/
2025-03-14T04:14:10.6770794Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml  
2025-03-14T04:14:10.6772031Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/
2025-03-14T04:14:10.6773295Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeSystem.cmake  
2025-03-14T04:14:10.6774607Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdC/
2025-03-14T04:14:10.6775948Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdC/tmp/
2025-03-14T04:14:10.6777456Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdC/CMakeCCompilerId.c  
2025-03-14T04:14:10.6779022Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdC/a.out  
2025-03-14T04:14:10.6780463Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeCCompiler.cmake  
2025-03-14T04:14:10.6781822Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCXX/
2025-03-14T04:14:10.6783163Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCXX/tmp/
2025-03-14T04:14:10.6784773Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCXX/CMakeCXXCompilerId.cpp  
2025-03-14T04:14:10.6786474Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CompilerIdCXX/a.out  
2025-03-14T04:14:10.6788287Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeCXXCompiler.cmake  
2025-03-14T04:14:10.6789940Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_C.bin  
2025-03-14T04:14:10.6791744Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_CXX.bin  
2025-03-14T04:14:10.6793395Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/
2025-03-14T04:14:10.6794679Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache  
2025-03-14T04:14:10.6796028Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/
2025-03-14T04:14:10.6797507Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts  
2025-03-14T04:14:10.6799252Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make  
2025-03-14T04:14:10.6800948Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make  
2025-03-14T04:14:10.6802344Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt  
2025-03-14T04:14:10.6803798Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake  
2025-03-14T04:14:10.6805257Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make  
2025-03-14T04:14:10.6806739Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake  
2025-03-14T04:14:10.6808201Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make  
2025-03-14T04:14:10.6809662Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make  
2025-03-14T04:14:10.6818533Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d  
2025-03-14T04:14:10.6878266Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o  
2025-03-14T04:14:10.6879991Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake  
2025-03-14T04:14:10.6881492Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt  
2025-03-14T04:14:10.6882842Z  extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks  
2025-03-14T04:14:10.6884071Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2  
2025-03-14T04:14:10.6885307Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake  
2025-03-14T04:14:10.6886492Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt  
2025-03-14T04:14:10.6887545Z   inflating: build/custom_test_artifacts/jit-hook-build/Makefile  
2025-03-14T04:14:10.6888658Z   inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake  
2025-03-14T04:14:10.6934483Z   inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks  
2025-03-14T04:14:10.6935624Z    creating: build/custom_test_artifacts/custom-backend-build/
2025-03-14T04:14:10.6936626Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/
2025-03-14T04:14:10.6937757Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/
2025-03-14T04:14:10.6939855Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml  
2025-03-14T04:14:10.6941240Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/
2025-03-14T04:14:10.6942623Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeSystem.cmake  
2025-03-14T04:14:10.6944058Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdC/
2025-03-14T04:14:10.6945533Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdC/tmp/
2025-03-14T04:14:10.6947358Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdC/CMakeCCompilerId.c  
2025-03-14T04:14:10.6949079Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdC/a.out  
2025-03-14T04:14:10.6950607Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeCCompiler.cmake  
2025-03-14T04:14:10.6952125Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCXX/
2025-03-14T04:14:10.6953580Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCXX/tmp/
2025-03-14T04:14:10.6955286Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCXX/CMakeCXXCompilerId.cpp  
2025-03-14T04:14:10.6956971Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CompilerIdCXX/a.out  
2025-03-14T04:14:10.6958583Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeCXXCompiler.cmake  
2025-03-14T04:14:10.6960573Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_C.bin  
2025-03-14T04:14:10.6962496Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/3.31.2/CMakeDetermineCompilerABI_CXX.bin  
2025-03-14T04:14:10.6964173Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/
2025-03-14T04:14:10.6965534Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache  
2025-03-14T04:14:10.6966932Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/
2025-03-14T04:14:10.6968419Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts  
2025-03-14T04:14:10.6970226Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make  
2025-03-14T04:14:10.6972110Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make  
2025-03-14T04:14:10.6973790Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt  
2025-03-14T04:14:10.6975380Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake  
2025-03-14T04:14:10.6977071Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make  
2025-03-14T04:14:10.6978809Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake  
2025-03-14T04:14:10.6980417Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make  
2025-03-14T04:14:10.6982069Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make  
2025-03-14T04:14:10.6983894Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d  
2025-03-14T04:14:10.7086365Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o  
2025-03-14T04:14:10.7088140Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/
2025-03-14T04:14:10.7089893Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts  
2025-03-14T04:14:10.7091868Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make  
2025-03-14T04:14:10.7093740Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make  
2025-03-14T04:14:10.7095380Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt  
2025-03-14T04:14:10.7097068Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake  
2025-03-14T04:14:10.7099091Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make  
2025-03-14T04:14:10.7100821Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake  
2025-03-14T04:14:10.7102478Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make  
2025-03-14T04:14:10.7104152Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make  
2025-03-14T04:14:10.7114882Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d  
2025-03-14T04:14:10.7166188Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o  
2025-03-14T04:14:10.7168134Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake  
2025-03-14T04:14:10.7169912Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt  
2025-03-14T04:14:10.7171366Z  extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks  
2025-03-14T04:14:10.7172741Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2  
2025-03-14T04:14:10.7174097Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake  
2025-03-14T04:14:10.7175378Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt  
2025-03-14T04:14:10.7176529Z   inflating: build/custom_test_artifacts/custom-backend-build/Makefile  
2025-03-14T04:14:10.7177756Z   inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake  
2025-03-14T04:14:10.7271499Z   inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so  
2025-03-14T04:14:10.7312909Z   inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend  
2025-03-14T04:14:10.7313870Z    creating: build/lib/
2025-03-14T04:14:10.7392223Z   inflating: build/lib/libprotobuf-lite.a  
2025-03-14T04:14:10.7811927Z   inflating: build/lib/libprotobuf.a  
2025-03-14T04:14:10.8282364Z   inflating: build/lib/libprotoc.a   
2025-03-14T04:14:10.8292034Z   inflating: build/lib/libpthreadpool.a  
2025-03-14T04:14:10.8300685Z   inflating: build/lib/libcpuinfo.a  
2025-03-14T04:14:10.8308831Z   inflating: build/lib/libcpuinfo_internals.a  
2025-03-14T04:14:10.8309838Z   inflating: build/lib/libclog.a     
2025-03-14T04:14:10.8328582Z   inflating: build/lib/libpytorch_qnnpack.a  
2025-03-14T04:14:10.8331479Z   inflating: build/lib/libnnpack_reference_layers.a  
2025-03-14T04:14:10.8509456Z   inflating: build/lib/libmicrokernels-prod.a  
2025-03-14T04:14:10.8527096Z   inflating: build/lib/libnnpack.a   
2025-03-14T04:14:10.9369035Z   inflating: build/lib/libmicrokernels-all.a  
2025-03-14T04:14:10.9435609Z   inflating: build/lib/libgtest.a    
2025-03-14T04:14:10.9452499Z   inflating: build/lib/libgmock.a    
2025-03-14T04:14:10.9469672Z   inflating: build/lib/libgtest_main.a  
2025-03-14T04:14:10.9470364Z   inflating: build/lib/libgmock_main.a  
2025-03-14T04:14:10.9541306Z   inflating: build/lib/libXNNPACK.a  
2025-03-14T04:14:10.9611126Z   inflating: build/lib/libbenchmark.a  
2025-03-14T04:14:10.9611848Z   inflating: build/lib/libbenchmark_main.a  
2025-03-14T04:14:10.9619709Z   inflating: build/lib/libittnotify.a  
2025-03-14T04:14:10.9681212Z   inflating: build/lib/libasmjit.a   
2025-03-14T04:14:11.0657486Z   inflating: build/lib/libfbgemm.a   
2025-03-14T04:14:11.0684152Z   inflating: build/lib/libtensorpipe_uv.a  
2025-03-14T04:14:11.1210410Z   inflating: build/lib/libtensorpipe.a  
2025-03-14T04:14:11.1318339Z   inflating: build/lib/libgloo.a     
2025-03-14T04:14:11.1357126Z   inflating: build/lib/libonnx_proto.a  
2025-03-14T04:14:11.2043035Z   inflating: build/lib/libonnx.a     
2025-03-14T04:14:12.1636502Z   inflating: build/lib/libdnnl.a     
2025-03-14T04:14:12.1653822Z   inflating: build/lib/libfmt.a      
2025-03-14T04:14:12.1899702Z   inflating: build/lib/libkineto.a   
2025-03-14T04:14:12.2007036Z   inflating: build/lib/libc10.so     
2025-03-14T04:14:12.2008597Z   inflating: build/lib/libtorch_global_deps.so  
2025-03-14T04:14:14.5195949Z   inflating: build/lib/libtorch_cpu.so  
2025-03-14T04:14:14.5196831Z   inflating: build/lib/libtorch.so   
2025-03-14T04:14:14.5201624Z   inflating: build/lib/libunbox_lib.a  
2025-03-14T04:14:14.5267049Z   inflating: build/lib/libtorchbind_test.so  
2025-03-14T04:14:14.5284982Z   inflating: build/lib/libjitbackend_test.so  
2025-03-14T04:14:14.5308534Z   inflating: build/lib/libbackend_with_compiler.so  
2025-03-14T04:14:14.5331213Z   inflating: build/lib/libaoti_custom_ops.so  
2025-03-14T04:14:14.5335107Z   inflating: build/lib/libshm.so     
2025-03-14T04:14:14.7158011Z   inflating: build/lib/libtorch_python.so  
2025-03-14T04:14:14.7190010Z   inflating: build/lib/libnnapi_backend.so  
2025-03-14T04:14:14.7190422Z    creating: build/bin/
2025-03-14T04:14:14.7190721Z    creating: build/bin/CMakeFiles/
2025-03-14T04:14:14.7191374Z   inflating: build/bin/cmake_install.cmake  
2025-03-14T04:14:14.7191766Z   inflating: build/bin/CTestTestfile.cmake  
2025-03-14T04:14:14.7633225Z   inflating: build/bin/protoc-3.13.0.0  
2025-03-14T04:14:14.8070280Z   inflating: build/bin/protoc        
2025-03-14T04:14:14.8123202Z   inflating: build/bin/c10_CompileTimeFunctionPointer_test  
2025-03-14T04:14:14.8177426Z   inflating: build/bin/c10_DeviceGuard_test  
2025-03-14T04:14:14.8232484Z   inflating: build/bin/c10_Device_test  
2025-03-14T04:14:14.8294199Z   inflating: build/bin/c10_DispatchKeySet_test  
2025-03-14T04:14:14.8351263Z   inflating: build/bin/c10_Scalar_test  
2025-03-14T04:14:14.8403155Z   inflating: build/bin/c10_StreamGuard_test  
2025-03-14T04:14:14.8456920Z   inflating: build/bin/c10_SymInt_test  
2025-03-14T04:14:14.8514170Z   inflating: build/bin/c10_InlineDeviceGuard_test  
2025-03-14T04:14:14.8572403Z   inflating: build/bin/c10_InlineStreamGuard_test  
2025-03-14T04:14:14.8624260Z   inflating: build/bin/c10_ConstexprCrc_test  
2025-03-14T04:14:14.8683264Z   inflating: build/bin/c10_SizesAndStrides_test  
2025-03-14T04:14:14.8738973Z   inflating: build/bin/c10_Bitset_test  
2025-03-14T04:14:14.8791711Z   inflating: build/bin/c10_DeadlockDetection_test  
2025-03-14T04:14:14.8865637Z   inflating: build/bin/c10_cow_test  
2025-03-14T04:14:14.8918496Z   inflating: build/bin/c10_ArrayRef_test  
2025-03-14T04:14:14.8977181Z   inflating: build/bin/c10_LeftRight_test  
2025-03-14T04:14:14.9030705Z   inflating: build/bin/c10_Half_test  
2025-03-14T04:14:14.9089052Z   inflating: build/bin/c10_Metaprogramming_test  
2025-03-14T04:14:14.9144895Z   inflating: build/bin/c10_NetworkFlow_test  
2025-03-14T04:14:14.9197751Z   inflating: build/bin/c10_Synchronized_test  
2025-03-14T04:14:14.9255981Z   inflating: build/bin/c10_ThreadLocal_test  
2025-03-14T04:14:14.9310818Z   inflating: build/bin/c10_TypeIndex_test  
2025-03-14T04:14:14.9364494Z   inflating: build/bin/c10_TypeList_test  
2025-03-14T04:14:14.9418795Z   inflating: build/bin/c10_accumulate_test  
2025-03-14T04:14:14.9470666Z   inflating: build/bin/c10_TypeTraits_test  
2025-03-14T04:14:14.9529064Z   inflating: build/bin/c10_bfloat16_test  
2025-03-14T04:14:14.9588355Z   inflating: build/bin/c10_complex_math_test  
2025-03-14T04:14:14.9642197Z   inflating: build/bin/c10_bit_cast_test  
2025-03-14T04:14:14.9694600Z   inflating: build/bin/c10_error_test  
2025-03-14T04:14:14.9752835Z   inflating: build/bin/c10_complex_test  
2025-03-14T04:14:14.9808123Z   inflating: build/bin/c10_exception_test  
2025-03-14T04:14:14.9861178Z   inflating: build/bin/c10_flags_test  
2025-03-14T04:14:15.0025537Z   inflating: build/bin/c10_intrusive_ptr_test  
2025-03-14T04:14:15.0078353Z   inflating: build/bin/c10_generic_math_test  
2025-03-14T04:14:15.0132030Z   inflating: build/bin/c10_irange_test  
2025-03-14T04:14:15.0188375Z   inflating: build/bin/c10_lazy_test  
2025-03-14T04:14:15.0248215Z   inflating: build/bin/c10_logging_test  
2025-03-14T04:14:15.0325520Z   inflating: build/bin/c10_optional_test  
2025-03-14T04:14:15.0389972Z   inflating: build/bin/c10_ordered_preserving_dict_test  
2025-03-14T04:14:15.0446157Z   inflating: build/bin/c10_registry_test  
2025-03-14T04:14:15.0600295Z   inflating: build/bin/c10_small_vector_test  
2025-03-14T04:14:15.0654754Z   inflating: build/bin/c10_ssize_test  
2025-03-14T04:14:15.0709646Z   inflating: build/bin/c10_string_util_test  
2025-03-14T04:14:15.0762675Z   inflating: build/bin/c10_tempfile_test  
2025-03-14T04:14:15.0814881Z   inflating: build/bin/c10_string_view_test  
2025-03-14T04:14:15.0861488Z   inflating: build/bin/c10_intrusive_ptr_benchmark  
2025-03-14T04:14:15.0920418Z   inflating: build/bin/c10_typeid_test  
2025-03-14T04:14:15.1310112Z   inflating: build/bin/vec_test_all_types_DEFAULT  
2025-03-14T04:14:15.1716357Z   inflating: build/bin/vec_test_all_types_AVX512  
2025-03-14T04:14:15.2132600Z   inflating: build/bin/vec_test_all_types_AVX2  
2025-03-14T04:14:15.2188250Z   inflating: build/bin/test_edge_op_registration  
2025-03-14T04:14:15.2245289Z   inflating: build/bin/static_runtime_bench  
2025-03-14T04:14:15.2493871Z   inflating: build/bin/static_runtime_test  
2025-03-14T04:14:15.2570269Z   inflating: build/bin/Dict_test     
2025-03-14T04:14:15.2626602Z   inflating: build/bin/Dimname_test  
2025-03-14T04:14:15.2694163Z   inflating: build/bin/MaybeOwned_test  
2025-03-14T04:14:15.2754074Z   inflating: build/bin/NamedTensor_test  
2025-03-14T04:14:15.2815491Z   inflating: build/bin/apply_utils_test  
2025-03-14T04:14:15.2876947Z   inflating: build/bin/atest         
2025-03-14T04:14:15.2942764Z   inflating: build/bin/basic         
2025-03-14T04:14:15.3001047Z   inflating: build/bin/broadcast_test  
2025-03-14T04:14:15.3054795Z   inflating: build/bin/cpu_allocator_test  
2025-03-14T04:14:15.3115860Z   inflating: build/bin/cpu_generator_test  
2025-03-14T04:14:15.3171559Z   inflating: build/bin/cpu_profiling_allocator_test  
2025-03-14T04:14:15.3264843Z   inflating: build/bin/cpu_rng_test  
2025-03-14T04:14:15.3317954Z   inflating: build/bin/dispatch_key_set_test  
2025-03-14T04:14:15.3371045Z   inflating: build/bin/dlconvertor_test  
2025-03-14T04:14:15.3431311Z   inflating: build/bin/extension_backend_test  
2025-03-14T04:14:15.3489623Z   inflating: build/bin/half_test     
2025-03-14T04:14:15.3588222Z   inflating: build/bin/ivalue_test   
2025-03-14T04:14:15.3641326Z   inflating: build/bin/lazy_tensor_test  
2025-03-14T04:14:15.3697846Z   inflating: build/bin/math_kernel_test  
2025-03-14T04:14:15.3754624Z   inflating: build/bin/memory_format_test  
2025-03-14T04:14:15.3810759Z   inflating: build/bin/memory_overlapping_test  
2025-03-14T04:14:15.3866825Z   inflating: build/bin/mobile_memory_cleanup  
2025-03-14T04:14:15.3925754Z   inflating: build/bin/native_test   
2025-03-14T04:14:15.3979244Z   inflating: build/bin/operator_name_test  
2025-03-14T04:14:15.4034470Z   inflating: build/bin/operators_test  
2025-03-14T04:14:15.4090738Z   inflating: build/bin/packedtensoraccessor_test  
2025-03-14T04:14:15.4160406Z   inflating: build/bin/pow_test      
2025-03-14T04:14:15.4220919Z   inflating: build/bin/quantized_test  
2025-03-14T04:14:15.4273666Z   inflating: build/bin/reduce_ops_test  
2025-03-14T04:14:15.4327597Z   inflating: build/bin/reportMemoryUsage_test  
2025-03-14T04:14:15.4387283Z   inflating: build/bin/scalar_tensor_test  
2025-03-14T04:14:15.4448522Z   inflating: build/bin/scalar_test   
2025-03-14T04:14:15.4502631Z   inflating: build/bin/StorageUtils_test  
2025-03-14T04:14:15.4557567Z   inflating: build/bin/stride_properties_test  
2025-03-14T04:14:15.4638339Z   inflating: build/bin/tensor_iterator_test  
2025-03-14T04:14:15.4695731Z   inflating: build/bin/test_parallel  
2025-03-14T04:14:15.4698562Z   inflating: build/bin/thread_init_test  
2025-03-14T04:14:15.4756550Z   inflating: build/bin/type_ptr_test  
2025-03-14T04:14:15.4819335Z   inflating: build/bin/type_test     
2025-03-14T04:14:15.4874764Z   inflating: build/bin/undefined_tensor_test  
2025-03-14T04:14:15.4876321Z   inflating: build/bin/verify_api_visibility  
2025-03-14T04:14:15.4949012Z   inflating: build/bin/legacy_vmap_test  
2025-03-14T04:14:15.5003547Z   inflating: build/bin/weakref_test  
2025-03-14T04:14:15.5059216Z   inflating: build/bin/wrapdim_test  
2025-03-14T04:14:15.5113756Z   inflating: build/bin/xla_tensor_test  
2025-03-14T04:14:15.5175850Z   inflating: build/bin/IListRef_test  
2025-03-14T04:14:15.5282986Z   inflating: build/bin/List_test     
2025-03-14T04:14:15.5351362Z   inflating: build/bin/KernelFunction_test  
2025-03-14T04:14:15.5472695Z   inflating: build/bin/kernel_function_legacy_test  
2025-03-14T04:14:15.5569435Z   inflating: build/bin/kernel_function_test  
2025-03-14T04:14:15.5695758Z   inflating: build/bin/kernel_lambda_legacy_test  
2025-03-14T04:14:15.5798659Z   inflating: build/bin/kernel_lambda_test  
2025-03-14T04:14:15.5861882Z   inflating: build/bin/kernel_stackbased_test  
2025-03-14T04:14:15.5958622Z   inflating: build/bin/make_boxed_from_unboxed_functor_test  
2025-03-14T04:14:15.6012721Z   inflating: build/bin/CppSignature_test  
2025-03-14T04:14:15.6070637Z   inflating: build/bin/backend_fallback_test  
2025-03-14T04:14:15.6122521Z   inflating: build/bin/op_allowlist_test  
2025-03-14T04:14:15.6426038Z   inflating: build/bin/op_registration_test  
2025-03-14T04:14:15.6492270Z   inflating: build/bin/inline_container_test  
2025-03-14T04:14:15.7049196Z   inflating: build/bin/test_jit      
2025-03-14T04:14:15.7104516Z   inflating: build/bin/BackoffTest   
2025-03-14T04:14:15.7160710Z   inflating: build/bin/FileStoreTest  
2025-03-14T04:14:15.7220080Z   inflating: build/bin/TCPStoreTest  
2025-03-14T04:14:15.7276203Z   inflating: build/bin/HashStoreTest  
2025-03-14T04:14:15.8046135Z   inflating: build/bin/test_tensorexpr  
2025-03-14T04:14:15.8058307Z   inflating: build/bin/tutorial_tensorexpr  
2025-03-14T04:14:15.8061056Z   inflating: build/bin/example_allreduce  
2025-03-14T04:14:15.8130018Z   inflating: build/bin/ProcessGroupGlooTest  
2025-03-14T04:14:15.8187970Z   inflating: build/bin/test_dist_autograd  
2025-03-14T04:14:15.8258411Z   inflating: build/bin/test_cpp_rpc  
2025-03-14T04:14:15.8260868Z   inflating: build/bin/parallel_benchmark  
2025-03-14T04:14:15.9371571Z   inflating: build/bin/test_api      
2025-03-14T04:14:15.9440724Z   inflating: build/bin/test_mobile_nnc  
2025-03-14T04:14:15.9449493Z   inflating: build/bin/aot_model_compiler_test  
2025-03-14T04:14:15.9786073Z   inflating: build/bin/test_lazy     
2025-03-14T04:14:15.9790162Z   inflating: build/bin/torch_shm_manager  
2025-03-14T04:14:15.9790535Z    creating: .additional_ci_files/
2025-03-14T04:14:15.9891866Z   inflating: .additional_ci_files/test-times.json  
2025-03-14T04:14:16.0289076Z   inflating: .additional_ci_files/test-class-times.json  
2025-03-14T04:14:16.0317397Z ##[group]Run rm artifacts.zip
2025-03-14T04:14:16.0317739Z rm artifacts.zip
2025-03-14T04:14:16.0323939Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:14:16.0324364Z env:
2025-03-14T04:14:16.0324606Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:16.0324901Z ##[endgroup]
2025-03-14T04:14:16.3291094Z ##[group]Run df -H
2025-03-14T04:14:16.3291384Z df -H
2025-03-14T04:14:16.3298168Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:14:16.3298834Z env:
2025-03-14T04:14:16.3299079Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:16.3299371Z ##[endgroup]
2025-03-14T04:14:16.3573036Z Filesystem        Size  Used Avail Use% Mounted on
2025-03-14T04:14:16.3573475Z devtmpfs          4.2M     0  4.2M   0% /dev
2025-03-14T04:14:16.3573847Z tmpfs             8.2G   25k  8.2G   1% /dev/shm
2025-03-14T04:14:16.3574208Z tmpfs             3.3G  488k  3.3G   1% /run
2025-03-14T04:14:16.3574557Z /dev/nvme0n1p1    161G   42G  120G  26% /
2025-03-14T04:14:16.3574906Z tmpfs             8.2G   33k  8.2G   1% /tmp
2025-03-14T04:14:16.3575422Z /dev/nvme0n1p128   11M  1.4M  9.2M  13% /boot/efi
2025-03-14T04:14:16.3703322Z Prepare all required actions
2025-03-14T04:14:16.3704130Z Getting action download info
2025-03-14T04:14:16.4999002Z ##[group]Run ./.github/actions/download-td-artifacts
2025-03-14T04:14:16.4999655Z with:
2025-03-14T04:14:16.5000016Z env:
2025-03-14T04:14:16.5000414Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:16.5000895Z ##[endgroup]
2025-03-14T04:14:16.5121952Z ##[group]Run seemethere/download-artifact-s3@v4
2025-03-14T04:14:16.5122566Z with:
2025-03-14T04:14:16.5122932Z   name: td_results
2025-03-14T04:14:16.5123350Z   s3-bucket: gha-artifacts
2025-03-14T04:14:16.5123816Z   region: us-east-1
2025-03-14T04:14:16.5124223Z env:
2025-03-14T04:14:16.5124610Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:16.5125069Z ##[endgroup]
2025-03-14T04:14:16.9602436Z (node:626530) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.
2025-03-14T04:14:16.9602971Z 
2025-03-14T04:14:16.9603282Z Please migrate your code to use AWS SDK for JavaScript (v3).
2025-03-14T04:14:16.9603864Z For more information, check the migration guide at https://a.co/7PzMCcy
2025-03-14T04:14:16.9604471Z (Use `node --trace-warnings ...` to show where the warning was created)
2025-03-14T04:14:17.0685415Z Found 1 objects with prefix pytorch/pytorch/13849515290/td_results/
2025-03-14T04:14:17.0686124Z Starting download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json
2025-03-14T04:14:17.1325782Z Finished download (1/1): /home/ec2-user/actions-runner/_work/pytorch/pytorch/td_results.json
2025-03-14T04:14:17.1331405Z Artifact download has finished successfully
2025-03-14T04:14:17.1582375Z ##[group]Run mkdir -p .additional_ci_files
2025-03-14T04:14:17.1582783Z mkdir -p .additional_ci_files
2025-03-14T04:14:17.1583248Z mv td_results.json .additional_ci_files/td_results.json || true
2025-03-14T04:14:17.1590401Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:14:17.1590811Z env:
2025-03-14T04:14:17.1591061Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:17.1591352Z ##[endgroup]
2025-03-14T04:14:17.2094453Z ##[group]Run .github/scripts/parse_ref.py
2025-03-14T04:14:17.2094865Z .github/scripts/parse_ref.py
2025-03-14T04:14:17.2100600Z shell: /usr/bin/bash -e {0}
2025-03-14T04:14:17.2100887Z env:
2025-03-14T04:14:17.2101129Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:17.2101435Z ##[endgroup]
2025-03-14T04:14:17.2725712Z Prepare all required actions
2025-03-14T04:14:17.2726494Z Getting action download info
2025-03-14T04:14:17.3840601Z ##[group]Run ./.github/actions/filter-test-configs
2025-03-14T04:14:17.3840987Z with:
2025-03-14T04:14:17.3841424Z   github-token: ***
2025-03-14T04:14:17.3844434Z   test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "lf.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "lf.linux.2xlarge"}, {"config": "numpy_2_x", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}]}
2025-03-14T04:14:17.3847623Z   job-name: linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)
2025-03-14T04:14:17.3848109Z env:
2025-03-14T04:14:17.3848347Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:17.3848630Z ##[endgroup]
2025-03-14T04:14:17.3935292Z ##[group]Run nick-fields/retry@v3.0.0
2025-03-14T04:14:17.3935628Z with:
2025-03-14T04:14:17.3935861Z   shell: bash
2025-03-14T04:14:17.3936109Z   timeout_minutes: 10
2025-03-14T04:14:17.3936505Z   max_attempts: 5
2025-03-14T04:14:17.3936767Z   retry_wait_seconds: 30
2025-03-14T04:14:17.3937602Z   command: set -eux
# PyYAML 6.0 doesn't work with MacOS x86 anymore
# This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2
python3 -m pip install requests==2.27.1 pyyaml==6.0.1

2025-03-14T04:14:17.3938491Z   polling_interval_seconds: 1
2025-03-14T04:14:17.3938799Z   warning_on_retry: true
2025-03-14T04:14:17.3939081Z   continue_on_error: false
2025-03-14T04:14:17.3939356Z env:
2025-03-14T04:14:17.3939572Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:17.3940124Z   GITHUB_TOKEN: ***
2025-03-14T04:14:17.3940366Z ##[endgroup]
2025-03-14T04:14:17.4958311Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1
2025-03-14T04:14:17.7270584Z Defaulting to user installation because normal site-packages is not writeable
2025-03-14T04:14:17.7420992Z Requirement already satisfied: requests==2.27.1 in /home/ec2-user/.local/lib/python3.9/site-packages (2.27.1)
2025-03-14T04:14:17.7425300Z Requirement already satisfied: pyyaml==6.0.1 in /home/ec2-user/.local/lib/python3.9/site-packages (6.0.1)
2025-03-14T04:14:17.7538191Z Requirement already satisfied: charset-normalizer~=2.0.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2.0.12)
2025-03-14T04:14:17.7541613Z Requirement already satisfied: certifi>=2017.4.17 in /home/ec2-user/.local/lib/python3.9/site-packages (from requests==2.27.1) (2025.1.31)
2025-03-14T04:14:17.7550178Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (2.10)
2025-03-14T04:14:17.7555833Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests==2.27.1) (1.25.10)
2025-03-14T04:14:18.4691195Z Command completed after 1 attempt(s).
2025-03-14T04:14:18.4833234Z ##[group]Run set -x
2025-03-14T04:14:18.4833531Z set -x
2025-03-14T04:14:18.4833781Z 
2025-03-14T04:14:18.4834193Z # Use relative path here as this could be checked out anywhere, not necessarily
2025-03-14T04:14:18.4834730Z # in runner workspace
2025-03-14T04:14:18.4835147Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py"
2025-03-14T04:14:18.4841147Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:14:18.4841556Z env:
2025-03-14T04:14:18.4841957Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:18.4842251Z ##[endgroup]
2025-03-14T04:14:18.4866922Z + python3 /home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py
2025-03-14T04:14:18.5184681Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}"
2025-03-14T04:14:18.5185131Z echo "Workflow: ${GITHUB_WORKFLOW}"
2025-03-14T04:14:18.5185608Z echo "Job name: ${JOB_NAME}"
2025-03-14T04:14:18.5185935Z 
2025-03-14T04:14:18.5186347Z # Use relative path here as this could be checked out anywhere, not necessarily
2025-03-14T04:14:18.5186864Z # in runner workspace
2025-03-14T04:14:18.5187344Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \
2025-03-14T04:14:18.5187857Z   --workflow "${GITHUB_WORKFLOW}" \
2025-03-14T04:14:18.5188221Z   --job-name "${JOB_NAME}" \
2025-03-14T04:14:18.5191278Z   --test-matrix "{"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "lf.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "lf.linux.2xlarge"}, {"config": "numpy_2_x", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}]}" \
2025-03-14T04:14:18.5194571Z   --selected-test-configs "" \
2025-03-14T04:14:18.5194949Z   --pr-number "${PR_NUMBER}" \
2025-03-14T04:14:18.5195293Z   --tag "${TAG}" \
2025-03-14T04:14:18.5195615Z   --event-name "${EVENT_NAME}" \
2025-03-14T04:14:18.5195955Z   --schedule "${SCHEDULE}" \
2025-03-14T04:14:18.5196297Z   --branch "${HEAD_BRANCH}"
2025-03-14T04:14:18.5202134Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:14:18.5202540Z env:
2025-03-14T04:14:18.5202765Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:18.5203260Z   GITHUB_TOKEN: ***
2025-03-14T04:14:18.5203683Z   JOB_NAME: linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)
2025-03-14T04:14:18.5204153Z   PR_NUMBER: 
2025-03-14T04:14:18.5204409Z   TAG: 
2025-03-14T04:14:18.5204636Z   EVENT_NAME: push
2025-03-14T04:14:18.5204887Z   SCHEDULE: 
2025-03-14T04:14:18.5205122Z   HEAD_BRANCH: 
2025-03-14T04:14:18.5205365Z ##[endgroup]
2025-03-14T04:14:18.5226888Z Workflow: pull
2025-03-14T04:14:18.5227329Z Job name: linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)
2025-03-14T04:14:18.8479868Z ##[group]Run echo "Filtered matrix:"
2025-03-14T04:14:18.8480273Z echo "Filtered matrix:"
2025-03-14T04:14:18.8483452Z echo "{"include": [{"config": "default", "shard": 1, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 2, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 3, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 4, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "default", "shard": 5, "num_shards": 5, "runner": "lf.linux.2xlarge"}, {"config": "docs_test", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}, {"config": "jit_legacy", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}, {"config": "backwards_compat", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}, {"config": "distributed", "shard": 1, "num_shards": 2, "runner": "lf.linux.2xlarge"}, {"config": "distributed", "shard": 2, "num_shards": 2, "runner": "lf.linux.2xlarge"}, {"config": "numpy_2_x", "shard": 1, "num_shards": 1, "runner": "lf.linux.2xlarge"}]}"
2025-03-14T04:14:18.8486488Z 
2025-03-14T04:14:18.8486718Z echo
2025-03-14T04:14:18.8487011Z echo "Is the current job unstable? False"
2025-03-14T04:14:18.8487380Z 
2025-03-14T04:14:18.8487614Z echo
2025-03-14T04:14:18.8487902Z echo "Is keep-going label set? False"
2025-03-14T04:14:18.8488256Z 
2025-03-14T04:14:18.8488480Z echo
2025-03-14T04:14:18.8488736Z echo "Renabled issues? "
2025-03-14T04:14:18.8494382Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:14:18.8494806Z env:
2025-03-14T04:14:18.8495057Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:18.8495345Z ##[endgroup]
2025-03-14T04:14:18.8520968Z Filtered matrix:
2025-03-14T04:14:18.8526548Z {include: [{config: default, shard: 1, num_shards: 5, runner: lf.linux.2xlarge}, {config: default, shard: 2, num_shards: 5, runner: lf.linux.2xlarge}, {config: default, shard: 3, num_shards: 5, runner: lf.linux.2xlarge}, {config: default, shard: 4, num_shards: 5, runner: lf.linux.2xlarge}, {config: default, shard: 5, num_shards: 5, runner: lf.linux.2xlarge}, {config: docs_test, shard: 1, num_shards: 1, runner: lf.linux.2xlarge}, {config: jit_legacy, shard: 1, num_shards: 1, runner: lf.linux.2xlarge}, {config: backwards_compat, shard: 1, num_shards: 1, runner: lf.linux.2xlarge}, {config: distributed, shard: 1, num_shards: 2, runner: lf.linux.2xlarge}, {config: distributed, shard: 2, num_shards: 2, runner: lf.linux.2xlarge}, {config: numpy_2_x, shard: 1, num_shards: 1, runner: lf.linux.2xlarge}]}
2025-03-14T04:14:18.8532032Z 
2025-03-14T04:14:18.8532272Z Is the current job unstable? False
2025-03-14T04:14:18.8532705Z 
2025-03-14T04:14:18.8532908Z Is keep-going label set? False
2025-03-14T04:14:18.8533282Z 
2025-03-14T04:14:18.8533443Z Renabled issues? 
2025-03-14T04:14:18.8601307Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}"
2025-03-14T04:14:18.8601865Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}"
2025-03-14T04:14:18.8607139Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T04:14:18.8607552Z env:
2025-03-14T04:14:18.8607791Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:18.8608080Z   JOB_TIMEOUT: 240
2025-03-14T04:14:18.8608321Z ##[endgroup]
2025-03-14T04:14:18.8691581Z ##[group]Run set -x
2025-03-14T04:14:18.8692248Z set -x
2025-03-14T04:14:18.8692503Z 
2025-03-14T04:14:18.8692774Z if [[ $TEST_CONFIG == 'multigpu' ]]; then
2025-03-14T04:14:18.8693205Z   TEST_COMMAND=.ci/pytorch/multigpu-test.sh
2025-03-14T04:14:18.8693651Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then
2025-03-14T04:14:18.8694043Z   TEST_COMMAND=.ci/onnx/test.sh
2025-03-14T04:14:18.8694375Z else
2025-03-14T04:14:18.8694653Z   TEST_COMMAND=.ci/pytorch/test.sh
2025-03-14T04:14:18.8694990Z fi
2025-03-14T04:14:18.8695219Z 
2025-03-14T04:14:18.8695507Z # Leaving 1GB for the runner and other things
2025-03-14T04:14:18.8696121Z TOTAL_AVAILABLE_MEMORY_IN_GB=$(awk '/MemTotal/ { printf "%.3f \n", $2/1024/1024 - 1 }' /proc/meminfo)
2025-03-14T04:14:18.8697054Z # https://docs.docker.com/engine/containers/resource_constraints/#--memory-swap-details, the 3GB swap
2025-03-14T04:14:18.8697807Z # comes from https://github.com/pytorch/test-infra/pull/6058
2025-03-14T04:14:18.8698599Z TOTAL_MEMORY_WITH_SWAP=$(("${TOTAL_AVAILABLE_MEMORY_IN_GB%.*}" + 3))
2025-03-14T04:14:18.8699054Z 
2025-03-14T04:14:18.8699345Z if [[ ${BUILD_ENVIRONMENT} == *"s390x"* ]]; then
2025-03-14T04:14:18.8699728Z   SHM_OPTS=
2025-03-14T04:14:18.8700005Z   JENKINS_USER=
2025-03-14T04:14:18.8700383Z   # ensure that docker container cleanly exits in 12 hours
2025-03-14T04:14:18.8700883Z   # if for some reason cleanup action doesn't stop container
2025-03-14T04:14:18.8701314Z   # when job is cancelled
2025-03-14T04:14:18.8701652Z   DOCKER_SHELL_CMD="sleep 12h"
2025-03-14T04:14:18.8701975Z 
2025-03-14T04:14:18.8702378Z   # since some steps are skipped on s390x, if they are necessary, run them here
2025-03-14T04:14:18.8702962Z   env | grep '^GITHUB' >> "/tmp/github_env_${GITHUB_RUN_ID}"
2025-03-14T04:14:18.8703447Z   env | grep '^CI' >> "/tmp/github_env_${GITHUB_RUN_ID}"
2025-03-14T04:14:18.8703833Z else
2025-03-14T04:14:18.8704110Z   SHM_OPTS="--shm-size=${SHM_SIZE}"
2025-03-14T04:14:18.8704478Z   JENKINS_USER="--user jenkins"
2025-03-14T04:14:18.8704825Z   DOCKER_SHELL_CMD=
2025-03-14T04:14:18.8705112Z fi
2025-03-14T04:14:18.8705405Z 
2025-03-14T04:14:18.8705776Z # detached container should get cleaned up by teardown_ec2_linux
2025-03-14T04:14:18.8706348Z # TODO: Stop building test binaries as part of the build phase
2025-03-14T04:14:18.8707005Z # Used for GPU_FLAG, SHM_OPTS, JENKINS_USER and DOCKER_SHELL_CMD since that doesn't play nice
2025-03-14T04:14:18.8707588Z # shellcheck disable=SC2086,SC2090
2025-03-14T04:14:18.8707960Z container_name=$(docker run \
2025-03-14T04:14:18.8708304Z   ${GPU_FLAG:-} \
2025-03-14T04:14:18.8708644Z   ${SCCACHE_SERVER_PORT_DOCKER_FLAG:-} \
2025-03-14T04:14:18.8709023Z   -e BUILD_ENVIRONMENT \
2025-03-14T04:14:18.8709346Z   -e PR_NUMBER \
2025-03-14T04:14:18.8709648Z   -e GITHUB_ACTIONS \
2025-03-14T04:14:18.8709952Z   -e GITHUB_REPOSITORY \
2025-03-14T04:14:18.8710438Z   -e GITHUB_WORKFLOW \
2025-03-14T04:14:18.8710755Z   -e GITHUB_JOB \
2025-03-14T04:14:18.8711068Z   -e GITHUB_RUN_ID \
2025-03-14T04:14:18.8711379Z   -e GITHUB_RUN_NUMBER \
2025-03-14T04:14:18.8711709Z   -e GITHUB_RUN_ATTEMPT \
2025-03-14T04:14:18.8712031Z   -e JOB_ID \
2025-03-14T04:14:18.8712314Z   -e JOB_NAME \
2025-03-14T04:14:18.8712602Z   -e BASE_SHA \
2025-03-14T04:14:18.8712889Z   -e BRANCH \
2025-03-14T04:14:18.8713171Z   -e SHA1 \
2025-03-14T04:14:18.8713532Z   -e AWS_DEFAULT_REGION \
2025-03-14T04:14:18.8714027Z   -e IN_WHEEL_TEST \
2025-03-14T04:14:18.8714337Z   -e SHARD_NUMBER \
2025-03-14T04:14:18.8714649Z   -e TEST_CONFIG \
2025-03-14T04:14:18.8714960Z   -e NUM_TEST_SHARDS \
2025-03-14T04:14:18.8715284Z   -e REENABLED_ISSUES \
2025-03-14T04:14:18.8715620Z   -e CONTINUE_THROUGH_ERROR \
2025-03-14T04:14:18.8715975Z   -e VERBOSE_TEST_LOGS \
2025-03-14T04:14:18.8716305Z   -e TEST_SHOWLOCALS \
2025-03-14T04:14:18.8716611Z   -e NO_TEST_TIMEOUT \
2025-03-14T04:14:18.8716920Z   -e NO_TD \
2025-03-14T04:14:18.8717201Z   -e TD_DISTRIBUTED \
2025-03-14T04:14:18.8717511Z   -e PR_LABELS \
2025-03-14T04:14:18.8717832Z   -e MAX_JOBS="$(nproc --ignore=2)" \
2025-03-14T04:14:18.8718195Z   -e SCCACHE_BUCKET \
2025-03-14T04:14:18.8718502Z   -e SCCACHE_REGION \
2025-03-14T04:14:18.8718803Z   -e XLA_CUDA \
2025-03-14T04:14:18.8719118Z   -e XLA_CLANG_CACHE_S3_BUCKET_NAME \
2025-03-14T04:14:18.8719515Z   -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \
2025-03-14T04:14:18.8719912Z   -e PYTORCH_TEST_RERUN_DISABLED_TESTS \
2025-03-14T04:14:18.8720309Z   -e SKIP_SCCACHE_INITIALIZATION=1 \
2025-03-14T04:14:18.8720685Z   -e HUGGING_FACE_HUB_TOKEN \
2025-03-14T04:14:18.8721050Z   -e SCRIBE_GRAPHQL_ACCESS_TOKEN \
2025-03-14T04:14:18.8721406Z   -e DASHBOARD_TAG \
2025-03-14T04:14:18.8721710Z   -e IS_A100_RUNNER \
2025-03-14T04:14:18.8722033Z   -e ARTIFACTS_FILE_SUFFIX \
2025-03-14T04:14:18.8722430Z   --memory="${TOTAL_AVAILABLE_MEMORY_IN_GB%.*}g" \
2025-03-14T04:14:18.8722879Z   --memory-swap="${TOTAL_MEMORY_WITH_SWAP}g" \
2025-03-14T04:14:18.8723324Z   --env-file="/tmp/github_env_${GITHUB_RUN_ID}" \
2025-03-14T04:14:18.8723750Z   --security-opt seccomp=unconfined \
2025-03-14T04:14:18.8724107Z   --cap-add=SYS_PTRACE \
2025-03-14T04:14:18.8724433Z   --ipc=host \
2025-03-14T04:14:18.8724719Z   ${SHM_OPTS} \
2025-03-14T04:14:18.8725000Z   --tty \
2025-03-14T04:14:18.8725260Z   --detach \
2025-03-14T04:14:18.8725550Z   --name="${container_name}" \
2025-03-14T04:14:18.8725892Z   ${JENKINS_USER} \
2025-03-14T04:14:18.8726274Z   -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \
2025-03-14T04:14:18.8726705Z   -w /var/lib/jenkins/workspace \
2025-03-14T04:14:18.8727056Z   "${DOCKER_IMAGE}" \
2025-03-14T04:14:18.8727363Z   ${DOCKER_SHELL_CMD}
2025-03-14T04:14:18.8727653Z )
2025-03-14T04:14:18.8727970Z # Propagate download.pytorch.org IP to container
2025-03-14T04:14:18.8728680Z grep download.pytorch.org /etc/hosts | docker exec -i "${container_name}" sudo bash -c "/bin/cat >> /etc/hosts"
2025-03-14T04:14:18.8729437Z echo "DOCKER_CONTAINER_ID=${container_name}" >> "${GITHUB_ENV}"
2025-03-14T04:14:18.8729883Z 
2025-03-14T04:14:18.8730170Z if [[ ${BUILD_ENVIRONMENT} == *"s390x"* ]]; then
2025-03-14T04:14:18.8730785Z   docker exec -t "${container_name}" sh -c "python3 -m pip install -r .ci/docker/requirements-ci.txt"
2025-03-14T04:14:18.8731340Z fi
2025-03-14T04:14:18.8731654Z 
2025-03-14T04:14:18.8732183Z docker exec -t "${container_name}" sh -c "python3 -m pip install $(echo dist/*.whl)[opt-einsum] && ${TEST_COMMAND}"
2025-03-14T04:14:18.8737657Z shell: /usr/bin/bash -e {0}
2025-03-14T04:14:18.8737958Z env:
2025-03-14T04:14:18.8738199Z   GIT_DEFAULT_BRANCH: main
2025-03-14T04:14:18.8738532Z   BUILD_ENVIRONMENT: linux-jammy-py3.9-gcc11
2025-03-14T04:14:18.8738885Z   PR_NUMBER: 
2025-03-14T04:14:18.8739153Z   GITHUB_REPOSITORY: pytorch/pytorch
2025-03-14T04:14:18.8739473Z   GITHUB_WORKFLOW: pull
2025-03-14T04:14:18.8739747Z   GITHUB_JOB: test
2025-03-14T04:14:18.8740008Z   GITHUB_RUN_ID: 13849515290
2025-03-14T04:14:18.8740302Z   GITHUB_RUN_NUMBER: 299021
2025-03-14T04:14:18.8740688Z   GITHUB_RUN_ATTEMPT: 1
2025-03-14T04:14:18.8740961Z   JOB_ID: 38754724585
2025-03-14T04:14:18.8741381Z   JOB_NAME: linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)
2025-03-14T04:14:18.8741871Z   BRANCH: main
2025-03-14T04:14:18.8742155Z   SHA1: aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:14:18.8742567Z   BASE_SHA: aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:14:18.8742941Z   TEST_CONFIG: distributed
2025-03-14T04:14:18.8743228Z   SHARD_NUMBER: 2
2025-03-14T04:14:18.8743481Z   NUM_TEST_SHARDS: 2
2025-03-14T04:14:18.8743744Z   REENABLED_ISSUES: 
2025-03-14T04:14:18.8744021Z   CONTINUE_THROUGH_ERROR: False
2025-03-14T04:14:18.8744329Z   VERBOSE_TEST_LOGS: False
2025-03-14T04:14:18.8744622Z   TEST_SHOWLOCALS: False
2025-03-14T04:14:18.8744893Z   NO_TEST_TIMEOUT: False
2025-03-14T04:14:18.8745165Z   NO_TD: False
2025-03-14T04:14:18.8745499Z   TD_DISTRIBUTED: False
2025-03-14T04:14:18.8745839Z   SCCACHE_BUCKET: ossci-compiler-cache-circleci-v2
2025-03-14T04:14:18.8746241Z   SCCACHE_REGION: us-east-1
2025-03-14T04:14:18.8746534Z   SHM_SIZE: 1g
2025-03-14T04:14:18.8747219Z   DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:14:18.8747975Z   XLA_CUDA: 
2025-03-14T04:14:18.8748370Z   XLA_CLANG_CACHE_S3_BUCKET_NAME: ossci-compiler-clang-cache-circleci-xla
2025-03-14T04:14:18.8748865Z   PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0
2025-03-14T04:14:18.8749222Z   PYTORCH_TEST_RERUN_DISABLED_TESTS: 0
2025-03-14T04:14:18.8749558Z   DASHBOARD_TAG: 
2025-03-14T04:14:18.8750018Z   HUGGING_FACE_HUB_TOKEN: ***
2025-03-14T04:14:18.8750466Z   SCRIBE_GRAPHQL_ACCESS_TOKEN: ***
2025-03-14T04:14:18.8750789Z   IS_A100_RUNNER: 0
2025-03-14T04:14:18.8751197Z   ARTIFACTS_FILE_SUFFIX: test-distributed-2-2-lf.linux.2xlarge_38754724585
2025-03-14T04:14:18.8751671Z ##[endgroup]
2025-03-14T04:14:18.8788546Z + [[ distributed == \m\u\l\t\i\g\p\u ]]
2025-03-14T04:14:18.8789192Z + [[ linux-jammy-py3.9-gcc11 == *onnx* ]]
2025-03-14T04:14:18.8789553Z + TEST_COMMAND=.ci/pytorch/test.sh
2025-03-14T04:14:18.8792068Z ++ awk '/MemTotal/ { printf "%.3f \n", $2/1024/1024 - 1 }' /proc/meminfo
2025-03-14T04:14:18.8969284Z + TOTAL_AVAILABLE_MEMORY_IN_GB='14.244 '
2025-03-14T04:14:18.8969915Z + TOTAL_MEMORY_WITH_SWAP=17
2025-03-14T04:14:18.8970542Z + [[ linux-jammy-py3.9-gcc11 == *\s\3\9\0\x* ]]
2025-03-14T04:14:18.8970909Z + SHM_OPTS=--shm-size=1g
2025-03-14T04:14:18.8971208Z + JENKINS_USER='--user jenkins'
2025-03-14T04:14:18.8971516Z + DOCKER_SHELL_CMD=
2025-03-14T04:14:18.8977553Z +++ nproc --ignore=2
2025-03-14T04:14:18.9049516Z ++ docker run -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BASE_SHA -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e TD_DISTRIBUTED -e PR_LABELS -e MAX_JOBS=6 -e SCCACHE_BUCKET -e SCCACHE_REGION -e XLA_CUDA -e XLA_CLANG_CACHE_S3_BUCKET_NAME -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e SKIP_SCCACHE_INITIALIZATION=1 -e HUGGING_FACE_HUB_TOKEN -e SCRIBE_GRAPHQL_ACCESS_TOKEN -e DASHBOARD_TAG -e IS_A100_RUNNER -e ARTIFACTS_FILE_SUFFIX --memory=14g --memory-swap=17g --env-file=/tmp/github_env_13849515290 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=1g --tty --detach --name= --user jenkins -v /home/ec2-user/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T04:14:29.9833577Z + container_name=fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T04:14:29.9836404Z + grep download.pytorch.org /etc/hosts
2025-03-14T04:14:29.9837965Z + docker exec -i fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c sudo bash -c '/bin/cat >> /etc/hosts'
2025-03-14T04:14:30.2334488Z + echo DOCKER_CONTAINER_ID=fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T04:14:30.2335440Z + [[ linux-jammy-py3.9-gcc11 == *\s\3\9\0\x* ]]
2025-03-14T04:14:30.2337379Z ++ echo dist/torch-2.8.0a0+gitaed0b7a-cp39-cp39-linux_x86_64.whl
2025-03-14T04:14:30.2340448Z + docker exec -t fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c sh -c 'python3 -m pip install dist/torch-2.8.0a0+gitaed0b7a-cp39-cp39-linux_x86_64.whl[opt-einsum] && .ci/pytorch/test.sh'
2025-03-14T04:14:31.1206863Z Processing ./dist/torch-2.8.0a0+gitaed0b7a-cp39-cp39-linux_x86_64.whl (from torch==2.8.0a0+gitaed0b7a)
2025-03-14T04:14:31.5800356Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.9/lib/python3.9/site-packages (from torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (3.16.1)
2025-03-14T04:14:31.5804912Z Requirement already satisfied: typing-extensions>=4.10.0 in /opt/conda/envs/py_3.9/lib/python3.9/site-packages (from torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (4.12.2)
2025-03-14T04:14:31.5810280Z Requirement already satisfied: sympy>=1.13.3 in /opt/conda/envs/py_3.9/lib/python3.9/site-packages (from torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (1.13.3)
2025-03-14T04:14:31.5813246Z Requirement already satisfied: networkx in /opt/conda/envs/py_3.9/lib/python3.9/site-packages (from torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (2.8.8)
2025-03-14T04:14:31.5816750Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.9/lib/python3.9/site-packages (from torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (3.1.6)
2025-03-14T04:14:31.5820382Z Requirement already satisfied: fsspec in /opt/conda/envs/py_3.9/lib/python3.9/site-packages (from torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (2024.10.0)
2025-03-14T04:14:31.5838876Z Requirement already satisfied: opt-einsum>=3.3 in /opt/conda/envs/py_3.9/lib/python3.9/site-packages (from torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (3.3.0)
2025-03-14T04:14:31.5863674Z Requirement already satisfied: numpy>=1.7 in /opt/conda/envs/py_3.9/lib/python3.9/site-packages (from opt-einsum>=3.3->torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (1.22.4)
2025-03-14T04:14:31.5886363Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.9/lib/python3.9/site-packages (from sympy>=1.13.3->torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (1.3.0)
2025-03-14T04:14:31.6278220Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.9/lib/python3.9/site-packages (from jinja2->torch==2.8.0a0+gitaed0b7a->torch==2.8.0a0+gitaed0b7a) (3.0.2)
2025-03-14T04:14:32.2091795Z Installing collected packages: torch
2025-03-14T04:14:40.1730492Z Successfully installed torch-2.8.0a0+gitaed0b7a
2025-03-14T04:14:40.2428304Z + export TERM=vt100
2025-03-14T04:14:40.2428787Z + TERM=vt100
2025-03-14T04:14:40.2431410Z ++ dirname .ci/pytorch/test.sh
2025-03-14T04:14:40.2447190Z + source .ci/pytorch/common.sh
2025-03-14T04:14:40.2454241Z +++ dirname .ci/pytorch/common.sh
2025-03-14T04:14:40.2461099Z ++ source .ci/pytorch/common_utils.sh
2025-03-14T04:14:40.2466781Z +++ declare -f -t trap_add
2025-03-14T04:14:40.2470528Z ++ set -ex -o pipefail
2025-03-14T04:14:40.2471308Z ++ [[ linux-jammy-py3.9-gcc11 == *rocm* ]]
2025-03-14T04:14:40.2471781Z ++ BUILD_TEST_LIBTORCH=0
2025-03-14T04:14:40.2472211Z + [[ linux-jammy-py3.9-gcc11 != *rocm* ]]
2025-03-14T04:14:40.2472711Z + [[ linux-jammy-py3.9-gcc11 != *s390x* ]]
2025-03-14T04:14:40.2473413Z + [[ -d /var/lib/jenkins/workspace ]]
2025-03-14T04:14:40.2474741Z ++ stat -c %u /var/lib/jenkins/workspace
2025-03-14T04:14:40.2495977Z + WORKSPACE_ORIGINAL_OWNER_ID=1000
2025-03-14T04:14:40.2496656Z + trap_add cleanup_workspace EXIT
2025-03-14T04:14:40.2497020Z + trap_add_cmd=cleanup_workspace
2025-03-14T04:14:40.2497311Z + shift
2025-03-14T04:14:40.2497558Z + for trap_add_name in "$@"
2025-03-14T04:14:40.2502884Z +++ trap -p EXIT
2025-03-14T04:14:40.2504780Z ++ eval 'extract_trap_cmd '
2025-03-14T04:14:40.2505197Z +++ extract_trap_cmd
2025-03-14T04:14:40.2505538Z +++ printf '%s\n' ''
2025-03-14T04:14:40.2505830Z ++ printf '%s\n' cleanup_workspace
2025-03-14T04:14:40.2508796Z + trap -- '
2025-03-14T04:14:40.2509101Z cleanup_workspace' EXIT
2025-03-14T04:14:40.2509445Z + sudo chown -R jenkins /var/lib/jenkins/workspace
2025-03-14T04:14:40.7642808Z + git config --global --add safe.directory /var/lib/jenkins/workspace
2025-03-14T04:14:40.7809868Z + echo 'Environment variables:'
2025-03-14T04:14:40.7810284Z Environment variables:
2025-03-14T04:14:40.7810566Z + env
2025-03-14T04:14:40.7828418Z INSTALLED_DB=yes
2025-03-14T04:14:40.7829096Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch
2025-03-14T04:14:40.7829788Z CONTINUE_THROUGH_ERROR=False
2025-03-14T04:14:40.7830263Z BUILD_ENVIRONMENT=linux-jammy-py3.9-gcc11
2025-03-14T04:14:40.7830798Z HOSTNAME=fc1c1f3bb76f
2025-03-14T04:14:40.7831759Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_02fe6574-e14d-4f4c-b7b4-62e2bef615fd
2025-03-14T04:14:40.7832438Z GITHUB_ACTION=__self
2025-03-14T04:14:40.7832733Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0
2025-03-14T04:14:40.7833183Z GITHUB_RUN_NUMBER=299021
2025-03-14T04:14:40.7833593Z TEST_CONFIG=distributed
2025-03-14T04:14:40.7834037Z GITHUB_REPOSITORY_OWNER_ID=21003710
2025-03-14T04:14:40.7834674Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all
2025-03-14T04:14:40.7835018Z IS_A100_RUNNER=0
2025-03-14T04:14:40.7835538Z SCRIBE_GRAPHQL_ACCESS_TOKEN=***
2025-03-14T04:14:40.7835878Z GITHUB_TRIGGERING_ACTOR=pytorchmergebot
2025-03-14T04:14:40.7836227Z GITHUB_REF_TYPE=branch
2025-03-14T04:14:40.7836515Z TORCH_CUDA_ARCH_LIST=Maxwell
2025-03-14T04:14:40.7836856Z BASE_SHA=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:14:40.7837214Z XLA_CUDA=
2025-03-14T04:14:40.7837562Z HUGGING_FACE_HUB_TOKEN=***
2025-03-14T04:14:40.7839259Z ***
2025-03-14T04:14:40.7839507Z GITHUB_REPOSITORY_ID=65600975
2025-03-14T04:14:40.7839836Z GITHUB_ACTIONS=true
2025-03-14T04:14:40.7840170Z SHA1=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:14:40.7840569Z GITHUB_SHA=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:14:40.7841110Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/heads/main
2025-03-14T04:14:40.7841622Z UCC_HOME=/usr
2025-03-14T04:14:40.7841878Z VERBOSE_TEST_LOGS=False
2025-03-14T04:14:40.7842164Z GITHUB_REF=refs/heads/main
2025-03-14T04:14:40.7842450Z SHARD_NUMBER=2
2025-03-14T04:14:40.7842710Z GITHUB_REF_PROTECTED=true
2025-03-14T04:14:40.7843002Z HOME=/var/lib/jenkins
2025-03-14T04:14:40.7843303Z GITHUB_API_URL=https://api.github.com
2025-03-14T04:14:40.7843660Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0
2025-03-14T04:14:40.7843961Z UCX_COMMIT=
2025-03-14T04:14:40.7844194Z NUM_TEST_SHARDS=2
2025-03-14T04:14:40.7844444Z UCX_HOME=/usr
2025-03-14T04:14:40.7845058Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_02fe6574-e14d-4f4c-b7b4-62e2bef615fd
2025-03-14T04:14:40.7845889Z JOB_NAME=linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)
2025-03-14T04:14:40.7846700Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_02fe6574-e14d-4f4c-b7b4-62e2bef615fd
2025-03-14T04:14:40.7847555Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json
2025-03-14T04:14:40.7848274Z GITHUB_EVENT_NAME=push
2025-03-14T04:14:40.7848555Z DASHBOARD_TAG=
2025-03-14T04:14:40.7848815Z GITHUB_RUN_ID=13849515290
2025-03-14T04:14:40.7849493Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_02fe6574-e14d-4f4c-b7b4-62e2bef615fd
2025-03-14T04:14:40.7850229Z GITHUB_ACTOR=pytorchmergebot
2025-03-14T04:14:40.7850528Z PR_NUMBER=
2025-03-14T04:14:40.7850763Z DESIRED_CUDA=
2025-03-14T04:14:40.7851007Z GITHUB_RUN_ATTEMPT=1
2025-03-14T04:14:40.7851288Z ANACONDA_PYTHON_VERSION=3.9
2025-03-14T04:14:40.7851647Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql
2025-03-14T04:14:40.7852124Z TERM=vt100
2025-03-14T04:14:40.7852355Z INSTALLED_VISION=yes
2025-03-14T04:14:40.7852620Z BRANCH=main
2025-03-14T04:14:40.7852872Z SCCACHE_REGION=us-east-1
2025-03-14T04:14:40.7853171Z OPENSSL_ROOT_DIR=/opt/openssl
2025-03-14T04:14:40.7853485Z CUDA_PATH=/usr/local/cuda
2025-03-14T04:14:40.7854048Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux
2025-03-14T04:14:40.7854665Z GITHUB_SERVER_URL=https://github.com
2025-03-14T04:14:40.7854992Z UCC_COMMIT=
2025-03-14T04:14:40.7855229Z REENABLED_ISSUES=
2025-03-14T04:14:40.7855475Z DOCS=yes
2025-03-14T04:14:40.7855693Z SHLVL=1
2025-03-14T04:14:40.7855914Z MAX_JOBS=6
2025-03-14T04:14:40.7856151Z GITHUB_ACTOR_ID=97764156
2025-03-14T04:14:40.7856519Z GITHUB_WORKFLOW_SHA=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:14:40.7856931Z GITHUB_REF_NAME=main
2025-03-14T04:14:40.7857324Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla
2025-03-14T04:14:40.7857785Z GITHUB_JOB=test
2025-03-14T04:14:40.7858038Z NO_TEST_TIMEOUT=False
2025-03-14T04:14:40.7858309Z TD_DISTRIBUTED=False
2025-03-14T04:14:40.7858602Z GITHUB_REPOSITORY=pytorch/pytorch
2025-03-14T04:14:40.7858937Z GITHUB_RETENTION_DAYS=90
2025-03-14T04:14:40.7859230Z OPENSSL_DIR=/opt/openssl
2025-03-14T04:14:40.7859528Z GITHUB_ACTION_REPOSITORY=
2025-03-14T04:14:40.7860345Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.9/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2025-03-14T04:14:40.7861188Z GITHUB_BASE_REF=
2025-03-14T04:14:40.7861441Z INSTALLED_ACL=
2025-03-14T04:14:40.7861843Z ARTIFACTS_FILE_SUFFIX=test-distributed-2-2-lf.linux.2xlarge_38754724585
2025-03-14T04:14:40.7862306Z CI=true
2025-03-14T04:14:40.7862552Z GITHUB_REPOSITORY_OWNER=pytorch
2025-03-14T04:14:40.7862857Z JOB_ID=38754724585
2025-03-14T04:14:40.7863121Z INSTALLED_PROTOBUF=yes
2025-03-14T04:14:40.7863382Z GITHUB_HEAD_REF=
2025-03-14T04:14:40.7863642Z GITHUB_ACTION_REF=
2025-03-14T04:14:40.7863966Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2
2025-03-14T04:14:40.7864348Z TEST_SHOWLOCALS=False
2025-03-14T04:14:40.7864626Z GITHUB_WORKFLOW=pull
2025-03-14T04:14:40.7864916Z DEBIAN_FRONTEND=noninteractive
2025-03-14T04:14:40.7865672Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_02fe6574-e14d-4f4c-b7b4-62e2bef615fd
2025-03-14T04:14:40.7866369Z NO_TD=False
2025-03-14T04:14:40.7866630Z SKIP_SCCACHE_INITIALIZATION=1
2025-03-14T04:14:40.7866935Z _=/usr/bin/env
2025-03-14T04:14:40.7867285Z ++ python -c 'import site; print(site.getsitepackages()[0])'
2025-03-14T04:14:40.8090007Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.9/lib/python3.9/site-packages/torch
2025-03-14T04:14:40.8091055Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.9/lib/python3.9/site-packages/torch/bin
2025-03-14T04:14:40.8092167Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.9/lib/python3.9/site-packages/torch/lib
2025-03-14T04:14:40.8092817Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.9/lib/python3.9/site-packages/torch/test
2025-03-14T04:14:40.8093304Z + BUILD_DIR=build
2025-03-14T04:14:40.8093580Z + BUILD_RENAMED_DIR=build_renamed
2025-03-14T04:14:40.8093911Z + BUILD_BIN_DIR=build/bin
2025-03-14T04:14:40.8094200Z + SHARD_NUMBER=2
2025-03-14T04:14:40.8094461Z + NUM_TEST_SHARDS=2
2025-03-14T04:14:40.8095011Z + export TORCH_SERIALIZATION_DEBUG=1
2025-03-14T04:14:40.8095409Z + TORCH_SERIALIZATION_DEBUG=1
2025-03-14T04:14:40.8095739Z + export VALGRIND=ON
2025-03-14T04:14:40.8096071Z + VALGRIND=ON
2025-03-14T04:14:40.8096358Z + [[ linux-jammy-py3.9-gcc11 == *clang9* ]]
2025-03-14T04:14:40.8096722Z + [[ linux-jammy-py3.9-gcc11 == *xpu* ]]
2025-03-14T04:14:40.8097078Z + [[ linux-jammy-py3.9-gcc11 == *s390x* ]]
2025-03-14T04:14:40.8097406Z + [[ 0 == \1 ]]
2025-03-14T04:14:40.8097647Z + [[ False == \1 ]]
2025-03-14T04:14:40.8097925Z + [[ linux-jammy-py3.9-gcc11 != *bazel* ]]
2025-03-14T04:14:40.8098446Z ++ realpath build/custom_test_artifacts
2025-03-14T04:14:40.8114474Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/workspace/build/custom_test_artifacts
2025-03-14T04:14:40.8115053Z + [[ -n '' ]]
2025-03-14T04:14:40.8115330Z + echo 'Environment variables'
2025-03-14T04:14:40.8115648Z Environment variables
2025-03-14T04:14:40.8115910Z + env
2025-03-14T04:14:40.8121186Z INSTALLED_DB=yes
2025-03-14T04:14:40.8121930Z GITHUB_WORKSPACE=/home/ec2-user/actions-runner/_work/pytorch/pytorch
2025-03-14T04:14:40.8122630Z CONTINUE_THROUGH_ERROR=False
2025-03-14T04:14:40.8123124Z BUILD_ENVIRONMENT=linux-jammy-py3.9-gcc11
2025-03-14T04:14:40.8123532Z HOSTNAME=fc1c1f3bb76f
2025-03-14T04:14:40.8124219Z GITHUB_PATH=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/add_path_02fe6574-e14d-4f4c-b7b4-62e2bef615fd
2025-03-14T04:14:40.8125036Z GITHUB_ACTION=__self
2025-03-14T04:14:40.8125344Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0
2025-03-14T04:14:40.8125717Z GITHUB_RUN_NUMBER=299021
2025-03-14T04:14:40.8126009Z TEST_CONFIG=distributed
2025-03-14T04:14:40.8126357Z GITHUB_REPOSITORY_OWNER_ID=21003710
2025-03-14T04:14:40.8126727Z TORCH_NVCC_FLAGS=-Xfatbin -compress-all
2025-03-14T04:14:40.8127072Z IS_A100_RUNNER=0
2025-03-14T04:14:40.8127599Z SCRIBE_GRAPHQL_ACCESS_TOKEN=***
2025-03-14T04:14:40.8127941Z GITHUB_TRIGGERING_ACTOR=pytorchmergebot
2025-03-14T04:14:40.8128291Z GITHUB_REF_TYPE=branch
2025-03-14T04:14:40.8128628Z TORCH_CUDA_ARCH_LIST=Maxwell
2025-03-14T04:14:40.8128984Z BASE_SHA=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:14:40.8129376Z XLA_CUDA=
2025-03-14T04:14:40.8129744Z HUGGING_FACE_HUB_TOKEN=***
2025-03-14T04:14:40.8130170Z ***
2025-03-14T04:14:40.8130411Z GITHUB_REPOSITORY_ID=65600975
2025-03-14T04:14:40.8130773Z GITHUB_ACTIONS=true
2025-03-14T04:14:40.8131072Z SHA1=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:14:40.8131476Z GITHUB_SHA=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:14:40.8132020Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/pull.yml@refs/heads/main
2025-03-14T04:14:40.8132517Z UCC_HOME=/usr
2025-03-14T04:14:40.8132773Z TORCH_SERIALIZATION_DEBUG=1
2025-03-14T04:14:40.8133081Z VERBOSE_TEST_LOGS=False
2025-03-14T04:14:40.8133350Z GITHUB_REF=refs/heads/main
2025-03-14T04:14:40.8133634Z SHARD_NUMBER=2
2025-03-14T04:14:40.8133890Z GITHUB_REF_PROTECTED=true
2025-03-14T04:14:40.8134177Z HOME=/var/lib/jenkins
2025-03-14T04:14:40.8134527Z GITHUB_API_URL=https://api.github.com
2025-03-14T04:14:40.8134892Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0
2025-03-14T04:14:40.8135205Z UCX_COMMIT=
2025-03-14T04:14:40.8135442Z NUM_TEST_SHARDS=2
2025-03-14T04:14:40.8135690Z UCX_HOME=/usr
2025-03-14T04:14:40.8136289Z GITHUB_STATE=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/save_state_02fe6574-e14d-4f4c-b7b4-62e2bef615fd
2025-03-14T04:14:40.8137126Z JOB_NAME=linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)
2025-03-14T04:14:40.8137936Z GITHUB_ENV=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_env_02fe6574-e14d-4f4c-b7b4-62e2bef615fd
2025-03-14T04:14:40.8138798Z GITHUB_EVENT_PATH=/home/ec2-user/actions-runner/_work/_temp/_github_workflow/event.json
2025-03-14T04:14:40.8139336Z GITHUB_EVENT_NAME=push
2025-03-14T04:14:40.8139609Z DASHBOARD_TAG=
2025-03-14T04:14:40.8139863Z GITHUB_RUN_ID=13849515290
2025-03-14T04:14:40.8140531Z GITHUB_STEP_SUMMARY=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/step_summary_02fe6574-e14d-4f4c-b7b4-62e2bef615fd
2025-03-14T04:14:40.8141554Z GITHUB_ACTOR=pytorchmergebot
2025-03-14T04:14:40.8141842Z PR_NUMBER=
2025-03-14T04:14:40.8142078Z DESIRED_CUDA=
2025-03-14T04:14:40.8142329Z GITHUB_RUN_ATTEMPT=1
2025-03-14T04:14:40.8142612Z VALGRIND=ON
2025-03-14T04:14:40.8142865Z ANACONDA_PYTHON_VERSION=3.9
2025-03-14T04:14:40.8143230Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql
2025-03-14T04:14:40.8143610Z TERM=vt100
2025-03-14T04:14:40.8143852Z INSTALLED_VISION=yes
2025-03-14T04:14:40.8144123Z BRANCH=main
2025-03-14T04:14:40.8144371Z SCCACHE_REGION=us-east-1
2025-03-14T04:14:40.8144673Z OPENSSL_ROOT_DIR=/opt/openssl
2025-03-14T04:14:40.8144985Z CUDA_PATH=/usr/local/cuda
2025-03-14T04:14:40.8145698Z GITHUB_ACTION_PATH=/home/ec2-user/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-linux
2025-03-14T04:14:40.8146325Z GITHUB_SERVER_URL=https://github.com
2025-03-14T04:14:40.8146656Z UCC_COMMIT=
2025-03-14T04:14:40.8146885Z REENABLED_ISSUES=
2025-03-14T04:14:40.8147145Z DOCS=yes
2025-03-14T04:14:40.8147377Z SHLVL=1
2025-03-14T04:14:40.8147609Z MAX_JOBS=6
2025-03-14T04:14:40.8147857Z GITHUB_ACTOR_ID=97764156
2025-03-14T04:14:40.8148229Z GITHUB_WORKFLOW_SHA=aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T04:14:40.8148640Z GITHUB_REF_NAME=main
2025-03-14T04:14:40.8149045Z XLA_CLANG_CACHE_S3_BUCKET_NAME=ossci-compiler-clang-cache-circleci-xla
2025-03-14T04:14:40.8149501Z GITHUB_JOB=test
2025-03-14T04:14:40.8149750Z NO_TEST_TIMEOUT=False
2025-03-14T04:14:40.8150023Z TD_DISTRIBUTED=False
2025-03-14T04:14:40.8150309Z GITHUB_REPOSITORY=pytorch/pytorch
2025-03-14T04:14:40.8150636Z GITHUB_RETENTION_DAYS=90
2025-03-14T04:14:40.8150924Z OPENSSL_DIR=/opt/openssl
2025-03-14T04:14:40.8151213Z GITHUB_ACTION_REPOSITORY=
2025-03-14T04:14:40.8152024Z PATH=/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.9/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2025-03-14T04:14:40.8152865Z GITHUB_BASE_REF=
2025-03-14T04:14:40.8153116Z INSTALLED_ACL=
2025-03-14T04:14:40.8153515Z ARTIFACTS_FILE_SUFFIX=test-distributed-2-2-lf.linux.2xlarge_38754724585
2025-03-14T04:14:40.8153984Z CI=true
2025-03-14T04:14:40.8154236Z GITHUB_REPOSITORY_OWNER=pytorch
2025-03-14T04:14:40.8154542Z JOB_ID=38754724585
2025-03-14T04:14:40.8154868Z INSTALLED_PROTOBUF=yes
2025-03-14T04:14:40.8155144Z GITHUB_HEAD_REF=
2025-03-14T04:14:40.8155395Z GITHUB_ACTION_REF=
2025-03-14T04:14:40.8155715Z SCCACHE_BUCKET=ossci-compiler-cache-circleci-v2
2025-03-14T04:14:40.8156091Z TEST_SHOWLOCALS=False
2025-03-14T04:14:40.8156367Z GITHUB_WORKFLOW=pull
2025-03-14T04:14:40.8156654Z DEBIAN_FRONTEND=noninteractive
2025-03-14T04:14:40.8157321Z GITHUB_OUTPUT=/home/ec2-user/actions-runner/_work/_temp/_runner_file_commands/set_output_02fe6574-e14d-4f4c-b7b4-62e2bef615fd
2025-03-14T04:14:40.8158000Z NO_TD=False
2025-03-14T04:14:40.8158255Z SKIP_SCCACHE_INITIALIZATION=1
2025-03-14T04:14:40.8158545Z _=/usr/bin/env
2025-03-14T04:14:40.8158799Z + echo 'Testing pytorch'
2025-03-14T04:14:40.8159081Z Testing pytorch
2025-03-14T04:14:40.8159352Z + export LANG=C.UTF-8
2025-03-14T04:14:40.8159619Z + LANG=C.UTF-8
2025-03-14T04:14:40.8167610Z + PR_NUMBER=
2025-03-14T04:14:40.8168033Z + [[ distributed == \d\e\f\a\u\l\t ]]
2025-03-14T04:14:40.8168578Z + [[ distributed == \d\i\s\t\r\i\b\u\t\e\d ]]
2025-03-14T04:14:40.8169043Z + [[ linux-jammy-py3.9-gcc11 == *rocm* ]]
2025-03-14T04:14:40.8169393Z + [[ distributed == \s\l\o\w ]]
2025-03-14T04:14:40.8169770Z + [[ linux-jammy-py3.9-gcc11 == *slow-gradcheck* ]]
2025-03-14T04:14:40.8170353Z + [[ linux-jammy-py3.9-gcc11 == *cuda* ]]
2025-03-14T04:14:40.8170832Z + [[ linux-jammy-py3.9-gcc11 == *rocm* ]]
2025-03-14T04:14:40.8171343Z + [[ linux-jammy-py3.9-gcc11 == *xpu* ]]
2025-03-14T04:14:40.8171704Z + [[ distributed == *crossref* ]]
2025-03-14T04:14:40.8172039Z + [[ linux-jammy-py3.9-gcc11 == *rocm* ]]
2025-03-14T04:14:40.8172394Z + [[ linux-jammy-py3.9-gcc11 == *xpu* ]]
2025-03-14T04:14:40.8172753Z + [[ linux-jammy-py3.9-gcc11 != *-bazel-* ]]
2025-03-14T04:14:40.8173118Z + pip_install --user ninja==1.10.2
2025-03-14T04:14:40.8173658Z + pip_install_pkg='python3 -m pip install --progress-bar off'
2025-03-14T04:14:40.8174186Z + python3 -m pip install --progress-bar off --user ninja==1.10.2
2025-03-14T04:14:41.2675612Z Collecting ninja==1.10.2
2025-03-14T04:14:41.2771235Z   Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB)
2025-03-14T04:14:41.2889347Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB)
2025-03-14T04:14:41.7596767Z Installing collected packages: ninja
2025-03-14T04:14:41.7684968Z   WARNING: The script ninja is installed in '/var/lib/jenkins/.local/bin' which is not on PATH.
2025-03-14T04:14:41.7686232Z   Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
2025-03-14T04:14:41.7994898Z Successfully installed ninja-1.10.2
2025-03-14T04:14:41.8646065Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.9/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2025-03-14T04:14:41.8648136Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.9/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2025-03-14T04:14:41.8649141Z + [[ linux-jammy-py3.9-gcc11 == *aarch64* ]]
2025-03-14T04:14:41.8649496Z + install_tlparse
2025-03-14T04:14:41.8649796Z + pip_install --user tlparse==0.3.30
2025-03-14T04:14:41.8650224Z + pip_install_pkg='python3 -m pip install --progress-bar off'
2025-03-14T04:14:41.8650752Z + python3 -m pip install --progress-bar off --user tlparse==0.3.30
2025-03-14T04:14:42.2931721Z Collecting tlparse==0.3.30
2025-03-14T04:14:42.3009297Z   Downloading tlparse-0.3.30-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.9 kB)
2025-03-14T04:14:42.3334122Z Downloading tlparse-0.3.30-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.3 MB)
2025-03-14T04:14:42.8554838Z Installing collected packages: tlparse
2025-03-14T04:14:42.9477867Z Successfully installed tlparse-0.3.30
2025-03-14T04:14:43.0174750Z ++ python -m site --user-base
2025-03-14T04:14:43.0464733Z + PATH=/var/lib/jenkins/.local/bin:/var/lib/jenkins/.local/bin:/opt/cache/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/opt/conda/envs/py_3.9/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2025-03-14T04:14:43.0466131Z + [[ linux-jammy-py3.9-gcc11 == *asan* ]]
2025-03-14T04:14:43.0466507Z + [[ linux-jammy-py3.9-gcc11 == *-debug* ]]
2025-03-14T04:14:43.0466885Z + [[ linux-jammy-py3.9-gcc11 != *-bazel-* ]]
2025-03-14T04:14:43.0467438Z + echo 'We are not in debug mode: linux-jammy-py3.9-gcc11. Expect the assertion to pass'
2025-03-14T04:14:43.0468135Z We are not in debug mode: linux-jammy-py3.9-gcc11. Expect the assertion to pass
2025-03-14T04:14:43.0468863Z + cd test
2025-03-14T04:14:43.0469944Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)'
2025-03-14T04:14:45.8947501Z + [[ distributed == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]]
2025-03-14T04:14:45.8947955Z + [[ distributed == \n\o\g\p\u\_\A\V\X\5\1\2 ]]
2025-03-14T04:14:45.8951113Z + DYNAMO_BENCHMARK_FLAGS=()
2025-03-14T04:14:45.8951563Z + [[ distributed == *pr_time_benchmarks* ]]
2025-03-14T04:14:45.8951929Z + [[ distributed == *dynamo_eager* ]]
2025-03-14T04:14:45.8952266Z + [[ distributed == *aot_eager* ]]
2025-03-14T04:14:45.8952584Z + [[ distributed == *aot_inductor* ]]
2025-03-14T04:14:45.8952938Z + [[ distributed == *max_autotune_inductor* ]]
2025-03-14T04:14:45.8953304Z + [[ distributed == *inductor* ]]
2025-03-14T04:14:45.8953638Z + [[ distributed == *dynamic* ]]
2025-03-14T04:14:45.8953980Z + [[ distributed == *cpu* ]]
2025-03-14T04:14:45.8954318Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda)
2025-03-14T04:14:45.8972879Z + [[ linux-jammy-py3.9-gcc11 == *libtorch* ]]
2025-03-14T04:14:45.8973282Z + [[ linux-jammy-py3.9-gcc11 == *-bazel-* ]]
2025-03-14T04:14:45.8975848Z + cd test
2025-03-14T04:14:45.8976591Z + python -c 'import torch; print(torch.__config__.show())'
2025-03-14T04:14:47.1163592Z PyTorch built with:
2025-03-14T04:14:47.1164054Z   - GCC 11.4
2025-03-14T04:14:47.1164418Z   - C++ Version: 201703
2025-03-14T04:14:47.1165335Z   - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
2025-03-14T04:14:47.1166505Z   - Intel(R) MKL-DNN v3.7.1 (Git Hash 8d263e693366ef8db40acc569cc7d8edf644556d)
2025-03-14T04:14:47.1167207Z   - OpenMP 201511 (a.k.a. OpenMP 4.5)
2025-03-14T04:14:47.1167750Z   - LAPACK is enabled (usually provided by MKL)
2025-03-14T04:14:47.1168286Z   - NNPACK is enabled
2025-03-14T04:14:47.1169042Z   - CPU capability usage: AVX512
2025-03-14T04:14:47.1176953Z   - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, COMMIT_SHA=aed0b7a742a2d7b7901790622829cbd2135049a4, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=1 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Werror -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, TORCH_VERSION=2.8.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF, 
2025-03-14T04:14:47.1187459Z 
2025-03-14T04:14:47.3325133Z + cd test
2025-03-14T04:14:47.3325661Z + python -c 'import torch; print(torch.__config__.parallel_info())'
2025-03-14T04:14:48.5560055Z ATen/Parallel:
2025-03-14T04:14:48.5560550Z 	at::get_num_threads() : 4
2025-03-14T04:14:48.5561048Z 	at::get_num_interop_threads() : 4
2025-03-14T04:14:48.5561553Z OpenMP 201511 (a.k.a. OpenMP 4.5)
2025-03-14T04:14:48.5562030Z 	omp_get_max_threads() : 4
2025-03-14T04:14:48.5562943Z Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
2025-03-14T04:14:48.5563937Z 	mkl_get_max_threads() : 4
2025-03-14T04:14:48.5564561Z Intel(R) MKL-DNN v3.7.1 (Git Hash 8d263e693366ef8db40acc569cc7d8edf644556d)
2025-03-14T04:14:48.5565287Z std::thread::hardware_concurrency() : 8
2025-03-14T04:14:48.5565835Z Environment variables:
2025-03-14T04:14:48.5566254Z 	OMP_NUM_THREADS : [not set]
2025-03-14T04:14:48.5566699Z 	MKL_NUM_THREADS : [not set]
2025-03-14T04:14:48.5567157Z ATen parallel backend: OpenMP
2025-03-14T04:14:48.5567457Z 
2025-03-14T04:14:48.7668721Z + [[ distributed == *numpy_2* ]]
2025-03-14T04:14:48.7669191Z + [[ linux-jammy-py3.9-gcc11 == *aarch64* ]]
2025-03-14T04:14:48.7669559Z + [[ distributed == *backward* ]]
2025-03-14T04:14:48.7669887Z + [[ distributed == *xla* ]]
2025-03-14T04:14:48.7670197Z + [[ distributed == *executorch* ]]
2025-03-14T04:14:48.7670535Z + [[ distributed == \j\i\t\_\l\e\g\a\c\y ]]
2025-03-14T04:14:48.7670904Z + [[ linux-jammy-py3.9-gcc11 == *libtorch* ]]
2025-03-14T04:14:48.7671272Z + [[ distributed == distributed ]]
2025-03-14T04:14:48.7671588Z + test_distributed
2025-03-14T04:14:48.7671886Z + echo 'Testing distributed python tests'
2025-03-14T04:14:48.7672247Z Testing distributed python tests
2025-03-14T04:14:48.7672698Z + python test/run_test.py --distributed-tests --shard 2 2 --verbose
2025-03-14T04:14:48.8812723Z /var/lib/jenkins/workspace/test/run_test.py:24: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
2025-03-14T04:14:48.8813647Z   import pkg_resources
2025-03-14T04:14:53.9851143Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/workspace/test/.pytorch-disabled-tests.json
2025-03-14T04:14:54.0890658Z Ignoring disabled issues:  ['']
2025-03-14T04:14:54.1065627Z Found test times from artifacts
2025-03-14T04:14:54.1749806Z Found test times from artifacts
2025-03-14T04:14:54.1776032Z Running all tests
2025-03-14T04:14:54.1899557Z Running parallel tests on 3 processes
2025-03-14T04:14:54.1906031Z Name: tests to run (est. time: 69.19min)
2025-03-14T04:14:54.1906523Z   Serial tests (112):
2025-03-14T04:14:54.1906898Z     distributed/fsdp/test_fsdp_dtensor_state_dict 1/1
2025-03-14T04:14:54.1907627Z     distributed/fsdp/test_fsdp_core 1/1
2025-03-14T04:14:54.1908078Z     distributed/fsdp/test_distributed_checkpoint 1/1
2025-03-14T04:14:54.1908512Z     distributed/fsdp/test_fsdp_misc 1/1
2025-03-14T04:14:54.1908975Z     distributed/test_c10d_nccl 1/1
2025-03-14T04:14:54.1909341Z     distributed/test_c10d_functional_native 1/1
2025-03-14T04:14:54.1909782Z     distributed/test_c10d_pypg 1/1
2025-03-14T04:14:54.1910124Z     distributed/test_c10d_logger 1/1
2025-03-14T04:14:54.1910651Z     distributed/_composable/test_composability/test_pp_composability 1/1
2025-03-14T04:14:54.1911228Z     distributed/test_c10d_object_collectives 1/1
2025-03-14T04:14:54.1911620Z     distributed/_tools/test_mod_tracker 1/1
2025-03-14T04:14:54.1912054Z     distributed/tensor/debug/test_op_coverage 1/1
2025-03-14T04:14:54.1912445Z     distributed/test_c10d_spawn_gloo 1/1
2025-03-14T04:14:54.1912854Z     distributed/test_c10d_spawn_nccl 1/1
2025-03-14T04:14:54.1913201Z     distributed/test_c10d_gloo 1/1
2025-03-14T04:14:54.1913605Z     distributed/fsdp/test_fsdp_fx 1/1
2025-03-14T04:14:54.1913975Z     distributed/tensor/test_experimental_ops 1/1
2025-03-14T04:14:54.1914454Z     distributed/checkpoint/test_fsdp_optim_state 1/1
2025-03-14T04:14:54.1914880Z     distributed/checkpoint/test_save_load_api 1/1
2025-03-14T04:14:54.1915330Z     distributed/_tools/test_memory_tracker 1/1
2025-03-14T04:14:54.1915760Z     distributed/test_collective_utils 1/1
2025-03-14T04:14:54.1916124Z     distributed/tensor/test_random_ops 1/1
2025-03-14T04:14:54.1916553Z     distributed/pipelining/test_backward 1/1
2025-03-14T04:14:54.1916930Z     distributed/tensor/test_dtensor_ops 1/1
2025-03-14T04:14:54.1917342Z     distributed/tensor/test_api 1/1
2025-03-14T04:14:54.1917707Z     distributed/checkpoint/test_format_utils 1/1
2025-03-14T04:14:54.1918155Z     distributed/tensor/test_redistribute 1/1
2025-03-14T04:14:54.1918577Z     distributed/tensor/parallel/test_tp_random_state 1/1
2025-03-14T04:14:54.1919097Z     distributed/tensor/debug/test_comm_mode_features 1/1
2025-03-14T04:14:54.1919560Z     distributed/_tools/test_mem_tracker 1/1
2025-03-14T04:14:54.1919941Z     distributed/pipelining/test_microbatch 1/1
2025-03-14T04:14:54.1920416Z     distributed/tensor/experimental/test_local_map 1/1
2025-03-14T04:14:54.1920848Z     distributed/tensor/parallel/test_tp_style 1/1
2025-03-14T04:14:54.1921394Z     distributed/_composable/fsdp/test_fully_shard_ignore_params 1/1
2025-03-14T04:14:54.1921933Z     distributed/checkpoint/e2e/test_pipeline 1/1
2025-03-14T04:14:54.1922339Z     distributed/pipelining/test_unflatten 1/1
2025-03-14T04:14:54.1922790Z     distributed/checkpoint/test_tp_checkpoint 1/1
2025-03-14T04:14:54.1923202Z     distributed/_tools/test_runtime_estimator 1/1
2025-03-14T04:14:54.1923647Z     distributed/checkpoint/test_fsspec 1/1
2025-03-14T04:14:54.1924038Z     distributed/tensor/test_xla_integration 1/1
2025-03-14T04:14:54.1924482Z     distributed/tensor/test_convolution_ops 1/1
2025-03-14T04:14:54.1924937Z     distributed/_tools/test_fsdp2_mem_tracker 1/1
2025-03-14T04:14:54.1925345Z     distributed/tensor/test_embedding_ops 1/1
2025-03-14T04:14:54.1925770Z     distributed/test_functional_api 1/1
2025-03-14T04:14:54.1926126Z     distributed/tensor/test_utils 1/1
2025-03-14T04:14:54.1926597Z     distributed/_composable/fsdp/test_fully_shard_overlap 1/1
2025-03-14T04:14:54.1927288Z     distributed/pipelining/test_stage 1/1
2025-03-14T04:14:54.1927689Z     distributed/checkpoint/test_hsdp_checkpoint 1/1
2025-03-14T04:14:54.1928183Z     distributed/checkpoint/test_fsdp_model_state 1/1
2025-03-14T04:14:54.1928630Z     distributed/checkpoint/test_dtensor_checkpoint 1/1
2025-03-14T04:14:54.1929077Z     distributed/checkpoint/test_planner 1/1
2025-03-14T04:14:54.1929539Z     distributed/checkpoint/test_dtensor_resharding 1/1
2025-03-14T04:14:54.1929970Z     distributed/checkpoint/test_compatibility 1/1
2025-03-14T04:14:54.1930417Z     distributed/fsdp/test_fsdp_memory 1/1
2025-03-14T04:14:54.1930787Z     distributed/tensor/test_op_strategy 1/1
2025-03-14T04:14:54.1931384Z     distributed/_composable/fsdp/test_fully_shard_grad_scaler 1/1
2025-03-14T04:14:54.1931886Z     distributed/test_fake_pg 1/1
2025-03-14T04:14:54.1932238Z     distributed/_composable/test_replicate 1/1
2025-03-14T04:14:54.1932678Z     distributed/optim/test_named_optimizer 1/1
2025-03-14T04:14:54.1933066Z     distributed/tensor/test_optimizers 1/1
2025-03-14T04:14:54.1933526Z     distributed/fsdp/test_fsdp_backward_prefetch 1/1
2025-03-14T04:14:54.1933937Z     distributed/checkpoint/test_utils 1/1
2025-03-14T04:14:54.1934381Z     distributed/tensor/parallel/test_tp_examples 1/1
2025-03-14T04:14:54.1934832Z     distributed/test_serialization 1/1
2025-03-14T04:14:54.1935188Z     distributed/test_pg_wrapper 1/1
2025-03-14T04:14:54.1935656Z     distributed/_composable/fsdp/test_fully_shard_frozen 1/1
2025-03-14T04:14:54.1936141Z     distributed/_composable/fsdp/test_fully_shard_init 1/1
2025-03-14T04:14:54.1936629Z     distributed/fsdp/test_fsdp_optim_state 1/1
2025-03-14T04:14:54.1937000Z     distributed/fsdp/test_fsdp_grad_acc 1/1
2025-03-14T04:14:54.1937420Z     distributed/test_multi_threaded_pg 1/1
2025-03-14T04:14:54.1937856Z     distributed/_composable/test_replicate_with_compiler 1/1
2025-03-14T04:14:54.1938298Z     distributed/fsdp/test_fsdp_fine_tune 1/1
2025-03-14T04:14:54.1938725Z     distributed/nn/jit/test_instantiator 1/1
2025-03-14T04:14:54.1939134Z     distributed/fsdp/test_fsdp_ignored_modules 1/1
2025-03-14T04:14:54.1939538Z     distributed/fsdp/test_fsdp_traversal 1/1
2025-03-14T04:14:54.1939988Z     distributed/_composable/fsdp/test_fully_shard_state_dict 1/1
2025-03-14T04:14:54.1940432Z     distributed/algorithms/test_join 1/1
2025-03-14T04:14:54.1940872Z     distributed/fsdp/test_fsdp_tp_integration 1/1
2025-03-14T04:14:54.1941262Z     distributed/rpc/test_faulty_agent 1/1
2025-03-14T04:14:54.1941638Z     distributed/rpc/test_tensorpipe_agent 2/5
2025-03-14T04:14:54.1942022Z     distributed/rpc/test_tensorpipe_agent 4/5
2025-03-14T04:14:54.1942405Z     distributed/fsdp/test_fsdp_checkpoint 1/1
2025-03-14T04:14:54.1942781Z     distributed/launcher/test_run 1/1
2025-03-14T04:14:54.1943148Z     distributed/fsdp/test_fsdp_hybrid_shard 1/1
2025-03-14T04:14:54.1943561Z     distributed/fsdp/test_fsdp_multiple_forward 1/1
2025-03-14T04:14:54.1944025Z     distributed/_shard/sharded_tensor/ops/test_tensor_ops 1/1
2025-03-14T04:14:54.1944575Z     distributed/_shard/sharded_optim/test_sharded_optim 1/1
2025-03-14T04:14:54.1945064Z     distributed/_shard/sharding_plan/test_sharding_plan 1/1
2025-03-14T04:14:54.1945544Z     distributed/test_launcher 1/1
2025-03-14T04:14:54.1945914Z     distributed/fsdp/test_fsdp_clip_grad_norm 1/1
2025-03-14T04:14:54.1946308Z     distributed/fsdp/test_fsdp_comm 1/1
2025-03-14T04:14:54.1946731Z     distributed/_shard/sharded_tensor/ops/test_embedding 1/1
2025-03-14T04:14:54.1947246Z     distributed/_shard/sharded_tensor/ops/test_embedding_bag 1/1
2025-03-14T04:14:54.1947801Z     distributed/_composable/fsdp/test_fully_shard_extensions 1/1
2025-03-14T04:14:54.1948305Z     distributed/_shard/sharded_tensor/ops/test_init 1/1
2025-03-14T04:14:54.1948787Z     distributed/_composable/fsdp/test_fully_shard_training 1/1
2025-03-14T04:14:54.1949229Z     distributed/_shard/test_sharder 1/1
2025-03-14T04:14:54.1949585Z     distributed/fsdp/test_wrap 1/1
2025-03-14T04:14:54.1949993Z     distributed/optim/test_apply_optimizer_in_backward 1/1
2025-03-14T04:14:54.1950540Z     distributed/fsdp/test_fsdp_unshard_params 1/1
2025-03-14T04:14:54.1950962Z     distributed/fsdp/test_fsdp_use_orig_params 1/1
2025-03-14T04:14:54.1951431Z     distributed/_shard/sharded_tensor/test_sharded_tensor 1/1
2025-03-14T04:14:54.1951942Z     distributed/_tools/test_fake_collectives 1/1
2025-03-14T04:14:54.1952330Z     distributed/elastic/events/lib_test 1/1
2025-03-14T04:14:54.1952701Z     distributed/elastic/metrics/api_test 1/1
2025-03-14T04:14:54.1953108Z     distributed/elastic/multiprocessing/api_test 1/1
2025-03-14T04:14:54.1953518Z     distributed/elastic/timer/api_test 1/1
2025-03-14T04:14:54.1954000Z     distributed/elastic/timer/local_timer_example 1/1
2025-03-14T04:14:54.1954424Z     distributed/elastic/timer/local_timer_test 1/1
2025-03-14T04:14:54.1954827Z     distributed/elastic/utils/distributed_test 1/1
2025-03-14T04:14:54.1955233Z     distributed/elastic/utils/logging_test 1/1
2025-03-14T04:14:54.1955611Z     distributed/elastic/utils/util_test 1/1
2025-03-14T04:14:54.1955963Z   Parallel tests (0):
2025-03-14T04:14:54.1956254Z Name: excluded (est. time: 0.0min)
2025-03-14T04:14:54.1956576Z   Serial tests (0):
2025-03-14T04:14:54.1956847Z   Parallel tests (0):
2025-03-14T04:14:54.2159978Z Running distributed/fsdp/test_fsdp_dtensor_state_dict 1/1 ... [2025-03-14 04:14:54.215529]
2025-03-14T04:14:54.2161060Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:14:54.2168031Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_dtensor_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:14:54.216342]
2025-03-14T04:14:59.0477483Z 
2025-03-14T04:14:59.0478680Z distributed/fsdp/test_fsdp_dtensor_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_dtensor_state_dict_1.1_27f29faadcc8f1d9_.log
2025-03-14T04:14:59.0480137Z Running 0 items in this shard:
2025-03-14T04:14:59.0480357Z 
2025-03-14T04:14:59.0486446Z Running distributed/fsdp/test_fsdp_core 1/1 ... [2025-03-14 04:14:59.048477]
2025-03-14T04:14:59.0486973Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:14:59.0491897Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_core.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:14:59.048970]
2025-03-14T04:15:03.1269006Z 
2025-03-14T04:15:03.1270478Z distributed/fsdp/test_fsdp_core 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_core_1.1_1f3cf1055d4f142a_.log
2025-03-14T04:15:03.1271410Z Running 0 items in this shard:
2025-03-14T04:15:03.1271626Z 
2025-03-14T04:15:03.1275167Z Running distributed/fsdp/test_distributed_checkpoint 1/1 ... [2025-03-14 04:15:03.127311]
2025-03-14T04:15:03.1275834Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:15:03.1279746Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_distributed_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:15:03.127723]
2025-03-14T04:15:07.2545172Z 
2025-03-14T04:15:07.2547155Z distributed/fsdp/test_distributed_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_distributed_checkpoint_1.1_97bd9714d438be1f_.log
2025-03-14T04:15:07.2549117Z Running 0 items in this shard:
2025-03-14T04:15:07.2549519Z 
2025-03-14T04:15:07.2554109Z Running distributed/fsdp/test_fsdp_misc 1/1 ... [2025-03-14 04:15:07.255198]
2025-03-14T04:15:07.2554924Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:15:07.2560528Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_misc.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:15:07.255718]
2025-03-14T04:16:06.6143656Z 
2025-03-14T04:16:06.6145101Z distributed/fsdp/test_fsdp_misc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_misc_1.1_917045c2fda1d041_.log
2025-03-14T04:16:06.6159584Z Running 28 items in this shard: test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_cpu_init_with_sync_module_states, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_cpu_init_stays_on_cpu, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_cpu_training, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_device_id_use_index_False, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_device_id_use_index_True, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_False_sharding_strategy0, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_False_sharding_strategy1, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_True_sharding_strategy0, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_module_no_compute_grad_use_second_layer_True_sharding_strategy1, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_not_all_outputs_used_in_loss, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_optim_overlap_no_use_orig_params_error, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_optimizer_overlap, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiProcess::test_fsdp_zero2_eval_with_prefetch, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_cpu_gpu_module, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_device_id_auto_wrap, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_device_id_cpu_offload, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_device_id_no_move_ignored_params_and_bufs, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_ignored_module_meta, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_namedtuple, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_same_model_across_ranks, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_fsdp_unsupported_module_cls, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_homogeneous_attributes, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_module_device_mismatches_device_id, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_multigpu_module, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscMultiThread::test_no_params, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscWorldSize1::test_training_device_mismatch_errors, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscWorldSize1::test_unsafe_setattr, test/distributed/fsdp/test_fsdp_misc.py::TestFSDPMiscWorldSize1::test_world_size_1_sharding_strategy_warning
2025-03-14T04:16:06.6172772Z 
2025-03-14T04:16:06.6173021Z Running distributed/test_c10d_nccl 1/1 ... [2025-03-14 04:16:06.614913]
2025-03-14T04:16:06.6173501Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:16:06.6174741Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/test_c10d_nccl.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:16:06.615372]
2025-03-14T04:16:08.1405868Z 
2025-03-14T04:16:08.1407131Z distributed/test_c10d_nccl 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_nccl_1.1_60d56c6a7c95f624_.log
2025-03-14T04:16:08.1408312Z 
2025-03-14T04:16:08.1411374Z Running distributed/test_c10d_functional_native 1/1 ... [2025-03-14 04:16:08.140961]
2025-03-14T04:16:08.1411935Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:16:08.1416578Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/test_c10d_functional_native.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:16:08.141425]
2025-03-14T04:16:20.3805284Z 
2025-03-14T04:16:20.3806722Z distributed/test_c10d_functional_native 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_functional_native_1.1_dd6f12771316463b_.log
2025-03-14T04:16:20.3819880Z Running 30 items in this shard: test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_gather_into_tensor_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_gather_into_tensor_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_coalesced_, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_reduce_single_, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_all_to_all_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_broadcast, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_fixed_striding, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_inductor_dtypeview_memory_leak, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_reduce_scatter_tensor_single, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_threading, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_unwaited, test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_wait_tensor, test/distributed/test_c10d_functional_native.py::PyWorkTest::test_collectives, test/distributed/test_c10d_functional_native.py::PyWorkTest::test_wait_tensor, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_gather_into_tensor_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_gather_into_tensor_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_reduce_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_reduce_non_contig_input, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_reduce_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_all_to_all_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_broadcast, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_inplace_op_on_view, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reduce_scatter_tensor_single, test/distributed/test_c10d_functional_native.py::CompileTest::test_inductor_reuse_buffer_after_inplace_collective, test/distributed/test_c10d_functional_native.py::CompileTest::test_ranks_and_tag, test/distributed/test_c10d_functional_native.py::CompileTest::test_wait_tensor
2025-03-14T04:16:20.3832030Z 
2025-03-14T04:16:20.3832275Z Running distributed/test_c10d_pypg 1/1 ... [2025-03-14 04:16:20.381043]
2025-03-14T04:16:20.3832757Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:16:20.3833866Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/test_c10d_pypg.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:16:20.381492]
2025-03-14T04:16:24.6089533Z 
2025-03-14T04:16:24.6091033Z distributed/test_c10d_pypg 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_pypg_1.1_c37b93fab008fc91_.log
2025-03-14T04:16:24.6127309Z Running 46 items in this shard: test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_dataclass_output, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_dataclass_output_unused_param, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_dynamic_module, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_dynamic_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_once_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_once_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_static_graph_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_static_graph_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_twice_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_unused_params_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_unused_params_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_weight_sharing_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_checkpointing_weight_sharing_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_invoke_work_object, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_no_init_sync, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_with_pypg, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_ddp_with_pypg_with_grad_views, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_invalid_powerSGD_state, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_sync_batch_norm_empty_input, test/distributed/test_c10d_pypg.py::TestDDPWithWorkSubclass::test_sync_batch_norm_only_empty_input, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_dataclass_output, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_dataclass_output_unused_param, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_dynamic_module, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_dynamic_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_once_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_once_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_static_graph_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_static_graph_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_twice_weight_sharing, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_unused_params_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_unused_params_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_weight_sharing_use_reentrant_False, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_checkpointing_weight_sharing_use_reentrant_True, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_invoke_work_object, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_no_init_sync, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_with_pypg, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_ddp_with_pypg_with_grad_views, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_invalid_powerSGD_state, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_sync_batch_norm_empty_input, test/distributed/test_c10d_pypg.py::TestDDPWithWorkWrapper::test_sync_batch_norm_only_empty_input, test/distributed/test_c10d_pypg.py::TestPyProcessGroup::test_abort_shutdown, test/distributed/test_c10d_pypg.py::TestPyProcessGroup::test_attr_overrides
2025-03-14T04:16:24.6160464Z 
2025-03-14T04:16:24.6160751Z Running distributed/test_c10d_logger 1/1 ... [2025-03-14 04:16:24.609880]
2025-03-14T04:16:24.6161254Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:16:24.6162370Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/test_c10d_logger.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:16:24.610471]
2025-03-14T04:16:36.3532378Z 
2025-03-14T04:16:36.3534070Z distributed/test_c10d_logger 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_logger_1.1_102273f250240740_.log
2025-03-14T04:16:36.3537106Z Running 2 items in this shard: test/distributed/test_c10d_logger.py::C10dErrorLoggerTest::test_exception_logger, test/distributed/test_c10d_logger.py::C10dErrorLoggerTest::test_get_or_create_logger
2025-03-14T04:16:36.3538858Z 
2025-03-14T04:16:36.3541337Z Running distributed/_composable/test_composability/test_pp_composability 1/1 ... [2025-03-14 04:16:36.353720]
2025-03-14T04:16:36.3542108Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:16:36.3546019Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_composable/test_composability/test_pp_composability.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:16:36.354302]
2025-03-14T04:16:40.5317627Z 
2025-03-14T04:16:40.5319393Z distributed/_composable/test_composability/test_pp_composability 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_composability.test_pp_composability_1.1_e331726cd66b54f9_.log
2025-03-14T04:16:40.5327899Z Running 11 items in this shard: test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_3d_with_tp_dp_pp_ScheduleClass0_bfloat16, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_3d_with_tp_dp_pp_ScheduleClass0_float32, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_3d_with_tp_dp_pp_ScheduleClass1_bfloat16, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_3d_with_tp_dp_pp_ScheduleClass1_float32, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_3d_with_tp_dp_pp_ScheduleClass2_bfloat16, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_3d_with_tp_dp_pp_ScheduleClass2_float32, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_3d_with_tp_dp_pp_ScheduleClass3_bfloat16, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_3d_with_tp_dp_pp_ScheduleClass3_float32, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_3d_with_tp_dp_pp_ScheduleClass4_bfloat16, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_3d_with_tp_dp_pp_ScheduleClass4_float32, test/distributed/_composable/test_composability/test_pp_composability.py::ComposabilityTest::test_pp_and_dcp
2025-03-14T04:16:40.5335570Z 
2025-03-14T04:16:40.5335872Z Running distributed/test_c10d_object_collectives 1/1 ... [2025-03-14 04:16:40.532199]
2025-03-14T04:16:40.5336419Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:16:40.5337745Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/test_c10d_object_collectives.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:16:40.532672]
2025-03-14T04:17:22.0899162Z 
2025-03-14T04:17:22.0900408Z distributed/test_c10d_object_collectives 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_object_collectives_1.1_e9444a739eb4dfd7_.log
2025-03-14T04:17:22.0906226Z Running 9 items in this shard: test/distributed/test_c10d_object_collectives.py::TestObjectCollectivesCPU::test_all_gather_object_cpu, test/distributed/test_c10d_object_collectives.py::TestObjectCollectivesCPU::test_broadcast_object_list_cpu, test/distributed/test_c10d_object_collectives.py::TestObjectCollectivesCPU::test_gather_object_cpu, test/distributed/test_c10d_object_collectives.py::TestObjectCollectivesCPU::test_scatter_object_list_cpu, test/distributed/test_c10d_object_collectives.py::TestObjectCollectivesCPU::test_send_recv_object_list_cpu, test/distributed/test_c10d_object_collectives.py::TestObjectCollectivesCPU::test_subpg_all_gather_object_cpu, test/distributed/test_c10d_object_collectives.py::TestObjectCollectivesCPU::test_subpg_broadcast_object_cpu, test/distributed/test_c10d_object_collectives.py::TestObjectCollectivesCPU::test_subpg_gather_object_cpu, test/distributed/test_c10d_object_collectives.py::TestObjectCollectivesCPU::test_subpg_scatter_object_cpu
2025-03-14T04:17:22.0911031Z 
2025-03-14T04:17:22.0911365Z Running distributed/_tools/test_mod_tracker 1/1 ... [2025-03-14 04:17:22.090455]
2025-03-14T04:17:22.0911917Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:17:22.0913122Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_tools/test_mod_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:17:22.090928]
2025-03-14T04:17:26.2186375Z 
2025-03-14T04:17:26.2187491Z distributed/_tools/test_mod_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_mod_tracker_1.1_8a6455da247091c2_.log
2025-03-14T04:17:26.2189729Z Running 4 items in this shard: test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_ac, test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_bw_detection, test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_module_hierarchy, test/distributed/_tools/test_mod_tracker.py::TestModTracker::test_user_hooks
2025-03-14T04:17:26.2191231Z 
2025-03-14T04:17:26.2193332Z Running distributed/tensor/debug/test_op_coverage 1/1 ... [2025-03-14 04:17:26.219118]
2025-03-14T04:17:26.2193892Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:17:26.2198421Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/debug/test_op_coverage.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:17:26.219579]
2025-03-14T04:17:30.5475940Z 
2025-03-14T04:17:30.5477176Z distributed/tensor/debug/test_op_coverage 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.debug.test_op_coverage_1.1_22c714e57cb502e4_.log
2025-03-14T04:17:30.5478624Z Running 1 items in this shard: test/distributed/tensor/debug/test_op_coverage.py::TestOpCoverage::test_trace_with_inductor_decomp
2025-03-14T04:17:30.5479523Z 
2025-03-14T04:17:30.5482944Z Running distributed/test_c10d_spawn_gloo 1/1 ... [2025-03-14 04:17:30.548106]
2025-03-14T04:17:30.5483515Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:17:30.5487967Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/test_c10d_spawn_gloo.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:17:30.548570]
2025-03-14T04:18:52.6402462Z 
2025-03-14T04:18:52.6405830Z distributed/test_c10d_spawn_gloo 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_spawn_gloo_1.1_85ea6b4a208590aa_.log
2025-03-14T04:18:52.6416026Z Running 11 items in this shard: test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cpu, test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cuda, test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_rnn, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_gather, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all_single, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_allreduce, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_broadcast, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_gather, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_reduce, test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_scatter
2025-03-14T04:18:52.6425157Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cpu
2025-03-14T04:18:52.6426806Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_cuda
2025-03-14T04:18:52.6428319Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::DistributedDataParallelSingleProcessTest::test_rnn
2025-03-14T04:18:52.6429814Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_gather
2025-03-14T04:18:52.6431314Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all
2025-03-14T04:18:52.6432855Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_all_to_all_single
2025-03-14T04:18:52.6434409Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_allreduce
2025-03-14T04:18:52.6435798Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_broadcast
2025-03-14T04:18:52.6437123Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_gather
2025-03-14T04:18:52.6438479Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_reduce
2025-03-14T04:18:52.6439856Z Running 1 items in this shard: test/distributed/test_c10d_spawn_gloo.py::TestDistributedNNFunctionsGloo::test_scatter
2025-03-14T04:18:52.6440709Z 
2025-03-14T04:18:52.6441089Z Running distributed/test_c10d_spawn_nccl 1/1 ... [2025-03-14 04:18:52.641322]
2025-03-14T04:18:52.6458504Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:18:52.6460317Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/test_c10d_spawn_nccl.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:18:52.641920]
2025-03-14T04:19:34.7464175Z 
2025-03-14T04:19:34.7465647Z distributed/test_c10d_spawn_nccl 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_spawn_nccl_1.1_d823d18f16e1f359_.log
2025-03-14T04:19:34.7471508Z Running 10 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather_base, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_reduce_non_contiguous, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all_single, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_allreduce, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_broadcast, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter, test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter_non_contiguous
2025-03-14T04:19:34.7477156Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather
2025-03-14T04:19:34.7478182Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_gather_base
2025-03-14T04:19:34.7479278Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_reduce_non_contiguous
2025-03-14T04:19:34.7480479Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all
2025-03-14T04:19:34.7481619Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_all_to_all_single
2025-03-14T04:19:34.7482684Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_allreduce
2025-03-14T04:19:34.7483664Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_broadcast
2025-03-14T04:19:34.7484629Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce
2025-03-14T04:19:34.7485614Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter
2025-03-14T04:19:34.7486718Z Running 1 items in this shard: test/distributed/test_c10d_spawn_nccl.py::TestDistributedNNFunctionsNccl::test_reduce_scatter_non_contiguous
2025-03-14T04:19:34.7487358Z 
2025-03-14T04:19:34.7487600Z Running distributed/test_c10d_gloo 1/1 ... [2025-03-14 04:19:34.747328]
2025-03-14T04:19:34.7488073Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:19:34.7489237Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/test_c10d_gloo.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:19:34.747950]
2025-03-14T04:35:43.4042221Z 
2025-03-14T04:35:43.4043274Z distributed/test_c10d_gloo 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_c10d_gloo_1.1_7f023791449f76f0_.log
2025-03-14T04:35:43.4109283Z Running 122 items in this shard: test/distributed/test_c10d_gloo.py::RendezvousEnvTest::test_logging_init, test/distributed/test_c10d_gloo.py::TimeoutTest::test_default_store_timeout_gloo, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_async, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_inference_mode, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_into_tensor_coalesced, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_noncontiguous_input, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_async, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_barrier_implies_wait, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_empty_tensors, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_noncontiguous_input, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_multi_device_constructor, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor_coalesced, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_send_recv_all_to_all, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_set_gloo_pg_timeout, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics_cuda, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_checks, test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_cuda_dispatched, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output_unused_param, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_weight_sharing, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_weight_sharing, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_False, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_True, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_cpu, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_gpu_gloo, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_register_just_once, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_sparse_gradients, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_init, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_return_type, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_find_unused_parameters_when_unused_parameters_empty, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_grad_is_view, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_static_graph, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_integer_list, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_torch_device_list, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_2gpu_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_4gpu_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module_grad_is_view, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output_with_unused_parameters, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_sharded_tensor, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_invalid_powerSGD_state, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_save_load_checkpoint, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients_grad_is_view, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_empty_input, test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_only_empty_input, test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward, test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_optimizer, test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_unused_parameters, test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_multi_bucket, test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_single_bucket, test/distributed/test_c10d_gloo.py::ReducerTest::test_single_dtype_single_bucket, test/distributed/test_c10d_gloo.py::CommTest::test_bool_tensors, test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cpu, test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cuda, test/distributed/test_c10d_gloo.py::CommTest::test_gloo_rank_membership, test/distributed/test_c10d_gloo.py::CommTest::test_gloo_warn_not_in_group, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_default, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_subgroup, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_default_pg_gloo, test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_gloo_new_group, test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_complex, test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_mismatch, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_all_to_all_single, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allgather_coalesced, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allreduce_coalesced, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_collectives, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_default_process_group, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_for_all_backends, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_optional_backend, test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_monitored_barrier, test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync, test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_duplicate_pg, test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_sanity_check
2025-03-14T04:35:43.4159837Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::RendezvousEnvTest::test_logging_init
2025-03-14T04:35:43.4160722Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::TimeoutTest::test_default_store_timeout_gloo
2025-03-14T04:35:43.4161625Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics
2025-03-14T04:35:43.4162537Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_basics_cuda
2025-03-14T04:35:43.4163453Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_checks
2025-03-14T04:35:43.4164384Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_async
2025-03-14T04:35:43.4165362Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_coalesced_checks
2025-03-14T04:35:43.4166338Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_inference_mode
2025-03-14T04:35:43.4167331Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_into_tensor_coalesced
2025-03-14T04:35:43.4168449Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_noncontiguous_input
2025-03-14T04:35:43.4169404Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress
2025-03-14T04:35:43.4170317Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allgather_stress_cuda
2025-03-14T04:35:43.4171232Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics
2025-03-14T04:35:43.4172150Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_basics_cuda
2025-03-14T04:35:43.4173118Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_checks
2025-03-14T04:35:43.4174056Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_async
2025-03-14T04:35:43.4175035Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_basics
2025-03-14T04:35:43.4176009Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks
2025-03-14T04:35:43.4177031Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_checks_cuda
2025-03-14T04:35:43.4178035Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_coalesced_stress
2025-03-14T04:35:43.4178977Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress
2025-03-14T04:35:43.4179893Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_allreduce_stress_cuda
2025-03-14T04:35:43.4180827Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_barrier_implies_wait
2025-03-14T04:35:43.4181741Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics
2025-03-14T04:35:43.4182658Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_basics_cuda
2025-03-14T04:35:43.4183581Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_checks
2025-03-14T04:35:43.4184513Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress
2025-03-14T04:35:43.4185558Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_broadcast_stress_cuda
2025-03-14T04:35:43.4187763Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_empty_tensors
2025-03-14T04:35:43.4188641Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics
2025-03-14T04:35:43.4189537Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_basics_cuda
2025-03-14T04:35:43.4190443Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_checks
2025-03-14T04:35:43.4191374Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_noncontiguous_input
2025-03-14T04:35:43.4192309Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress
2025-03-14T04:35:43.4193200Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_gather_stress_cuda
2025-03-14T04:35:43.4194136Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_multi_device_constructor
2025-03-14T04:35:43.4195050Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics
2025-03-14T04:35:43.4195939Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_basics_cuda
2025-03-14T04:35:43.4196943Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_checks
2025-03-14T04:35:43.4197854Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor
2025-03-14T04:35:43.4199046Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_scatter_tensor_coalesced
2025-03-14T04:35:43.4199998Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress
2025-03-14T04:35:43.4201004Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_reduce_stress_cuda
2025-03-14T04:35:43.4201900Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics
2025-03-14T04:35:43.4202796Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_basics_cuda
2025-03-14T04:35:43.4203701Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_checks
2025-03-14T04:35:43.4204577Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress
2025-03-14T04:35:43.4205472Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_scatter_stress_cuda
2025-03-14T04:35:43.4206395Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_send_recv_all_to_all
2025-03-14T04:35:43.4207324Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_set_gloo_pg_timeout
2025-03-14T04:35:43.4208262Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics
2025-03-14T04:35:43.4209237Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_basics_cuda
2025-03-14T04:35:43.4210221Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_checks
2025-03-14T04:35:43.4211216Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ProcessGroupGlooTest::test_sparse_allreduce_cuda_dispatched
2025-03-14T04:35:43.4212214Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output
2025-03-14T04:35:43.4213238Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_dataclass_output_unused_param
2025-03-14T04:35:43.4214352Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_module
2025-03-14T04:35:43.4215520Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_dynamic_weight_sharing
2025-03-14T04:35:43.4216731Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_False
2025-03-14T04:35:43.4217949Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_once_use_reentrant_True
2025-03-14T04:35:43.4219222Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_False
2025-03-14T04:35:43.4220542Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_static_graph_use_reentrant_True
2025-03-14T04:35:43.4221818Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_False
2025-03-14T04:35:43.4223030Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_use_reentrant_True
2025-03-14T04:35:43.4224307Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_twice_weight_sharing
2025-03-14T04:35:43.4225615Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_False
2025-03-14T04:35:43.4226897Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_unused_params_use_reentrant_True
2025-03-14T04:35:43.4228174Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_False
2025-03-14T04:35:43.4229529Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_checkpointing_weight_sharing_use_reentrant_True
2025-03-14T04:35:43.4230733Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_cpu
2025-03-14T04:35:43.4231876Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_future_passing_gpu_gloo
2025-03-14T04:35:43.4233010Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_register_just_once
2025-03-14T04:35:43.4234114Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_comm_hook_sparse_gradients
2025-03-14T04:35:43.4235198Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_init
2025-03-14T04:35:43.4236294Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ddp_invalid_comm_hook_return_type
2025-03-14T04:35:43.4237494Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_find_unused_parameters_when_unused_parameters_empty
2025-03-14T04:35:43.4238691Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad
2025-03-14T04:35:43.4239869Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_grad_is_view
2025-03-14T04:35:43.4241127Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_global_local_unused_params_grad_with_static_graph
2025-03-14T04:35:43.4242374Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_integer_list
2025-03-14T04:35:43.4243645Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_1gpu_module_device_ids_torch_device_list
2025-03-14T04:35:43.4244810Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_2gpu_module
2025-03-14T04:35:43.4245853Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_4gpu_module
2025-03-14T04:35:43.4246883Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module
2025-03-14T04:35:43.4247969Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_gloo_backend_cpu_module_grad_is_view
2025-03-14T04:35:43.4249015Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output
2025-03-14T04:35:43.4250076Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_output_with_unused_parameters
2025-03-14T04:35:43.4251167Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_ignored_sharded_tensor
2025-03-14T04:35:43.4252180Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_invalid_powerSGD_state
2025-03-14T04:35:43.4253248Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_save_load_checkpoint
2025-03-14T04:35:43.4254222Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients
2025-03-14T04:35:43.4255244Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sparse_gradients_grad_is_view
2025-03-14T04:35:43.4256326Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_empty_input
2025-03-14T04:35:43.4257491Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::DistributedDataParallelTest::test_sync_batch_norm_only_empty_input
2025-03-14T04:35:43.4258467Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward
2025-03-14T04:35:43.4259331Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_optimizer
2025-03-14T04:35:43.4260257Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_forward_backward_unused_parameters
2025-03-14T04:35:43.4261170Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_multi_bucket
2025-03-14T04:35:43.4262047Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_multi_dtype_single_bucket
2025-03-14T04:35:43.4262931Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::ReducerTest::test_single_dtype_single_bucket
2025-03-14T04:35:43.4263754Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_bool_tensors
2025-03-14T04:35:43.4264573Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cpu
2025-03-14T04:35:43.4265541Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_broadcast_coalesced_gloo_cuda
2025-03-14T04:35:43.4266414Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_gloo_rank_membership
2025-03-14T04:35:43.4267251Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_gloo_warn_not_in_group
2025-03-14T04:35:43.4268149Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_default
2025-03-14T04:35:43.4269108Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_incremented_gloo_subgroup
2025-03-14T04:35:43.4270049Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_default_pg_gloo
2025-03-14T04:35:43.4270971Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_sequence_num_set_gloo_new_group
2025-03-14T04:35:43.4271843Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_complex
2025-03-14T04:35:43.4272687Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::CommTest::test_tensor_dtype_mismatch
2025-03-14T04:35:43.4273724Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_all_to_all_single
2025-03-14T04:35:43.4275052Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allgather_coalesced
2025-03-14T04:35:43.4276287Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_allreduce_coalesced
2025-03-14T04:35:43.4277483Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_collectives
2025-03-14T04:35:43.4278682Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_default_process_group
2025-03-14T04:35:43.4279994Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_for_all_backends
2025-03-14T04:35:43.4281459Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_init_process_group_optional_backend
2025-03-14T04:35:43.4282757Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::GlooProcessGroupWithDispatchedCollectivesTests::test_monitored_barrier
2025-03-14T04:35:43.4283802Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync
2025-03-14T04:35:43.4284992Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_duplicate_pg
2025-03-14T04:35:43.4286015Z Running 1 items in this shard: test/distributed/test_c10d_gloo.py::LargeCommTest::test_new_group_local_sync_sanity_check
2025-03-14T04:35:43.4286550Z 
2025-03-14T04:35:43.4286817Z Running distributed/fsdp/test_fsdp_fx 1/1 ... [2025-03-14 04:35:43.408364]
2025-03-14T04:35:43.4287317Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:35:43.4288441Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_fx.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:35:43.408850]
2025-03-14T04:35:47.3860242Z 
2025-03-14T04:35:47.3861252Z distributed/fsdp/test_fsdp_fx 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_fx_1.1_91e8bed7e84ddc65_.log
2025-03-14T04:35:47.3862155Z Running 0 items in this shard:
2025-03-14T04:35:47.3862404Z 
2025-03-14T04:35:47.3866825Z Running distributed/tensor/test_experimental_ops 1/1 ... [2025-03-14 04:35:47.386498]
2025-03-14T04:35:47.3867375Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:35:47.3872052Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/test_experimental_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:35:47.386979]
2025-03-14T04:36:03.5817398Z 
2025-03-14T04:36:03.5818764Z distributed/tensor/test_experimental_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.test_experimental_ops_1.1_51d72d2486387279_.log
2025-03-14T04:36:03.5820840Z Running 3 items in this shard: test/distributed/tensor/test_experimental_ops.py::DistOtherOpsTest::test_bernoulli, test/distributed/tensor/test_experimental_ops.py::DistOtherOpsTest::test_nll, test/distributed/tensor/test_experimental_ops.py::DistOtherOpsTest::test_slice
2025-03-14T04:36:03.5822100Z 
2025-03-14T04:36:03.5823941Z Running distributed/checkpoint/test_fsdp_optim_state 1/1 ... [2025-03-14 04:36:03.582200]
2025-03-14T04:36:03.5824626Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:36:03.5829469Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/checkpoint/test_fsdp_optim_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:36:03.582666]
2025-03-14T04:36:16.2280801Z 
2025-03-14T04:36:16.2282164Z distributed/checkpoint/test_fsdp_optim_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_fsdp_optim_state_1.1_cd9da7094ebca3fe_.log
2025-03-14T04:36:16.2285194Z Running 2 items in this shard: test/distributed/checkpoint/test_fsdp_optim_state.py::FsdpOptimStateCheckpoint::test_load_sharded_optimizer_state_dict_pass_planner_False, test/distributed/checkpoint/test_fsdp_optim_state.py::FsdpOptimStateCheckpoint::test_load_sharded_optimizer_state_dict_pass_planner_True
2025-03-14T04:36:16.2286605Z 
2025-03-14T04:36:16.2288354Z Running distributed/checkpoint/test_save_load_api 1/1 ... [2025-03-14 04:36:16.228636]
2025-03-14T04:36:16.2289156Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:36:16.2293587Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/checkpoint/test_save_load_api.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:36:16.229139]
2025-03-14T04:36:28.4694991Z 
2025-03-14T04:36:28.4696109Z distributed/checkpoint/test_save_load_api 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_save_load_api_1.1_200aaa8ef57841fd_.log
2025-03-14T04:36:28.4698160Z Running 2 items in this shard: test/distributed/checkpoint/test_save_load_api.py::TestSaveAndLoadAPI::test_assert_same_keys, test/distributed/checkpoint/test_save_load_api.py::TestSaveAndLoadAPI::test_auto_detect
2025-03-14T04:36:28.4699282Z 
2025-03-14T04:36:28.4706488Z Running distributed/_tools/test_memory_tracker 1/1 ... [2025-03-14 04:36:28.470436]
2025-03-14T04:36:28.4707046Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:36:28.4713217Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_tools/test_memory_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:36:28.471102]
2025-03-14T04:36:32.5001441Z 
2025-03-14T04:36:32.5003275Z distributed/_tools/test_memory_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_memory_tracker_1.1_f12fbce4b0259b9a_.log
2025-03-14T04:36:32.5005488Z Running 1 items in this shard: test/distributed/_tools/test_memory_tracker.py::TestMemoryTracker::test_local_model
2025-03-14T04:36:32.5006499Z 
2025-03-14T04:36:32.5007009Z Running distributed/test_collective_utils 1/1 ... [2025-03-14 04:36:32.500471]
2025-03-14T04:36:32.5007838Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:36:32.5013730Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/test_collective_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:36:32.501035]
2025-03-14T04:36:36.1288133Z 
2025-03-14T04:36:36.1289641Z distributed/test_collective_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_collective_utils_1.1_7f0d9f0304992aec_.log
2025-03-14T04:36:36.1290476Z 
2025-03-14T04:36:36.1293397Z Running distributed/tensor/test_random_ops 1/1 ... [2025-03-14 04:36:36.129149]
2025-03-14T04:36:36.1293975Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:36:36.1299002Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/test_random_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:36:36.129661]
2025-03-14T04:37:33.4173180Z 
2025-03-14T04:37:33.4176073Z distributed/tensor/test_random_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.test_random_ops_1.1_64768b4cd84ae001_.log
2025-03-14T04:37:33.4182560Z Running 12 items in this shard: test/distributed/tensor/test_random_ops.py::DistTensorRandomInitTest::test_fsdp_tp_model_meta_init, test/distributed/tensor/test_random_ops.py::DistTensorRandomInitTest::test_init_ops, test/distributed/tensor/test_random_ops.py::DistTensorRandomInitTest::test_meta_tensor_init, test/distributed/tensor/test_random_ops.py::DistTensorRandomInitTest::test_tp_model_meta_init, test/distributed/tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_dropout_1d, test/distributed/tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_rand_1d, test/distributed/tensor/test_random_ops.py::DistTensorRandomOpTest::test_deterministic_uniform_2d, test/distributed/tensor/test_random_ops.py::DistTensorRandomOpTest::test_manual_seed, test/distributed/tensor/test_random_ops.py::DistTensorRandomOpTest::test_manual_seed_submesh, test/distributed/tensor/test_random_ops.py::DistTensorRandomOpTest::test_pipeline_parallel_manual_seed, test/distributed/tensor/test_random_ops.py::DistTensorRandomOpTest::test_rng_tracker_init, test/distributed/tensor/test_random_ops.py::DistTensorRandomOpsTest3D::test_hsdp_tp_model_meta_init
2025-03-14T04:37:33.4188397Z 
2025-03-14T04:37:33.4188699Z Running distributed/pipelining/test_backward 1/1 ... [2025-03-14 04:37:33.417536]
2025-03-14T04:37:33.4189234Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:37:33.4190527Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/pipelining/test_backward.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:37:33.418031]
2025-03-14T04:37:37.5967192Z 
2025-03-14T04:37:37.5968436Z distributed/pipelining/test_backward 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_backward_1.1_b7d4963b4ce280b9_.log
2025-03-14T04:37:37.5971569Z Running 4 items in this shard: test/distributed/pipelining/test_backward.py::StageBackwardTestsCPU::test_stage_backward_cpu, test/distributed/pipelining/test_backward.py::StageBackwardTestsCPU::test_stage_backward_input_cpu, test/distributed/pipelining/test_backward.py::StageBackwardTestsCPU::test_stage_backward_weight_cpu, test/distributed/pipelining/test_backward.py::StageBackwardTestsCPU::test_stage_backward_weight_multiple_iters_cpu
2025-03-14T04:37:37.5973718Z 
2025-03-14T04:37:37.5977149Z Running distributed/tensor/test_dtensor_ops 1/1 ... [2025-03-14 04:37:37.597511]
2025-03-14T04:37:37.5977684Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:37:37.5982506Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/test_dtensor_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:37:37.598038]
2025-03-14T04:39:17.1483336Z 
2025-03-14T04:39:17.1484435Z distributed/tensor/test_dtensor_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.test_dtensor_ops_1.1_61b123688546946d_.log
2025-03-14T04:39:17.2087063Z Running 670 items in this shard: test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_H_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_T_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___getitem___cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___radd___cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rdiv___cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rmatmul___cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rmod___cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rmul___cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rpow___cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db___rsub___cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__batch_norm_with_update_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__chunk_cat_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__native_batch_norm_legit_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__segment_reduce_lengths_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__segment_reduce_offsets_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__softmax_backward_data_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__unsafe_masked_index_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__unsafe_masked_index_put_accumulate_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db__upsample_bilinear2d_aa_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_abs_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_acos_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_acosh_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_add_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addbmm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addcdiv_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addcmul_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addmm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addmm_decomposed_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addmv_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_addr_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_alias_copy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_all_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_allclose_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_amax_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_amin_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_aminmax_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_angle_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_any_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_arange_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argmax_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argmin_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argsort_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_argwhere_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_copy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_partial_views_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_as_strided_scatter_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_asin_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_asinh_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atan2_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atan_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atanh_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atleast_1d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atleast_2d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_atleast_3d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_baddbmm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bernoulli_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bfloat16_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_block_diag_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bmm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bool_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_broadcast_shapes_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_broadcast_tensors_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_broadcast_to_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_bucketize_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_byte_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cartesian_prod_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cat_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cauchy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cdist_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cdouble_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ceil_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cfloat_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_chalf_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_char_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cholesky_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cholesky_inverse_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cholesky_solve_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_chunk_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clamp_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clamp_max_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clamp_min_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_clone_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_column_stack_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_combinations_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_complex_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_conj_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_conj_physical_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_constant_pad_nd_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_contiguous_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_copysign_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_corrcoef_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cos_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cosh_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_count_nonzero_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cov_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cross_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cummax_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cummin_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cumprod_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cumsum_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_cumulative_trapezoid_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_deg2rad_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diag_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diag_embed_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagflat_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagonal_copy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagonal_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diagonal_scatter_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_diff_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_digamma_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dist_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_div_floor_rounding_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_div_no_rounding_mode_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_div_trunc_rounding_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dot_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_double_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dsplit_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_dstack_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_einsum_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_like_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_permuted_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_empty_strided_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_eq_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_equal_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_erf_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_erfc_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_erfinv_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_exp2_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_exp_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_expand_as_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_expand_copy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_expand_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_expm1_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_exponential_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_eye_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fft2_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fft_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fftn_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_fftshift_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_hfft2_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_hfft_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_hfftn_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifft2_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifft_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifftn_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ifftshift_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ihfft2_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ihfft_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_ihfftn_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_irfft2_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_irfft_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_irfftn_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_rfft2_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_rfft_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fft_rfftn_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fill_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_flatten_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_flip_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fliplr_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_flipud_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_float_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_float_power_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_floor_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_floor_divide_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fmax_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fmin_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_fmod_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_frac_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_frexp_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_full_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_full_like_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_gather_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ge_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_geometric_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_geqrf_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_gradient_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_grid_sampler_2d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_gt_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_half_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_heaviside_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_histc_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_histogram_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_histogramdd_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_hsplit_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_hstack_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_hypot_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_i0_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_igamma_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_igammac_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_add_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_copy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_fill_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_put_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_amax_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_amin_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_mean_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_reduce_prod_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_index_select_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_inner_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_int_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isclose_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isfinite_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isin_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isinf_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isnan_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isneginf_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isposinf_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_isreal_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_item_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_2inputs_2outputs_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_4inputs_with_extra_args_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_binary_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_binary_return_by_ref_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_jiterator_unary_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_kron_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_kthvalue_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ldexp_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_le_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lerp_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lgamma_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cholesky_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cholesky_ex_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cond_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_cross_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_det_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_diagonal_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eig_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eigh_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eigvals_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_eigvalsh_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_householder_product_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_inv_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_inv_ex_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_ldl_factor_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_ldl_factor_ex_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_ldl_solve_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lstsq_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lstsq_grad_oriented_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_factor_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_factor_ex_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_lu_solve_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_norm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_power_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_rank_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_matrix_rank_hermitian_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_multi_dot_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_norm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_norm_subgradients_at_zero_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_pinv_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_pinv_hermitian_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_pinv_singular_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_qr_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_slogdet_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_solve_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_solve_ex_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_solve_triangular_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_svd_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_svdvals_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_tensorinv_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_tensorsolve_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_vander_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_vecdot_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linalg_vector_norm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linspace_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_linspace_tensor_overload_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log10_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log1p_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log2_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_normal_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_softmax_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_log_softmax_with_dtype_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logaddexp2_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logaddexp_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logcumsumexp_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logdet_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_and_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_not_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_or_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logical_xor_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logit_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logspace_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logspace_tensor_overload_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_logsumexp_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_long_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lt_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lu_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lu_solve_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_lu_unpack_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mH_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mT_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_amax_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_amin_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_argmax_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_argmin_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_cumprod_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_cumsum_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_fill_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_log_softmax_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_logaddexp_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_logsumexp_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_mean_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_median_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_norm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_normalize_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_prod_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_scatter_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_select_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_softmax_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_softmin_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_std_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_sum_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_masked_var_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_matmul_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_matrix_exp_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_binary_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_pool2d_with_indices_backward_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_reduction_no_dim_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_max_reduction_with_dim_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_maximum_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mean_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_median_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_meshgrid_list_of_tensors_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_meshgrid_variadic_tensors_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_min_binary_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_min_reduction_no_dim_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_min_reduction_with_dim_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_minimum_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mode_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_movedim_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_msort_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mul_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_multinomial_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mv_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mvlgamma_mvlgamma_p_1_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mvlgamma_mvlgamma_p_3_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_mvlgamma_mvlgamma_p_5_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nan_to_num_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nanmean_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nanmedian_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nanquantile_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nansum_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_narrow_copy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_narrow_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_native_batch_norm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_native_dropout_backward_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_native_layer_norm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ne_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_neg_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_empty_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_empty_strided_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_full_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_ones_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_new_zeros_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nextafter_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_avg_pool1d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_avg_pool2d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_avg_pool3d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_max_pool1d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_max_pool2d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_adaptive_max_pool3d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_alpha_dropout_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_avg_pool1d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_avg_pool2d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_avg_pool3d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_batch_norm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_bilinear_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_binary_cross_entropy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_binary_cross_entropy_with_logits_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_celu_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_channel_shuffle_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv1d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv2d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv3d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv_transpose1d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv_transpose2d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_conv_transpose3d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_cosine_embedding_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_cosine_similarity_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_cross_entropy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_ctc_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_dropout2d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_dropout3d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_dropout_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_elu_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_embedding_bag_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_embedding_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_feature_alpha_dropout_with_train_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_feature_alpha_dropout_without_train_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_fractional_max_pool2d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_fractional_max_pool3d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_gaussian_nll_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_gelu_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_glu_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_grid_sample_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_group_norm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardshrink_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardsigmoid_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardswish_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hardtanh_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_hinge_embedding_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_huber_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_instance_norm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_area_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_bicubic_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_bilinear_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_linear_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_nearest-exact_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_nearest_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_interpolate_trilinear_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_kl_div_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_l1_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_layer_norm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_leaky_relu_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_linear_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_local_response_norm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_logsigmoid_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_margin_ranking_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_pool1d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_pool2d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_pool3d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool1d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool1d_grad_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool2d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool2d_grad_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool3d_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_max_unpool3d_grad_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_mish_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_mse_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multi_head_attention_forward_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multi_margin_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multilabel_margin_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_multilabel_soft_margin_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_nll_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_normalize_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_circular_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_constant_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_reflect_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_replicate_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pad_replicate_negative_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pairwise_distance_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pdist_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pixel_shuffle_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_pixel_unshuffle_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_poisson_nll_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_prelu_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_relu6_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_relu_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_rms_norm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_rrelu_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_scaled_dot_product_attention_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_selu_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_silu_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_smooth_l1_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_soft_margin_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softmin_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softmin_with_dtype_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softplus_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softshrink_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_softsign_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_tanhshrink_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_threshold_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_triplet_margin_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_triplet_margin_with_distance_loss_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_unfold_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_upsample_bilinear_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nn_functional_upsample_nearest_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nonzero_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_nonzero_static_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_fro_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_inf_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_norm_nuc_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_normal_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_normal_in_place_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_normal_number_mean_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ones_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ones_like_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ormqr_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_outer_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_pca_lowrank_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_permute_copy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_permute_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_pinverse_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polar_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_0_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_1_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_2_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_3_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_polygamma_polygamma_n_4_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_positive_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_pow_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_prod_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_put_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_qr_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_quantile_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rad2deg_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rand_like_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randint_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randint_like_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randn_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_randn_like_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_ravel_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_real_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_reciprocal_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_remainder_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_renorm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_repeat_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_repeat_interleave_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_reshape_as_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_reshape_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resize__cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resize_as__cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resolve_conj_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_resolve_neg_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_roll_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rot90_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_decimals_0_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_decimals_3_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_round_decimals_neg_3_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rsqrt_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_rsub_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scalar_tensor_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_add_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_amax_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_amin_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_mean_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_prod_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_scatter_reduce_sum_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_searchsorted_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_select_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_select_scatter_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sgn_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_short_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sigmoid_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sign_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_bartlett_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_blackman_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_cosine_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_exponential_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_gaussian_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_general_cosine_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_general_hamming_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_hamming_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_hann_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_kaiser_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signal_windows_nuttall_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_signbit_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sin_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sinc_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sinh_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_slice_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_slice_scatter_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_softmax_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_softmax_with_dtype_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sort_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sparse_mm_reduce_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sparse_sampled_addmm_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_airy_ai_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_j0_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_j1_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_y0_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_bessel_y1_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_t_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_u_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_v_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_chebyshev_polynomial_w_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_entr_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_erfcx_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_hermite_polynomial_h_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_hermite_polynomial_he_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_i0e_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_i1_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_i1e_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_laguerre_polynomial_l_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_legendre_polynomial_p_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_log_ndtr_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_i0_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_i1_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_k0_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_modified_bessel_k1_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_ndtr_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_ndtri_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_polygamma_special_polygamma_n_0_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_scaled_modified_bessel_k0_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_scaled_modified_bessel_k1_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_t_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_u_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_v_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_shifted_chebyshev_polynomial_w_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_spherical_bessel_j0_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_xlog1py_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_special_zeta_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_list_args_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_with_sizes_copy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_split_with_sizes_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sqrt_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_square_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_squeeze_copy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_squeeze_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_squeeze_multiple_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_stack_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_mean_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_mean_unbiased_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_std_unbiased_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_stft_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sub_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sum_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_sum_to_size_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_svd_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_svd_lowrank_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_t_copy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_t_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_take_along_dim_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_take_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tan_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tanh_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tensor_split_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tensordot_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tile_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_to_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_to_sparse_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_topk_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_torch_ops_aten__safe_softmax_default_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trace_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_transpose_copy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_transpose_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trapezoid_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trapz_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_triangular_solve_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_tril_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_triu_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_true_divide_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_trunc_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unbind_copy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unbind_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unflatten_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unfold_copy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unfold_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_uniform_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unique_consecutive_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unique_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unsafe_chunk_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unsafe_split_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unsqueeze_copy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_unsqueeze_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_mean_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_mean_unbiased_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_var_unbiased_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_vdot_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_as_complex_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_as_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_copy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_view_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_vsplit_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_vstack_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_where_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_xlogy_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_zero__cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_zeros_cpu_float32, test/distributed/tensor/test_dtensor_ops.py::TestDTensorOpsCPU::test_dtensor_op_db_zeros_like_cpu_float32
2025-03-14T04:39:17.2408460Z 
2025-03-14T04:39:17.2408748Z Running distributed/tensor/test_api 1/1 ... [2025-03-14 04:39:17.238437]
2025-03-14T04:39:17.2409284Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:39:17.2410400Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/test_api.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:39:17.238922]
2025-03-14T04:39:55.9392746Z 
2025-03-14T04:39:55.9394254Z distributed/tensor/test_api 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.test_api_1.1_eba86e128ad99438_.log
2025-03-14T04:39:55.9401191Z Running 8 items in this shard: test/distributed/tensor/test_api.py::DTensorAPITest::test_distribute_module, test/distributed/tensor/test_api.py::DTensorAPITest::test_distribute_module_casting, test/distributed/tensor/test_api.py::DTensorAPITest::test_distribute_module_input_fn_output_fn, test/distributed/tensor/test_api.py::DTensorAPITest::test_distribute_module_input_fn_output_fn_warning, test/distributed/tensor/test_api.py::DTensorAPITest::test_distribute_module_meta, test/distributed/tensor/test_api.py::DTensorAPITest::test_distribute_tensor_errors, test/distributed/tensor/test_api.py::DTensorAPITest::test_distribute_tensor_rank, test/distributed/tensor/test_api.py::DTensorAPITest::test_distribute_tensor_uneven_sharding
2025-03-14T04:39:55.9405796Z 
2025-03-14T04:39:55.9406127Z Running distributed/checkpoint/test_format_utils 1/1 ... [2025-03-14 04:39:55.939776]
2025-03-14T04:39:55.9406694Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:39:55.9408008Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/checkpoint/test_format_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:39:55.940282]
2025-03-14T04:40:13.0445703Z 
2025-03-14T04:40:13.0447520Z distributed/checkpoint/test_format_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_format_utils_1.1_651c37566ec63c02_.log
2025-03-14T04:40:13.0451747Z Running 3 items in this shard: test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_dcp_to_torch_save, test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_online_torch_save_to_dcp, test/distributed/checkpoint/test_format_utils.py::TestFormatUtils::test_torch_save_to_dcp
2025-03-14T04:40:13.0454198Z 
2025-03-14T04:40:13.0455528Z Running distributed/tensor/test_redistribute 1/1 ... [2025-03-14 04:40:13.045355]
2025-03-14T04:40:13.0456431Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:40:13.0462734Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/test_redistribute.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:40:13.045929]
2025-03-14T04:41:19.0599168Z 
2025-03-14T04:41:19.0600967Z distributed/tensor/test_redistribute 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.test_redistribute_1.1_aa709b9c5ee4585d_.log
2025-03-14T04:41:19.0614325Z Running 13 items in this shard: test/distributed/tensor/test_redistribute.py::RedistributeTest::test_partial_to_replicate_forward_backward, test/distributed/tensor/test_redistribute.py::RedistributeTest::test_partial_to_shard, test/distributed/tensor/test_redistribute.py::RedistributeTest::test_redistribute_negative_shard_dim, test/distributed/tensor/test_redistribute.py::RedistributeTest::test_redistribute_shard_dim_change, test/distributed/tensor/test_redistribute.py::RedistributeTest::test_redistribute_uneven_sharding, test/distributed/tensor/test_redistribute.py::RedistributeTest::test_replicate_to_local_partial_grad, test/distributed/tensor/test_redistribute.py::RedistributeTest::test_replicate_to_partial, test/distributed/tensor/test_redistribute.py::RedistributeTest::test_replicate_to_replicate_forward_backward, test/distributed/tensor/test_redistribute.py::RedistributeTest::test_replicate_to_shard_forward_backward, test/distributed/tensor/test_redistribute.py::RedistributeTest::test_shard_dim_alltoall, test/distributed/tensor/test_redistribute.py::RedistributeTest::test_shard_to_replicate_forward_backward, test/distributed/tensor/test_redistribute.py::MultiDimRedistributeTest::test_multi_dim_mesh, test/distributed/tensor/test_redistribute.py::MultiDimRedistributeTest::test_redistribute_shard_dim_multi_dim_mesh
2025-03-14T04:41:19.0625992Z 
2025-03-14T04:41:19.0626567Z Running distributed/tensor/parallel/test_tp_random_state 1/1 ... [2025-03-14 04:41:19.061979]
2025-03-14T04:41:19.0627495Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:41:19.0629648Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/parallel/test_tp_random_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:41:19.062576]
2025-03-14T04:41:27.4493668Z 
2025-03-14T04:41:27.4495863Z distributed/tensor/parallel/test_tp_random_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_tp_random_state_1.1_6c2442b76039efb4_.log
2025-03-14T04:41:27.4498889Z Running 1 items in this shard: test/distributed/tensor/parallel/test_tp_random_state.py::TensorParallelRandomStateTests::test_model_init
2025-03-14T04:41:27.4500131Z 
2025-03-14T04:41:27.4503740Z Running distributed/tensor/debug/test_comm_mode_features 1/1 ... [2025-03-14 04:41:27.450137]
2025-03-14T04:41:27.4505791Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:41:27.4510588Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/debug/test_comm_mode_features.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:41:27.450714]
2025-03-14T04:41:49.0645610Z 
2025-03-14T04:41:49.0646874Z distributed/tensor/debug/test_comm_mode_features 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.debug.test_comm_mode_features_1.1_b8811ab911863edb_.log
2025-03-14T04:41:49.0650017Z Running 4 items in this shard: test/distributed/tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_MLPStacked_distributed_sharding_display, test/distributed/tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_MLP_distributed_sharding_display, test/distributed/tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_MLP_module_tracing, test/distributed/tensor/debug/test_comm_mode_features.py::TestCommModeFeatures::test_transformer_module_tracing
2025-03-14T04:41:49.0652709Z 
2025-03-14T04:41:49.0652992Z Running distributed/_tools/test_mem_tracker 1/1 ... [2025-03-14 04:41:49.065075]
2025-03-14T04:41:49.0653522Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:41:49.0657538Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_tools/test_mem_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:41:49.065532]
2025-03-14T04:41:53.1933092Z 
2025-03-14T04:41:53.1934186Z distributed/_tools/test_mem_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_mem_tracker_1.1_c6d105d6eefdc61e_.log
2025-03-14T04:41:53.1936307Z Running 3 items in this shard: test/distributed/_tools/test_mem_tracker.py::TestMemTracker::test_cuda_tracker_equivalence, test/distributed/_tools/test_mem_tracker.py::TestMemTracker::test_tracker_attribution, test/distributed/_tools/test_mem_tracker.py::TestMemTracker::test_tracker_with_activation_checkpointing
2025-03-14T04:41:53.1937700Z 
2025-03-14T04:41:53.1939876Z Running distributed/pipelining/test_microbatch 1/1 ... [2025-03-14 04:41:53.193794]
2025-03-14T04:41:53.1940528Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:41:53.1944858Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/pipelining/test_microbatch.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:41:53.194257]
2025-03-14T04:41:57.4223352Z 
2025-03-14T04:41:57.4224778Z distributed/pipelining/test_microbatch 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_microbatch_1.1_7aac26247d4f7fc3_.log
2025-03-14T04:41:57.4226769Z Running 2 items in this shard: test/distributed/pipelining/test_microbatch.py::MicrobatchTestsCPU::test_chunk_spec_cpu, test/distributed/pipelining/test_microbatch.py::MicrobatchTestsCPU::test_split_and_merge_cpu
2025-03-14T04:41:57.4227889Z 
2025-03-14T04:41:57.4232858Z Running distributed/tensor/experimental/test_local_map 1/1 ... [2025-03-14 04:41:57.423074]
2025-03-14T04:41:57.4233738Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:41:57.4238446Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/experimental/test_local_map.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:41:57.423603]
2025-03-14T04:42:17.6257735Z 
2025-03-14T04:42:17.6260547Z distributed/tensor/experimental/test_local_map 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.experimental.test_local_map_1.1_bdda98c24cbb0e5d_.log
2025-03-14T04:42:17.6263957Z Running 4 items in this shard: test/distributed/tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_correctness, test/distributed/tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_in_placements, test/distributed/tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_out_placements, test/distributed/tensor/experimental/test_local_map.py::TestLocalMap::test_local_map_redistribute
2025-03-14T04:42:17.6266361Z 
2025-03-14T04:42:17.6266692Z Running distributed/tensor/parallel/test_tp_style 1/1 ... [2025-03-14 04:42:17.626298]
2025-03-14T04:42:17.6267304Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:42:17.6271653Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/parallel/test_tp_style.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:42:17.626793]
2025-03-14T04:43:01.4480752Z 
2025-03-14T04:43:01.4481922Z distributed/tensor/parallel/test_tp_style 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_tp_style_1.1_5575f44de7731ebd_.log
2025-03-14T04:43:01.4487574Z Running 9 items in this shard: test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_colwise_parallel_embedding, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_colwise_parallel_style, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_prepare_module_input, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_prepare_module_input_multiple_inputs, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_prepare_module_kwargs_input, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_prepare_module_output, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_rowwise_parallel_embedding, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_rowwise_parallel_style, test/distributed/tensor/parallel/test_tp_style.py::TensorParallelStyleTest::test_sequence_parallel_style
2025-03-14T04:43:01.4492339Z 
2025-03-14T04:43:01.4492738Z Running distributed/_composable/fsdp/test_fully_shard_ignore_params 1/1 ... [2025-03-14 04:43:01.448537]
2025-03-14T04:43:01.4493363Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:43:01.4494617Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_ignore_params.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:43:01.448975]
2025-03-14T04:43:09.5321605Z 
2025-03-14T04:43:09.5323281Z distributed/_composable/fsdp/test_fully_shard_ignore_params 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_ignore_params_1.1_75ba17fd6329681a_.log
2025-03-14T04:43:09.5324958Z Running 1 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_ignore_params.py::TestFullyShardIgnoreParams::test_ddp_A_fsdp_B_ddp_C
2025-03-14T04:43:09.5325669Z 
2025-03-14T04:43:09.5327750Z Running distributed/checkpoint/e2e/test_pipeline 1/1 ... [2025-03-14 04:43:09.532575]
2025-03-14T04:43:09.5328601Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:43:09.5332691Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/checkpoint/e2e/test_pipeline.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:43:09.533022]
2025-03-14T04:43:13.7108323Z 
2025-03-14T04:43:13.7110141Z distributed/checkpoint/e2e/test_pipeline 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.e2e.test_pipeline_1.1_c1219c66ec0d3c75_.log
2025-03-14T04:43:13.7111795Z Running 1 items in this shard: test/distributed/checkpoint/e2e/test_pipeline.py::TestPipeline::test_pipeline
2025-03-14T04:43:13.7112321Z 
2025-03-14T04:43:13.7115133Z Running distributed/pipelining/test_unflatten 1/1 ... [2025-03-14 04:43:13.711311]
2025-03-14T04:43:13.7115693Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:43:13.7121022Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/pipelining/test_unflatten.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:43:13.711828]
2025-03-14T04:43:18.0400032Z 
2025-03-14T04:43:18.0401205Z distributed/pipelining/test_unflatten 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_unflatten_1.1_d76f02edf1c24a5e_.log
2025-03-14T04:43:18.0402589Z Running 1 items in this shard: test/distributed/pipelining/test_unflatten.py::UnflattenTestsCPU::test_unflatten_cpu
2025-03-14T04:43:18.0403157Z 
2025-03-14T04:43:18.0407108Z Running distributed/checkpoint/test_tp_checkpoint 1/1 ... [2025-03-14 04:43:18.040518]
2025-03-14T04:43:18.0407734Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:43:18.0412114Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/checkpoint/test_tp_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:43:18.040962]
2025-03-14T04:43:31.0872137Z 
2025-03-14T04:43:31.0873622Z distributed/checkpoint/test_tp_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_tp_checkpoint_1.1_eb09c03af9bc6620_.log
2025-03-14T04:43:31.0875531Z Running 2 items in this shard: test/distributed/checkpoint/test_tp_checkpoint.py::TestTpCheckpoint::test_tp_checkpoint, test/distributed/checkpoint/test_tp_checkpoint.py::TestTpCheckpoint::test_tp_checkpoint_load_on_meta_device
2025-03-14T04:43:31.0876680Z 
2025-03-14T04:43:31.0877881Z Running distributed/_tools/test_runtime_estimator 1/1 ... [2025-03-14 04:43:31.087615]
2025-03-14T04:43:31.0878410Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:43:31.0882697Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_tools/test_runtime_estimator.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:43:31.088055]
2025-03-14T04:43:35.2159051Z 
2025-03-14T04:43:35.2160885Z distributed/_tools/test_runtime_estimator 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_runtime_estimator_1.1_cd1645da6fbd8d4c_.log
2025-03-14T04:43:35.2164313Z Running 2 items in this shard: test/distributed/_tools/test_runtime_estimator.py::TestRuntimeEstimator::test_conv_model_runtime, test/distributed/_tools/test_runtime_estimator.py::TestRuntimeEstimator::test_transformer_runtime
2025-03-14T04:43:35.2166038Z 
2025-03-14T04:43:35.2166802Z Running distributed/checkpoint/test_fsspec 1/1 ... [2025-03-14 04:43:35.216481]
2025-03-14T04:43:35.2167688Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:43:35.2174047Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/checkpoint/test_fsspec.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:43:35.217056]
2025-03-14T04:43:39.3949238Z 
2025-03-14T04:43:39.3951133Z distributed/checkpoint/test_fsspec 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_fsspec_1.1_a32523780ff57360_.log
2025-03-14T04:43:39.3955052Z Running 3 items in this shard: test/distributed/checkpoint/test_fsspec.py::TestFSSpec::test_fsspec, test/distributed/checkpoint/test_fsspec.py::TestFSSpec::test_overwrite, test/distributed/checkpoint/test_fsspec.py::TestFileSystem::test_remove_on_fail
2025-03-14T04:43:39.3957123Z 
2025-03-14T04:43:39.3959191Z Running distributed/tensor/test_xla_integration 1/1 ... [2025-03-14 04:43:39.395688]
2025-03-14T04:43:39.3960079Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:43:39.3966092Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/test_xla_integration.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:43:39.396260]
2025-03-14T04:43:43.5241037Z 
2025-03-14T04:43:43.5242823Z distributed/tensor/test_xla_integration 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.test_xla_integration_1.1_8d26803beefcea73_.log
2025-03-14T04:43:43.5247589Z Running 3 items in this shard: test/distributed/tensor/test_xla_integration.py::DTensorXLAIntegrationTest::test_xla_distribute_tensor_1d_replicate, test/distributed/tensor/test_xla_integration.py::DTensorXLAIntegrationTest::test_xla_distribute_tensor_1d_shard, test/distributed/tensor/test_xla_integration.py::DTensorXLAIntegrationTest::test_xla_distribute_tensor_2d
2025-03-14T04:43:43.5250560Z 
2025-03-14T04:43:43.5251329Z Running distributed/tensor/test_convolution_ops 1/1 ... [2025-03-14 04:43:43.524882]
2025-03-14T04:43:43.5251936Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:43:43.5256300Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/test_convolution_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:43:43.525409]
2025-03-14T04:44:15.7552658Z 
2025-03-14T04:44:15.7554407Z distributed/tensor/test_convolution_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.test_convolution_ops_1.1_a68aa596e20eec3b_.log
2025-03-14T04:44:15.7558569Z Running 3 items in this shard: test/distributed/tensor/test_convolution_ops.py::DistConvolutionOpsTest::test_conv_backward_none_grad_inp, test/distributed/tensor/test_convolution_ops.py::DistConvolutionOpsTest::test_depthwise_convolution, test/distributed/tensor/test_convolution_ops.py::DistConvolutionOpsTest::test_downsampling_convolution
2025-03-14T04:44:15.7561374Z 
2025-03-14T04:44:15.7561850Z Running distributed/_tools/test_fsdp2_mem_tracker 1/1 ... [2025-03-14 04:44:15.755771]
2025-03-14T04:44:15.7562645Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:44:15.7565552Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_tools/test_fsdp2_mem_tracker.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:44:15.756304]
2025-03-14T04:44:20.0850886Z 
2025-03-14T04:44:20.0852002Z distributed/_tools/test_fsdp2_mem_tracker 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_fsdp2_mem_tracker_1.1_754fd7513841c48a_.log
2025-03-14T04:44:20.0854648Z Running 3 items in this shard: test/distributed/_tools/test_fsdp2_mem_tracker.py::TestTrackerFullyShard1DTrainingCore::test_tracker_multi_group_eager, test/distributed/_tools/test_fsdp2_mem_tracker.py::TestTrackerFullyShard1DTrainingCore::test_tracker_non_root_forward_backward, test/distributed/_tools/test_fsdp2_mem_tracker.py::TestTrackerFullyShard1DTrainingCompose::test_tracker_with_activation_checkpointing
2025-03-14T04:44:20.0871598Z 
2025-03-14T04:44:20.0872055Z Running distributed/tensor/test_embedding_ops 1/1 ... [2025-03-14 04:44:20.085588]
2025-03-14T04:44:20.0872798Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:44:20.0886448Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/test_embedding_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:44:20.086050]
2025-03-14T04:44:42.3071104Z 
2025-03-14T04:44:42.3072253Z distributed/tensor/test_embedding_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.test_embedding_ops_1.1_8eae67bed32f5213_.log
2025-03-14T04:44:42.3074999Z Running 4 items in this shard: test/distributed/tensor/test_embedding_ops.py::TestEmbeddingOp::test_multiple_embeddings_rowwise, test/distributed/tensor/test_embedding_ops.py::TestEmbeddingOp::test_sharded_embedding_colwise, test/distributed/tensor/test_embedding_ops.py::TestEmbeddingOp::test_sharded_embedding_colwise_max_norm_errors, test/distributed/tensor/test_embedding_ops.py::TestEmbeddingOp::test_sharded_embedding_rowwise
2025-03-14T04:44:42.3077088Z 
2025-03-14T04:44:42.3077400Z Running distributed/test_functional_api 1/1 ... [2025-03-14 04:44:42.307543]
2025-03-14T04:44:42.3077909Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:44:42.3082835Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/test_functional_api.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:44:42.308014]
2025-03-14T04:44:50.8424105Z 
2025-03-14T04:44:50.8425276Z distributed/test_functional_api 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_functional_api_1.1_36d899f2a174fc29_.log
2025-03-14T04:44:50.8427671Z Running 2 items in this shard: test/distributed/test_functional_api.py::TestMetaCollectives::test_all_reduce, test/distributed/test_functional_api.py::TestMakeFx::test_all_reduce_tracing
2025-03-14T04:44:50.8440074Z 
2025-03-14T04:44:50.8440371Z Running distributed/tensor/test_utils 1/1 ... [2025-03-14 04:44:50.842785]
2025-03-14T04:44:50.8440886Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:44:50.8442041Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/test_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:44:50.843213]
2025-03-14T04:45:50.3857113Z 
2025-03-14T04:45:50.3858421Z distributed/tensor/test_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.test_utils_1.1_81b5fff5086a8b1f_.log
2025-03-14T04:45:50.3864110Z Running 10 items in this shard: test/distributed/tensor/test_utils.py::UtilTest::test_compute_local_shape_and_global_offset_1D, test/distributed/tensor/test_utils.py::UtilTest::test_compute_local_shape_and_global_offset_2D, test/distributed/tensor/test_utils.py::UtilTest::test_fsdp_tp_meta_compute, test/distributed/tensor/test_utils.py::UtilTest::test_hsdp_tp_meta_compute, test/distributed/tensor/test_utils.py::UtilTest::test_strided_sharding_assumption_in_meta_compute, test/distributed/tensor/test_utils.py::TestStridedSharding::test_1d_mesh_strided_sharding, test/distributed/tensor/test_utils.py::TestStridedSharding::test_2d_mesh_2d_tensor_strided_sharding, test/distributed/tensor/test_utils.py::TestStridedSharding::test_2d_mesh_strided_sharding, test/distributed/tensor/test_utils.py::Test2DStridedLocalShard::test_fsdp1_tp_2d_dtensor_local_shards_and_offsets, test/distributed/tensor/test_utils.py::Test2DStridedLocalShard::test_fsdp2_tp_2d_dtensor_local_shards_and_offsets
2025-03-14T04:45:50.3870041Z 
2025-03-14T04:45:50.3870426Z Running distributed/_composable/fsdp/test_fully_shard_overlap 1/1 ... [2025-03-14 04:45:50.386250]
2025-03-14T04:45:50.3871021Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:45:50.3872246Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_overlap.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:45:50.386708]
2025-03-14T04:45:54.5654081Z 
2025-03-14T04:45:54.5655736Z distributed/_composable/fsdp/test_fully_shard_overlap 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_overlap_1.1_f68fcf7b19f054c4_.log
2025-03-14T04:45:54.5657913Z Running 2 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_overlap.py::TestFullyShardOverlap::test_fully_shard_post_optim_event_overlap, test/distributed/_composable/fsdp/test_fully_shard_overlap.py::TestFullyShardOverlap::test_fully_shard_training_overlap
2025-03-14T04:45:54.5659212Z 
2025-03-14T04:45:54.5661359Z Running distributed/pipelining/test_stage 1/1 ... [2025-03-14 04:45:54.565891]
2025-03-14T04:45:54.5662256Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:45:54.5667622Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/pipelining/test_stage.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:45:54.566426]
2025-03-14T04:45:58.4660013Z 
2025-03-14T04:45:58.4661107Z distributed/pipelining/test_stage 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.pipelining.test_stage_1.1_b2313c4d97ecbca0_.log
2025-03-14T04:45:58.4661982Z 
2025-03-14T04:45:58.4666983Z Running distributed/checkpoint/test_hsdp_checkpoint 1/1 ... [2025-03-14 04:45:58.466486]
2025-03-14T04:45:58.4667552Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:45:58.4672109Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/checkpoint/test_hsdp_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:45:58.466979]
2025-03-14T04:46:19.7326479Z 
2025-03-14T04:46:19.7328114Z distributed/checkpoint/test_hsdp_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_hsdp_checkpoint_1.1_ded21cf0541b4c0e_.log
2025-03-14T04:46:19.7333278Z Running 4 items in this shard: test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_checkpoint_is_even_sharded_model_False, test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_checkpoint_is_even_sharded_model_True, test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_fsdp_checkpoint_conversion_is_even_sharded_model_False, test/distributed/checkpoint/test_hsdp_checkpoint.py::TestHSDPCheckpoint::test_hsdp_fsdp_checkpoint_conversion_is_even_sharded_model_True
2025-03-14T04:46:19.7337446Z 
2025-03-14T04:46:19.7338062Z Running distributed/checkpoint/test_fsdp_model_state 1/1 ... [2025-03-14 04:46:19.733258]
2025-03-14T04:46:19.7339021Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:46:19.7340939Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/checkpoint/test_fsdp_model_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:46:19.733804]
2025-03-14T04:46:32.6305797Z 
2025-03-14T04:46:32.6307015Z distributed/checkpoint/test_fsdp_model_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_fsdp_model_state_1.1_067197f602c0452e_.log
2025-03-14T04:46:32.6309550Z Running 2 items in this shard: test/distributed/checkpoint/test_fsdp_model_state.py::FsdpModelStateCheckpoint::test_fsdp_model_state_no_resharding, test/distributed/checkpoint/test_fsdp_model_state.py::FsdpModelStateCheckpoint::test_fsdp_model_state_with_resharding
2025-03-14T04:46:32.6310942Z 
2025-03-14T04:46:32.6311297Z Running distributed/checkpoint/test_dtensor_checkpoint 1/1 ... [2025-03-14 04:46:32.630880]
2025-03-14T04:46:32.6311877Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:46:32.6317633Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/checkpoint/test_dtensor_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:46:32.631351]
2025-03-14T04:46:41.1231277Z 
2025-03-14T04:46:41.1233241Z distributed/checkpoint/test_dtensor_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_dtensor_checkpoint_1.1_fe717f8e330b9cd9_.log
2025-03-14T04:46:41.1236092Z Running 1 items in this shard: test/distributed/checkpoint/test_dtensor_checkpoint.py::DTensorPlanner::test_distributed_tensor_planner
2025-03-14T04:46:41.1237308Z 
2025-03-14T04:46:41.1240588Z Running distributed/checkpoint/test_planner 1/1 ... [2025-03-14 04:46:41.123818]
2025-03-14T04:46:41.1241462Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:46:41.1247315Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/checkpoint/test_planner.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:46:41.124372]
2025-03-14T04:46:45.3026250Z 
2025-03-14T04:46:45.3027362Z distributed/checkpoint/test_planner 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_planner_1.1_ff298acb17ba064d_.log
2025-03-14T04:46:45.3033668Z Running 13 items in this shard: test/distributed/checkpoint/test_planner.py::TestSavePlan::test_finish_plan_with_caching, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_global_plan, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_global_plan_with_caching, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_load_with_resharding, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_load_with_world_size_diff_by_one, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_local_load_plan, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_local_plan, test/distributed/checkpoint/test_planner.py::TestSavePlan::test_local_plan_with_caching, test/distributed/checkpoint/test_planner.py::TestPlannerHelpers::test_compare_save_plans, test/distributed/checkpoint/test_planner.py::TestPlannerHelpers::test_create_read_item_from_chunks, test/distributed/checkpoint/test_planner.py::TestPlannerHelpers::test_merge_delta_local_plans, test/distributed/checkpoint/test_planner.py::TestLoadPlanner::test_load_different_sizes_throws, test/distributed/checkpoint/test_planner.py::TestLoadPlanner::test_strict
2025-03-14T04:46:45.3039410Z 
2025-03-14T04:46:45.3039758Z Running distributed/checkpoint/test_dtensor_resharding 1/1 ... [2025-03-14 04:46:45.303092]
2025-03-14T04:46:45.3040337Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:46:45.3041555Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/checkpoint/test_dtensor_resharding.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:46:45.303576]
2025-03-14T04:47:20.6441744Z 
2025-03-14T04:47:20.6443003Z distributed/checkpoint/test_dtensor_resharding 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_dtensor_resharding_1.1_eadd69445acf162a_.log
2025-03-14T04:47:20.6449401Z Running 7 items in this shard: test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardPlacementChange::test_1d_to_1d_reshard_placement_change_extensions0, test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardPlacementChange::test_1d_to_1d_reshard_placement_change_extensions1, test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardPlacementChange::test_1d_to_1d_reshard_placement_change_extensions2, test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardPlacementChange::test_2d_to_2d_reshard_placement_change, test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardMeshChange::test_1d_to_2d_reshard_mesh_change, test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardMeshChange::test_2d_to_1d_reshard_mesh_change, test/distributed/checkpoint/test_dtensor_resharding.py::TestDTensorReshardMeshChange::test_dtensor_checkpoint_resharding_with_empty_shard
2025-03-14T04:47:20.6454110Z 
2025-03-14T04:47:20.6454430Z Running distributed/checkpoint/test_compatibility 1/1 ... [2025-03-14 04:47:20.644467]
2025-03-14T04:47:20.6454986Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:47:20.6456175Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/checkpoint/test_compatibility.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:47:20.644955]
2025-03-14T04:47:24.7728594Z 
2025-03-14T04:47:24.7729787Z distributed/checkpoint/test_compatibility 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_compatibility_1.1_48cdfd6b68b80f38_.log
2025-03-14T04:47:24.7732517Z Running 4 items in this shard: test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_metadata, test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_sharded_tensor_dependency, test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_storage_meta, test/distributed/checkpoint/test_compatibility.py::TestDCPCompatbility::test_with_v_2_3
2025-03-14T04:47:24.7734644Z 
2025-03-14T04:47:24.7735028Z Running distributed/fsdp/test_fsdp_memory 1/1 ... [2025-03-14 04:47:24.773322]
2025-03-14T04:47:24.7735548Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:47:24.7739869Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_memory.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:47:24.773759]
2025-03-14T04:47:36.7134402Z 
2025-03-14T04:47:36.7135787Z distributed/fsdp/test_fsdp_memory 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_memory_1.1_4ca8f5593c0e52d8_.log
2025-03-14T04:47:36.7137458Z Running 2 items in this shard: test/distributed/fsdp/test_fsdp_memory.py::TestFSDPMemory::test_fsdp_memory_ckpt_ckpt, test/distributed/fsdp/test_fsdp_memory.py::TestFSDPMemory::test_fsdp_memory_ckpt_no_ckpt
2025-03-14T04:47:36.7138401Z 
2025-03-14T04:47:36.7141098Z Running distributed/tensor/test_op_strategy 1/1 ... [2025-03-14 04:47:36.713900]
2025-03-14T04:47:36.7141726Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:47:36.7146455Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/test_op_strategy.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:47:36.714379]
2025-03-14T04:47:40.9936360Z 
2025-03-14T04:47:40.9937432Z distributed/tensor/test_op_strategy 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.test_op_strategy_1.1_55f516f9033d309d_.log
2025-03-14T04:47:40.9944159Z Running 15 items in this shard: test/distributed/tensor/test_op_strategy.py::TestEinsumDims::test_batch_dims, test/distributed/tensor/test_op_strategy.py::TestEinsumDims::test_bmm_dims, test/distributed/tensor/test_op_strategy.py::TestEinsumDims::test_free_dims, test/distributed/tensor/test_op_strategy.py::TestEinsumDims::test_mm_dims, test/distributed/tensor/test_op_strategy.py::TestEinsumStrategies::test_bmm_1d_mesh, test/distributed/tensor/test_op_strategy.py::TestEinsumStrategies::test_bmm_2d_mesh, test/distributed/tensor/test_op_strategy.py::TestEinsumStrategies::test_linearity_1d_mesh, test/distributed/tensor/test_op_strategy.py::TestEinsumStrategies::test_mm_1d_mesh, test/distributed/tensor/test_op_strategy.py::TestEinsumStrategies::test_mm_2d_mesh, test/distributed/tensor/test_op_strategy.py::TestEinsumStrategies::test_pointwise_1d_mesh, test/distributed/tensor/test_op_strategy.py::TestCostModel::test_bmm_strategies, test/distributed/tensor/test_op_strategy.py::TestCostModel::test_mm_strategies, test/distributed/tensor/test_op_strategy.py::TestCostModel::test_redistribute_cost_latency, test/distributed/tensor/test_op_strategy.py::TestCostModel::test_redistribute_cost_mesh_1d, test/distributed/tensor/test_op_strategy.py::TestCostModel::test_redistribute_cost_mesh_2d
2025-03-14T04:47:40.9950402Z 
2025-03-14T04:47:40.9950785Z Running distributed/_composable/fsdp/test_fully_shard_grad_scaler 1/1 ... [2025-03-14 04:47:40.993915]
2025-03-14T04:47:40.9951395Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:47:40.9952645Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_grad_scaler.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:47:40.994404]
2025-03-14T04:47:48.9785907Z 
2025-03-14T04:47:48.9787180Z distributed/_composable/fsdp/test_fully_shard_grad_scaler 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_grad_scaler_1.1_2aa7ecebf5c5e5fe_.log
2025-03-14T04:47:48.9788839Z Running 1 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_grad_scaler.py::TestFullyShardGradientScaler::test_gradient_scaler
2025-03-14T04:47:48.9789531Z 
2025-03-14T04:47:48.9791236Z Running distributed/test_fake_pg 1/1 ... [2025-03-14 04:47:48.978954]
2025-03-14T04:47:48.9791732Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:47:48.9796492Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/test_fake_pg.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:47:48.979422]
2025-03-14T04:47:53.2077708Z 
2025-03-14T04:47:53.2078740Z distributed/test_fake_pg 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_fake_pg_1.1_f7f48e8b565959a8_.log
2025-03-14T04:47:53.2083457Z Running 13 items in this shard: test/distributed/test_fake_pg.py::TestFakePG::test_all_reduce, test/distributed/test_fake_pg.py::TestFakePG::test_allgather, test/distributed/test_fake_pg.py::TestFakePG::test_alltoall, test/distributed/test_fake_pg.py::TestFakePG::test_alltoall_base, test/distributed/test_fake_pg.py::TestFakePG::test_broadcast, test/distributed/test_fake_pg.py::TestFakePG::test_construct_fsdp, test/distributed/test_fake_pg.py::TestFakePG::test_fake_pg_tracing, test/distributed/test_fake_pg.py::TestFakePG::test_fsdp_fake_e2e, test/distributed/test_fake_pg.py::TestFakePG::test_fsdp_tp_fake_e2e, test/distributed/test_fake_pg.py::TestFakePG::test_recv, test/distributed/test_fake_pg.py::TestFakePG::test_reduce_scatter, test/distributed/test_fake_pg.py::TestFakePG::test_scatter, test/distributed/test_fake_pg.py::TestFakePG::test_send
2025-03-14T04:47:53.2087365Z 
2025-03-14T04:47:53.2087797Z Running distributed/_composable/test_replicate 1/1 ... [2025-03-14 04:47:53.208205]
2025-03-14T04:47:53.2088430Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:47:53.2089882Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_composable/test_replicate.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:47:53.208659]
2025-03-14T04:49:05.8416524Z 
2025-03-14T04:49:05.8417648Z distributed/_composable/test_replicate 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_replicate_1.1_9c1adc9c8605f1e0_.log
2025-03-14T04:49:05.8426717Z Running 17 items in this shard: test/distributed/_composable/test_replicate.py::ReplicateStateDictTest::test_replicate_non_root_multiple_save_load, test/distributed/_composable/test_replicate.py::ReplicateStateDictTest::test_replicate_single_module_save_load, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_device_id, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_ignore_module, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_move_args_kwargs_to_device, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_multi_module, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_single_module, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_with_kwargs, test/distributed/_composable/test_replicate.py::ReplicateTest::test_replicate_wrong_device_id_type, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_device_id, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_fully_shard_init, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_ignore_module, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_move_args_kwargs_to_device, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_multi_module, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_single_module, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_with_kwargs, test/distributed/_composable/test_replicate.py::ReplicateFullyShardInit::test_replicate_wrong_device_id_type
2025-03-14T04:49:05.8434837Z 
2025-03-14T04:49:05.8435152Z Running distributed/optim/test_named_optimizer 1/1 ... [2025-03-14 04:49:05.842162]
2025-03-14T04:49:05.8435694Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:49:05.8436867Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/optim/test_named_optimizer.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:49:05.842663]
2025-03-14T04:49:08.5692649Z 
2025-03-14T04:49:08.5694466Z distributed/optim/test_named_optimizer 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.optim.test_named_optimizer_1.1_f28991977885b79d_.log
2025-03-14T04:49:08.5696139Z 
2025-03-14T04:49:08.5701460Z Running distributed/tensor/test_optimizers 1/1 ... [2025-03-14 04:49:08.569907]
2025-03-14T04:49:08.5702305Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:49:08.5707370Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/test_optimizers.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:49:08.570467]
2025-03-14T04:50:04.0030369Z 
2025-03-14T04:50:04.0033933Z distributed/tensor/test_optimizers 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.test_optimizers_1.1_be26ac985f0a17e9_.log
2025-03-14T04:50:04.0041017Z Running 11 items in this shard: test/distributed/tensor/test_optimizers.py::TestDTensorOptimizer::test_RMSprop_1d_sharding, test/distributed/tensor/test_optimizers.py::TestDTensorOptimizer::test_adadelta_1d_sharding, test/distributed/tensor/test_optimizers.py::TestDTensorOptimizer::test_adagrad_1d_sharding, test/distributed/tensor/test_optimizers.py::TestDTensorOptimizer::test_adam_1d_sharding, test/distributed/tensor/test_optimizers.py::TestDTensorOptimizer::test_adamax_1d_sharding, test/distributed/tensor/test_optimizers.py::TestDTensorOptimizer::test_adamw_1d_sharding, test/distributed/tensor/test_optimizers.py::TestDTensorOptimizer::test_asgd_1d_sharding, test/distributed/tensor/test_optimizers.py::TestDTensorOptimizer::test_nadam_1d_sharding, test/distributed/tensor/test_optimizers.py::TestDTensorOptimizer::test_optimizer_foreach_supported_types_include_DTensor, test/distributed/tensor/test_optimizers.py::TestDTensorOptimizer::test_radam_1d_sharding, test/distributed/tensor/test_optimizers.py::TestDTensorOptimizer::test_sgd_1d_sharding
2025-03-14T04:50:04.0046504Z 
2025-03-14T04:50:04.0046990Z Running distributed/fsdp/test_fsdp_backward_prefetch 1/1 ... [2025-03-14 04:50:04.004090]
2025-03-14T04:50:04.0047563Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:50:04.0048770Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_backward_prefetch.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:50:04.004532]
2025-03-14T04:50:12.1384159Z 
2025-03-14T04:50:12.1386431Z distributed/fsdp/test_fsdp_backward_prefetch 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_backward_prefetch_1.1_ea449d38780daa41_.log
2025-03-14T04:50:12.1389155Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_backward_prefetch.py::TestBackwardPrefetch::test_backward_prefetch
2025-03-14T04:50:12.1390210Z 
2025-03-14T04:50:12.1393189Z Running distributed/checkpoint/test_utils 1/1 ... [2025-03-14 04:50:12.139063]
2025-03-14T04:50:12.1393805Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:50:12.1397824Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/checkpoint/test_utils.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:50:12.139557]
2025-03-14T04:50:16.3202488Z 
2025-03-14T04:50:16.3204251Z distributed/checkpoint/test_utils 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.checkpoint.test_utils_1.1_b6eac901d82948e7_.log
2025-03-14T04:50:16.3214906Z Running 13 items in this shard: test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_dcp_logger, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_flat_data, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_index_hint_ignored_on_equals, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_index_hint_ignored_on_hash, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_init_convert_offset, test/distributed/checkpoint/test_utils.py::TestMedatadaIndex::test_sharded_tensor_lookup, test/distributed/checkpoint/test_utils.py::TestReaderView::testAllRead, test/distributed/checkpoint/test_utils.py::TestReaderView::testLongRead, test/distributed/checkpoint/test_utils.py::TestReaderView::testLongReadinto, test/distributed/checkpoint/test_utils.py::TestReaderView::testShortRead, test/distributed/checkpoint/test_utils.py::TestReaderView::testShortReadinto, test/distributed/checkpoint/test_utils.py::TestDistWrapper::test_gather_object, test/distributed/checkpoint/test_utils.py::TestDistWrapper::test_scatter_object
2025-03-14T04:50:16.3223442Z 
2025-03-14T04:50:16.3223970Z Running distributed/tensor/parallel/test_tp_examples 1/1 ... [2025-03-14 04:50:16.321052]
2025-03-14T04:50:16.3224765Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:50:16.3226568Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/tensor/parallel/test_tp_examples.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:50:16.321660]
2025-03-14T04:51:29.7923522Z 
2025-03-14T04:51:29.7924977Z distributed/tensor/parallel/test_tp_examples 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.tensor.parallel.test_tp_examples_1.1_1ac91a3db32fd90e_.log
2025-03-14T04:51:29.7938968Z Running 16 items in this shard: test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_loss_parallel, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_mlp_inference, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_mlp_training_is_seq_parallel_False_recompute_activation_False, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_mlp_training_is_seq_parallel_True_recompute_activation_False, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_req_grad_float64_thaw_all, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_req_grad_seq_parallel_float32_thaw_all, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_req_grad_seq_parallel_float32_thaw_layers_0_attention_wv__layers_0_feed_forward_w1__layers_1_feed_forward_w2__layers_1_ffn_norm__output__tok_embeddings, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_req_grad_seq_parallel_float32_thaw_layers_1_ffn_norm__norm__output__tok_embeddings, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_req_grad_seq_parallel_float32_thaw_norm__output, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_req_grad_seq_parallel_float32_thaw_norm__output__tok_embeddings, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_req_grad_seq_parallel_float32_thaw_output__tok_embeddings, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_False_float32, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_False_float64, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_True_float32, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_transformer_training_is_seq_parallel_True_float64, test/distributed/tensor/parallel/test_tp_examples.py::DistTensorParallelExampleTest::test_weight_tying
2025-03-14T04:51:29.7950155Z 
2025-03-14T04:51:29.7950441Z Running distributed/test_serialization 1/1 ... [2025-03-14 04:51:29.792824]
2025-03-14T04:51:29.7950944Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:51:29.7952086Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/test_serialization.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:51:29.793284]
2025-03-14T04:51:33.5710743Z 
2025-03-14T04:51:33.5711780Z distributed/test_serialization 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_serialization_1.1_bc7729ab92ac867a_.log
2025-03-14T04:51:33.5716230Z Running 10 items in this shard: test/distributed/test_serialization.py::TestSerialization::test_cuda, test/distributed/test_serialization.py::TestSerialization::test_dtensor, test/distributed/test_serialization.py::TestSerialization::test_nested_tensors, test/distributed/test_serialization.py::TestSerialization::test_python_object, test/distributed/test_serialization.py::TestSerialization::test_scalar_tensor, test/distributed/test_serialization.py::TestSerialization::test_str_utf8, test/distributed/test_serialization.py::TestSerialization::test_strided_tensor, test/distributed/test_serialization.py::TestSerialization::test_tensor_with_offset, test/distributed/test_serialization.py::TestSerialization::test_various_data_types, test/distributed/test_serialization.py::TestSerialization::test_weights_only
2025-03-14T04:51:33.5720301Z 
2025-03-14T04:51:33.5720555Z Running distributed/test_pg_wrapper 1/1 ... [2025-03-14 04:51:33.571361]
2025-03-14T04:51:33.5721043Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:51:33.5722337Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/test_pg_wrapper.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:51:33.571818]
2025-03-14T04:53:25.9575009Z 
2025-03-14T04:53:25.9576593Z distributed/test_pg_wrapper 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_pg_wrapper_1.1_a25960a7545250fd_.log
2025-03-14T04:53:25.9589354Z Running 17 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_coalescing_manager_debug_mode_detail, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_hang, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_detail, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_off, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_debug_level_detail_no_gloo, test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_new_group_no_gloo, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_hang, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode_off, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda_debug_mode, test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_debug_mode
2025-03-14T04:53:25.9598422Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_coalescing_manager_debug_mode_detail
2025-03-14T04:53:25.9599541Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_hang
2025-03-14T04:53:25.9600656Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_detail
2025-03-14T04:53:25.9601882Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collective_shape_mismatch_debug_mode_off
2025-03-14T04:53:25.9603015Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch
2025-03-14T04:53:25.9604128Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_collectives_op_mismatch_debug_mode
2025-03-14T04:53:25.9605248Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_debug_level_detail_no_gloo
2025-03-14T04:53:25.9606568Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupNCCLWrapperTest::test_new_group_no_gloo
2025-03-14T04:53:25.9607551Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_hang
2025-03-14T04:53:25.9608601Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda
2025-03-14T04:53:25.9609769Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_cuda_debug_mode
2025-03-14T04:53:25.9611082Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode
2025-03-14T04:53:25.9612279Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collective_shape_mismatch_debug_mode_off
2025-03-14T04:53:25.9613410Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch
2025-03-14T04:53:25.9614498Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda
2025-03-14T04:53:25.9615646Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_cuda_debug_mode
2025-03-14T04:53:25.9616826Z Running 1 items in this shard: test/distributed/test_pg_wrapper.py::ProcessGroupGlooWrapperTest::test_collectives_op_mismatch_debug_mode
2025-03-14T04:53:25.9617460Z 
2025-03-14T04:53:25.9617814Z Running distributed/_composable/fsdp/test_fully_shard_frozen 1/1 ... [2025-03-14 04:53:25.958415]
2025-03-14T04:53:25.9618394Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:53:25.9619611Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_frozen.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:53:25.958911]
2025-03-14T04:53:30.0369173Z 
2025-03-14T04:53:30.0370568Z distributed/_composable/fsdp/test_fully_shard_frozen 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_frozen_1.1_98db8a7e7c1d584e_.log
2025-03-14T04:53:30.0373305Z Running 3 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_frozen.py::TestFullyShardFrozen::test_multi_forward_mixed_requires_grad, test/distributed/_composable/fsdp/test_fully_shard_frozen.py::TestFullyShardFrozen::test_train_mixed_requires_grad_across_groups, test/distributed/_composable/fsdp/test_fully_shard_frozen.py::TestFullyShardFrozen::test_train_mixed_requires_grad_per_group
2025-03-14T04:53:30.0375560Z 
2025-03-14T04:53:30.0375922Z Running distributed/_composable/fsdp/test_fully_shard_init 1/1 ... [2025-03-14 04:53:30.037352]
2025-03-14T04:53:30.0376514Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:53:30.0381033Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_init.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:53:30.037825]
2025-03-14T04:53:34.2660475Z 
2025-03-14T04:53:34.2661973Z distributed/_composable/fsdp/test_fully_shard_init 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_init_1.1_be095c2f5731700f_.log
2025-03-14T04:53:34.2686378Z Running 40 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardDeviceTensor::test_move_states_to_device_tensor, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardDeviceDTensor::test_move_states_to_device_dtensor_invalid, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardDeviceDTensor::test_move_states_to_device_dtensor_valid, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMeshArg::test_2d_mesh_without_mesh_dim_names, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMeshArg::test_invalid_mesh_ndim, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_duplicate, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_list_of_mlps, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_nested, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_nested_fully_shard_and_replicate, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_modules_single, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_states_list_of_mlps, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_states_nested_fully_shard, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardManagedModulesAndStates::test_managed_states_shared_params_and_buffers, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardParamModuleInfos::test_get_param_module_infos_duplicates, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardParamModuleInfos::test_get_param_module_infos_list_of_mlps, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardParamModuleInfos::test_get_param_module_infos_shared_params, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardedParameterTensor::test_raise_noncontiguous_parameter, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardedParameterTensor::test_raise_scalar_parameter, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardedParameterTensor::test_shard_tensor_parameters, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardedParameterDTensor::test_shard_dtensor_parameters, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_double_lazy_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_is_root, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_module_and_param_fqns, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_fully_shard_multi_module_root, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardLazyInit::test_reset_sharded_param_in_lazy_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_invalid_meta_device_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_meta_device_1d_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_meta_device_2d_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardMetaDeviceInit::test_rank0_broadcast_meta_device_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardProcessGroupInit::test_1d_process_group_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardProcessGroupInit::test_2d_process_group_init, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardHSDPBroadcast::test_hsdp_broadcast_across_replicas, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestHSDPWithCustomHook::test_custom_hook_custom_stream, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestHSDPWithCustomHook::test_custom_hsdp_all_reduce_hook, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardPlacementFn::test_init_1d_transformer_shard_dim_neg1, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardPlacementFn::test_init_1d_transformer_shard_largest_dim, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardPlacementFn::test_init_1d_uneven_shard_largest_dim, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardPlacementFn::test_init_2d_transformer_shard_diff_dim, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardShardPlacementFn::test_invalid_shard_dim, test/distributed/_composable/fsdp/test_fully_shard_init.py::TestFullyShardOldImport::test_old_import_training
2025-03-14T04:53:34.2709312Z 
2025-03-14T04:53:34.2709686Z Running distributed/fsdp/test_fsdp_optim_state 1/1 ... [2025-03-14 04:53:34.266356]
2025-03-14T04:53:34.2710215Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:53:34.2711378Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_optim_state.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:53:34.267606]
2025-03-14T04:57:33.6276437Z 
2025-03-14T04:57:33.6278210Z distributed/fsdp/test_fsdp_optim_state 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_optim_state_1.1_ebe950b107b2e744_.log
2025-03-14T04:57:33.6342802Z Running 60 items in this shard: test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_compatible_with_trec, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_flatten_sharded_optim_state_dict_nested, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_flatten_sharded_optim_state_dict_transformer, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_full_optim_state_dict_keys, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_full_optim_state_dict_nested_invalid, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_interface_arguments, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_no_grad, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_input_warning, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type0_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_False_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_dict_nested_state_dict_type1_use_multiple_param_groups_True_rank0_only_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_optim_state_without_param_groups, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type0_use_multiple_param_groups_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type0_use_multiple_param_groups_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type1_use_multiple_param_groups_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_ids_state_dict_type1_use_multiple_param_groups_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_rekey_optim_state_dict_to_names, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_save_load_without_0th_param_state_state_dict_type0, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_save_load_without_0th_param_state_state_dict_type1, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_halve_world_size, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_scatter_full_optim_state_dict_transformer, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_halve_world_size, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_False_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_False_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_nested_use_multiple_param_groups_True_wrap_alt_True_use_diff_optim_inputs_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_transformer, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type0_add_to_fsdp_module_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type0_add_to_fsdp_module_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type1_add_to_fsdp_module_False, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_shard_full_optim_state_dict_unmanaged_params_state_dict_type1_add_to_fsdp_module_True, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_state_dict_with_none_tensor_state, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_use_orig_params, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_with_empty_optimizer_state, test/distributed/fsdp/test_fsdp_optim_state.py::TestFSDPOptimState::test_with_no_shard
2025-03-14T04:57:33.6386285Z 
2025-03-14T04:57:33.6386613Z Running distributed/fsdp/test_fsdp_grad_acc 1/1 ... [2025-03-14 04:57:33.628564]
2025-03-14T04:57:33.6387143Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:57:33.6388298Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_grad_acc.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:57:33.629159]
2025-03-14T04:58:01.1924347Z 
2025-03-14T04:58:01.1926004Z distributed/fsdp/test_fsdp_grad_acc 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_grad_acc_1.1_74c46386e446db13_.log
2025-03-14T04:58:01.1931990Z Running 6 items in this shard: test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_configs0_use_orig_params_False, test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_configs0_use_orig_params_True, test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_configs1_use_orig_params_False, test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_configs1_use_orig_params_True, test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_cpu_offload_use_orig_params_False, test/distributed/fsdp/test_fsdp_grad_acc.py::TestGradAcc::test_grad_acc_cpu_offload_use_orig_params_True
2025-03-14T04:58:01.1936204Z 
2025-03-14T04:58:01.1936508Z Running distributed/test_multi_threaded_pg 1/1 ... [2025-03-14 04:58:01.192924]
2025-03-14T04:58:01.1937024Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:58:01.1938170Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/test_multi_threaded_pg.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:58:01.193431]
2025-03-14T04:58:05.3719579Z 
2025-03-14T04:58:05.3721691Z distributed/test_multi_threaded_pg 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_multi_threaded_pg_1.1_6a612da2f4e22401_.log
2025-03-14T04:58:05.3740252Z Running 22 items in this shard: test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_all_to_all_single_list, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_all_to_all_single_none, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_all_to_all_single_tensor, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_broadcast_object_list, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_collective_error_on_rank_non_zero, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_collective_error_on_rank_non_zero_all, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_collective_error_on_rank_zero, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithWrapper::test_skip, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_reduce, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_reduce_coalesced, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_reduce_ops, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_all_to_all, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_allgather, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_assert_equal_on_rank, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_broadcast, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_broadcast_object_list, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_bwd_sees_fwd_pg, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_gather, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_reduce_scatter, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_scatter, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_subpg, test/distributed/test_multi_threaded_pg.py::TestCollectivesWithBaseClass::test_using_pg_from_another_thread
2025-03-14T04:58:05.3754860Z 
2025-03-14T04:58:05.3755403Z Running distributed/_composable/test_replicate_with_compiler 1/1 ... [2025-03-14 04:58:05.372858]
2025-03-14T04:58:05.3756284Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:58:05.3758098Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_composable/test_replicate_with_compiler.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:58:05.373450]
2025-03-14T04:59:32.4527605Z 
2025-03-14T04:59:32.4528985Z distributed/_composable/test_replicate_with_compiler 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.test_replicate_with_compiler_1.1_7dfe42292ed47b44_.log
2025-03-14T04:59:32.4534882Z Running 10 items in this shard: test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_bucketing_coalesced_op, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_bucketing_concat_op, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_backward_only, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_bf16, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_cpu, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_cpu_no_sync, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_fp16, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_gpu, test/distributed/_composable/test_replicate_with_compiler.py::ReplicateTest::test_compile_gpu_ac, test/distributed/_composable/test_replicate_with_compiler.py::DDP_TP_Test::test_ddp_tp
2025-03-14T04:59:32.4539657Z 
2025-03-14T04:59:32.4539945Z Running distributed/fsdp/test_fsdp_fine_tune 1/1 ... [2025-03-14 04:59:32.453228]
2025-03-14T04:59:32.4540466Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:59:32.4541619Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_fine_tune.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:59:32.453686]
2025-03-14T04:59:36.5318202Z 
2025-03-14T04:59:36.5319588Z distributed/fsdp/test_fsdp_fine_tune 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_fine_tune_1.1_2c669406e1c2eebd_.log
2025-03-14T04:59:36.5320622Z Running 0 items in this shard:
2025-03-14T04:59:36.5320882Z 
2025-03-14T04:59:36.5324625Z Running distributed/nn/jit/test_instantiator 1/1 ... [2025-03-14 04:59:36.532275]
2025-03-14T04:59:36.5325187Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:59:36.5329683Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/nn/jit/test_instantiator.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:59:36.532726]
2025-03-14T04:59:40.6105946Z 
2025-03-14T04:59:40.6107076Z distributed/nn/jit/test_instantiator 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.nn.jit.test_instantiator_1.1_9660011738600130_.log
2025-03-14T04:59:40.6109471Z Running 3 items in this shard: test/distributed/nn/jit/test_instantiator.py::TestInstantiator::test_get_arg_return_types_from_interface, test/distributed/nn/jit/test_instantiator.py::TestInstantiator::test_instantiate_non_scripted_remote_module_template, test/distributed/nn/jit/test_instantiator.py::TestInstantiator::test_instantiate_scripted_remote_module_template
2025-03-14T04:59:40.6111239Z 
2025-03-14T04:59:40.6113522Z Running distributed/fsdp/test_fsdp_ignored_modules 1/1 ... [2025-03-14 04:59:40.611129]
2025-03-14T04:59:40.6114110Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:59:40.6118450Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_ignored_modules.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:59:40.611606]
2025-03-14T04:59:44.6896731Z 
2025-03-14T04:59:44.6897945Z distributed/fsdp/test_fsdp_ignored_modules 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_ignored_modules_1.1_2e973940520a287a_.log
2025-03-14T04:59:44.6903479Z Running 8 items in this shard: test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_diff_ignored_modules_across_ranks, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_invalid, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_nested, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_False, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_not_under_wrapped_root_ignore_modules_True, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_modules_transformer, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_states_auto_wrap, test/distributed/fsdp/test_fsdp_ignored_modules.py::TestFSDPIgnoredModules::test_ignored_states_check
2025-03-14T04:59:44.6908325Z 
2025-03-14T04:59:44.6908654Z Running distributed/fsdp/test_fsdp_traversal 1/1 ... [2025-03-14 04:59:44.690212]
2025-03-14T04:59:44.6909210Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:59:44.6910493Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_traversal.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:59:44.690687]
2025-03-14T04:59:48.7184653Z 
2025-03-14T04:59:48.7185870Z distributed/fsdp/test_fsdp_traversal 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_traversal_1.1_dd6d83364709a20b_.log
2025-03-14T04:59:48.7186886Z Running 0 items in this shard:
2025-03-14T04:59:48.7187091Z 
2025-03-14T04:59:48.7191543Z Running distributed/_composable/fsdp/test_fully_shard_state_dict 1/1 ... [2025-03-14 04:59:48.718929]
2025-03-14T04:59:48.7192165Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:59:48.7197220Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_state_dict.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:59:48.719413]
2025-03-14T04:59:52.8475349Z 
2025-03-14T04:59:52.8476672Z distributed/_composable/fsdp/test_fully_shard_state_dict 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_state_dict_1.1_10c47f23318b96f0_.log
2025-03-14T04:59:52.8481945Z Running 6 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_2d_state_dict_correctness, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_dp_state_dict_cpu_offload, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_dp_state_dict_save_load, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_dp_tp_state_dict_save_load, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiProcess::test_hsdp_tp_state_dict_save_load, test/distributed/_composable/fsdp/test_fully_shard_state_dict.py::TestFullyShardStateDictMultiThread::test_rank0_offload_full_state_dict
2025-03-14T04:59:52.8486099Z 
2025-03-14T04:59:52.8486447Z Running distributed/algorithms/test_join 1/1 ... [2025-03-14 04:59:52.847997]
2025-03-14T04:59:52.8486975Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T04:59:52.8488224Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/algorithms/test_join.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 04:59:52.848465]
2025-03-14T05:00:32.2291020Z 
2025-03-14T05:00:32.2292254Z distributed/algorithms/test_join 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.algorithms.test_join_1.1_15b2adbba069b553_.log
2025-03-14T05:00:32.2297047Z Running 9 items in this shard: test/distributed/algorithms/test_join.py::TestJoin::test_join_kwargs, test/distributed/algorithms/test_join.py::TestJoin::test_multiple_joinable_disable, test/distributed/algorithms/test_join.py::TestJoin::test_multiple_joinables, test/distributed/algorithms/test_join.py::TestJoin::test_multiple_joinables_throw, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_disable, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_main_hooks, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_post_hooks, test/distributed/algorithms/test_join.py::TestJoin::test_single_joinable_throw
2025-03-14T05:00:32.2301371Z 
2025-03-14T05:00:32.2301740Z Running distributed/fsdp/test_fsdp_tp_integration 1/1 ... [2025-03-14 05:00:32.229554]
2025-03-14T05:00:32.2302527Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:00:32.2304465Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_tp_integration.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:00:32.230111]
2025-03-14T05:00:44.0694909Z 
2025-03-14T05:00:44.0696324Z distributed/fsdp/test_fsdp_tp_integration 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_tp_integration_1.1_22b1e11151dfe328_.log
2025-03-14T05:00:44.0698890Z Running 3 items in this shard: test/distributed/fsdp/test_fsdp_tp_integration.py::TestTPFSDPIntegration::test_fsdp_tp_extension_grad, test/distributed/fsdp/test_fsdp_tp_integration.py::TestTPFSDPIntegration::test_fsdp_tp_integration, test/distributed/fsdp/test_fsdp_tp_integration.py::TestTPFSDPIntegration::test_fsdp_tp_sync_module_state
2025-03-14T05:00:44.0700444Z 
2025-03-14T05:00:44.0705064Z Running distributed/rpc/test_faulty_agent 1/1 ... [2025-03-14 05:00:44.070273]
2025-03-14T05:00:44.0705730Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:00:44.0711430Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/rpc/test_faulty_agent.py', '--shard-id=1', '--num-shards=1', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:00:44.070910]
2025-03-14T05:07:37.7157481Z 
2025-03-14T05:07:37.7159171Z distributed/rpc/test_faulty_agent 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_faulty_agent_1.1_9c6b146c252d55da_.log
2025-03-14T05:07:37.7177882Z Running 28 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_check_failed_messages, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_faulty_messages, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_messages_to_delay, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_no_faulty_messages, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_builtin_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_script_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rref_to_here_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout_to_self, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_verify_backend_options, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_context_cleanup_tensor_with_grad, test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_verify_backend_options, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_remote_timeout_to_here_in_jit, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_in_jit, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_script_func, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_to_here_timeout_in_jit, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_python, test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_torchscript_function
2025-03-14T05:07:37.7192298Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout
2025-03-14T05:07:37.7193552Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_builtin_remote_message_dropped_timeout_to_self
2025-03-14T05:07:37.7194724Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_check_failed_messages
2025-03-14T05:07:37.7195779Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_faulty_messages
2025-03-14T05:07:37.7196845Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_custom_messages_to_delay
2025-03-14T05:07:37.7197891Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_no_faulty_messages
2025-03-14T05:07:37.7199184Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout
2025-03-14T05:07:37.7200412Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_builtin_delay_timeout_to_self
2025-03-14T05:07:37.7201620Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle
2025-03-14T05:07:37.7202794Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_dropped_pickle_to_self
2025-03-14T05:07:37.7204002Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout
2025-03-14T05:07:37.7205222Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_remote_message_script_delay_timeout_to_self
2025-03-14T05:07:37.7206363Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_builtin_timeout
2025-03-14T05:07:37.7207382Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rpc_script_timeout
2025-03-14T05:07:37.7208405Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_rref_to_here_timeout
2025-03-14T05:07:37.7209496Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout
2025-03-14T05:07:37.7210682Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_delay_timeout_to_self
2025-03-14T05:07:37.7212002Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout
2025-03-14T05:07:37.7213221Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_udf_remote_message_dropped_timeout_to_self
2025-03-14T05:07:37.7214375Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentRpcTest::test_verify_backend_options
2025-03-14T05:07:37.7215544Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_context_cleanup_tensor_with_grad
2025-03-14T05:07:37.7216842Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyFaultyAgentDistAutogradTest::test_verify_backend_options
2025-03-14T05:07:37.7218013Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_remote_timeout_to_here_in_jit
2025-03-14T05:07:37.7219157Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_in_jit
2025-03-14T05:07:37.7220312Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_timeout_pickle_script_func
2025-03-14T05:07:37.7221491Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_rref_to_here_timeout_in_jit
2025-03-14T05:07:37.7222658Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_python
2025-03-14T05:07:37.7223776Z Running 1 items in this shard: test/distributed/rpc/test_faulty_agent.py::FaultyJitFaultyAgentRpcTest::test_timeout_in_torchscript_function
2025-03-14T05:07:37.7224422Z 
2025-03-14T05:07:37.7224714Z Running distributed/rpc/test_tensorpipe_agent 2/5 ... [2025-03-14 05:07:37.717083]
2025-03-14T05:07:37.7225242Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:07:37.7226538Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/rpc/test_tensorpipe_agent.py', '--shard-id=2', '--num-shards=5', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:07:37.717564]
2025-03-14T05:19:26.3223546Z 
2025-03-14T05:19:26.3224648Z distributed/rpc/test_tensorpipe_agent 2/5 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_tensorpipe_agent_2.5_39cbedd9249bb07c_.log
2025-03-14T05:19:26.3273409Z Running 74 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_method_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_chained_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_legacy, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_expected_src, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_shutdown, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_non_garbage_collected_user_rref_due_to_local_circular_dependency, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_rpc_key_names, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_remote_rpc_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_built_in, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_instance_method, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_multi_async_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_nested_pickle, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rref_args_user_share, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_partial_subset, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_get_future, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_non_exist, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_tensor_self, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_with_error_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_rpc_arg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_self_rpc_arg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_set_and_get_default_rpc_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_shutdown_errors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_stress_light_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rpc_pickler, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rrefs_confirmed, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rrefs_confirmed_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_multiple_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_worker_info_pickle, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_complex_python_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_dtypes, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_tensor_dims, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_round_trips, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_multi, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_nested, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_python_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_no_tensors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_tensor_with_grad, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_error_in_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_copy_sparse_indices_extra_ref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_builtin_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_remote_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_python_remote_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_mixed_requires_grad, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_nested_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_grad_copy_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_rpc_complex_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_all_kwargs_are_populated_by_defaults, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_args_and_kwargs_contain_different_types, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_wrong_return_type_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_script_throw, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_rpc_with_profiling, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_chain, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_record_function_jit_end_callbacks_with_fork, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_return_local_script_module_rref_in_py_and_use_in_script, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_python_annotation, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_restore_context_after_swtich_to_jit_thread, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_bad_module, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rref_stress_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_sparse_tensors_in_container, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_wait_all_workers_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_no_grad_on_tensor_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_multi_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_nested_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backwards_nested_python_udf_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_no_grad_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_with_grad_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_python_call_sparse
2025-03-14T05:19:26.3309030Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_method_remote
2025-03-14T05:19:26.3310152Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_chained_remote
2025-03-14T05:19:26.3311249Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_simple
2025-03-14T05:19:26.3312314Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_legacy
2025-03-14T05:19:26.3313336Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_expected_src
2025-03-14T05:19:26.3314300Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_shutdown
2025-03-14T05:19:26.3315476Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_non_garbage_collected_user_rref_due_to_local_circular_dependency
2025-03-14T05:19:26.3316762Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_rpc_key_names
2025-03-14T05:19:26.3317909Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin_single_threaded
2025-03-14T05:19:26.3319075Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_udf
2025-03-14T05:19:26.3320235Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_remote_rpc_single_threaded
2025-03-14T05:19:26.3321526Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin
2025-03-14T05:19:26.3322551Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_built_in
2025-03-14T05:19:26.3323553Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_class_instance_method
2025-03-14T05:19:26.3324582Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_multi_async_call
2025-03-14T05:19:26.3325647Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_nested_pickle
2025-03-14T05:19:26.3326656Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rref_args_user_share
2025-03-14T05:19:26.3327705Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_partial_subset
2025-03-14T05:19:26.3328725Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_get_future
2025-03-14T05:19:26.3329718Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_non_exist
2025-03-14T05:19:26.3330739Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_proxy_tensor_self
2025-03-14T05:19:26.3331748Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_blocking
2025-03-14T05:19:26.3332785Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_with_error_blocking
2025-03-14T05:19:26.3333864Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_rpc_arg
2025-03-14T05:19:26.3334957Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_self_rpc_arg
2025-03-14T05:19:26.3336069Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_set_and_get_default_rpc_timeout
2025-03-14T05:19:26.3337106Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_shutdown_errors
2025-03-14T05:19:26.3338079Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_stress_light_rpc
2025-03-14T05:19:26.3339049Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rpc_pickler
2025-03-14T05:19:26.3340041Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rrefs_confirmed
2025-03-14T05:19:26.3341078Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_user_rrefs_confirmed_remote
2025-03-14T05:19:26.3342135Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_multiple_call
2025-03-14T05:19:26.3343149Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_worker_info_pickle
2025-03-14T05:19:26.3344236Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_complex_python_udf
2025-03-14T05:19:26.3345488Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_dtypes
2025-03-14T05:19:26.3346685Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_different_tensor_dims
2025-03-14T05:19:26.3347892Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_multiple_round_trips
2025-03-14T05:19:26.3349016Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref
2025-03-14T05:19:26.3350158Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_multi
2025-03-14T05:19:26.3351257Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_rref_nested
2025-03-14T05:19:26.3352391Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_simple_python_udf
2025-03-14T05:19:26.3353624Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_no_tensors
2025-03-14T05:19:26.3354818Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_context_cleanup_tensor_with_grad
2025-03-14T05:19:26.3355975Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_error_in_context
2025-03-14T05:19:26.3357140Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_grad_copy_sparse_indices_extra_ref
2025-03-14T05:19:26.3358333Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_builtin_call
2025-03-14T05:19:26.3359500Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_py_nested_remote_call
2025-03-14T05:19:26.3360703Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_graph_for_python_remote_call
2025-03-14T05:19:26.3361848Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_mixed_requires_grad
2025-03-14T05:19:26.3362919Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_nested_context
2025-03-14T05:19:26.3363995Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_no_grad_copy_sparse
2025-03-14T05:19:26.3365074Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_rpc_complex_args
2025-03-14T05:19:26.3366204Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_all_kwargs_are_populated_by_defaults
2025-03-14T05:19:26.3367407Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_args_and_kwargs_contain_different_types
2025-03-14T05:19:26.3368776Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_wrong_return_type_remote
2025-03-14T05:19:26.3369899Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_script_throw
2025-03-14T05:19:26.3370951Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_call_rpc_with_profiling
2025-03-14T05:19:26.3371979Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_chain
2025-03-14T05:19:26.3373097Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_record_function_jit_end_callbacks_with_fork
2025-03-14T05:19:26.3374259Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_remote_script_udf
2025-03-14T05:19:26.3375451Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_return_local_script_module_rref_in_py_and_use_in_script
2025-03-14T05:19:26.3376661Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_rref_python_annotation
2025-03-14T05:19:26.3377867Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_restore_context_after_swtich_to_jit_thread
2025-03-14T05:19:26.3379119Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_bad_module
2025-03-14T05:19:26.3380240Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_nested_rref_stress_sparse
2025-03-14T05:19:26.3381501Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_sparse_tensors_in_container
2025-03-14T05:19:26.3382802Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_wait_all_workers_sparse
2025-03-14T05:19:26.3384118Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_no_grad_on_tensor_sparse
2025-03-14T05:19:26.3385559Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_multi_sparse
2025-03-14T05:19:26.3386918Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_nested_sparse
2025-03-14T05:19:26.3388310Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backwards_nested_python_udf_sparse
2025-03-14T05:19:26.3389748Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_no_grad_sparse
2025-03-14T05:19:26.3391212Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_context_cleanup_tensor_with_grad_sparse
2025-03-14T05:19:26.3392630Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_python_call_sparse
2025-03-14T05:19:26.3393389Z 
2025-03-14T05:19:26.3393667Z Running distributed/rpc/test_tensorpipe_agent 4/5 ... [2025-03-14 05:19:26.325645]
2025-03-14T05:19:26.3394193Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:19:26.3395417Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/rpc/test_tensorpipe_agent.py', '--shard-id=4', '--num-shards=5', '-v', '--subprocess', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:19:26.326093]
2025-03-14T05:34:06.6454789Z 
2025-03-14T05:34:06.6455942Z distributed/rpc/test_tensorpipe_agent 4/5 was successful, full logs can be found in artifacts with path test/test-reports/distributed.rpc.test_tensorpipe_agent_4.5_60e3af3ab37756ec_.log
2025-03-14T05:34:06.6538523Z Running 92 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_all_gather, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_fanout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_nested, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_nested_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_with_future_ctor, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_async, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_cbs_jit_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_static_method, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_call_method_on_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_chain, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_with_ret, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_in_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_rpc_then_pg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_value_not_on_owner, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_layer_nested_async_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_my_parameter_server, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rref, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rref_stress, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_owner_rref_backward, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_pass_local_rrefs, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_export_trace, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_udf_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_autograd_context_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_builtin_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_sync_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin_single_threaded, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_no_return_result, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_raise_in_user_func_escaped_str, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rref_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors_in_container, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors_multi_async_call, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_register_rpc_backend_and_set_and_start_rpc_backend, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_remote_throw, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_local_rrefs, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_all, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_async_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_remote_record_function, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_forward_chain, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_owner_non_blocking, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_remote_arg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_self_remote_arg, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_server_process_global_profiler, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rref_after_shutdown, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_raise_in_body, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_node_failure_python_udf, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_without_rpc, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_debug_info, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_dist_autograd_profiling, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_trainer_ps_torchscript_functions, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_exception_on_constructor, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_simple, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_create_local_script_class_rref_in_py, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_future_passed_between_python_and_jit, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_less_than_needed_args_are_specified, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_more_than_needed_args_are_specified, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_user_rrefs_confirmed_remote, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_jit_fork_within_context, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_send_remote_module_with_a_new_attribute_not_pickled_over_the_wire, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_inside, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_comparison, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_comparison_uneven_inputs, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_builtin_remote_ret_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_op_with_invalid_args, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_user_share_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_py_udf_remote_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_rpc_arg_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_self_rpc_arg_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_set_and_get_num_worker_threads, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_tensorpipe_options_throw_on_timedelta_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_tensorpipe_set_default_timeout, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_self_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_remote_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_remote_call_itself_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_remote_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_python_remote_call_sparse, test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_rpc_complex_args_sparse
2025-03-14T05:34:06.6594315Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_all_gather
2025-03-14T05:34:06.6595351Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy_async
2025-03-14T05:34:06.6596602Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_class_rref_proxy_remote
2025-03-14T05:34:06.6597694Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_multi_fanout
2025-03-14T05:34:06.6598928Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_nested
2025-03-14T05:34:06.6600001Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_nested_remote
2025-03-14T05:34:06.6601099Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_with_future_ctor
2025-03-14T05:34:06.6602247Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_async
2025-03-14T05:34:06.6603434Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_function_wrong_return_type_remote
2025-03-14T05:34:06.6604602Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_record_function_cbs_jit_call
2025-03-14T05:34:06.6605669Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_async_static_method
2025-03-14T05:34:06.6606668Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_call_method_on_rref
2025-03-14T05:34:06.6607644Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_chain
2025-03-14T05:34:06.6608606Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_simple
2025-03-14T05:34:06.6609580Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_callback_with_ret
2025-03-14T05:34:06.6610770Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_future_in_rpc
2025-03-14T05:34:06.6611737Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_init_rpc_then_pg
2025-03-14T05:34:06.6612815Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_local_value_not_on_owner
2025-03-14T05:34:06.6614332Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_multi_layer_nested_async_rpc
2025-03-14T05:34:06.6615374Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_my_parameter_server
2025-03-14T05:34:06.6616343Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rpc
2025-03-14T05:34:06.6617282Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rref
2025-03-14T05:34:06.6618258Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_nested_rref_stress
2025-03-14T05:34:06.6619252Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_owner_rref_backward
2025-03-14T05:34:06.6620242Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_pass_local_rrefs
2025-03-14T05:34:06.6621411Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_export_trace
2025-03-14T05:34:06.6622479Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_builtin
2025-03-14T05:34:06.6623643Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_async_rpc_udf_single_threaded
2025-03-14T05:34:06.6624991Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_autograd_context_single_threaded
2025-03-14T05:34:06.6626455Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_remote_builtin_single_threaded
2025-03-14T05:34:06.6627620Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_script_sync_rpc
2025-03-14T05:34:06.6628794Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_profiler_with_sync_rpc_builtin_single_threaded
2025-03-14T05:34:06.6629914Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_no_return_result
2025-03-14T05:34:06.6630982Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_raise_in_user_func_escaped_str
2025-03-14T05:34:06.6632031Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_rref_args
2025-03-14T05:34:06.6632981Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors
2025-03-14T05:34:06.6633968Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors_in_container
2025-03-14T05:34:06.6635026Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_py_tensors_multi_async_call
2025-03-14T05:34:06.6636203Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_register_rpc_backend_and_set_and_start_rpc_backend
2025-03-14T05:34:06.6637316Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_remote_throw
2025-03-14T05:34:06.6638273Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future
2025-03-14T05:34:06.6639255Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_future_remote
2025-03-14T05:34:06.6640252Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_return_local_rrefs
2025-03-14T05:34:06.6641229Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_barrier_all
2025-03-14T05:34:06.6642251Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_async_function
2025-03-14T05:34:06.6643370Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rpc_profiling_remote_record_function
2025-03-14T05:34:06.6644443Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_forward_chain
2025-03-14T05:34:06.6645510Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_timeout
2025-03-14T05:34:06.6646534Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_rref_type_owner_non_blocking
2025-03-14T05:34:06.6647618Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_remote_arg
2025-03-14T05:34:06.6648737Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_self_remote_rref_as_self_remote_arg
2025-03-14T05:34:06.6649940Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_server_process_global_profiler
2025-03-14T05:34:06.6651015Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_use_rref_after_shutdown
2025-03-14T05:34:06.6651988Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all
2025-03-14T05:34:06.6652960Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRpcTest::test_wait_all_raise_in_body
2025-03-14T05:34:06.6654139Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_node_failure_python_udf
2025-03-14T05:34:06.6655305Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_backward_without_rpc
2025-03-14T05:34:06.6656363Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_debug_info
2025-03-14T05:34:06.6657429Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_dist_autograd_profiling
2025-03-14T05:34:06.6658603Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistAutogradTest::test_trainer_ps_torchscript_functions
2025-03-14T05:34:06.6659852Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDistOptimizerTest::test_dist_optim_exception_on_constructor
2025-03-14T05:34:06.6661014Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_async_function_simple
2025-03-14T05:34:06.6662035Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_callback_simple
2025-03-14T05:34:06.6663132Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_create_local_script_class_rref_in_py
2025-03-14T05:34:06.6664334Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_future_passed_between_python_and_jit
2025-03-14T05:34:06.6665621Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_less_than_needed_args_are_specified
2025-03-14T05:34:06.6666804Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_more_than_needed_args_are_specified
2025-03-14T05:34:06.6667950Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitRpcTest::test_user_rrefs_confirmed_remote
2025-03-14T05:34:06.6669085Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeJitDistAutogradTest::test_jit_fork_within_context
2025-03-14T05:34:06.6670418Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeRemoteModuleTest::test_send_remote_module_with_a_new_attribute_not_pickled_over_the_wire
2025-03-14T05:34:06.6671860Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeThreeWorkersRemoteModuleTest::test_send_remote_module_over_the_wire
2025-03-14T05:34:06.6673138Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpUnderDistAutogradTest::test_backward_ddp_inside
2025-03-14T05:34:06.6674262Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_comparison
2025-03-14T05:34:06.6675400Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeDdpComparisonTest::test_ddp_comparison_uneven_inputs
2025-03-14T05:34:06.6676620Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_builtin_remote_ret_sparse
2025-03-14T05:34:06.6677819Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_op_with_invalid_args
2025-03-14T05:34:06.6679059Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_sparse
2025-03-14T05:34:06.6680281Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_py_rref_args_user_share_sparse
2025-03-14T05:34:06.6681563Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_py_udf_remote_sparse
2025-03-14T05:34:06.6682890Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_rpc_arg_sparse
2025-03-14T05:34:06.6684225Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_self_remote_rref_as_self_rpc_arg_sparse
2025-03-14T05:34:06.6685547Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_set_and_get_num_worker_threads
2025-03-14T05:34:06.6687004Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_tensorpipe_options_throw_on_timedelta_timeout
2025-03-14T05:34:06.6688353Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentRpcTest::test_tensorpipe_set_default_timeout
2025-03-14T05:34:06.6689643Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_rref_sparse
2025-03-14T05:34:06.6690975Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_backward_simple_self_sparse
2025-03-14T05:34:06.6692340Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_call_sparse
2025-03-14T05:34:06.6693767Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_builtin_remote_call_sparse
2025-03-14T05:34:06.6695244Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_remote_call_itself_sparse
2025-03-14T05:34:06.6696732Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_py_nested_remote_call_sparse
2025-03-14T05:34:06.6698183Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_graph_for_python_remote_call_sparse
2025-03-14T05:34:06.6699726Z Running 1 items in this shard: test/distributed/rpc/test_tensorpipe_agent.py::TensorPipeTensorPipeAgentDistAutogradTest::test_rpc_complex_args_sparse
2025-03-14T05:34:06.6700451Z 
2025-03-14T05:34:06.6700743Z Running distributed/fsdp/test_fsdp_checkpoint 1/1 ... [2025-03-14 05:34:06.650347]
2025-03-14T05:34:06.6701271Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:34:06.6702442Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_checkpoint.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:34:06.650951]
2025-03-14T05:35:15.0694592Z 
2025-03-14T05:35:15.0697311Z distributed/fsdp/test_fsdp_checkpoint 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_checkpoint_1.1_553be86bbec018f1_.log
2025-03-14T05:35:15.0711594Z Running 16 items in this shard: test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload0_offload_activations_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_basic_checkpoint_end_to_end_cpu_offload1_offload_activations_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload0_offload_activations_True_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_False_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_False_use_orig_params_True, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_True_use_orig_params_False, test/distributed/fsdp/test_fsdp_checkpoint.py::TestFSDPCheckpoint::test_checkpoint_fsdp_wrapping_cpu_offload1_offload_activations_True_use_orig_params_True
2025-03-14T05:35:15.0722932Z 
2025-03-14T05:35:15.0723191Z Running distributed/launcher/test_run 1/1 ... [2025-03-14 05:35:15.069983]
2025-03-14T05:35:15.0723691Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:35:15.0724824Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/launcher/test_run.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:35:15.070429]
2025-03-14T05:36:01.4646890Z 
2025-03-14T05:36:01.4648074Z distributed/launcher/test_run 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.launcher.test_run_1.1_6be2e3093d5e099e_.log
2025-03-14T05:36:01.4660213Z Running 24 items in this shard: test/distributed/launcher/test_run.py::ElasticLaunchTest::test_capture_logs_using_default_logs_specs, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_init_method_env_with_torchelastic, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_init_method_tcp_with_torchelastic, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_is_not_torchelastic_launched, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_is_torchelastic_launched, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_is_torchelastic_launched_with_logs_spec_defined, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic_agent_raise_exception, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic_multiple_agents, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_elastic_worker_raise_exception, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_run_path, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_shutdown, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_standalone, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_bash, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_default_nproc, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_python, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_user_script_python_caffe2_bc, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_launch_with_env_vars, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_logs_logs_spec_entrypoint_must_be_defined, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_min_max_nodes_parse, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_gpu_launch_configurations, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_launch_auto_configurations, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_launch_number_configurations, test/distributed/launcher/test_run.py::ElasticLaunchTest::test_nproc_launch_unknown_configurations
2025-03-14T05:36:01.4670869Z 
2025-03-14T05:36:01.4671170Z Running distributed/fsdp/test_fsdp_hybrid_shard 1/1 ... [2025-03-14 05:36:01.465654]
2025-03-14T05:36:01.4671705Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:36:01.4672877Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_hybrid_shard.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:36:01.466129]
2025-03-14T05:36:28.8783110Z 
2025-03-14T05:36:28.8784230Z distributed/fsdp/test_fsdp_hybrid_shard 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_hybrid_shard_1.1_7b5e95e63c1af4be_.log
2025-03-14T05:36:28.8788003Z Running 6 items in this shard: test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_fsdp_hybrid_shard_basic_setup, test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_fsdp_hybrid_shard_parity, test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_hsdp_save_load_state_dict, test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_hsdp_sync_module_state, test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_invalid_pg_specification_raises, test/distributed/fsdp/test_fsdp_hybrid_shard.py::TestFSDPHybridShard::test_raises_manual_wrap_hybrid_shard_when_none_policy
2025-03-14T05:36:28.8791118Z 
2025-03-14T05:36:28.8791442Z Running distributed/fsdp/test_fsdp_multiple_forward 1/1 ... [2025-03-14 05:36:28.878765]
2025-03-14T05:36:28.8791978Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:36:28.8794561Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_multiple_forward.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:36:28.879225]
2025-03-14T05:36:37.0128268Z 
2025-03-14T05:36:37.0129412Z distributed/fsdp/test_fsdp_multiple_forward 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_multiple_forward_1.1_2559ca3246f5160b_.log
2025-03-14T05:36:37.0131536Z Running 1 items in this shard: test/distributed/fsdp/test_fsdp_multiple_forward.py::TestMultiForwardCPU::test_multi_forward_cpu
2025-03-14T05:36:37.0132140Z 
2025-03-14T05:36:37.0135085Z Running distributed/_shard/sharded_tensor/ops/test_tensor_ops 1/1 ... [2025-03-14 05:36:37.013274]
2025-03-14T05:36:37.0135739Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:36:37.0140077Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_tensor_ops.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:36:37.013720]
2025-03-14T05:36:40.8915664Z 
2025-03-14T05:36:40.8917216Z distributed/_shard/sharded_tensor/ops/test_tensor_ops 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_tensor_ops_1.1_eaa79eba291b1750_.log
2025-03-14T05:36:40.8920558Z Running 5 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_clone, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_deep_copy, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_detach, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_inplace_copy, test/distributed/_shard/sharded_tensor/ops/test_tensor_ops.py::TestTensorOps::test_set_requires_grad
2025-03-14T05:36:40.8923411Z 
2025-03-14T05:36:40.8923779Z Running distributed/_shard/sharded_optim/test_sharded_optim 1/1 ... [2025-03-14 05:36:40.892009]
2025-03-14T05:36:40.8924363Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:36:40.8927471Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_shard/sharded_optim/test_sharded_optim.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:36:40.892477]
2025-03-14T05:36:44.7203243Z 
2025-03-14T05:36:44.7204767Z distributed/_shard/sharded_optim/test_sharded_optim 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_optim.test_sharded_optim_1.1_b9e4292b7cc7982f_.log
2025-03-14T05:36:44.7206857Z Running 2 items in this shard: test/distributed/_shard/sharded_optim/test_sharded_optim.py::TestShardedOptimizer::test_named_params_with_sharded_tensor, test/distributed/_shard/sharded_optim/test_sharded_optim.py::TestShardedOptimizer::test_sharded_optim
2025-03-14T05:36:44.7208655Z 
2025-03-14T05:36:44.7209025Z Running distributed/_shard/sharding_plan/test_sharding_plan 1/1 ... [2025-03-14 05:36:44.720610]
2025-03-14T05:36:44.7209614Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:36:44.7213268Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_shard/sharding_plan/test_sharding_plan.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:36:44.721053]
2025-03-14T05:36:48.5984887Z 
2025-03-14T05:36:48.5986521Z distributed/_shard/sharding_plan/test_sharding_plan 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharding_plan.test_sharding_plan_1.1_b9b36863c9c93618_.log
2025-03-14T05:36:48.5989004Z Running 3 items in this shard: test/distributed/_shard/sharding_plan/test_sharding_plan.py::TestShardingPlan::test_custom_sharding_planner, test/distributed/_shard/sharding_plan/test_sharding_plan.py::TestShardingPlan::test_shard_module_sub_process_group, test/distributed/_shard/sharding_plan/test_sharding_plan.py::TestShardingPlan::test_sharding_plan_errors
2025-03-14T05:36:48.5990814Z 
2025-03-14T05:36:48.5991222Z Running distributed/test_launcher 1/1 ... [2025-03-14 05:36:48.598950]
2025-03-14T05:36:48.5992054Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:36:48.5996753Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/test_launcher.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:36:48.599421]
2025-03-14T05:36:53.7292404Z 
2025-03-14T05:36:53.7293417Z distributed/test_launcher 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.test_launcher_1.1_c9641116225c6cfb_.log
2025-03-14T05:36:53.7295252Z Running 1 items in this shard: test/distributed/test_launcher.py::TestDistributedLaunch::test_launch_user_script
2025-03-14T05:36:53.7295777Z 
2025-03-14T05:36:53.7299610Z Running distributed/fsdp/test_fsdp_clip_grad_norm 1/1 ... [2025-03-14 05:36:53.729745]
2025-03-14T05:36:53.7300157Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:36:53.7305657Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_clip_grad_norm.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:36:53.730317]
2025-03-14T05:36:57.7582406Z 
2025-03-14T05:36:57.7583806Z distributed/fsdp/test_fsdp_clip_grad_norm 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_clip_grad_norm_1.1_e6ef88816ec3101c_.log
2025-03-14T05:36:57.7584833Z Running 0 items in this shard:
2025-03-14T05:36:57.7585051Z 
2025-03-14T05:36:57.7589135Z Running distributed/fsdp/test_fsdp_comm 1/1 ... [2025-03-14 05:36:57.758721]
2025-03-14T05:36:57.7589658Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:36:57.7594003Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_comm.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:36:57.759178]
2025-03-14T05:37:01.8371831Z 
2025-03-14T05:37:01.8372898Z distributed/fsdp/test_fsdp_comm 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_comm_1.1_bc351c302f393730_.log
2025-03-14T05:37:01.8373859Z Running 0 items in this shard:
2025-03-14T05:37:01.8374075Z 
2025-03-14T05:37:01.8378508Z Running distributed/_shard/sharded_tensor/ops/test_embedding 1/1 ... [2025-03-14 05:37:01.837662]
2025-03-14T05:37:01.8379117Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:37:01.8383918Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_embedding.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:37:01.838144]
2025-03-14T05:37:05.6663667Z 
2025-03-14T05:37:05.6665947Z distributed/_shard/sharded_tensor/ops/test_embedding 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_1.1_533bec517705edb2_.log
2025-03-14T05:37:05.6670023Z Running 2 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_embedding.py::TestShardedEmbedding::test_sharded_embedding_colwise, test/distributed/_shard/sharded_tensor/ops/test_embedding.py::TestShardedEmbedding::test_sharded_embedding_rowwise
2025-03-14T05:37:05.6672068Z 
2025-03-14T05:37:05.6674000Z Running distributed/_shard/sharded_tensor/ops/test_embedding_bag 1/1 ... [2025-03-14 05:37:05.667148]
2025-03-14T05:37:05.6675006Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:37:05.6680841Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_embedding_bag.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:37:05.667734]
2025-03-14T05:37:09.5455028Z 
2025-03-14T05:37:09.5456415Z distributed/_shard/sharded_tensor/ops/test_embedding_bag 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_bag_1.1_a31e13cb7dcda492_.log
2025-03-14T05:37:09.5458856Z Running 2 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_embedding_bag.py::TestShardedEmbeddingBag::test_sharded_embedding_bag_colwise, test/distributed/_shard/sharded_tensor/ops/test_embedding_bag.py::TestShardedEmbeddingBag::test_sharded_embedding_bag_rowwise
2025-03-14T05:37:09.5460391Z 
2025-03-14T05:37:09.5462031Z Running distributed/_composable/fsdp/test_fully_shard_extensions 1/1 ... [2025-03-14 05:37:09.545958]
2025-03-14T05:37:09.5462709Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:37:09.5467403Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_extensions.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:37:09.546467]
2025-03-14T05:37:17.8311009Z 
2025-03-14T05:37:17.8312927Z distributed/_composable/fsdp/test_fully_shard_extensions 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_extensions_1.1_a001279f475464b2_.log
2025-03-14T05:37:17.8317872Z Running 5 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_extensions.py::TestFullyShardAllGatherExtensionsMultiProcess::test_all_gather_extensions_train_parity, test/distributed/_composable/fsdp/test_fully_shard_extensions.py::TestFullyShardAllGatherExtensionsMultiThread::test_all_gather_extension_hsdp_mesh, test/distributed/_composable/fsdp/test_fully_shard_extensions.py::TestFullyShardAllGatherExtensionsMultiThread::test_all_gather_extension_outer_size_stride, test/distributed/_composable/fsdp/test_fully_shard_extensions.py::TestFullyShardAllGatherExtensionsMultiThread::test_all_gather_extensions_end_to_end, test/distributed/_composable/fsdp/test_fully_shard_extensions.py::TestFullyShardAllGatherExtensionsMultiThread::test_all_gather_extensions_monkey_patch
2025-03-14T05:37:17.8321804Z 
2025-03-14T05:37:17.8322304Z Running distributed/_shard/sharded_tensor/ops/test_init 1/1 ... [2025-03-14 05:37:17.831563]
2025-03-14T05:37:17.8322943Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:37:17.8324250Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_shard/sharded_tensor/ops/test_init.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:37:17.832071]
2025-03-14T05:37:21.7100858Z 
2025-03-14T05:37:21.7102222Z distributed/_shard/sharded_tensor/ops/test_init 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.ops.test_init_1.1_bcc3673b5317c186_.log
2025-03-14T05:37:21.7105184Z Running 3 items in this shard: test/distributed/_shard/sharded_tensor/ops/test_init.py::TestShardedTensorNNInit::test_init_sharded_tensor_with_kaiming_uniform, test/distributed/_shard/sharded_tensor/ops/test_init.py::TestShardedTensorNNInit::test_init_sharded_tensor_with_normal, test/distributed/_shard/sharded_tensor/ops/test_init.py::TestShardedTensorNNInit::test_init_sharded_tensor_with_uniform
2025-03-14T05:37:21.7107032Z 
2025-03-14T05:37:21.7107409Z Running distributed/_composable/fsdp/test_fully_shard_training 1/1 ... [2025-03-14 05:37:21.710363]
2025-03-14T05:37:21.7108009Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:37:21.7110756Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_composable/fsdp/test_fully_shard_training.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:37:21.710844]
2025-03-14T05:37:25.8890199Z 
2025-03-14T05:37:25.8891593Z distributed/_composable/fsdp/test_fully_shard_training 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._composable.fsdp.test_fully_shard_training_1.1_069be336e479bfd3_.log
2025-03-14T05:37:25.8908202Z Running 23 items in this shard: test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardForwardInputs::test_root_move_forward_input_to_device, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardRegisteredParams::test_param_registration_after_backward, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardRegisteredParams::test_param_registration_after_forward, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardCastAfterInit::test_to_float64_after_init, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_explicit_prefetching, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_multi_forward_module, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_non_root_forward_backward, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_post_optim_event, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_multi_group, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_multi_group_cpu_offload_eager, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_multi_group_unshard_async_op, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_single_group_shard_dim0, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCore::test_train_parity_single_group_shard_largest_dim, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard1DTrainingCompose::test_train_parity_with_activation_checkpointing, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardShardPlacementFnMultiProcess::test_train_parity_shard_placement_fn_shard_largest_dim, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardShardPlacementFnMultiThread::test_shard_placement_fn_contiguous_params_grads, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardSharedParams::test_train_parity_with_shared_params, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardGradientAccumulation::test_1f1b_microbatching, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardGradientAccumulation::test_gradient_accumulation, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardNDTraining::test_2d_mlp_with_nd_mesh, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardHSDP3DTraining::test_3d_mlp_with_nd_mesh, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardHSDPTraining::test_train_parity_hsdp, test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShardCustomForwardMethod::test_register_fsdp_forward_method
2025-03-14T05:37:25.8923396Z 
2025-03-14T05:37:25.8923665Z Running distributed/_shard/test_sharder 1/1 ... [2025-03-14 05:37:25.889523]
2025-03-14T05:37:25.8924243Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:37:25.8925480Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_shard/test_sharder.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:37:25.890000]
2025-03-14T05:37:29.7679437Z 
2025-03-14T05:37:29.7680820Z distributed/_shard/test_sharder 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.test_sharder_1.1_2f83c0d84c5588c9_.log
2025-03-14T05:37:29.7682424Z Running 2 items in this shard: test/distributed/_shard/test_sharder.py::TestCustomSharder::test_custom_sharder, test/distributed/_shard/test_sharder.py::TestCustomSharder::test_custom_sharder_errors
2025-03-14T05:37:29.7683512Z 
2025-03-14T05:37:29.7684264Z Running distributed/fsdp/test_wrap 1/1 ... [2025-03-14 05:37:29.768222]
2025-03-14T05:37:29.7684901Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:37:29.7689294Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_wrap.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:37:29.768675]
2025-03-14T05:39:10.3289587Z 
2025-03-14T05:39:10.3291105Z distributed/fsdp/test_wrap 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_wrap_1.1_f324c2185c03c5df_.log
2025-03-14T05:39:10.3320553Z Running 52 items in this shard: test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_bn_always_wrapped_individually, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_False_device_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_False_device_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_True_device_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_error_already_wrapped_nested_True_device_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_False_device_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_False_device_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_True_device_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch0_forward_prefetch_True_device_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_False_device_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_False_device_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_True_device_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload0_backward_prefetch1_forward_prefetch_True_device_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_False_device_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_False_device_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_True_device_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch0_forward_prefetch_True_device_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_False_device_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_False_device_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_True_device_init_mode0, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_main_wrap_api_cpu_offload1_backward_prefetch1_forward_prefetch_True_device_init_mode1, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_wrap_batchnorm_individually_use_or_policy_False, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_wrap_batchnorm_individually_use_or_policy_True, test/distributed/fsdp/test_wrap.py::TestFSDPWrap::test_zero_argument, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_always_wrap, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_always_wrap_with_ignored_modules_wrap_method0, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_always_wrap_with_ignored_modules_wrap_method1, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_api, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_exclude_wrap, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_exclude_wrap_include_children, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_force_leaf, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_preset_force_leaf_custom, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_device_init_mode0_cpu_offload0_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_device_init_mode0_cpu_offload0_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_device_init_mode0_cpu_offload1_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_device_init_mode0_cpu_offload1_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_device_init_mode1_cpu_offload0_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_device_init_mode1_cpu_offload0_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_device_init_mode1_cpu_offload1_use_device_id_False, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_smoke_test_device_init_mode1_cpu_offload1_use_device_id_True, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_with_ignored_modules_wrap_method0, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_auto_wrap_with_ignored_modules_wrap_method1, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_custom_policy, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_frozen_params, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_module_wrap_policy, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_module_wrap_policy_callable, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_transformer_auto_wrap_policy, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_disabled_outside_context, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_override_defaults, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_wrap_method0, test/distributed/fsdp/test_wrap.py::TestAutoWrap::test_wrap_wrap_method1, test/distributed/fsdp/test_wrap.py::TestWrapUtils::test_validate_frozen_params
2025-03-14T05:39:10.3345215Z 
2025-03-14T05:39:10.3345631Z Running distributed/optim/test_apply_optimizer_in_backward 1/1 ... [2025-03-14 05:39:10.329643]
2025-03-14T05:39:10.3346219Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:39:10.3347435Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/optim/test_apply_optimizer_in_backward.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:39:10.330189]
2025-03-14T05:39:13.0356677Z 
2025-03-14T05:39:13.0357942Z distributed/optim/test_apply_optimizer_in_backward 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.optim.test_apply_optimizer_in_backward_1.1_f2c2805c0cee032f_.log
2025-03-14T05:39:13.0358913Z 
2025-03-14T05:39:13.0363099Z Running distributed/fsdp/test_fsdp_unshard_params 1/1 ... [2025-03-14 05:39:13.036135]
2025-03-14T05:39:13.0363645Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:39:13.0368545Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_unshard_params.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:39:13.036633]
2025-03-14T05:40:15.1989366Z 
2025-03-14T05:40:15.1990577Z distributed/fsdp/test_fsdp_unshard_params 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_unshard_params_1.1_cf1e67d285a7d65c_.log
2025-03-14T05:40:15.2000035Z Running 15 items in this shard: test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_named_parameters_and_buffers, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_param_data, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_recurse, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_respects_reshard, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_params_writeback, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_singleton_param_writeback, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_unshard_submodule, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_with_grads_core, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParams::test_with_grads_none_grads, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsNoShard::test_unshard_params_param_data_no_shard, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsNoShard::test_unshard_params_writeback_no_shard, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_offload_to_cpu_no_shard_raises, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_rank0_only_with_writeback_raises, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_unshard_params_from_backward_raises, test/distributed/fsdp/test_fsdp_unshard_params.py::TestUnshardParamsErrors::test_unshard_params_from_forward_raises
2025-03-14T05:40:15.2008103Z 
2025-03-14T05:40:15.2008483Z Running distributed/fsdp/test_fsdp_use_orig_params 1/1 ... [2025-03-14 05:40:15.199385]
2025-03-14T05:40:15.2009079Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:40:15.2010335Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/fsdp/test_fsdp_use_orig_params.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:40:15.199882]
2025-03-14T05:44:01.9066322Z 
2025-03-14T05:44:01.9068172Z distributed/fsdp/test_fsdp_use_orig_params 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.fsdp.test_fsdp_use_orig_params_1.1_06f7e427f0cce0fd_.log
2025-03-14T05:44:01.9094536Z Running 25 items in this shard: test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_cpu_offload_sharding_strategy_str_full_shard, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_cpu_offload_sharding_strategy_str_no_shard, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_cpu_offload_sharding_strategy_str_shard_grad_op, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_sharding_strategy_str_full_shard, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_sharding_strategy_str_no_shard, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_hyperparams_sharding_strategy_str_shard_grad_op, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_diff_trainability, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_fsdp_compile, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsMultipleParamGroups::test_multiple_optimizers, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsUnshardReshard::test_multiple_forward_offload_params_False, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsUnshardReshard::test_multiple_forward_offload_params_True, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsUnshardReshard::test_summon_between_two_forwards_offload_params_False, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsUnshardReshard::test_summon_between_two_forwards_offload_params_True, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsParamAccess::test_access_params_after_forward, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsWriteback::test_grad_writeback, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsWriteback::test_no_reshard_and_mixed_precision, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsWriteback::test_param_writeback, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsWriteback::test_writeback_between_fwd_and_bwd_for_no_reshard_raises, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsWriteback::test_writeback_shape_mismatch, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsFQNs::test_named_parameters_in_forward, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsNoSync::test_no_sync_correctness, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsNoSync::test_no_sync_mixed_precision, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestFSDPUseOrigParamsInit::test_non_uniform_requires_grad, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestMultiTensorApply::test_multi_tensor_apply_size0_tensors_cpu, test/distributed/fsdp/test_fsdp_use_orig_params.py::TestMultiTensorApply::test_multi_tensor_apply_size0_tensors_cuda
2025-03-14T05:44:01.9117861Z 
2025-03-14T05:44:01.9118447Z Running distributed/_shard/sharded_tensor/test_sharded_tensor 1/1 ... [2025-03-14 05:44:01.907477]
2025-03-14T05:44:01.9119321Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:44:01.9121070Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_shard/sharded_tensor/test_sharded_tensor.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:44:01.908078]
2025-03-14T05:44:14.9561400Z 
2025-03-14T05:44:14.9562623Z distributed/_shard/sharded_tensor/test_sharded_tensor 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_1.1_aacf32e09be9654a_.log
2025-03-14T05:44:14.9601794Z Running 69 items in this shard: test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorMetadata::test_serialize_and_deserialize, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestCreateTensorFromParams::test_empty, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardParameter::test_shard_parameter, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardParameter::test_shard_parameter_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardTensor::test_shard_tensor, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardTensor::test_shard_tensor_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardTensor::test_shard_tensor_with_empty_shard, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestModuleHookApi::test_collect_local_shard, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestModuleHookApi::test_reshard_output, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestLocalTensor::test_local_tensor, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestLocalTensor::test_local_tensor_error, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_cleanup, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_complete_world_size, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_like, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_full, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_ones, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_rand, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_create_sharded_tensor_with_zeros, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_gather_even, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_gather_uneven, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_insufficient_sharding_dims, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_invalid_pg_rpc_ranks, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_invalid_sharding, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_load_state_dict_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_multiple_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_partial_world_size, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_sharded_tensor_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_sharded_tensor_sizes, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_sharding_columns, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_state_dict, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_state_dict_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorChunked::test_state_dict_no_sharded_tensors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_create_sharded_tensor_with_ones, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_gather_even, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_gather_uneven, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_grid_sharding, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_multiple_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_partial_world_size, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_device, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_to_cpu, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_to_cuda, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_sharded_tensor_to_test, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_uneven_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorEnumerable::test_with_rpc_names, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalTensor::test_init_from_local_tensor, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalTensor::test_init_from_local_tensor_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_and_global_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_and_global_metadata_invalid_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_pin_memory, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_property_cross_ranks, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_shards_gaps, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_invalid_shards_overlap, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_init_from_local_shards_new_group, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_local_shards, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorFromLocalShards::test_st_base_init_from_local_shards_and_global_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorCustomOps::test_custom_op, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorCustomOps::test_custom_op_errors, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorCustomOps::test_custom_op_override, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardMetadata::test_create_shard_with_no_placement, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardMetadata::test_shard_metadata_init, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorSubGroupInit::test_sub_process_group_placement_validation, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestShardedTensorSubGroupInit::test_sub_process_group_sharded_tensor_init, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestCreateTensorNoProcessGroupMode::test_init_from_local_shards_and_global_metadata, test/distributed/_shard/sharded_tensor/test_sharded_tensor.py::TestCreateTensorNoProcessGroupMode::test_non_contiguous_local_shards
2025-03-14T05:44:14.9638765Z 
2025-03-14T05:44:14.9639072Z Running distributed/_tools/test_fake_collectives 1/1 ... [2025-03-14 05:44:14.956636]
2025-03-14T05:44:14.9639610Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:44:14.9640777Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/_tools/test_fake_collectives.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:44:14.957094]
2025-03-14T05:44:18.9849276Z 
2025-03-14T05:44:18.9850894Z distributed/_tools/test_fake_collectives 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed._tools.test_fake_collectives_1.1_bfa57d8268792dcb_.log
2025-03-14T05:44:18.9852261Z Running 1 items in this shard: test/distributed/_tools/test_fake_collectives.py::TestFakeCollectives::test_collectives
2025-03-14T05:44:18.9852807Z 
2025-03-14T05:44:18.9856125Z Running distributed/elastic/events/lib_test 1/1 ... [2025-03-14 05:44:18.985400]
2025-03-14T05:44:18.9856731Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:44:18.9861431Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/elastic/events/lib_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:44:18.985883]
2025-03-14T05:44:23.1640194Z 
2025-03-14T05:44:23.1641644Z distributed/elastic/events/lib_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.events.lib_test_1.1_ab1a9d1fe2eb6f72_.log
2025-03-14T05:44:23.1645850Z Running 8 items in this shard: test/distributed/elastic/events/lib_test.py::EventLibTest::test_event_created, test/distributed/elastic/events/lib_test.py::EventLibTest::test_event_deser, test/distributed/elastic/events/lib_test.py::EventLibTest::test_get_or_create_logger, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_construct_and_record_rdzv_event, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_construct_and_record_rdzv_event_does_not_run_if_invalid_dest, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_rdzv_event_created, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_rdzv_event_deserialize, test/distributed/elastic/events/lib_test.py::RdzvEventLibTest::test_rdzv_event_str
2025-03-14T05:44:23.1650134Z 
2025-03-14T05:44:23.1650601Z Running distributed/elastic/metrics/api_test 1/1 ... [2025-03-14 05:44:23.164499]
2025-03-14T05:44:23.1651370Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:44:23.1652737Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/elastic/metrics/api_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:44:23.164965]
2025-03-14T05:44:27.2932316Z 
2025-03-14T05:44:27.2933747Z distributed/elastic/metrics/api_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.metrics.api_test_1.1_749f776b0e0ddac8_.log
2025-03-14T05:44:27.2935766Z Running 3 items in this shard: test/distributed/elastic/metrics/api_test.py::MetricsApiTest::test_get_metric_name, test/distributed/elastic/metrics/api_test.py::MetricsApiTest::test_inheritance, test/distributed/elastic/metrics/api_test.py::MetricsApiTest::test_profile
2025-03-14T05:44:27.2937043Z 
2025-03-14T05:44:27.2939052Z Running distributed/elastic/multiprocessing/api_test 1/1 ... [2025-03-14 05:44:27.293678]
2025-03-14T05:44:27.2939719Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:44:27.2944087Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/elastic/multiprocessing/api_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:44:27.294139]
2025-03-14T05:45:03.4201267Z 
2025-03-14T05:45:03.4202676Z distributed/elastic/multiprocessing/api_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.multiprocessing.api_test_1.1_e8fa40a0549bf30e_.log
2025-03-14T05:45:03.4217777Z Running 25 items in this shard: test/distributed/elastic/multiprocessing/api_test.py::RunProcResultsTest::test_get_failures, test/distributed/elastic/multiprocessing/api_test.py::RunProcResultsTest::test_is_failed, test/distributed/elastic/multiprocessing/api_test.py::StdTest::test_from_str_bad_input, test/distributed/elastic/multiprocessing/api_test.py::StdTest::test_from_value, test/distributed/elastic/multiprocessing/api_test.py::StdTest::test_from_value_map, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsFuncTest::test_args_env_len_mismatch, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsFuncTest::test_function_large_ret_val, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsFuncTest::test_function_raise, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsFuncTest::test_function_with_tensor, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsFuncTest::test_invalid_log_dir, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsFuncTest::test_multiprocess_context_close, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsFuncTest::test_multiprocessing_context_poll_raises_exception, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsFuncTest::test_pcontext_wait, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsFuncTest::test_pcontext_wait_on_a_child_thread, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsFuncTest::test_to_map, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsFuncTest::test_void_function, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsFuncTest::test_wait_for_all_child_procs_to_exit, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsBinaryTest::test_binary_exit, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsBinaryTest::test_binary_incorrect_entrypoint, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsBinaryTest::test_binary_raises, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsBinaryTest::test_subprocess_context_close, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesAsBinaryTest::test_validate_full_rank, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListAsFuncTest::test_function, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListAsBinaryTest::test_binary, test/distributed/elastic/multiprocessing/api_test.py::StartProcessesListAsBinaryTest::test_binary_redirect_and_tee
2025-03-14T05:45:03.4230550Z 
2025-03-14T05:45:03.4230848Z Running distributed/elastic/timer/api_test 1/1 ... [2025-03-14 05:45:03.420651]
2025-03-14T05:45:03.4231367Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:45:03.4232519Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/elastic/timer/api_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:45:03.421116]
2025-03-14T05:45:04.8678846Z 
2025-03-14T05:45:04.8680422Z distributed/elastic/timer/api_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.timer.api_test_1.1_caaf4c5fd9e908d8_.log
2025-03-14T05:45:04.8681290Z 
2025-03-14T05:45:04.8685663Z Running distributed/elastic/timer/local_timer_example 1/1 ... [2025-03-14 05:45:04.868345]
2025-03-14T05:45:04.8686304Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:45:04.8690700Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/elastic/timer/local_timer_example.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:45:04.868806]
2025-03-14T05:45:28.8034036Z 
2025-03-14T05:45:28.8036131Z distributed/elastic/timer/local_timer_example 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.timer.local_timer_example_1.1_75f544fdac4daf9d_.log
2025-03-14T05:45:28.8039468Z Running 2 items in this shard: test/distributed/elastic/timer/local_timer_example.py::LocalTimerExample::test_example_start_method_spawn, test/distributed/elastic/timer/local_timer_example.py::LocalTimerExample::test_torch_mp_example
2025-03-14T05:45:28.8041501Z 
2025-03-14T05:45:28.8044208Z Running distributed/elastic/timer/local_timer_test 1/1 ... [2025-03-14 05:45:28.804176]
2025-03-14T05:45:28.8044875Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:45:28.8049680Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/elastic/timer/local_timer_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:45:28.804733]
2025-03-14T05:45:38.2409389Z 
2025-03-14T05:45:38.2410710Z distributed/elastic/timer/local_timer_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.timer.local_timer_test_1.1_97889ed767288475_.log
2025-03-14T05:45:38.2418875Z Running 14 items in this shard: test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_client_interaction, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_exception_propagation, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_get_timer_recursive, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_happy_path, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_no_client, test/distributed/elastic/timer/local_timer_test.py::LocalTimerTest::test_timer, test/distributed/elastic/timer/local_timer_test.py::MultiprocessingRequestQueueTest::test_get, test/distributed/elastic/timer/local_timer_test.py::MultiprocessingRequestQueueTest::test_get_less_than_size, test/distributed/elastic/timer/local_timer_test.py::MultiprocessingRequestQueueTest::test_get_size, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_acquire_release, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_expired_timers, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_valid_timers, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_watchdog_call_count, test/distributed/elastic/timer/local_timer_test.py::LocalTimerServerTest::test_watchdog_empty_queue
2025-03-14T05:45:38.2426105Z 
2025-03-14T05:45:38.2426440Z Running distributed/elastic/utils/distributed_test 1/1 ... [2025-03-14 05:45:38.241459]
2025-03-14T05:45:38.2427004Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:45:38.2428209Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/elastic/utils/distributed_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:45:38.241945]
2025-03-14T05:45:47.3788823Z 
2025-03-14T05:45:47.3790441Z distributed/elastic/utils/distributed_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.utils.distributed_test_1.1_02e442c9d1a6500b_.log
2025-03-14T05:45:47.3795416Z Running 8 items in this shard: test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_multi, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_no_port_multi, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_single_server, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_timeout_on_server, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_timeout_on_worker, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_create_store_with_libuv_support, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_port_already_in_use_on_server, test/distributed/elastic/utils/distributed_test.py::DistributedUtilTest::test_port_already_in_use_on_worker
2025-03-14T05:45:47.3800385Z 
2025-03-14T05:45:47.3800698Z Running distributed/elastic/utils/logging_test 1/1 ... [2025-03-14 05:45:47.379554]
2025-03-14T05:45:47.3801312Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:45:47.3803157Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/elastic/utils/logging_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:45:47.380042]
2025-03-14T05:45:51.5083258Z 
2025-03-14T05:45:51.5084368Z distributed/elastic/utils/logging_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.utils.logging_test_1.1_edffe3af707aa02f_.log
2025-03-14T05:45:51.5086076Z Running 2 items in this shard: test/distributed/elastic/utils/logging_test.py::LoggingTest::test_derive_module_name, test/distributed/elastic/utils/logging_test.py::LoggingTest::test_logger_name
2025-03-14T05:45:51.5086976Z 
2025-03-14T05:45:51.5088791Z Running distributed/elastic/utils/util_test 1/1 ... [2025-03-14 05:45:51.508683]
2025-03-14T05:45:51.5089404Z SCRIBE_GRAPHQL_ACCESS_TOKEN is set
2025-03-14T05:45:51.5094234Z Executing ['/opt/conda/envs/py_3.9/bin/python', '-bb', 'distributed/elastic/utils/util_test.py', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-03-14 05:45:51.509182]
2025-03-14T05:45:55.7878222Z 
2025-03-14T05:45:55.7879321Z distributed/elastic/utils/util_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/distributed.elastic.utils.util_test_1.1_a7a753c87e98f723_.log
2025-03-14T05:45:55.7885101Z Running 12 items in this shard: test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_barrier, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_barrier_hash_store, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_barrier_timeout_operations, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_barrier_timeout_rank_tracing, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_get_all_rank_0, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_get_all_rank_n, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_synchronize, test/distributed/elastic/utils/util_test.py::StoreUtilTest::test_synchronize_hash_store, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger_custom_name, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger_different, test/distributed/elastic/utils/util_test.py::UtilTest::test_get_logger_none
2025-03-14T05:45:55.7892759Z 
2025-03-14T05:45:56.5443863Z Running test batch 'tests to run' cost 5462.35 seconds
2025-03-14T05:45:57.1988873Z 
2025-03-14T05:45:57.1989226Z real	91m8.432s
2025-03-14T05:45:57.1989601Z user	185m35.295s
2025-03-14T05:45:57.1989926Z sys	33m31.066s
2025-03-14T05:45:57.1990277Z + assert_git_not_dirty
2025-03-14T05:45:57.1990766Z + [[ linux-jammy-py3.9-gcc11 != *rocm* ]]
2025-03-14T05:45:57.1991390Z + [[ linux-jammy-py3.9-gcc11 != *xla* ]]
2025-03-14T05:45:57.1997628Z ++ git status --porcelain
2025-03-14T05:45:57.1998463Z ++ grep -v '?? third_party'
2025-03-14T05:46:34.0507199Z ++ true
2025-03-14T05:46:34.0509665Z + git_status=
2025-03-14T05:46:34.0510031Z + [[ -n '' ]]
2025-03-14T05:46:34.0510376Z + [[ linux-jammy-py3.9-gcc11 == *cuda* ]]
2025-03-14T05:46:34.0510767Z + [[ linux-jammy-py3.9-gcc11 == *rocm* ]]
2025-03-14T05:46:34.0511253Z + [[ 2 == 1 ]]
2025-03-14T05:46:34.0511665Z + cleanup_workspace
2025-03-14T05:46:34.0512520Z + echo 'sudo may print the following warning message that can be ignored. The chown command will still run.'
2025-03-14T05:46:34.0513359Z sudo may print the following warning message that can be ignored. The chown command will still run.
2025-03-14T05:46:34.0514099Z + echo '    sudo: setrlimit(RLIMIT_STACK): Operation not permitted'
2025-03-14T05:46:34.0514615Z     sudo: setrlimit(RLIMIT_STACK): Operation not permitted
2025-03-14T05:46:34.0515187Z + echo 'For more details refer to https://github.com/sudo-project/sudo/issues/42'
2025-03-14T05:46:34.0515817Z For more details refer to https://github.com/sudo-project/sudo/issues/42
2025-03-14T05:46:34.0516321Z + sudo chown -R 1000 /var/lib/jenkins/workspace
2025-03-14T05:46:37.1188556Z ##[group]Run pytorch/test-infra/.github/actions/upload-benchmark-results@main
2025-03-14T05:46:37.1189081Z with:
2025-03-14T05:46:37.1189363Z   benchmark-results-dir: test/test-reports
2025-03-14T05:46:37.1189717Z   dry-run: false
2025-03-14T05:46:37.1189977Z   schema-version: v3
2025-03-14T05:46:37.1190445Z   github-token: ***
2025-03-14T05:46:37.1190701Z env:
2025-03-14T05:46:37.1190938Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:37.1191441Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:37.1191977Z ##[endgroup]
2025-03-14T05:46:37.1219321Z ##[group]Run set -eux
2025-03-14T05:46:37.1219624Z set -eux
2025-03-14T05:46:37.1219928Z python3 -mpip install boto3==1.35.33
2025-03-14T05:46:37.1266994Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:37.1267418Z env:
2025-03-14T05:46:37.1267683Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:37.1268410Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:37.1268955Z ##[endgroup]
2025-03-14T05:46:37.1296055Z + python3 -mpip install boto3==1.35.33
2025-03-14T05:46:37.4455001Z Defaulting to user installation because normal site-packages is not writeable
2025-03-14T05:46:37.4648063Z Requirement already satisfied: boto3==1.35.33 in /home/ec2-user/.local/lib/python3.9/site-packages (1.35.33)
2025-03-14T05:46:37.4698178Z Requirement already satisfied: s3transfer<0.11.0,>=0.10.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from boto3==1.35.33) (0.10.4)
2025-03-14T05:46:37.4702453Z Requirement already satisfied: botocore<1.36.0,>=1.35.33 in /home/ec2-user/.local/lib/python3.9/site-packages (from boto3==1.35.33) (1.35.99)
2025-03-14T05:46:37.4708126Z Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/lib/python3.9/site-packages (from boto3==1.35.33) (0.10.0)
2025-03-14T05:46:37.4763116Z Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.25.10)
2025-03-14T05:46:37.4767513Z Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /home/ec2-user/.local/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (2.8.2)
2025-03-14T05:46:37.4808464Z Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.15.0)
2025-03-14T05:46:37.6826670Z ##[group]Run set -eux
2025-03-14T05:46:37.6826973Z set -eux
2025-03-14T05:46:37.6827226Z 
2025-03-14T05:46:37.6827494Z if [[ -z "${GITHUB_TOKEN}" ]]; then
2025-03-14T05:46:37.6827876Z   echo "Missing github-token input"
2025-03-14T05:46:37.6828221Z   exit 1
2025-03-14T05:46:37.6828467Z fi
2025-03-14T05:46:37.6835160Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:37.6835578Z env:
2025-03-14T05:46:37.6835810Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:37.6836352Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:37.6837265Z   GITHUB_TOKEN: ***
2025-03-14T05:46:37.6837609Z ##[endgroup]
2025-03-14T05:46:37.6866639Z + [[ -z *** ]]
2025-03-14T05:46:37.6934955Z ##[group]Run pytorch/test-infra/.github/actions/get-workflow-job-id@main
2025-03-14T05:46:37.6935428Z with:
2025-03-14T05:46:37.6935818Z   github-token: ***
2025-03-14T05:46:37.6936074Z env:
2025-03-14T05:46:37.6936310Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:37.6936814Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:37.6937353Z ##[endgroup]
2025-03-14T05:46:37.6972619Z ##[group]Run set -eux
2025-03-14T05:46:37.6972919Z set -eux
2025-03-14T05:46:37.6973170Z 
2025-03-14T05:46:37.6973683Z python3 "${GITHUB_ACTION_PATH}/../../scripts/get_workflow_job_id.py" "${GITHUB_RUN_ID}" "${RUNNER_NAME}"
2025-03-14T05:46:37.6979288Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:37.6979728Z env:
2025-03-14T05:46:37.6979960Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:37.6980470Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:37.6981310Z   GITHUB_TOKEN: ***
2025-03-14T05:46:37.6981563Z ##[endgroup]
2025-03-14T05:46:37.7004966Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/get-workflow-job-id/../../scripts/get_workflow_job_id.py 13849515290 i-09d49014f68772654
2025-03-14T05:46:39.9574815Z setting job-id=38754724585
2025-03-14T05:46:39.9575835Z setting job-name=linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)
2025-03-14T05:46:39.9689623Z ##[group]Run set -eux
2025-03-14T05:46:39.9689969Z set -eux
2025-03-14T05:46:39.9690214Z 
2025-03-14T05:46:39.9690647Z python3 "${GITHUB_ACTION_PATH}/../../scripts/benchmarks/gather_metadata.py" \
2025-03-14T05:46:39.9691399Z   --schema-version "${SCHEMA_VERSION}" \
2025-03-14T05:46:39.9691784Z   --repo "${REPO}" \
2025-03-14T05:46:39.9692127Z   --head-branch "${HEAD_BRANCH}" \
2025-03-14T05:46:39.9692497Z   --head-sha "${HEAD_SHA}" \
2025-03-14T05:46:39.9692866Z   --workflow-id "${WORKFLOW_RUN_ID}" \
2025-03-14T05:46:39.9693259Z   --run-attempt "${RUN_ATTEMPT}" \
2025-03-14T05:46:39.9693619Z   --job-id "${JOB_ID}" \
2025-03-14T05:46:39.9693947Z   --job-name "${JOB_NAME}"
2025-03-14T05:46:39.9699849Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:39.9700271Z env:
2025-03-14T05:46:39.9700516Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:39.9701022Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:39.9701565Z   SCHEMA_VERSION: v3
2025-03-14T05:46:39.9701838Z   REPO: pytorch/pytorch
2025-03-14T05:46:39.9702126Z   HEAD_BRANCH: refs/heads/main
2025-03-14T05:46:39.9702495Z   HEAD_SHA: aed0b7a742a2d7b7901790622829cbd2135049a4
2025-03-14T05:46:39.9702870Z   WORKFLOW_RUN_ID: 13849515290
2025-03-14T05:46:39.9703165Z   RUN_ATTEMPT: 1
2025-03-14T05:46:39.9703417Z   JOB_ID: 38754724585
2025-03-14T05:46:39.9703840Z   JOB_NAME: linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)
2025-03-14T05:46:39.9704319Z ##[endgroup]
2025-03-14T05:46:39.9729320Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/benchmarks/gather_metadata.py --schema-version v3 --repo pytorch/pytorch --head-branch refs/heads/main --head-sha aed0b7a742a2d7b7901790622829cbd2135049a4 --workflow-id 13849515290 --run-attempt 1 --job-id 38754724585 --job-name 'linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)'
2025-03-14T05:46:40.0046905Z ##[group]Run set -eux
2025-03-14T05:46:40.0047196Z set -eux
2025-03-14T05:46:40.0047445Z 
2025-03-14T05:46:40.0047720Z # TODO (huydhn): Implement this part
2025-03-14T05:46:40.0048142Z echo "runners=[]" >> "${GITHUB_OUTPUT}"
2025-03-14T05:46:40.0054030Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:40.0054435Z env:
2025-03-14T05:46:40.0054677Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:40.0055179Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:40.0055720Z ##[endgroup]
2025-03-14T05:46:40.0168245Z + echo 'runners=[]'
2025-03-14T05:46:40.0194204Z ##[group]Run set -eux
2025-03-14T05:46:40.0194500Z set -eux
2025-03-14T05:46:40.0194750Z 
2025-03-14T05:46:40.0195023Z # TODO (huydhn): Implement this part
2025-03-14T05:46:40.0195436Z echo "dependencies={}" >> "${GITHUB_OUTPUT}"
2025-03-14T05:46:40.0200778Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:40.0201184Z env:
2025-03-14T05:46:40.0201424Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:40.0202068Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:40.0202621Z ##[endgroup]
2025-03-14T05:46:40.0223165Z + echo 'dependencies={}'
2025-03-14T05:46:40.0250238Z ##[group]Run set -eux
2025-03-14T05:46:40.0250543Z set -eux
2025-03-14T05:46:40.0250794Z 
2025-03-14T05:46:40.0251082Z if [[ ! -d "${BENCHMARK_RESULTS_DIR}" ]]; then
2025-03-14T05:46:40.0251555Z   echo "${BENCHMARK_RESULTS_DIR} does not exist, skipping"
2025-03-14T05:46:40.0252080Z   # We don't want the job to fail if the directory doesn't exist
2025-03-14T05:46:40.0252499Z   exit 0
2025-03-14T05:46:40.0252742Z fi
2025-03-14T05:46:40.0252969Z 
2025-03-14T05:46:40.0253223Z if [[ "${DRY_RUN}" == "true" ]]; then
2025-03-14T05:46:40.0253734Z   python3 "${GITHUB_ACTION_PATH}/../../scripts/upload_benchmark_results.py" \
2025-03-14T05:46:40.0254324Z     --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \
2025-03-14T05:46:40.0254933Z     --metadata "${BENCHMARK_METADATA}" \
2025-03-14T05:46:40.0255314Z     --runners "${RUNNER_INFO}" \
2025-03-14T05:46:40.0255689Z     --dependencies "${DEPENDENCIES}" \
2025-03-14T05:46:40.0256047Z     --dry-run
2025-03-14T05:46:40.0256316Z else
2025-03-14T05:46:40.0256711Z   python3 "${GITHUB_ACTION_PATH}/../../scripts/upload_benchmark_results.py" \
2025-03-14T05:46:40.0257300Z     --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \
2025-03-14T05:46:40.0257757Z     --metadata "${BENCHMARK_METADATA}" \
2025-03-14T05:46:40.0258141Z     --runners "${RUNNER_INFO}" \
2025-03-14T05:46:40.0258515Z     --dependencies "${DEPENDENCIES}"
2025-03-14T05:46:40.0258858Z fi
2025-03-14T05:46:40.0263688Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:40.0264095Z env:
2025-03-14T05:46:40.0264334Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:40.0264853Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:40.0265515Z   BENCHMARK_RESULTS_DIR: test/test-reports
2025-03-14T05:46:40.0265865Z   DRY_RUN: false
2025-03-14T05:46:40.0267204Z   BENCHMARK_METADATA: {"timestamp": 1741931199, "schema_version": "v3", "name": "linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)", "repo": "pytorch/pytorch", "head_branch": "refs/heads/main", "head_sha": "aed0b7a742a2d7b7901790622829cbd2135049a4", "workflow_id": 13849515290, "run_attempt": 1, "job_id": 38754724585}
2025-03-14T05:46:40.0268595Z   RUNNER_INFO: []
2025-03-14T05:46:40.0268856Z   DEPENDENCIES: {}
2025-03-14T05:46:40.0269113Z ##[endgroup]
2025-03-14T05:46:40.0289320Z + [[ ! -d test/test-reports ]]
2025-03-14T05:46:40.0289653Z + [[ false == \t\r\u\e ]]
2025-03-14T05:46:40.0292048Z + python3 /home/ec2-user/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/upload_benchmark_results.py --benchmark-results-dir test/test-reports --metadata '{"timestamp": 1741931199, "schema_version": "v3", "name": "linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)", "repo": "pytorch/pytorch", "head_branch": "refs/heads/main", "head_sha": "aed0b7a742a2d7b7901790622829cbd2135049a4", "workflow_id": 13849515290, "run_attempt": 1, "job_id": 38754724585}' --runners '[]' --dependencies '{}'
2025-03-14T05:46:40.2213196Z ##[group]Run cat test/**/*_toprint.log || true
2025-03-14T05:46:40.2213936Z cat test/**/*_toprint.log || true
2025-03-14T05:46:40.2222038Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:40.2222739Z env:
2025-03-14T05:46:40.2223137Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:40.2224039Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:40.2225013Z ##[endgroup]
2025-03-14T05:46:40.2302047Z cat: 'test/**/*_toprint.log': No such file or directory
2025-03-14T05:46:40.2336905Z ##[group]Run kill "$MONITOR_SCRIPT_PID"
2025-03-14T05:46:40.2337295Z kill "$MONITOR_SCRIPT_PID"
2025-03-14T05:46:40.2342767Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:40.2343172Z env:
2025-03-14T05:46:40.2343414Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:40.2343909Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:40.2344465Z   MONITOR_SCRIPT_PID: 626501
2025-03-14T05:46:40.2344767Z ##[endgroup]
2025-03-14T05:46:40.2518787Z Prepare all required actions
2025-03-14T05:46:40.2519508Z Getting action download info
2025-03-14T05:46:40.3680441Z Download action repository 'actions/upload-artifact@v4' (SHA:4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1)
2025-03-14T05:46:40.8613809Z ##[group]Run ./.github/actions/upload-test-artifacts
2025-03-14T05:46:40.8614209Z with:
2025-03-14T05:46:40.8614564Z   file-suffix: test-distributed-2-2-lf.linux.2xlarge_38754724585
2025-03-14T05:46:40.8615014Z   s3-bucket: gha-artifacts
2025-03-14T05:46:40.8615453Z env:
2025-03-14T05:46:40.8615683Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:40.8616190Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:40.8616727Z ##[endgroup]
2025-03-14T05:46:40.8664597Z ##[group]Run # Remove any previous test jsons if they exist
2025-03-14T05:46:40.8665098Z # Remove any previous test jsons if they exist
2025-03-14T05:46:40.8665595Z rm -f test-jsons-*.zip
2025-03-14T05:46:40.8666107Z zip -r "test-jsons-${FILE_SUFFIX}.zip" test/test-reports -i '*.json'
2025-03-14T05:46:40.8671698Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:40.8672110Z env:
2025-03-14T05:46:40.8672352Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:40.8672856Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:40.8673503Z   FILE_SUFFIX: test-distributed-2-2-lf.linux.2xlarge_38754724585
2025-03-14T05:46:40.8673928Z ##[endgroup]
2025-03-14T05:46:40.8750412Z   adding: test/test-reports/td_exclusions-94cb759e0ae33cb29012.json (deflated 85%)
2025-03-14T05:46:40.8788701Z ##[group]Run # Remove any previous test reports if they exist
2025-03-14T05:46:40.8789515Z # Remove any previous test reports if they exist
2025-03-14T05:46:40.8790172Z rm -f test-reports-*.zip
2025-03-14T05:46:40.8790987Z zip -r "test-reports-${FILE_SUFFIX}.zip" test/test-reports -i '*.xml' -i '*.csv'
2025-03-14T05:46:40.8799137Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:40.8799840Z env:
2025-03-14T05:46:40.8800236Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:40.8801105Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:40.8802256Z   FILE_SUFFIX: test-distributed-2-2-lf.linux.2xlarge_38754724585
2025-03-14T05:46:40.8802920Z ##[endgroup]
2025-03-14T05:46:40.8893732Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_dtensor_state_dict/distributed.fsdp.test_fsdp_dtensor_state_dict-9c1b4868949595c3.xml (deflated 27%)
2025-03-14T05:46:40.8899127Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_core/distributed.fsdp.test_fsdp_core-92262779e04c6127.xml (deflated 28%)
2025-03-14T05:46:40.8904202Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_distributed_checkpoint/distributed.fsdp.test_distributed_checkpoint-6cd2586ec55140af.xml (deflated 28%)
2025-03-14T05:46:40.8910994Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_misc/distributed.fsdp.test_fsdp_misc-780de02a7c00ccd3.xml (deflated 90%)
2025-03-14T05:46:40.8917871Z   adding: test/test-reports/python-pytest/distributed.test_c10d_functional_native/distributed.test_c10d_functional_native-92808a69a24b5288.xml (deflated 93%)
2025-03-14T05:46:40.8925057Z   adding: test/test-reports/python-pytest/distributed.test_c10d_pypg/distributed.test_c10d_pypg-99e5b5d6fecb860e.xml (deflated 94%)
2025-03-14T05:46:40.8930369Z   adding: test/test-reports/python-pytest/distributed.test_c10d_logger/distributed.test_c10d_logger-e3b55d5e142f8f11.xml (deflated 51%)
2025-03-14T05:46:40.8936381Z   adding: test/test-reports/python-pytest/distributed._composable.test_composability.test_pp_composability/distributed._composable.test_composability.test_pp_composability-6467d4f0c52cfbc8.xml (deflated 92%)
2025-03-14T05:46:40.8941716Z   adding: test/test-reports/python-pytest/distributed.test_c10d_object_collectives/distributed.test_c10d_object_collectives-a81894b38a1690a7.xml (deflated 81%)
2025-03-14T05:46:40.8946449Z   adding: test/test-reports/python-pytest/distributed._tools.test_mod_tracker/distributed._tools.test_mod_tracker-d2a8c6b624cc9c69.xml (deflated 64%)
2025-03-14T05:46:40.8951306Z   adding: test/test-reports/python-pytest/distributed.tensor.debug.test_op_coverage/distributed.tensor.debug.test_op_coverage-f1bc8308c42725c9.xml (deflated 35%)
2025-03-14T05:46:40.8956291Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-c41e9ca921a89561.xml (deflated 36%)
2025-03-14T05:46:40.8961537Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-bb1febf3cc838f37.xml (deflated 43%)
2025-03-14T05:46:40.8966402Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-5044abaa2e6f9d90.xml (deflated 43%)
2025-03-14T05:46:40.8971931Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-3e5e266dcc881287.xml (deflated 43%)
2025-03-14T05:46:40.8977483Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-c583abfff39e2e4b.xml (deflated 44%)
2025-03-14T05:46:40.8982498Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-144241c5523a868b.xml (deflated 44%)
2025-03-14T05:46:40.8987670Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-ebaeef951123be8f.xml (deflated 44%)
2025-03-14T05:46:40.8992905Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-475f5b53d11ad6d9.xml (deflated 44%)
2025-03-14T05:46:40.8998092Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-4f7dc0778ba521d1.xml (deflated 44%)
2025-03-14T05:46:40.9003118Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-a7af268fc3800751.xml (deflated 44%)
2025-03-14T05:46:40.9008513Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_gloo/distributed.test_c10d_spawn_gloo-af0507103cd59b68.xml (deflated 44%)
2025-03-14T05:46:40.9013995Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-7f022497680c316b.xml (deflated 46%)
2025-03-14T05:46:40.9019765Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-19be6017dcf5d304.xml (deflated 45%)
2025-03-14T05:46:40.9026143Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-f171afb9867ebf76.xml (deflated 45%)
2025-03-14T05:46:40.9031926Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-7698e6428403cc0f.xml (deflated 46%)
2025-03-14T05:46:40.9037401Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-29eb6182886a637e.xml (deflated 46%)
2025-03-14T05:46:40.9043072Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-f2fd37131045b800.xml (deflated 45%)
2025-03-14T05:46:40.9049211Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-4b531e5008387196.xml (deflated 45%)
2025-03-14T05:46:40.9054635Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-96a994452570f0e8.xml (deflated 46%)
2025-03-14T05:46:40.9060276Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-bf268c9348de3dda.xml (deflated 45%)
2025-03-14T05:46:40.9066406Z   adding: test/test-reports/python-pytest/distributed.test_c10d_spawn_nccl/distributed.test_c10d_spawn_nccl-193a4317365a6ff9.xml (deflated 45%)
2025-03-14T05:46:40.9072746Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-fa23a1dbfdea9761.xml (deflated 34%)
2025-03-14T05:46:40.9078611Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3f43b48f4edaa9c3.xml (deflated 37%)
2025-03-14T05:46:40.9086197Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-991cae1778ba1451.xml (deflated 35%)
2025-03-14T05:46:40.9093857Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f1f8736be76de496.xml (deflated 42%)
2025-03-14T05:46:40.9100672Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9476bfa6f190ec9f.xml (deflated 35%)
2025-03-14T05:46:40.9106533Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-086048d04fe12c66.xml (deflated 35%)
2025-03-14T05:46:40.9112206Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-635be0bb902c2f36.xml (deflated 35%)
2025-03-14T05:46:40.9117980Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-037dd61820f72438.xml (deflated 35%)
2025-03-14T05:46:40.9123263Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-41f2480495c45256.xml (deflated 36%)
2025-03-14T05:46:40.9128559Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-17e1d27e52de740a.xml (deflated 35%)
2025-03-14T05:46:40.9133755Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-80aaafd26c9738e0.xml (deflated 35%)
2025-03-14T05:46:40.9139176Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-45d53ebb53c4f9f2.xml (deflated 42%)
2025-03-14T05:46:40.9144529Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c1e6e81537f7412e.xml (deflated 35%)
2025-03-14T05:46:40.9149977Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-55c51a37919303c2.xml (deflated 43%)
2025-03-14T05:46:40.9155261Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9ae133ecedaed34f.xml (deflated 36%)
2025-03-14T05:46:40.9160951Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4c1c5730cdd0aceb.xml (deflated 35%)
2025-03-14T05:46:40.9166824Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-88b13f4b28412cb7.xml (deflated 35%)
2025-03-14T05:46:40.9172232Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-af9d1f33ac9858cf.xml (deflated 36%)
2025-03-14T05:46:40.9177843Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9400b7a85b06f972.xml (deflated 42%)
2025-03-14T05:46:40.9183399Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b5a6346badaa0309.xml (deflated 35%)
2025-03-14T05:46:40.9189222Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e117666be3e12322.xml (deflated 36%)
2025-03-14T05:46:40.9195443Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-10499e3199f3e2bc.xml (deflated 43%)
2025-03-14T05:46:40.9201400Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-39c676992f64859d.xml (deflated 35%)
2025-03-14T05:46:40.9207234Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4af46b8d0a459608.xml (deflated 36%)
2025-03-14T05:46:40.9212685Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c8e996e4f5c9ca19.xml (deflated 43%)
2025-03-14T05:46:40.9218381Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3b1e15c7cb405e9c.xml (deflated 35%)
2025-03-14T05:46:40.9223874Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2a7236f9d0402daa.xml (deflated 36%)
2025-03-14T05:46:40.9229769Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-39d4374a61f1aa6e.xml (deflated 43%)
2025-03-14T05:46:40.9235263Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e1f37d52cf99f80e.xml (deflated 35%)
2025-03-14T05:46:40.9241002Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f50ea4eeee7c5e7f.xml (deflated 35%)
2025-03-14T05:46:40.9246738Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2acbeb2bfee4cfe7.xml (deflated 42%)
2025-03-14T05:46:40.9252953Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f31f74f82b28ad13.xml (deflated 36%)
2025-03-14T05:46:40.9258559Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-42cdcfb75a5ede96.xml (deflated 35%)
2025-03-14T05:46:40.9264207Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-73bd523b797596c8.xml (deflated 35%)
2025-03-14T05:46:40.9270162Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-cf53f6d71b4f3094.xml (deflated 42%)
2025-03-14T05:46:40.9276248Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0d357a2e10333424.xml (deflated 34%)
2025-03-14T05:46:40.9282246Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4dcf889ecbd011af.xml (deflated 35%)
2025-03-14T05:46:40.9288260Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-6e4010a0939a4438.xml (deflated 43%)
2025-03-14T05:46:40.9294125Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-71ebc0e0bc3649ad.xml (deflated 35%)
2025-03-14T05:46:40.9300086Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-75691552d9d9c3b0.xml (deflated 35%)
2025-03-14T05:46:40.9306008Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a3917af3d4af275e.xml (deflated 35%)
2025-03-14T05:46:40.9312083Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-accae35e4edc8264.xml (deflated 36%)
2025-03-14T05:46:40.9318154Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-dc855408623fd42d.xml (deflated 42%)
2025-03-14T05:46:40.9323842Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-23676c427ef017fc.xml (deflated 36%)
2025-03-14T05:46:40.9329825Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-142010923c2ea541.xml (deflated 42%)
2025-03-14T05:46:40.9335647Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-6bb953016c42f0dd.xml (deflated 35%)
2025-03-14T05:46:40.9341475Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-bad4fa625b8c9123.xml (deflated 36%)
2025-03-14T05:46:40.9347704Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-edac53109ab1f490.xml (deflated 47%)
2025-03-14T05:46:40.9353811Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8687df54db247df5.xml (deflated 35%)
2025-03-14T05:46:40.9359924Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-97b11dc3ce0b5a51.xml (deflated 36%)
2025-03-14T05:46:40.9366183Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-395b6eccfa703fa2.xml (deflated 35%)
2025-03-14T05:46:40.9372245Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-53dc4cd69e5f6149.xml (deflated 43%)
2025-03-14T05:46:40.9378272Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8f0e7aa32b862f98.xml (deflated 35%)
2025-03-14T05:46:40.9384634Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9a5168f6a12a1142.xml (deflated 42%)
2025-03-14T05:46:40.9390750Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-39b073912a270af9.xml (deflated 43%)
2025-03-14T05:46:40.9396594Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-26d4c42e1945720a.xml (deflated 43%)
2025-03-14T05:46:40.9402525Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-7e1b7ffa83ca7313.xml (deflated 42%)
2025-03-14T05:46:40.9408445Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-72ace09a3d306b1c.xml (deflated 42%)
2025-03-14T05:46:40.9414540Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-6e60d0eb3010b9ed.xml (deflated 41%)
2025-03-14T05:46:40.9420333Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b87463f870d28a85.xml (deflated 42%)
2025-03-14T05:46:40.9426439Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-849493acb72f76df.xml (deflated 41%)
2025-03-14T05:46:40.9432475Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e75bda59e7fa7fd9.xml (deflated 42%)
2025-03-14T05:46:40.9438376Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-448749c3db9a4174.xml (deflated 41%)
2025-03-14T05:46:40.9444726Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-97aa748cfa6d1f38.xml (deflated 42%)
2025-03-14T05:46:40.9450957Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2c840a3c982279bd.xml (deflated 42%)
2025-03-14T05:46:40.9457021Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2be211784b2f168d.xml (deflated 42%)
2025-03-14T05:46:40.9464433Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4378bbda2bfe25ae.xml (deflated 42%)
2025-03-14T05:46:40.9472251Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e6e26c8ab0666cf6.xml (deflated 41%)
2025-03-14T05:46:40.9477765Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-7db3a950c780b0bf.xml (deflated 42%)
2025-03-14T05:46:40.9483471Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2dd71a50ae9b2bbc.xml (deflated 36%)
2025-03-14T05:46:40.9489550Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-20217a73bcfe578a.xml (deflated 43%)
2025-03-14T05:46:40.9497842Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f1203b3bba84c32b.xml (deflated 36%)
2025-03-14T05:46:40.9506178Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2a949bb27d4580c5.xml (deflated 57%)
2025-03-14T05:46:40.9512067Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9ef6571f158780cb.xml (deflated 35%)
2025-03-14T05:46:40.9517763Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-9aa439d3218e3369.xml (deflated 36%)
2025-03-14T05:46:40.9524463Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-66a8e694e5b89783.xml (deflated 45%)
2025-03-14T05:46:40.9530539Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c04deab3354ee12d.xml (deflated 43%)
2025-03-14T05:46:40.9536399Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-24b554e16b84bc42.xml (deflated 44%)
2025-03-14T05:46:40.9542304Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-13893b389c304683.xml (deflated 43%)
2025-03-14T05:46:40.9548115Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-fa51a75b2eb224e8.xml (deflated 43%)
2025-03-14T05:46:40.9553552Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b45dc756d3b55dbe.xml (deflated 44%)
2025-03-14T05:46:40.9559408Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b70c12c5c6aa8863.xml (deflated 43%)
2025-03-14T05:46:40.9565215Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-8fa34c7adb81260a.xml (deflated 43%)
2025-03-14T05:46:40.9570760Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3604fb0b0f71a47f.xml (deflated 36%)
2025-03-14T05:46:40.9576920Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-fd38f313fd900b8f.xml (deflated 36%)
2025-03-14T05:46:40.9583107Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2a7d27ee90d69606.xml (deflated 36%)
2025-03-14T05:46:40.9588905Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-30a24e06592f7468.xml (deflated 36%)
2025-03-14T05:46:40.9594824Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-626e306192fb5d73.xml (deflated 44%)
2025-03-14T05:46:40.9600563Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-6eb7391d6fd7f4e8.xml (deflated 35%)
2025-03-14T05:46:40.9606037Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-5c8d2a4216208756.xml (deflated 43%)
2025-03-14T05:46:40.9611820Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-02460c426e0b54df.xml (deflated 36%)
2025-03-14T05:46:40.9617639Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f43952a2c219368a.xml (deflated 57%)
2025-03-14T05:46:40.9623469Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-e6ee3f400b8325f8.xml (deflated 42%)
2025-03-14T05:46:40.9629266Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2583399cca554f21.xml (deflated 41%)
2025-03-14T05:46:40.9635312Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-a27e7927cdb522bc.xml (deflated 36%)
2025-03-14T05:46:40.9641553Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2038695ee892109d.xml (deflated 35%)
2025-03-14T05:46:40.9647428Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-6cf972a15d642d5b.xml (deflated 36%)
2025-03-14T05:46:40.9653160Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0d910cbdd447990b.xml (deflated 36%)
2025-03-14T05:46:40.9659109Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-1835a0a5577f6a04.xml (deflated 36%)
2025-03-14T05:46:40.9664914Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4107911b1e32d30a.xml (deflated 36%)
2025-03-14T05:46:40.9670565Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-5204beebce095c06.xml (deflated 36%)
2025-03-14T05:46:40.9676956Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-b1b434967c506031.xml (deflated 36%)
2025-03-14T05:46:40.9684052Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0106f15a1f892e22.xml (deflated 44%)
2025-03-14T05:46:40.9689823Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4d21551aad96994d.xml (deflated 44%)
2025-03-14T05:46:40.9705204Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4eee43eb0bdd1f84.xml (deflated 43%)
2025-03-14T05:46:40.9716450Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-0bd2a776ef9c973e.xml (deflated 43%)
2025-03-14T05:46:40.9721502Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-3366fde4ba17dac5.xml (deflated 43%)
2025-03-14T05:46:40.9726838Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f0844fa8614abf1b.xml (deflated 43%)
2025-03-14T05:46:40.9732154Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2688a66b98fc9454.xml (deflated 43%)
2025-03-14T05:46:40.9737423Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-54228196c7cc71e3.xml (deflated 58%)
2025-03-14T05:46:40.9742660Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-d918bc990c4740aa.xml (deflated 43%)
2025-03-14T05:46:40.9747944Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-cad61b004bcb205f.xml (deflated 34%)
2025-03-14T05:46:40.9753981Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4a8cf4b98095a2ab.xml (deflated 33%)
2025-03-14T05:46:40.9758802Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-fe60307be03dd305.xml (deflated 34%)
2025-03-14T05:46:40.9764000Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-f5218f92b2daf2b9.xml (deflated 35%)
2025-03-14T05:46:40.9769728Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-4c1fdd4592cb6556.xml (deflated 34%)
2025-03-14T05:46:40.9775683Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-911f7e3d9ed960fb.xml (deflated 34%)
2025-03-14T05:46:40.9781253Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-13a4aaf16273316c.xml (deflated 34%)
2025-03-14T05:46:40.9787099Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-82cf5d9156187b12.xml (deflated 33%)
2025-03-14T05:46:40.9792930Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-c6f27d877dbe1ced.xml (deflated 34%)
2025-03-14T05:46:40.9799263Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-2130a32e8d660268.xml (deflated 34%)
2025-03-14T05:46:40.9803978Z   adding: test/test-reports/python-pytest/distributed.test_c10d_gloo/distributed.test_c10d_gloo-373877a3cd3d5bef.xml (deflated 34%)
2025-03-14T05:46:40.9809427Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_fx/distributed.fsdp.test_fsdp_fx-aaff532243ea335c.xml (deflated 27%)
2025-03-14T05:46:40.9815268Z   adding: test/test-reports/python-pytest/distributed.tensor.test_experimental_ops/distributed.tensor.test_experimental_ops-ef96ef1db4d76fde.xml (deflated 60%)
2025-03-14T05:46:40.9821662Z   adding: test/test-reports/python-pytest/distributed.checkpoint.test_fsdp_optim_state/distributed.checkpoint.test_fsdp_optim_state-99246c37b19dd011.xml (deflated 65%)
2025-03-14T05:46:40.9827562Z   adding: test/test-reports/python-pytest/distributed.checkpoint.test_save_load_api/distributed.checkpoint.test_save_load_api-ce0fd43f58b2b9ce.xml (deflated 61%)
2025-03-14T05:46:40.9834575Z   adding: test/test-reports/python-pytest/distributed._tools.test_memory_tracker/distributed._tools.test_memory_tracker-f995e25b62df273c.xml (deflated 44%)
2025-03-14T05:46:40.9839843Z   adding: test/test-reports/python-pytest/distributed.tensor.test_random_ops/distributed.tensor.test_random_ops-24bc668e05c818f5.xml (deflated 87%)
2025-03-14T05:46:40.9841195Z   adding: test/test-reports/python-pytest/distributed.pipelining.test_backward/distributed.pipelining.test_backward-f03cc547397567ee.xml (deflated 68%)
2025-03-14T05:46:40.9854606Z   adding: test/test-reports/python-pytest/distributed.tensor.test_dtensor_ops/distributed.tensor.test_dtensor_ops-4d13dc0c14a5e7e6.xml (deflated 96%)
2025-03-14T05:46:40.9856025Z   adding: test/test-reports/python-pytest/distributed.tensor.test_api/distributed.tensor.test_api-7e2c38ebd6d44559.xml (deflated 78%)
2025-03-14T05:46:40.9857337Z   adding: test/test-reports/python-pytest/distributed.checkpoint.test_format_utils/distributed.checkpoint.test_format_utils-bffd391f58182883.xml (deflated 59%)
2025-03-14T05:46:40.9858747Z   adding: test/test-reports/python-pytest/distributed.tensor.test_redistribute/distributed.tensor.test_redistribute-d78e69fde325ed86.xml (deflated 82%)
2025-03-14T05:46:40.9860212Z   adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_tp_random_state/distributed.tensor.parallel.test_tp_random_state-cb36056db7a13421.xml (deflated 46%)
2025-03-14T05:46:40.9861768Z   adding: test/test-reports/python-pytest/distributed.tensor.debug.test_comm_mode_features/distributed.tensor.debug.test_comm_mode_features-014b569eae9a9efb.xml (deflated 65%)
2025-03-14T05:46:40.9863185Z   adding: test/test-reports/python-pytest/distributed._tools.test_mem_tracker/distributed._tools.test_mem_tracker-383a3ecb2be2d2c5.xml (deflated 65%)
2025-03-14T05:46:40.9864533Z   adding: test/test-reports/python-pytest/distributed.pipelining.test_microbatch/distributed.pipelining.test_microbatch-d2ecc833081a2738.xml (deflated 60%)
2025-03-14T05:46:40.9866101Z   adding: test/test-reports/python-pytest/distributed.tensor.experimental.test_local_map/distributed.tensor.experimental.test_local_map-dc57daa55a89502c.xml (deflated 67%)
2025-03-14T05:46:40.9867608Z   adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_tp_style/distributed.tensor.parallel.test_tp_style-37f7770a0ff4d6a5.xml (deflated 80%)
2025-03-14T05:46:40.9869185Z   adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_ignore_params/distributed._composable.fsdp.test_fully_shard_ignore_params-2e330bd390e556f4.xml (deflated 45%)
2025-03-14T05:46:40.9870761Z   adding: test/test-reports/python-pytest/distributed.checkpoint.e2e.test_pipeline/distributed.checkpoint.e2e.test_pipeline-f3d4658907d83873.xml (deflated 38%)
2025-03-14T05:46:40.9872177Z   adding: test/test-reports/python-pytest/distributed.pipelining.test_unflatten/distributed.pipelining.test_unflatten-c0e0c691ae610e2e.xml (deflated 56%)
2025-03-14T05:46:40.9873605Z   adding: test/test-reports/python-pytest/distributed.checkpoint.test_tp_checkpoint/distributed.checkpoint.test_tp_checkpoint-4965e4ef2a8f6a7e.xml (deflated 65%)
2025-03-14T05:46:40.9875042Z   adding: test/test-reports/python-pytest/distributed._tools.test_runtime_estimator/distributed._tools.test_runtime_estimator-62803e6014463f94.xml (deflated 64%)
2025-03-14T05:46:40.9876396Z   adding: test/test-reports/python-pytest/distributed.checkpoint.test_fsspec/distributed.checkpoint.test_fsspec-44562920b855f27b.xml (deflated 65%)
2025-03-14T05:46:40.9877748Z   adding: test/test-reports/python-pytest/distributed.tensor.test_xla_integration/distributed.tensor.test_xla_integration-0e88037ec394879c.xml (deflated 74%)
2025-03-14T05:46:40.9879134Z   adding: test/test-reports/python-pytest/distributed.tensor.test_convolution_ops/distributed.tensor.test_convolution_ops-cea8224dacf5b46a.xml (deflated 66%)
2025-03-14T05:46:40.9880522Z   adding: test/test-reports/python-pytest/distributed._tools.test_fsdp2_mem_tracker/distributed._tools.test_fsdp2_mem_tracker-ad012309fbbcddfb.xml (deflated 60%)
2025-03-14T05:46:40.9881959Z   adding: test/test-reports/python-pytest/distributed.tensor.test_embedding_ops/distributed.tensor.test_embedding_ops-2f32702181bc849c.xml (deflated 67%)
2025-03-14T05:46:40.9883252Z   adding: test/test-reports/python-pytest/distributed.test_functional_api/distributed.test_functional_api-be2e4a4e35e04b5a.xml (deflated 48%)
2025-03-14T05:46:40.9884478Z   adding: test/test-reports/python-pytest/distributed.tensor.test_utils/distributed.tensor.test_utils-45018c010575e255.xml (deflated 77%)
2025-03-14T05:46:40.9885886Z   adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_overlap/distributed._composable.fsdp.test_fully_shard_overlap-bc665506903b8183.xml (deflated 62%)
2025-03-14T05:46:40.9887502Z   adding: test/test-reports/python-pytest/distributed.checkpoint.test_hsdp_checkpoint/distributed.checkpoint.test_hsdp_checkpoint-5d5dd4cb9fc7cb2b.xml (deflated 79%)
2025-03-14T05:46:40.9888984Z   adding: test/test-reports/python-pytest/distributed.checkpoint.test_fsdp_model_state/distributed.checkpoint.test_fsdp_model_state-7a83cab9384ec697.xml (deflated 65%)
2025-03-14T05:46:40.9890506Z   adding: test/test-reports/python-pytest/distributed.checkpoint.test_dtensor_checkpoint/distributed.checkpoint.test_dtensor_checkpoint-d17070bb190bbdbd.xml (deflated 47%)
2025-03-14T05:46:40.9891961Z   adding: test/test-reports/python-pytest/distributed.checkpoint.test_planner/distributed.checkpoint.test_planner-9d1ab7b5cc5f4d22.xml (deflated 74%)
2025-03-14T05:46:40.9893413Z   adding: test/test-reports/python-pytest/distributed.checkpoint.test_dtensor_resharding/distributed.checkpoint.test_dtensor_resharding-9e02ab442081987d.xml (deflated 84%)
2025-03-14T05:46:40.9894933Z   adding: test/test-reports/python-pytest/distributed.checkpoint.test_compatibility/distributed.checkpoint.test_compatibility-db0dbcfdb033dd48.xml (deflated 69%)
2025-03-14T05:46:40.9896311Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_memory/distributed.fsdp.test_fsdp_memory-fc9568ecd8d1241b.xml (deflated 64%)
2025-03-14T05:46:40.9897609Z   adding: test/test-reports/python-pytest/distributed.tensor.test_op_strategy/distributed.tensor.test_op_strategy-d1381d7f611e190a.xml (deflated 84%)
2025-03-14T05:46:40.9899271Z   adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_grad_scaler/distributed._composable.fsdp.test_fully_shard_grad_scaler-247805871cd399fc.xml (deflated 47%)
2025-03-14T05:46:40.9900664Z   adding: test/test-reports/python-pytest/distributed.test_fake_pg/distributed.test_fake_pg-706bb47931dcfde8.xml (deflated 82%)
2025-03-14T05:46:40.9901914Z   adding: test/test-reports/python-pytest/distributed._composable.test_replicate/distributed._composable.test_replicate-d80c2dd5a14cae01.xml (deflated 87%)
2025-03-14T05:46:40.9903270Z   adding: test/test-reports/python-pytest/distributed.tensor.test_optimizers/distributed.tensor.test_optimizers-68ed3a6bca1d0eec.xml (deflated 81%)
2025-03-14T05:46:40.9904665Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_backward_prefetch/distributed.fsdp.test_fsdp_backward_prefetch-3b899e9dad678063.xml (deflated 47%)
2025-03-14T05:46:40.9906152Z   adding: test/test-reports/python-pytest/distributed.checkpoint.test_utils/distributed.checkpoint.test_utils-58d70161aae32ac5.xml (deflated 80%)
2025-03-14T05:46:40.9907564Z   adding: test/test-reports/python-pytest/distributed.tensor.parallel.test_tp_examples/distributed.tensor.parallel.test_tp_examples-1b153667a583262e.xml (deflated 90%)
2025-03-14T05:46:40.9908943Z   adding: test/test-reports/python-pytest/distributed.test_serialization/distributed.test_serialization-36cc7f3e4b343d5c.xml (deflated 68%)
2025-03-14T05:46:40.9910158Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-f1a0a55f7fd43cb4.xml (deflated 44%)
2025-03-14T05:46:40.9911327Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-1b59a11949849efc.xml (deflated 44%)
2025-03-14T05:46:40.9912651Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-4cd0cb1253282744.xml (deflated 44%)
2025-03-14T05:46:40.9913810Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-307544534bb78f6d.xml (deflated 43%)
2025-03-14T05:46:40.9914968Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-f476485ee8ab0157.xml (deflated 44%)
2025-03-14T05:46:40.9916130Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-ebedca822949c6ed.xml (deflated 44%)
2025-03-14T05:46:40.9917298Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-465ebf088208948c.xml (deflated 44%)
2025-03-14T05:46:40.9918536Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-d065d0b759d358d7.xml (deflated 44%)
2025-03-14T05:46:40.9919696Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-18289687f7729195.xml (deflated 35%)
2025-03-14T05:46:40.9920864Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-373ea5c85b1eca16.xml (deflated 42%)
2025-03-14T05:46:40.9922025Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-3b8481bb5e5e5338.xml (deflated 42%)
2025-03-14T05:46:40.9923279Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-dadf824d68d9399b.xml (deflated 35%)
2025-03-14T05:46:40.9924441Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-5d13be3f2227375a.xml (deflated 35%)
2025-03-14T05:46:40.9925612Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-27edfafe1a459ea6.xml (deflated 35%)
2025-03-14T05:46:40.9926773Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-271f08e276a6a114.xml (deflated 42%)
2025-03-14T05:46:40.9927934Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-8927570b28d684ef.xml (deflated 42%)
2025-03-14T05:46:40.9929086Z   adding: test/test-reports/python-pytest/distributed.test_pg_wrapper/distributed.test_pg_wrapper-847a36e9b848ed1a.xml (deflated 34%)
2025-03-14T05:46:40.9930468Z   adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_frozen/distributed._composable.fsdp.test_fully_shard_frozen-e738a866361fade5.xml (deflated 62%)
2025-03-14T05:46:40.9932052Z   adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_init/distributed._composable.fsdp.test_fully_shard_init-181901fe23faabed.xml (deflated 92%)
2025-03-14T05:46:40.9933513Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_optim_state/distributed.fsdp.test_fsdp_optim_state-4b3a2b2568ab45fb.xml (deflated 96%)
2025-03-14T05:46:40.9934834Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_grad_acc/distributed.fsdp.test_fsdp_grad_acc-3d3bad13e33464eb.xml (deflated 84%)
2025-03-14T05:46:40.9936113Z   adding: test/test-reports/python-pytest/distributed.test_multi_threaded_pg/distributed.test_multi_threaded_pg-0f266c333352d022.xml (deflated 81%)
2025-03-14T05:46:40.9937554Z   adding: test/test-reports/python-pytest/distributed._composable.test_replicate_with_compiler/distributed._composable.test_replicate_with_compiler-98603fbe64b9dbe3.xml (deflated 83%)
2025-03-14T05:46:40.9939019Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_fine_tune/distributed.fsdp.test_fsdp_fine_tune-6cdb43647c4456c6.xml (deflated 28%)
2025-03-14T05:46:40.9940331Z   adding: test/test-reports/python-pytest/distributed.nn.jit.test_instantiator/distributed.nn.jit.test_instantiator-48c62a62271c6ade.xml (deflated 61%)
2025-03-14T05:46:40.9941833Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_ignored_modules/distributed.fsdp.test_fsdp_ignored_modules-041abde3fbbc75a4.xml (deflated 79%)
2025-03-14T05:46:40.9943306Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_traversal/distributed.fsdp.test_fsdp_traversal-0fa5e2701cc079bf.xml (deflated 27%)
2025-03-14T05:46:40.9944796Z   adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_state_dict/distributed._composable.fsdp.test_fully_shard_state_dict-0d6484ed78e44331.xml (deflated 75%)
2025-03-14T05:46:40.9946341Z   adding: test/test-reports/python-pytest/distributed.algorithms.test_join/distributed.algorithms.test_join-ad0affab6f9bb4c5.xml (deflated 79%)
2025-03-14T05:46:40.9947690Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_tp_integration/distributed.fsdp.test_fsdp_tp_integration-5489f12654667a6f.xml (deflated 70%)
2025-03-14T05:46:40.9949107Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-576b5b53d5402547.xml (deflated 36%)
2025-03-14T05:46:40.9950366Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-e212f52566d220e2.xml (deflated 36%)
2025-03-14T05:46:40.9951623Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-77cc9b5fbee420db.xml (deflated 36%)
2025-03-14T05:46:40.9952887Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-c2a97c0dd60dcf54.xml (deflated 36%)
2025-03-14T05:46:40.9954150Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-911640bf5bb16932.xml (deflated 36%)
2025-03-14T05:46:40.9955418Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-f98feeebe0a039f1.xml (deflated 37%)
2025-03-14T05:46:40.9956678Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-76bfd5c720f0e4c1.xml (deflated 35%)
2025-03-14T05:46:40.9957934Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-d2e4586b429e3337.xml (deflated 36%)
2025-03-14T05:46:40.9959187Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-d145da94c2764074.xml (deflated 36%)
2025-03-14T05:46:40.9960446Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-0ce50960cc6acf37.xml (deflated 36%)
2025-03-14T05:46:40.9961705Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-8df08475dcd6c435.xml (deflated 36%)
2025-03-14T05:46:40.9962973Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-cb99e511b566cded.xml (deflated 36%)
2025-03-14T05:46:40.9964235Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-4ff699c6fca54eb9.xml (deflated 36%)
2025-03-14T05:46:40.9965498Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-ba132aa88a1c1828.xml (deflated 36%)
2025-03-14T05:46:40.9966755Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-1130d4486cf9028a.xml (deflated 35%)
2025-03-14T05:46:40.9968005Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-f7041504878f1cf7.xml (deflated 36%)
2025-03-14T05:46:40.9969255Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-50b1b400311060ed.xml (deflated 36%)
2025-03-14T05:46:40.9970515Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-cbbc607f569edfc0.xml (deflated 36%)
2025-03-14T05:46:40.9971781Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-320bae630dbe2a51.xml (deflated 36%)
2025-03-14T05:46:40.9973309Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-32acd6dc68e471b9.xml (deflated 35%)
2025-03-14T05:46:40.9974571Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-eb4c81c5312d020c.xml (deflated 36%)
2025-03-14T05:46:40.9975830Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-1c046a09bdabf34e.xml (deflated 35%)
2025-03-14T05:46:40.9977088Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-09ae36b53f402eb7.xml (deflated 36%)
2025-03-14T05:46:40.9978404Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-d8e2b90e3e2c3d79.xml (deflated 35%)
2025-03-14T05:46:40.9979665Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-fddca96b49a0f2cc.xml (deflated 35%)
2025-03-14T05:46:40.9980935Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-0cffb7b01ff2dfe6.xml (deflated 35%)
2025-03-14T05:46:40.9982195Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-d964a4b12ec31485.xml (deflated 35%)
2025-03-14T05:46:40.9983449Z   adding: test/test-reports/python-pytest/distributed.rpc.test_faulty_agent/distributed.rpc.test_faulty_agent-5c4d20bd8767cad7.xml (deflated 35%)
2025-03-14T05:46:40.9984756Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a8cfcc7cdbcc4192.xml (deflated 36%)
2025-03-14T05:46:40.9986180Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-062a165227c60295.xml (deflated 36%)
2025-03-14T05:46:40.9987528Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1ed18b3e774df711.xml (deflated 36%)
2025-03-14T05:46:40.9988877Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ec961c23e84b5acd.xml (deflated 35%)
2025-03-14T05:46:40.9990218Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-63ff369e65ab51f6.xml (deflated 35%)
2025-03-14T05:46:40.9991558Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-05bf9341b4644323.xml (deflated 35%)
2025-03-14T05:46:40.9992901Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c5ad2790772e3abc.xml (deflated 35%)
2025-03-14T05:46:40.9994247Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-58fa80dde17cb9d4.xml (deflated 56%)
2025-03-14T05:46:40.9995589Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-44e58d08f28c04cb.xml (deflated 35%)
2025-03-14T05:46:40.9996934Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-041ca5e442d7604f.xml (deflated 35%)
2025-03-14T05:46:40.9998438Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f61e1fc98dd7c99d.xml (deflated 35%)
2025-03-14T05:46:40.9999799Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d7043cf6afb8a9df.xml (deflated 35%)
2025-03-14T05:46:41.0001155Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0dab411cbc3508ea.xml (deflated 36%)
2025-03-14T05:46:41.0002501Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f917cdd2794e895f.xml (deflated 36%)
2025-03-14T05:46:41.0003951Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2ef67a175436b6cd.xml (deflated 35%)
2025-03-14T05:46:41.0005295Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1d85d28428bae7ed.xml (deflated 36%)
2025-03-14T05:46:41.0006637Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c4dab2ff66a8fc54.xml (deflated 35%)
2025-03-14T05:46:41.0008063Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2b2b543838bf9069.xml (deflated 36%)
2025-03-14T05:46:41.0009402Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-00ed097b4d5bcab3.xml (deflated 35%)
2025-03-14T05:46:41.0010744Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5c6740d56d530d6b.xml (deflated 35%)
2025-03-14T05:46:41.0012099Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-55a15e9b2531c7d5.xml (deflated 36%)
2025-03-14T05:46:41.0013444Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fbed8e8a174cb561.xml (deflated 35%)
2025-03-14T05:46:41.0014797Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-377dafe2ccb81fac.xml (deflated 36%)
2025-03-14T05:46:41.0016145Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-69b9895ceedd0e02.xml (deflated 35%)
2025-03-14T05:46:41.0017491Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c5619f37704ecc18.xml (deflated 36%)
2025-03-14T05:46:41.0018843Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5b00439d823a03b1.xml (deflated 36%)
2025-03-14T05:46:41.0020178Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-43b00e462210572c.xml (deflated 36%)
2025-03-14T05:46:41.0021522Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-977297fdbf41cc14.xml (deflated 35%)
2025-03-14T05:46:41.0022860Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e8276faf1e99161a.xml (deflated 36%)
2025-03-14T05:46:41.0024296Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b7f0a353cd9ab8cd.xml (deflated 36%)
2025-03-14T05:46:41.0025725Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-64dec08706193f2f.xml (deflated 36%)
2025-03-14T05:46:41.0027071Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-990e0ffcb5280d1e.xml (deflated 35%)
2025-03-14T05:46:41.0028430Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-53ee3f909804c7a1.xml (deflated 35%)
2025-03-14T05:46:41.0029775Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ecf4b2040b5e4b40.xml (deflated 34%)
2025-03-14T05:46:41.0031122Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-485d8fbf7b8478ce.xml (deflated 35%)
2025-03-14T05:46:41.0032465Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3c0960d951feeb41.xml (deflated 35%)
2025-03-14T05:46:41.0033871Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-493ae3f6aeb6a342.xml (deflated 35%)
2025-03-14T05:46:41.0035208Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9073d2ce8d166b51.xml (deflated 34%)
2025-03-14T05:46:41.0036547Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-19b7f4b7233666de.xml (deflated 34%)
2025-03-14T05:46:41.0037889Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8513e46afa6f9d37.xml (deflated 35%)
2025-03-14T05:46:41.0039281Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-be57f351ac813582.xml (deflated 34%)
2025-03-14T05:46:41.0040622Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3c711845dd8fef3d.xml (deflated 35%)
2025-03-14T05:46:41.0041962Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d0babf197156a4ac.xml (deflated 35%)
2025-03-14T05:46:41.0043299Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-82446e48b9e3ac55.xml (deflated 35%)
2025-03-14T05:46:41.0044634Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-075b706546a88ae5.xml (deflated 35%)
2025-03-14T05:46:41.0045977Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cf841a8bec28217f.xml (deflated 34%)
2025-03-14T05:46:41.0047315Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ee41433e4166db3f.xml (deflated 35%)
2025-03-14T05:46:41.0048655Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-835ac486d157cd8e.xml (deflated 35%)
2025-03-14T05:46:41.0049992Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e14460af1420c5d1.xml (deflated 35%)
2025-03-14T05:46:41.0051330Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7ef2d7fe2fe19993.xml (deflated 35%)
2025-03-14T05:46:41.0052673Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d9d8a8d0a3c3cad9.xml (deflated 35%)
2025-03-14T05:46:41.0054078Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-11d1dd67460135f4.xml (deflated 34%)
2025-03-14T05:46:41.0055423Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ce9f6875ec42ac63.xml (deflated 35%)
2025-03-14T05:46:41.0056770Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e8ca853fc2e6b7b4.xml (deflated 35%)
2025-03-14T05:46:41.0058118Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d1e0db3c2d5bf828.xml (deflated 35%)
2025-03-14T05:46:41.0059459Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7bc5099d6d2379be.xml (deflated 35%)
2025-03-14T05:46:41.0060804Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f432e3386caaac2d.xml (deflated 35%)
2025-03-14T05:46:41.0062148Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-98a5400b1499fc7a.xml (deflated 35%)
2025-03-14T05:46:41.0063567Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-66d4a3efbd6ff48a.xml (deflated 34%)
2025-03-14T05:46:41.0064911Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1b5b64c95b712603.xml (deflated 35%)
2025-03-14T05:46:41.0066320Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a4b966e6e521860c.xml (deflated 36%)
2025-03-14T05:46:41.0067667Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c3c50f54b6d15db8.xml (deflated 35%)
2025-03-14T05:46:41.0069069Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8d0f64f5804666ba.xml (deflated 34%)
2025-03-14T05:46:41.0070415Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-407b1a21de9aaaaa.xml (deflated 36%)
2025-03-14T05:46:41.0071765Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1c139b1fad233250.xml (deflated 37%)
2025-03-14T05:46:41.0073108Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-97bf52affeba9867.xml (deflated 38%)
2025-03-14T05:46:41.0074448Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cbd7474e51a53924.xml (deflated 36%)
2025-03-14T05:46:41.0075792Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cbf20f22d27a143e.xml (deflated 37%)
2025-03-14T05:46:41.0077128Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-35001b93526975d5.xml (deflated 36%)
2025-03-14T05:46:41.0078463Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-67059de0e12d9deb.xml (deflated 36%)
2025-03-14T05:46:41.0079804Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6c8f924d9f719f8a.xml (deflated 36%)
2025-03-14T05:46:41.0081143Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0420f4261d3ec9bf.xml (deflated 38%)
2025-03-14T05:46:41.0082484Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-85bd5ff312b20d7b.xml (deflated 37%)
2025-03-14T05:46:41.0083833Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3acfe7fb90cf0114.xml (deflated 36%)
2025-03-14T05:46:41.0085174Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-dfce074ffdcc2575.xml (deflated 36%)
2025-03-14T05:46:41.0086526Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c1bfff4f1f7f10c3.xml (deflated 36%)
2025-03-14T05:46:41.0087867Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5aeb9c5f65a905e3.xml (deflated 36%)
2025-03-14T05:46:41.0089209Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-923c5659becb25f2.xml (deflated 36%)
2025-03-14T05:46:41.0090545Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d61c55c44e520689.xml (deflated 36%)
2025-03-14T05:46:41.0091891Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b3e921faede90c86.xml (deflated 36%)
2025-03-14T05:46:41.0093234Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-eaa493bc72fea597.xml (deflated 36%)
2025-03-14T05:46:41.0094629Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-69b3586be0748432.xml (deflated 36%)
2025-03-14T05:46:41.0095968Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ee132175cf9cd668.xml (deflated 35%)
2025-03-14T05:46:41.0097303Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-25d17ffd1f607b6d.xml (deflated 35%)
2025-03-14T05:46:41.0098824Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-eae4b08db5c07586.xml (deflated 35%)
2025-03-14T05:46:41.0100411Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-eafd786ff7130cd0.xml (deflated 35%)
2025-03-14T05:46:41.0102561Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ba68cde94537b9f7.xml (deflated 35%)
2025-03-14T05:46:41.0104825Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-20fc15be16d78f0c.xml (deflated 35%)
2025-03-14T05:46:41.0107248Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b2fec7f419576ee9.xml (deflated 35%)
2025-03-14T05:46:41.0109663Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5741026089963d8d.xml (deflated 35%)
2025-03-14T05:46:41.0112130Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0d603d61622e1e4a.xml (deflated 36%)
2025-03-14T05:46:41.0114619Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6494b5611c3f4ba0.xml (deflated 35%)
2025-03-14T05:46:41.0117194Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-674d86b5645e96e1.xml (deflated 36%)
2025-03-14T05:46:41.0119752Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ce199abd6a17fd02.xml (deflated 35%)
2025-03-14T05:46:41.0122234Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-79559291794ac108.xml (deflated 36%)
2025-03-14T05:46:41.0124550Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2c5c1de840af5ba7.xml (deflated 36%)
2025-03-14T05:46:41.0127053Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ebdce61965382034.xml (deflated 36%)
2025-03-14T05:46:41.0129505Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-99a8394d07f576e4.xml (deflated 35%)
2025-03-14T05:46:41.0132010Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c7c1624c3e49dbbb.xml (deflated 35%)
2025-03-14T05:46:41.0134525Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-068375ccbfb63d61.xml (deflated 36%)
2025-03-14T05:46:41.0136953Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9be9c1e27f801bfe.xml (deflated 35%)
2025-03-14T05:46:41.0139515Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-7fb1a944efba17b8.xml (deflated 35%)
2025-03-14T05:46:41.0142013Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9708ce8d70d31bbd.xml (deflated 35%)
2025-03-14T05:46:41.0144688Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-59fe254264a53b2e.xml (deflated 35%)
2025-03-14T05:46:41.0147298Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2c4f1a7ba1f337a2.xml (deflated 35%)
2025-03-14T05:46:41.0149788Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-876f4b43c3203cc1.xml (deflated 35%)
2025-03-14T05:46:41.0152322Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-04d096c9fb2bc3c2.xml (deflated 36%)
2025-03-14T05:46:41.0154932Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-3cbbe088b20ebf27.xml (deflated 35%)
2025-03-14T05:46:41.0157439Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8977792bed46ee3f.xml (deflated 36%)
2025-03-14T05:46:41.0160027Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d203721f6709a0e0.xml (deflated 36%)
2025-03-14T05:46:41.0162598Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c8ffbd8c4c57ac98.xml (deflated 36%)
2025-03-14T05:46:41.0165189Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-182f4944f13b169d.xml (deflated 36%)
2025-03-14T05:46:41.0167780Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-36aa5741069a237e.xml (deflated 38%)
2025-03-14T05:46:41.0170341Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-bebc62c5af13708f.xml (deflated 35%)
2025-03-14T05:46:41.0172886Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-25954989386f9ea8.xml (deflated 35%)
2025-03-14T05:46:41.0175510Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-efbfb6b0f8c61f26.xml (deflated 36%)
2025-03-14T05:46:41.0178092Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f656ede095b9b0d8.xml (deflated 36%)
2025-03-14T05:46:41.0180666Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-bf289e477708ff09.xml (deflated 36%)
2025-03-14T05:46:41.0183205Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-8b31bd682a7b7645.xml (deflated 36%)
2025-03-14T05:46:41.0185918Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c4208203982472f2.xml (deflated 35%)
2025-03-14T05:46:41.0188501Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-982784653188f45d.xml (deflated 35%)
2025-03-14T05:46:41.0191051Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-097979c4a84746f8.xml (deflated 36%)
2025-03-14T05:46:41.0193623Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6af4b97a5d0b5cee.xml (deflated 35%)
2025-03-14T05:46:41.0196146Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2f1d70910c90df4f.xml (deflated 37%)
2025-03-14T05:46:41.0198870Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ecf201043c9fb04a.xml (deflated 38%)
2025-03-14T05:46:41.0201351Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6d7b35fef97076fd.xml (deflated 35%)
2025-03-14T05:46:41.0204040Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0993570960167cd9.xml (deflated 35%)
2025-03-14T05:46:41.0206599Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-15b8c8fb400a4d2e.xml (deflated 35%)
2025-03-14T05:46:41.0209173Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-592ef0dad9445a98.xml (deflated 35%)
2025-03-14T05:46:41.0211932Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9b92091f5cf482f6.xml (deflated 35%)
2025-03-14T05:46:41.0214522Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f4efe5c07b1c6b79.xml (deflated 35%)
2025-03-14T05:46:41.0217108Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6871c7b02c3237f6.xml (deflated 34%)
2025-03-14T05:46:41.0219186Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2d2fcc9276c8cfbe.xml (deflated 36%)
2025-03-14T05:46:41.0220556Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4cca37560b8fcd2d.xml (deflated 35%)
2025-03-14T05:46:41.0221913Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f0c47ad85b2b7feb.xml (deflated 36%)
2025-03-14T05:46:41.0223367Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c97440acfb366c0b.xml (deflated 35%)
2025-03-14T05:46:41.0224712Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-cb48221526411b4b.xml (deflated 35%)
2025-03-14T05:46:41.0226140Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ae190cc1af16c1a2.xml (deflated 36%)
2025-03-14T05:46:41.0227483Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-2ec62b470366e4f6.xml (deflated 35%)
2025-03-14T05:46:41.0228825Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-178c3e95ca310381.xml (deflated 35%)
2025-03-14T05:46:41.0230161Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9c8f3e460e71d077.xml (deflated 35%)
2025-03-14T05:46:41.0231504Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-14052d7667ae14af.xml (deflated 35%)
2025-03-14T05:46:41.0232839Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-4c918a8969753249.xml (deflated 34%)
2025-03-14T05:46:41.0234175Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-1696955283836ff6.xml (deflated 37%)
2025-03-14T05:46:41.0235513Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-90d22296a80c4e3b.xml (deflated 35%)
2025-03-14T05:46:41.0236855Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-0c25513f5b15ef28.xml (deflated 55%)
2025-03-14T05:46:41.0238201Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ae8cd226938b8783.xml (deflated 36%)
2025-03-14T05:46:41.0239542Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-6f3e2acdbfa48d6f.xml (deflated 36%)
2025-03-14T05:46:41.0241021Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-def45bc056638939.xml (deflated 37%)
2025-03-14T05:46:41.0242368Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-da594aa60c9f2996.xml (deflated 36%)
2025-03-14T05:46:41.0243715Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-5dc3924c25d8d0d3.xml (deflated 37%)
2025-03-14T05:46:41.0245066Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-9f85db334bd0493b.xml (deflated 37%)
2025-03-14T05:46:41.0246467Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-14afb069d654fc55.xml (deflated 37%)
2025-03-14T05:46:41.0247804Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-956960ce0fcc0bc5.xml (deflated 37%)
2025-03-14T05:46:41.0249150Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-b83bde09d1ddf0b4.xml (deflated 38%)
2025-03-14T05:46:41.0250491Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-f5305abaa1521663.xml (deflated 36%)
2025-03-14T05:46:41.0251831Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d3aba43121c17447.xml (deflated 38%)
2025-03-14T05:46:41.0253175Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-a3b89943fd3ac771.xml (deflated 39%)
2025-03-14T05:46:41.0254514Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-e63201c782e7fa2f.xml (deflated 36%)
2025-03-14T05:46:41.0255862Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-c62ae30ceb01c56f.xml (deflated 36%)
2025-03-14T05:46:41.0257213Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-143d6d2bd1bff6ef.xml (deflated 37%)
2025-03-14T05:46:41.0258555Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-693b79fc4b57e89f.xml (deflated 37%)
2025-03-14T05:46:41.0259893Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-08f967c8ee6655da.xml (deflated 37%)
2025-03-14T05:46:41.0261245Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-d701bd4216aff09c.xml (deflated 37%)
2025-03-14T05:46:41.0262591Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-fff6c03ca1274448.xml (deflated 36%)
2025-03-14T05:46:41.0263935Z   adding: test/test-reports/python-pytest/distributed.rpc.test_tensorpipe_agent/distributed.rpc.test_tensorpipe_agent-ab406948d5038ace.xml (deflated 36%)
2025-03-14T05:46:41.0265341Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_checkpoint/distributed.fsdp.test_fsdp_checkpoint-18beadcbcab5a17d.xml (deflated 93%)
2025-03-14T05:46:41.0266635Z   adding: test/test-reports/python-pytest/distributed.launcher.test_run/distributed.launcher.test_run-b58a2db8d18a9939.xml (deflated 84%)
2025-03-14T05:46:41.0267927Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_hybrid_shard/distributed.fsdp.test_fsdp_hybrid_shard-13a9a3bd5373d264.xml (deflated 81%)
2025-03-14T05:46:41.0269337Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_multiple_forward/distributed.fsdp.test_fsdp_multiple_forward-3f46a6e1af2cd42a.xml (deflated 45%)
2025-03-14T05:46:41.0270917Z   adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_tensor_ops/distributed._shard.sharded_tensor.ops.test_tensor_ops-e75399d6111ec6e8.xml (deflated 83%)
2025-03-14T05:46:41.0272501Z   adding: test/test-reports/python-pytest/distributed._shard.sharded_optim.test_sharded_optim/distributed._shard.sharded_optim.test_sharded_optim-eb46ee577c8eaee0.xml (deflated 67%)
2025-03-14T05:46:41.0274071Z   adding: test/test-reports/python-pytest/distributed._shard.sharding_plan.test_sharding_plan/distributed._shard.sharding_plan.test_sharding_plan-730e9a7d09df53e0.xml (deflated 75%)
2025-03-14T05:46:41.0275424Z   adding: test/test-reports/python-pytest/distributed.test_launcher/distributed.test_launcher-3f463712d56a59fa.xml (deflated 39%)
2025-03-14T05:46:41.0276733Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_clip_grad_norm/distributed.fsdp.test_fsdp_clip_grad_norm-02b46a7435d91d46.xml (deflated 28%)
2025-03-14T05:46:41.0278044Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_comm/distributed.fsdp.test_fsdp_comm-5fe9d01817bba68c.xml (deflated 28%)
2025-03-14T05:46:41.0279468Z   adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_embedding/distributed._shard.sharded_tensor.ops.test_embedding-bc73cdcb83dfdfd2.xml (deflated 68%)
2025-03-14T05:46:41.0281102Z   adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_embedding_bag/distributed._shard.sharded_tensor.ops.test_embedding_bag-96b085576ff5d1f7.xml (deflated 68%)
2025-03-14T05:46:41.0282760Z   adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_extensions/distributed._composable.fsdp.test_fully_shard_extensions-7ee9388293113789.xml (deflated 80%)
2025-03-14T05:46:41.0284348Z   adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.ops.test_init/distributed._shard.sharded_tensor.ops.test_init-d12dc90e6f18d389.xml (deflated 76%)
2025-03-14T05:46:41.0285928Z   adding: test/test-reports/python-pytest/distributed._composable.fsdp.test_fully_shard_training/distributed._composable.fsdp.test_fully_shard_training-35d0fcf4d6fccbd7.xml (deflated 85%)
2025-03-14T05:46:41.0287358Z   adding: test/test-reports/python-pytest/distributed._shard.test_sharder/distributed._shard.test_sharder-8ab38b3d6a3d376b.xml (deflated 66%)
2025-03-14T05:46:41.0288555Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_wrap/distributed.fsdp.test_wrap-f0064a576be35218.xml (deflated 94%)
2025-03-14T05:46:41.0289830Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_unshard_params/distributed.fsdp.test_fsdp_unshard_params-17e3ca78b97f84de.xml (deflated 89%)
2025-03-14T05:46:41.0291231Z   adding: test/test-reports/python-pytest/distributed.fsdp.test_fsdp_use_orig_params/distributed.fsdp.test_fsdp_use_orig_params-eaae42023d009c1a.xml (deflated 90%)
2025-03-14T05:46:41.0292735Z   adding: test/test-reports/python-pytest/distributed._shard.sharded_tensor.test_sharded_tensor/distributed._shard.sharded_tensor.test_sharded_tensor-a855aa03dff6e096.xml (deflated 95%)
2025-03-14T05:46:41.0294228Z   adding: test/test-reports/python-pytest/distributed._tools.test_fake_collectives/distributed._tools.test_fake_collectives-7315dd097643014c.xml (deflated 46%)
2025-03-14T05:46:41.0295599Z   adding: test/test-reports/python-pytest/distributed.elastic.events.lib_test/distributed.elastic.events.lib_test-aa600a7c7ed48373.xml (deflated 76%)
2025-03-14T05:46:41.0296944Z   adding: test/test-reports/python-pytest/distributed.elastic.metrics.api_test/distributed.elastic.metrics.api_test-b67dda6eeaf20d14.xml (deflated 60%)
2025-03-14T05:46:41.0298587Z   adding: test/test-reports/python-pytest/distributed.elastic.multiprocessing.api_test/distributed.elastic.multiprocessing.api_test-daf8d33489084f79.xml (deflated 84%)
2025-03-14T05:46:41.0300135Z   adding: test/test-reports/python-pytest/distributed.elastic.timer.local_timer_example/distributed.elastic.timer.local_timer_example-d0fdb580ae6867b4.xml (deflated 51%)
2025-03-14T05:46:41.0301725Z   adding: test/test-reports/python-pytest/distributed.elastic.timer.local_timer_test/distributed.elastic.timer.local_timer_test-120b2d99602b4deb.xml (deflated 80%)
2025-03-14T05:46:41.0303201Z   adding: test/test-reports/python-pytest/distributed.elastic.utils.distributed_test/distributed.elastic.utils.distributed_test-102fc69947e52485.xml (deflated 78%)
2025-03-14T05:46:41.0304650Z   adding: test/test-reports/python-pytest/distributed.elastic.utils.logging_test/distributed.elastic.utils.logging_test-dc6379d8273f329d.xml (deflated 51%)
2025-03-14T05:46:41.0306091Z   adding: test/test-reports/python-pytest/distributed.elastic.utils.util_test/distributed.elastic.utils.util_test-dba8f3a0e41f58bc.xml (deflated 81%)
2025-03-14T05:46:41.0338017Z ##[group]Run # Remove any previous usage logs if they exist
2025-03-14T05:46:41.0338530Z # Remove any previous usage logs if they exist
2025-03-14T05:46:41.0338935Z rm -f logs-*.zip
2025-03-14T05:46:41.0339433Z # this workflow is also run in bazel build test, but we dont generate usage reports for it
2025-03-14T05:46:41.0340024Z # so check to see if the file exists first
2025-03-14T05:46:41.0340413Z if [ -f 'usage_log.txt' ]; then
2025-03-14T05:46:41.0340814Z     zip "logs-${FILE_SUFFIX}.zip" 'usage_log.txt'
2025-03-14T05:46:41.0341188Z fi
2025-03-14T05:46:41.0341587Z if find "test/test-reports" -name "*.log" 2>/dev/null | grep -q .; then
2025-03-14T05:46:41.0342175Z     zip -r "logs-${FILE_SUFFIX}.zip" test/test-reports -i '*.log'
2025-03-14T05:46:41.0342600Z fi
2025-03-14T05:46:41.0348170Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:41.0348586Z env:
2025-03-14T05:46:41.0348831Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:41.0349332Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:41.0349983Z   FILE_SUFFIX: test-distributed-2-2-lf.linux.2xlarge_38754724585
2025-03-14T05:46:41.0350410Z ##[endgroup]
2025-03-14T05:46:41.0425831Z   adding: usage_log.txt (deflated 94%)
2025-03-14T05:46:41.0533655Z   adding: test/test-reports/distributed.elastic.multiprocessing.api_test_1.1_e8fa40a0549bf30e_.log (deflated 84%)
2025-03-14T05:46:41.0540132Z   adding: test/test-reports/distributed.fsdp.test_fsdp_dtensor_state_dict_1.1_27f29faadcc8f1d9_.log (deflated 61%)
2025-03-14T05:46:41.0545738Z   adding: test/test-reports/distributed.fsdp.test_fsdp_core_1.1_1f3cf1055d4f142a_.log (deflated 51%)
2025-03-14T05:46:41.0546587Z   adding: test/test-reports/distributed.elastic.timer.api_test_1.1_caaf4c5fd9e908d8_.log (stored 0%)
2025-03-14T05:46:41.0552070Z   adding: test/test-reports/distributed.fsdp.test_distributed_checkpoint_1.1_97bd9714d438be1f_.log (deflated 53%)
2025-03-14T05:46:41.0559346Z   adding: test/test-reports/distributed.fsdp.test_fsdp_misc_1.1_917045c2fda1d041_.log (deflated 84%)
2025-03-14T05:46:41.0565360Z   adding: test/test-reports/distributed.test_c10d_nccl_1.1_60d56c6a7c95f624_.log (stored 0%)
2025-03-14T05:46:41.0572530Z   adding: test/test-reports/distributed.test_c10d_functional_native_1.1_dd6f12771316463b_.log (deflated 86%)
2025-03-14T05:46:41.0580173Z   adding: test/test-reports/distributed.test_c10d_pypg_1.1_c37b93fab008fc91_.log (deflated 90%)
2025-03-14T05:46:41.0586446Z   adding: test/test-reports/distributed.test_c10d_logger_1.1_102273f250240740_.log (deflated 55%)
2025-03-14T05:46:41.0587849Z   adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_training_1.1_069be336e479bfd3_.log (deflated 85%)
2025-03-14T05:46:41.0594208Z   adding: test/test-reports/distributed._composable.test_composability.test_pp_composability_1.1_e331726cd66b54f9_.log (deflated 84%)
2025-03-14T05:46:41.0600975Z   adding: test/test-reports/distributed.test_c10d_object_collectives_1.1_e9444a739eb4dfd7_.log (deflated 76%)
2025-03-14T05:46:41.0607562Z   adding: test/test-reports/distributed._tools.test_mod_tracker_1.1_8a6455da247091c2_.log (deflated 63%)
2025-03-14T05:46:41.0613403Z   adding: test/test-reports/distributed.tensor.debug.test_op_coverage_1.1_22c714e57cb502e4_.log (deflated 54%)
2025-03-14T05:46:41.0638065Z   adding: test/test-reports/distributed.test_c10d_spawn_gloo_1.1_85ea6b4a208590aa_.log (deflated 93%)
2025-03-14T05:46:41.0654284Z   adding: test/test-reports/distributed.test_c10d_spawn_nccl_1.1_d823d18f16e1f359_.log (deflated 93%)
2025-03-14T05:46:41.0680966Z   adding: test/test-reports/distributed.test_c10d_gloo_1.1_7f023791449f76f0_.log (deflated 96%)
2025-03-14T05:46:41.0686286Z   adding: test/test-reports/distributed.fsdp.test_fsdp_fx_1.1_91e8bed7e84ddc65_.log (deflated 51%)
2025-03-14T05:46:41.0691732Z   adding: test/test-reports/distributed.tensor.test_experimental_ops_1.1_51d72d2486387279_.log (deflated 64%)
2025-03-14T05:46:41.0692989Z   adding: test/test-reports/distributed.elastic.timer.local_timer_example_1.1_75f544fdac4daf9d_.log (deflated 68%)
2025-03-14T05:46:41.0697521Z   adding: test/test-reports/distributed.checkpoint.test_fsdp_optim_state_1.1_cd9da7094ebca3fe_.log (deflated 62%)
2025-03-14T05:46:41.0702497Z   adding: test/test-reports/distributed.checkpoint.test_save_load_api_1.1_200aaa8ef57841fd_.log (deflated 58%)
2025-03-14T05:46:41.0707429Z   adding: test/test-reports/distributed._tools.test_memory_tracker_1.1_f12fbce4b0259b9a_.log (deflated 54%)
2025-03-14T05:46:41.0708291Z   adding: test/test-reports/distributed.test_collective_utils_1.1_7f0d9f0304992aec_.log (stored 0%)
2025-03-14T05:46:41.0713504Z   adding: test/test-reports/distributed.tensor.test_random_ops_1.1_64768b4cd84ae001_.log (deflated 79%)
2025-03-14T05:46:41.0714388Z   adding: test/test-reports/distributed.pipelining.test_backward_1.1_b7d4963b4ce280b9_.log (deflated 67%)
2025-03-14T05:46:41.1910821Z   adding: test/test-reports/distributed.tensor.test_dtensor_ops_1.1_61b123688546946d_.log (deflated 96%)
2025-03-14T05:46:41.1911687Z   adding: test/test-reports/distributed.tensor.test_api_1.1_eba86e128ad99438_.log (deflated 72%)
2025-03-14T05:46:41.1912994Z   adding: test/test-reports/distributed.checkpoint.test_format_utils_1.1_651c37566ec63c02_.log (deflated 85%)
2025-03-14T05:46:41.1926668Z   adding: test/test-reports/distributed.tensor.test_redistribute_1.1_aa709b9c5ee4585d_.log (deflated 96%)
2025-03-14T05:46:41.1927698Z   adding: test/test-reports/distributed.elastic.timer.local_timer_test_1.1_97889ed767288475_.log (deflated 77%)
2025-03-14T05:46:41.1929407Z   adding: test/test-reports/distributed.tensor.parallel.test_tp_random_state_1.1_6c2442b76039efb4_.log (deflated 55%)
2025-03-14T05:46:41.1932520Z   adding: test/test-reports/distributed.elastic.utils.distributed_test_1.1_02e442c9d1a6500b_.log (deflated 92%)
2025-03-14T05:46:41.1934442Z   adding: test/test-reports/distributed.tensor.debug.test_comm_mode_features_1.1_b8811ab911863edb_.log (deflated 76%)
2025-03-14T05:46:41.1936226Z   adding: test/test-reports/distributed._tools.test_mem_tracker_1.1_c6d105d6eefdc61e_.log (deflated 61%)
2025-03-14T05:46:41.1937936Z   adding: test/test-reports/distributed.pipelining.test_microbatch_1.1_7aac26247d4f7fc3_.log (deflated 58%)
2025-03-14T05:46:41.1939726Z   adding: test/test-reports/distributed.elastic.utils.logging_test_1.1_edffe3af707aa02f_.log (deflated 58%)
2025-03-14T05:46:41.1941581Z   adding: test/test-reports/distributed.tensor.experimental.test_local_map_1.1_bdda98c24cbb0e5d_.log (deflated 70%)
2025-03-14T05:46:41.1943461Z   adding: test/test-reports/distributed.tensor.parallel.test_tp_style_1.1_5575f44de7731ebd_.log (deflated 83%)
2025-03-14T05:46:41.1945148Z   adding: test/test-reports/distributed._shard.test_sharder_1.1_2f83c0d84c5588c9_.log (deflated 59%)
2025-03-14T05:46:41.1947080Z   adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_ignore_params_1.1_75ba17fd6329681a_.log (deflated 56%)
2025-03-14T05:46:41.1948985Z   adding: test/test-reports/distributed.checkpoint.e2e.test_pipeline_1.1_c1219c66ec0d3c75_.log (deflated 52%)
2025-03-14T05:46:41.1950757Z   adding: test/test-reports/distributed.pipelining.test_unflatten_1.1_d76f02edf1c24a5e_.log (deflated 54%)
2025-03-14T05:46:41.1952775Z   adding: test/test-reports/distributed.checkpoint.test_tp_checkpoint_1.1_eb09c03af9bc6620_.log (deflated 60%)
2025-03-14T05:46:41.1954587Z   adding: test/test-reports/distributed._tools.test_runtime_estimator_1.1_cd1645da6fbd8d4c_.log (deflated 60%)
2025-03-14T05:46:41.1956342Z   adding: test/test-reports/distributed.checkpoint.test_fsspec_1.1_a32523780ff57360_.log (deflated 59%)
2025-03-14T05:46:41.1958032Z   adding: test/test-reports/distributed.tensor.test_xla_integration_1.1_8d26803beefcea73_.log (deflated 66%)
2025-03-14T05:46:41.1959782Z   adding: test/test-reports/distributed.tensor.test_convolution_ops_1.1_a68aa596e20eec3b_.log (deflated 62%)
2025-03-14T05:46:41.1961659Z   adding: test/test-reports/distributed._tools.test_fsdp2_mem_tracker_1.1_754fd7513841c48a_.log (deflated 66%)
2025-03-14T05:46:41.1963394Z   adding: test/test-reports/distributed.tensor.test_embedding_ops_1.1_8eae67bed32f5213_.log (deflated 66%)
2025-03-14T05:46:41.1964932Z   adding: test/test-reports/distributed.test_functional_api_1.1_36d899f2a174fc29_.log (deflated 57%)
2025-03-14T05:46:41.1966454Z   adding: test/test-reports/distributed.tensor.test_utils_1.1_81b5fff5086a8b1f_.log (deflated 74%)
2025-03-14T05:46:41.1967991Z   adding: test/test-reports/distributed.fsdp.test_wrap_1.1_f324c2185c03c5df_.log (deflated 89%)
2025-03-14T05:46:41.1969674Z   adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_overlap_1.1_f68fcf7b19f054c4_.log (deflated 58%)
2025-03-14T05:46:41.1971414Z   adding: test/test-reports/distributed.pipelining.test_stage_1.1_b2313c4d97ecbca0_.log (deflated 2%)
2025-03-14T05:46:41.1973068Z   adding: test/test-reports/distributed.checkpoint.test_hsdp_checkpoint_1.1_ded21cf0541b4c0e_.log (deflated 70%)
2025-03-14T05:46:41.1974836Z   adding: test/test-reports/distributed.elastic.utils.util_test_1.1_a7a753c87e98f723_.log (deflated 75%)
2025-03-14T05:46:41.1976590Z   adding: test/test-reports/distributed.checkpoint.test_fsdp_model_state_1.1_067197f602c0452e_.log (deflated 61%)
2025-03-14T05:46:41.1978450Z   adding: test/test-reports/distributed.checkpoint.test_dtensor_checkpoint_1.1_fe717f8e330b9cd9_.log (deflated 55%)
2025-03-14T05:46:41.1980277Z   adding: test/test-reports/distributed.checkpoint.test_planner_1.1_ff298acb17ba064d_.log (deflated 74%)
2025-03-14T05:46:41.1982020Z   adding: test/test-reports/distributed.checkpoint.test_dtensor_resharding_1.1_eadd69445acf162a_.log (deflated 76%)
2025-03-14T05:46:41.1983888Z   adding: test/test-reports/distributed.checkpoint.test_compatibility_1.1_48cdfd6b68b80f38_.log (deflated 65%)
2025-03-14T05:46:41.1985700Z   adding: test/test-reports/distributed.fsdp.test_fsdp_memory_1.1_4ca8f5593c0e52d8_.log (deflated 60%)
2025-03-14T05:46:41.1987378Z   adding: test/test-reports/distributed.tensor.test_op_strategy_1.1_55f516f9033d309d_.log (deflated 78%)
2025-03-14T05:46:41.1989074Z   adding: test/test-reports/distributed.optim.test_apply_optimizer_in_backward_1.1_f2c2805c0cee032f_.log (stored 0%)
2025-03-14T05:46:41.1991033Z   adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_grad_scaler_1.1_2aa7ecebf5c5e5fe_.log (deflated 56%)
2025-03-14T05:46:41.1992757Z   adding: test/test-reports/distributed.test_fake_pg_1.1_f7f48e8b565959a8_.log (deflated 73%)
2025-03-14T05:46:41.1994288Z   adding: test/test-reports/distributed._composable.test_replicate_1.1_9c1adc9c8605f1e0_.log (deflated 81%)
2025-03-14T05:46:41.1995981Z   adding: test/test-reports/distributed.optim.test_named_optimizer_1.1_f28991977885b79d_.log (stored 0%)
2025-03-14T05:46:41.1997622Z   adding: test/test-reports/distributed.tensor.test_optimizers_1.1_be26ac985f0a17e9_.log (deflated 95%)
2025-03-14T05:46:41.1999533Z   adding: test/test-reports/distributed.fsdp.test_fsdp_backward_prefetch_1.1_ea449d38780daa41_.log (deflated 55%)
2025-03-14T05:46:41.2001302Z   adding: test/test-reports/distributed.checkpoint.test_utils_1.1_b6eac901d82948e7_.log (deflated 72%)
2025-03-14T05:46:41.2003058Z   adding: test/test-reports/distributed.tensor.parallel.test_tp_examples_1.1_1ac91a3db32fd90e_.log (deflated 84%)
2025-03-14T05:46:41.2005004Z   adding: test/test-reports/distributed.test_serialization_1.1_bc7729ab92ac867a_.log (deflated 72%)
2025-03-14T05:46:41.2006520Z   adding: test/test-reports/distributed.test_pg_wrapper_1.1_a25960a7545250fd_.log (deflated 93%)
2025-03-14T05:46:41.2008022Z   adding: test/test-reports/distributed.fsdp.test_fsdp_unshard_params_1.1_cf1e67d285a7d65c_.log (deflated 82%)
2025-03-14T05:46:41.2009888Z   adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_frozen_1.1_98db8a7e7c1d584e_.log (deflated 67%)
2025-03-14T05:46:41.2011827Z   adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_init_1.1_be095c2f5731700f_.log (deflated 88%)
2025-03-14T05:46:41.2013600Z   adding: test/test-reports/distributed.fsdp.test_fsdp_optim_state_1.1_ebe950b107b2e744_.log (deflated 93%)
2025-03-14T05:46:41.2015220Z   adding: test/test-reports/distributed.fsdp.test_fsdp_grad_acc_1.1_74c46386e446db13_.log (deflated 74%)
2025-03-14T05:46:41.2016833Z   adding: test/test-reports/distributed.test_multi_threaded_pg_1.1_6a612da2f4e22401_.log (deflated 86%)
2025-03-14T05:46:41.2018590Z   adding: test/test-reports/distributed._shard.sharded_tensor.test_sharded_tensor_1.1_aacf32e09be9654a_.log (deflated 91%)
2025-03-14T05:46:41.2020504Z   adding: test/test-reports/distributed._composable.test_replicate_with_compiler_1.1_7dfe42292ed47b44_.log (deflated 80%)
2025-03-14T05:46:41.2022171Z   adding: test/test-reports/distributed.fsdp.test_fsdp_fine_tune_1.1_2c669406e1c2eebd_.log (deflated 51%)
2025-03-14T05:46:41.2023758Z   adding: test/test-reports/distributed.nn.jit.test_instantiator_1.1_9660011738600130_.log (deflated 63%)
2025-03-14T05:46:41.2025490Z   adding: test/test-reports/distributed.fsdp.test_fsdp_ignored_modules_1.1_2e973940520a287a_.log (deflated 79%)
2025-03-14T05:46:41.2027172Z   adding: test/test-reports/distributed.fsdp.test_fsdp_traversal_1.1_dd6d83364709a20b_.log (deflated 52%)
2025-03-14T05:46:41.2028824Z   adding: test/test-reports/distributed.fsdp.test_fsdp_use_orig_params_1.1_06f7e427f0cce0fd_.log (deflated 86%)
2025-03-14T05:46:41.2030558Z   adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_state_dict_1.1_10c47f23318b96f0_.log (deflated 76%)
2025-03-14T05:46:41.2032357Z   adding: test/test-reports/distributed.algorithms.test_join_1.1_15b2adbba069b553_.log (deflated 72%)
2025-03-14T05:46:41.2033945Z   adding: test/test-reports/distributed.fsdp.test_fsdp_tp_integration_1.1_22b1e11151dfe328_.log (deflated 61%)
2025-03-14T05:46:41.2035445Z   adding: test/test-reports/distributed.rpc.test_faulty_agent_1.1_9c6b146c252d55da_.log (deflated 94%)
2025-03-14T05:46:41.2037108Z   adding: test/test-reports/distributed.rpc.test_tensorpipe_agent_2.5_39cbedd9249bb07c_.log (deflated 96%)
2025-03-14T05:46:41.2054671Z   adding: test/test-reports/distributed.rpc.test_tensorpipe_agent_4.5_60e3af3ab37756ec_.log (deflated 96%)
2025-03-14T05:46:41.2056298Z   adding: test/test-reports/distributed.fsdp.test_fsdp_checkpoint_1.1_553be86bbec018f1_.log (deflated 87%)
2025-03-14T05:46:41.2059363Z   adding: test/test-reports/distributed.launcher.test_run_1.1_6be2e3093d5e099e_.log (deflated 89%)
2025-03-14T05:46:41.2061037Z   adding: test/test-reports/distributed.fsdp.test_fsdp_hybrid_shard_1.1_7b5e95e63c1af4be_.log (deflated 72%)
2025-03-14T05:46:41.2062801Z   adding: test/test-reports/distributed.fsdp.test_fsdp_multiple_forward_1.1_2559ca3246f5160b_.log (deflated 54%)
2025-03-14T05:46:41.2064573Z   adding: test/test-reports/distributed._tools.test_fake_collectives_1.1_bfa57d8268792dcb_.log (deflated 54%)
2025-03-14T05:46:41.2066520Z   adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_tensor_ops_1.1_eaa79eba291b1750_.log (deflated 71%)
2025-03-14T05:46:41.2068484Z   adding: test/test-reports/distributed._shard.sharded_optim.test_sharded_optim_1.1_b9e4292b7cc7982f_.log (deflated 62%)
2025-03-14T05:46:41.2070408Z   adding: test/test-reports/distributed._shard.sharding_plan.test_sharding_plan_1.1_b9b36863c9c93618_.log (deflated 66%)
2025-03-14T05:46:41.2071966Z   adding: test/test-reports/distributed.test_launcher_1.1_c9641116225c6cfb_.log (deflated 58%)
2025-03-14T05:46:41.2073422Z   adding: test/test-reports/distributed.fsdp.test_fsdp_clip_grad_norm_1.1_e6ef88816ec3101c_.log (deflated 52%)
2025-03-14T05:46:41.2074785Z   adding: test/test-reports/distributed.fsdp.test_fsdp_comm_1.1_bc351c302f393730_.log (deflated 51%)
2025-03-14T05:46:41.2075710Z   adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_1.1_533bec517705edb2_.log (deflated 62%)
2025-03-14T05:46:41.2076730Z   adding: test/test-reports/distributed.elastic.events.lib_test_1.1_ab1a9d1fe2eb6f72_.log (deflated 72%)
2025-03-14T05:46:41.2078526Z   adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_embedding_bag_1.1_a31e13cb7dcda492_.log (deflated 63%)
2025-03-14T05:46:41.2079502Z   adding: test/test-reports/distributed.elastic.metrics.api_test_1.1_749f776b0e0ddac8_.log (deflated 62%)
2025-03-14T05:46:41.2080472Z   adding: test/test-reports/distributed._composable.fsdp.test_fully_shard_extensions_1.1_a001279f475464b2_.log (deflated 73%)
2025-03-14T05:46:41.2081484Z   adding: test/test-reports/distributed._shard.sharded_tensor.ops.test_init_1.1_bcc3673b5317c186_.log (deflated 67%)
2025-03-14T05:46:41.2106854Z ##[group]Run # Remove any previous debugging artifacts if they exist
2025-03-14T05:46:41.2107422Z # Remove any previous debugging artifacts if they exist
2025-03-14T05:46:41.2107863Z rm -f debug-*.zip
2025-03-14T05:46:41.2108173Z if [ -d 'test/debug' ]; then
2025-03-14T05:46:41.2108559Z   zip -r "debug-${FILE_SUFFIX}.zip" test/debug
2025-03-14T05:46:41.2108948Z fi
2025-03-14T05:46:41.2114799Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:41.2115214Z env:
2025-03-14T05:46:41.2115464Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:41.2115970Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:41.2116641Z   FILE_SUFFIX: test-distributed-2-2-lf.linux.2xlarge_38754724585
2025-03-14T05:46:41.2117056Z ##[endgroup]
2025-03-14T05:46:41.2194746Z ##[group]Run seemethere/upload-artifact-s3@v5
2025-03-14T05:46:41.2195108Z with:
2025-03-14T05:46:41.2195357Z   s3-bucket: gha-artifacts
2025-03-14T05:46:41.2195704Z   s3-prefix: pytorch/pytorch/13849515290/1/artifact

2025-03-14T05:46:41.2196079Z   retention-days: 14
2025-03-14T05:46:41.2196356Z   if-no-files-found: warn
2025-03-14T05:46:41.2196651Z   path: test-jsons-*.zip
2025-03-14T05:46:41.2196931Z   name: artifact
2025-03-14T05:46:41.2197181Z   region: us-east-1
2025-03-14T05:46:41.2197418Z env:
2025-03-14T05:46:41.2197651Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:41.2198159Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:41.2198929Z ##[endgroup]
2025-03-14T05:46:41.5832036Z NOTE: s3-prefix specified, ignoring name parameter
2025-03-14T05:46:41.5832539Z With the provided path, there will be 1 file uploaded
2025-03-14T05:46:41.5833019Z Uploading to s3 prefix: pytorch/pytorch/13849515290/1/artifact
2025-03-14T05:46:41.5914541Z Starting upload of test-jsons-test-distributed-2-2-lf.linux.2xlarge_38754724585.zip
2025-03-14T05:46:41.7417302Z Finished upload of test-jsons-test-distributed-2-2-lf.linux.2xlarge_38754724585.zip
2025-03-14T05:46:41.7588610Z ##[group]Run seemethere/upload-artifact-s3@v5
2025-03-14T05:46:41.7588979Z with:
2025-03-14T05:46:41.7589224Z   s3-bucket: gha-artifacts
2025-03-14T05:46:41.7589574Z   s3-prefix: pytorch/pytorch/13849515290/1/artifact

2025-03-14T05:46:41.7589950Z   retention-days: 14
2025-03-14T05:46:41.7590228Z   if-no-files-found: error
2025-03-14T05:46:41.7590543Z   path: test-reports-*.zip
2025-03-14T05:46:41.7590834Z   name: artifact
2025-03-14T05:46:41.7591075Z   region: us-east-1
2025-03-14T05:46:41.7591326Z env:
2025-03-14T05:46:41.7591566Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:41.7592065Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:41.7592600Z ##[endgroup]
2025-03-14T05:46:42.0812368Z NOTE: s3-prefix specified, ignoring name parameter
2025-03-14T05:46:42.0813524Z With the provided path, there will be 1 file uploaded
2025-03-14T05:46:42.0814453Z Uploading to s3 prefix: pytorch/pytorch/13849515290/1/artifact
2025-03-14T05:46:42.0854782Z Starting upload of test-reports-test-distributed-2-2-lf.linux.2xlarge_38754724585.zip
2025-03-14T05:46:42.2623734Z Finished upload of test-reports-test-distributed-2-2-lf.linux.2xlarge_38754724585.zip
2025-03-14T05:46:42.2799938Z ##[group]Run seemethere/upload-artifact-s3@v5
2025-03-14T05:46:42.2800307Z with:
2025-03-14T05:46:42.2800559Z   s3-bucket: gha-artifacts
2025-03-14T05:46:42.2801058Z   s3-prefix: pytorch/pytorch/13849515290/1/artifact

2025-03-14T05:46:42.2801441Z   retention-days: 14
2025-03-14T05:46:42.2801728Z   if-no-files-found: ignore
2025-03-14T05:46:42.2802027Z   path: logs-*.zip
2025-03-14T05:46:42.2802282Z   name: artifact
2025-03-14T05:46:42.2802525Z   region: us-east-1
2025-03-14T05:46:42.2802771Z env:
2025-03-14T05:46:42.2803007Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:42.2803515Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:42.2804049Z ##[endgroup]
2025-03-14T05:46:42.6000370Z NOTE: s3-prefix specified, ignoring name parameter
2025-03-14T05:46:42.6001234Z With the provided path, there will be 1 file uploaded
2025-03-14T05:46:42.6002089Z Uploading to s3 prefix: pytorch/pytorch/13849515290/1/artifact
2025-03-14T05:46:42.6042394Z Starting upload of logs-test-distributed-2-2-lf.linux.2xlarge_38754724585.zip
2025-03-14T05:46:42.8293918Z Finished upload of logs-test-distributed-2-2-lf.linux.2xlarge_38754724585.zip
2025-03-14T05:46:42.8465966Z ##[group]Run seemethere/upload-artifact-s3@v5
2025-03-14T05:46:42.8466345Z with:
2025-03-14T05:46:42.8466597Z   s3-bucket: gha-artifacts
2025-03-14T05:46:42.8466947Z   s3-prefix: pytorch/pytorch/13849515290/1/artifact

2025-03-14T05:46:42.8467325Z   retention-days: 14
2025-03-14T05:46:42.8467608Z   if-no-files-found: ignore
2025-03-14T05:46:42.8467907Z   path: debug-*.zip
2025-03-14T05:46:42.8468178Z   name: artifact
2025-03-14T05:46:42.8468419Z   region: us-east-1
2025-03-14T05:46:42.8468668Z env:
2025-03-14T05:46:42.8468903Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:42.8469403Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:42.8469939Z ##[endgroup]
2025-03-14T05:46:43.1617479Z No files were found with the provided path: debug-*.zip. No artifacts will be uploaded.
2025-03-14T05:46:43.1793898Z ##[group]Run # shellcheck disable=SC2156
2025-03-14T05:46:43.1794314Z # shellcheck disable=SC2156
2025-03-14T05:46:43.1794951Z find . -iname "core.[1-9]*" -exec docker exec "${DOCKER_CONTAINER_ID}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \;
2025-03-14T05:46:43.1801142Z shell: /usr/bin/bash -e {0}
2025-03-14T05:46:43.1801446Z env:
2025-03-14T05:46:43.1801682Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:43.1802193Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:43.1802771Z ##[endgroup]
2025-03-14T05:46:43.4148240Z Prepare all required actions
2025-03-14T05:46:43.4148668Z Getting action download info
2025-03-14T05:46:43.5345393Z ##[group]Run ./.github/actions/upload-utilization-stats
2025-03-14T05:46:43.5345797Z with:
2025-03-14T05:46:43.5346029Z   job_id: 38754724585
2025-03-14T05:46:43.5346459Z   job_name: linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)
2025-03-14T05:46:43.5346951Z   workflow_name: pull
2025-03-14T05:46:43.5347237Z   workflow_run_id: 13849515290
2025-03-14T05:46:43.5347539Z   workflow_attempt: 1
2025-03-14T05:46:43.5347822Z env:
2025-03-14T05:46:43.5348059Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:43.5348558Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:43.5349095Z ##[endgroup]
2025-03-14T05:46:43.5369226Z ##[group]Run echo "workflow_id: 13849515290"
2025-03-14T05:46:43.5369618Z echo "workflow_id: 13849515290"
2025-03-14T05:46:43.5369973Z echo "workflow_attempt: 1"
2025-03-14T05:46:43.5370380Z echo "workflow_Name: pull"
2025-03-14T05:46:43.5370714Z echo "job_id: 38754724585"
2025-03-14T05:46:43.5371239Z echo "job_name:  linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)"
2025-03-14T05:46:43.5376939Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:43.5377346Z env:
2025-03-14T05:46:43.5377588Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:43.5378091Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:43.5378778Z ##[endgroup]
2025-03-14T05:46:43.5401002Z workflow_id: 13849515290
2025-03-14T05:46:43.5401340Z workflow_attempt: 1
2025-03-14T05:46:43.5401618Z workflow_Name: pull
2025-03-14T05:46:43.5401879Z job_id: 38754724585
2025-03-14T05:46:43.5402451Z job_name:  linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)
2025-03-14T05:46:43.5440241Z ##[group]Run nick-fields/retry@v3.0.0
2025-03-14T05:46:43.5440571Z with:
2025-03-14T05:46:43.5440816Z   shell: bash
2025-03-14T05:46:43.5441059Z   timeout_minutes: 5
2025-03-14T05:46:43.5441316Z   max_attempts: 5
2025-03-14T05:46:43.5441576Z   retry_wait_seconds: 30
2025-03-14T05:46:43.5442064Z   command: set -eu
python3 -m pip install python-dateutil==2.8.2 boto3==1.35.42 pandas==2.1.3

2025-03-14T05:46:43.5442610Z   polling_interval_seconds: 1
2025-03-14T05:46:43.5442914Z   warning_on_retry: true
2025-03-14T05:46:43.5443201Z   continue_on_error: false
2025-03-14T05:46:43.5443481Z env:
2025-03-14T05:46:43.5443725Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:43.5444238Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:43.5444775Z ##[endgroup]
2025-03-14T05:46:43.8700564Z Defaulting to user installation because normal site-packages is not writeable
2025-03-14T05:46:43.8860969Z Requirement already satisfied: python-dateutil==2.8.2 in /home/ec2-user/.local/lib/python3.9/site-packages (2.8.2)
2025-03-14T05:46:44.8723359Z Collecting boto3==1.35.42
2025-03-14T05:46:44.8789491Z   Using cached boto3-1.35.42-py3-none-any.whl (139 kB)
2025-03-14T05:46:44.8815420Z Requirement already satisfied: pandas==2.1.3 in /home/ec2-user/.local/lib/python3.9/site-packages (2.1.3)
2025-03-14T05:46:44.8830360Z Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil==2.8.2) (1.15.0)
2025-03-14T05:46:44.8876074Z Requirement already satisfied: botocore<1.36.0,>=1.35.42 in /home/ec2-user/.local/lib/python3.9/site-packages (from boto3==1.35.42) (1.35.99)
2025-03-14T05:46:44.8879979Z Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/lib/python3.9/site-packages (from boto3==1.35.42) (0.10.0)
2025-03-14T05:46:44.8884146Z Requirement already satisfied: s3transfer<0.11.0,>=0.10.0 in /home/ec2-user/.local/lib/python3.9/site-packages (from boto3==1.35.42) (0.10.4)
2025-03-14T05:46:44.9742502Z Requirement already satisfied: pytz>=2020.1 in /usr/lib/python3.9/site-packages (from pandas==2.1.3) (2022.7.1)
2025-03-14T05:46:44.9747649Z Requirement already satisfied: tzdata>=2022.1 in /home/ec2-user/.local/lib/python3.9/site-packages (from pandas==2.1.3) (2025.1)
2025-03-14T05:46:44.9757532Z Requirement already satisfied: numpy<2,>=1.22.4 in /home/ec2-user/.local/lib/python3.9/site-packages (from pandas==2.1.3) (1.26.4)
2025-03-14T05:46:44.9813343Z Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python3.9/site-packages (from botocore<1.36.0,>=1.35.42->boto3==1.35.42) (1.25.10)
2025-03-14T05:46:45.0635807Z Installing collected packages: boto3
2025-03-14T05:46:45.0636233Z   Attempting uninstall: boto3
2025-03-14T05:46:45.0637734Z     Found existing installation: boto3 1.35.33
2025-03-14T05:46:45.0740635Z     Uninstalling boto3-1.35.33:
2025-03-14T05:46:45.0771666Z       Successfully uninstalled boto3-1.35.33
2025-03-14T05:46:45.1578875Z Successfully installed boto3-1.35.42
2025-03-14T05:46:45.6201287Z Command completed after 1 attempt(s).
2025-03-14T05:46:45.6254467Z ##[group]Run python3 -m tools.stats.upload_utilization_stats.upload_utilization_stats \
2025-03-14T05:46:45.6255206Z python3 -m tools.stats.upload_utilization_stats.upload_utilization_stats \
2025-03-14T05:46:45.6255748Z   --workflow-run-id "13849515290" \
2025-03-14T05:46:45.6256117Z   --workflow-name "pull" \
2025-03-14T05:46:45.6256472Z   --workflow-run-attempt "1" \
2025-03-14T05:46:45.6256825Z   --job-id "38754724585" \
2025-03-14T05:46:45.6257338Z   --job-name "linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)"
2025-03-14T05:46:45.6263260Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:45.6263682Z env:
2025-03-14T05:46:45.6263933Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:45.6264448Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:45.6264987Z ##[endgroup]
2025-03-14T05:46:47.7281769Z repo: pytorch/pytorch
2025-03-14T05:46:47.7282759Z Downloading logs-test-distributed-2-2-lf.linux.2xlarge_38754724585.zip
2025-03-14T05:46:47.7283769Z Converted Log Model: UtilizationMetadata:
2025-03-14T05:46:47.7286497Z  UtilizationMetadata(level='metadata', workflow_id='13849515290', job_id='38754724585', workflow_name='pull', job_name='linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)', usage_collect_interval=1.0, data_model_version=1.0, start_at=1741925644, gpu_count=0, cpu_count=8, gpu_type='', error=None)
2025-03-14T05:46:47.7289377Z [Db Segments] detected pytest cmd: 10, generated segments: 10
2025-03-14T05:46:47.7290232Z [db model] Peek db timeseries 
2025-03-14T05:46:47.7290796Z :{
2025-03-14T05:46:47.7291236Z     "created_at": 1741931207,
2025-03-14T05:46:47.7291812Z     "type": "utilization",
2025-03-14T05:46:47.7292347Z     "tags": [
2025-03-14T05:46:47.7292794Z         "record"
2025-03-14T05:46:47.7293258Z     ],
2025-03-14T05:46:47.7293705Z     "time_stamp": 1741925644,
2025-03-14T05:46:47.7294293Z     "repo": "pytorch/pytorch",
2025-03-14T05:46:47.7294880Z     "workflow_id": 13849515290,
2025-03-14T05:46:47.7295454Z     "run_attempt": 1,
2025-03-14T05:46:47.7295967Z     "job_id": 38754724585,
2025-03-14T05:46:47.7296525Z     "workflow_name": "pull",
2025-03-14T05:46:47.7297404Z     "job_name": "linux-jammy-py3.9-gcc11 / test (distributed, 2, 2, lf.linux.2xlarge)",
2025-03-14T05:46:47.7298581Z     "json_data": "{}"
2025-03-14T05:46:47.7299091Z }
2025-03-14T05:46:47.7300121Z Writing 1 documents to S3 ossci-utilization/util_metadata/v_1.0/pytorch/pytorch/13849515290/1/38754724585/metadata
2025-03-14T05:46:47.7302026Z Done! Finish writing document to S3 ossci-utilization/util_metadata/v_1.0/pytorch/pytorch/13849515290/1/38754724585/metadata 
2025-03-14T05:46:47.7304035Z Writing 1087 documents to S3 ossci-utilization/util_timeseries/v_1.0/pytorch/pytorch/13849515290/1/38754724585/time_series
2025-03-14T05:46:47.7306139Z Done! Finish writing document to S3 ossci-utilization/util_timeseries/v_1.0/pytorch/pytorch/13849515290/1/38754724585/time_series 
2025-03-14T05:46:47.8332925Z ##[group]Run pytorch/test-infra/.github/actions/teardown-linux@main
2025-03-14T05:46:47.8333399Z with:
2025-03-14T05:46:47.8333630Z env:
2025-03-14T05:46:47.8333879Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:47.8334383Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:47.8334920Z ##[endgroup]
2025-03-14T05:46:47.8355519Z ##[group]Run set -eou pipefail
2025-03-14T05:46:47.8355867Z set -eou pipefail
2025-03-14T05:46:47.8356162Z 
2025-03-14T05:46:47.8356570Z echo "Holding runner for 2 hours until all ssh sessions have logged out"
2025-03-14T05:46:47.8357080Z for _ in $(seq 1440); do
2025-03-14T05:46:47.8357437Z     # Break if no ssh session exists anymore
2025-03-14T05:46:47.8357812Z     if [ "$(who)" = "" ]; then
2025-03-14T05:46:47.8358133Z       break
2025-03-14T05:46:47.8358392Z     fi
2025-03-14T05:46:47.8358683Z     echo "."
2025-03-14T05:46:47.8358952Z     sleep 5
2025-03-14T05:46:47.8359215Z done
2025-03-14T05:46:47.8364804Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:47.8365218Z env:
2025-03-14T05:46:47.8365458Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:47.8365960Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:47.8366505Z ##[endgroup]
2025-03-14T05:46:47.8388568Z Holding runner for 2 hours until all ssh sessions have logged out
2025-03-14T05:46:47.8451149Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty
2025-03-14T05:46:47.8451885Z # ignore expansion of "docker ps -q" since it could be empty
2025-03-14T05:46:47.8452351Z # shellcheck disable=SC2046
2025-03-14T05:46:47.8452714Z docker stop $(docker ps -q) || true
2025-03-14T05:46:47.8453090Z # Prune all of the docker images
2025-03-14T05:46:47.8453448Z docker system prune -af
2025-03-14T05:46:47.8458607Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:46:47.8459016Z env:
2025-03-14T05:46:47.8459258Z   GIT_DEFAULT_BRANCH: main
2025-03-14T05:46:47.8459764Z   DOCKER_CONTAINER_ID: fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:47.8460303Z ##[endgroup]
2025-03-14T05:46:58.3982403Z fc1c1f3bb76f
2025-03-14T05:46:58.9177741Z Deleted Containers:
2025-03-14T05:46:58.9178231Z fc1c1f3bb76fbb0b0e51d88aff67f4d29585833bc7e4c2bbdc430f995353a54c
2025-03-14T05:46:58.9178591Z 
2025-03-14T05:47:06.2543255Z Deleted Images:
2025-03-14T05:47:06.2544206Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11:aa89d6e739080d90fa18625d57297c6734465849
2025-03-14T05:47:06.2545719Z untagged: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/pytorch-linux-jammy-py3.9-gcc11@sha256:af98f5b94e2a6aa569662f00ae8e7d849202075bbcf06a2f7dd51e480c3adbef
2025-03-14T05:47:06.2546883Z deleted: sha256:106fb2ab21a44c9d34c6968e6196c73d5cf7eaa50499fe5f98d7dd73b69b1298
2025-03-14T05:47:06.2547626Z deleted: sha256:be88725bf63c0f7732108470833b2115ec462de48b07da2e91e744078b48b880
2025-03-14T05:47:06.2548357Z deleted: sha256:6c78e4cacf7c502c8f9d3b836413a412342b876b9e5f4a14504cd630bd0f3848
2025-03-14T05:47:06.2549082Z deleted: sha256:5107bbbc77743b0b6e9d8383ae5dd8ffac0973f8e04d745b79f9a13c06b05ef3
2025-03-14T05:47:06.2549762Z deleted: sha256:608f058a4e328b0c74aaee18c4b0c37a8ed5c454ce6a73a2129925481d8e61ed
2025-03-14T05:47:06.2550445Z deleted: sha256:9edea689489aa6547c7025d394b4e51aaca3f26bcaccf86dbcbfcc40fefb2a99
2025-03-14T05:47:06.2551138Z deleted: sha256:7fb60e18111feaee573b5d066a216e82ce351a8d239b06b07fd81dc805df1c64
2025-03-14T05:47:06.2551815Z deleted: sha256:6e5d0ef27a35e0d96b9873e6dce09866db6e037307bc7efe147f19666739b5a2
2025-03-14T05:47:06.2552482Z deleted: sha256:a5476900314ffa5fb72930c4ca39d26400327f771b2518a31086a4dcf04f85d5
2025-03-14T05:47:06.2553405Z deleted: sha256:25ca006e55fe1ceacd87060da73e3e7da9f08338c5d195c4d9e8a98088aa78e7
2025-03-14T05:47:06.2554104Z deleted: sha256:fdf943d91d10654ac7d8c5ce43b0ce43d6e835e1cb84d22235d18c8558b05502
2025-03-14T05:47:06.2554788Z deleted: sha256:1f96322d2f1406408a177c75b1870bf664b2dee11046d30d636e4307c157dd5f
2025-03-14T05:47:06.2555477Z deleted: sha256:561c760199ca6a83eb54c0541243e1045fdb1a8e16788e4619de065cc816d540
2025-03-14T05:47:06.2556146Z deleted: sha256:aa1d4490ae2961719200b6b2f6858298681a5cfae28e7a78801f64aa75caafa0
2025-03-14T05:47:06.2556822Z deleted: sha256:dabcbb6854d27689bcd2354e57456cc4914b253fbf8716f2f384c9c52f9ab24d
2025-03-14T05:47:06.2557497Z deleted: sha256:2cbb3c75d283221fe61c6d8f41b024155fb52319d11955e648f7732b6ea20f92
2025-03-14T05:47:06.2558185Z deleted: sha256:d378f8f4c77e5bef745dc123c847f5ac1e1acc8b02e5b8f8d0ee0b5184e0fe3a
2025-03-14T05:47:06.2558863Z deleted: sha256:5746b0b6d890732f5741449eb70a216d4998dd9e9203be5f616452b5e298ed8e
2025-03-14T05:47:06.2559547Z deleted: sha256:8c0dbd3f6e25c1834bf2c656ac1f051ecfb55deae8634dbb63efb02789fc71d7
2025-03-14T05:47:06.2560245Z deleted: sha256:7bb0b71e17f76ef8e56fa0f4565aef30d6e717dc8b3efa55b3181d02346401cb
2025-03-14T05:47:06.2560927Z deleted: sha256:2e69ff6b7c6f957af967a7127c3896e8a32694ce82ca750b722781e110a43767
2025-03-14T05:47:06.2561608Z deleted: sha256:33b38ce5873910da0de3ccdb6c96d22cad2641cdabe34a2f08b8031c902545cb
2025-03-14T05:47:06.2562280Z deleted: sha256:9c86e146483daeb34ac7d351140b9ea9405d1d599431843fcc9433af49a9c8bd
2025-03-14T05:47:06.2562953Z deleted: sha256:b88d9a9b28be82b61f075f1ab20a16091da806bd1d118ee4e6bd4472b952ea9e
2025-03-14T05:47:06.2563635Z deleted: sha256:fa2e822725ef51b11c54ec7b905e9c6c0c58b53bca8de47443600d568ece0fb8
2025-03-14T05:47:06.2564432Z deleted: sha256:606215b13092f9c00918d337491e12a3f957cc900995b6f53833219432b8fb1a
2025-03-14T05:47:06.2565110Z deleted: sha256:03fd8edb49ddecfc00b50f84a8f90f1b62c620fea1ddc3d51b6a3333d13e07b9
2025-03-14T05:47:06.2565806Z deleted: sha256:2b8031cce4ff1382215e3e73c7faf55c2f12d50058bbefb29b798a0054e729cc
2025-03-14T05:47:06.2566492Z deleted: sha256:60cd9cc6191349d276babca840e8756263717b8834763d9d99cd40defd441588
2025-03-14T05:47:06.2567159Z deleted: sha256:cc627a90a7169228d8f29c5538830c3f1f262792a4a220a45596a481cef29458
2025-03-14T05:47:06.2567841Z deleted: sha256:44f9cffe0f14e6fc5eddad4d8517e9dd291e2ea0df35ea5bb25cdda5e6fc3f15
2025-03-14T05:47:06.2568540Z deleted: sha256:aa5e0fe2e5d8f42cc9e970417f3cb51f6d8a978d81b7b006f833511ca87dae61
2025-03-14T05:47:06.2569227Z deleted: sha256:b405f54ff16cefed2a7da1c11c73a0fbf790950e62381de974bf4a26f9a75c37
2025-03-14T05:47:06.2569909Z deleted: sha256:37edf893fd575596586cb4f58d1efe809bb90005bfec5985ff218af4febcf573
2025-03-14T05:47:06.2570590Z deleted: sha256:08c70b9e380c6b6208e40abb63201db3c0dea9e92b5e46eab2361045cac0d988
2025-03-14T05:47:06.2571250Z deleted: sha256:1b15953232b4a21cee76900847e82888d0650c575b0911336ef5800dc5579b87
2025-03-14T05:47:06.2571911Z deleted: sha256:bcde3fdde8851fa9764780597878be269db6d1ca615124b714f9112f5d465b8b
2025-03-14T05:47:06.2572574Z deleted: sha256:c7f62c1554e06299d862672b939073f83915c3197c430531562f02defff7a331
2025-03-14T05:47:06.2573228Z deleted: sha256:19a2fd8e5dc140ffc4b74821f242f3d9613c9931dd99618654789d0168ef2c39
2025-03-14T05:47:06.2573905Z deleted: sha256:1439acd579c86cbed2f9c9ae9a6fda445cae8f25a8c470a90813ed47f126afe0
2025-03-14T05:47:06.2574587Z deleted: sha256:f23595dc04505bbbd4d8499cda0c3d453e7654eb46bad0985fc31728551c0bf5
2025-03-14T05:47:06.2575252Z deleted: sha256:b758871f978672c57435479f3d9a52c12a2cd31465c8acea6745101ffaf64772
2025-03-14T05:47:06.2575911Z deleted: sha256:58ef958d0d3790b885bb38517fc08c4aa285447c1506f79d2ce4fa1341ff6eda
2025-03-14T05:47:06.2576590Z deleted: sha256:0c2707db6d9a56f4281419e5f868cfcc4f177eaacc4b24b89dc74d6cc182474d
2025-03-14T05:47:06.2577286Z deleted: sha256:e02e3d15ba943a6a05f8eab2205eddbb8549f468ab2bb97690da4e72d41d2fe8
2025-03-14T05:47:06.2577987Z deleted: sha256:d3d3cd9bbeddf430a8fd3bfca4eda7f601c6ef7432e1b6da65fdaa70a8e00358
2025-03-14T05:47:06.2578684Z deleted: sha256:d76761ed20f7ea8faa1203ed713605061e6477c00cbed15df7ca0108d64b71eb
2025-03-14T05:47:06.2579455Z deleted: sha256:98d345936754b8d6c44ec96fccc36bd8388dd2c09b52875c75b852d161384e92
2025-03-14T05:47:06.2580116Z deleted: sha256:379802a1107422b46188f23c3fb407694021a1a2837bef5e445545f27cf766ae
2025-03-14T05:47:06.2580768Z deleted: sha256:59e9e2f215914ba90e56345ff249212105f0b1d601bd0dd879572d1818f11a5a
2025-03-14T05:47:06.2581431Z deleted: sha256:72e91e5bc788b9b783a3593b44742b87aa5e0bfead0b02244778c383ae24404a
2025-03-14T05:47:06.2582104Z deleted: sha256:d2b8b7f91a608bfd454873909d870e8b53a5e624099c4a4cb0664bc5e68cb3dc
2025-03-14T05:47:06.2582782Z deleted: sha256:a6b2bc07fe268bbba9324489885805decde60693eb53d546bcd31ff3c303ac94
2025-03-14T05:47:06.2583481Z deleted: sha256:5d0b54c4cfd4af35a2dcd75b3fe3b99c77c97bde809f1eb1e98db71cfe95fe5c
2025-03-14T05:47:06.2584179Z deleted: sha256:db838ea7caaadda9cafbdb1f498902b964c95f0a939a1627d34edb6b9be1ab71
2025-03-14T05:47:06.2584857Z deleted: sha256:4c27d1576346c5a42dca61b191ba414225c539281465ae7cb1739bf82ca4fb74
2025-03-14T05:47:06.2585624Z deleted: sha256:dcca946e769579db828e9945c21e729372d6d124caf6d379577a1178f99a143a
2025-03-14T05:47:06.2586295Z deleted: sha256:35621af69ff9484fa8ea176ffb60debf3a882ebd7322f441e3138d4fbedb9678
2025-03-14T05:47:06.2586981Z deleted: sha256:e743507cdd87d8ad176c7cebdd4372269691f6b913f8fbcd8339ac469fee18c1
2025-03-14T05:47:06.2587669Z deleted: sha256:d8d55be8333f89e86149bfe3fb1eecf511d702da0206421e917af7cb1b3bbcbc
2025-03-14T05:47:06.2588355Z deleted: sha256:7a7f13a6d4ed9a94d6d0ca14f2cd0c2140be43e041d73e4c2759b3e7c2ca9ef9
2025-03-14T05:47:06.2589032Z deleted: sha256:f664b90c9641a5264e61f99486ec2816098bb970d38700b49becb941440545c3
2025-03-14T05:47:06.2589782Z deleted: sha256:acfc5e931ee26e2fe4faabe3812a3a0110a2269e11e4a36ed7c8e21da29f7aeb
2025-03-14T05:47:06.2590475Z deleted: sha256:eee342f6807e77b06fc1e7aceb958f650aef23a727a23cffe8663950300a0df3
2025-03-14T05:47:06.2591152Z deleted: sha256:1e174748bd1c3b30c3835613f3f130f59d2cea4dcd8a3d3a4c89a0cc5bd3a61e
2025-03-14T05:47:06.2591831Z deleted: sha256:33eb94b59a9a66959c48281978f5bdfe4f5208e4d4d09e55f6f9bf8d25f285e2
2025-03-14T05:47:06.2592521Z deleted: sha256:cf45ef618cd3c176bea6ade4b24ac87469f03d0db3ffcc3f2e01f2390a507522
2025-03-14T05:47:06.2593205Z deleted: sha256:2b5331a79d87d10f384461eecafebc7687e52a9c47f57ce46949693e336ad49a
2025-03-14T05:47:06.2593875Z deleted: sha256:4ef700b913b2c4a5c8b70e02de98683288586e3a6bd25c2f1f9092b920cfedd1
2025-03-14T05:47:06.2594540Z deleted: sha256:121990c6caa9f0d40a9f671ea78a728328a69a357920a8f1711bef986be7a2ee
2025-03-14T05:47:06.2595214Z deleted: sha256:50f7b6a1a2bc40b93a8e583feaff5277f6ebf19054d499a379aaee08f1187679
2025-03-14T05:47:06.2595895Z deleted: sha256:e009549e81f44b512a8cbc35d297a267a57ec6c795641a1593d0885c8ba6ee01
2025-03-14T05:47:06.2596568Z deleted: sha256:85bb3c8af2ac2fce3c60a9e1e59e399a21f1db4733565831e6abacf070df491e
2025-03-14T05:47:06.2597242Z deleted: sha256:62f51c268e9004618dedd852f746692894c806027c2f34c8cc4f5858f049857c
2025-03-14T05:47:06.2597901Z deleted: sha256:e9c58e53f511410f68d1239af517233ef82b03f809c6d709ead85e90253682dd
2025-03-14T05:47:06.2598804Z deleted: sha256:2a688edcc6314fac4ea4b995a562f088d5a9b17fe8a76d60d29ce6fad32398f3
2025-03-14T05:47:06.2599501Z deleted: sha256:245ecec2e6a703f8ec0fb28a14dfadf025bc9f8d41ddd1b7c9633f77cb1033eb
2025-03-14T05:47:06.2600175Z deleted: sha256:29c5f8d42c1453955708c54b28e40526918789ed17c6f57dfc2c96f72e099764
2025-03-14T05:47:06.2600846Z deleted: sha256:ceb8a1babf9556c79661f228bb6a64f7a777feaf74001cd263d7d75d8ed7d87f
2025-03-14T05:47:06.2601536Z deleted: sha256:17733f1f1ba3553efd6afb76aa7b7fe5cce54d2ed54aa83c16df07f952e32fb0
2025-03-14T05:47:06.2602220Z deleted: sha256:4b467b5423a19fba3b5937fecd51e29ac81b1e07762a83bcacd0d2683f53e782
2025-03-14T05:47:06.2602903Z deleted: sha256:f56dd9ad53ed88024031ebf5f126ddbb21877e8a6c9a9d1919c461bede45661f
2025-03-14T05:47:06.2603577Z deleted: sha256:3b7b11dd7f9192872ce43c477c3980fa29bb9d7084dc1a3938641291dc149d91
2025-03-14T05:47:06.2604237Z deleted: sha256:092550c6482cb487787a1944c600b2f7c577a535cdbf0c7544528c8b95132dda
2025-03-14T05:47:06.2605070Z deleted: sha256:030383ea992306d456e898fdf7d1c97f843721c16cedb3aadeea8e148675935e
2025-03-14T05:47:06.2605749Z deleted: sha256:d826537aa9dc5fee54a4c084ff8d11bdac512107953441ff47f6b03793fe8c90
2025-03-14T05:47:06.2606429Z deleted: sha256:6d53d2f6a059af0b0da89c0bc757eeb05ef531658a7ce2946652778396a73996
2025-03-14T05:47:06.2607106Z deleted: sha256:e77ef3c970304cd80732a648cd94409f2dafed1740be592b49e3d70d4427027b
2025-03-14T05:47:06.2607783Z deleted: sha256:1f8f6029ff0b5de95db44674b2c2a04f941474e889683837d4bf42ac97ebc27a
2025-03-14T05:47:06.2608455Z deleted: sha256:9a3be655b6eaaf9450c7fc185591623c78f38112c4d26ca74a7026222fc21f74
2025-03-14T05:47:06.2609131Z deleted: sha256:0f437542f529403032135ccdfbe3eb8726d6e3955095552e676c41fadd49f8d3
2025-03-14T05:47:06.2609778Z deleted: sha256:270a1170e7e398434ff1b31e17e233f7d7b71aa99a40473615860068e86720af
2025-03-14T05:47:06.2610180Z 
2025-03-14T05:47:06.2653417Z Total reclaimed space: 26.99GB
2025-03-14T05:47:06.2783988Z Post job cleanup.
2025-03-14T05:47:06.2838210Z Post job cleanup.
2025-03-14T05:47:06.3841477Z [command]/usr/bin/git version
2025-03-14T05:47:06.3892122Z git version 2.47.1
2025-03-14T05:47:06.3926385Z Copying '/home/ec2-user/.gitconfig' to '/home/ec2-user/actions-runner/_work/_temp/a6085c14-9244-4a4d-ad68-4ac7fd1f2ff1/.gitconfig'
2025-03-14T05:47:06.3940853Z Temporarily overriding HOME='/home/ec2-user/actions-runner/_work/_temp/a6085c14-9244-4a4d-ad68-4ac7fd1f2ff1' before making global git config changes
2025-03-14T05:47:06.3941852Z Adding repository directory to the temporary git global config as a safe directory
2025-03-14T05:47:06.3946342Z [command]/usr/bin/git config --global --add safe.directory /home/ec2-user/actions-runner/_work/pytorch/pytorch
2025-03-14T05:47:06.3981167Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2025-03-14T05:47:06.4016678Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2025-03-14T05:47:06.4295794Z Entering 'android/libs/fbjni'
2025-03-14T05:47:06.4349008Z Entering 'third_party/FP16'
2025-03-14T05:47:06.4397941Z Entering 'third_party/FXdiv'
2025-03-14T05:47:06.4448330Z Entering 'third_party/NNPACK'
2025-03-14T05:47:06.4497529Z Entering 'third_party/NVTX'
2025-03-14T05:47:06.4547723Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T05:47:06.4596533Z Entering 'third_party/XNNPACK'
2025-03-14T05:47:06.4660921Z Entering 'third_party/benchmark'
2025-03-14T05:47:06.4709800Z Entering 'third_party/composable_kernel'
2025-03-14T05:47:06.4764058Z Entering 'third_party/cpp-httplib'
2025-03-14T05:47:06.4812556Z Entering 'third_party/cpuinfo'
2025-03-14T05:47:06.4861392Z Entering 'third_party/cudnn_frontend'
2025-03-14T05:47:06.4910145Z Entering 'third_party/cutlass'
2025-03-14T05:47:06.4966172Z Entering 'third_party/eigen'
2025-03-14T05:47:06.5018668Z Entering 'third_party/fbgemm'
2025-03-14T05:47:06.5068015Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T05:47:06.5116804Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T05:47:06.5164465Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T05:47:06.5218878Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T05:47:06.5267072Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T05:47:06.5315614Z Entering 'third_party/flash-attention'
2025-03-14T05:47:06.5364851Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T05:47:06.5419469Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T05:47:06.5477075Z Entering 'third_party/flatbuffers'
2025-03-14T05:47:06.5530433Z Entering 'third_party/fmt'
2025-03-14T05:47:06.5578759Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T05:47:06.5628093Z Entering 'third_party/gloo'
2025-03-14T05:47:06.5677506Z Entering 'third_party/googletest'
2025-03-14T05:47:06.5726404Z Entering 'third_party/ideep'
2025-03-14T05:47:06.5774063Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T05:47:06.5830660Z Entering 'third_party/ittapi'
2025-03-14T05:47:06.5879997Z Entering 'third_party/kineto'
2025-03-14T05:47:06.5928701Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T05:47:06.5976290Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T05:47:06.6025661Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T05:47:06.6073742Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T05:47:06.6122262Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T05:47:06.6169407Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T05:47:06.6219990Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T05:47:06.6267428Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T05:47:06.6316323Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T05:47:06.6364932Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T05:47:06.6414000Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T05:47:06.6461410Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T05:47:06.6510576Z Entering 'third_party/kleidiai'
2025-03-14T05:47:06.6560337Z Entering 'third_party/mimalloc'
2025-03-14T05:47:06.6609523Z Entering 'third_party/nlohmann'
2025-03-14T05:47:06.6659251Z Entering 'third_party/onnx'
2025-03-14T05:47:06.6723315Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T05:47:06.6773730Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T05:47:06.6824800Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T05:47:06.6871658Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T05:47:06.6919811Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T05:47:06.6968134Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T05:47:06.7017614Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T05:47:06.7065545Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T05:47:06.7113109Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T05:47:06.7160606Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T05:47:06.7211939Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T05:47:06.7261633Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T05:47:06.7330125Z Entering 'third_party/pocketfft'
2025-03-14T05:47:06.7380747Z Entering 'third_party/protobuf'
2025-03-14T05:47:06.7435242Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T05:47:06.7484467Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T05:47:06.7535984Z Entering 'third_party/psimd'
2025-03-14T05:47:06.7586189Z Entering 'third_party/pthreadpool'
2025-03-14T05:47:06.7638120Z Entering 'third_party/pybind11'
2025-03-14T05:47:06.7687069Z Entering 'third_party/python-peachpy'
2025-03-14T05:47:06.7736599Z Entering 'third_party/sleef'
2025-03-14T05:47:06.7786289Z Entering 'third_party/tensorpipe'
2025-03-14T05:47:06.7836209Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T05:47:06.7884388Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T05:47:06.7931800Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T05:47:06.7980016Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T05:47:06.8029987Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T05:47:06.8098843Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2025-03-14T05:47:06.8117626Z http.https://github.com/.extraheader
2025-03-14T05:47:06.8127594Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
2025-03-14T05:47:06.8156201Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2025-03-14T05:47:06.8426946Z Entering 'android/libs/fbjni'
2025-03-14T05:47:06.8462043Z http.https://github.com/.extraheader
2025-03-14T05:47:06.8492746Z Entering 'third_party/FP16'
2025-03-14T05:47:06.8525829Z http.https://github.com/.extraheader
2025-03-14T05:47:06.8556837Z Entering 'third_party/FXdiv'
2025-03-14T05:47:06.8589836Z http.https://github.com/.extraheader
2025-03-14T05:47:06.8620940Z Entering 'third_party/NNPACK'
2025-03-14T05:47:06.8654084Z http.https://github.com/.extraheader
2025-03-14T05:47:06.8684627Z Entering 'third_party/NVTX'
2025-03-14T05:47:06.8718332Z http.https://github.com/.extraheader
2025-03-14T05:47:06.8749102Z Entering 'third_party/VulkanMemoryAllocator'
2025-03-14T05:47:06.8782270Z http.https://github.com/.extraheader
2025-03-14T05:47:06.8812780Z Entering 'third_party/XNNPACK'
2025-03-14T05:47:06.8846234Z http.https://github.com/.extraheader
2025-03-14T05:47:06.8891670Z Entering 'third_party/benchmark'
2025-03-14T05:47:06.8926343Z http.https://github.com/.extraheader
2025-03-14T05:47:06.8956538Z Entering 'third_party/composable_kernel'
2025-03-14T05:47:06.8989881Z http.https://github.com/.extraheader
2025-03-14T05:47:06.9029131Z Entering 'third_party/cpp-httplib'
2025-03-14T05:47:06.9061820Z http.https://github.com/.extraheader
2025-03-14T05:47:06.9092065Z Entering 'third_party/cpuinfo'
2025-03-14T05:47:06.9126337Z http.https://github.com/.extraheader
2025-03-14T05:47:06.9157380Z Entering 'third_party/cudnn_frontend'
2025-03-14T05:47:06.9189768Z http.https://github.com/.extraheader
2025-03-14T05:47:06.9221902Z Entering 'third_party/cutlass'
2025-03-14T05:47:06.9255450Z http.https://github.com/.extraheader
2025-03-14T05:47:06.9293655Z Entering 'third_party/eigen'
2025-03-14T05:47:06.9328077Z http.https://github.com/.extraheader
2025-03-14T05:47:06.9360632Z Entering 'third_party/fbgemm'
2025-03-14T05:47:06.9393335Z http.https://github.com/.extraheader
2025-03-14T05:47:06.9425706Z Entering 'third_party/fbgemm/third_party/asmjit'
2025-03-14T05:47:06.9458230Z http.https://github.com/.extraheader
2025-03-14T05:47:06.9487536Z Entering 'third_party/fbgemm/third_party/cpuinfo'
2025-03-14T05:47:06.9520519Z http.https://github.com/.extraheader
2025-03-14T05:47:06.9551134Z Entering 'third_party/fbgemm/third_party/cutlass'
2025-03-14T05:47:06.9582967Z http.https://github.com/.extraheader
2025-03-14T05:47:06.9620109Z Entering 'third_party/fbgemm/third_party/googletest'
2025-03-14T05:47:06.9652792Z http.https://github.com/.extraheader
2025-03-14T05:47:06.9682720Z Entering 'third_party/fbgemm/third_party/hipify_torch'
2025-03-14T05:47:06.9714951Z http.https://github.com/.extraheader
2025-03-14T05:47:06.9745686Z Entering 'third_party/flash-attention'
2025-03-14T05:47:06.9779205Z http.https://github.com/.extraheader
2025-03-14T05:47:06.9810950Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-03-14T05:47:06.9842969Z http.https://github.com/.extraheader
2025-03-14T05:47:06.9879386Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-03-14T05:47:06.9912275Z http.https://github.com/.extraheader
2025-03-14T05:47:06.9951041Z Entering 'third_party/flatbuffers'
2025-03-14T05:47:06.9983902Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0017466Z Entering 'third_party/fmt'
2025-03-14T05:47:07.0050449Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0080284Z Entering 'third_party/gemmlowp/gemmlowp'
2025-03-14T05:47:07.0114362Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0144373Z Entering 'third_party/gloo'
2025-03-14T05:47:07.0178098Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0208206Z Entering 'third_party/googletest'
2025-03-14T05:47:07.0241708Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0271417Z Entering 'third_party/ideep'
2025-03-14T05:47:07.0306507Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0335642Z Entering 'third_party/ideep/mkl-dnn'
2025-03-14T05:47:07.0368245Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0406428Z Entering 'third_party/ittapi'
2025-03-14T05:47:07.0440645Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0470568Z Entering 'third_party/kineto'
2025-03-14T05:47:07.0504130Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0534431Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-03-14T05:47:07.0566979Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0597573Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-03-14T05:47:07.0629998Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0661933Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-03-14T05:47:07.0695283Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0726329Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-03-14T05:47:07.0758446Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0789997Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-03-14T05:47:07.0824586Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0854910Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-03-14T05:47:07.0887590Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0921101Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-03-14T05:47:07.0953771Z http.https://github.com/.extraheader
2025-03-14T05:47:07.0984324Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-03-14T05:47:07.1018052Z http.https://github.com/.extraheader
2025-03-14T05:47:07.1048976Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-03-14T05:47:07.1081579Z http.https://github.com/.extraheader
2025-03-14T05:47:07.1114337Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-03-14T05:47:07.1146947Z http.https://github.com/.extraheader
2025-03-14T05:47:07.1179775Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-03-14T05:47:07.1212981Z http.https://github.com/.extraheader
2025-03-14T05:47:07.1243157Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-03-14T05:47:07.1275546Z http.https://github.com/.extraheader
2025-03-14T05:47:07.1308115Z Entering 'third_party/kleidiai'
2025-03-14T05:47:07.1342159Z http.https://github.com/.extraheader
2025-03-14T05:47:07.1374121Z Entering 'third_party/mimalloc'
2025-03-14T05:47:07.1408402Z http.https://github.com/.extraheader
2025-03-14T05:47:07.1439262Z Entering 'third_party/nlohmann'
2025-03-14T05:47:07.1473033Z http.https://github.com/.extraheader
2025-03-14T05:47:07.1505725Z Entering 'third_party/onnx'
2025-03-14T05:47:07.1539451Z http.https://github.com/.extraheader
2025-03-14T05:47:07.1586671Z Entering 'third_party/onnx/third_party/pybind11'
2025-03-14T05:47:07.1621040Z http.https://github.com/.extraheader
2025-03-14T05:47:07.1654046Z Entering 'third_party/opentelemetry-cpp'
2025-03-14T05:47:07.1688986Z http.https://github.com/.extraheader
2025-03-14T05:47:07.1723324Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-03-14T05:47:07.1756245Z http.https://github.com/.extraheader
2025-03-14T05:47:07.1786724Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-03-14T05:47:07.1820869Z http.https://github.com/.extraheader
2025-03-14T05:47:07.1851392Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-03-14T05:47:07.1883052Z http.https://github.com/.extraheader
2025-03-14T05:47:07.1914506Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-03-14T05:47:07.1947100Z http.https://github.com/.extraheader
2025-03-14T05:47:07.1978899Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-03-14T05:47:07.2011780Z http.https://github.com/.extraheader
2025-03-14T05:47:07.2042141Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-03-14T05:47:07.2073808Z http.https://github.com/.extraheader
2025-03-14T05:47:07.2104528Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-03-14T05:47:07.2141006Z http.https://github.com/.extraheader
2025-03-14T05:47:07.2172165Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-03-14T05:47:07.2205177Z http.https://github.com/.extraheader
2025-03-14T05:47:07.2237552Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-03-14T05:47:07.2269481Z http.https://github.com/.extraheader
2025-03-14T05:47:07.2302219Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-03-14T05:47:07.2334982Z http.https://github.com/.extraheader
2025-03-14T05:47:07.2385852Z Entering 'third_party/pocketfft'
2025-03-14T05:47:07.2419877Z http.https://github.com/.extraheader
2025-03-14T05:47:07.2449707Z Entering 'third_party/protobuf'
2025-03-14T05:47:07.2483031Z http.https://github.com/.extraheader
2025-03-14T05:47:07.2516314Z Entering 'third_party/protobuf/third_party/benchmark'
2025-03-14T05:47:07.2550030Z http.https://github.com/.extraheader
2025-03-14T05:47:07.2579541Z Entering 'third_party/protobuf/third_party/googletest'
2025-03-14T05:47:07.2612711Z http.https://github.com/.extraheader
2025-03-14T05:47:07.2644279Z Entering 'third_party/psimd'
2025-03-14T05:47:07.2677310Z http.https://github.com/.extraheader
2025-03-14T05:47:07.2708490Z Entering 'third_party/pthreadpool'
2025-03-14T05:47:07.2742192Z http.https://github.com/.extraheader
2025-03-14T05:47:07.2771681Z Entering 'third_party/pybind11'
2025-03-14T05:47:07.2806119Z http.https://github.com/.extraheader
2025-03-14T05:47:07.2837180Z Entering 'third_party/python-peachpy'
2025-03-14T05:47:07.2870440Z http.https://github.com/.extraheader
2025-03-14T05:47:07.2901966Z Entering 'third_party/sleef'
2025-03-14T05:47:07.2935818Z http.https://github.com/.extraheader
2025-03-14T05:47:07.2965770Z Entering 'third_party/tensorpipe'
2025-03-14T05:47:07.2999836Z http.https://github.com/.extraheader
2025-03-14T05:47:07.3030884Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-03-14T05:47:07.3063061Z http.https://github.com/.extraheader
2025-03-14T05:47:07.3093266Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-03-14T05:47:07.3125667Z http.https://github.com/.extraheader
2025-03-14T05:47:07.3155160Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-03-14T05:47:07.3187130Z http.https://github.com/.extraheader
2025-03-14T05:47:07.3217725Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-03-14T05:47:07.3249773Z http.https://github.com/.extraheader
2025-03-14T05:47:07.3278671Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-03-14T05:47:07.3310712Z http.https://github.com/.extraheader
2025-03-14T05:47:07.3428978Z A job completed hook has been configured by the self-hosted runner administrator
2025-03-14T05:47:07.3454377Z ##[group]Run '/home/ec2-user/runner-scripts/after_job.sh'
2025-03-14T05:47:07.3459404Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-03-14T05:47:07.3459823Z ##[endgroup]
2025-03-14T05:47:14.3991268Z Cleaning up orphan processes