2025-07-24T04:31:16.2555254Z Current runner version: '2.326.0'
2025-07-24T04:31:16.2561778Z Runner name: 'gpud553'
2025-07-24T04:31:16.2562529Z Runner group name: 'linux.rocm.gpu.group'
2025-07-24T04:31:16.2563389Z Machine name: 'gpud553'
2025-07-24T04:31:16.2566350Z ##[group]GITHUB_TOKEN Permissions
2025-07-24T04:31:16.2568523Z Contents: read
2025-07-24T04:31:16.2569239Z Metadata: read
2025-07-24T04:31:16.2569963Z ##[endgroup]
2025-07-24T04:31:16.2571909Z Secret source: Actions
2025-07-24T04:31:16.2572511Z Prepare workflow directory
2025-07-24T04:31:16.9558792Z Prepare all required actions
2025-07-24T04:31:16.9598019Z Getting action download info
2025-07-24T04:31:17.2634427Z Download action repository 'pytorch/pytorch@main' (SHA:e09d80c545d9f006f9fa28de6144531bb2af33ff)
2025-07-24T04:31:22.0912275Z Download action repository 'aws-actions/configure-aws-credentials@ececac1a45f3b08a01d2dd070d28d111c5fe6722' (SHA:ececac1a45f3b08a01d2dd070d28d111c5fe6722)
2025-07-24T04:31:22.7137401Z Download action repository 'aws-actions/amazon-ecr-login@062b18b96a7aff071d4dc91bc00c4c1a7945b076' (SHA:062b18b96a7aff071d4dc91bc00c4c1a7945b076)
2025-07-24T04:31:23.2974823Z Download action repository 'pytorch/test-infra@main' (SHA:e32c622fbd62d34f173b9f131f1eca78782b2be2)
2025-07-24T04:31:24.4287904Z Download action repository 'actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02' (SHA:ea165f8d65b6e75b540449e92b4886f43607fa02)
2025-07-24T04:31:25.1797995Z Getting action download info
2025-07-24T04:31:25.3010260Z Download action repository 'actions/checkout@v4' (SHA:11bd71901bbe5b1630ceea73d27597364c9af683)
2025-07-24T04:31:25.9826555Z Getting action download info
2025-07-24T04:31:26.1092759Z Download action repository 'nick-fields/retry@v3.0.0' (SHA:7152eba30c6575329ac0576536151aca5a72780e)
2025-07-24T04:31:26.6588330Z Getting action download info
2025-07-24T04:31:26.7927766Z Uses: pytorch/pytorch/.github/workflows/_rocm-test.yml@refs/tags/ciflow/trunk/149961 (27c8ef1839e0747b36769efb192befc9308ff93c)
2025-07-24T04:31:26.7931339Z ##[group] Inputs
2025-07-24T04:31:26.7931674Z   build-environment: linux-jammy-rocm-py3.10
2025-07-24T04:31:26.7932504Z   test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "default", "shard": 2, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "distributed", "shard": 1, "num_shards": 1, "runner": "linux.rocm.gpu.4"}]}
2025-07-24T04:31:26.7933548Z   docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:31:26.7934090Z   sync-tag: 
2025-07-24T04:31:26.7934727Z   timeout-minutes: 300
2025-07-24T04:31:26.7935349Z   tests-to-include: test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl
2025-07-24T04:31:26.7935966Z   dashboard-tag: 
2025-07-24T04:31:26.7936427Z   disable-monitor: true
2025-07-24T04:31:26.7936661Z   monitor-log-interval: 5
2025-07-24T04:31:26.7936920Z   monitor-data-collect-interval: 1
2025-07-24T04:31:26.7937178Z ##[endgroup]
2025-07-24T04:31:26.7937497Z Complete job name: linux-jammy-rocm-py3.10 / test (default, 2, 2, linux.rocm.gpu.2)
2025-07-24T04:31:26.9675729Z ##[group]Run pytorch/pytorch/.github/actions/checkout-pytorch@main
2025-07-24T04:31:26.9676314Z with:
2025-07-24T04:31:26.9676485Z   no-sudo: true
2025-07-24T04:31:26.9676678Z   submodules: recursive
2025-07-24T04:31:26.9676879Z   fetch-depth: 0
2025-07-24T04:31:26.9677175Z env:
2025-07-24T04:31:26.9677336Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:26.9677528Z ##[endgroup]
2025-07-24T04:31:26.9748709Z ##[group]Run echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT"
2025-07-24T04:31:26.9749453Z echo "IN_CONTAINER_RUNNER=$(if [ -f /.inarc ] || [ -f /.incontainer ]; then echo true ; else echo false; fi)" >> "$GITHUB_OUTPUT"
2025-07-24T04:31:26.9783955Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:26.9784562Z env:
2025-07-24T04:31:26.9784720Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:26.9784921Z ##[endgroup]
2025-07-24T04:31:27.0125606Z ##[group]Run # Use all available CPUs for fetching
2025-07-24T04:31:27.0126355Z # Use all available CPUs for fetching
2025-07-24T04:31:27.0126907Z cd "${GITHUB_WORKSPACE}"
2025-07-24T04:31:27.0127444Z git config --global fetch.parallel 0
2025-07-24T04:31:27.0128045Z git config --global submodule.fetchJobs 0
2025-07-24T04:31:27.0128578Z 
2025-07-24T04:31:27.0129116Z # Clean workspace. The default checkout action should also do this, but
2025-07-24T04:31:27.0129837Z # do it here as well just in case
2025-07-24T04:31:27.0130318Z if [[ -d .git ]]; then
2025-07-24T04:31:27.0130776Z   if [ -z "${NO_SUDO}" ]; then
2025-07-24T04:31:27.0131262Z     sudo git clean -ffdx
2025-07-24T04:31:27.0131684Z   else
2025-07-24T04:31:27.0132065Z     git clean -ffdx
2025-07-24T04:31:27.0132446Z   fi
2025-07-24T04:31:27.0132765Z fi
2025-07-24T04:31:27.0193456Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:27.0194072Z env:
2025-07-24T04:31:27.0194531Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:27.0194940Z   NO_SUDO: true
2025-07-24T04:31:27.0195409Z ##[endgroup]
2025-07-24T04:31:27.5295189Z Removing .additional_ci_files/
2025-07-24T04:31:27.5295923Z Removing build/
2025-07-24T04:31:27.5296360Z Removing dist/
2025-07-24T04:31:27.5296794Z Removing test/test-reports/
2025-07-24T04:31:27.5424038Z ##[group]Run actions/checkout@v4
2025-07-24T04:31:27.5424567Z with:
2025-07-24T04:31:27.5425024Z   ref: 27c8ef1839e0747b36769efb192befc9308ff93c
2025-07-24T04:31:27.5425606Z   fetch-depth: 0
2025-07-24T04:31:27.5426014Z   submodules: recursive
2025-07-24T04:31:27.5426468Z   show-progress: false
2025-07-24T04:31:27.5426925Z   repository: pytorch/pytorch
2025-07-24T04:31:27.5427702Z   token: ***
2025-07-24T04:31:27.5428098Z   ssh-strict: true
2025-07-24T04:31:27.5428506Z   ssh-user: git
2025-07-24T04:31:27.5428912Z   persist-credentials: true
2025-07-24T04:31:27.5429367Z   clean: true
2025-07-24T04:31:27.5429804Z   sparse-checkout-cone-mode: true
2025-07-24T04:31:27.5430357Z   fetch-tags: false
2025-07-24T04:31:27.5430746Z   lfs: false
2025-07-24T04:31:27.5431119Z   set-safe-directory: true
2025-07-24T04:31:27.5431552Z env:
2025-07-24T04:31:27.5431899Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:27.5432330Z ##[endgroup]
2025-07-24T04:31:27.6737833Z Syncing repository: pytorch/pytorch
2025-07-24T04:31:27.6740012Z ##[group]Getting Git version info
2025-07-24T04:31:27.6740838Z Working directory is '/var/home/pytorchci/actions-runner/_work/pytorch/pytorch'
2025-07-24T04:31:27.6741927Z [command]/usr/bin/git version
2025-07-24T04:31:27.6742367Z git version 2.34.1
2025-07-24T04:31:27.6743741Z ##[endgroup]
2025-07-24T04:31:27.6749956Z Copying '/var/home/pytorchci/.gitconfig' to '/var/home/pytorchci/actions-runner/_work/_temp/131c2fd3-fd86-4c6c-85bb-dd3b4d2c55ae/.gitconfig'
2025-07-24T04:31:27.6829035Z Temporarily overriding HOME='/var/home/pytorchci/actions-runner/_work/_temp/131c2fd3-fd86-4c6c-85bb-dd3b4d2c55ae' before making global git config changes
2025-07-24T04:31:27.6830893Z Adding repository directory to the temporary git global config as a safe directory
2025-07-24T04:31:27.6832166Z [command]/usr/bin/git config --global --add safe.directory /var/home/pytorchci/actions-runner/_work/pytorch/pytorch
2025-07-24T04:31:27.6835726Z [command]/usr/bin/git config --local --get remote.origin.url
2025-07-24T04:31:27.6886100Z https://github.com/pytorch/pytorch
2025-07-24T04:31:27.6903925Z ##[group]Removing previously created refs, to avoid conflicts
2025-07-24T04:31:27.6906692Z [command]/usr/bin/git rev-parse --symbolic-full-name --verify --quiet HEAD
2025-07-24T04:31:27.6952936Z HEAD
2025-07-24T04:31:27.7042253Z ##[endgroup]
2025-07-24T04:31:27.7044590Z [command]/usr/bin/git submodule status
2025-07-24T04:31:27.7821733Z  7e1e1fe3858c63c251c637ae41a20de425dde96f android/libs/fbjni (v0.1.0-12-g7e1e1fe)
2025-07-24T04:31:27.8045956Z  4dfe081cf6bcd15db339cf2680b9281b8451eeb3 third_party/FP16 (4dfe081)
2025-07-24T04:31:27.8269931Z  b408327ac2a15ec3e43352421954f5b1967701d1 third_party/FXdiv (b408327)
2025-07-24T04:31:27.8515775Z  c07e3a0400713d546e0dea2d5466dd22ea389c73 third_party/NNPACK (c07e3a0)
2025-07-24T04:31:27.8648324Z  2942f167cc30c5e3a44a2aecd5b0d9c07ff61a07 third_party/NVTX (v3.1.0-263-g2942f16)
2025-07-24T04:31:27.8845562Z  1d8f600fd424278486eade7ed3e877c99f0846b1 third_party/VulkanMemoryAllocator (v2.1.0-982-g1d8f600)
2025-07-24T04:31:27.9756611Z  51a0103656eff6fc9bfd39a4597923c4b542c883 third_party/XNNPACK (remotes/origin/ds/ndk-1243-g51a0103656)
2025-07-24T04:31:27.9850611Z  01aae101b9e5e94d6c16a9514c9fb8df99c93150 third_party/aiter (v0.1.1-92-g01aae101)
2025-07-24T04:31:27.9921668Z  299e5928955cc62af9968370293b916f5130916f third_party/benchmark (v1.9.3)
2025-07-24T04:31:28.0079252Z  df6023e305f389bbf7249b0c4414e649f3ad6598 third_party/composable_kernel (rocm-6.4.2-337-gdf6023e30)
2025-07-24T04:31:28.0258156Z  3af7f2c16147f3fbc6e4d717032daf505dc1652c third_party/cpp-httplib (v0.20.1)
2025-07-24T04:31:28.0562349Z  5e3d2445e6a84d9599bee2bf78edbb4d80865e1d third_party/cpuinfo (5e3d244)
2025-07-24T04:31:28.0660062Z  f937055efc6d414d11f4c6577e3977fe74f35fb6 third_party/cudnn_frontend (v0.5-52-gf937055)
2025-07-24T04:31:28.0905934Z  b995f933179c22d3fe0d871c3a53d11e4681950f third_party/cutlass (v4.0.0)
2025-07-24T04:31:28.1024738Z  157e88b750c452bef2ab4653fe9d1eeb151ce4c3 third_party/fbgemm (v1.2.0-186-g157e88b7)
2025-07-24T04:31:28.1230808Z  979702c87a8713a8e0a5e9fee122b90d2ef13be5 third_party/flash-attention (v2.7.4)
2025-07-24T04:31:28.1299227Z  a2cd1ea3b6d3fee220106b5fed3f7ce8da9eb757 third_party/flatbuffers (v24.12.23)
2025-07-24T04:31:28.2058012Z  40626af88bd7df9a5fb80be7b25ac85b122d6c21 third_party/fmt (11.2.0)
2025-07-24T04:31:28.2353999Z  3fb5c176c17c765a3492cd2f0321b0dab712f350 third_party/gemmlowp/gemmlowp (remotes/origin/revert-87-master-135-g3fb5c17)
2025-07-24T04:31:28.2673770Z  c7b7b022c124d9643957d9bd55f57ac59fce8fa2 third_party/gloo (remotes/origin/gh/c-p-i-o/1/base-33-gc7b7b02)
2025-07-24T04:31:28.3151188Z  52eb8108c5bdec04579160ae17225d66034bd723 third_party/googletest (release-1.8.0-3544-g52eb8108)
2025-07-24T04:31:28.3360060Z  719d8e6cd7f7a0e01b155657526d693acf97c2b3 third_party/ideep (pytorch-rls-v3.7.1)
2025-07-24T04:31:28.3522519Z  dec1d23ca65ab069d225dfe40dea14f455170959 third_party/ittapi (v3.25.5)
2025-07-24T04:31:28.4129942Z  5e7501833f1021ce6f618572d3baf657b6319658 third_party/kineto (remotes/origin/sraikund/test-98-g5e75018)
2025-07-24T04:31:28.4199823Z  cca02c2f69dd18e1f12647c1c0bdc8cf90e680c7 third_party/kleidiai (v1.8.0)
2025-07-24T04:31:28.4271137Z  fbd8b99c2b828428947d70fdc046bb55609be93e third_party/mimalloc (v2.2.4)
2025-07-24T04:31:28.4341223Z  55f93686c01528224f448c19128836e7df245f72 third_party/nlohmann (v3.12.0)
2025-07-24T04:31:28.5004805Z  e709452ef2bbc1d113faf678c24e6d3467696e83 third_party/onnx (v1.18.0)
2025-07-24T04:31:28.5079442Z  a799f4aed9c94b765dcdaabaeab7d5e7e2310878 third_party/opentelemetry-cpp (v1.14.2)
2025-07-24T04:31:28.5154818Z  0fa0ef591e38c2758e3184c6c23e497b9f732ffa third_party/pocketfft (release_for_eigen-40-g0fa0ef5)
2025-07-24T04:31:28.5824508Z  d1eca4e4b421cd2997495c4b4e65cea6be4e9b8a third_party/protobuf (v3.7.0-rc.2-1279-gd1eca4e4b)
2025-07-24T04:31:28.6049671Z  072586a71b55b7f8c584153d223e95687148a900 third_party/psimd (heads/master)
2025-07-24T04:31:28.6201879Z  4fe0e1e183925bf8cfa6aae24237e724a96479b8 third_party/pthreadpool (0.1-144-g4fe0e1e)
2025-07-24T04:31:28.6337574Z  a2e59f0e7065404b44dfe92a28aca47ba1378dc4 third_party/pybind11 (v2.11.0-182-ga2e59f0e)
2025-07-24T04:31:28.6561425Z  f45429b087dd7d5bc78bb40dc7cf06425c252d67 third_party/python-peachpy (remotes/origin/pre-generated)
2025-07-24T04:31:28.6758453Z  5a1d179df9cf652951b59010a2d2075372d67f68 third_party/sleef (3.8)
2025-07-24T04:31:28.6973337Z  52791a2fd214b2a9dc5759d36725909c1daa7f2e third_party/tensorpipe (remotes/origin/master)
2025-07-24T04:31:28.7009738Z ##[group]Cleaning the repository
2025-07-24T04:31:28.7018150Z [command]/usr/bin/git clean -ffdx
2025-07-24T04:31:28.7654807Z [command]/usr/bin/git reset --hard HEAD
2025-07-24T04:31:28.9478868Z HEAD is now at 4214c869d37 FXConverter handling of generic output in inductor fallback kernel
2025-07-24T04:31:28.9520416Z ##[endgroup]
2025-07-24T04:31:28.9525075Z ##[group]Disabling automatic garbage collection
2025-07-24T04:31:28.9534836Z [command]/usr/bin/git config --local gc.auto 0
2025-07-24T04:31:28.9604580Z ##[endgroup]
2025-07-24T04:31:28.9605368Z ##[group]Setting up auth
2025-07-24T04:31:28.9618605Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2025-07-24T04:31:28.9716228Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2025-07-24T04:31:29.0404759Z Entering 'android/libs/fbjni'
2025-07-24T04:31:29.0532318Z Entering 'third_party/FP16'
2025-07-24T04:31:29.0654350Z Entering 'third_party/FXdiv'
2025-07-24T04:31:29.0780012Z Entering 'third_party/NNPACK'
2025-07-24T04:31:29.0903269Z Entering 'third_party/NVTX'
2025-07-24T04:31:29.1028045Z Entering 'third_party/VulkanMemoryAllocator'
2025-07-24T04:31:29.1154886Z Entering 'third_party/XNNPACK'
2025-07-24T04:31:29.1318246Z Entering 'third_party/aiter'
2025-07-24T04:31:29.1438673Z Entering 'third_party/aiter/3rdparty/composable_kernel'
2025-07-24T04:31:29.1579399Z Entering 'third_party/benchmark'
2025-07-24T04:31:29.1701580Z Entering 'third_party/composable_kernel'
2025-07-24T04:31:29.1845402Z Entering 'third_party/cpp-httplib'
2025-07-24T04:31:29.1966936Z Entering 'third_party/cpuinfo'
2025-07-24T04:31:29.2085978Z Entering 'third_party/cudnn_frontend'
2025-07-24T04:31:29.2208595Z Entering 'third_party/cutlass'
2025-07-24T04:31:29.2350013Z Entering 'third_party/fbgemm'
2025-07-24T04:31:29.2471706Z Entering 'third_party/fbgemm/external/asmjit'
2025-07-24T04:31:29.2581914Z Entering 'third_party/fbgemm/external/composable_kernel'
2025-07-24T04:31:29.2714953Z Entering 'third_party/fbgemm/external/cpuinfo'
2025-07-24T04:31:29.2829262Z Entering 'third_party/fbgemm/external/cutlass'
2025-07-24T04:31:29.2958612Z Entering 'third_party/fbgemm/external/googletest'
2025-07-24T04:31:29.3069420Z Entering 'third_party/fbgemm/external/hipify_torch'
2025-07-24T04:31:29.3180000Z Entering 'third_party/fbgemm/external/json'
2025-07-24T04:31:29.3301303Z Entering 'third_party/flash-attention'
2025-07-24T04:31:29.3418090Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-07-24T04:31:29.3546335Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-07-24T04:31:29.3681805Z Entering 'third_party/flatbuffers'
2025-07-24T04:31:29.3806012Z Entering 'third_party/fmt'
2025-07-24T04:31:29.3925919Z Entering 'third_party/gemmlowp/gemmlowp'
2025-07-24T04:31:29.4047948Z Entering 'third_party/gloo'
2025-07-24T04:31:29.4169454Z Entering 'third_party/googletest'
2025-07-24T04:31:29.4289875Z Entering 'third_party/ideep'
2025-07-24T04:31:29.4405995Z Entering 'third_party/ideep/mkl-dnn'
2025-07-24T04:31:29.4538506Z Entering 'third_party/ittapi'
2025-07-24T04:31:29.4657861Z Entering 'third_party/kineto'
2025-07-24T04:31:29.4774983Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-07-24T04:31:29.4885266Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-07-24T04:31:29.5003357Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-07-24T04:31:29.5115895Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-07-24T04:31:29.5226320Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-07-24T04:31:29.5332490Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-07-24T04:31:29.5451837Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-07-24T04:31:29.5561581Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-07-24T04:31:29.5673510Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-07-24T04:31:29.5786834Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-07-24T04:31:29.5904837Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-07-24T04:31:29.6017006Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-07-24T04:31:29.6135730Z Entering 'third_party/kleidiai'
2025-07-24T04:31:29.6254873Z Entering 'third_party/mimalloc'
2025-07-24T04:31:29.6372758Z Entering 'third_party/nlohmann'
2025-07-24T04:31:29.6497187Z Entering 'third_party/onnx'
2025-07-24T04:31:29.6652162Z Entering 'third_party/onnx/third_party/pybind11'
2025-07-24T04:31:29.6777449Z Entering 'third_party/opentelemetry-cpp'
2025-07-24T04:31:29.6895764Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-07-24T04:31:29.7008461Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-07-24T04:31:29.7120853Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-07-24T04:31:29.7232012Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-07-24T04:31:29.7345399Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-07-24T04:31:29.7451422Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-07-24T04:31:29.7564857Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-07-24T04:31:29.7673121Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-07-24T04:31:29.7788513Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-07-24T04:31:29.7907442Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-07-24T04:31:29.8071121Z Entering 'third_party/pocketfft'
2025-07-24T04:31:29.8189196Z Entering 'third_party/protobuf'
2025-07-24T04:31:29.8312119Z Entering 'third_party/protobuf/third_party/benchmark'
2025-07-24T04:31:29.8424316Z Entering 'third_party/protobuf/third_party/googletest'
2025-07-24T04:31:29.8545973Z Entering 'third_party/psimd'
2025-07-24T04:31:29.8664566Z Entering 'third_party/pthreadpool'
2025-07-24T04:31:29.8782116Z Entering 'third_party/pybind11'
2025-07-24T04:31:29.8901805Z Entering 'third_party/python-peachpy'
2025-07-24T04:31:29.9022502Z Entering 'third_party/sleef'
2025-07-24T04:31:29.9144057Z Entering 'third_party/tensorpipe'
2025-07-24T04:31:29.9260655Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-07-24T04:31:29.9374426Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-07-24T04:31:29.9487439Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-07-24T04:31:29.9600070Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-07-24T04:31:29.9705846Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-07-24T04:31:29.9868576Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2025-07-24T04:31:29.9948151Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2025-07-24T04:31:30.0642424Z Entering 'android/libs/fbjni'
2025-07-24T04:31:30.0771221Z Entering 'third_party/FP16'
2025-07-24T04:31:30.0892907Z Entering 'third_party/FXdiv'
2025-07-24T04:31:30.1015780Z Entering 'third_party/NNPACK'
2025-07-24T04:31:30.1139820Z Entering 'third_party/NVTX'
2025-07-24T04:31:30.1263731Z Entering 'third_party/VulkanMemoryAllocator'
2025-07-24T04:31:30.1385573Z Entering 'third_party/XNNPACK'
2025-07-24T04:31:30.1540624Z Entering 'third_party/aiter'
2025-07-24T04:31:30.1663717Z Entering 'third_party/aiter/3rdparty/composable_kernel'
2025-07-24T04:31:30.1804612Z Entering 'third_party/benchmark'
2025-07-24T04:31:30.1926626Z Entering 'third_party/composable_kernel'
2025-07-24T04:31:30.2065726Z Entering 'third_party/cpp-httplib'
2025-07-24T04:31:30.2188576Z Entering 'third_party/cpuinfo'
2025-07-24T04:31:30.2311861Z Entering 'third_party/cudnn_frontend'
2025-07-24T04:31:30.2433443Z Entering 'third_party/cutlass'
2025-07-24T04:31:30.2578377Z Entering 'third_party/fbgemm'
2025-07-24T04:31:30.2707823Z Entering 'third_party/fbgemm/external/asmjit'
2025-07-24T04:31:30.2827355Z Entering 'third_party/fbgemm/external/composable_kernel'
2025-07-24T04:31:30.2959359Z Entering 'third_party/fbgemm/external/cpuinfo'
2025-07-24T04:31:30.3081234Z Entering 'third_party/fbgemm/external/cutlass'
2025-07-24T04:31:30.3211369Z Entering 'third_party/fbgemm/external/googletest'
2025-07-24T04:31:30.3326556Z Entering 'third_party/fbgemm/external/hipify_torch'
2025-07-24T04:31:30.3440263Z Entering 'third_party/fbgemm/external/json'
2025-07-24T04:31:30.3564681Z Entering 'third_party/flash-attention'
2025-07-24T04:31:30.3689765Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-07-24T04:31:30.3818864Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-07-24T04:31:30.3964303Z Entering 'third_party/flatbuffers'
2025-07-24T04:31:30.4092864Z Entering 'third_party/fmt'
2025-07-24T04:31:30.4214609Z Entering 'third_party/gemmlowp/gemmlowp'
2025-07-24T04:31:30.4335727Z Entering 'third_party/gloo'
2025-07-24T04:31:30.4456887Z Entering 'third_party/googletest'
2025-07-24T04:31:30.4577631Z Entering 'third_party/ideep'
2025-07-24T04:31:30.4696666Z Entering 'third_party/ideep/mkl-dnn'
2025-07-24T04:31:30.4833205Z Entering 'third_party/ittapi'
2025-07-24T04:31:30.4957878Z Entering 'third_party/kineto'
2025-07-24T04:31:30.5078569Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-07-24T04:31:30.5192797Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-07-24T04:31:30.5314044Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-07-24T04:31:30.5433308Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-07-24T04:31:30.5552350Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-07-24T04:31:30.5666820Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-07-24T04:31:30.5790261Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-07-24T04:31:30.5905889Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-07-24T04:31:30.6021371Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-07-24T04:31:30.6136408Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-07-24T04:31:30.6261336Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-07-24T04:31:30.6377056Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-07-24T04:31:30.6501042Z Entering 'third_party/kleidiai'
2025-07-24T04:31:30.6624982Z Entering 'third_party/mimalloc'
2025-07-24T04:31:30.6751125Z Entering 'third_party/nlohmann'
2025-07-24T04:31:30.6878030Z Entering 'third_party/onnx'
2025-07-24T04:31:30.7029876Z Entering 'third_party/onnx/third_party/pybind11'
2025-07-24T04:31:30.7161126Z Entering 'third_party/opentelemetry-cpp'
2025-07-24T04:31:30.7285137Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-07-24T04:31:30.7399738Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-07-24T04:31:30.7519113Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-07-24T04:31:30.7634911Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-07-24T04:31:30.7754510Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-07-24T04:31:30.7869683Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-07-24T04:31:30.7979176Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-07-24T04:31:30.8093586Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-07-24T04:31:30.8211922Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-07-24T04:31:30.8336096Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-07-24T04:31:30.8499131Z Entering 'third_party/pocketfft'
2025-07-24T04:31:30.8621883Z Entering 'third_party/protobuf'
2025-07-24T04:31:30.8750377Z Entering 'third_party/protobuf/third_party/benchmark'
2025-07-24T04:31:30.8865835Z Entering 'third_party/protobuf/third_party/googletest'
2025-07-24T04:31:30.8990889Z Entering 'third_party/psimd'
2025-07-24T04:31:30.9112919Z Entering 'third_party/pthreadpool'
2025-07-24T04:31:30.9233145Z Entering 'third_party/pybind11'
2025-07-24T04:31:30.9354803Z Entering 'third_party/python-peachpy'
2025-07-24T04:31:30.9474771Z Entering 'third_party/sleef'
2025-07-24T04:31:30.9595004Z Entering 'third_party/tensorpipe'
2025-07-24T04:31:30.9718922Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-07-24T04:31:30.9833483Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-07-24T04:31:30.9948168Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-07-24T04:31:31.0062253Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-07-24T04:31:31.0173657Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-07-24T04:31:31.0354771Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***
2025-07-24T04:31:31.0454745Z ##[endgroup]
2025-07-24T04:31:31.0455593Z ##[group]Fetching the repository
2025-07-24T04:31:31.0469716Z [command]/usr/bin/git -c protocol.version=2 fetch --prune --no-recurse-submodules origin +refs/heads/*:refs/remotes/origin/* +refs/tags/*:refs/tags/*
2025-07-24T04:31:32.2017544Z [command]/usr/bin/git rev-parse --verify --quiet 27c8ef1839e0747b36769efb192befc9308ff93c^{object}
2025-07-24T04:31:32.2085537Z 27c8ef1839e0747b36769efb192befc9308ff93c
2025-07-24T04:31:32.2098359Z ##[endgroup]
2025-07-24T04:31:32.2099254Z ##[group]Determining the checkout info
2025-07-24T04:31:32.2101122Z ##[endgroup]
2025-07-24T04:31:32.2108102Z [command]/usr/bin/git sparse-checkout disable
2025-07-24T04:31:32.2544337Z [command]/usr/bin/git config --local --unset-all extensions.worktreeConfig
2025-07-24T04:31:32.2616954Z ##[group]Checking out the ref
2025-07-24T04:31:32.2625216Z [command]/usr/bin/git checkout --progress --force 27c8ef1839e0747b36769efb192befc9308ff93c
2025-07-24T04:31:32.5367137Z Previous HEAD position was 4214c869d37 FXConverter handling of generic output in inductor fallback kernel
2025-07-24T04:31:32.5385416Z HEAD is now at 27c8ef1839e Update
2025-07-24T04:31:32.5450993Z ##[endgroup]
2025-07-24T04:31:32.5451761Z ##[group]Setting up auth for fetching submodules
2025-07-24T04:31:32.5462033Z [command]/usr/bin/git config --global http.https://github.com/.extraheader AUTHORIZATION: basic ***
2025-07-24T04:31:32.5556638Z [command]/usr/bin/git config --global --unset-all url.https://github.com/.insteadOf
2025-07-24T04:31:32.5638684Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf git@github.com:
2025-07-24T04:31:32.5718685Z [command]/usr/bin/git config --global --add url.https://github.com/.insteadOf org-21003710@github.com:
2025-07-24T04:31:32.5794765Z ##[endgroup]
2025-07-24T04:31:32.5795511Z ##[group]Fetching submodules
2025-07-24T04:31:32.5804333Z [command]/usr/bin/git submodule sync --recursive
2025-07-24T04:31:32.6504338Z Synchronizing submodule url for 'android/libs/fbjni'
2025-07-24T04:31:32.6613946Z Synchronizing submodule url for 'third_party/FP16'
2025-07-24T04:31:32.6723132Z Synchronizing submodule url for 'third_party/FXdiv'
2025-07-24T04:31:32.6825969Z Synchronizing submodule url for 'third_party/NNPACK'
2025-07-24T04:31:32.6929277Z Synchronizing submodule url for 'third_party/NVTX'
2025-07-24T04:31:32.7037001Z Synchronizing submodule url for 'third_party/VulkanMemoryAllocator'
2025-07-24T04:31:32.7143738Z Synchronizing submodule url for 'third_party/XNNPACK'
2025-07-24T04:31:32.7280205Z Synchronizing submodule url for 'third_party/aiter'
2025-07-24T04:31:32.7380595Z Synchronizing submodule url for 'third_party/aiter/3rdparty/composable_kernel'
2025-07-24T04:31:32.7501249Z Synchronizing submodule url for 'third_party/benchmark'
2025-07-24T04:31:32.7604867Z Synchronizing submodule url for 'third_party/composable_kernel'
2025-07-24T04:31:32.7726667Z Synchronizing submodule url for 'third_party/cpp-httplib'
2025-07-24T04:31:32.7831710Z Synchronizing submodule url for 'third_party/cpuinfo'
2025-07-24T04:31:32.7937217Z Synchronizing submodule url for 'third_party/cudnn_frontend'
2025-07-24T04:31:32.8042925Z Synchronizing submodule url for 'third_party/cutlass'
2025-07-24T04:31:32.8168207Z Synchronizing submodule url for 'third_party/fbgemm'
2025-07-24T04:31:32.8270407Z Synchronizing submodule url for 'third_party/fbgemm/external/asmjit'
2025-07-24T04:31:32.8366823Z Synchronizing submodule url for 'third_party/fbgemm/external/composable_kernel'
2025-07-24T04:31:32.8476452Z Synchronizing submodule url for 'third_party/fbgemm/external/cpuinfo'
2025-07-24T04:31:32.8572700Z Synchronizing submodule url for 'third_party/fbgemm/external/cutlass'
2025-07-24T04:31:32.8691365Z Synchronizing submodule url for 'third_party/fbgemm/external/googletest'
2025-07-24T04:31:32.8785004Z Synchronizing submodule url for 'third_party/fbgemm/external/hipify_torch'
2025-07-24T04:31:32.8880280Z Synchronizing submodule url for 'third_party/fbgemm/external/json'
2025-07-24T04:31:32.8989206Z Synchronizing submodule url for 'third_party/flash-attention'
2025-07-24T04:31:32.9085658Z Synchronizing submodule url for 'third_party/flash-attention/csrc/composable_kernel'
2025-07-24T04:31:32.9193827Z Synchronizing submodule url for 'third_party/flash-attention/csrc/cutlass'
2025-07-24T04:31:32.9319988Z Synchronizing submodule url for 'third_party/flatbuffers'
2025-07-24T04:31:32.9426908Z Synchronizing submodule url for 'third_party/fmt'
2025-07-24T04:31:32.9530830Z Synchronizing submodule url for 'third_party/gemmlowp/gemmlowp'
2025-07-24T04:31:32.9634696Z Synchronizing submodule url for 'third_party/gloo'
2025-07-24T04:31:32.9741879Z Synchronizing submodule url for 'third_party/googletest'
2025-07-24T04:31:32.9845490Z Synchronizing submodule url for 'third_party/ideep'
2025-07-24T04:31:32.9939548Z Synchronizing submodule url for 'third_party/ideep/mkl-dnn'
2025-07-24T04:31:33.0056303Z Synchronizing submodule url for 'third_party/ittapi'
2025-07-24T04:31:33.0165003Z Synchronizing submodule url for 'third_party/kineto'
2025-07-24T04:31:33.0260846Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog'
2025-07-24T04:31:33.0350355Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-07-24T04:31:33.0457787Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-07-24T04:31:33.0553934Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-07-24T04:31:33.0653530Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-07-24T04:31:33.0742750Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-07-24T04:31:33.0847577Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-07-24T04:31:33.0944111Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-07-24T04:31:33.1039227Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-07-24T04:31:33.1136074Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-07-24T04:31:33.1238745Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/fmt'
2025-07-24T04:31:33.1331794Z Synchronizing submodule url for 'third_party/kineto/libkineto/third_party/googletest'
2025-07-24T04:31:33.1434516Z Synchronizing submodule url for 'third_party/kleidiai'
2025-07-24T04:31:33.1546613Z Synchronizing submodule url for 'third_party/mimalloc'
2025-07-24T04:31:33.1654686Z Synchronizing submodule url for 'third_party/nlohmann'
2025-07-24T04:31:33.1763184Z Synchronizing submodule url for 'third_party/onnx'
2025-07-24T04:31:33.1900776Z Synchronizing submodule url for 'third_party/onnx/third_party/pybind11'
2025-07-24T04:31:33.2013380Z Synchronizing submodule url for 'third_party/opentelemetry-cpp'
2025-07-24T04:31:33.2113086Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-07-24T04:31:33.2211757Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/googletest'
2025-07-24T04:31:33.2306504Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-07-24T04:31:33.2400268Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-07-24T04:31:33.2506890Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-07-24T04:31:33.2604436Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-07-24T04:31:33.2697154Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-07-24T04:31:33.2787637Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-07-24T04:31:33.2890896Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-07-24T04:31:33.2992634Z Synchronizing submodule url for 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-07-24T04:31:33.3140225Z Synchronizing submodule url for 'third_party/pocketfft'
2025-07-24T04:31:33.3242377Z Synchronizing submodule url for 'third_party/protobuf'
2025-07-24T04:31:33.3347408Z Synchronizing submodule url for 'third_party/protobuf/third_party/benchmark'
2025-07-24T04:31:33.3440679Z Synchronizing submodule url for 'third_party/protobuf/third_party/googletest'
2025-07-24T04:31:33.3545449Z Synchronizing submodule url for 'third_party/psimd'
2025-07-24T04:31:33.3651635Z Synchronizing submodule url for 'third_party/pthreadpool'
2025-07-24T04:31:33.3755384Z Synchronizing submodule url for 'third_party/pybind11'
2025-07-24T04:31:33.3858978Z Synchronizing submodule url for 'third_party/python-peachpy'
2025-07-24T04:31:33.3961792Z Synchronizing submodule url for 'third_party/sleef'
2025-07-24T04:31:33.4064031Z Synchronizing submodule url for 'third_party/tensorpipe'
2025-07-24T04:31:33.4164549Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/googletest'
2025-07-24T04:31:33.4258035Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/libnop'
2025-07-24T04:31:33.4350594Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/libuv'
2025-07-24T04:31:33.4445259Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/pybind11'
2025-07-24T04:31:33.4532202Z Synchronizing submodule url for 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-07-24T04:31:33.4675717Z [command]/usr/bin/git -c protocol.version=2 submodule update --init --force --recursive
2025-07-24T04:31:33.5953072Z Submodule path 'android/libs/fbjni': checked out '7e1e1fe3858c63c251c637ae41a20de425dde96f'
2025-07-24T04:31:33.6569577Z Submodule path 'third_party/FP16': checked out '4dfe081cf6bcd15db339cf2680b9281b8451eeb3'
2025-07-24T04:31:33.7182884Z Submodule path 'third_party/FXdiv': checked out 'b408327ac2a15ec3e43352421954f5b1967701d1'
2025-07-24T04:31:33.7820483Z Submodule path 'third_party/NNPACK': checked out 'c07e3a0400713d546e0dea2d5466dd22ea389c73'
2025-07-24T04:31:33.8495411Z Submodule path 'third_party/NVTX': checked out '2942f167cc30c5e3a44a2aecd5b0d9c07ff61a07'
2025-07-24T04:31:33.9139601Z Submodule path 'third_party/VulkanMemoryAllocator': checked out '1d8f600fd424278486eade7ed3e877c99f0846b1'
2025-07-24T04:31:34.0176636Z Submodule path 'third_party/XNNPACK': checked out '51a0103656eff6fc9bfd39a4597923c4b542c883'
2025-07-24T04:31:34.1025475Z Submodule path 'third_party/aiter': checked out '01aae101b9e5e94d6c16a9514c9fb8df99c93150'
2025-07-24T04:31:34.1935462Z Submodule path 'third_party/aiter/3rdparty/composable_kernel': checked out 'cffe8fa2a442ac8e80dd236a1a5d24fe3d7e0cbf'
2025-07-24T04:31:34.2639946Z Submodule path 'third_party/benchmark': checked out '299e5928955cc62af9968370293b916f5130916f'
2025-07-24T04:31:34.5062103Z Submodule path 'third_party/composable_kernel': checked out '434d19f696da62c12b5372b32cbc9ba968588d7e'
2025-07-24T04:31:34.5777707Z Submodule path 'third_party/cpp-httplib': checked out '3af7f2c16147f3fbc6e4d717032daf505dc1652c'
2025-07-24T04:31:34.6442697Z Submodule path 'third_party/cpuinfo': checked out '5e3d2445e6a84d9599bee2bf78edbb4d80865e1d'
2025-07-24T04:31:34.7154098Z Submodule path 'third_party/cudnn_frontend': checked out 'f937055efc6d414d11f4c6577e3977fe74f35fb6'
2025-07-24T04:31:34.8000515Z Submodule path 'third_party/cutlass': checked out 'b995f933179c22d3fe0d871c3a53d11e4681950f'
2025-07-24T04:31:34.8884993Z Submodule path 'third_party/fbgemm': checked out '157e88b750c452bef2ab4653fe9d1eeb151ce4c3'
2025-07-24T04:31:34.9454631Z Submodule path 'third_party/fbgemm/external/asmjit': checked out 'e5d7c0bd5d9aec44d68830187138149e6a8c4e32'
2025-07-24T04:31:35.0166811Z Submodule path 'third_party/fbgemm/external/composable_kernel': checked out '4a61bdd4bd4ed730e078aebc7c0fcf046ff29406'
2025-07-24T04:31:35.0803556Z Submodule path 'third_party/fbgemm/external/cpuinfo': checked out '6543fec09b2f04ac4a666882998b534afc9c1349'
2025-07-24T04:31:35.1550281Z Submodule path 'third_party/fbgemm/external/cutlass': checked out '3ed8d2ec4ba35ef5d9d8353826209b6f868f63d3'
2025-07-24T04:31:35.2185078Z Submodule path 'third_party/fbgemm/external/googletest': checked out 'f8d7d77c06936315286eb55f8de22cd23c188571'
2025-07-24T04:31:35.2756367Z Submodule path 'third_party/fbgemm/external/hipify_torch': checked out 'a4337c69fe0e2552a7b7b0669178926beeed828c'
2025-07-24T04:31:35.3464949Z Submodule path 'third_party/fbgemm/external/json': checked out '9cca280a4d0ccf0c08f47a99aa71d1b0e52f8d03'
2025-07-24T04:31:35.4202183Z Submodule path 'third_party/flash-attention': checked out '979702c87a8713a8e0a5e9fee122b90d2ef13be5'
2025-07-24T04:31:35.5102231Z Submodule path 'third_party/flash-attention/csrc/composable_kernel': checked out '888317e698e9803c62bd38568abc9e05d7709f33'
2025-07-24T04:31:35.5909366Z Submodule path 'third_party/flash-attention/csrc/cutlass': checked out 'c506e16788cb08416a4a57e11a9067beeee29420'
2025-07-24T04:31:35.6696511Z Submodule path 'third_party/flatbuffers': checked out 'a2cd1ea3b6d3fee220106b5fed3f7ce8da9eb757'
2025-07-24T04:31:35.7374769Z Submodule path 'third_party/fmt': checked out '40626af88bd7df9a5fb80be7b25ac85b122d6c21'
2025-07-24T04:31:35.8008503Z Submodule path 'third_party/gemmlowp/gemmlowp': checked out '3fb5c176c17c765a3492cd2f0321b0dab712f350'
2025-07-24T04:31:35.8665647Z Submodule path 'third_party/gloo': checked out 'c7b7b022c124d9643957d9bd55f57ac59fce8fa2'
2025-07-24T04:31:35.9320176Z Submodule path 'third_party/googletest': checked out '52eb8108c5bdec04579160ae17225d66034bd723'
2025-07-24T04:31:35.9974631Z Submodule path 'third_party/ideep': checked out '719d8e6cd7f7a0e01b155657526d693acf97c2b3'
2025-07-24T04:31:36.0923472Z Submodule path 'third_party/ideep/mkl-dnn': checked out '8d263e693366ef8db40acc569cc7d8edf644556d'
2025-07-24T04:31:36.1592038Z Submodule path 'third_party/ittapi': checked out 'dec1d23ca65ab069d225dfe40dea14f455170959'
2025-07-24T04:31:36.2274923Z Submodule path 'third_party/kineto': checked out '5e7501833f1021ce6f618572d3baf657b6319658'
2025-07-24T04:31:36.2919621Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog': checked out '7d04a0053a845370ae06ce317a22a48e9edcc74e'
2025-07-24T04:31:36.3539228Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM': checked out 'ffde4e54bc7249a6039a5e6b45b395141e1217f9'
2025-07-24T04:31:36.4149631Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr': checked out '871ed52d350214a034f6ef8a3b8f51c5ce1bd400'
2025-07-24T04:31:36.4754694Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt': checked out 'cd4af11efc9c622896a3e4cb599fa28668ca3d05'
2025-07-24T04:31:36.5349577Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags': checked out 'e171aa2d15ed9eb17054558e0b3a6a413bb01067'
2025-07-24T04:31:36.5918418Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc': checked out '8411df715cf522606e3b1aca386ddfc0b63d34b4'
2025-07-24T04:31:36.6523522Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog': checked out 'b33e3bad4c46c8a6345525fd822af355e5ef9446'
2025-07-24T04:31:36.7131986Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest': checked out '58d77fa8070e8cec2dc1ed015d66b454c8d78850'
2025-07-24T04:31:36.7828290Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/json': checked out '4f8fba14066156b73f1189a2b8bd568bde5284c5'
2025-07-24T04:31:36.8407552Z Submodule path 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs': checked out 'f68a2fa8ea36c783bdd760371411fcb495aa3150'
2025-07-24T04:31:36.9024167Z Submodule path 'third_party/kineto/libkineto/third_party/fmt': checked out '0041a40c1350ba702d475b9c4ad62da77caea164'
2025-07-24T04:31:36.9620971Z Submodule path 'third_party/kineto/libkineto/third_party/googletest': checked out '7aca84427f224eeed3144123d5230d5871e93347'
2025-07-24T04:31:37.0311823Z Submodule path 'third_party/kleidiai': checked out 'cca02c2f69dd18e1f12647c1c0bdc8cf90e680c7'
2025-07-24T04:31:37.1005417Z Submodule path 'third_party/mimalloc': checked out 'fbd8b99c2b828428947d70fdc046bb55609be93e'
2025-07-24T04:31:37.1752283Z Submodule path 'third_party/nlohmann': checked out '55f93686c01528224f448c19128836e7df245f72'
2025-07-24T04:31:37.2774239Z Submodule path 'third_party/onnx': checked out 'e709452ef2bbc1d113faf678c24e6d3467696e83'
2025-07-24T04:31:37.3491478Z Submodule path 'third_party/onnx/third_party/pybind11': checked out 'a2e59f0e7065404b44dfe92a28aca47ba1378dc4'
2025-07-24T04:31:37.4273714Z Submodule path 'third_party/opentelemetry-cpp': checked out 'a799f4aed9c94b765dcdaabaeab7d5e7e2310878'
2025-07-24T04:31:37.4861210Z Submodule path 'third_party/opentelemetry-cpp/third_party/benchmark': checked out 'd572f4777349d43653b21d6c2fc63020ab326db2'
2025-07-24T04:31:37.5453930Z Submodule path 'third_party/opentelemetry-cpp/third_party/googletest': checked out 'b796f7d44681514f58a683a3a71ff17c94edb0c1'
2025-07-24T04:31:37.6033131Z Submodule path 'third_party/opentelemetry-cpp/third_party/ms-gsl': checked out '6f4529395c5b7c2d661812257cd6780c67e54afa'
2025-07-24T04:31:37.6721404Z Submodule path 'third_party/opentelemetry-cpp/third_party/nlohmann-json': checked out 'bc889afb4c5bf1c0d8ee29ef35eaaf4c8bef8a5d'
2025-07-24T04:31:37.7310382Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto': checked out '4ca4f0335c63cda7ab31ea7ed70d6553aee14dce'
2025-07-24T04:31:37.7870962Z Submodule path 'third_party/opentelemetry-cpp/third_party/opentracing-cpp': checked out '06b57f48ded1fa3bdd3d4346f6ef29e40e08eaf5'
2025-07-24T04:31:37.8482750Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp': checked out 'c9ffcdda9086ffd9e1283ea7a0276d831f3c8a8d'
2025-07-24T04:31:37.9107712Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb': checked out 'eefb26f82b233268fc98577d265352720d477ba4'
2025-07-24T04:31:37.9724120Z Submodule path 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest': checked out 'e2239ee6043f73722e7aa812a459f54a28552929'
2025-07-24T04:31:38.0684988Z Submodule path 'third_party/opentelemetry-cpp/tools/vcpkg': checked out '8eb57355a4ffb410a2e94c07b4dca2dffbee8e50'
2025-07-24T04:31:38.1403194Z Submodule path 'third_party/pocketfft': checked out '0fa0ef591e38c2758e3184c6c23e497b9f732ffa'
2025-07-24T04:31:38.2468380Z Submodule path 'third_party/protobuf': checked out 'd1eca4e4b421cd2997495c4b4e65cea6be4e9b8a'
2025-07-24T04:31:38.3055397Z Submodule path 'third_party/protobuf/third_party/benchmark': checked out '5b7683f49e1e9223cf9927b24f6fd3d6bd82e3f8'
2025-07-24T04:31:38.3658797Z Submodule path 'third_party/protobuf/third_party/googletest': checked out '5ec7f0c4a113e2f18ac2c6cc7df51ad6afc24081'
2025-07-24T04:31:38.4282225Z Submodule path 'third_party/psimd': checked out '072586a71b55b7f8c584153d223e95687148a900'
2025-07-24T04:31:38.4895256Z Submodule path 'third_party/pthreadpool': checked out '4fe0e1e183925bf8cfa6aae24237e724a96479b8'
2025-07-24T04:31:38.5589518Z Submodule path 'third_party/pybind11': checked out 'a2e59f0e7065404b44dfe92a28aca47ba1378dc4'
2025-07-24T04:31:38.6210653Z Submodule path 'third_party/python-peachpy': checked out 'f45429b087dd7d5bc78bb40dc7cf06425c252d67'
2025-07-24T04:31:38.6843599Z Submodule path 'third_party/sleef': checked out '5a1d179df9cf652951b59010a2d2075372d67f68'
2025-07-24T04:31:38.7505450Z Submodule path 'third_party/tensorpipe': checked out '52791a2fd214b2a9dc5759d36725909c1daa7f2e'
2025-07-24T04:31:38.8081842Z Submodule path 'third_party/tensorpipe/third_party/googletest': checked out 'aee0f9d9b5b87796ee8a0ab26b7587ec30e8858e'
2025-07-24T04:31:38.8654835Z Submodule path 'third_party/tensorpipe/third_party/libnop': checked out '910b55815be16109f04f4180e9adee14fb4ce281'
2025-07-24T04:31:38.9543344Z Submodule path 'third_party/tensorpipe/third_party/libuv': checked out '1dff88e5161cba5c59276d2070d2e304e4dcb242'
2025-07-24T04:31:39.0166793Z Submodule path 'third_party/tensorpipe/third_party/pybind11': checked out 'a23996fce38ff6ccfbcdc09f1e63f2c4be5ea2ef'
2025-07-24T04:31:39.0711913Z Submodule path 'third_party/tensorpipe/third_party/pybind11/tools/clang': checked out '6a00cbc4a9b8e68b71caf7f774b3f9c753ae84d5'
2025-07-24T04:31:39.0947516Z [command]/usr/bin/git submodule foreach --recursive git config --local gc.auto 0
2025-07-24T04:31:39.1620019Z Entering 'android/libs/fbjni'
2025-07-24T04:31:39.1733100Z Entering 'third_party/FP16'
2025-07-24T04:31:39.1842943Z Entering 'third_party/FXdiv'
2025-07-24T04:31:39.1950608Z Entering 'third_party/NNPACK'
2025-07-24T04:31:39.2058280Z Entering 'third_party/NVTX'
2025-07-24T04:31:39.2168060Z Entering 'third_party/VulkanMemoryAllocator'
2025-07-24T04:31:39.2275926Z Entering 'third_party/XNNPACK'
2025-07-24T04:31:39.2415596Z Entering 'third_party/aiter'
2025-07-24T04:31:39.2522199Z Entering 'third_party/aiter/3rdparty/composable_kernel'
2025-07-24T04:31:39.2646875Z Entering 'third_party/benchmark'
2025-07-24T04:31:39.2759635Z Entering 'third_party/composable_kernel'
2025-07-24T04:31:39.2881246Z Entering 'third_party/cpp-httplib'
2025-07-24T04:31:39.2988998Z Entering 'third_party/cpuinfo'
2025-07-24T04:31:39.3101735Z Entering 'third_party/cudnn_frontend'
2025-07-24T04:31:39.3210548Z Entering 'third_party/cutlass'
2025-07-24T04:31:39.3341288Z Entering 'third_party/fbgemm'
2025-07-24T04:31:39.3451622Z Entering 'third_party/fbgemm/external/asmjit'
2025-07-24T04:31:39.3553120Z Entering 'third_party/fbgemm/external/composable_kernel'
2025-07-24T04:31:39.3667734Z Entering 'third_party/fbgemm/external/cpuinfo'
2025-07-24T04:31:39.3769333Z Entering 'third_party/fbgemm/external/cutlass'
2025-07-24T04:31:39.3887642Z Entering 'third_party/fbgemm/external/googletest'
2025-07-24T04:31:39.3986173Z Entering 'third_party/fbgemm/external/hipify_torch'
2025-07-24T04:31:39.4085763Z Entering 'third_party/fbgemm/external/json'
2025-07-24T04:31:39.4194842Z Entering 'third_party/flash-attention'
2025-07-24T04:31:39.4302808Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-07-24T04:31:39.4416025Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-07-24T04:31:39.4540179Z Entering 'third_party/flatbuffers'
2025-07-24T04:31:39.4652456Z Entering 'third_party/fmt'
2025-07-24T04:31:39.4759602Z Entering 'third_party/gemmlowp/gemmlowp'
2025-07-24T04:31:39.4866798Z Entering 'third_party/gloo'
2025-07-24T04:31:39.4974008Z Entering 'third_party/googletest'
2025-07-24T04:31:39.5080439Z Entering 'third_party/ideep'
2025-07-24T04:31:39.5184583Z Entering 'third_party/ideep/mkl-dnn'
2025-07-24T04:31:39.5304685Z Entering 'third_party/ittapi'
2025-07-24T04:31:39.5412008Z Entering 'third_party/kineto'
2025-07-24T04:31:39.5514023Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-07-24T04:31:39.5611965Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-07-24T04:31:39.5713354Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-07-24T04:31:39.5811538Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-07-24T04:31:39.5909508Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-07-24T04:31:39.6002987Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-07-24T04:31:39.6108371Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-07-24T04:31:39.6208488Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-07-24T04:31:39.6309171Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-07-24T04:31:39.6409827Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-07-24T04:31:39.6514611Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-07-24T04:31:39.6613153Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-07-24T04:31:39.6719770Z Entering 'third_party/kleidiai'
2025-07-24T04:31:39.6827358Z Entering 'third_party/mimalloc'
2025-07-24T04:31:39.6934995Z Entering 'third_party/nlohmann'
2025-07-24T04:31:39.7042951Z Entering 'third_party/onnx'
2025-07-24T04:31:39.7182649Z Entering 'third_party/onnx/third_party/pybind11'
2025-07-24T04:31:39.7295036Z Entering 'third_party/opentelemetry-cpp'
2025-07-24T04:31:39.7401066Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-07-24T04:31:39.7497658Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-07-24T04:31:39.7595019Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-07-24T04:31:39.7693464Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-07-24T04:31:39.7794606Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-07-24T04:31:39.7891827Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-07-24T04:31:39.7988847Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-07-24T04:31:39.8082474Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-07-24T04:31:39.8184061Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-07-24T04:31:39.8285591Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-07-24T04:31:39.8429363Z Entering 'third_party/pocketfft'
2025-07-24T04:31:39.8537006Z Entering 'third_party/protobuf'
2025-07-24T04:31:39.8647187Z Entering 'third_party/protobuf/third_party/benchmark'
2025-07-24T04:31:39.8744663Z Entering 'third_party/protobuf/third_party/googletest'
2025-07-24T04:31:39.8850891Z Entering 'third_party/psimd'
2025-07-24T04:31:39.8959479Z Entering 'third_party/pthreadpool'
2025-07-24T04:31:39.9066851Z Entering 'third_party/pybind11'
2025-07-24T04:31:39.9173651Z Entering 'third_party/python-peachpy'
2025-07-24T04:31:39.9279377Z Entering 'third_party/sleef'
2025-07-24T04:31:39.9385450Z Entering 'third_party/tensorpipe'
2025-07-24T04:31:39.9490288Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-07-24T04:31:39.9588434Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-07-24T04:31:39.9685111Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-07-24T04:31:39.9785900Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-07-24T04:31:39.9878717Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-07-24T04:31:40.0016327Z ##[endgroup]
2025-07-24T04:31:40.0017241Z ##[group]Persisting credentials for submodules
2025-07-24T04:31:40.0028726Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'url\.https\:\/\/github\.com\/\.insteadOf' && git config --local --unset-all 'url.https://github.com/.insteadOf' || :"
2025-07-24T04:31:40.0686201Z Entering 'android/libs/fbjni'
2025-07-24T04:31:40.0752347Z url.https://github.com/.insteadof
2025-07-24T04:31:40.0753018Z url.https://github.com/.insteadof
2025-07-24T04:31:40.0871584Z Entering 'third_party/FP16'
2025-07-24T04:31:40.0936410Z url.https://github.com/.insteadof
2025-07-24T04:31:40.0937083Z url.https://github.com/.insteadof
2025-07-24T04:31:40.1029729Z Entering 'third_party/FXdiv'
2025-07-24T04:31:40.1094242Z url.https://github.com/.insteadof
2025-07-24T04:31:40.1094909Z url.https://github.com/.insteadof
2025-07-24T04:31:40.1190702Z Entering 'third_party/NNPACK'
2025-07-24T04:31:40.1255315Z url.https://github.com/.insteadof
2025-07-24T04:31:40.1255985Z url.https://github.com/.insteadof
2025-07-24T04:31:40.1350212Z Entering 'third_party/NVTX'
2025-07-24T04:31:40.1414338Z url.https://github.com/.insteadof
2025-07-24T04:31:40.1415005Z url.https://github.com/.insteadof
2025-07-24T04:31:40.1509833Z Entering 'third_party/VulkanMemoryAllocator'
2025-07-24T04:31:40.1574588Z url.https://github.com/.insteadof
2025-07-24T04:31:40.1575250Z url.https://github.com/.insteadof
2025-07-24T04:31:40.1668626Z Entering 'third_party/XNNPACK'
2025-07-24T04:31:40.1732166Z url.https://github.com/.insteadof
2025-07-24T04:31:40.1732830Z url.https://github.com/.insteadof
2025-07-24T04:31:40.1859164Z Entering 'third_party/aiter'
2025-07-24T04:31:40.1923763Z url.https://github.com/.insteadof
2025-07-24T04:31:40.1924417Z url.https://github.com/.insteadof
2025-07-24T04:31:40.2014841Z Entering 'third_party/aiter/3rdparty/composable_kernel'
2025-07-24T04:31:40.2074636Z url.https://github.com/.insteadof
2025-07-24T04:31:40.2075309Z url.https://github.com/.insteadof
2025-07-24T04:31:40.2190941Z Entering 'third_party/benchmark'
2025-07-24T04:31:40.2254560Z url.https://github.com/.insteadof
2025-07-24T04:31:40.2255208Z url.https://github.com/.insteadof
2025-07-24T04:31:40.2346991Z Entering 'third_party/composable_kernel'
2025-07-24T04:31:40.2409961Z url.https://github.com/.insteadof
2025-07-24T04:31:40.2410607Z url.https://github.com/.insteadof
2025-07-24T04:31:40.2522155Z Entering 'third_party/cpp-httplib'
2025-07-24T04:31:40.2585244Z url.https://github.com/.insteadof
2025-07-24T04:31:40.2585911Z url.https://github.com/.insteadof
2025-07-24T04:31:40.2675319Z Entering 'third_party/cpuinfo'
2025-07-24T04:31:40.2737789Z url.https://github.com/.insteadof
2025-07-24T04:31:40.2738458Z url.https://github.com/.insteadof
2025-07-24T04:31:40.2831743Z Entering 'third_party/cudnn_frontend'
2025-07-24T04:31:40.2893801Z url.https://github.com/.insteadof
2025-07-24T04:31:40.2894459Z url.https://github.com/.insteadof
2025-07-24T04:31:40.2984478Z Entering 'third_party/cutlass'
2025-07-24T04:31:40.3046476Z url.https://github.com/.insteadof
2025-07-24T04:31:40.3047138Z url.https://github.com/.insteadof
2025-07-24T04:31:40.3158464Z Entering 'third_party/fbgemm'
2025-07-24T04:31:40.3219617Z url.https://github.com/.insteadof
2025-07-24T04:31:40.3220268Z url.https://github.com/.insteadof
2025-07-24T04:31:40.3311591Z Entering 'third_party/fbgemm/external/asmjit'
2025-07-24T04:31:40.3370743Z url.https://github.com/.insteadof
2025-07-24T04:31:40.3371435Z url.https://github.com/.insteadof
2025-07-24T04:31:40.3457855Z Entering 'third_party/fbgemm/external/composable_kernel'
2025-07-24T04:31:40.3521427Z url.https://github.com/.insteadof
2025-07-24T04:31:40.3522116Z url.https://github.com/.insteadof
2025-07-24T04:31:40.3616402Z Entering 'third_party/fbgemm/external/cpuinfo'
2025-07-24T04:31:40.3674728Z url.https://github.com/.insteadof
2025-07-24T04:31:40.3675380Z url.https://github.com/.insteadof
2025-07-24T04:31:40.3762637Z Entering 'third_party/fbgemm/external/cutlass'
2025-07-24T04:31:40.3825525Z url.https://github.com/.insteadof
2025-07-24T04:31:40.3826191Z url.https://github.com/.insteadof
2025-07-24T04:31:40.3927486Z Entering 'third_party/fbgemm/external/googletest'
2025-07-24T04:31:40.3985801Z url.https://github.com/.insteadof
2025-07-24T04:31:40.3986464Z url.https://github.com/.insteadof
2025-07-24T04:31:40.4075161Z Entering 'third_party/fbgemm/external/hipify_torch'
2025-07-24T04:31:40.4135707Z url.https://github.com/.insteadof
2025-07-24T04:31:40.4136909Z url.https://github.com/.insteadof
2025-07-24T04:31:40.4224124Z Entering 'third_party/fbgemm/external/json'
2025-07-24T04:31:40.4285574Z url.https://github.com/.insteadof
2025-07-24T04:31:40.4286257Z url.https://github.com/.insteadof
2025-07-24T04:31:40.4385316Z Entering 'third_party/flash-attention'
2025-07-24T04:31:40.4447625Z url.https://github.com/.insteadof
2025-07-24T04:31:40.4448297Z url.https://github.com/.insteadof
2025-07-24T04:31:40.4537538Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-07-24T04:31:40.4601067Z url.https://github.com/.insteadof
2025-07-24T04:31:40.4601719Z url.https://github.com/.insteadof
2025-07-24T04:31:40.4701962Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-07-24T04:31:40.4761123Z url.https://github.com/.insteadof
2025-07-24T04:31:40.4761782Z url.https://github.com/.insteadof
2025-07-24T04:31:40.4875736Z Entering 'third_party/flatbuffers'
2025-07-24T04:31:40.4939153Z url.https://github.com/.insteadof
2025-07-24T04:31:40.4939851Z url.https://github.com/.insteadof
2025-07-24T04:31:40.5042063Z Entering 'third_party/fmt'
2025-07-24T04:31:40.5102045Z url.https://github.com/.insteadof
2025-07-24T04:31:40.5102744Z url.https://github.com/.insteadof
2025-07-24T04:31:40.5194888Z Entering 'third_party/gemmlowp/gemmlowp'
2025-07-24T04:31:40.5258320Z url.https://github.com/.insteadof
2025-07-24T04:31:40.5258976Z url.https://github.com/.insteadof
2025-07-24T04:31:40.5351843Z Entering 'third_party/gloo'
2025-07-24T04:31:40.5415125Z url.https://github.com/.insteadof
2025-07-24T04:31:40.5415787Z url.https://github.com/.insteadof
2025-07-24T04:31:40.5506678Z Entering 'third_party/googletest'
2025-07-24T04:31:40.5569053Z url.https://github.com/.insteadof
2025-07-24T04:31:40.5569732Z url.https://github.com/.insteadof
2025-07-24T04:31:40.5660788Z Entering 'third_party/ideep'
2025-07-24T04:31:40.5723797Z url.https://github.com/.insteadof
2025-07-24T04:31:40.5724467Z url.https://github.com/.insteadof
2025-07-24T04:31:40.5809641Z Entering 'third_party/ideep/mkl-dnn'
2025-07-24T04:31:40.5867498Z url.https://github.com/.insteadof
2025-07-24T04:31:40.5868161Z url.https://github.com/.insteadof
2025-07-24T04:31:40.5977562Z Entering 'third_party/ittapi'
2025-07-24T04:31:40.6040909Z url.https://github.com/.insteadof
2025-07-24T04:31:40.6041571Z url.https://github.com/.insteadof
2025-07-24T04:31:40.6131971Z Entering 'third_party/kineto'
2025-07-24T04:31:40.6195780Z url.https://github.com/.insteadof
2025-07-24T04:31:40.6196446Z url.https://github.com/.insteadof
2025-07-24T04:31:40.6283528Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-07-24T04:31:40.6342916Z url.https://github.com/.insteadof
2025-07-24T04:31:40.6343599Z url.https://github.com/.insteadof
2025-07-24T04:31:40.6427885Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-07-24T04:31:40.6486550Z url.https://github.com/.insteadof
2025-07-24T04:31:40.6487214Z url.https://github.com/.insteadof
2025-07-24T04:31:40.6578356Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-07-24T04:31:40.6640509Z url.https://github.com/.insteadof
2025-07-24T04:31:40.6641173Z url.https://github.com/.insteadof
2025-07-24T04:31:40.6728917Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-07-24T04:31:40.6786961Z url.https://github.com/.insteadof
2025-07-24T04:31:40.6787636Z url.https://github.com/.insteadof
2025-07-24T04:31:40.6880484Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-07-24T04:31:40.6943394Z url.https://github.com/.insteadof
2025-07-24T04:31:40.6944052Z url.https://github.com/.insteadof
2025-07-24T04:31:40.7032197Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-07-24T04:31:40.7092469Z url.https://github.com/.insteadof
2025-07-24T04:31:40.7093122Z url.https://github.com/.insteadof
2025-07-24T04:31:40.7188573Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-07-24T04:31:40.7246270Z url.https://github.com/.insteadof
2025-07-24T04:31:40.7246916Z url.https://github.com/.insteadof
2025-07-24T04:31:40.7335902Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-07-24T04:31:40.7392927Z url.https://github.com/.insteadof
2025-07-24T04:31:40.7393581Z url.https://github.com/.insteadof
2025-07-24T04:31:40.7482356Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-07-24T04:31:40.7539664Z url.https://github.com/.insteadof
2025-07-24T04:31:40.7540321Z url.https://github.com/.insteadof
2025-07-24T04:31:40.7631147Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-07-24T04:31:40.7694197Z url.https://github.com/.insteadof
2025-07-24T04:31:40.7694859Z url.https://github.com/.insteadof
2025-07-24T04:31:40.7791144Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-07-24T04:31:40.7851512Z url.https://github.com/.insteadof
2025-07-24T04:31:40.7852188Z url.https://github.com/.insteadof
2025-07-24T04:31:40.7941301Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-07-24T04:31:40.8000288Z url.https://github.com/.insteadof
2025-07-24T04:31:40.8000946Z url.https://github.com/.insteadof
2025-07-24T04:31:40.8097347Z Entering 'third_party/kleidiai'
2025-07-24T04:31:40.8161097Z url.https://github.com/.insteadof
2025-07-24T04:31:40.8161765Z url.https://github.com/.insteadof
2025-07-24T04:31:40.8259258Z Entering 'third_party/mimalloc'
2025-07-24T04:31:40.8318566Z url.https://github.com/.insteadof
2025-07-24T04:31:40.8319238Z url.https://github.com/.insteadof
2025-07-24T04:31:40.8412338Z Entering 'third_party/nlohmann'
2025-07-24T04:31:40.8475590Z url.https://github.com/.insteadof
2025-07-24T04:31:40.8476257Z url.https://github.com/.insteadof
2025-07-24T04:31:40.8570345Z Entering 'third_party/onnx'
2025-07-24T04:31:40.8631481Z url.https://github.com/.insteadof
2025-07-24T04:31:40.8632137Z url.https://github.com/.insteadof
2025-07-24T04:31:40.8756143Z Entering 'third_party/onnx/third_party/pybind11'
2025-07-24T04:31:40.8816935Z url.https://github.com/.insteadof
2025-07-24T04:31:40.8817598Z url.https://github.com/.insteadof
2025-07-24T04:31:40.8918155Z Entering 'third_party/opentelemetry-cpp'
2025-07-24T04:31:40.8980162Z url.https://github.com/.insteadof
2025-07-24T04:31:40.8980825Z url.https://github.com/.insteadof
2025-07-24T04:31:40.9073690Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-07-24T04:31:40.9132533Z url.https://github.com/.insteadof
2025-07-24T04:31:40.9133219Z url.https://github.com/.insteadof
2025-07-24T04:31:40.9221407Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-07-24T04:31:40.9279916Z url.https://github.com/.insteadof
2025-07-24T04:31:40.9280567Z url.https://github.com/.insteadof
2025-07-24T04:31:40.9366858Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-07-24T04:31:40.9425396Z url.https://github.com/.insteadof
2025-07-24T04:31:40.9426063Z url.https://github.com/.insteadof
2025-07-24T04:31:40.9512101Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-07-24T04:31:40.9569106Z url.https://github.com/.insteadof
2025-07-24T04:31:40.9569775Z url.https://github.com/.insteadof
2025-07-24T04:31:40.9660395Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-07-24T04:31:40.9716935Z url.https://github.com/.insteadof
2025-07-24T04:31:40.9717604Z url.https://github.com/.insteadof
2025-07-24T04:31:40.9804860Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-07-24T04:31:40.9862991Z url.https://github.com/.insteadof
2025-07-24T04:31:40.9863669Z url.https://github.com/.insteadof
2025-07-24T04:31:40.9949300Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-07-24T04:31:41.0007580Z url.https://github.com/.insteadof
2025-07-24T04:31:41.0008233Z url.https://github.com/.insteadof
2025-07-24T04:31:41.0090873Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-07-24T04:31:41.0148948Z url.https://github.com/.insteadof
2025-07-24T04:31:41.0150178Z url.https://github.com/.insteadof
2025-07-24T04:31:41.0243266Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-07-24T04:31:41.0302922Z url.https://github.com/.insteadof
2025-07-24T04:31:41.0303590Z url.https://github.com/.insteadof
2025-07-24T04:31:41.0402460Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-07-24T04:31:41.0466018Z url.https://github.com/.insteadof
2025-07-24T04:31:41.0466694Z url.https://github.com/.insteadof
2025-07-24T04:31:41.0608757Z Entering 'third_party/pocketfft'
2025-07-24T04:31:41.0671226Z url.https://github.com/.insteadof
2025-07-24T04:31:41.0671897Z url.https://github.com/.insteadof
2025-07-24T04:31:41.0796526Z Entering 'third_party/protobuf'
2025-07-24T04:31:41.0827334Z url.https://github.com/.insteadof
2025-07-24T04:31:41.0827989Z url.https://github.com/.insteadof
2025-07-24T04:31:41.0923718Z Entering 'third_party/protobuf/third_party/benchmark'
2025-07-24T04:31:41.0983196Z url.https://github.com/.insteadof
2025-07-24T04:31:41.0983891Z url.https://github.com/.insteadof
2025-07-24T04:31:41.1070011Z Entering 'third_party/protobuf/third_party/googletest'
2025-07-24T04:31:41.1131215Z url.https://github.com/.insteadof
2025-07-24T04:31:41.1131898Z url.https://github.com/.insteadof
2025-07-24T04:31:41.1227155Z Entering 'third_party/psimd'
2025-07-24T04:31:41.1291194Z url.https://github.com/.insteadof
2025-07-24T04:31:41.1291873Z url.https://github.com/.insteadof
2025-07-24T04:31:41.1381585Z Entering 'third_party/pthreadpool'
2025-07-24T04:31:41.1443590Z url.https://github.com/.insteadof
2025-07-24T04:31:41.1444246Z url.https://github.com/.insteadof
2025-07-24T04:31:41.1533551Z Entering 'third_party/pybind11'
2025-07-24T04:31:41.1595582Z url.https://github.com/.insteadof
2025-07-24T04:31:41.1596252Z url.https://github.com/.insteadof
2025-07-24T04:31:41.1691431Z Entering 'third_party/python-peachpy'
2025-07-24T04:31:41.1759906Z url.https://github.com/.insteadof
2025-07-24T04:31:41.1760566Z url.https://github.com/.insteadof
2025-07-24T04:31:41.1854527Z Entering 'third_party/sleef'
2025-07-24T04:31:41.1921574Z url.https://github.com/.insteadof
2025-07-24T04:31:41.1922261Z url.https://github.com/.insteadof
2025-07-24T04:31:41.2010330Z Entering 'third_party/tensorpipe'
2025-07-24T04:31:41.2072962Z url.https://github.com/.insteadof
2025-07-24T04:31:41.2073635Z url.https://github.com/.insteadof
2025-07-24T04:31:41.2163570Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-07-24T04:31:41.2223649Z url.https://github.com/.insteadof
2025-07-24T04:31:41.2224318Z url.https://github.com/.insteadof
2025-07-24T04:31:41.2312809Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-07-24T04:31:41.2371589Z url.https://github.com/.insteadof
2025-07-24T04:31:41.2372238Z url.https://github.com/.insteadof
2025-07-24T04:31:41.2461022Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-07-24T04:31:41.2519021Z url.https://github.com/.insteadof
2025-07-24T04:31:41.2519673Z url.https://github.com/.insteadof
2025-07-24T04:31:41.2607668Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-07-24T04:31:41.2667032Z url.https://github.com/.insteadof
2025-07-24T04:31:41.2667702Z url.https://github.com/.insteadof
2025-07-24T04:31:41.2752569Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-07-24T04:31:41.2812538Z url.https://github.com/.insteadof
2025-07-24T04:31:41.2813192Z url.https://github.com/.insteadof
2025-07-24T04:31:41.2958200Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local 'http.https://github.com/.extraheader' 'AUTHORIZATION: basic ***' && git config --local --show-origin --name-only --get-regexp remote.origin.url"
2025-07-24T04:31:41.3632990Z Entering 'android/libs/fbjni'
2025-07-24T04:31:41.3735447Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/android/libs/fbjni/config	remote.origin.url
2025-07-24T04:31:41.3794721Z Entering 'third_party/FP16'
2025-07-24T04:31:41.3902340Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FP16/config	remote.origin.url
2025-07-24T04:31:41.3962661Z Entering 'third_party/FXdiv'
2025-07-24T04:31:41.4061612Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/FXdiv/config	remote.origin.url
2025-07-24T04:31:41.4121759Z Entering 'third_party/NNPACK'
2025-07-24T04:31:41.4218779Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK/config	remote.origin.url
2025-07-24T04:31:41.4275727Z Entering 'third_party/NVTX'
2025-07-24T04:31:41.4378516Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NVTX/config	remote.origin.url
2025-07-24T04:31:41.4438829Z Entering 'third_party/VulkanMemoryAllocator'
2025-07-24T04:31:41.4539996Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/VulkanMemoryAllocator/config	remote.origin.url
2025-07-24T04:31:41.4600345Z Entering 'third_party/XNNPACK'
2025-07-24T04:31:41.4697449Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/XNNPACK/config	remote.origin.url
2025-07-24T04:31:41.4789484Z Entering 'third_party/aiter'
2025-07-24T04:31:41.4888603Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/config	remote.origin.url
2025-07-24T04:31:41.4945156Z Entering 'third_party/aiter/3rdparty/composable_kernel'
2025-07-24T04:31:41.5044287Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/aiter/modules/3rdparty/composable_kernel/config	remote.origin.url
2025-07-24T04:31:41.5122407Z Entering 'third_party/benchmark'
2025-07-24T04:31:41.5222364Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/benchmark/config	remote.origin.url
2025-07-24T04:31:41.5279774Z Entering 'third_party/composable_kernel'
2025-07-24T04:31:41.5377453Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/composable_kernel/config	remote.origin.url
2025-07-24T04:31:41.5450831Z Entering 'third_party/cpp-httplib'
2025-07-24T04:31:41.5548886Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpp-httplib/config	remote.origin.url
2025-07-24T04:31:41.5605333Z Entering 'third_party/cpuinfo'
2025-07-24T04:31:41.5704338Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cpuinfo/config	remote.origin.url
2025-07-24T04:31:41.5763143Z Entering 'third_party/cudnn_frontend'
2025-07-24T04:31:41.5865185Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cudnn_frontend/config	remote.origin.url
2025-07-24T04:31:41.5924491Z Entering 'third_party/cutlass'
2025-07-24T04:31:41.6024869Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/cutlass/config	remote.origin.url
2025-07-24T04:31:41.6102665Z Entering 'third_party/fbgemm'
2025-07-24T04:31:41.6202500Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/config	remote.origin.url
2025-07-24T04:31:41.6258878Z Entering 'third_party/fbgemm/external/asmjit'
2025-07-24T04:31:41.6353853Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/asmjit/config	remote.origin.url
2025-07-24T04:31:41.6407041Z Entering 'third_party/fbgemm/external/composable_kernel'
2025-07-24T04:31:41.6509204Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/composable_kernel/config	remote.origin.url
2025-07-24T04:31:41.6576349Z Entering 'third_party/fbgemm/external/cpuinfo'
2025-07-24T04:31:41.6671243Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cpuinfo/config	remote.origin.url
2025-07-24T04:31:41.6722897Z Entering 'third_party/fbgemm/external/cutlass'
2025-07-24T04:31:41.6817755Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/cutlass/config	remote.origin.url
2025-07-24T04:31:41.6890164Z Entering 'third_party/fbgemm/external/googletest'
2025-07-24T04:31:41.6985942Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/googletest/config	remote.origin.url
2025-07-24T04:31:41.7038541Z Entering 'third_party/fbgemm/external/hipify_torch'
2025-07-24T04:31:41.7133561Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/hipify_torch/config	remote.origin.url
2025-07-24T04:31:41.7183907Z Entering 'third_party/fbgemm/external/json'
2025-07-24T04:31:41.7279631Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fbgemm/modules/external/json/config	remote.origin.url
2025-07-24T04:31:41.7340958Z Entering 'third_party/flash-attention'
2025-07-24T04:31:41.7442574Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/config	remote.origin.url
2025-07-24T04:31:41.7497883Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-07-24T04:31:41.7593892Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/composable_kernel/config	remote.origin.url
2025-07-24T04:31:41.7660283Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-07-24T04:31:41.7757154Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flash-attention/modules/csrc/cutlass/config	remote.origin.url
2025-07-24T04:31:41.7840813Z Entering 'third_party/flatbuffers'
2025-07-24T04:31:41.7942811Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/flatbuffers/config	remote.origin.url
2025-07-24T04:31:41.8008878Z Entering 'third_party/fmt'
2025-07-24T04:31:41.8108725Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/fmt/config	remote.origin.url
2025-07-24T04:31:41.8165772Z Entering 'third_party/gemmlowp/gemmlowp'
2025-07-24T04:31:41.8264460Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gemmlowp/gemmlowp/config	remote.origin.url
2025-07-24T04:31:41.8324990Z Entering 'third_party/gloo'
2025-07-24T04:31:41.8422914Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/gloo/config	remote.origin.url
2025-07-24T04:31:41.8484975Z Entering 'third_party/googletest'
2025-07-24T04:31:41.8583523Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/googletest/config	remote.origin.url
2025-07-24T04:31:41.8643535Z Entering 'third_party/ideep'
2025-07-24T04:31:41.8742537Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/config	remote.origin.url
2025-07-24T04:31:41.8794941Z Entering 'third_party/ideep/mkl-dnn'
2025-07-24T04:31:41.8889863Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ideep/modules/mkl-dnn/config	remote.origin.url
2025-07-24T04:31:41.8964911Z Entering 'third_party/ittapi'
2025-07-24T04:31:41.9066693Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/ittapi/config	remote.origin.url
2025-07-24T04:31:41.9127484Z Entering 'third_party/kineto'
2025-07-24T04:31:41.9227258Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/config	remote.origin.url
2025-07-24T04:31:41.9281629Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-07-24T04:31:41.9408354Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/config	remote.origin.url
2025-07-24T04:31:41.9460599Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-07-24T04:31:41.9563750Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/DCGM/config	remote.origin.url
2025-07-24T04:31:41.9619676Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-07-24T04:31:41.9717223Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/cpr/config	remote.origin.url
2025-07-24T04:31:41.9777960Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-07-24T04:31:41.9873465Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/fmt/config	remote.origin.url
2025-07-24T04:31:41.9928247Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-07-24T04:31:42.0024563Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/config	remote.origin.url
2025-07-24T04:31:42.0075445Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-07-24T04:31:42.0174936Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/gflags/modules/doc/config	remote.origin.url
2025-07-24T04:31:42.0236070Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-07-24T04:31:42.0334030Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/glog/config	remote.origin.url
2025-07-24T04:31:42.0387260Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-07-24T04:31:42.0484425Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/googletest/config	remote.origin.url
2025-07-24T04:31:42.0538348Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-07-24T04:31:42.0633833Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/json/config	remote.origin.url
2025-07-24T04:31:42.0690313Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-07-24T04:31:42.0787298Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/dynolog/modules/third_party/pfs/config	remote.origin.url
2025-07-24T04:31:42.0848410Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-07-24T04:31:42.0947063Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/fmt/config	remote.origin.url
2025-07-24T04:31:42.1000344Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-07-24T04:31:42.1098235Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kineto/modules/libkineto/third_party/googletest/config	remote.origin.url
2025-07-24T04:31:42.1162390Z Entering 'third_party/kleidiai'
2025-07-24T04:31:42.1256366Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/kleidiai/config	remote.origin.url
2025-07-24T04:31:42.1311495Z Entering 'third_party/mimalloc'
2025-07-24T04:31:42.1412464Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/mimalloc/config	remote.origin.url
2025-07-24T04:31:42.1470822Z Entering 'third_party/nlohmann'
2025-07-24T04:31:42.1570308Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/nlohmann/config	remote.origin.url
2025-07-24T04:31:42.1630779Z Entering 'third_party/onnx'
2025-07-24T04:31:42.1732806Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/config	remote.origin.url
2025-07-24T04:31:42.1824868Z Entering 'third_party/onnx/third_party/pybind11'
2025-07-24T04:31:42.1925514Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/onnx/modules/third_party/pybind11/config	remote.origin.url
2025-07-24T04:31:42.1992168Z Entering 'third_party/opentelemetry-cpp'
2025-07-24T04:31:42.2095296Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/config	remote.origin.url
2025-07-24T04:31:42.2152560Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-07-24T04:31:42.2252680Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/benchmark/config	remote.origin.url
2025-07-24T04:31:42.2305351Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-07-24T04:31:42.2402872Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/googletest/config	remote.origin.url
2025-07-24T04:31:42.2454187Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-07-24T04:31:42.2548617Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/ms-gsl/config	remote.origin.url
2025-07-24T04:31:42.2600469Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-07-24T04:31:42.2697044Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/nlohmann-json/config	remote.origin.url
2025-07-24T04:31:42.2753130Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-07-24T04:31:42.2850165Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentelemetry-proto/config	remote.origin.url
2025-07-24T04:31:42.2902075Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-07-24T04:31:42.3002587Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/opentracing-cpp/config	remote.origin.url
2025-07-24T04:31:42.3052992Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-07-24T04:31:42.3149901Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/config	remote.origin.url
2025-07-24T04:31:42.3201252Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-07-24T04:31:42.3298244Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/civetweb/config	remote.origin.url
2025-07-24T04:31:42.3355606Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-07-24T04:31:42.3456697Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/third_party/prometheus-cpp/modules/googletest/config	remote.origin.url
2025-07-24T04:31:42.3515090Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-07-24T04:31:42.3610680Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/opentelemetry-cpp/modules/tools/vcpkg/config	remote.origin.url
2025-07-24T04:31:42.3710649Z Entering 'third_party/pocketfft'
2025-07-24T04:31:42.3807574Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pocketfft/config	remote.origin.url
2025-07-24T04:31:42.3862347Z Entering 'third_party/protobuf'
2025-07-24T04:31:42.3959632Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/config	remote.origin.url
2025-07-24T04:31:42.4018510Z Entering 'third_party/protobuf/third_party/benchmark'
2025-07-24T04:31:42.4113003Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/benchmark/config	remote.origin.url
2025-07-24T04:31:42.4165544Z Entering 'third_party/protobuf/third_party/googletest'
2025-07-24T04:31:42.4260011Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/protobuf/modules/third_party/googletest/config	remote.origin.url
2025-07-24T04:31:42.4320691Z Entering 'third_party/psimd'
2025-07-24T04:31:42.4421058Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/psimd/config	remote.origin.url
2025-07-24T04:31:42.4480291Z Entering 'third_party/pthreadpool'
2025-07-24T04:31:42.4575824Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/NNPACK_deps/pthreadpool/config	remote.origin.url
2025-07-24T04:31:42.4631908Z Entering 'third_party/pybind11'
2025-07-24T04:31:42.4731710Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/pybind11/config	remote.origin.url
2025-07-24T04:31:42.4788283Z Entering 'third_party/python-peachpy'
2025-07-24T04:31:42.4885640Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/python-peachpy/config	remote.origin.url
2025-07-24T04:31:42.4941229Z Entering 'third_party/sleef'
2025-07-24T04:31:42.5039117Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/sleef/config	remote.origin.url
2025-07-24T04:31:42.5095439Z Entering 'third_party/tensorpipe'
2025-07-24T04:31:42.5192979Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/config	remote.origin.url
2025-07-24T04:31:42.5244602Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-07-24T04:31:42.5336078Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/googletest/config	remote.origin.url
2025-07-24T04:31:42.5387549Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-07-24T04:31:42.5482060Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libnop/config	remote.origin.url
2025-07-24T04:31:42.5532920Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-07-24T04:31:42.5626686Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/libuv/config	remote.origin.url
2025-07-24T04:31:42.5678065Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-07-24T04:31:42.5772485Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/config	remote.origin.url
2025-07-24T04:31:42.5819489Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-07-24T04:31:42.5911820Z file:/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/.git/modules/third_party/tensorpipe/modules/third_party/pybind11/modules/tools/clang/config	remote.origin.url
2025-07-24T04:31:42.6525813Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'git@github.com:'
2025-07-24T04:31:42.7203261Z Entering 'android/libs/fbjni'
2025-07-24T04:31:42.7308426Z Entering 'third_party/FP16'
2025-07-24T04:31:42.7416885Z Entering 'third_party/FXdiv'
2025-07-24T04:31:42.7525907Z Entering 'third_party/NNPACK'
2025-07-24T04:31:42.7633415Z Entering 'third_party/NVTX'
2025-07-24T04:31:42.7745072Z Entering 'third_party/VulkanMemoryAllocator'
2025-07-24T04:31:42.7854638Z Entering 'third_party/XNNPACK'
2025-07-24T04:31:42.7994907Z Entering 'third_party/aiter'
2025-07-24T04:31:42.8102599Z Entering 'third_party/aiter/3rdparty/composable_kernel'
2025-07-24T04:31:42.8226483Z Entering 'third_party/benchmark'
2025-07-24T04:31:42.8335324Z Entering 'third_party/composable_kernel'
2025-07-24T04:31:42.8459266Z Entering 'third_party/cpp-httplib'
2025-07-24T04:31:42.8565949Z Entering 'third_party/cpuinfo'
2025-07-24T04:31:42.8673845Z Entering 'third_party/cudnn_frontend'
2025-07-24T04:31:42.8780115Z Entering 'third_party/cutlass'
2025-07-24T04:31:42.8907498Z Entering 'third_party/fbgemm'
2025-07-24T04:31:42.9015583Z Entering 'third_party/fbgemm/external/asmjit'
2025-07-24T04:31:42.9116113Z Entering 'third_party/fbgemm/external/composable_kernel'
2025-07-24T04:31:42.9229433Z Entering 'third_party/fbgemm/external/cpuinfo'
2025-07-24T04:31:42.9332896Z Entering 'third_party/fbgemm/external/cutlass'
2025-07-24T04:31:42.9450505Z Entering 'third_party/fbgemm/external/googletest'
2025-07-24T04:31:42.9550959Z Entering 'third_party/fbgemm/external/hipify_torch'
2025-07-24T04:31:42.9648854Z Entering 'third_party/fbgemm/external/json'
2025-07-24T04:31:42.9759258Z Entering 'third_party/flash-attention'
2025-07-24T04:31:42.9864841Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-07-24T04:31:42.9974882Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-07-24T04:31:43.0098653Z Entering 'third_party/flatbuffers'
2025-07-24T04:31:43.0210858Z Entering 'third_party/fmt'
2025-07-24T04:31:43.0317502Z Entering 'third_party/gemmlowp/gemmlowp'
2025-07-24T04:31:43.0423802Z Entering 'third_party/gloo'
2025-07-24T04:31:43.0529694Z Entering 'third_party/googletest'
2025-07-24T04:31:43.0635186Z Entering 'third_party/ideep'
2025-07-24T04:31:43.0739725Z Entering 'third_party/ideep/mkl-dnn'
2025-07-24T04:31:43.0861375Z Entering 'third_party/ittapi'
2025-07-24T04:31:43.0967255Z Entering 'third_party/kineto'
2025-07-24T04:31:43.1070185Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-07-24T04:31:43.1166959Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-07-24T04:31:43.1268950Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-07-24T04:31:43.1368392Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-07-24T04:31:43.1468584Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-07-24T04:31:43.1562741Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-07-24T04:31:43.1670000Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-07-24T04:31:43.1767929Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-07-24T04:31:43.1865126Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-07-24T04:31:43.1964693Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-07-24T04:31:43.2068894Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-07-24T04:31:43.2167327Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-07-24T04:31:43.2275973Z Entering 'third_party/kleidiai'
2025-07-24T04:31:43.2383558Z Entering 'third_party/mimalloc'
2025-07-24T04:31:43.2489384Z Entering 'third_party/nlohmann'
2025-07-24T04:31:43.2597655Z Entering 'third_party/onnx'
2025-07-24T04:31:43.2736975Z Entering 'third_party/onnx/third_party/pybind11'
2025-07-24T04:31:43.2846808Z Entering 'third_party/opentelemetry-cpp'
2025-07-24T04:31:43.2951508Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-07-24T04:31:43.3048439Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-07-24T04:31:43.3144878Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-07-24T04:31:43.3239093Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-07-24T04:31:43.3338185Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-07-24T04:31:43.3435336Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-07-24T04:31:43.3537950Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-07-24T04:31:43.3631015Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-07-24T04:31:43.3731555Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-07-24T04:31:43.3836793Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-07-24T04:31:43.3980432Z Entering 'third_party/pocketfft'
2025-07-24T04:31:43.4086035Z Entering 'third_party/protobuf'
2025-07-24T04:31:43.4193801Z Entering 'third_party/protobuf/third_party/benchmark'
2025-07-24T04:31:43.4290661Z Entering 'third_party/protobuf/third_party/googletest'
2025-07-24T04:31:43.4401599Z Entering 'third_party/psimd'
2025-07-24T04:31:43.4502193Z Entering 'third_party/pthreadpool'
2025-07-24T04:31:43.4608727Z Entering 'third_party/pybind11'
2025-07-24T04:31:43.4714675Z Entering 'third_party/python-peachpy'
2025-07-24T04:31:43.4821693Z Entering 'third_party/sleef'
2025-07-24T04:31:43.4926975Z Entering 'third_party/tensorpipe'
2025-07-24T04:31:43.5027465Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-07-24T04:31:43.5126173Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-07-24T04:31:43.5222951Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-07-24T04:31:43.5321846Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-07-24T04:31:43.5414335Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-07-24T04:31:43.5564076Z [command]/usr/bin/git submodule foreach --recursive git config --local --add 'url.https://github.com/.insteadOf' 'org-21003710@github.com:'
2025-07-24T04:31:43.6246413Z Entering 'android/libs/fbjni'
2025-07-24T04:31:43.6361718Z Entering 'third_party/FP16'
2025-07-24T04:31:43.6472242Z Entering 'third_party/FXdiv'
2025-07-24T04:31:43.6581561Z Entering 'third_party/NNPACK'
2025-07-24T04:31:43.6693717Z Entering 'third_party/NVTX'
2025-07-24T04:31:43.6808438Z Entering 'third_party/VulkanMemoryAllocator'
2025-07-24T04:31:43.6919298Z Entering 'third_party/XNNPACK'
2025-07-24T04:31:43.7061777Z Entering 'third_party/aiter'
2025-07-24T04:31:43.7173104Z Entering 'third_party/aiter/3rdparty/composable_kernel'
2025-07-24T04:31:43.7299412Z Entering 'third_party/benchmark'
2025-07-24T04:31:43.7411743Z Entering 'third_party/composable_kernel'
2025-07-24T04:31:43.7540422Z Entering 'third_party/cpp-httplib'
2025-07-24T04:31:43.7653515Z Entering 'third_party/cpuinfo'
2025-07-24T04:31:43.7767379Z Entering 'third_party/cudnn_frontend'
2025-07-24T04:31:43.7877493Z Entering 'third_party/cutlass'
2025-07-24T04:31:43.8009308Z Entering 'third_party/fbgemm'
2025-07-24T04:31:43.8119474Z Entering 'third_party/fbgemm/external/asmjit'
2025-07-24T04:31:43.8224947Z Entering 'third_party/fbgemm/external/composable_kernel'
2025-07-24T04:31:43.8341525Z Entering 'third_party/fbgemm/external/cpuinfo'
2025-07-24T04:31:43.8445725Z Entering 'third_party/fbgemm/external/cutlass'
2025-07-24T04:31:43.8565408Z Entering 'third_party/fbgemm/external/googletest'
2025-07-24T04:31:43.8667406Z Entering 'third_party/fbgemm/external/hipify_torch'
2025-07-24T04:31:43.8767937Z Entering 'third_party/fbgemm/external/json'
2025-07-24T04:31:43.8879530Z Entering 'third_party/flash-attention'
2025-07-24T04:31:43.8989005Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-07-24T04:31:43.9107612Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-07-24T04:31:43.9235663Z Entering 'third_party/flatbuffers'
2025-07-24T04:31:43.9354661Z Entering 'third_party/fmt'
2025-07-24T04:31:43.9464456Z Entering 'third_party/gemmlowp/gemmlowp'
2025-07-24T04:31:43.9574382Z Entering 'third_party/gloo'
2025-07-24T04:31:43.9681721Z Entering 'third_party/googletest'
2025-07-24T04:31:43.9790601Z Entering 'third_party/ideep'
2025-07-24T04:31:43.9897411Z Entering 'third_party/ideep/mkl-dnn'
2025-07-24T04:31:44.0020876Z Entering 'third_party/ittapi'
2025-07-24T04:31:44.0128031Z Entering 'third_party/kineto'
2025-07-24T04:31:44.0236493Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-07-24T04:31:44.0336148Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-07-24T04:31:44.0443552Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-07-24T04:31:44.0548132Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-07-24T04:31:44.0651064Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-07-24T04:31:44.0750677Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-07-24T04:31:44.0862108Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-07-24T04:31:44.0965864Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-07-24T04:31:44.1068047Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-07-24T04:31:44.1171733Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-07-24T04:31:44.1279634Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-07-24T04:31:44.1381253Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-07-24T04:31:44.1487007Z Entering 'third_party/kleidiai'
2025-07-24T04:31:44.1595120Z Entering 'third_party/mimalloc'
2025-07-24T04:31:44.1704251Z Entering 'third_party/nlohmann'
2025-07-24T04:31:44.1814497Z Entering 'third_party/onnx'
2025-07-24T04:31:44.1955197Z Entering 'third_party/onnx/third_party/pybind11'
2025-07-24T04:31:44.2071157Z Entering 'third_party/opentelemetry-cpp'
2025-07-24T04:31:44.2179776Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-07-24T04:31:44.2278272Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-07-24T04:31:44.2379450Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-07-24T04:31:44.2478712Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-07-24T04:31:44.2581823Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-07-24T04:31:44.2679377Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-07-24T04:31:44.2777118Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-07-24T04:31:44.2870244Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-07-24T04:31:44.2971553Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-07-24T04:31:44.3082627Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-07-24T04:31:44.3227309Z Entering 'third_party/pocketfft'
2025-07-24T04:31:44.3336141Z Entering 'third_party/protobuf'
2025-07-24T04:31:44.3447099Z Entering 'third_party/protobuf/third_party/benchmark'
2025-07-24T04:31:44.3545651Z Entering 'third_party/protobuf/third_party/googletest'
2025-07-24T04:31:44.3653052Z Entering 'third_party/psimd'
2025-07-24T04:31:44.3761059Z Entering 'third_party/pthreadpool'
2025-07-24T04:31:44.3867118Z Entering 'third_party/pybind11'
2025-07-24T04:31:44.3977797Z Entering 'third_party/python-peachpy'
2025-07-24T04:31:44.4091823Z Entering 'third_party/sleef'
2025-07-24T04:31:44.4202883Z Entering 'third_party/tensorpipe'
2025-07-24T04:31:44.4310496Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-07-24T04:31:44.4412485Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-07-24T04:31:44.4513229Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-07-24T04:31:44.4618533Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-07-24T04:31:44.4715537Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-07-24T04:31:44.4863461Z ##[endgroup]
2025-07-24T04:31:44.4962846Z [command]/usr/bin/git log -1 --format=%H
2025-07-24T04:31:44.5030255Z 27c8ef1839e0747b36769efb192befc9308ff93c
2025-07-24T04:31:44.5430631Z Prepare all required actions
2025-07-24T04:31:44.5431565Z Getting action download info
2025-07-24T04:31:44.6824066Z ##[group]Run ./.github/actions/setup-rocm
2025-07-24T04:31:44.6824622Z env:
2025-07-24T04:31:44.6824988Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:44.6825430Z ##[endgroup]
2025-07-24T04:31:44.6868145Z ##[group]Run dpkg -l | grep -E "  rocm"
2025-07-24T04:31:44.6868686Z dpkg -l | grep -E "  rocm"
2025-07-24T04:31:44.6926089Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:44.6926660Z env:
2025-07-24T04:31:44.6926993Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:44.6927373Z ##[endgroup]
2025-07-24T04:31:44.7365916Z ii  rocm                                   6.3.3.60303-74~22.04                    amd64        Radeon Open Compute (ROCm) software stack meta package
2025-07-24T04:31:44.7367199Z ii  rocm-cmake                             0.14.0.60303-74~22.04                   amd64        rocm-cmake built using CMake
2025-07-24T04:31:44.7368421Z ii  rocm-core                              6.3.3.60303-74~22.04                    amd64        Radeon Open Compute (ROCm) Runtime software stack
2025-07-24T04:31:44.7370210Z ii  rocm-dbgapi                            0.77.0.60303-74~22.04                   amd64        Library to provide AMD GPU debugger API
2025-07-24T04:31:44.7371451Z ii  rocm-debug-agent                       2.0.3.60303-74~22.04                    amd64        Radeon Open Compute Debug Agent (ROCdebug-agent)
2025-07-24T04:31:44.7372792Z ii  rocm-developer-tools                   6.3.3.60303-74~22.04                    amd64        Radeon Open Compute (ROCm) Runtime software stack
2025-07-24T04:31:44.7374051Z ii  rocm-device-libs                       1.0.0.60303-74~22.04                    amd64        Radeon Open Compute - device libraries
2025-07-24T04:31:44.7375266Z ii  rocm-gdb                               15.2.60303-74~22.04                     amd64        ROCgdb
2025-07-24T04:31:44.7376365Z ii  rocm-hip-libraries                     6.3.3.60303-74~22.04                    amd64        Radeon Open Compute (ROCm) Runtime software stack
2025-07-24T04:31:44.7377658Z ii  rocm-hip-runtime                       6.3.3.60303-74~22.04                    amd64        Radeon Open Compute (ROCm) Runtime software stack
2025-07-24T04:31:44.7378920Z ii  rocm-hip-runtime-dev                   6.3.3.60303-74~22.04                    amd64        Radeon Open Compute (ROCm) Runtime software stack
2025-07-24T04:31:44.7380129Z ii  rocm-hip-sdk                           6.3.3.60303-74~22.04                    amd64        Radeon Open Compute (ROCm) Runtime software stack
2025-07-24T04:31:44.7381371Z ii  rocm-language-runtime                  6.3.3.60303-74~22.04                    amd64        Radeon Open Compute (ROCm) Runtime software stack
2025-07-24T04:31:44.7382545Z ii  rocm-llvm                              18.0.0.25012.60303-74~22.04             amd64        ROCm core compiler
2025-07-24T04:31:44.7383679Z ii  rocm-ml-libraries                      6.3.3.60303-74~22.04                    amd64        Radeon Open Compute (ROCm) Runtime software stack
2025-07-24T04:31:44.7384906Z ii  rocm-ml-sdk                            6.3.3.60303-74~22.04                    amd64        Radeon Open Compute (ROCm) Runtime software stack
2025-07-24T04:31:44.7386034Z ii  rocm-opencl                            2.0.0.60303-74~22.04                    amd64        clr built using CMake
2025-07-24T04:31:44.7387063Z ii  rocm-opencl-dev                        2.0.0.60303-74~22.04                    amd64        clr built using CMake
2025-07-24T04:31:44.7388271Z ii  rocm-opencl-runtime                    6.3.3.60303-74~22.04                    amd64        Radeon Open Compute (ROCm) Runtime software stack
2025-07-24T04:31:44.7389538Z ii  rocm-opencl-sdk                        6.3.3.60303-74~22.04                    amd64        Radeon Open Compute (ROCm) Runtime software stack
2025-07-24T04:31:44.7391340Z ii  rocm-openmp-sdk                        6.3.3.60303-74~22.04                    amd64        Radeon Open Compute (ROCm) OpenMP Software development Kit.
2025-07-24T04:31:44.7392695Z ii  rocm-smi-lib                           7.4.0.60303-74~22.04                    amd64        AMD System Management libraries
2025-07-24T04:31:44.7393850Z ii  rocm-utils                             6.3.3.60303-74~22.04                    amd64        Radeon Open Compute (ROCm) Runtime software stack
2025-07-24T04:31:44.7395228Z ii  rocminfo                               1.0.0.60303-74~22.04                    amd64        Radeon Open Compute (ROCm) Runtime rocminfo tool
2025-07-24T04:31:44.7502926Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty
2025-07-24T04:31:44.7503974Z # ignore expansion of "docker ps -q" since it could be empty
2025-07-24T04:31:44.7504769Z # shellcheck disable=SC2046
2025-07-24T04:31:44.7505373Z docker stop $(docker ps -q) || true
2025-07-24T04:31:44.7506026Z # Prune all stopped containers.
2025-07-24T04:31:44.7507043Z docker container prune -f
2025-07-24T04:31:44.7577932Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:44.7578653Z env:
2025-07-24T04:31:44.7579052Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:44.7579538Z ##[endgroup]
2025-07-24T04:31:44.8319910Z "docker stop" requires at least 1 argument.
2025-07-24T04:31:44.8320652Z See 'docker stop --help'.
2025-07-24T04:31:44.8320985Z 
2025-07-24T04:31:44.8321297Z Usage:  docker stop [OPTIONS] CONTAINER [CONTAINER...]
2025-07-24T04:31:44.8321809Z 
2025-07-24T04:31:44.8322028Z Stop one or more running containers
2025-07-24T04:31:44.8677923Z Total reclaimed space: 0B
2025-07-24T04:31:44.8779482Z ##[group]Run cat /etc/os-release || true
2025-07-24T04:31:44.8780139Z cat /etc/os-release || true
2025-07-24T04:31:44.8780769Z cat /etc/apt/sources.list.d/rocm.list || true
2025-07-24T04:31:44.8781405Z cat /opt/rocm/.info/version || true
2025-07-24T04:31:44.8781940Z whoami
2025-07-24T04:31:44.8848800Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:44.8849501Z env:
2025-07-24T04:31:44.8849905Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:44.8850346Z ##[endgroup]
2025-07-24T04:31:44.8976662Z PRETTY_NAME="Ubuntu 22.04.4 LTS"
2025-07-24T04:31:44.8977287Z NAME="Ubuntu"
2025-07-24T04:31:44.8977670Z VERSION_ID="22.04"
2025-07-24T04:31:44.8978116Z VERSION="22.04.4 LTS (Jammy Jellyfish)"
2025-07-24T04:31:44.8978642Z VERSION_CODENAME=jammy
2025-07-24T04:31:44.8979060Z ID=ubuntu
2025-07-24T04:31:44.8979403Z ID_LIKE=debian
2025-07-24T04:31:44.8979889Z HOME_URL="https://www.ubuntu.com/"
2025-07-24T04:31:44.8980489Z SUPPORT_URL="https://help.ubuntu.com/"
2025-07-24T04:31:44.8981172Z BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
2025-07-24T04:31:44.8982100Z PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
2025-07-24T04:31:44.8982942Z UBUNTU_CODENAME=jammy
2025-07-24T04:31:44.9005524Z deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/6.3.3 jammy main
2025-07-24T04:31:44.9033470Z 6.3.3-74
2025-07-24T04:31:44.9077993Z pytorchci
2025-07-24T04:31:44.9126791Z ##[group]Run dpkg -l | grep -E "  amdgpu"
2025-07-24T04:31:44.9127451Z dpkg -l | grep -E "  amdgpu"
2025-07-24T04:31:44.9193964Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:44.9195019Z env:
2025-07-24T04:31:44.9195407Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:44.9195854Z ##[endgroup]
2025-07-24T04:31:44.9628956Z ii  amdgpu-core                            1:6.3.60303-2119913.22.04               all          Core meta package for unified amdgpu driver.
2025-07-24T04:31:44.9630071Z ii  amdgpu-dkms                            1:6.10.5.60303-2119913.22.04            all          amdgpu driver in DKMS format.
2025-07-24T04:31:44.9631182Z ii  amdgpu-dkms-firmware                   1:6.10.5.60303-2119913.22.04            all          firmware blobs used by amdgpu driver in DKMS format
2025-07-24T04:31:44.9632788Z ii  amdgpu-install                         6.3.60303-2119913.22.04                 all          AMDGPU driver repository and installer
2025-07-24T04:31:44.9680340Z ##[group]Run rocm-smi
2025-07-24T04:31:44.9680796Z rocm-smi
2025-07-24T04:31:44.9747848Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:44.9748494Z env:
2025-07-24T04:31:44.9748868Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:44.9749320Z ##[endgroup]
2025-07-24T04:31:45.1724574Z 
2025-07-24T04:31:45.1724651Z 
2025-07-24T04:31:45.1725249Z ========================================= ROCm System Management Interface =========================================
2025-07-24T04:31:45.1726246Z =================================================== Concise Info ===================================================
2025-07-24T04:31:45.1727253Z Device  Node  IDs              Temp    Power   Partitions          SCLK    MCLK     Fan  Perf  PwrCap  VRAM%  GPU%  
2025-07-24T04:31:45.1728870Z               (DID,     GUID)  (Edge)  (Avg)   (Mem, Compute, ID)                                                   
2025-07-24T04:31:45.1730386Z ====================================================================================================================
2025-07-24T04:31:45.1731801Z 0       4     0x740c,   57586  43.0°C  93.0W   N/A, N/A, 0         800Mhz  1600Mhz  0%   auto  560.0W  0%     0%    
2025-07-24T04:31:45.1733015Z 1       5     0x740c,   45873  45.0°C  N/A     N/A, N/A, 0         800Mhz  1600Mhz  0%   auto  0.0W    0%     0%    
2025-07-24T04:31:45.1734185Z 2       2     0x740c,   51627  40.0°C  102.0W  N/A, N/A, 0         800Mhz  1600Mhz  0%   auto  560.0W  0%     0%    
2025-07-24T04:31:45.1735481Z 3       3     0x740c,   64489  33.0°C  N/A     N/A, N/A, 0         800Mhz  1600Mhz  0%   auto  0.0W    0%     0%    
2025-07-24T04:31:45.1736779Z 4       8     0x740c,   30939  37.0°C  99.0W   N/A, N/A, 0         800Mhz  1600Mhz  0%   auto  560.0W  0%     0%    
2025-07-24T04:31:45.1737908Z 5       9     0x740c,   8466   38.0°C  N/A     N/A, N/A, 0         800Mhz  1600Mhz  0%   auto  0.0W    0%     0%    
2025-07-24T04:31:45.1738983Z 6       6     0x740c,   41154  36.0°C  93.0W   N/A, N/A, 0         800Mhz  1600Mhz  0%   auto  560.0W  0%     0%    
2025-07-24T04:31:45.1740082Z 7       7     0x740c,   63755  36.0°C  N/A     N/A, N/A, 0         800Mhz  1600Mhz  0%   auto  0.0W    0%     0%    
2025-07-24T04:31:45.1741499Z ====================================================================================================================
2025-07-24T04:31:45.1742371Z =============================================== End of ROCm SMI Log ================================================
2025-07-24T04:31:45.1895773Z ##[group]Run rocminfo
2025-07-24T04:31:45.1896229Z rocminfo
2025-07-24T04:31:45.1959598Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:45.1960236Z env:
2025-07-24T04:31:45.1960602Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:45.1961030Z ##[endgroup]
2025-07-24T04:31:45.3434804Z ROCk module version 6.10.5 is loaded
2025-07-24T04:31:45.3435852Z =====================    
2025-07-24T04:31:45.3436404Z HSA System Attributes    
2025-07-24T04:31:45.3436957Z =====================    
2025-07-24T04:31:45.3437457Z Runtime Version:         1.14
2025-07-24T04:31:45.3437930Z Runtime Ext Version:     1.6
2025-07-24T04:31:45.3438428Z System Timestamp Freq.:  1000.000000MHz
2025-07-24T04:31:45.3439374Z Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
2025-07-24T04:31:45.3440477Z Machine Model:           LARGE                              
2025-07-24T04:31:45.3441282Z System Endianness:       LITTLE                             
2025-07-24T04:31:45.3441904Z Mwaitx:                  DISABLED
2025-07-24T04:31:45.3442402Z DMAbuf Support:          YES
2025-07-24T04:31:45.3442689Z 
2025-07-24T04:31:45.3442853Z ==========               
2025-07-24T04:31:45.3443273Z HSA Agents               
2025-07-24T04:31:45.3443671Z ==========               
2025-07-24T04:31:45.3444654Z *******                  
2025-07-24T04:31:45.3445098Z Agent 1                  
2025-07-24T04:31:45.3445486Z *******                  
2025-07-24T04:31:45.3445992Z   Name:                    AMD EPYC 7713 64-Core Processor    
2025-07-24T04:31:45.3446674Z   Uuid:                    CPU-XX                             
2025-07-24T04:31:45.3447355Z   Marketing Name:          AMD EPYC 7713 64-Core Processor    
2025-07-24T04:31:45.3448039Z   Vendor Name:             CPU                                
2025-07-24T04:31:45.3448714Z   Feature:                 None specified                     
2025-07-24T04:31:45.3449369Z   Profile:                 FULL_PROFILE                       
2025-07-24T04:31:45.3450017Z   Float Round Mode:        NEAR                               
2025-07-24T04:31:45.3450699Z   Max Queue Number:        0(0x0)                             
2025-07-24T04:31:45.3451345Z   Queue Min Size:          0(0x0)                             
2025-07-24T04:31:45.3452011Z   Queue Max Size:          0(0x0)                             
2025-07-24T04:31:45.3453029Z   Queue Type:              MULTI                              
2025-07-24T04:31:45.3453636Z   Node:                    0                                  
2025-07-24T04:31:45.3454237Z   Device Type:             CPU                                
2025-07-24T04:31:45.3454830Z   Cache Info:              
2025-07-24T04:31:45.3455351Z     L1:                      32768(0x8000) KB                   
2025-07-24T04:31:45.3456033Z   Chip ID:                 0(0x0)                             
2025-07-24T04:31:45.3456770Z   ASIC Revision:           0(0x0)                             
2025-07-24T04:31:45.3457439Z   Cacheline Size:          64(0x40)                           
2025-07-24T04:31:45.3458101Z   Max Clock Freq. (MHz):   2000                               
2025-07-24T04:31:45.3458709Z   BDFID:                   0                                  
2025-07-24T04:31:45.3459335Z   Internal Node ID:        0                                  
2025-07-24T04:31:45.3460011Z   Compute Unit:            64                                 
2025-07-24T04:31:45.3460641Z   SIMDs per CU:            0                                  
2025-07-24T04:31:45.3461291Z   Shader Engines:          0                                  
2025-07-24T04:31:45.3461949Z   Shader Arrs. per Eng.:   0                                  
2025-07-24T04:31:45.3462640Z   WatchPts on Addr. Ranges:1                                  
2025-07-24T04:31:45.3463246Z   Memory Properties:       
2025-07-24T04:31:45.3463725Z   Features:                None
2025-07-24T04:31:45.3464181Z   Pool Info:               
2025-07-24T04:31:45.3464608Z     Pool 1                   
2025-07-24T04:31:45.3465159Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:31:45.3465815Z       Size:                    528249788(0x1f7c73bc) KB           
2025-07-24T04:31:45.3466473Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3467148Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3467896Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:31:45.3468608Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3469297Z       Accessible by all:       TRUE                               
2025-07-24T04:31:45.3469889Z     Pool 2                   
2025-07-24T04:31:45.3470420Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:31:45.3471066Z       Size:                    528249788(0x1f7c73bc) KB           
2025-07-24T04:31:45.3471684Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3472359Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3473045Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:31:45.3473748Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3474882Z       Accessible by all:       TRUE                               
2025-07-24T04:31:45.3475547Z     Pool 3                   
2025-07-24T04:31:45.3476085Z       Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
2025-07-24T04:31:45.3476709Z       Size:                    528249788(0x1f7c73bc) KB           
2025-07-24T04:31:45.3477358Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3478031Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3478742Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:31:45.3479456Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3480135Z       Accessible by all:       TRUE                               
2025-07-24T04:31:45.3480737Z     Pool 4                   
2025-07-24T04:31:45.3481256Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:31:45.3481901Z       Size:                    528249788(0x1f7c73bc) KB           
2025-07-24T04:31:45.3482847Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3483506Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3484199Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:31:45.3484907Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3485608Z       Accessible by all:       TRUE                               
2025-07-24T04:31:45.3486185Z   ISA Info:                
2025-07-24T04:31:45.3486607Z *******                  
2025-07-24T04:31:45.3487010Z Agent 2                  
2025-07-24T04:31:45.3487402Z *******                  
2025-07-24T04:31:45.3487884Z   Name:                    AMD EPYC 7713 64-Core Processor    
2025-07-24T04:31:45.3488533Z   Uuid:                    CPU-XX                             
2025-07-24T04:31:45.3489182Z   Marketing Name:          AMD EPYC 7713 64-Core Processor    
2025-07-24T04:31:45.3489881Z   Vendor Name:             CPU                                
2025-07-24T04:31:45.3490557Z   Feature:                 None specified                     
2025-07-24T04:31:45.3491196Z   Profile:                 FULL_PROFILE                       
2025-07-24T04:31:45.3491865Z   Float Round Mode:        NEAR                               
2025-07-24T04:31:45.3492537Z   Max Queue Number:        0(0x0)                             
2025-07-24T04:31:45.3493199Z   Queue Min Size:          0(0x0)                             
2025-07-24T04:31:45.3493843Z   Queue Max Size:          0(0x0)                             
2025-07-24T04:31:45.3494492Z   Queue Type:              MULTI                              
2025-07-24T04:31:45.3495106Z   Node:                    1                                  
2025-07-24T04:31:45.3495695Z   Device Type:             CPU                                
2025-07-24T04:31:45.3496283Z   Cache Info:              
2025-07-24T04:31:45.3496753Z     L1:                      32768(0x8000) KB                   
2025-07-24T04:31:45.3497377Z   Chip ID:                 0(0x0)                             
2025-07-24T04:31:45.3497994Z   ASIC Revision:           0(0x0)                             
2025-07-24T04:31:45.3498666Z   Cacheline Size:          64(0x40)                           
2025-07-24T04:31:45.3499334Z   Max Clock Freq. (MHz):   2000                               
2025-07-24T04:31:45.3499949Z   BDFID:                   0                                  
2025-07-24T04:31:45.3500569Z   Internal Node ID:        1                                  
2025-07-24T04:31:45.3501221Z   Compute Unit:            64                                 
2025-07-24T04:31:45.3501851Z   SIMDs per CU:            0                                  
2025-07-24T04:31:45.3502487Z   Shader Engines:          0                                  
2025-07-24T04:31:45.3503161Z   Shader Arrs. per Eng.:   0                                  
2025-07-24T04:31:45.3503851Z   WatchPts on Addr. Ranges:1                                  
2025-07-24T04:31:45.3504748Z   Memory Properties:       
2025-07-24T04:31:45.3505220Z   Features:                None
2025-07-24T04:31:45.3505648Z   Pool Info:               
2025-07-24T04:31:45.3506091Z     Pool 1                   
2025-07-24T04:31:45.3506624Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:31:45.3507287Z       Size:                    528402452(0x1f7ec814) KB           
2025-07-24T04:31:45.3507923Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3508596Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3509303Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:31:45.3510003Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3510690Z       Accessible by all:       TRUE                               
2025-07-24T04:31:45.3511270Z     Pool 2                   
2025-07-24T04:31:45.3511817Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:31:45.3512726Z       Size:                    528402452(0x1f7ec814) KB           
2025-07-24T04:31:45.3513377Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3514088Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3515198Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:31:45.3515932Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3516635Z       Accessible by all:       TRUE                               
2025-07-24T04:31:45.3517230Z     Pool 3                   
2025-07-24T04:31:45.3517788Z       Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
2025-07-24T04:31:45.3518430Z       Size:                    528402452(0x1f7ec814) KB           
2025-07-24T04:31:45.3519076Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3519898Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3520610Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:31:45.3521324Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3522002Z       Accessible by all:       TRUE                               
2025-07-24T04:31:45.3522603Z     Pool 4                   
2025-07-24T04:31:45.3523136Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:31:45.3523770Z       Size:                    528402452(0x1f7ec814) KB           
2025-07-24T04:31:45.3524407Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3525073Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3525789Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:31:45.3526483Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3527186Z       Accessible by all:       TRUE                               
2025-07-24T04:31:45.3527788Z   ISA Info:                
2025-07-24T04:31:45.3528226Z *******                  
2025-07-24T04:31:45.3528637Z Agent 3                  
2025-07-24T04:31:45.3529035Z *******                  
2025-07-24T04:31:45.3529519Z   Name:                    gfx90a                             
2025-07-24T04:31:45.3530134Z   Uuid:                    GPU-46d3423fd94a977d               
2025-07-24T04:31:45.3530812Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:31:45.3531482Z   Vendor Name:             AMD                                
2025-07-24T04:31:45.3532136Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:31:45.3532772Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:31:45.3533430Z   Float Round Mode:        NEAR                               
2025-07-24T04:31:45.3534105Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:31:45.3535099Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:31:45.3535772Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:31:45.3536407Z   Queue Type:              MULTI                              
2025-07-24T04:31:45.3537014Z   Node:                    2                                  
2025-07-24T04:31:45.3537612Z   Device Type:             GPU                                
2025-07-24T04:31:45.3538189Z   Cache Info:              
2025-07-24T04:31:45.3538675Z     L1:                      16(0x10) KB                        
2025-07-24T04:31:45.3539232Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:31:45.3539822Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:31:45.3540448Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:31:45.3541122Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:31:45.3541790Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:31:45.3542785Z   BDFID:                   12800                              
2025-07-24T04:31:45.3543420Z   Internal Node ID:        2                                  
2025-07-24T04:31:45.3544058Z   Compute Unit:            104                                
2025-07-24T04:31:45.3544696Z   SIMDs per CU:            4                                  
2025-07-24T04:31:45.3545327Z   Shader Engines:          8                                  
2025-07-24T04:31:45.3546002Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:31:45.3546689Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:31:45.3547395Z   Coherent Host Access:    FALSE                              
2025-07-24T04:31:45.3548012Z   Memory Properties:       
2025-07-24T04:31:45.3548500Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:31:45.3549133Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:31:45.3549818Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:31:45.3550525Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3551155Z   Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3551687Z     x                        1024(0x400)                        
2025-07-24T04:31:45.3552221Z     y                        1024(0x400)                        
2025-07-24T04:31:45.3552767Z     z                        1024(0x400)                        
2025-07-24T04:31:45.3553373Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:31:45.3554040Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:31:45.3554832Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3555423Z   Grid Max Size per Dimension:
2025-07-24T04:31:45.3555913Z     x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3556457Z     y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3557025Z     z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3557677Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:31:45.3567164Z   Packet Processor uCode:: 92                                 
2025-07-24T04:31:45.3568028Z   SDMA engine uCode::      9                                  
2025-07-24T04:31:45.3568735Z   IOMMU Support::          None                               
2025-07-24T04:31:45.3569343Z   Pool Info:               
2025-07-24T04:31:45.3569784Z     Pool 1                   
2025-07-24T04:31:45.3570346Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:31:45.3571031Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3571673Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3572339Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3573467Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3574218Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3574899Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3575502Z     Pool 2                   
2025-07-24T04:31:45.3576049Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:31:45.3576718Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3577394Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3578063Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3578779Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3579488Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3580181Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3580782Z     Pool 3                   
2025-07-24T04:31:45.3581320Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:31:45.3582283Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3582901Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3583567Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3584260Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3584968Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3585638Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3586226Z     Pool 4                   
2025-07-24T04:31:45.3586745Z       Segment:                 GROUP                              
2025-07-24T04:31:45.3587350Z       Size:                    64(0x40) KB                        
2025-07-24T04:31:45.3587986Z       Allocatable:             FALSE                              
2025-07-24T04:31:45.3588661Z       Alloc Granule:           0KB                                
2025-07-24T04:31:45.3589377Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:31:45.3590071Z       Alloc Alignment:         0KB                                
2025-07-24T04:31:45.3590759Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3591357Z   ISA Info:                
2025-07-24T04:31:45.3591778Z     ISA 1                    
2025-07-24T04:31:45.3592357Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:31:45.3593081Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:31:45.3593789Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:31:45.3594885Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3595614Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3596293Z       Fast f16:                TRUE                               
2025-07-24T04:31:45.3596964Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3597620Z       Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3598172Z         x                        1024(0x400)                        
2025-07-24T04:31:45.3598732Z         y                        1024(0x400)                        
2025-07-24T04:31:45.3599269Z         z                        1024(0x400)                        
2025-07-24T04:31:45.3599889Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3600501Z       Grid Max Size per Dimension:
2025-07-24T04:31:45.3601000Z         x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3601564Z         y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3602100Z         z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3603072Z       FBarrier Max Size:       32                                 
2025-07-24T04:31:45.3603677Z *******                  
2025-07-24T04:31:45.3604106Z Agent 4                  
2025-07-24T04:31:45.3604494Z *******                  
2025-07-24T04:31:45.3604956Z   Name:                    gfx90a                             
2025-07-24T04:31:45.3605583Z   Uuid:                    GPU-95cf8055be9ab54a               
2025-07-24T04:31:45.3606240Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:31:45.3606875Z   Vendor Name:             AMD                                
2025-07-24T04:31:45.3607436Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:31:45.3608011Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:31:45.3608584Z   Float Round Mode:        NEAR                               
2025-07-24T04:31:45.3609182Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:31:45.3609766Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:31:45.3610342Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:31:45.3611208Z   Queue Type:              MULTI                              
2025-07-24T04:31:45.3611730Z   Node:                    3                                  
2025-07-24T04:31:45.3612271Z   Device Type:             GPU                                
2025-07-24T04:31:45.3612767Z   Cache Info:              
2025-07-24T04:31:45.3613187Z     L1:                      16(0x10) KB                        
2025-07-24T04:31:45.3613681Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:31:45.3614201Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:31:45.3614767Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:31:45.3615348Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:31:45.3615935Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:31:45.3616484Z   BDFID:                   13568                              
2025-07-24T04:31:45.3617050Z   Internal Node ID:        3                                  
2025-07-24T04:31:45.3617618Z   Compute Unit:            104                                
2025-07-24T04:31:45.3618257Z   SIMDs per CU:            4                                  
2025-07-24T04:31:45.3619029Z   Shader Engines:          8                                  
2025-07-24T04:31:45.3619630Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:31:45.3620239Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:31:45.3620869Z   Coherent Host Access:    FALSE                              
2025-07-24T04:31:45.3621415Z   Memory Properties:       
2025-07-24T04:31:45.3621858Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:31:45.3622420Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:31:45.3623068Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:31:45.3623706Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3624267Z   Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3624745Z     x                        1024(0x400)                        
2025-07-24T04:31:45.3625220Z     y                        1024(0x400)                        
2025-07-24T04:31:45.3625696Z     z                        1024(0x400)                        
2025-07-24T04:31:45.3626223Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:31:45.3626827Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:31:45.3627425Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3627941Z   Grid Max Size per Dimension:
2025-07-24T04:31:45.3628372Z     x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3628853Z     y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3629344Z     z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3630157Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:31:45.3630819Z   Packet Processor uCode:: 92                                 
2025-07-24T04:31:45.3631444Z   SDMA engine uCode::      9                                  
2025-07-24T04:31:45.3632042Z   IOMMU Support::          None                               
2025-07-24T04:31:45.3632561Z   Pool Info:               
2025-07-24T04:31:45.3632943Z     Pool 1                   
2025-07-24T04:31:45.3633433Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:31:45.3634014Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3634701Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3635293Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3635913Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3636559Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3637456Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3637986Z     Pool 2                   
2025-07-24T04:31:45.3638457Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:31:45.3639040Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3639609Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3640191Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3640815Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3641429Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3642041Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3642553Z     Pool 3                   
2025-07-24T04:31:45.3643028Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:31:45.3643595Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3644167Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3644764Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3645374Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3645998Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3646599Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3647121Z     Pool 4                   
2025-07-24T04:31:45.3647564Z       Segment:                 GROUP                              
2025-07-24T04:31:45.3648105Z       Size:                    64(0x40) KB                        
2025-07-24T04:31:45.3648662Z       Allocatable:             FALSE                              
2025-07-24T04:31:45.3649253Z       Alloc Granule:           0KB                                
2025-07-24T04:31:45.3649918Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:31:45.3650538Z       Alloc Alignment:         0KB                                
2025-07-24T04:31:45.3651154Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3651674Z   ISA Info:                
2025-07-24T04:31:45.3652059Z     ISA 1                    
2025-07-24T04:31:45.3652555Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:31:45.3653188Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:31:45.3653808Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:31:45.3654417Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3655050Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3655625Z       Fast f16:                TRUE                               
2025-07-24T04:31:45.3656544Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3657134Z       Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3657616Z         x                        1024(0x400)                        
2025-07-24T04:31:45.3658119Z         y                        1024(0x400)                        
2025-07-24T04:31:45.3658588Z         z                        1024(0x400)                        
2025-07-24T04:31:45.3659127Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3659646Z       Grid Max Size per Dimension:
2025-07-24T04:31:45.3660090Z         x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3660582Z         y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3661066Z         z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3661616Z       FBarrier Max Size:       32                                 
2025-07-24T04:31:45.3662116Z *******                  
2025-07-24T04:31:45.3662731Z Agent 5                  
2025-07-24T04:31:45.3663089Z *******                  
2025-07-24T04:31:45.3663505Z   Name:                    gfx90a                             
2025-07-24T04:31:45.3664050Z   Uuid:                    GPU-e94c1f19ce37e5a1               
2025-07-24T04:31:45.3664654Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:31:45.3665254Z   Vendor Name:             AMD                                
2025-07-24T04:31:45.3665818Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:31:45.3666395Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:31:45.3666969Z   Float Round Mode:        NEAR                               
2025-07-24T04:31:45.3667570Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:31:45.3668140Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:31:45.3668717Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:31:45.3669312Z   Queue Type:              MULTI                              
2025-07-24T04:31:45.3669842Z   Node:                    4                                  
2025-07-24T04:31:45.3670384Z   Device Type:             GPU                                
2025-07-24T04:31:45.3670886Z   Cache Info:              
2025-07-24T04:31:45.3671312Z     L1:                      16(0x10) KB                        
2025-07-24T04:31:45.3671807Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:31:45.3672322Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:31:45.3672868Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:31:45.3673462Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:31:45.3674056Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:31:45.3674678Z   BDFID:                   4352                               
2025-07-24T04:31:45.3675250Z   Internal Node ID:        4                                  
2025-07-24T04:31:45.3675923Z   Compute Unit:            104                                
2025-07-24T04:31:45.3676490Z   SIMDs per CU:            4                                  
2025-07-24T04:31:45.3677060Z   Shader Engines:          8                                  
2025-07-24T04:31:45.3677655Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:31:45.3678271Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:31:45.3678945Z   Coherent Host Access:    FALSE                              
2025-07-24T04:31:45.3679502Z   Memory Properties:       
2025-07-24T04:31:45.3679926Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:31:45.3680479Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:31:45.3681072Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:31:45.3681684Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3682527Z   Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3683013Z     x                        1024(0x400)                        
2025-07-24T04:31:45.3683502Z     y                        1024(0x400)                        
2025-07-24T04:31:45.3683966Z     z                        1024(0x400)                        
2025-07-24T04:31:45.3684508Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:31:45.3685096Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:31:45.3685687Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3686215Z   Grid Max Size per Dimension:
2025-07-24T04:31:45.3686635Z     x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3687126Z     y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3687601Z     z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3688184Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:31:45.3689117Z   Packet Processor uCode:: 92                                 
2025-07-24T04:31:45.3689753Z   SDMA engine uCode::      9                                  
2025-07-24T04:31:45.3690379Z   IOMMU Support::          None                               
2025-07-24T04:31:45.3690903Z   Pool Info:               
2025-07-24T04:31:45.3691296Z     Pool 1                   
2025-07-24T04:31:45.3691777Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:31:45.3692373Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3692941Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3693544Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3694162Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3694823Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3695468Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3696005Z     Pool 2                   
2025-07-24T04:31:45.3696485Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:31:45.3697053Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3697618Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3698205Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3698829Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3699462Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3700060Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3700600Z     Pool 3                   
2025-07-24T04:31:45.3701259Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:31:45.3701839Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3702403Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3702998Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3703618Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3704232Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3704848Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3705365Z     Pool 4                   
2025-07-24T04:31:45.3705817Z       Segment:                 GROUP                              
2025-07-24T04:31:45.3706353Z       Size:                    64(0x40) KB                        
2025-07-24T04:31:45.3706922Z       Allocatable:             FALSE                              
2025-07-24T04:31:45.3707522Z       Alloc Granule:           0KB                                
2025-07-24T04:31:45.3708381Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:31:45.3709040Z       Alloc Alignment:         0KB                                
2025-07-24T04:31:45.3709651Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3710183Z   ISA Info:                
2025-07-24T04:31:45.3710552Z     ISA 1                    
2025-07-24T04:31:45.3711043Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:31:45.3711686Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:31:45.3712297Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:31:45.3712920Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3713543Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3714150Z       Fast f16:                TRUE                               
2025-07-24T04:31:45.3714964Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3715549Z       Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3716352Z         x                        1024(0x400)                        
2025-07-24T04:31:45.3716841Z         y                        1024(0x400)                        
2025-07-24T04:31:45.3717328Z         z                        1024(0x400)                        
2025-07-24T04:31:45.3717856Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3718389Z       Grid Max Size per Dimension:
2025-07-24T04:31:45.3718857Z         x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3719365Z         y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3719860Z         z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3720408Z       FBarrier Max Size:       32                                 
2025-07-24T04:31:45.3720933Z *******                  
2025-07-24T04:31:45.3721287Z Agent 6                  
2025-07-24T04:31:45.3721661Z *******                  
2025-07-24T04:31:45.3722093Z   Name:                    gfx90a                             
2025-07-24T04:31:45.3722646Z   Uuid:                    GPU-5159f7f6b913f996               
2025-07-24T04:31:45.3723237Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:31:45.3723839Z   Vendor Name:             AMD                                
2025-07-24T04:31:45.3724422Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:31:45.3724996Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:31:45.3725585Z   Float Round Mode:        NEAR                               
2025-07-24T04:31:45.3726169Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:31:45.3726759Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:31:45.3727322Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:31:45.3727901Z   Queue Type:              MULTI                              
2025-07-24T04:31:45.3728454Z   Node:                    5                                  
2025-07-24T04:31:45.3728989Z   Device Type:             GPU                                
2025-07-24T04:31:45.3729502Z   Cache Info:              
2025-07-24T04:31:45.3729913Z     L1:                      16(0x10) KB                        
2025-07-24T04:31:45.3730424Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:31:45.3730933Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:31:45.3731502Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:31:45.3732091Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:31:45.3732671Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:31:45.3733227Z   BDFID:                   5120                               
2025-07-24T04:31:45.3733768Z   Internal Node ID:        5                                  
2025-07-24T04:31:45.3758332Z   Compute Unit:            104                                
2025-07-24T04:31:45.3759045Z   SIMDs per CU:            4                                  
2025-07-24T04:31:45.3759670Z   Shader Engines:          8                                  
2025-07-24T04:31:45.3760295Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:31:45.3760915Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:31:45.3761540Z   Coherent Host Access:    FALSE                              
2025-07-24T04:31:45.3762092Z   Memory Properties:       
2025-07-24T04:31:45.3762532Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:31:45.3763083Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:31:45.3763684Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:31:45.3764262Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3764801Z   Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3765269Z     x                        1024(0x400)                        
2025-07-24T04:31:45.3766055Z     y                        1024(0x400)                        
2025-07-24T04:31:45.3766532Z     z                        1024(0x400)                        
2025-07-24T04:31:45.3767057Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:31:45.3767650Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:31:45.3768217Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3768724Z   Grid Max Size per Dimension:
2025-07-24T04:31:45.3769141Z     x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3769616Z     y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3770088Z     z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3770625Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:31:45.3771275Z   Packet Processor uCode:: 92                                 
2025-07-24T04:31:45.3771885Z   SDMA engine uCode::      9                                  
2025-07-24T04:31:45.3772465Z   IOMMU Support::          None                               
2025-07-24T04:31:45.3772961Z   Pool Info:               
2025-07-24T04:31:45.3773343Z     Pool 1                   
2025-07-24T04:31:45.3773823Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:31:45.3774382Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3774937Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3775504Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3776104Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3776706Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3777293Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3777809Z     Pool 2                   
2025-07-24T04:31:45.3778283Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:31:45.3778852Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3779422Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3779998Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3780596Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3781211Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3781826Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3782344Z     Pool 3                   
2025-07-24T04:31:45.3782803Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:31:45.3783343Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3784166Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3784757Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3785366Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3785972Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3786564Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3787068Z     Pool 4                   
2025-07-24T04:31:45.3787508Z       Segment:                 GROUP                              
2025-07-24T04:31:45.3788025Z       Size:                    64(0x40) KB                        
2025-07-24T04:31:45.3788561Z       Allocatable:             FALSE                              
2025-07-24T04:31:45.3789138Z       Alloc Granule:           0KB                                
2025-07-24T04:31:45.3789740Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:31:45.3790358Z       Alloc Alignment:         0KB                                
2025-07-24T04:31:45.3791179Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3791684Z   ISA Info:                
2025-07-24T04:31:45.3792050Z     ISA 1                    
2025-07-24T04:31:45.3792517Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:31:45.3793147Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:31:45.3793748Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:31:45.3794678Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3795313Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3795873Z       Fast f16:                TRUE                               
2025-07-24T04:31:45.3796444Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3796991Z       Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3797483Z         x                        1024(0x400)                        
2025-07-24T04:31:45.3797964Z         y                        1024(0x400)                        
2025-07-24T04:31:45.3798427Z         z                        1024(0x400)                        
2025-07-24T04:31:45.3798945Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3799458Z       Grid Max Size per Dimension:
2025-07-24T04:31:45.3799891Z         x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3800357Z         y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3800824Z         z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3801353Z       FBarrier Max Size:       32                                 
2025-07-24T04:31:45.3801854Z *******                  
2025-07-24T04:31:45.3802193Z Agent 7                  
2025-07-24T04:31:45.3802533Z *******                  
2025-07-24T04:31:45.3802938Z   Name:                    gfx90a                             
2025-07-24T04:31:45.3803467Z   Uuid:                    GPU-b7e9520488228d14               
2025-07-24T04:31:45.3804036Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:31:45.3804612Z   Vendor Name:             AMD                                
2025-07-24T04:31:45.3805165Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:31:45.3805715Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:31:45.3806277Z   Float Round Mode:        NEAR                               
2025-07-24T04:31:45.3806847Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:31:45.3807404Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:31:45.3807960Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:31:45.3808508Z   Queue Type:              MULTI                              
2025-07-24T04:31:45.3809323Z   Node:                    6                                  
2025-07-24T04:31:45.3809870Z   Device Type:             GPU                                
2025-07-24T04:31:45.3810372Z   Cache Info:              
2025-07-24T04:31:45.3810785Z     L1:                      16(0x10) KB                        
2025-07-24T04:31:45.3811268Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:31:45.3811760Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:31:45.3812294Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:31:45.3812865Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:31:45.3813437Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:31:45.3813971Z   BDFID:                   44544                              
2025-07-24T04:31:45.3814505Z   Internal Node ID:        6                                  
2025-07-24T04:31:45.3815072Z   Compute Unit:            104                                
2025-07-24T04:31:45.3815908Z   SIMDs per CU:            4                                  
2025-07-24T04:31:45.3816468Z   Shader Engines:          8                                  
2025-07-24T04:31:45.3817046Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:31:45.3817641Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:31:45.3818237Z   Coherent Host Access:    FALSE                              
2025-07-24T04:31:45.3818759Z   Memory Properties:       
2025-07-24T04:31:45.3819178Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:31:45.3819703Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:31:45.3820283Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:31:45.3820864Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3821395Z   Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3821833Z     x                        1024(0x400)                        
2025-07-24T04:31:45.3822309Z     y                        1024(0x400)                        
2025-07-24T04:31:45.3822769Z     z                        1024(0x400)                        
2025-07-24T04:31:45.3823283Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:31:45.3823859Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:31:45.3824439Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3824944Z   Grid Max Size per Dimension:
2025-07-24T04:31:45.3825351Z     x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3825820Z     y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3826302Z     z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3826836Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:31:45.3827479Z   Packet Processor uCode:: 92                                 
2025-07-24T04:31:45.3828095Z   SDMA engine uCode::      9                                  
2025-07-24T04:31:45.3828686Z   IOMMU Support::          None                               
2025-07-24T04:31:45.3829184Z   Pool Info:               
2025-07-24T04:31:45.3829544Z     Pool 1                   
2025-07-24T04:31:45.3830012Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:31:45.3830577Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3831124Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3831708Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3832311Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3832920Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3833504Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3834007Z     Pool 2                   
2025-07-24T04:31:45.3834865Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:31:45.3835487Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3836063Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3836647Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3837249Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3837858Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3838456Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3838967Z     Pool 3                   
2025-07-24T04:31:45.3839429Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:31:45.3839971Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3840510Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3841098Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3841999Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3842603Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3843190Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3843702Z     Pool 4                   
2025-07-24T04:31:45.3844146Z       Segment:                 GROUP                              
2025-07-24T04:31:45.3844671Z       Size:                    64(0x40) KB                        
2025-07-24T04:31:45.3845208Z       Allocatable:             FALSE                              
2025-07-24T04:31:45.3845784Z       Alloc Granule:           0KB                                
2025-07-24T04:31:45.3846384Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:31:45.3846983Z       Alloc Alignment:         0KB                                
2025-07-24T04:31:45.3847587Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3848106Z   ISA Info:                
2025-07-24T04:31:45.3848469Z     ISA 1                    
2025-07-24T04:31:45.3848943Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:31:45.3849568Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:31:45.3850175Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:31:45.3850766Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3851379Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3851938Z       Fast f16:                TRUE                               
2025-07-24T04:31:45.3852504Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3853043Z       Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3853509Z         x                        1024(0x400)                        
2025-07-24T04:31:45.3854008Z         y                        1024(0x400)                        
2025-07-24T04:31:45.3854468Z         z                        1024(0x400)                        
2025-07-24T04:31:45.3854989Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3855507Z       Grid Max Size per Dimension:
2025-07-24T04:31:45.3855939Z         x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3856413Z         y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3856882Z         z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3857422Z       FBarrier Max Size:       32                                 
2025-07-24T04:31:45.3857911Z *******                  
2025-07-24T04:31:45.3858265Z Agent 8                  
2025-07-24T04:31:45.3858598Z *******                  
2025-07-24T04:31:45.3859004Z   Name:                    gfx90a                             
2025-07-24T04:31:45.3859788Z   Uuid:                    GPU-9dc1bf8373e5d434               
2025-07-24T04:31:45.3860404Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:31:45.3860992Z   Vendor Name:             AMD                                
2025-07-24T04:31:45.3861547Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:31:45.3862113Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:31:45.3862677Z   Float Round Mode:        NEAR                               
2025-07-24T04:31:45.3863250Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:31:45.3863818Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:31:45.3864374Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:31:45.3864930Z   Queue Type:              MULTI                              
2025-07-24T04:31:45.3865447Z   Node:                    7                                  
2025-07-24T04:31:45.3865989Z   Device Type:             GPU                                
2025-07-24T04:31:45.3866759Z   Cache Info:              
2025-07-24T04:31:45.3867170Z     L1:                      16(0x10) KB                        
2025-07-24T04:31:45.3867669Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:31:45.3868175Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:31:45.3868726Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:31:45.3869298Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:31:45.3869875Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:31:45.3870409Z   BDFID:                   45824                              
2025-07-24T04:31:45.3870961Z   Internal Node ID:        7                                  
2025-07-24T04:31:45.3871529Z   Compute Unit:            104                                
2025-07-24T04:31:45.3872088Z   SIMDs per CU:            4                                  
2025-07-24T04:31:45.3872668Z   Shader Engines:          8                                  
2025-07-24T04:31:45.3873241Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:31:45.3873846Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:31:45.3874564Z   Coherent Host Access:    FALSE                              
2025-07-24T04:31:45.3875107Z   Memory Properties:       
2025-07-24T04:31:45.3875526Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:31:45.3876057Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:31:45.3876653Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:31:45.3877243Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3877781Z   Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3878230Z     x                        1024(0x400)                        
2025-07-24T04:31:45.3878705Z     y                        1024(0x400)                        
2025-07-24T04:31:45.3879179Z     z                        1024(0x400)                        
2025-07-24T04:31:45.3879705Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:31:45.3880281Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:31:45.3880872Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3881383Z   Grid Max Size per Dimension:
2025-07-24T04:31:45.3881791Z     x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3882273Z     y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3882744Z     z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3883299Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:31:45.3883932Z   Packet Processor uCode:: 92                                 
2025-07-24T04:31:45.3884547Z   SDMA engine uCode::      9                                  
2025-07-24T04:31:45.3885436Z   IOMMU Support::          None                               
2025-07-24T04:31:45.3885965Z   Pool Info:               
2025-07-24T04:31:45.3886346Z     Pool 1                   
2025-07-24T04:31:45.3886819Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:31:45.3887404Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3887964Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3888548Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3889163Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3889778Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3890374Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3890879Z     Pool 2                   
2025-07-24T04:31:45.3891351Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:31:45.3892205Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3892763Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3893355Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3893961Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3894573Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3895172Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3895689Z     Pool 3                   
2025-07-24T04:31:45.3896139Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:31:45.3896693Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3897236Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3897820Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3898445Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3899055Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3899657Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3900158Z     Pool 4                   
2025-07-24T04:31:45.3900604Z       Segment:                 GROUP                              
2025-07-24T04:31:45.3901127Z       Size:                    64(0x40) KB                        
2025-07-24T04:31:45.3901673Z       Allocatable:             FALSE                              
2025-07-24T04:31:45.3902264Z       Alloc Granule:           0KB                                
2025-07-24T04:31:45.3902868Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:31:45.3903479Z       Alloc Alignment:         0KB                                
2025-07-24T04:31:45.3904074Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3904605Z   ISA Info:                
2025-07-24T04:31:45.3904981Z     ISA 1                    
2025-07-24T04:31:45.3905462Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:31:45.3906092Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:31:45.3906693Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:31:45.3907302Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3907912Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3908490Z       Fast f16:                TRUE                               
2025-07-24T04:31:45.3909063Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3909615Z       Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3910089Z         x                        1024(0x400)                        
2025-07-24T04:31:45.3910800Z         y                        1024(0x400)                        
2025-07-24T04:31:45.3911304Z         z                        1024(0x400)                        
2025-07-24T04:31:45.3911825Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3912354Z       Grid Max Size per Dimension:
2025-07-24T04:31:45.3912786Z         x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3913279Z         y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3913765Z         z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3914557Z       FBarrier Max Size:       32                                 
2025-07-24T04:31:45.3915084Z *******                  
2025-07-24T04:31:45.3915435Z Agent 9                  
2025-07-24T04:31:45.3915784Z *******                  
2025-07-24T04:31:45.3916177Z   Name:                    gfx90a                             
2025-07-24T04:31:45.3916716Z   Uuid:                    GPU-2eced76570c2b642               
2025-07-24T04:31:45.3917604Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:31:45.3918192Z   Vendor Name:             AMD                                
2025-07-24T04:31:45.3918763Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:31:45.3919317Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:31:45.3919896Z   Float Round Mode:        NEAR                               
2025-07-24T04:31:45.3920480Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:31:45.3921061Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:31:45.3921617Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:31:45.3922179Z   Queue Type:              MULTI                              
2025-07-24T04:31:45.3922709Z   Node:                    8                                  
2025-07-24T04:31:45.3923234Z   Device Type:             GPU                                
2025-07-24T04:31:45.3923752Z   Cache Info:              
2025-07-24T04:31:45.3924168Z     L1:                      16(0x10) KB                        
2025-07-24T04:31:45.3924661Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:31:45.3925149Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:31:45.3925705Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:31:45.3926341Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:31:45.3927023Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:31:45.3927657Z   BDFID:                   36352                              
2025-07-24T04:31:45.3928296Z   Internal Node ID:        8                                  
2025-07-24T04:31:45.3928968Z   Compute Unit:            104                                
2025-07-24T04:31:45.3929611Z   SIMDs per CU:            4                                  
2025-07-24T04:31:45.3930283Z   Shader Engines:          8                                  
2025-07-24T04:31:45.3930984Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:31:45.3931702Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:31:45.3932286Z   Coherent Host Access:    FALSE                              
2025-07-24T04:31:45.3932585Z   Memory Properties:       
2025-07-24T04:31:45.3932806Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:31:45.3933052Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:31:45.3933327Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:31:45.3933601Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3933856Z   Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3934066Z     x                        1024(0x400)                        
2025-07-24T04:31:45.3934285Z     y                        1024(0x400)                        
2025-07-24T04:31:45.3934630Z     z                        1024(0x400)                        
2025-07-24T04:31:45.3934879Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:31:45.3935150Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:31:45.3935420Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3935664Z   Grid Max Size per Dimension:
2025-07-24T04:31:45.3935860Z     x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3936085Z     y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3936315Z     z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3936572Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:31:45.3936867Z   Packet Processor uCode:: 92                                 
2025-07-24T04:31:45.3937152Z   SDMA engine uCode::      9                                  
2025-07-24T04:31:45.3937432Z   IOMMU Support::          None                               
2025-07-24T04:31:45.3937798Z   Pool Info:               
2025-07-24T04:31:45.3937971Z     Pool 1                   
2025-07-24T04:31:45.3938196Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:31:45.3938461Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3938727Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3939002Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3939302Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3939589Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3939872Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3940119Z     Pool 2                   
2025-07-24T04:31:45.3940337Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:31:45.3940611Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3940879Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3941157Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3941438Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3941731Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3942015Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3942256Z     Pool 3                   
2025-07-24T04:31:45.3942480Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:31:45.3942733Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3942995Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3943265Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3943557Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3943855Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3944135Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3944374Z     Pool 4                   
2025-07-24T04:31:45.3944577Z       Segment:                 GROUP                              
2025-07-24T04:31:45.3944822Z       Size:                    64(0x40) KB                        
2025-07-24T04:31:45.3945069Z       Allocatable:             FALSE                              
2025-07-24T04:31:45.3945347Z       Alloc Granule:           0KB                                
2025-07-24T04:31:45.3945630Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:31:45.3945915Z       Alloc Alignment:         0KB                                
2025-07-24T04:31:45.3946197Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3946432Z   ISA Info:                
2025-07-24T04:31:45.3946724Z     ISA 1                    
2025-07-24T04:31:45.3946959Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:31:45.3947256Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:31:45.3947537Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:31:45.3947818Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3948106Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3948377Z       Fast f16:                TRUE                               
2025-07-24T04:31:45.3948645Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3948894Z       Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3949119Z         x                        1024(0x400)                        
2025-07-24T04:31:45.3949345Z         y                        1024(0x400)                        
2025-07-24T04:31:45.3949578Z         z                        1024(0x400)                        
2025-07-24T04:31:45.3949959Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3950205Z       Grid Max Size per Dimension:
2025-07-24T04:31:45.3950414Z         x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3950640Z         y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3950870Z         z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3951115Z       FBarrier Max Size:       32                                 
2025-07-24T04:31:45.3951348Z *******                  
2025-07-24T04:31:45.3951520Z Agent 10                 
2025-07-24T04:31:45.3951682Z *******                  
2025-07-24T04:31:45.3951885Z   Name:                    gfx90a                             
2025-07-24T04:31:45.3952132Z   Uuid:                    GPU-404705ee5bcc7f87               
2025-07-24T04:31:45.3952416Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:31:45.3952698Z   Vendor Name:             AMD                                
2025-07-24T04:31:45.3952966Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:31:45.3953227Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:31:45.3953498Z   Float Round Mode:        NEAR                               
2025-07-24T04:31:45.3953782Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:31:45.3954059Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:31:45.3954401Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:31:45.3954673Z   Queue Type:              MULTI                              
2025-07-24T04:31:45.3954937Z   Node:                    9                                  
2025-07-24T04:31:45.3955190Z   Device Type:             GPU                                
2025-07-24T04:31:45.3955444Z   Cache Info:              
2025-07-24T04:31:45.3955658Z     L1:                      16(0x10) KB                        
2025-07-24T04:31:45.3955903Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:31:45.3956156Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:31:45.3956421Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:31:45.3956709Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:31:45.3956987Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:31:45.3957255Z   BDFID:                   37632                              
2025-07-24T04:31:45.3957524Z   Internal Node ID:        9                                  
2025-07-24T04:31:45.3957797Z   Compute Unit:            104                                
2025-07-24T04:31:45.3958071Z   SIMDs per CU:            4                                  
2025-07-24T04:31:45.3958386Z   Shader Engines:          8                                  
2025-07-24T04:31:45.3958834Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:31:45.3959160Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:31:45.3959462Z   Coherent Host Access:    FALSE                              
2025-07-24T04:31:45.3959733Z   Memory Properties:       
2025-07-24T04:31:45.3959936Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:31:45.3960196Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:31:45.3960476Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:31:45.3960763Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3961023Z   Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3961252Z     x                        1024(0x400)                        
2025-07-24T04:31:45.3961489Z     y                        1024(0x400)                        
2025-07-24T04:31:45.3961714Z     z                        1024(0x400)                        
2025-07-24T04:31:45.3961982Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:31:45.3962403Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:31:45.3962692Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3962937Z   Grid Max Size per Dimension:
2025-07-24T04:31:45.3963149Z     x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3963379Z     y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3963621Z     z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3963898Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:31:45.3964202Z   Packet Processor uCode:: 92                                 
2025-07-24T04:31:45.3964500Z   SDMA engine uCode::      9                                  
2025-07-24T04:31:45.3964782Z   IOMMU Support::          None                               
2025-07-24T04:31:45.3965037Z   Pool Info:               
2025-07-24T04:31:45.3965225Z     Pool 1                   
2025-07-24T04:31:45.3965470Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:31:45.3965753Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3966015Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3966296Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3966592Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3966900Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3967186Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3967444Z     Pool 2                   
2025-07-24T04:31:45.3967680Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:31:45.3967956Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3968235Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3968529Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3968834Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3969126Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3969426Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3969688Z     Pool 3                   
2025-07-24T04:31:45.3969920Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:31:45.3970209Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:31:45.3970471Z       Allocatable:             TRUE                               
2025-07-24T04:31:45.3970762Z       Alloc Granule:           4KB                                
2025-07-24T04:31:45.3971072Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:31:45.3971502Z       Alloc Alignment:         4KB                                
2025-07-24T04:31:45.3971817Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3972071Z     Pool 4                   
2025-07-24T04:31:45.3972298Z       Segment:                 GROUP                              
2025-07-24T04:31:45.3972549Z       Size:                    64(0x40) KB                        
2025-07-24T04:31:45.3972816Z       Allocatable:             FALSE                              
2025-07-24T04:31:45.3973092Z       Alloc Granule:           0KB                                
2025-07-24T04:31:45.3973395Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:31:45.3973696Z       Alloc Alignment:         0KB                                
2025-07-24T04:31:45.3973980Z       Accessible by all:       FALSE                              
2025-07-24T04:31:45.3974232Z   ISA Info:                
2025-07-24T04:31:45.3974414Z     ISA 1                    
2025-07-24T04:31:45.3974659Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:31:45.3975072Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:31:45.3975369Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:31:45.3975655Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3975962Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:31:45.3976249Z       Fast f16:                TRUE                               
2025-07-24T04:31:45.3976524Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:31:45.3976802Z       Workgroup Max Size per Dimension:
2025-07-24T04:31:45.3977031Z         x                        1024(0x400)                        
2025-07-24T04:31:45.3977270Z         y                        1024(0x400)                        
2025-07-24T04:31:45.3977496Z         z                        1024(0x400)                        
2025-07-24T04:31:45.3977765Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:31:45.3978034Z       Grid Max Size per Dimension:
2025-07-24T04:31:45.3978247Z         x                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3978489Z         y                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3978721Z         z                        4294967295(0xffffffff)             
2025-07-24T04:31:45.3978995Z       FBarrier Max Size:       32                                 
2025-07-24T04:31:45.3979237Z *** Done ***             
2025-07-24T04:31:45.3997659Z ##[group]Run ngpu=$(rocminfo | grep -c -E 'Name:.*\sgfx')
2025-07-24T04:31:45.3998000Z ngpu=$(rocminfo | grep -c -E 'Name:.*\sgfx')
2025-07-24T04:31:45.3998529Z msg="Please file an issue on pytorch/pytorch reporting the faulty runner. Include a link to the runner logs so the runner can be identified"
2025-07-24T04:31:45.3999029Z if [[ $ngpu -eq 0 ]]; then
2025-07-24T04:31:45.3999299Z     echo "Error: Failed to detect any GPUs on the runner"
2025-07-24T04:31:45.3999576Z     echo "$msg"
2025-07-24T04:31:45.3999747Z     exit 1
2025-07-24T04:31:45.3999908Z fi
2025-07-24T04:31:45.4000058Z if [[ $ngpu -eq 1 ]]; then
2025-07-24T04:31:45.4000380Z     echo "Error: only 1 GPU detected, at least 2 GPUs are needed for distributed jobs"
2025-07-24T04:31:45.4000701Z     echo "$msg"
2025-07-24T04:31:45.4000879Z     exit 1
2025-07-24T04:31:45.4001043Z fi
2025-07-24T04:31:45.4030254Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:45.4030531Z env:
2025-07-24T04:31:45.4030681Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:45.4030872Z ##[endgroup]
2025-07-24T04:31:45.5737046Z ##[group]Run pytorch/pytorch/.github/actions/diskspace-cleanup@main
2025-07-24T04:31:45.5737754Z with:
2025-07-24T04:31:45.5738141Z   diskspace-cutoff: 70
2025-07-24T04:31:45.5738528Z env:
2025-07-24T04:31:45.5739328Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:45.5739811Z ##[endgroup]
2025-07-24T04:31:45.5812784Z ##[group]Run set -ex
2025-07-24T04:31:45.5813321Z set -ex
2025-07-24T04:31:45.5813736Z diskspace_cutoff=70
2025-07-24T04:31:45.5814381Z docker_root_dir=$(docker info -f '{{.DockerRootDir}}')
2025-07-24T04:31:45.5815090Z if [ ! -d "$docker_root_dir" ]; then
2025-07-24T04:31:45.5815997Z     echo "Docker root directory ($docker_root_dir) does not exist. Skipping disk space check."
2025-07-24T04:31:45.5816834Z     exit 0
2025-07-24T04:31:45.5817221Z fi
2025-07-24T04:31:45.5817949Z diskspace=$(df -H --output=pcent ${docker_root_dir} | sed -n 2p | sed 's/%//' | sed 's/ //')
2025-07-24T04:31:45.5819467Z msg="Please file an issue on pytorch/pytorch reporting the faulty runner. Include a link to the runner logs so the runner can be identified"
2025-07-24T04:31:45.5820784Z if [[ "$diskspace" -ge "$diskspace_cutoff" ]] ; then
2025-07-24T04:31:45.5821839Z     docker system prune -af
2025-07-24T04:31:45.5822703Z     diskspace_new=$(df -H --output=pcent ${docker_root_dir} | sed -n 2p | sed 's/%//' | sed 's/ //')
2025-07-24T04:31:45.5823718Z     if [[ "$diskspace_new" -gt "$diskspace_cutoff" ]] ; then
2025-07-24T04:31:45.5824800Z         echo "Error: Available diskspace is less than $diskspace_cutoff percent. Not enough diskspace."
2025-07-24T04:31:45.5825722Z         echo "$msg"
2025-07-24T04:31:45.5826163Z         exit 1
2025-07-24T04:31:45.5826576Z     else
2025-07-24T04:31:45.5827056Z         difference=$((diskspace - diskspace_new))
2025-07-24T04:31:45.5827774Z         echo "Diskspace saved: $difference percent"
2025-07-24T04:31:45.5828356Z     fi
2025-07-24T04:31:45.5828705Z fi
2025-07-24T04:31:45.5895063Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:45.5895667Z env:
2025-07-24T04:31:45.5896028Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:45.5896418Z ##[endgroup]
2025-07-24T04:31:45.5990706Z + diskspace_cutoff=70
2025-07-24T04:31:45.5999400Z ++ docker info -f '{{.DockerRootDir}}'
2025-07-24T04:31:45.6979215Z + docker_root_dir=/media/4TB/docker-rootless
2025-07-24T04:31:45.6979974Z + '[' '!' -d /media/4TB/docker-rootless ']'
2025-07-24T04:31:45.6995709Z ++ df -H --output=pcent /media/4TB/docker-rootless
2025-07-24T04:31:45.6997824Z ++ sed -n 2p
2025-07-24T04:31:45.7002984Z ++ sed s/%//
2025-07-24T04:31:45.7005252Z ++ sed 's/ //'
2025-07-24T04:31:45.7046538Z + diskspace=61
2025-07-24T04:31:45.7047710Z + msg='Please file an issue on pytorch/pytorch reporting the faulty runner. Include a link to the runner logs so the runner can be identified'
2025-07-24T04:31:45.7048875Z + [[ 61 -ge 70 ]]
2025-07-24T04:31:45.7102105Z ##[group]Run RUNNER_ARTIFACT_DIR="${RUNNER_TEMP}/artifacts"
2025-07-24T04:31:45.7102924Z RUNNER_ARTIFACT_DIR="${RUNNER_TEMP}/artifacts"
2025-07-24T04:31:45.7103614Z rm -rf "${RUNNER_ARTIFACT_DIR}"
2025-07-24T04:31:45.7104182Z mkdir -p "${RUNNER_ARTIFACT_DIR}"
2025-07-24T04:31:45.7104942Z echo "RUNNER_ARTIFACT_DIR=${RUNNER_ARTIFACT_DIR}" >> "${GITHUB_ENV}"
2025-07-24T04:31:45.7105642Z 
2025-07-24T04:31:45.7106160Z RUNNER_TEST_RESULTS_DIR="${RUNNER_TEMP}/test-results"
2025-07-24T04:31:45.7106854Z rm -rf "${RUNNER_TEST_RESULTS_DIR}"
2025-07-24T04:31:45.7107441Z mkdir -p "${RUNNER_TEST_RESULTS_DIR}"
2025-07-24T04:31:45.7108246Z echo "RUNNER_TEST_RESULTS_DIR=${RUNNER_TEST_RESULTS_DIR}" >> "${GITHUB_ENV}"
2025-07-24T04:31:45.7108993Z 
2025-07-24T04:31:45.7109396Z RUNNER_DOCS_DIR="${RUNNER_TEMP}/docs"
2025-07-24T04:31:45.7109953Z rm -rf "${RUNNER_DOCS_DIR}"
2025-07-24T04:31:45.7110484Z mkdir -p "${RUNNER_DOCS_DIR}"
2025-07-24T04:31:45.7111154Z echo "RUNNER_DOCS_DIR=${RUNNER_DOCS_DIR}" >> "${GITHUB_ENV}"
2025-07-24T04:31:45.7173202Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:45.7173889Z env:
2025-07-24T04:31:45.7174246Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:45.7174711Z ##[endgroup]
2025-07-24T04:31:45.7509864Z ##[group]Run env | grep '^GITHUB' >> "${RUNNER_TEMP}/github_env_${GITHUB_RUN_ID}"
2025-07-24T04:31:45.7510851Z env | grep '^GITHUB' >> "${RUNNER_TEMP}/github_env_${GITHUB_RUN_ID}"
2025-07-24T04:31:45.7511717Z env | grep '^CI' >> "${RUNNER_TEMP}/github_env_${GITHUB_RUN_ID}"
2025-07-24T04:31:45.7574440Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:45.7575115Z env:
2025-07-24T04:31:45.7575523Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:45.7576394Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:31:45.7577606Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:31:45.7578641Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:31:45.7579787Z ##[endgroup]
2025-07-24T04:31:45.7807948Z ##[group]Run # All GPUs are visible to the runner; visibility, if needed, will be set by run_test.py.
2025-07-24T04:31:45.7809239Z # All GPUs are visible to the runner; visibility, if needed, will be set by run_test.py.
2025-07-24T04:31:45.7810171Z # Add render group for container creation.
2025-07-24T04:31:45.7810937Z render_gid=`cat /etc/group | grep render | cut -d: -f3`
2025-07-24T04:31:45.7811851Z # Ensure GPU isolation if pod is part of kubernetes setup with DEVICE_FLAG.
2025-07-24T04:31:45.7812770Z if [ -f "/etc/podinfo/gha-render-devices" ]; then
2025-07-24T04:31:45.7813525Z   DEVICE_FLAG=$(cat /etc/podinfo/gha-render-devices)
2025-07-24T04:31:45.7814151Z else
2025-07-24T04:31:45.7814583Z   DEVICE_FLAG="--device /dev/dri"
2025-07-24T04:31:45.7815091Z fi
2025-07-24T04:31:45.7815953Z # The --group-add daemon and --group-add bin are needed in the Ubuntu 24.04 and Almalinux OSs respectively.
2025-07-24T04:31:45.7817484Z # This is due to the device files (/dev/kfd & /dev/dri) being owned by video group on bare metal.
2025-07-24T04:31:45.7818671Z # This video group ID maps to subgid 1 inside the docker image due to the /etc/subgid entries.
2025-07-24T04:31:45.7819910Z # The group name corresponding to group ID 1 can change depending on the OS, so both are necessary.
2025-07-24T04:31:45.7821998Z echo "GPU_FLAG=--device=/dev/mem --device=/dev/kfd $DEVICE_FLAG --group-add video --group-add $render_gid --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host" >> "${GITHUB_ENV}"
2025-07-24T04:31:45.7884985Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:45.7885654Z env:
2025-07-24T04:31:45.7886020Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:45.7886776Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:31:45.7887898Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:31:45.7888912Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:31:45.7889607Z ##[endgroup]
2025-07-24T04:31:45.8147642Z ##[group]Run aws-actions/configure-aws-credentials@ececac1a45f3b08a01d2dd070d28d111c5fe6722
2025-07-24T04:31:45.8148490Z with:
2025-07-24T04:31:45.8149053Z   role-to-assume: arn:aws:iam::308535385114:role/gha_workflow_s3_and_ecr_read_only
2025-07-24T04:31:45.8149717Z   aws-region: us-east-1
2025-07-24T04:31:45.8150140Z   role-duration-seconds: 18000
2025-07-24T04:31:45.8150578Z   audience: sts.amazonaws.com
2025-07-24T04:31:45.8150978Z env:
2025-07-24T04:31:45.8151281Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:45.8151917Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:31:45.8152857Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:31:45.8153747Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:31:45.8155646Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:31:45.8157169Z ##[endgroup]
2025-07-24T04:31:46.1142217Z Assuming role with OIDC
2025-07-24T04:31:46.2922457Z Authenticated as assumedRoleId AROAUPVRELQNLLCOPFEJR:GitHubActions
2025-07-24T04:31:46.3636964Z ##[group]Run aws-actions/amazon-ecr-login@062b18b96a7aff071d4dc91bc00c4c1a7945b076
2025-07-24T04:31:46.3638008Z with:
2025-07-24T04:31:46.3638456Z   mask-password: true
2025-07-24T04:31:46.3638937Z   registry-type: private
2025-07-24T04:31:46.3639404Z   skip-logout: false
2025-07-24T04:31:46.3639823Z env:
2025-07-24T04:31:46.3640213Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:46.3640974Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:31:46.3642551Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:31:46.3643576Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:31:46.3645281Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:31:46.3646803Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:31:46.3647368Z   AWS_REGION: us-east-1
2025-07-24T04:31:46.3648493Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:31:46.3649202Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:31:46.3659940Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:31:46.3660409Z ##[endgroup]
2025-07-24T04:31:46.7145541Z Logging into registry 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-07-24T04:31:47.0713837Z ##[group]Run pytorch/test-infra/.github/actions/calculate-docker-image@main
2025-07-24T04:31:47.0714971Z with:
2025-07-24T04:31:47.0716237Z   docker-image-name: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:31:47.0717770Z   use-custom-docker-registry: true
2025-07-24T04:31:47.0718333Z   docker-build-dir: .ci/docker
2025-07-24T04:31:47.0718813Z   docker-build-script: ./build.sh
2025-07-24T04:31:47.0719277Z   working-directory: .
2025-07-24T04:31:47.0719834Z   docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-07-24T04:31:47.0720461Z   force-push: false
2025-07-24T04:31:47.0720833Z env:
2025-07-24T04:31:47.0721187Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:47.0721886Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:31:47.0722882Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:31:47.0724006Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:31:47.0725828Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:31:47.0727465Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:31:47.0727789Z   AWS_REGION: us-east-1
2025-07-24T04:31:47.0728185Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:31:47.0728535Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:31:47.0733607Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:31:47.0733884Z ##[endgroup]
2025-07-24T04:31:47.0751534Z ##[group]Run set -ex
2025-07-24T04:31:47.0751769Z set -ex
2025-07-24T04:31:47.0751944Z 
2025-07-24T04:31:47.0752244Z # If the docker build directory or the build script doesn't exist, the action will
2025-07-24T04:31:47.0752734Z # gracefully return the docker image name as it is.  Pulling docker image in Linux
2025-07-24T04:31:47.0753149Z # job could then download the pre-built image as usual
2025-07-24T04:31:47.0753645Z if [[ -d "${DOCKER_BUILD_DIR}" ]] && [[ -f "${DOCKER_BUILD_DIR}/${DOCKER_BUILD_SCRIPT}" ]] && [[ "${USE_CUSTOM_DOCKER_REGISTRY}" == "true" ]]; then
2025-07-24T04:31:47.0754125Z   echo "skip=false" >> "${GITHUB_OUTPUT}"
2025-07-24T04:31:47.0754542Z else
2025-07-24T04:31:47.0754748Z   echo "skip=true" >> "${GITHUB_OUTPUT}"
2025-07-24T04:31:47.0755077Z   echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}"
2025-07-24T04:31:47.0755378Z 
2025-07-24T04:31:47.0755796Z   echo "Not using custom ECR registry.  Either it was not requested or there is no Docker build script in the ${REPO_NAME} repo..."
2025-07-24T04:31:47.0756337Z   exit 0
2025-07-24T04:31:47.0756564Z fi
2025-07-24T04:31:47.0756792Z 
2025-07-24T04:31:47.0757156Z if [[ "${DOCKER_IMAGE_NAME}" == *"${DOCKER_REGISTRY}/${REPO_NAME}"* ]]; then
2025-07-24T04:31:47.0757782Z   # The docker image name already includes the ECR prefix and tag, so we can just
2025-07-24T04:31:47.0758503Z   # use it as it is, but first let's extract the tag
2025-07-24T04:31:47.0758927Z   DOCKER_TAG=$(echo "${DOCKER_IMAGE_NAME}" | awk -F '[:,]' '{print $2}')
2025-07-24T04:31:47.0759376Z   echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}"
2025-07-24T04:31:47.0759794Z   echo "docker-image=${DOCKER_IMAGE_NAME}" >> "${GITHUB_OUTPUT}"
2025-07-24T04:31:47.0760151Z else
2025-07-24T04:31:47.0760395Z   if [[ "${DOCKER_IMAGE_NAME}" == *:* ]]; then
2025-07-24T04:31:47.0760734Z     CUSTOM_TAG_PREFIX=${DOCKER_IMAGE_NAME#*:}
2025-07-24T04:31:47.0761082Z     DOCKER_IMAGE_NAME=${DOCKER_IMAGE_NAME%%:*}
2025-07-24T04:31:47.0761378Z   fi
2025-07-24T04:31:47.0761987Z   DOCKER_TAG=${CUSTOM_TAG_PREFIX:+${CUSTOM_TAG_PREFIX}-}$(git rev-parse HEAD:"${DOCKER_BUILD_DIR}")
2025-07-24T04:31:47.0762546Z   echo "docker-tag=${DOCKER_TAG}" >> "${GITHUB_OUTPUT}"
2025-07-24T04:31:47.0763099Z   echo "docker-image=${DOCKER_REGISTRY}/${REPO_NAME}/${DOCKER_IMAGE_NAME}:${DOCKER_TAG}" >> "${GITHUB_OUTPUT}"
2025-07-24T04:31:47.0763687Z   echo "custom-tag-prefix=${CUSTOM_TAG_PREFIX}" >> "${GITHUB_OUTPUT}"
2025-07-24T04:31:47.0764052Z fi
2025-07-24T04:31:47.0797243Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:47.0797650Z env:
2025-07-24T04:31:47.0797896Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:47.0798291Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:31:47.0798843Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:31:47.0799356Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:31:47.0800208Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:31:47.0800972Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:31:47.0801223Z   AWS_REGION: us-east-1
2025-07-24T04:31:47.0801577Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:31:47.0801927Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:31:47.0806942Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:31:47.0807182Z   REPO_NAME: pytorch
2025-07-24T04:31:47.0807814Z   DOCKER_IMAGE_NAME: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:31:47.0808489Z   DOCKER_BUILD_DIR: .ci/docker
2025-07-24T04:31:47.0808743Z   DOCKER_BUILD_SCRIPT: ./build.sh
2025-07-24T04:31:47.0809072Z   DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-07-24T04:31:47.0809429Z   USE_CUSTOM_DOCKER_REGISTRY: true
2025-07-24T04:31:47.0809688Z   CUSTOM_TAG_PREFIX: 
2025-07-24T04:31:47.0809916Z ##[endgroup]
2025-07-24T04:31:47.0873441Z + [[ -d .ci/docker ]]
2025-07-24T04:31:47.0874055Z + [[ -f .ci/docker/./build.sh ]]
2025-07-24T04:31:47.0874802Z + [[ true == \t\r\u\e ]]
2025-07-24T04:31:47.0875295Z + echo skip=false
2025-07-24T04:31:47.0877045Z + [[ 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f == *\3\0\8\5\3\5\3\8\5\1\1\4\.\d\k\r\.\e\c\r\.\u\s\-\e\a\s\t\-\1\.\a\m\a\z\o\n\a\w\s\.\c\o\m\/\p\y\t\o\r\c\h* ]]
2025-07-24T04:31:47.0894350Z ++ echo 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:31:47.0896636Z ++ awk -F '[:,]' '{print $2}'
2025-07-24T04:31:47.0954160Z + DOCKER_TAG=pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:31:47.0955733Z + echo docker-tag=pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:31:47.0957594Z + echo docker-image=308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:31:47.1018695Z ##[group]Run set +e
2025-07-24T04:31:47.1019258Z set +e
2025-07-24T04:31:47.1019673Z set -x
2025-07-24T04:31:47.1020055Z 
2025-07-24T04:31:47.1020447Z login() {
2025-07-24T04:31:47.1021441Z   aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1"
2025-07-24T04:31:47.1022523Z }
2025-07-24T04:31:47.1022964Z 
2025-07-24T04:31:47.1023448Z retry () {
2025-07-24T04:31:47.1024022Z   $*  || (sleep 1 && $*) || (sleep 2 && $*)
2025-07-24T04:31:47.1025173Z }
2025-07-24T04:31:47.1025739Z 
2025-07-24T04:31:47.1026436Z retry login "${DOCKER_REGISTRY}"
2025-07-24T04:31:47.1027308Z 
2025-07-24T04:31:47.1055511Z START_TIME=$(date +%s)
2025-07-24T04:31:47.1056760Z # Wait up to 120 minutes
2025-07-24T04:31:47.1057493Z while [[ $(( $(date +%s) - 7200 )) -lt $START_TIME ]]; do
2025-07-24T04:31:47.1058393Z   # Check if image already exists, if it does then skip building it
2025-07-24T04:31:47.1059272Z   if docker manifest inspect "${DOCKER_IMAGE}"; then
2025-07-24T04:31:47.1059923Z     exit 0
2025-07-24T04:31:47.1060353Z   fi
2025-07-24T04:31:47.1060743Z 
2025-07-24T04:31:47.1061438Z   # NB: This flag is used by Docker build workflow to push the image to ECR, so we can
2025-07-24T04:31:47.1062575Z   # use this to differentiate between the Docker build and regular build jobs. For the
2025-07-24T04:31:47.1063690Z   # latter, it will wait for the Docker images to become available before continuing
2025-07-24T04:31:47.1064573Z   if [ "${DOCKER_PUSH:-false}" == "true" ]; then
2025-07-24T04:31:47.1065274Z     # It's a Docker build job, let's build the image
2025-07-24T04:31:47.1065890Z     break
2025-07-24T04:31:47.1066307Z   else
2025-07-24T04:31:47.1066908Z     # It's a regular build job, wait for the image to become available
2025-07-24T04:31:47.1067651Z     sleep 300
2025-07-24T04:31:47.1068109Z   fi
2025-07-24T04:31:47.1068499Z done
2025-07-24T04:31:47.1068879Z 
2025-07-24T04:31:47.1069501Z # NB: This part requires a full checkout. Otherwise, the merge base will
2025-07-24T04:31:47.1070504Z # be empty.  The default action would be to continue rebuild the image
2025-07-24T04:31:47.1071402Z if [[ "$BASE_REVISION" = "$(git rev-parse HEAD)" ]]; then
2025-07-24T04:31:47.1072196Z   # if we're on the base branch then use the parent commit
2025-07-24T04:31:47.1072902Z   MERGE_BASE=$(git rev-parse HEAD~)
2025-07-24T04:31:47.1073448Z else
2025-07-24T04:31:47.1074035Z   # otherwise we're on a PR, so use the most recent base commit
2025-07-24T04:31:47.1074995Z   MERGE_BASE=$(git merge-base HEAD "$BASE_REVISION")
2025-07-24T04:31:47.1075627Z fi
2025-07-24T04:31:47.1076016Z 
2025-07-24T04:31:47.1076433Z if [[ -z "${MERGE_BASE}" ]]; then
2025-07-24T04:31:47.1077074Z   echo "rebuild=true" >> "${GITHUB_OUTPUT}"
2025-07-24T04:31:47.1077665Z 
2025-07-24T04:31:47.1078485Z   echo "Finding merge base only works with full checkout, please set fetch-depth to 0, continuing ..."
2025-07-24T04:31:47.1079413Z   exit 0
2025-07-24T04:31:47.1079809Z fi
2025-07-24T04:31:47.1080177Z 
2025-07-24T04:31:47.1080719Z if ! git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}"; then
2025-07-24T04:31:47.1082072Z   echo "Directory '${DOCKER_BUILD_DIR}' not found in commit $MERGE_BASE, you should rebase onto a more recent commit"
2025-07-24T04:31:47.1083243Z   exit 1
2025-07-24T04:31:47.1083707Z fi
2025-07-24T04:31:47.1084165Z 
2025-07-24T04:31:47.1084916Z PREVIOUS_DOCKER_TAG=$(git rev-parse "${MERGE_BASE}:${DOCKER_BUILD_DIR}")
2025-07-24T04:31:47.1086812Z # If no image exists but the hash is the same as the previous hash then we should error out here
2025-07-24T04:31:47.1087938Z if [[ "${PREVIOUS_DOCKER_TAG}" == "${DOCKER_TAG}" ]]; then
2025-07-24T04:31:47.1089103Z   echo "WARNING: Something has gone wrong and the previous image isn't available for the merge-base of your branch"
2025-07-24T04:31:47.1090463Z   echo "         Will re-build docker image to store in local cache, TTS may be longer"
2025-07-24T04:31:47.1091260Z fi
2025-07-24T04:31:47.1091637Z 
2025-07-24T04:31:47.1092102Z echo "rebuild=true" >> "${GITHUB_OUTPUT}"
2025-07-24T04:31:47.1161608Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:47.1162449Z env:
2025-07-24T04:31:47.1162930Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:47.1164240Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:31:47.1165613Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:31:47.1166891Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:31:47.1168713Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:31:47.1170305Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:31:47.1170832Z   AWS_REGION: us-east-1
2025-07-24T04:31:47.1171457Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:31:47.1172191Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:31:47.1183163Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:31:47.1183656Z   DOCKER_BUILD_DIR: .ci/docker
2025-07-24T04:31:47.1184262Z   BASE_REVISION: 27c8ef1839e0747b36769efb192befc9308ff93c
2025-07-24T04:31:47.1185722Z   DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:31:47.1187401Z   DOCKER_TAG: pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:31:47.1188471Z   DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-07-24T04:31:47.1189158Z   DOCKER_PUSH: 
2025-07-24T04:31:47.1189563Z ##[endgroup]
2025-07-24T04:31:47.1293343Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-07-24T04:31:47.1294195Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-07-24T04:31:47.1302375Z + aws ecr get-login-password --region us-east-1
2025-07-24T04:31:47.1306285Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-07-24T04:31:48.2326080Z WARNING! Your password will be stored unencrypted in /var/home/pytorchci/.docker/config.json.
2025-07-24T04:31:48.2327224Z Configure a credential helper to remove this warning. See
2025-07-24T04:31:48.2328314Z https://docs.docker.com/engine/reference/commandline/login/#credential-stores
2025-07-24T04:31:48.2329018Z 
2025-07-24T04:31:48.2337084Z Login Succeeded
2025-07-24T04:31:48.2391217Z ++ date +%s
2025-07-24T04:31:48.2417244Z + START_TIME=1753331508
2025-07-24T04:31:48.2426430Z ++ date +%s
2025-07-24T04:31:48.2450896Z + [[ 1753324308 -lt 1753331508 ]]
2025-07-24T04:31:48.2452465Z + docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:31:49.0085376Z {
2025-07-24T04:31:49.0087054Z 	"schemaVersion": 2,
2025-07-24T04:31:49.0088199Z 	"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
2025-07-24T04:31:49.0089075Z 	"config": {
2025-07-24T04:31:49.0089732Z 		"mediaType": "application/vnd.docker.container.image.v1+json",
2025-07-24T04:31:49.0090487Z 		"size": 28426,
2025-07-24T04:31:49.0091231Z 		"digest": "sha256:441c23f68601012989b37ca64859d010f9dc3cbc8ea65694e4ace6ed235631cc"
2025-07-24T04:31:49.0092150Z 	},
2025-07-24T04:31:49.0092516Z 	"layers": [
2025-07-24T04:31:49.0092907Z 		{
2025-07-24T04:31:49.0093445Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0094799Z 			"size": 30446997,
2025-07-24T04:31:49.0095523Z 			"digest": "sha256:66587c81b81a58d07e40c48d900a1517516bbf58c4378c687d89d645824f5e5f"
2025-07-24T04:31:49.0096357Z 		},
2025-07-24T04:31:49.0096721Z 		{
2025-07-24T04:31:49.0097341Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0097998Z 			"size": 1554,
2025-07-24T04:31:49.0098617Z 			"digest": "sha256:c467531473a7140e2846e8648a245301c77a465d0561b48e8dd31b30e191e523"
2025-07-24T04:31:49.0099311Z 		},
2025-07-24T04:31:49.0099607Z 		{
2025-07-24T04:31:49.0100114Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0100747Z 			"size": 313286081,
2025-07-24T04:31:49.0101407Z 			"digest": "sha256:6edafb9a3aa208d2ac734c6205f560f7b7dcdec1a7b832bc7894e1278787f2b1"
2025-07-24T04:31:49.0102541Z 		},
2025-07-24T04:31:49.0102882Z 		{
2025-07-24T04:31:49.0103385Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0104028Z 			"size": 704,
2025-07-24T04:31:49.0104669Z 			"digest": "sha256:67bbdb9be01f9e05df0be4996ff180f6ee99763b4e8bed028f22e8c5c687f1f3"
2025-07-24T04:31:49.0105372Z 		},
2025-07-24T04:31:49.0105683Z 		{
2025-07-24T04:31:49.0106195Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0106823Z 			"size": 1214,
2025-07-24T04:31:49.0107462Z 			"digest": "sha256:d30866a5fbe67a4d3ed6411fd4d9040a5715bdb1f83db530ab44c7275bf0cd12"
2025-07-24T04:31:49.0108181Z 		},
2025-07-24T04:31:49.0108490Z 		{
2025-07-24T04:31:49.0108986Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0109605Z 			"size": 485,
2025-07-24T04:31:49.0110232Z 			"digest": "sha256:ef2b71d5d5187f9bc8eb30b15d328bc65465f4b4b4c67061cfd8027aba8f72dd"
2025-07-24T04:31:49.0110945Z 		},
2025-07-24T04:31:49.0111265Z 		{
2025-07-24T04:31:49.0111849Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0112593Z 			"size": 110343867,
2025-07-24T04:31:49.0113375Z 			"digest": "sha256:3a79f1be9328db38db5d9682fef0e9659b2d24e666d5ef2f39f2a233084e548c"
2025-07-24T04:31:49.0114202Z 		},
2025-07-24T04:31:49.0114726Z 		{
2025-07-24T04:31:49.0115237Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0115862Z 			"size": 4164,
2025-07-24T04:31:49.0116547Z 			"digest": "sha256:ab67e3b4c18a18c51c22703a2493006d45815828baf0b1df64b12da7ff0bd2e7"
2025-07-24T04:31:49.0117359Z 		},
2025-07-24T04:31:49.0117708Z 		{
2025-07-24T04:31:49.0118290Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0118923Z 			"size": 1710,
2025-07-24T04:31:49.0119538Z 			"digest": "sha256:035eb636c57bd7202decf2a456e8537bddc1be808adbd80f6dd75e12c13e1be3"
2025-07-24T04:31:49.0120241Z 		},
2025-07-24T04:31:49.0120532Z 		{
2025-07-24T04:31:49.0121029Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0121641Z 			"size": 724,
2025-07-24T04:31:49.0122267Z 			"digest": "sha256:d4ec37220e3a61f253b7bff3b60dce616d51caf1327ea11d322d85cd281dbf07"
2025-07-24T04:31:49.0122983Z 		},
2025-07-24T04:31:49.0123274Z 		{
2025-07-24T04:31:49.0123757Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0124387Z 			"size": 3305033147,
2025-07-24T04:31:49.0125025Z 			"digest": "sha256:9d9b3bb5166b5733af7605b65690330a088bd8636c36433e8e173a6659b16421"
2025-07-24T04:31:49.0125714Z 		},
2025-07-24T04:31:49.0126005Z 		{
2025-07-24T04:31:49.0126497Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0127119Z 			"size": 381,
2025-07-24T04:31:49.0127713Z 			"digest": "sha256:554a1868b180d4e2032cc390ea00a9f7c180d8885392be12550bc3e1c5e1a23c"
2025-07-24T04:31:49.0128404Z 		},
2025-07-24T04:31:49.0128709Z 		{
2025-07-24T04:31:49.0129211Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0129948Z 			"size": 233567,
2025-07-24T04:31:49.0130688Z 			"digest": "sha256:090ade5d5ee9d68cd421dbc7cb6e401709f6c178a61d3f87019b041c6774b2db"
2025-07-24T04:31:49.0131771Z 		},
2025-07-24T04:31:49.0132055Z 		{
2025-07-24T04:31:49.0132557Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0133184Z 			"size": 792,
2025-07-24T04:31:49.0133836Z 			"digest": "sha256:a998e898d9939b4c1d30567427b67cc709d6852ee65ef64fa2d9c318db592126"
2025-07-24T04:31:49.0134567Z 		},
2025-07-24T04:31:49.0134873Z 		{
2025-07-24T04:31:49.0135393Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0136028Z 			"size": 106,
2025-07-24T04:31:49.0136662Z 			"digest": "sha256:cfaabb19fb471ea7d79bd4c76b041e6a316873783814724e35c08c71bf89cd2c"
2025-07-24T04:31:49.0137374Z 		},
2025-07-24T04:31:49.0137670Z 		{
2025-07-24T04:31:49.0138161Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0139056Z 			"size": 1495,
2025-07-24T04:31:49.0139693Z 			"digest": "sha256:53b92662290aaff40a4779c14e8bf37ca7e400c814eadabbdeea922a157f26f0"
2025-07-24T04:31:49.0140412Z 		},
2025-07-24T04:31:49.0140709Z 		{
2025-07-24T04:31:49.0141204Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0141821Z 			"size": 455235035,
2025-07-24T04:31:49.0142454Z 			"digest": "sha256:95b6332c3b55e9d6fe681de21373e3993ddc9b286515039dc2f7ed9914277167"
2025-07-24T04:31:49.0143161Z 		},
2025-07-24T04:31:49.0143460Z 		{
2025-07-24T04:31:49.0143960Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0144574Z 			"size": 163,
2025-07-24T04:31:49.0145206Z 			"digest": "sha256:b67085f0f4fdd797574ea5df363a0cbde9ca856b230bbdd7aae29b90ad5e79cb"
2025-07-24T04:31:49.0145920Z 		},
2025-07-24T04:31:49.0146215Z 		{
2025-07-24T04:31:49.0146701Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0147323Z 			"size": 2438,
2025-07-24T04:31:49.0147968Z 			"digest": "sha256:5e2e975cce85260c1165f6d3d8280f9d678fed13e21fc8b6efba9f1a2bca97bc"
2025-07-24T04:31:49.0148667Z 		},
2025-07-24T04:31:49.0148973Z 		{
2025-07-24T04:31:49.0149484Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0150244Z 			"size": 7164124491,
2025-07-24T04:31:49.0151002Z 			"digest": "sha256:90308017eb7efa2434edc95a07bc92d8710942262bf3d76dda289feca59fb5f5"
2025-07-24T04:31:49.0151827Z 		},
2025-07-24T04:31:49.0152132Z 		{
2025-07-24T04:31:49.0152630Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0153260Z 			"size": 105,
2025-07-24T04:31:49.0153904Z 			"digest": "sha256:7f8c898d8b8417ffa2b0a975a6e9a7ccaee952d8e8acb7e90906c53f9f41a7a6"
2025-07-24T04:31:49.0154737Z 		},
2025-07-24T04:31:49.0155049Z 		{
2025-07-24T04:31:49.0155561Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0156185Z 			"size": 612,
2025-07-24T04:31:49.0156809Z 			"digest": "sha256:5b1565991f5c0f1defeb2448eaac6383107d330f11fa60e3c9f8488752ee12e8"
2025-07-24T04:31:49.0157511Z 		},
2025-07-24T04:31:49.0157810Z 		{
2025-07-24T04:31:49.0158316Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0158951Z 			"size": 677677324,
2025-07-24T04:31:49.0159592Z 			"digest": "sha256:cfa6c02ce350f691ca01e9bbc91554b3448765199582f80d0dc44b8b4d226f46"
2025-07-24T04:31:49.0160298Z 		},
2025-07-24T04:31:49.0160596Z 		{
2025-07-24T04:31:49.0161086Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0161705Z 			"size": 111,
2025-07-24T04:31:49.0162312Z 			"digest": "sha256:4690cd0b661d36beb72956d1933777b5e2eb63ac53f3549cbe117619cbed11ab"
2025-07-24T04:31:49.0162994Z 		},
2025-07-24T04:31:49.0163298Z 		{
2025-07-24T04:31:49.0163783Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0164447Z 			"size": 1557,
2025-07-24T04:31:49.0165090Z 			"digest": "sha256:8dab92a6cabfe1c1cc4cf921541571080c1db7eeaa5d6fb5e1dc421b7627954e"
2025-07-24T04:31:49.0165798Z 		},
2025-07-24T04:31:49.0166091Z 		{
2025-07-24T04:31:49.0166572Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0167626Z 			"size": 107,
2025-07-24T04:31:49.0168356Z 			"digest": "sha256:741b9f0a45dbdd117063541fe5928c8656c711df99ce2f6c93fb837df6c54000"
2025-07-24T04:31:49.0169079Z 		},
2025-07-24T04:31:49.0169365Z 		{
2025-07-24T04:31:49.0169856Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0170479Z 			"size": 166,
2025-07-24T04:31:49.0171280Z 			"digest": "sha256:78a6125437f3d51017ccb8735f1c3fce4ce00aa14f7490ba2d1f3a0254101595"
2025-07-24T04:31:49.0171997Z 		},
2025-07-24T04:31:49.0172301Z 		{
2025-07-24T04:31:49.0172807Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0173434Z 			"size": 2801110,
2025-07-24T04:31:49.0174087Z 			"digest": "sha256:3be44072ab4e7975f53fa3962eb807664f771ebafad2dae86f07404fb56be803"
2025-07-24T04:31:49.0175080Z 		},
2025-07-24T04:31:49.0175385Z 		{
2025-07-24T04:31:49.0175875Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0176508Z 			"size": 107,
2025-07-24T04:31:49.0177178Z 			"digest": "sha256:04044610df175f1bf00fb5ea1215ba9784924b5a20293aafe94a155cfc75d56d"
2025-07-24T04:31:49.0178002Z 		},
2025-07-24T04:31:49.0178346Z 		{
2025-07-24T04:31:49.0178922Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0179661Z 			"size": 802,
2025-07-24T04:31:49.0180385Z 			"digest": "sha256:152aa5a5158e010217f23fd8fbb0d9b0826c39bb13e6d8652c6965cd56b5fc7d"
2025-07-24T04:31:49.0181209Z 		},
2025-07-24T04:31:49.0181561Z 		{
2025-07-24T04:31:49.0182147Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0182901Z 			"size": 26113617,
2025-07-24T04:31:49.0183563Z 			"digest": "sha256:63f73895622ecab7d061d93953419fbdf7c48814c5008105fc0b432177c033a0"
2025-07-24T04:31:49.0184252Z 		},
2025-07-24T04:31:49.0184569Z 		{
2025-07-24T04:31:49.0185048Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0185662Z 			"size": 104,
2025-07-24T04:31:49.0186309Z 			"digest": "sha256:5af6f2f093ea5798a44a5decddf46cb875cbf4b54e5e938d3ad6d5a0d3b61967"
2025-07-24T04:31:49.0187028Z 		},
2025-07-24T04:31:49.0187326Z 		{
2025-07-24T04:31:49.0187820Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0188435Z 			"size": 425,
2025-07-24T04:31:49.0189053Z 			"digest": "sha256:fda40cf9ab8372afeefe16b77d0233d4b63b678cace2059714f7c35d772e1637"
2025-07-24T04:31:49.0189760Z 		},
2025-07-24T04:31:49.0190056Z 		{
2025-07-24T04:31:49.0190533Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0191152Z 			"size": 19308731,
2025-07-24T04:31:49.0191796Z 			"digest": "sha256:5aae7747cc25bf67f6bfe370770b12dccc584ea9afb3f8b825b03b0cb15918b1"
2025-07-24T04:31:49.0192497Z 		},
2025-07-24T04:31:49.0192791Z 		{
2025-07-24T04:31:49.0193290Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0193913Z 			"size": 425,
2025-07-24T04:31:49.0194797Z 			"digest": "sha256:5b62e7dff82033606f1b96c42c49540ebbe46772aeb12467df1e1bb07eb2e175"
2025-07-24T04:31:49.0195533Z 		},
2025-07-24T04:31:49.0195832Z 		{
2025-07-24T04:31:49.0196415Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0197138Z 			"size": 724,
2025-07-24T04:31:49.0197871Z 			"digest": "sha256:d4ec37220e3a61f253b7bff3b60dce616d51caf1327ea11d322d85cd281dbf07"
2025-07-24T04:31:49.0198579Z 		},
2025-07-24T04:31:49.0198874Z 		{
2025-07-24T04:31:49.0199378Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0200016Z 			"size": 141,
2025-07-24T04:31:49.0200713Z 			"digest": "sha256:02460b2a4f6a2d3bc23ff65db1afd3750ee9ae49a35ce9ddb8e74f3c46b5705f"
2025-07-24T04:31:49.0201554Z 		},
2025-07-24T04:31:49.0201903Z 		{
2025-07-24T04:31:49.0202483Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0203210Z 			"size": 136,
2025-07-24T04:31:49.0203823Z 			"digest": "sha256:344aa86a61d68f5f8573f99122b4efe5c555bdceaeb3a1f1b5502715dbaee4c7"
2025-07-24T04:31:49.0204854Z 		},
2025-07-24T04:31:49.0205147Z 		{
2025-07-24T04:31:49.0205631Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0206249Z 			"size": 5098992948,
2025-07-24T04:31:49.0206887Z 			"digest": "sha256:04b592ce3217c257cc0b889e15f3c3731d22baca68e5760ec146ef749d0ff1f1"
2025-07-24T04:31:49.0207576Z 		},
2025-07-24T04:31:49.0207868Z 		{
2025-07-24T04:31:49.0208350Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0208963Z 			"size": 195,
2025-07-24T04:31:49.0209582Z 			"digest": "sha256:e3aedebec93030bc06d19aacab68750c6e085bb1c27dc358d16858848935cf71"
2025-07-24T04:31:49.0210281Z 		},
2025-07-24T04:31:49.0210576Z 		{
2025-07-24T04:31:49.0211064Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0211971Z 			"size": 346,
2025-07-24T04:31:49.0212584Z 			"digest": "sha256:2b5211eebec64099919bea1197c9c43983976b5037fd1ff6236509f49dd4a979"
2025-07-24T04:31:49.0213286Z 		},
2025-07-24T04:31:49.0213581Z 		{
2025-07-24T04:31:49.0214076Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0214743Z + exit 0
2025-07-24T04:31:49.0215072Z 			"size": 88303,
2025-07-24T04:31:49.0215696Z 			"digest": "sha256:88aac01ec1db72a960d19e691f39b315730a97c2330390a8e6d0cbd2f80d3ca0"
2025-07-24T04:31:49.0216399Z 		},
2025-07-24T04:31:49.0216702Z 		{
2025-07-24T04:31:49.0217223Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0217944Z 			"size": 106,
2025-07-24T04:31:49.0218663Z 			"digest": "sha256:f154dd81b32fa92d4faa8040506993076e2912a919c2afe6dded50740d86d5be"
2025-07-24T04:31:49.0219410Z 		},
2025-07-24T04:31:49.0219727Z 		{
2025-07-24T04:31:49.0220222Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0220860Z 			"size": 1666,
2025-07-24T04:31:49.0221480Z 			"digest": "sha256:089f2260c49023e792a2bee3867520ef852c8a55f37ce04ea81b07b7dfc2dae7"
2025-07-24T04:31:49.0222187Z 		},
2025-07-24T04:31:49.0222500Z 		{
2025-07-24T04:31:49.0223006Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0223630Z 			"size": 724,
2025-07-24T04:31:49.0224243Z 			"digest": "sha256:d4ec37220e3a61f253b7bff3b60dce616d51caf1327ea11d322d85cd281dbf07"
2025-07-24T04:31:49.0224950Z 		},
2025-07-24T04:31:49.0225249Z 		{
2025-07-24T04:31:49.0225743Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0226367Z 			"size": 137,
2025-07-24T04:31:49.0226980Z 			"digest": "sha256:ca8b54819584ab34ad98cd66be3fceb44c157dc0b7f140070c4584816919bbe5"
2025-07-24T04:31:49.0227682Z 		},
2025-07-24T04:31:49.0227985Z 		{
2025-07-24T04:31:49.0228475Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0229092Z 			"size": 120,
2025-07-24T04:31:49.0229706Z 			"digest": "sha256:76870a4c3f8fd6d57b2ec17e72b7a85a29d885057ef18bf331afb97a748d6d40"
2025-07-24T04:31:49.0230414Z 		},
2025-07-24T04:31:49.0230731Z 		{
2025-07-24T04:31:49.0231224Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0231850Z 			"size": 5356440017,
2025-07-24T04:31:49.0232500Z 			"digest": "sha256:f4a9f30fcdabff1a1167cb66c7560f21b3668899a96b35b9fc4fde046e4ef265"
2025-07-24T04:31:49.0233210Z 		},
2025-07-24T04:31:49.0233514Z 		{
2025-07-24T04:31:49.0234008Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0234728Z 			"size": 175,
2025-07-24T04:31:49.0235345Z 			"digest": "sha256:110b5a00f043de5141a35fc11bdd3d6541ec027ba726178f89e3bd5b44f18d8b"
2025-07-24T04:31:49.0236087Z 		},
2025-07-24T04:31:49.0236431Z 		{
2025-07-24T04:31:49.0237030Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0237767Z 			"size": 1897,
2025-07-24T04:31:49.0238475Z 			"digest": "sha256:83772e66e52fac49c784a3a4dee3cad95c1653ee093350b48399492da564b335"
2025-07-24T04:31:49.0239173Z 		},
2025-07-24T04:31:49.0239480Z 		{
2025-07-24T04:31:49.0240351Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0240997Z 			"size": 197493594,
2025-07-24T04:31:49.0241651Z 			"digest": "sha256:eee6d9689c2d5b11deada1d789902ffa6c32c42bd82c1c99df6264532d0af8d8"
2025-07-24T04:31:49.0242366Z 		},
2025-07-24T04:31:49.0242659Z 		{
2025-07-24T04:31:49.0243165Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0243789Z 			"size": 303,
2025-07-24T04:31:49.0244389Z 			"digest": "sha256:40f61293f85973eb3d6df45c40671ffd92439d1bc29f39c887e4342e5bb05461"
2025-07-24T04:31:49.0245071Z 		},
2025-07-24T04:31:49.0245369Z 		{
2025-07-24T04:31:49.0245866Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0246482Z 			"size": 32,
2025-07-24T04:31:49.0247387Z 			"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
2025-07-24T04:31:49.0248109Z 		},
2025-07-24T04:31:49.0248409Z 		{
2025-07-24T04:31:49.0248894Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0249533Z 			"size": 108,
2025-07-24T04:31:49.0250157Z 			"digest": "sha256:dfc71386efac7d21b798410f8f2ead070171f7f687d570ebf35eed00ecaf48c8"
2025-07-24T04:31:49.0250867Z 		},
2025-07-24T04:31:49.0251170Z 		{
2025-07-24T04:31:49.0251662Z 			"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
2025-07-24T04:31:49.0252301Z 			"size": 54145699,
2025-07-24T04:31:49.0252959Z 			"digest": "sha256:f4813cdb60f2c95fd11144ffcc01f5908a02cabb3606fc1f84f9b6bdf44177e0"
2025-07-24T04:31:49.0253678Z 		}
2025-07-24T04:31:49.0253986Z 	]
2025-07-24T04:31:49.0254285Z }
2025-07-24T04:31:49.0301824Z ##[group]Run set -eux
2025-07-24T04:31:49.0302272Z set -eux
2025-07-24T04:31:49.0303620Z aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token | jq --raw-output '.SecretString' | jq -r .docker_hub_readonly_token | docker login --username pytorchbot --password-stdin
2025-07-24T04:31:49.0365431Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:49.0365745Z env:
2025-07-24T04:31:49.0365932Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:49.0366262Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:31:49.0366744Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:31:49.0367176Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:31:49.0367891Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:31:49.0368525Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:31:49.0368743Z   AWS_REGION: us-east-1
2025-07-24T04:31:49.0369039Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:31:49.0369335Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:31:49.0373590Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:31:49.0373789Z ##[endgroup]
2025-07-24T04:31:49.0445148Z + aws secretsmanager get-secret-value --secret-id docker_hub_readonly_token
2025-07-24T04:31:49.0447039Z + jq --raw-output .SecretString
2025-07-24T04:31:49.0452491Z + jq -r .docker_hub_readonly_token
2025-07-24T04:31:49.0455541Z + docker login --username pytorchbot --password-stdin
2025-07-24T04:31:49.7912072Z 
2025-07-24T04:31:49.7914927Z An error occurred (AccessDeniedException) when calling the GetSecretValue operation: User: arn:aws:sts::308535385114:assumed-role/gha_workflow_s3_and_ecr_read_only/GitHubActions is not authorized to perform: secretsmanager:GetSecretValue on resource: docker_hub_readonly_token because no identity-based policy allows the secretsmanager:GetSecretValue action
2025-07-24T04:31:49.9078227Z Error: Cannot perform an interactive login from a non TTY device
2025-07-24T04:31:49.9159009Z ##[error]Process completed with exit code 1.
2025-07-24T04:31:49.9350595Z ##[group]Run pytorch/test-infra/.github/actions/pull-docker-image@main
2025-07-24T04:31:49.9351432Z with:
2025-07-24T04:31:49.9353131Z   docker-image: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:31:49.9354794Z   docker-registry: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-07-24T04:31:49.9355474Z env:
2025-07-24T04:31:49.9355872Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:49.9356639Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:31:49.9357777Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:31:49.9358839Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:31:49.9360661Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:31:49.9362237Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:31:49.9362738Z   AWS_REGION: us-east-1
2025-07-24T04:31:49.9363398Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:31:49.9364088Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:31:49.9374472Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:31:49.9374939Z ##[endgroup]
2025-07-24T04:31:49.9404886Z ##[group]Run set -x
2025-07-24T04:31:49.9405360Z set -x
2025-07-24T04:31:49.9405964Z set +e
2025-07-24T04:31:49.9406436Z 
2025-07-24T04:31:49.9407238Z login() {
2025-07-24T04:31:49.9408249Z   aws ecr get-login-password --region us-east-1 | docker login -u AWS --password-stdin "$1"
2025-07-24T04:31:49.9409281Z }
2025-07-24T04:31:49.9409936Z 
2025-07-24T04:31:49.9410465Z retry () {
2025-07-24T04:31:49.9411111Z   $*  || (sleep 1 && $*) || (sleep 2 && $*)
2025-07-24T04:31:49.9411941Z }
2025-07-24T04:31:49.9412394Z 
2025-07-24T04:31:49.9412977Z retry login "${DOCKER_REGISTRY}"
2025-07-24T04:31:49.9413797Z 
2025-07-24T04:31:49.9441663Z IMAGE_SIZE=$(docker manifest inspect "${DOCKER_IMAGE}" | jq '[.layers[].size, .config.size] | add / 1024 / 1024')
2025-07-24T04:31:49.9442810Z echo "Compressed size of image in MB: ${IMAGE_SIZE}"
2025-07-24T04:31:49.9443410Z 
2025-07-24T04:31:49.9443759Z set -e
2025-07-24T04:31:49.9444307Z # ignore output since only exit code is used for conditional
2025-07-24T04:31:49.9445060Z # only pull docker image if it's not available locally
2025-07-24T04:31:49.9445905Z if ! docker inspect --type=image "${DOCKER_IMAGE}" >/dev/null 2>/dev/null; then
2025-07-24T04:31:49.9446683Z   retry docker pull "${DOCKER_IMAGE}"
2025-07-24T04:31:49.9447212Z fi
2025-07-24T04:31:49.9506997Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:49.9507621Z env:
2025-07-24T04:31:49.9507998Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:49.9508685Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:31:49.9509687Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:31:49.9510610Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:31:49.9512139Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:31:49.9513493Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:31:49.9513951Z   AWS_REGION: us-east-1
2025-07-24T04:31:49.9514763Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:31:49.9515377Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:31:49.9524574Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:31:49.9526125Z   DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:31:49.9527484Z   DOCKER_REGISTRY: 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-07-24T04:31:49.9528087Z ##[endgroup]
2025-07-24T04:31:49.9620136Z + set +e
2025-07-24T04:31:49.9621552Z + retry login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-07-24T04:31:49.9622497Z + login 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-07-24T04:31:49.9632334Z + aws ecr get-login-password --region us-east-1
2025-07-24T04:31:49.9633433Z + docker login -u AWS --password-stdin 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-07-24T04:31:51.0867553Z WARNING! Your password will be stored unencrypted in /var/home/pytorchci/.docker/config.json.
2025-07-24T04:31:51.0868728Z Configure a credential helper to remove this warning. See
2025-07-24T04:31:51.0869743Z https://docs.docker.com/engine/reference/commandline/login/#credential-stores
2025-07-24T04:31:51.0870415Z 
2025-07-24T04:31:51.0877613Z Login Succeeded
2025-07-24T04:31:51.0950616Z ++ docker manifest inspect 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:31:51.0952267Z ++ jq '[.layers[].size, .config.size] | add / 1024 / 1024'
2025-07-24T04:31:51.7920214Z + IMAGE_SIZE=21755.0470495224
2025-07-24T04:31:51.7920885Z Compressed size of image in MB: 21755.0470495224
2025-07-24T04:31:51.7921675Z + echo 'Compressed size of image in MB: 21755.0470495224'
2025-07-24T04:31:51.7922330Z + set -e
2025-07-24T04:31:51.7923672Z + docker inspect --type=image 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:31:51.8398344Z Prepare all required actions
2025-07-24T04:31:51.8460260Z ##[group]Run ./.github/actions/get-workflow-job-id
2025-07-24T04:31:51.8460912Z with:
2025-07-24T04:31:51.8461744Z   github-token: ***
2025-07-24T04:31:51.8462202Z env:
2025-07-24T04:31:51.8462625Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:51.8463440Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:31:51.8464679Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:31:51.8465793Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:31:51.8467635Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:31:51.8469287Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:31:51.8469823Z   AWS_REGION: us-east-1
2025-07-24T04:31:51.8470499Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:31:51.8471224Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:31:51.8482801Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:31:51.8483311Z ##[endgroup]
2025-07-24T04:31:51.8514953Z ##[group]Run set -eux
2025-07-24T04:31:51.8515485Z set -eux
2025-07-24T04:31:51.8516359Z python3 .github/scripts/get_workflow_job_id.py "${GITHUB_RUN_ID}" "${RUNNER_NAME}"
2025-07-24T04:31:51.8581560Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:31:51.8582181Z env:
2025-07-24T04:31:51.8582565Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:51.8583251Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:31:51.8584270Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:31:51.8585236Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:31:51.8586808Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:31:51.8588185Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:31:51.8588646Z   AWS_REGION: us-east-1
2025-07-24T04:31:51.8589163Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:31:51.8589859Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:31:51.8599238Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:31:51.8599852Z   GITHUB_TOKEN: ***
2025-07-24T04:31:51.8600247Z ##[endgroup]
2025-07-24T04:31:51.8693655Z + python3 .github/scripts/get_workflow_job_id.py 16487210679 gpud553
2025-07-24T04:31:52.7772237Z Setting output job-id=46615439132
2025-07-24T04:31:52.7773249Z Setting output job-name=linux-jammy-rocm-py3.10 / test (default, 2, 2, linux.rocm.gpu.2)
2025-07-24T04:31:52.8036863Z Prepare all required actions
2025-07-24T04:31:52.8037628Z Getting action download info
2025-07-24T04:31:52.9509509Z Download action repository 'seemethere/download-artifact-s3@v4' (SHA:1da556a7aa0a088e3153970611f6c432d58e80e6)
2025-07-24T04:31:53.5898421Z Download action repository 'actions/download-artifact@v4' (SHA:d3f86a106a0bac45b974a628896c90dbdf5c8093)
2025-07-24T04:31:54.2393585Z ##[group]Run ./.github/actions/download-build-artifacts
2025-07-24T04:31:54.2393887Z with:
2025-07-24T04:31:54.2394074Z   name: linux-jammy-rocm-py3.10
2025-07-24T04:31:54.2394533Z   s3-bucket: gha-artifacts
2025-07-24T04:31:54.2394736Z env:
2025-07-24T04:31:54.2394908Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:54.2395223Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:31:54.2395716Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:31:54.2396224Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:31:54.2396953Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:31:54.2397592Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:31:54.2397808Z   AWS_REGION: us-east-1
2025-07-24T04:31:54.2398077Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:31:54.2398420Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:31:54.2403662Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:31:54.2403913Z ##[endgroup]
2025-07-24T04:31:54.2431932Z ##[group]Run seemethere/download-artifact-s3@v4
2025-07-24T04:31:54.2432179Z with:
2025-07-24T04:31:54.2432358Z   name: linux-jammy-rocm-py3.10
2025-07-24T04:31:54.2432582Z   s3-bucket: gha-artifacts
2025-07-24T04:31:54.2432795Z   region: us-east-1
2025-07-24T04:31:54.2432963Z env:
2025-07-24T04:31:54.2433124Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:31:54.2433442Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:31:54.2433903Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:31:54.2434391Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:31:54.2435137Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:31:54.2435783Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:31:54.2436002Z   AWS_REGION: us-east-1
2025-07-24T04:31:54.2436235Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:31:54.2436517Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:31:54.2441558Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:31:54.2441796Z ##[endgroup]
2025-07-24T04:31:54.6900588Z (node:3202559) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.
2025-07-24T04:31:54.6901519Z 
2025-07-24T04:31:54.6901930Z Please migrate your code to use AWS SDK for JavaScript (v3).
2025-07-24T04:31:54.6902901Z For more information, check the migration guide at https://a.co/7PzMCcy
2025-07-24T04:31:54.6903878Z (Use `node --trace-warnings ...` to show where the warning was created)
2025-07-24T04:31:54.8496715Z Found 1 objects with prefix pytorch/pytorch/16487210679/linux-jammy-rocm-py3.10/
2025-07-24T04:31:54.8498032Z Starting download (1/1): /var/home/pytorchci/actions-runner/_work/pytorch/pytorch/artifacts.zip
2025-07-24T04:32:15.6315391Z Finished download (1/1): /var/home/pytorchci/actions-runner/_work/pytorch/pytorch/artifacts.zip
2025-07-24T04:32:15.6326815Z Artifact download has finished successfully
2025-07-24T04:32:15.6899431Z ##[group]Run unzip -o artifacts.zip
2025-07-24T04:32:15.6899914Z unzip -o artifacts.zip
2025-07-24T04:32:15.6951030Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:32:15.6951567Z env:
2025-07-24T04:32:15.6951877Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:32:15.6952857Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:32:15.6953757Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:32:15.6954818Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:32:15.6956167Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:32:15.6957363Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:32:15.6957759Z   AWS_REGION: us-east-1
2025-07-24T04:32:15.6958226Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:32:15.6958761Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:32:15.6967020Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:32:15.6967279Z ##[endgroup]
2025-07-24T04:32:15.7120194Z Archive:  artifacts.zip
2025-07-24T04:32:15.7121665Z    creating: dist/
2025-07-24T04:32:21.0382858Z   inflating: dist/torch-2.9.0a0+git27c8ef1-cp310-cp310-linux_x86_64.whl  
2025-07-24T04:32:21.0510247Z   inflating: dist/.ninja_log         
2025-07-24T04:32:21.0510961Z    creating: build/custom_test_artifacts/
2025-07-24T04:32:21.0511719Z    creating: build/custom_test_artifacts/custom-op-build/
2025-07-24T04:32:21.0512638Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/
2025-07-24T04:32:21.0513670Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/pkgRedirects/
2025-07-24T04:32:21.0515479Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeConfigureLog.yaml  
2025-07-24T04:32:21.0516628Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/
2025-07-24T04:32:21.0517738Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CMakeSystem.cmake  
2025-07-24T04:32:21.0518963Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdC/
2025-07-24T04:32:21.0520131Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdC/tmp/
2025-07-24T04:32:21.0521452Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdC/CMakeCCompilerId.c  
2025-07-24T04:32:21.0522818Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdC/a.out  
2025-07-24T04:32:21.0524098Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CMakeCCompiler.cmake  
2025-07-24T04:32:21.0525317Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdCXX/
2025-07-24T04:32:21.0526481Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdCXX/tmp/
2025-07-24T04:32:21.0527864Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdCXX/CMakeCXXCompilerId.cpp  
2025-07-24T04:32:21.0529296Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CompilerIdCXX/a.out  
2025-07-24T04:32:21.0530587Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CMakeCXXCompiler.cmake  
2025-07-24T04:32:21.0532073Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CMakeDetermineCompilerABI_C.bin  
2025-07-24T04:32:21.0533572Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/4.0.0/CMakeDetermineCompilerABI_CXX.bin  
2025-07-24T04:32:21.0534864Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeScratch/
2025-07-24T04:32:21.0535892Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeTmp/
2025-07-24T04:32:21.0536961Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/cmake.check_cache  
2025-07-24T04:32:21.0538090Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/
2025-07-24T04:32:21.0539309Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.ts  
2025-07-24T04:32:21.0541493Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/compiler_depend.make  
2025-07-24T04:32:21.0543268Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/depend.make  
2025-07-24T04:32:21.0544553Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/link.txt  
2025-07-24T04:32:21.0545835Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/cmake_clean.cmake  
2025-07-24T04:32:21.0547131Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/build.make  
2025-07-24T04:32:21.0548484Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/DependInfo.cmake  
2025-07-24T04:32:21.0549809Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/flags.make  
2025-07-24T04:32:21.0551121Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/progress.make  
2025-07-24T04:32:21.0556196Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o.d  
2025-07-24T04:32:21.0733987Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/custom_ops.dir/op.cpp.o  
2025-07-24T04:32:21.0735390Z    creating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/
2025-07-24T04:32:21.0736783Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.ts  
2025-07-24T04:32:21.0738295Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/compiler_depend.make  
2025-07-24T04:32:21.0739792Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/depend.make  
2025-07-24T04:32:21.0741140Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/link.txt  
2025-07-24T04:32:21.0742571Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/cmake_clean.cmake  
2025-07-24T04:32:21.0743984Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/build.make  
2025-07-24T04:32:21.0745369Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/DependInfo.cmake  
2025-07-24T04:32:21.0746751Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/flags.make  
2025-07-24T04:32:21.0748113Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/progress.make  
2025-07-24T04:32:21.0759098Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o.d  
2025-07-24T04:32:21.0833605Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/test_custom_ops.dir/test_custom_ops.cpp.o  
2025-07-24T04:32:21.0835423Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/CMakeDirectoryInformation.cmake  
2025-07-24T04:32:21.0836877Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/TargetDirectories.txt  
2025-07-24T04:32:21.0838171Z  extracting: build/custom_test_artifacts/custom-op-build/CMakeFiles/progress.marks  
2025-07-24T04:32:21.0839319Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile2  
2025-07-24T04:32:21.0840453Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/Makefile.cmake  
2025-07-24T04:32:21.0841650Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeFiles/InstallScripts.json  
2025-07-24T04:32:21.0842859Z   inflating: build/custom_test_artifacts/custom-op-build/hipblaslt_test_outer_vec.cc  
2025-07-24T04:32:21.0843991Z   inflating: build/custom_test_artifacts/custom-op-build/hipblaslt_test_vec_ext.cc  
2025-07-24T04:32:21.0845046Z   inflating: build/custom_test_artifacts/custom-op-build/CMakeCache.txt  
2025-07-24T04:32:21.0845992Z   inflating: build/custom_test_artifacts/custom-op-build/Makefile  
2025-07-24T04:32:21.0847519Z   inflating: build/custom_test_artifacts/custom-op-build/cmake_install.cmake  
2025-07-24T04:32:21.0990120Z   inflating: build/custom_test_artifacts/custom-op-build/libcustom_ops.so  
2025-07-24T04:32:21.1041286Z   inflating: build/custom_test_artifacts/custom-op-build/test_custom_ops  
2025-07-24T04:32:21.1042325Z    creating: build/custom_test_artifacts/jit-hook-build/
2025-07-24T04:32:21.1043212Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/
2025-07-24T04:32:21.1044229Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/pkgRedirects/
2025-07-24T04:32:21.1045473Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeConfigureLog.yaml  
2025-07-24T04:32:21.1046621Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/
2025-07-24T04:32:21.1047719Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CMakeSystem.cmake  
2025-07-24T04:32:21.1048893Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdC/
2025-07-24T04:32:21.1050064Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdC/tmp/
2025-07-24T04:32:21.1051402Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdC/CMakeCCompilerId.c  
2025-07-24T04:32:21.1052748Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdC/a.out  
2025-07-24T04:32:21.1054002Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CMakeCCompiler.cmake  
2025-07-24T04:32:21.1055262Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdCXX/
2025-07-24T04:32:21.1056457Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdCXX/tmp/
2025-07-24T04:32:21.1057858Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdCXX/CMakeCXXCompilerId.cpp  
2025-07-24T04:32:21.1059270Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CompilerIdCXX/a.out  
2025-07-24T04:32:21.1060566Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CMakeCXXCompiler.cmake  
2025-07-24T04:32:21.1061936Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CMakeDetermineCompilerABI_C.bin  
2025-07-24T04:32:21.1063447Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/4.0.0/CMakeDetermineCompilerABI_CXX.bin  
2025-07-24T04:32:21.1064715Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeScratch/
2025-07-24T04:32:21.1065761Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeTmp/
2025-07-24T04:32:21.1066833Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/cmake.check_cache  
2025-07-24T04:32:21.1067966Z    creating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/
2025-07-24T04:32:21.1069248Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.ts  
2025-07-24T04:32:21.1070719Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/compiler_depend.make  
2025-07-24T04:32:21.1072132Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/depend.make  
2025-07-24T04:32:21.1073432Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/link.txt  
2025-07-24T04:32:21.1075209Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/cmake_clean.cmake  
2025-07-24T04:32:21.1076568Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/build.make  
2025-07-24T04:32:21.1077916Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/DependInfo.cmake  
2025-07-24T04:32:21.1079262Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/flags.make  
2025-07-24T04:32:21.1080608Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/progress.make  
2025-07-24T04:32:21.1084753Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o.d  
2025-07-24T04:32:21.1143564Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/test_jit_hooks.dir/test_jit_hooks.cpp.o  
2025-07-24T04:32:21.1145086Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/CMakeDirectoryInformation.cmake  
2025-07-24T04:32:21.1146431Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/TargetDirectories.txt  
2025-07-24T04:32:21.1147658Z  extracting: build/custom_test_artifacts/jit-hook-build/CMakeFiles/progress.marks  
2025-07-24T04:32:21.1148771Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile2  
2025-07-24T04:32:21.1149860Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/Makefile.cmake  
2025-07-24T04:32:21.1151005Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeFiles/InstallScripts.json  
2025-07-24T04:32:21.1152182Z   inflating: build/custom_test_artifacts/jit-hook-build/hipblaslt_test_outer_vec.cc  
2025-07-24T04:32:21.1153282Z   inflating: build/custom_test_artifacts/jit-hook-build/hipblaslt_test_vec_ext.cc  
2025-07-24T04:32:21.1154613Z   inflating: build/custom_test_artifacts/jit-hook-build/CMakeCache.txt  
2025-07-24T04:32:21.1155570Z   inflating: build/custom_test_artifacts/jit-hook-build/Makefile  
2025-07-24T04:32:21.1156528Z   inflating: build/custom_test_artifacts/jit-hook-build/cmake_install.cmake  
2025-07-24T04:32:21.1185624Z   inflating: build/custom_test_artifacts/jit-hook-build/test_jit_hooks  
2025-07-24T04:32:21.1186549Z    creating: build/custom_test_artifacts/custom-backend-build/
2025-07-24T04:32:21.1187467Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/
2025-07-24T04:32:21.1188530Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/pkgRedirects/
2025-07-24T04:32:21.1190296Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeConfigureLog.yaml  
2025-07-24T04:32:21.1191518Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/
2025-07-24T04:32:21.1192734Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CMakeSystem.cmake  
2025-07-24T04:32:21.1194012Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdC/
2025-07-24T04:32:21.1195430Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdC/tmp/
2025-07-24T04:32:21.1196856Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdC/CMakeCCompilerId.c  
2025-07-24T04:32:21.1198318Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdC/a.out  
2025-07-24T04:32:21.1199687Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CMakeCCompiler.cmake  
2025-07-24T04:32:21.1201001Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdCXX/
2025-07-24T04:32:21.1202286Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdCXX/tmp/
2025-07-24T04:32:21.1203773Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdCXX/CMakeCXXCompilerId.cpp  
2025-07-24T04:32:21.1205304Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CompilerIdCXX/a.out  
2025-07-24T04:32:21.1206688Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CMakeCXXCompiler.cmake  
2025-07-24T04:32:21.1208185Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CMakeDetermineCompilerABI_C.bin  
2025-07-24T04:32:21.1209783Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/4.0.0/CMakeDetermineCompilerABI_CXX.bin  
2025-07-24T04:32:21.1211166Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeScratch/
2025-07-24T04:32:21.1212303Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeTmp/
2025-07-24T04:32:21.1213922Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/cmake.check_cache  
2025-07-24T04:32:21.1215451Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/
2025-07-24T04:32:21.1216874Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.ts  
2025-07-24T04:32:21.1218440Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/compiler_depend.make  
2025-07-24T04:32:21.1219943Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/depend.make  
2025-07-24T04:32:21.1221350Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/link.txt  
2025-07-24T04:32:21.1222819Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/cmake_clean.cmake  
2025-07-24T04:32:21.1224315Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/build.make  
2025-07-24T04:32:21.1225808Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/DependInfo.cmake  
2025-07-24T04:32:21.1227293Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/flags.make  
2025-07-24T04:32:21.1228742Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/progress.make  
2025-07-24T04:32:21.1230295Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o.d  
2025-07-24T04:32:21.1325717Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/custom_backend.dir/custom_backend.cpp.o  
2025-07-24T04:32:21.1327198Z    creating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/
2025-07-24T04:32:21.1328659Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.ts  
2025-07-24T04:32:21.1330310Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/compiler_depend.make  
2025-07-24T04:32:21.1331892Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/depend.make  
2025-07-24T04:32:21.1333396Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/link.txt  
2025-07-24T04:32:21.1334907Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/cmake_clean.cmake  
2025-07-24T04:32:21.1336440Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/build.make  
2025-07-24T04:32:21.1337998Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/DependInfo.cmake  
2025-07-24T04:32:21.1339524Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/flags.make  
2025-07-24T04:32:21.1341047Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/progress.make  
2025-07-24T04:32:21.1350561Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o.d  
2025-07-24T04:32:21.1400982Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/test_custom_backend.dir/test_custom_backend.cpp.o  
2025-07-24T04:32:21.1402618Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/CMakeDirectoryInformation.cmake  
2025-07-24T04:32:21.1404054Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/TargetDirectories.txt  
2025-07-24T04:32:21.1405343Z  extracting: build/custom_test_artifacts/custom-backend-build/CMakeFiles/progress.marks  
2025-07-24T04:32:21.1406541Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile2  
2025-07-24T04:32:21.1407699Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/Makefile.cmake  
2025-07-24T04:32:21.1409343Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeFiles/InstallScripts.json  
2025-07-24T04:32:21.1410900Z   inflating: build/custom_test_artifacts/custom-backend-build/hipblaslt_test_outer_vec.cc  
2025-07-24T04:32:21.1412098Z   inflating: build/custom_test_artifacts/custom-backend-build/hipblaslt_test_vec_ext.cc  
2025-07-24T04:32:21.1413221Z   inflating: build/custom_test_artifacts/custom-backend-build/CMakeCache.txt  
2025-07-24T04:32:21.1414246Z   inflating: build/custom_test_artifacts/custom-backend-build/Makefile  
2025-07-24T04:32:21.1415287Z   inflating: build/custom_test_artifacts/custom-backend-build/cmake_install.cmake  
2025-07-24T04:32:21.1501148Z   inflating: build/custom_test_artifacts/custom-backend-build/libcustom_backend.so  
2025-07-24T04:32:21.1536571Z   inflating: build/custom_test_artifacts/custom-backend-build/test_custom_backend  
2025-07-24T04:32:21.1537416Z    creating: build/lib/
2025-07-24T04:32:21.1612294Z   inflating: build/lib/libprotobuf-lite.a  
2025-07-24T04:32:21.2017409Z   inflating: build/lib/libprotobuf.a  
2025-07-24T04:32:21.2467300Z   inflating: build/lib/libprotoc.a   
2025-07-24T04:32:21.2475732Z   inflating: build/lib/libpthreadpool.a  
2025-07-24T04:32:21.2483367Z   inflating: build/lib/libcpuinfo.a  
2025-07-24T04:32:21.2490558Z   inflating: build/lib/libcpuinfo_internals.a  
2025-07-24T04:32:21.2491308Z   inflating: build/lib/libclog.a     
2025-07-24T04:32:21.2508346Z   inflating: build/lib/libpytorch_qnnpack.a  
2025-07-24T04:32:21.2509641Z   inflating: build/lib/libnnpack_reference_layers.a  
2025-07-24T04:32:21.2526288Z   inflating: build/lib/libnnpack.a   
2025-07-24T04:32:21.2694961Z   inflating: build/lib/libmicrokernels-prod.a  
2025-07-24T04:32:21.3485526Z   inflating: build/lib/libmicrokernels-all.a  
2025-07-24T04:32:21.3548922Z   inflating: build/lib/libgtest.a    
2025-07-24T04:32:21.3564491Z   inflating: build/lib/libgmock.a    
2025-07-24T04:32:21.3565196Z   inflating: build/lib/libgtest_main.a  
2025-07-24T04:32:21.3647334Z   inflating: build/lib/libXNNPACK.a  
2025-07-24T04:32:21.3648029Z   inflating: build/lib/libgmock_main.a  
2025-07-24T04:32:21.3715939Z   inflating: build/lib/libbenchmark.a  
2025-07-24T04:32:21.3716687Z   inflating: build/lib/libbenchmark_main.a  
2025-07-24T04:32:21.3723773Z   inflating: build/lib/libittnotify.a  
2025-07-24T04:32:21.3724501Z   inflating: build/lib/libjitprofiling.a  
2025-07-24T04:32:21.3782709Z   inflating: build/lib/libasmjit.a   
2025-07-24T04:32:21.4884021Z   inflating: build/lib/libfbgemm.a   
2025-07-24T04:32:21.4908496Z   inflating: build/lib/libtensorpipe_uv.a  
2025-07-24T04:32:21.5415168Z   inflating: build/lib/libtensorpipe.a  
2025-07-24T04:32:21.5524491Z   inflating: build/lib/libgloo.a     
2025-07-24T04:32:21.5567643Z   inflating: build/lib/libonnx_proto.a  
2025-07-24T04:32:21.5967633Z   inflating: build/lib/libgloo_hip.a  
2025-07-24T04:32:21.6623160Z   inflating: build/lib/libonnx.a     
2025-07-24T04:32:22.6055475Z   inflating: build/lib/libdnnl.a     
2025-07-24T04:32:22.6071685Z   inflating: build/lib/libfmt.a      
2025-07-24T04:32:22.6333579Z   inflating: build/lib/libkineto.a   
2025-07-24T04:32:22.6429350Z   inflating: build/lib/libc10.so     
2025-07-24T04:32:22.6430088Z   inflating: build/lib/libtorch_global_deps.so  
2025-07-24T04:32:22.6476431Z   inflating: build/lib/libc10_hip.so  
2025-07-24T04:32:22.6477149Z   inflating: build/lib/libcaffe2_nvrtc.so  
2025-07-24T04:32:25.6835640Z   inflating: build/lib/libtorch_cpu.so  
2025-07-24T04:32:25.6838082Z   inflating: build/lib/libshm.so     
2025-07-24T04:32:26.4552934Z   inflating: build/lib/libtorch_hip.so  
2025-07-24T04:32:26.4553641Z   inflating: build/lib/libtorch.so   
2025-07-24T04:32:26.4570882Z   inflating: build/lib/libjitbackend_test.so  
2025-07-24T04:32:26.4592192Z   inflating: build/lib/libbackend_with_compiler.so  
2025-07-24T04:32:26.4655660Z   inflating: build/lib/libtorchbind_test.so  
2025-07-24T04:32:26.4679509Z   inflating: build/lib/libaoti_custom_ops.so  
2025-07-24T04:32:26.6520040Z   inflating: build/lib/libtorch_python.so  
2025-07-24T04:32:26.6549944Z   inflating: build/lib/libnnapi_backend.so  
2025-07-24T04:32:26.6550623Z    creating: build/bin/
2025-07-24T04:32:26.6551676Z    creating: build/bin/CMakeFiles/
2025-07-24T04:32:26.6552404Z   inflating: build/bin/cmake_install.cmake  
2025-07-24T04:32:26.6553030Z   inflating: build/bin/CTestTestfile.cmake  
2025-07-24T04:32:26.6958024Z   inflating: build/bin/protoc-3.13.0.0  
2025-07-24T04:32:26.7365283Z   inflating: build/bin/protoc        
2025-07-24T04:32:26.7414684Z   inflating: build/bin/c10_CompileTimeFunctionPointer_test  
2025-07-24T04:32:26.7465535Z   inflating: build/bin/c10_DeviceGuard_test  
2025-07-24T04:32:26.7516353Z   inflating: build/bin/c10_Device_test  
2025-07-24T04:32:26.7574981Z   inflating: build/bin/c10_DispatchKeySet_test  
2025-07-24T04:32:26.7628245Z   inflating: build/bin/c10_Scalar_test  
2025-07-24T04:32:26.7677560Z   inflating: build/bin/c10_StreamGuard_test  
2025-07-24T04:32:26.7727040Z   inflating: build/bin/c10_SymInt_test  
2025-07-24T04:32:26.7781522Z   inflating: build/bin/c10_InlineDeviceGuard_test  
2025-07-24T04:32:26.7836266Z   inflating: build/bin/c10_InlineStreamGuard_test  
2025-07-24T04:32:26.7884847Z   inflating: build/bin/c10_ConstexprCrc_test  
2025-07-24T04:32:26.7939947Z   inflating: build/bin/c10_SizesAndStrides_test  
2025-07-24T04:32:26.8008408Z   inflating: build/bin/c10_cow_test  
2025-07-24T04:32:26.8057828Z   inflating: build/bin/c10_DeadlockDetection_test  
2025-07-24T04:32:26.8110173Z   inflating: build/bin/c10_Bitset_test  
2025-07-24T04:32:26.8159191Z   inflating: build/bin/c10_ArrayRef_test  
2025-07-24T04:32:26.8215429Z   inflating: build/bin/c10_Enumerate_test  
2025-07-24T04:32:26.8267437Z   inflating: build/bin/c10_IntrusiveList_test  
2025-07-24T04:32:26.8322475Z   inflating: build/bin/c10_Metaprogramming_test  
2025-07-24T04:32:26.8372269Z   inflating: build/bin/c10_Half_test  
2025-07-24T04:32:26.8427583Z   inflating: build/bin/c10_LeftRight_test  
2025-07-24T04:32:26.8481077Z   inflating: build/bin/c10_NetworkFlow_test  
2025-07-24T04:32:26.8530392Z   inflating: build/bin/c10_Semaphore_test  
2025-07-24T04:32:26.8579837Z   inflating: build/bin/c10_Synchronized_test  
2025-07-24T04:32:26.8635148Z   inflating: build/bin/c10_ThreadLocal_test  
2025-07-24T04:32:26.8685876Z   inflating: build/bin/c10_TypeIndex_test  
2025-07-24T04:32:26.8734543Z   inflating: build/bin/c10_TypeTraits_test  
2025-07-24T04:32:26.8784866Z   inflating: build/bin/c10_TypeList_test  
2025-07-24T04:32:26.8836118Z   inflating: build/bin/c10_accumulate_test  
2025-07-24T04:32:26.8890719Z   inflating: build/bin/c10_bfloat16_test  
2025-07-24T04:32:26.8940786Z   inflating: build/bin/c10_bit_cast_test  
2025-07-24T04:32:26.8996886Z   inflating: build/bin/c10_complex_math_test  
2025-07-24T04:32:26.9045971Z   inflating: build/bin/c10_error_test  
2025-07-24T04:32:26.9100224Z   inflating: build/bin/c10_complex_test  
2025-07-24T04:32:26.9152089Z   inflating: build/bin/c10_exception_test  
2025-07-24T04:32:26.9201797Z   inflating: build/bin/c10_flags_test  
2025-07-24T04:32:26.9251766Z   inflating: build/bin/c10_generic_math_test  
2025-07-24T04:32:26.9302175Z   inflating: build/bin/c10_irange_test  
2025-07-24T04:32:26.9355085Z   inflating: build/bin/c10_lazy_test  
2025-07-24T04:32:26.9513001Z   inflating: build/bin/c10_intrusive_ptr_test  
2025-07-24T04:32:26.9569381Z   inflating: build/bin/c10_logging_test  
2025-07-24T04:32:26.9642571Z   inflating: build/bin/c10_optional_test  
2025-07-24T04:32:26.9703841Z   inflating: build/bin/c10_ordered_preserving_dict_test  
2025-07-24T04:32:26.9756420Z   inflating: build/bin/c10_registry_test  
2025-07-24T04:32:26.9807340Z   inflating: build/bin/c10_ssize_test  
2025-07-24T04:32:26.9956560Z   inflating: build/bin/c10_small_vector_test  
2025-07-24T04:32:27.0011932Z   inflating: build/bin/c10_string_util_test  
2025-07-24T04:32:27.0061656Z   inflating: build/bin/c10_tempfile_test  
2025-07-24T04:32:27.0110080Z   inflating: build/bin/c10_string_view_test  
2025-07-24T04:32:27.0153252Z   inflating: build/bin/c10_intrusive_ptr_benchmark  
2025-07-24T04:32:27.0208499Z   inflating: build/bin/c10_typeid_test  
2025-07-24T04:32:27.0257892Z   inflating: build/bin/c10_hip_HIPAssertionsTest_1_var_test  
2025-07-24T04:32:27.0305756Z   inflating: build/bin/c10_hip_HIPAssertionsTest_catches_stream  
2025-07-24T04:32:27.0354765Z   inflating: build/bin/c10_hip_HIPAssertionsTest_catches_thread_and_block_and_device  
2025-07-24T04:32:27.0403038Z   inflating: build/bin/c10_hip_HIPAssertionsTest_from_2_processes  
2025-07-24T04:32:27.0451851Z   inflating: build/bin/c10_hip_HIPAssertionsTest_multiple_writes_from_blocks_and_threads  
2025-07-24T04:32:27.0500161Z   inflating: build/bin/c10_hip_HIPAssertionsTest_multiple_writes_from_multiple_blocks  
2025-07-24T04:32:27.0548883Z   inflating: build/bin/c10_hip_HIPTest  
2025-07-24T04:32:27.0597939Z   inflating: build/bin/c10_hip_HIPAssertionsTest_multiple_writes_from_same_block  
2025-07-24T04:32:27.1615234Z   inflating: build/bin/vec_test_all_types_DEFAULT  
2025-07-24T04:32:27.2646196Z   inflating: build/bin/vec_test_all_types_AVX512  
2025-07-24T04:32:27.3682716Z   inflating: build/bin/vec_test_all_types_AVX2  
2025-07-24T04:32:27.3734381Z   inflating: build/bin/BackoffTest   
2025-07-24T04:32:27.3786947Z   inflating: build/bin/FileStoreTest  
2025-07-24T04:32:27.3843312Z   inflating: build/bin/TCPStoreTest  
2025-07-24T04:32:27.3896578Z   inflating: build/bin/HashStoreTest  
2025-07-24T04:32:27.3960893Z   inflating: build/bin/ProcessGroupGlooTest  
2025-07-24T04:32:27.3962600Z   inflating: build/bin/example_allreduce  
2025-07-24T04:32:27.3967068Z   inflating: build/bin/torch_shm_manager  
2025-07-24T04:32:27.4019636Z   inflating: build/bin/static_runtime_bench  
2025-07-24T04:32:27.4257326Z   inflating: build/bin/static_runtime_test  
2025-07-24T04:32:27.4329160Z   inflating: build/bin/Dict_test     
2025-07-24T04:32:27.4381157Z   inflating: build/bin/Dimname_test  
2025-07-24T04:32:27.4444941Z   inflating: build/bin/MaybeOwned_test  
2025-07-24T04:32:27.4500822Z   inflating: build/bin/NamedTensor_test  
2025-07-24T04:32:27.4558272Z   inflating: build/bin/apply_utils_test  
2025-07-24T04:32:27.4616228Z   inflating: build/bin/atest         
2025-07-24T04:32:27.4678488Z   inflating: build/bin/basic         
2025-07-24T04:32:27.4733117Z   inflating: build/bin/broadcast_test  
2025-07-24T04:32:27.4783104Z   inflating: build/bin/cpu_allocator_test  
2025-07-24T04:32:27.4840285Z   inflating: build/bin/cpu_generator_test  
2025-07-24T04:32:27.4892330Z   inflating: build/bin/cpu_profiling_allocator_test  
2025-07-24T04:32:27.4981849Z   inflating: build/bin/cpu_rng_test  
2025-07-24T04:32:27.5032140Z   inflating: build/bin/dlconvertor_test  
2025-07-24T04:32:27.5088619Z   inflating: build/bin/extension_backend_test  
2025-07-24T04:32:27.5143011Z   inflating: build/bin/half_test     
2025-07-24T04:32:27.5235316Z   inflating: build/bin/ivalue_test   
2025-07-24T04:32:27.5284281Z   inflating: build/bin/lazy_tensor_test  
2025-07-24T04:32:27.5337318Z   inflating: build/bin/math_kernel_test  
2025-07-24T04:32:27.5389751Z   inflating: build/bin/memory_format_test  
2025-07-24T04:32:27.5442793Z   inflating: build/bin/memory_overlapping_test  
2025-07-24T04:32:27.5495092Z   inflating: build/bin/mobile_memory_cleanup  
2025-07-24T04:32:27.5545644Z   inflating: build/bin/operator_name_test  
2025-07-24T04:32:27.5600650Z   inflating: build/bin/native_test   
2025-07-24T04:32:27.5650904Z   inflating: build/bin/operators_test  
2025-07-24T04:32:27.5702190Z   inflating: build/bin/packedtensoraccessor_test  
2025-07-24T04:32:27.5758629Z   inflating: build/bin/quantized_test  
2025-07-24T04:32:27.5823863Z   inflating: build/bin/pow_test      
2025-07-24T04:32:27.5873554Z   inflating: build/bin/reduce_ops_test  
2025-07-24T04:32:27.5923896Z   inflating: build/bin/reportMemoryUsage_test  
2025-07-24T04:32:27.5974743Z   inflating: build/bin/StorageUtils_test  
2025-07-24T04:32:27.6030290Z   inflating: build/bin/scalar_tensor_test  
2025-07-24T04:32:27.6087905Z   inflating: build/bin/scalar_test   
2025-07-24T04:32:27.6139134Z   inflating: build/bin/stride_properties_test  
2025-07-24T04:32:27.6216659Z   inflating: build/bin/tensor_iterator_test  
2025-07-24T04:32:27.6271237Z   inflating: build/bin/type_ptr_test  
2025-07-24T04:32:27.6272391Z   inflating: build/bin/thread_init_test  
2025-07-24T04:32:27.6327065Z   inflating: build/bin/test_parallel  
2025-07-24T04:32:27.6385081Z   inflating: build/bin/type_test     
2025-07-24T04:32:27.6437248Z   inflating: build/bin/undefined_tensor_test  
2025-07-24T04:32:27.6486133Z   inflating: build/bin/verify_api_visibility  
2025-07-24T04:32:27.6554128Z   inflating: build/bin/legacy_vmap_test  
2025-07-24T04:32:27.6604776Z   inflating: build/bin/weakref_test  
2025-07-24T04:32:27.6663381Z   inflating: build/bin/IListRef_test  
2025-07-24T04:32:27.6713977Z   inflating: build/bin/wrapdim_test  
2025-07-24T04:32:27.6765324Z   inflating: build/bin/xla_tensor_test  
2025-07-24T04:32:27.6867979Z   inflating: build/bin/List_test     
2025-07-24T04:32:27.6982675Z   inflating: build/bin/kernel_function_legacy_test  
2025-07-24T04:32:27.7102898Z   inflating: build/bin/kernel_lambda_legacy_test  
2025-07-24T04:32:27.7194697Z   inflating: build/bin/kernel_function_test  
2025-07-24T04:32:27.7259105Z   inflating: build/bin/KernelFunction_test  
2025-07-24T04:32:27.7357508Z   inflating: build/bin/kernel_lambda_test  
2025-07-24T04:32:27.7416458Z   inflating: build/bin/kernel_stackbased_test  
2025-07-24T04:32:27.7508393Z   inflating: build/bin/make_boxed_from_unboxed_functor_test  
2025-07-24T04:32:27.7558555Z   inflating: build/bin/CppSignature_test  
2025-07-24T04:32:27.7607302Z   inflating: build/bin/op_allowlist_test  
2025-07-24T04:32:27.7902306Z   inflating: build/bin/op_registration_test  
2025-07-24T04:32:27.7967459Z   inflating: build/bin/inline_container_test  
2025-07-24T04:32:27.8021899Z   inflating: build/bin/backend_fallback_test  
2025-07-24T04:32:27.8070223Z   inflating: build/bin/hip_complex_math_test  
2025-07-24T04:32:27.8119083Z   inflating: build/bin/hip_complex_test  
2025-07-24T04:32:27.8171193Z   inflating: build/bin/hip_apply_test  
2025-07-24T04:32:27.8219894Z   inflating: build/bin/hip_distributions_test  
2025-07-24T04:32:27.8268097Z   inflating: build/bin/hip_generator_test  
2025-07-24T04:32:27.8317001Z   inflating: build/bin/hip_half_test  
2025-07-24T04:32:27.8365894Z   inflating: build/bin/hip_integer_divider_test  
2025-07-24T04:32:27.8414650Z   inflating: build/bin/hip_optional_test  
2025-07-24T04:32:27.8463194Z   inflating: build/bin/hip_packedtensoraccessor_test  
2025-07-24T04:32:27.8512528Z   inflating: build/bin/hip_vectorized_test  
2025-07-24T04:32:27.8563614Z   inflating: build/bin/hip_dlconvertor_test  
2025-07-24T04:32:27.9576124Z   inflating: build/bin/test_jit      
2025-07-24T04:32:27.9835269Z   inflating: build/bin/test_nativert  
2025-07-24T04:32:28.0561173Z   inflating: build/bin/test_tensorexpr  
2025-07-24T04:32:28.0572520Z   inflating: build/bin/tutorial_tensorexpr  
2025-07-24T04:32:28.0626664Z   inflating: build/bin/test_dist_autograd  
2025-07-24T04:32:28.0692665Z   inflating: build/bin/test_cpp_rpc  
2025-07-24T04:32:28.0693954Z   inflating: build/bin/parallel_benchmark  
2025-07-24T04:32:28.1760116Z   inflating: build/bin/test_api      
2025-07-24T04:32:28.1824504Z   inflating: build/bin/test_mobile_nnc  
2025-07-24T04:32:28.1832407Z   inflating: build/bin/aot_model_compiler_test  
2025-07-24T04:32:28.2159018Z   inflating: build/bin/test_lazy     
2025-07-24T04:32:28.2159686Z    creating: .additional_ci_files/
2025-07-24T04:32:28.2305303Z   inflating: .additional_ci_files/test-times.json  
2025-07-24T04:32:28.2862705Z   inflating: .additional_ci_files/test-class-times.json  
2025-07-24T04:32:28.2919578Z ##[group]Run rm artifacts.zip
2025-07-24T04:32:28.2920119Z rm artifacts.zip
2025-07-24T04:32:28.2983677Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:32:28.2984370Z env:
2025-07-24T04:32:28.2984773Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:32:28.2986013Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:32:28.2987140Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:32:28.2988550Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:32:28.2990328Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:32:28.2991853Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:32:28.2992347Z   AWS_REGION: us-east-1
2025-07-24T04:32:28.2992941Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:32:28.2993618Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:32:28.3004318Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:32:28.3004786Z ##[endgroup]
2025-07-24T04:32:28.6549541Z ##[group]Run df -H
2025-07-24T04:32:28.6549987Z df -H
2025-07-24T04:32:28.6612958Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:32:28.6613659Z env:
2025-07-24T04:32:28.6614045Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:32:28.6614792Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:32:28.6615973Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:32:28.6617033Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:32:28.6619013Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:32:28.6620796Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:32:28.6621308Z   AWS_REGION: us-east-1
2025-07-24T04:32:28.6621860Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:32:28.6622542Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:32:28.6632883Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:32:28.6633331Z ##[endgroup]
2025-07-24T04:32:28.6791797Z Filesystem                                          Size  Used Avail Use% Mounted on
2025-07-24T04:32:28.6792699Z tmpfs                                               109G   33M  109G   1% /run
2025-07-24T04:32:28.6793537Z /dev/nvme0n1p2                                      944G   71G  825G   8% /
2025-07-24T04:32:28.6794663Z tmpfs                                               542G   33k  542G   1% /dev/shm
2025-07-24T04:32:28.6795369Z tmpfs                                               5.3M     0  5.3M   0% /run/lock
2025-07-24T04:32:28.6796101Z /dev/nvme0n1p1                                      536M  6.4M  530M   2% /boot/efi
2025-07-24T04:32:28.6796900Z /dev/nvme1n1p1                                      3.8T  2.2T  1.5T  61% /media/4TB
2025-07-24T04:32:28.6797665Z tmpfs                                               109G   33k  109G   1% /run/user/1307800118
2025-07-24T04:32:28.6798453Z 172.18.148.8:/export/amd2                           5.5T  278G  5.3T   6% /mnt
2025-07-24T04:32:28.6799544Z pure1.jax.cs.cpe.ice.amd.com:/homes/amd-pytorch     108G  327M  108G   1% /home/amd-pytorch
2025-07-24T04:32:28.6800760Z pure2.jax.cs.cpe.ice.amd.com:/GroupStorage          165T  127T   39T  77% /groups
2025-07-24T04:32:28.6801782Z pure2.jax.cs.cpe.ice.amd.com:/GroupStorage/Scratch  5.5T  1.8T  3.8T  33% /scratch
2025-07-24T04:32:28.6866907Z Prepare all required actions
2025-07-24T04:32:28.6867620Z Getting action download info
2025-07-24T04:32:28.8627663Z ##[group]Run ./.github/actions/download-td-artifacts
2025-07-24T04:32:28.8628354Z with:
2025-07-24T04:32:28.8628760Z env:
2025-07-24T04:32:28.8629189Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:32:28.8629983Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:32:28.8656177Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:32:28.8657313Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:32:28.8659079Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:32:28.8661243Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:32:28.8661806Z   AWS_REGION: us-east-1
2025-07-24T04:32:28.8662514Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:32:28.8663251Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:32:28.8673674Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:32:28.8674175Z ##[endgroup]
2025-07-24T04:32:28.8725515Z ##[group]Run seemethere/download-artifact-s3@v4
2025-07-24T04:32:28.8726057Z with:
2025-07-24T04:32:28.8726411Z   name: td_results
2025-07-24T04:32:28.8726788Z   s3-bucket: gha-artifacts
2025-07-24T04:32:28.8727209Z   region: us-east-1
2025-07-24T04:32:28.8727571Z env:
2025-07-24T04:32:28.8727912Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:32:28.8728574Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:32:28.8729599Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:32:28.8730530Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:32:28.8732084Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:32:28.8733441Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:32:28.8733886Z   AWS_REGION: us-east-1
2025-07-24T04:32:28.8734373Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:32:28.8734966Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:32:28.8744188Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:32:28.8744619Z ##[endgroup]
2025-07-24T04:32:29.3243493Z (node:3202610) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.
2025-07-24T04:32:29.3244407Z 
2025-07-24T04:32:29.3244772Z Please migrate your code to use AWS SDK for JavaScript (v3).
2025-07-24T04:32:29.3245715Z For more information, check the migration guide at https://a.co/7PzMCcy
2025-07-24T04:32:29.3246753Z (Use `node --trace-warnings ...` to show where the warning was created)
2025-07-24T04:32:29.4749331Z Found 1 objects with prefix pytorch/pytorch/16487210679/td_results/
2025-07-24T04:32:29.4750582Z Starting download (1/1): /var/home/pytorchci/actions-runner/_work/pytorch/pytorch/td_results.json
2025-07-24T04:32:29.7400134Z Finished download (1/1): /var/home/pytorchci/actions-runner/_work/pytorch/pytorch/td_results.json
2025-07-24T04:32:29.7410076Z Artifact download has finished successfully
2025-07-24T04:32:29.7904605Z ##[group]Run mkdir -p .additional_ci_files
2025-07-24T04:32:29.7905279Z mkdir -p .additional_ci_files
2025-07-24T04:32:29.7906055Z mv td_results.json .additional_ci_files/td_results.json || true
2025-07-24T04:32:29.7970899Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:32:29.7971313Z env:
2025-07-24T04:32:29.7971536Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:32:29.7971973Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:32:29.7972567Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:32:29.7973139Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:32:29.7974313Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:32:29.7975132Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:32:29.7975392Z   AWS_REGION: us-east-1
2025-07-24T04:32:29.7975730Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:32:29.7976096Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:32:29.7986065Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:32:29.7986546Z ##[endgroup]
2025-07-24T04:32:29.8213472Z ##[group]Run .github/scripts/parse_ref.py
2025-07-24T04:32:29.8213877Z .github/scripts/parse_ref.py
2025-07-24T04:32:29.8270289Z shell: /usr/bin/bash -e {0}
2025-07-24T04:32:29.8270815Z env:
2025-07-24T04:32:29.8271244Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:32:29.8272066Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:32:29.8273244Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:32:29.8274625Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:32:29.8276377Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:32:29.8277916Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:32:29.8278438Z   AWS_REGION: us-east-1
2025-07-24T04:32:29.8279003Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:32:29.8279764Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:32:29.8291516Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:32:29.8291819Z ##[endgroup]
2025-07-24T04:32:29.8757325Z Setting output tag=ciflow/trunk/149961
2025-07-24T04:32:29.8994923Z Prepare all required actions
2025-07-24T04:32:29.8995782Z Getting action download info
2025-07-24T04:32:30.0520862Z ##[group]Run ./.github/actions/filter-test-configs
2025-07-24T04:32:30.0521416Z with:
2025-07-24T04:32:30.0522071Z   github-token: ***
2025-07-24T04:32:30.0523516Z   test-matrix: {"include": [{"config": "default", "shard": 1, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "default", "shard": 2, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "distributed", "shard": 1, "num_shards": 1, "runner": "linux.rocm.gpu.4"}]}
2025-07-24T04:32:30.0525224Z   job-name: linux-jammy-rocm-py3.10 / test (default, 2, 2, linux.rocm.gpu.2)
2025-07-24T04:32:30.0525924Z env:
2025-07-24T04:32:30.0526286Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:32:30.0526968Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:32:30.0527954Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:32:30.0528961Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:32:30.0530502Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:32:30.0531879Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:32:30.0532340Z   AWS_REGION: us-east-1
2025-07-24T04:32:30.0532856Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:32:30.0533512Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:32:30.0542735Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:32:30.0543146Z ##[endgroup]
2025-07-24T04:32:30.0600483Z ##[group]Run nick-fields/retry@v3.0.0
2025-07-24T04:32:30.0600971Z with:
2025-07-24T04:32:30.0601303Z   shell: bash
2025-07-24T04:32:30.0601717Z   timeout_minutes: 10
2025-07-24T04:32:30.0602109Z   max_attempts: 5
2025-07-24T04:32:30.0602492Z   retry_wait_seconds: 30
2025-07-24T04:32:30.0603724Z   command: set -eux
# PyYAML 6.0 doesn't work with MacOS x86 anymore
# This must run on Python-3.7 (AmazonLinux2) so can't use request=3.32.2
python3 -m pip install requests==2.27.1 pyyaml==6.0.1

2025-07-24T04:32:30.0605024Z   polling_interval_seconds: 1
2025-07-24T04:32:30.0605471Z   warning_on_retry: true
2025-07-24T04:32:30.0605872Z   continue_on_error: false
2025-07-24T04:32:30.0606282Z env:
2025-07-24T04:32:30.0606624Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:32:30.0607296Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:32:30.0608309Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:32:30.0609237Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:32:30.0610767Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:32:30.0612571Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:32:30.0613032Z   AWS_REGION: us-east-1
2025-07-24T04:32:30.0613533Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:32:30.0614150Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:32:30.0623369Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:32:30.0623974Z   GITHUB_TOKEN: ***
2025-07-24T04:32:30.0624369Z ##[endgroup]
2025-07-24T04:32:30.1659879Z + python3 -m pip install requests==2.27.1 pyyaml==6.0.1
2025-07-24T04:32:30.4506206Z Defaulting to user installation because normal site-packages is not writeable
2025-07-24T04:32:30.6189678Z Requirement already satisfied: requests==2.27.1 in /var/home/pytorchci/.local/lib/python3.10/site-packages (2.27.1)
2025-07-24T04:32:30.6193184Z Requirement already satisfied: pyyaml==6.0.1 in /var/home/pytorchci/.local/lib/python3.10/site-packages (6.0.1)
2025-07-24T04:32:30.6281697Z Requirement already satisfied: charset-normalizer~=2.0.0 in /var/home/pytorchci/.local/lib/python3.10/site-packages (from requests==2.27.1) (2.0.12)
2025-07-24T04:32:30.6285364Z Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests==2.27.1) (2020.6.20)
2025-07-24T04:32:30.6289218Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3/dist-packages (from requests==2.27.1) (1.26.5)
2025-07-24T04:32:30.6298816Z Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests==2.27.1) (3.3)
2025-07-24T04:32:31.1656287Z Command completed after 1 attempt(s).
2025-07-24T04:32:31.1786830Z ##[group]Run set -x
2025-07-24T04:32:31.1787316Z set -x
2025-07-24T04:32:31.1787725Z 
2025-07-24T04:32:31.1788395Z # Use relative path here as this could be checked out anywhere, not necessarily
2025-07-24T04:32:31.1789227Z # in runner workspace
2025-07-24T04:32:31.1789906Z python3 "${GITHUB_ACTION_PATH}/../../scripts/parse_ref.py"
2025-07-24T04:32:31.1855793Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:32:31.1856507Z env:
2025-07-24T04:32:31.1856995Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:32:31.1857761Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:32:31.1858885Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:32:31.1859932Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:32:31.1861635Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:32:31.1863156Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:32:31.1863673Z   AWS_REGION: us-east-1
2025-07-24T04:32:31.1864260Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:32:31.1864951Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:32:31.1875424Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:32:31.1875915Z ##[endgroup]
2025-07-24T04:32:31.1979073Z + python3 /var/home/pytorchci/actions-runner/_work/pytorch/pytorch/./.github/actions/filter-test-configs/../../scripts/parse_ref.py
2025-07-24T04:32:31.2302721Z Setting output tag=ciflow/trunk/149961
2025-07-24T04:32:31.2433883Z ##[group]Run echo "Workflow: ${GITHUB_WORKFLOW}"
2025-07-24T04:32:31.2434985Z echo "Workflow: ${GITHUB_WORKFLOW}"
2025-07-24T04:32:31.2435605Z echo "Job name: ${JOB_NAME}"
2025-07-24T04:32:31.2436190Z 
2025-07-24T04:32:31.2436992Z # Use relative path here as this could be checked out anywhere, not necessarily
2025-07-24T04:32:31.2438022Z # in runner workspace
2025-07-24T04:32:31.2438798Z python3 "${GITHUB_ACTION_PATH}/../../scripts/filter_test_configs.py" \
2025-07-24T04:32:31.2439660Z   --workflow "${GITHUB_WORKFLOW}" \
2025-07-24T04:32:31.2440274Z   --job-name "${JOB_NAME}" \
2025-07-24T04:32:31.2442081Z   --test-matrix "{"include": [{"config": "default", "shard": 1, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "default", "shard": 2, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "distributed", "shard": 1, "num_shards": 1, "runner": "linux.rocm.gpu.4"}]}" \
2025-07-24T04:32:31.2444385Z   --selected-test-configs "" \
2025-07-24T04:32:31.2444990Z   --pr-number "${PR_NUMBER}" \
2025-07-24T04:32:31.2445545Z   --tag "${TAG}" \
2025-07-24T04:32:31.2446068Z   --event-name "${EVENT_NAME}" \
2025-07-24T04:32:31.2446636Z   --schedule "${SCHEDULE}" \
2025-07-24T04:32:31.2447192Z   --branch "${HEAD_BRANCH}"
2025-07-24T04:32:31.2511886Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:32:31.2512584Z env:
2025-07-24T04:32:31.2512998Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:32:31.2513774Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:32:31.2515024Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:32:31.2516067Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:32:31.2518251Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:32:31.2519870Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:32:31.2520405Z   AWS_REGION: us-east-1
2025-07-24T04:32:31.2521003Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:32:31.2521697Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:32:31.2532081Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:32:31.2532806Z   GITHUB_TOKEN: ***
2025-07-24T04:32:31.2533517Z   JOB_NAME: linux-jammy-rocm-py3.10 / test (default, 2, 2, linux.rocm.gpu.2)
2025-07-24T04:32:31.2534315Z   PR_NUMBER: 
2025-07-24T04:32:31.2534751Z   TAG: ciflow/trunk/149961
2025-07-24T04:32:31.2535220Z   EVENT_NAME: push
2025-07-24T04:32:31.2535643Z   SCHEDULE: 
2025-07-24T04:32:31.2536042Z   HEAD_BRANCH: 
2025-07-24T04:32:31.2536454Z ##[endgroup]
2025-07-24T04:32:31.2636553Z Workflow: trunk
2025-07-24T04:32:31.2637438Z Job name: linux-jammy-rocm-py3.10 / test (default, 2, 2, linux.rocm.gpu.2)
2025-07-24T04:32:31.5514488Z INFO:root:Found no test-config label on the PR, so all test configs are included
2025-07-24T04:32:31.8969790Z Setting output keep-going=False
2025-07-24T04:32:31.8970597Z Setting output ci-verbose-test-logs=False
2025-07-24T04:32:31.8971283Z Setting output ci-test-showlocals=False
2025-07-24T04:32:31.8971875Z Setting output ci-no-test-timeout=False
2025-07-24T04:32:31.8972449Z Setting output ci-no-td=False
2025-07-24T04:32:31.8972997Z Setting output ci-td-distributed=False
2025-07-24T04:32:31.8973557Z Setting output is-unstable=False
2025-07-24T04:32:31.8974088Z Setting output reenabled-issues=
2025-07-24T04:32:31.8975871Z Setting output test-matrix={"include": [{"config": "default", "shard": 1, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "default", "shard": 2, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "distributed", "shard": 1, "num_shards": 1, "runner": "linux.rocm.gpu.4"}]}
2025-07-24T04:32:31.8977787Z Setting output is-test-matrix-empty=False
2025-07-24T04:32:31.9228070Z ##[group]Run echo "Filtered matrix:"
2025-07-24T04:32:31.9228699Z echo "Filtered matrix:"
2025-07-24T04:32:31.9230386Z echo "{"include": [{"config": "default", "shard": 1, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "default", "shard": 2, "num_shards": 2, "runner": "linux.rocm.gpu.2"}, {"config": "distributed", "shard": 1, "num_shards": 1, "runner": "linux.rocm.gpu.4"}]}"
2025-07-24T04:32:31.9232060Z 
2025-07-24T04:32:31.9232437Z echo
2025-07-24T04:32:31.9232937Z echo "Is the current job unstable? False"
2025-07-24T04:32:31.9233530Z 
2025-07-24T04:32:31.9233905Z echo
2025-07-24T04:32:31.9234524Z echo "Is keep-going label set? False"
2025-07-24T04:32:31.9235093Z 
2025-07-24T04:32:31.9235455Z echo
2025-07-24T04:32:31.9235878Z echo "Reenabled issues? "
2025-07-24T04:32:31.9296483Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:32:31.9296843Z env:
2025-07-24T04:32:31.9297052Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:32:31.9297454Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:32:31.9298052Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:32:31.9298593Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:32:31.9299694Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:32:31.9301525Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:32:31.9302124Z   AWS_REGION: us-east-1
2025-07-24T04:32:31.9302709Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:32:31.9303398Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:32:31.9313815Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:32:31.9314387Z ##[endgroup]
2025-07-24T04:32:31.9411937Z Filtered matrix:
2025-07-24T04:32:31.9413667Z {include: [{config: default, shard: 1, num_shards: 2, runner: linux.rocm.gpu.2}, {config: default, shard: 2, num_shards: 2, runner: linux.rocm.gpu.2}, {config: distributed, shard: 1, num_shards: 1, runner: linux.rocm.gpu.4}]}
2025-07-24T04:32:31.9415249Z 
2025-07-24T04:32:31.9415604Z Is the current job unstable? False
2025-07-24T04:32:31.9416059Z 
2025-07-24T04:32:31.9416284Z Is keep-going label set? False
2025-07-24T04:32:31.9416636Z 
2025-07-24T04:32:31.9416822Z Reenabled issues? 
2025-07-24T04:32:31.9490280Z ##[group]Run echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}"
2025-07-24T04:32:31.9491253Z echo "timeout=$((JOB_TIMEOUT-30))" >> "${GITHUB_OUTPUT}"
2025-07-24T04:32:31.9554126Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:32:31.9554963Z env:
2025-07-24T04:32:31.9555377Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:32:31.9556164Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:32:31.9557342Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:32:31.9558408Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:32:31.9560180Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:32:31.9561782Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:32:31.9562385Z   AWS_REGION: us-east-1
2025-07-24T04:32:31.9562983Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:32:31.9563690Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:32:31.9574103Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:32:31.9574588Z   JOB_TIMEOUT: 300
2025-07-24T04:32:31.9575033Z ##[endgroup]
2025-07-24T04:32:31.9766348Z ##[group]Run set -x
2025-07-24T04:32:31.9766974Z set -x
2025-07-24T04:32:31.9767405Z 
2025-07-24T04:32:31.9767909Z if [[ $TEST_CONFIG == 'multigpu' ]]; then
2025-07-24T04:32:31.9768655Z   TEST_COMMAND=.ci/pytorch/multigpu-test.sh
2025-07-24T04:32:31.9769379Z elif [[ $BUILD_ENVIRONMENT == *onnx* ]]; then
2025-07-24T04:32:31.9770069Z   TEST_COMMAND=.ci/caffe2/test.sh
2025-07-24T04:32:31.9770640Z else
2025-07-24T04:32:31.9771106Z   TEST_COMMAND=.ci/pytorch/test.sh
2025-07-24T04:32:31.9771651Z fi
2025-07-24T04:32:31.9772034Z 
2025-07-24T04:32:31.9772635Z # detached container should get cleaned up by teardown_ec2_linux
2025-07-24T04:32:31.9773699Z # TODO: Stop building test binaries as part of the build phase
2025-07-24T04:32:31.9774674Z # Used for GPU_FLAG since that doesn't play nice
2025-07-24T04:32:31.9775705Z # shellcheck disable=SC2086,SC2090
2025-07-24T04:32:31.9776447Z container_name=$(docker run \
2025-07-24T04:32:31.9777034Z   ${GPU_FLAG:-} \
2025-07-24T04:32:31.9778194Z   -e BUILD_ENVIRONMENT \
2025-07-24T04:32:31.9778829Z   -e PR_NUMBER \
2025-07-24T04:32:31.9779332Z   -e GITHUB_ACTIONS \
2025-07-24T04:32:31.9779873Z   -e GITHUB_REPOSITORY \
2025-07-24T04:32:31.9780407Z   -e GITHUB_WORKFLOW \
2025-07-24T04:32:31.9780915Z   -e GITHUB_JOB \
2025-07-24T04:32:31.9781403Z   -e GITHUB_RUN_ID \
2025-07-24T04:32:31.9781899Z   -e GITHUB_RUN_NUMBER \
2025-07-24T04:32:31.9782439Z   -e GITHUB_RUN_ATTEMPT \
2025-07-24T04:32:31.9782965Z   -e JOB_ID \
2025-07-24T04:32:31.9783430Z   -e JOB_NAME \
2025-07-24T04:32:31.9783888Z   -e BRANCH \
2025-07-24T04:32:31.9784356Z   -e SHA1 \
2025-07-24T04:32:31.9784810Z   -e AWS_DEFAULT_REGION \
2025-07-24T04:32:31.9785349Z   -e IN_WHEEL_TEST \
2025-07-24T04:32:31.9785848Z   -e SHARD_NUMBER \
2025-07-24T04:32:31.9786348Z   -e TEST_CONFIG \
2025-07-24T04:32:31.9786859Z   -e NUM_TEST_SHARDS \
2025-07-24T04:32:31.9787379Z   -e REENABLED_ISSUES \
2025-07-24T04:32:31.9787949Z   -e CONTINUE_THROUGH_ERROR \
2025-07-24T04:32:31.9788506Z   -e VERBOSE_TEST_LOGS \
2025-07-24T04:32:31.9789031Z   -e TEST_SHOWLOCALS \
2025-07-24T04:32:31.9789533Z   -e NO_TEST_TIMEOUT \
2025-07-24T04:32:31.9790021Z   -e NO_TD \
2025-07-24T04:32:31.9790540Z   -e MAX_JOBS="$(nproc --ignore=2)" \
2025-07-24T04:32:31.9791202Z   -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK \
2025-07-24T04:32:31.9791860Z   -e PYTORCH_TEST_RERUN_DISABLED_TESTS \
2025-07-24T04:32:31.9792464Z   -e TESTS_TO_INCLUDE \
2025-07-24T04:32:31.9792985Z   -e DASHBOARD_TAG \
2025-07-24T04:32:31.9793648Z   --env-file="${RUNNER_TEMP}/github_env_${GITHUB_RUN_ID}" \
2025-07-24T04:32:31.9794540Z   --ulimit stack=10485760:83886080 \
2025-07-24T04:32:31.9795114Z   --ulimit core=0 \
2025-07-24T04:32:31.9795684Z   --security-opt seccomp=unconfined \
2025-07-24T04:32:31.9796276Z   --cap-add=SYS_PTRACE \
2025-07-24T04:32:31.9796795Z   --shm-size="8g" \
2025-07-24T04:32:31.9797263Z   --tty \
2025-07-24T04:32:31.9797687Z   --detach \
2025-07-24T04:32:31.9798167Z   --name="${container_name}" \
2025-07-24T04:32:31.9798721Z   --user jenkins \
2025-07-24T04:32:31.9799334Z   -v "${GITHUB_WORKSPACE}:/var/lib/jenkins/workspace" \
2025-07-24T04:32:31.9800055Z   -w /var/lib/jenkins/workspace \
2025-07-24T04:32:31.9800612Z   "${DOCKER_IMAGE}"
2025-07-24T04:32:31.9801079Z )
2025-07-24T04:32:31.9801526Z # save container name for later step
2025-07-24T04:32:31.9802691Z echo "CONTAINER_NAME=${container_name}" >> "$GITHUB_ENV"
2025-07-24T04:32:31.9804028Z # jenkins user does not have write permission to mounted workspace; work-around by copying within container to jenkins home
2025-07-24T04:32:31.9805697Z docker exec -t "${container_name}" sh -c "cd .. && cp -R workspace pytorch && cd pytorch && pip install dist/*.whl && ${TEST_COMMAND}"
2025-07-24T04:32:31.9871870Z shell: /usr/bin/bash -e {0}
2025-07-24T04:32:31.9872392Z env:
2025-07-24T04:32:31.9872803Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:32:31.9873578Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:32:31.9875012Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:32:31.9876043Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:32:31.9877786Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:32:31.9879353Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:32:31.9879871Z   AWS_REGION: us-east-1
2025-07-24T04:32:31.9880888Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:32:31.9881575Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:32:31.9891936Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:32:31.9892501Z   BUILD_ENVIRONMENT: linux-jammy-rocm-py3.10
2025-07-24T04:32:31.9893115Z   PR_NUMBER: 
2025-07-24T04:32:31.9893578Z   GITHUB_REPOSITORY: pytorch/pytorch
2025-07-24T04:32:31.9894134Z   GITHUB_WORKFLOW: trunk
2025-07-24T04:32:31.9894579Z   GITHUB_JOB: test
2025-07-24T04:32:31.9895012Z   GITHUB_RUN_ID: 16487210679
2025-07-24T04:32:31.9895500Z   GITHUB_RUN_NUMBER: 137322
2025-07-24T04:32:31.9895978Z   GITHUB_RUN_ATTEMPT: 1
2025-07-24T04:32:31.9896415Z   JOB_ID: 46615439132
2025-07-24T04:32:31.9897096Z   JOB_NAME: linux-jammy-rocm-py3.10 / test (default, 2, 2, linux.rocm.gpu.2)
2025-07-24T04:32:31.9897878Z   BRANCH: 
2025-07-24T04:32:31.9898347Z   SHA1: 27c8ef1839e0747b36769efb192befc9308ff93c
2025-07-24T04:32:31.9898965Z   CONTINUE_THROUGH_ERROR: False
2025-07-24T04:32:31.9899484Z   VERBOSE_TEST_LOGS: False
2025-07-24T04:32:31.9899974Z   TEST_SHOWLOCALS: False
2025-07-24T04:32:31.9900432Z   NO_TEST_TIMEOUT: False
2025-07-24T04:32:31.9900908Z   NO_TD: False
2025-07-24T04:32:31.9901316Z   TEST_CONFIG: default
2025-07-24T04:32:31.9901745Z   SHARD_NUMBER: 2
2025-07-24T04:32:31.9902164Z   NUM_TEST_SHARDS: 2
2025-07-24T04:32:31.9902597Z   REENABLED_ISSUES: 
2025-07-24T04:32:31.9903840Z   DOCKER_IMAGE: 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:32:31.9905218Z   PYTORCH_TEST_CUDA_MEM_LEAK_CHECK: 0
2025-07-24T04:32:31.9905801Z   PYTORCH_TEST_RERUN_DISABLED_TESTS: 0
2025-07-24T04:32:31.9907308Z   TESTS_TO_INCLUDE: test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl
2025-07-24T04:32:31.9908796Z   DASHBOARD_TAG: 
2025-07-24T04:32:31.9909209Z ##[endgroup]
2025-07-24T04:32:32.0015929Z + [[ default == \m\u\l\t\i\g\p\u ]]
2025-07-24T04:32:32.0016644Z + [[ linux-jammy-rocm-py3.10 == *onnx* ]]
2025-07-24T04:32:32.0017294Z + TEST_COMMAND=.ci/pytorch/test.sh
2025-07-24T04:32:32.0037272Z +++ nproc --ignore=2
2025-07-24T04:32:32.0074779Z ++ docker run --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host -e BUILD_ENVIRONMENT -e PR_NUMBER -e GITHUB_ACTIONS -e GITHUB_REPOSITORY -e GITHUB_WORKFLOW -e GITHUB_JOB -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RUN_ATTEMPT -e JOB_ID -e JOB_NAME -e BRANCH -e SHA1 -e AWS_DEFAULT_REGION -e IN_WHEEL_TEST -e SHARD_NUMBER -e TEST_CONFIG -e NUM_TEST_SHARDS -e REENABLED_ISSUES -e CONTINUE_THROUGH_ERROR -e VERBOSE_TEST_LOGS -e TEST_SHOWLOCALS -e NO_TEST_TIMEOUT -e NO_TD -e MAX_JOBS=126 -e PYTORCH_TEST_CUDA_MEM_LEAK_CHECK -e PYTORCH_TEST_RERUN_DISABLED_TESTS -e TESTS_TO_INCLUDE -e DASHBOARD_TAG --env-file=/var/home/pytorchci/actions-runner/_work/_temp/github_env_16487210679 --ulimit stack=10485760:83886080 --ulimit core=0 --security-opt seccomp=unconfined --cap-add=SYS_PTRACE --shm-size=8g --tty --detach --name= --user jenkins -v /var/home/pytorchci/actions-runner/_work/pytorch/pytorch:/var/lib/jenkins/workspace -w /var/lib/jenkins/workspace 308535385114.dkr.ecr.us-east-1.amazonaws.com/pytorch/ci-image:pytorch-linux-jammy-rocm-n-py3-f28e9a7a8bc459f8b121db7008296463e7d7437f
2025-07-24T04:32:32.2428255Z + container_name=51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:32:32.2429524Z + echo CONTAINER_NAME=51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:32:32.2431672Z + docker exec -t 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c sh -c 'cd .. && cp -R workspace pytorch && cd pytorch && pip install dist/*.whl && .ci/pytorch/test.sh'
2025-07-24T04:32:47.2728951Z Processing ./dist/torch-2.9.0a0+git27c8ef1-cp310-cp310-linux_x86_64.whl
2025-07-24T04:32:48.0503302Z Requirement already satisfied: filelock in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.9.0a0+git27c8ef1) (3.18.0)
2025-07-24T04:32:48.0506520Z Requirement already satisfied: typing-extensions>=4.10.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.9.0a0+git27c8ef1) (4.14.1)
2025-07-24T04:32:48.0509715Z Requirement already satisfied: sympy>=1.13.3 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.9.0a0+git27c8ef1) (1.13.3)
2025-07-24T04:32:48.0513858Z Requirement already satisfied: networkx>=2.5.1 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.9.0a0+git27c8ef1) (2.8.8)
2025-07-24T04:32:48.0516187Z Requirement already satisfied: jinja2 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.9.0a0+git27c8ef1) (3.1.6)
2025-07-24T04:32:48.0520265Z Requirement already satisfied: fsspec>=0.8.5 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from torch==2.9.0a0+git27c8ef1) (2025.5.1)
2025-07-24T04:32:48.0827943Z Requirement already satisfied: mpmath<1.4,>=1.1.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from sympy>=1.13.3->torch==2.9.0a0+git27c8ef1) (1.3.0)
2025-07-24T04:32:48.0873472Z Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/envs/py_3.10/lib/python3.10/site-packages (from jinja2->torch==2.9.0a0+git27c8ef1) (3.0.2)
2025-07-24T04:32:48.6309290Z Installing collected packages: torch
2025-07-24T04:32:59.6686988Z ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
2025-07-24T04:32:59.6688660Z helion 0.0.10 requires filecheck, which is not installed.
2025-07-24T04:32:59.6689613Z timm 1.0.14 requires torchvision, which is not installed.
2025-07-24T04:32:59.6690496Z Successfully installed torch-2.9.0a0+git27c8ef1
2025-07-24T04:32:59.7257618Z + export TERM=vt100
2025-07-24T04:32:59.7258211Z + TERM=vt100
2025-07-24T04:32:59.7267394Z ++ dirname .ci/pytorch/test.sh
2025-07-24T04:32:59.7296881Z + source .ci/pytorch/common.sh
2025-07-24T04:32:59.7307228Z +++ dirname .ci/pytorch/common.sh
2025-07-24T04:32:59.7332569Z ++ source .ci/pytorch/common_utils.sh
2025-07-24T04:32:59.7335469Z +++ declare -f -t trap_add
2025-07-24T04:32:59.7345887Z ++ set -ex -o pipefail
2025-07-24T04:32:59.7346593Z ++ [[ linux-jammy-rocm-py3.10 == *rocm* ]]
2025-07-24T04:32:59.7347202Z ++ unset HIP_PLATFORM
2025-07-24T04:32:59.7347716Z ++ export PYTORCH_TEST_WITH_ROCM=1
2025-07-24T04:32:59.7348267Z ++ PYTORCH_TEST_WITH_ROCM=1
2025-07-24T04:32:59.7348757Z ++ BUILD_TEST_LIBTORCH=0
2025-07-24T04:32:59.7358157Z ++ dirname .ci/pytorch/test.sh
2025-07-24T04:32:59.7391631Z + source .ci/pytorch/common-build.sh
2025-07-24T04:32:59.7394849Z ++ [[ linux-jammy-rocm-py3.10 != *win-* ]]
2025-07-24T04:32:59.7414696Z ++++ dirname .ci/pytorch/common-build.sh
2025-07-24T04:32:59.7443511Z +++ cd .ci/pytorch
2025-07-24T04:32:59.7444083Z +++ pwd -P
2025-07-24T04:32:59.7450474Z ++ script_dir=/var/lib/jenkins/pytorch/.ci/pytorch
2025-07-24T04:32:59.7451400Z ++ [[ linux-jammy-rocm-py3.10 == *-pch* ]]
2025-07-24T04:32:59.7452026Z ++ which sccache
2025-07-24T04:32:59.7486602Z ++ [[ -z '' ]]
2025-07-24T04:32:59.7487146Z ++ unset SCCACHE_BUCKET
2025-07-24T04:32:59.7487653Z ++ unset SCCACHE_REGION
2025-07-24T04:32:59.7488137Z ++ sccache --stop-server
2025-07-24T04:32:59.7565181Z ++ true
2025-07-24T04:32:59.7565850Z ++ rm -f /var/lib/jenkins/sccache_error.log
2025-07-24T04:32:59.7597715Z ++ trap_add sccache_epilogue EXIT
2025-07-24T04:32:59.7598420Z ++ trap_add_cmd=sccache_epilogue
2025-07-24T04:32:59.7598972Z ++ shift
2025-07-24T04:32:59.7599383Z ++ for trap_add_name in "$@"
2025-07-24T04:32:59.7619151Z ++++ trap -p EXIT
2025-07-24T04:32:59.7626570Z +++ eval 'extract_trap_cmd '
2025-07-24T04:32:59.7627500Z ++++ extract_trap_cmd
2025-07-24T04:32:59.7628045Z ++++ printf '%s\n' ''
2025-07-24T04:32:59.7628733Z +++ printf '%s\n' sccache_epilogue
2025-07-24T04:32:59.7635217Z ++ trap -- '
2025-07-24T04:32:59.7635748Z sccache_epilogue' EXIT
2025-07-24T04:32:59.7636903Z ++ [[ -n '' ]]
2025-07-24T04:32:59.7637378Z ++ [[ linux-jammy-rocm-py3.10 == *rocm* ]]
2025-07-24T04:32:59.7638107Z ++ SCCACHE_ERROR_LOG=/var/lib/jenkins/sccache_error.log
2025-07-24T04:32:59.7638768Z ++ SCCACHE_IDLE_TIMEOUT=0
2025-07-24T04:32:59.7639257Z ++ sccache --start-server
2025-07-24T04:32:59.7697848Z sccache: Starting the server...
2025-07-24T04:32:59.8413527Z sccache: Listening on address 127.0.0.1:4226
2025-07-24T04:32:59.8445577Z ++ sccache --zero-stats
2025-07-24T04:32:59.8492977Z Statistics zeroed.
2025-07-24T04:32:59.8502865Z ++ which ccache
2025-07-24T04:32:59.8533995Z + [[ linux-jammy-rocm-py3.10 != *rocm* ]]
2025-07-24T04:32:59.8534672Z + echo 'Environment variables:'
2025-07-24T04:32:59.8535192Z Environment variables:
2025-07-24T04:32:59.8535641Z + env
2025-07-24T04:32:59.8564974Z GITHUB_WORKSPACE=/var/home/pytorchci/actions-runner/_work/pytorch/pytorch
2025-07-24T04:32:59.8565907Z CONTINUE_THROUGH_ERROR=False
2025-07-24T04:32:59.8566512Z BUILD_ENVIRONMENT=linux-jammy-rocm-py3.10
2025-07-24T04:32:59.8567212Z HOSTNAME=gpud553.jax.cs.cpe.ice.amd.com
2025-07-24T04:32:59.8568424Z GITHUB_PATH=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/add_path_b97dc80c-a4e1-4c3d-85a5-dddd5a801622
2025-07-24T04:32:59.8569571Z GITHUB_ACTION=__self
2025-07-24T04:32:59.8570054Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0
2025-07-24T04:32:59.8570589Z GITHUB_RUN_NUMBER=137322
2025-07-24T04:32:59.8571053Z TEST_CONFIG=default
2025-07-24T04:32:59.8571501Z GITHUB_REPOSITORY_OWNER_ID=21003710
2025-07-24T04:32:59.8572052Z AWS_DEFAULT_REGION=us-east-1
2025-07-24T04:32:59.8572582Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot]
2025-07-24T04:32:59.8573131Z GITHUB_REF_TYPE=tag
2025-07-24T04:32:59.8577482Z ***
2025-07-24T04:32:59.8577925Z GITHUB_REPOSITORY_ID=65600975
2025-07-24T04:32:59.8578513Z GITHUB_ACTIONS=true
2025-07-24T04:32:59.8579036Z SHA1=27c8ef1839e0747b36769efb192befc9308ff93c
2025-07-24T04:32:59.8579705Z GITHUB_SHA=27c8ef1839e0747b36769efb192befc9308ff93c
2025-07-24T04:32:59.8580767Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/trunk.yml@refs/tags/ciflow/trunk/149961
2025-07-24T04:32:59.8581682Z UCC_HOME=/usr
2025-07-24T04:32:59.8582087Z VERBOSE_TEST_LOGS=False
2025-07-24T04:32:59.8582581Z GITHUB_REF=refs/tags/ciflow/trunk/149961
2025-07-24T04:32:59.8583130Z SHARD_NUMBER=2
2025-07-24T04:32:59.8583544Z GITHUB_REF_PROTECTED=false
2025-07-24T04:32:59.8584007Z HOME=/var/lib/jenkins
2025-07-24T04:32:59.8584509Z GITHUB_API_URL=https://api.github.com
2025-07-24T04:32:59.8585119Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0
2025-07-24T04:32:59.8585656Z LANG=C.UTF-8
2025-07-24T04:32:59.8586167Z UCX_COMMIT=cc312eaa4655c0cc5c2bcd796db938f90563bcf6
2025-07-24T04:32:59.8586813Z PYTORCH_TEST_WITH_ROCM=1
2025-07-24T04:32:59.8587280Z NUM_TEST_SHARDS=2
2025-07-24T04:32:59.8587679Z UCX_HOME=/usr
2025-07-24T04:32:59.8589191Z GITHUB_STATE=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/save_state_b97dc80c-a4e1-4c3d-85a5-dddd5a801622
2025-07-24T04:32:59.8590641Z JOB_NAME=linux-jammy-rocm-py3.10 / test (default, 2, 2, linux.rocm.gpu.2)
2025-07-24T04:32:59.8591445Z MAGMA_HOME=/opt/rocm/magma
2025-07-24T04:32:59.8592528Z GITHUB_ENV=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/set_env_b97dc80c-a4e1-4c3d-85a5-dddd5a801622
2025-07-24T04:32:59.8594008Z GITHUB_EVENT_PATH=/var/home/pytorchci/actions-runner/_work/_temp/_github_workflow/event.json
2025-07-24T04:32:59.8595132Z GITHUB_EVENT_NAME=push
2025-07-24T04:32:59.8595652Z DASHBOARD_TAG=
2025-07-24T04:32:59.8596141Z GITHUB_RUN_ID=16487210679
2025-07-24T04:32:59.8597323Z GITHUB_STEP_SUMMARY=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/step_summary_b97dc80c-a4e1-4c3d-85a5-dddd5a801622
2025-07-24T04:32:59.8598618Z GITHUB_ACTOR=pytorch-bot[bot]
2025-07-24T04:32:59.8599198Z PR_NUMBER=
2025-07-24T04:32:59.8599657Z GITHUB_RUN_ATTEMPT=1
2025-07-24T04:32:59.8600206Z ANACONDA_PYTHON_VERSION=3.10
2025-07-24T04:32:59.8600843Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql
2025-07-24T04:32:59.8601796Z TERM=vt100
2025-07-24T04:32:59.8602189Z INSTALLED_VISION=yes
2025-07-24T04:32:59.8602606Z BRANCH=
2025-07-24T04:32:59.8602998Z OPENSSL_ROOT_DIR=/opt/openssl
2025-07-24T04:32:59.8604457Z TESTS_TO_INCLUDE=test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl
2025-07-24T04:32:59.8606480Z GITHUB_ACTION_PATH=/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-rocm
2025-07-24T04:32:59.8607538Z GITHUB_SERVER_URL=https://github.com
2025-07-24T04:32:59.8608118Z PYTORCH_ROCM_ARCH=gfx90a;gfx942
2025-07-24T04:32:59.8608695Z UCC_COMMIT=0c0fc21559835044ab107199e334f7157d6a0d3d
2025-07-24T04:32:59.8609287Z REENABLED_ISSUES=
2025-07-24T04:32:59.8609679Z SHLVL=1
2025-07-24T04:32:59.8610053Z MAX_JOBS=126
2025-07-24T04:32:59.8610462Z GITHUB_ACTOR_ID=54816060
2025-07-24T04:32:59.8611077Z GITHUB_WORKFLOW_SHA=27c8ef1839e0747b36769efb192befc9308ff93c
2025-07-24T04:32:59.8611786Z GITHUB_REF_NAME=ciflow/trunk/149961
2025-07-24T04:32:59.8612324Z ROCM_PATH=/opt/rocm
2025-07-24T04:32:59.8612738Z GITHUB_JOB=test
2025-07-24T04:32:59.8613140Z NO_TEST_TIMEOUT=False
2025-07-24T04:32:59.8613609Z GITHUB_REPOSITORY=pytorch/pytorch
2025-07-24T04:32:59.8614118Z LC_ALL=C.UTF-8
2025-07-24T04:32:59.8614530Z GITHUB_RETENTION_DAYS=90
2025-07-24T04:32:59.8615069Z OPENSSL_DIR=/opt/openssl
2025-07-24T04:32:59.8615618Z GITHUB_ACTION_REPOSITORY=
2025-07-24T04:32:59.8617392Z PATH=/opt/cache/bin:/opt/rocm/llvm/bin:/opt/rocm/opencl/bin:/opt/rocm/hip/bin:/opt/rocm/hcc/bin:/opt/rocm/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2025-07-24T04:32:59.8619097Z GITHUB_BASE_REF=
2025-07-24T04:32:59.8619489Z CI=true
2025-07-24T04:32:59.8619885Z GITHUB_REPOSITORY_OWNER=pytorch
2025-07-24T04:32:59.8620367Z JOB_ID=46615439132
2025-07-24T04:32:59.8620777Z GITHUB_HEAD_REF=
2025-07-24T04:32:59.8621184Z GITHUB_ACTION_REF=
2025-07-24T04:32:59.8621590Z TEST_SHOWLOCALS=False
2025-07-24T04:32:59.8622032Z GITHUB_WORKFLOW=trunk
2025-07-24T04:32:59.8622493Z DEBIAN_FRONTEND=noninteractive
2025-07-24T04:32:59.8623640Z GITHUB_OUTPUT=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/set_output_b97dc80c-a4e1-4c3d-85a5-dddd5a801622
2025-07-24T04:32:59.8624784Z NO_TD=False
2025-07-24T04:32:59.8625172Z OLDPWD=/var/lib/jenkins
2025-07-24T04:32:59.8625595Z _=/usr/bin/env
2025-07-24T04:32:59.8626160Z ++ python -c 'import site; print(site.getsitepackages()[0])'
2025-07-24T04:32:59.8835344Z + TORCH_INSTALL_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch
2025-07-24T04:32:59.8837168Z + TORCH_BIN_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/bin
2025-07-24T04:32:59.8838233Z + TORCH_LIB_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib
2025-07-24T04:32:59.8839929Z + TORCH_TEST_DIR=/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/test
2025-07-24T04:32:59.8840770Z + BUILD_DIR=build
2025-07-24T04:32:59.8841235Z + BUILD_RENAMED_DIR=build_renamed
2025-07-24T04:32:59.8841791Z + BUILD_BIN_DIR=build/bin
2025-07-24T04:32:59.8842246Z + SHARD_NUMBER=2
2025-07-24T04:32:59.8842647Z + NUM_TEST_SHARDS=2
2025-07-24T04:32:59.8843096Z + export TORCH_SERIALIZATION_DEBUG=1
2025-07-24T04:32:59.8843660Z + TORCH_SERIALIZATION_DEBUG=1
2025-07-24T04:32:59.8844147Z + export VALGRIND=ON
2025-07-24T04:32:59.8844568Z + VALGRIND=ON
2025-07-24T04:32:59.8845020Z + [[ linux-jammy-rocm-py3.10 == *clang9* ]]
2025-07-24T04:32:59.8845639Z + [[ linux-jammy-rocm-py3.10 == *xpu* ]]
2025-07-24T04:32:59.8846263Z + [[ linux-jammy-rocm-py3.10 == *s390x* ]]
2025-07-24T04:32:59.8846806Z + [[ 0 == \1 ]]
2025-07-24T04:32:59.8847200Z + [[ False == \1 ]]
2025-07-24T04:32:59.8847676Z + [[ linux-jammy-rocm-py3.10 != *bazel* ]]
2025-07-24T04:32:59.8848275Z ++ realpath build/custom_test_artifacts
2025-07-24T04:32:59.8884176Z + CUSTOM_TEST_ARTIFACT_BUILD_DIR=/var/lib/jenkins/pytorch/build/custom_test_artifacts
2025-07-24T04:32:59.8886021Z + [[ -n test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl ]]
2025-07-24T04:32:59.8887847Z + echo 'Setting INCLUDE_CLAUSE'
2025-07-24T04:32:59.8888357Z Setting INCLUDE_CLAUSE
2025-07-24T04:32:59.8889822Z + INCLUDE_CLAUSE='--include test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl'
2025-07-24T04:32:59.8891362Z + echo 'Environment variables'
2025-07-24T04:32:59.8891849Z Environment variables
2025-07-24T04:32:59.8892260Z + env
2025-07-24T04:32:59.8912941Z GITHUB_WORKSPACE=/var/home/pytorchci/actions-runner/_work/pytorch/pytorch
2025-07-24T04:32:59.8913885Z CONTINUE_THROUGH_ERROR=False
2025-07-24T04:32:59.8914583Z BUILD_ENVIRONMENT=linux-jammy-rocm-py3.10
2025-07-24T04:32:59.8915328Z HOSTNAME=gpud553.jax.cs.cpe.ice.amd.com
2025-07-24T04:32:59.8916532Z GITHUB_PATH=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/add_path_b97dc80c-a4e1-4c3d-85a5-dddd5a801622
2025-07-24T04:32:59.8917699Z GITHUB_ACTION=__self
2025-07-24T04:32:59.8918179Z PYTORCH_TEST_CUDA_MEM_LEAK_CHECK=0
2025-07-24T04:32:59.8918712Z GITHUB_RUN_NUMBER=137322
2025-07-24T04:32:59.8919170Z TEST_CONFIG=default
2025-07-24T04:32:59.8919618Z GITHUB_REPOSITORY_OWNER_ID=21003710
2025-07-24T04:32:59.8920148Z AWS_DEFAULT_REGION=us-east-1
2025-07-24T04:32:59.8920693Z GITHUB_TRIGGERING_ACTOR=pytorch-bot[bot]
2025-07-24T04:32:59.8921246Z GITHUB_REF_TYPE=tag
2025-07-24T04:32:59.8921891Z ***
2025-07-24T04:32:59.8922296Z GITHUB_REPOSITORY_ID=65600975
2025-07-24T04:32:59.8922809Z GITHUB_ACTIONS=true
2025-07-24T04:32:59.8923305Z SHA1=27c8ef1839e0747b36769efb192befc9308ff93c
2025-07-24T04:32:59.8923997Z GITHUB_SHA=27c8ef1839e0747b36769efb192befc9308ff93c
2025-07-24T04:32:59.8925001Z GITHUB_WORKFLOW_REF=pytorch/pytorch/.github/workflows/trunk.yml@refs/tags/ciflow/trunk/149961
2025-07-24T04:32:59.8925899Z UCC_HOME=/usr
2025-07-24T04:32:59.8926324Z TORCH_SERIALIZATION_DEBUG=1
2025-07-24T04:32:59.8926820Z VERBOSE_TEST_LOGS=False
2025-07-24T04:32:59.8927292Z GITHUB_REF=refs/tags/ciflow/trunk/149961
2025-07-24T04:32:59.8927828Z SHARD_NUMBER=2
2025-07-24T04:32:59.8928254Z GITHUB_REF_PROTECTED=false
2025-07-24T04:32:59.8947215Z HOME=/var/lib/jenkins
2025-07-24T04:32:59.8947870Z GITHUB_API_URL=https://api.github.com
2025-07-24T04:32:59.8948513Z PYTORCH_TEST_RERUN_DISABLED_TESTS=0
2025-07-24T04:32:59.8949072Z LANG=C.UTF-8
2025-07-24T04:32:59.8949587Z UCX_COMMIT=cc312eaa4655c0cc5c2bcd796db938f90563bcf6
2025-07-24T04:32:59.8950242Z PYTORCH_TEST_WITH_ROCM=1
2025-07-24T04:32:59.8950717Z NUM_TEST_SHARDS=2
2025-07-24T04:32:59.8951125Z UCX_HOME=/usr
2025-07-24T04:32:59.8952215Z GITHUB_STATE=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/save_state_b97dc80c-a4e1-4c3d-85a5-dddd5a801622
2025-07-24T04:32:59.8954074Z JOB_NAME=linux-jammy-rocm-py3.10 / test (default, 2, 2, linux.rocm.gpu.2)
2025-07-24T04:32:59.8955316Z MAGMA_HOME=/opt/rocm/magma
2025-07-24T04:32:59.8956465Z GITHUB_ENV=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/set_env_b97dc80c-a4e1-4c3d-85a5-dddd5a801622
2025-07-24T04:32:59.8957977Z GITHUB_EVENT_PATH=/var/home/pytorchci/actions-runner/_work/_temp/_github_workflow/event.json
2025-07-24T04:32:59.8958898Z GITHUB_EVENT_NAME=push
2025-07-24T04:32:59.8959350Z DASHBOARD_TAG=
2025-07-24T04:32:59.8959781Z GITHUB_RUN_ID=16487210679
2025-07-24T04:32:59.8960962Z GITHUB_STEP_SUMMARY=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/step_summary_b97dc80c-a4e1-4c3d-85a5-dddd5a801622
2025-07-24T04:32:59.8962191Z GITHUB_ACTOR=pytorch-bot[bot]
2025-07-24T04:32:59.8962680Z PR_NUMBER=
2025-07-24T04:32:59.8963067Z GITHUB_RUN_ATTEMPT=1
2025-07-24T04:32:59.8963500Z VALGRIND=ON
2025-07-24T04:32:59.8963915Z ANACONDA_PYTHON_VERSION=3.10
2025-07-24T04:32:59.8964524Z GITHUB_GRAPHQL_URL=https://api.github.com/graphql
2025-07-24T04:32:59.8965131Z TERM=vt100
2025-07-24T04:32:59.8965510Z INSTALLED_VISION=yes
2025-07-24T04:32:59.8966283Z BRANCH=
2025-07-24T04:32:59.8966685Z OPENSSL_ROOT_DIR=/opt/openssl
2025-07-24T04:32:59.8968141Z TESTS_TO_INCLUDE=test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl
2025-07-24T04:32:59.8970104Z GITHUB_ACTION_PATH=/var/home/pytorchci/actions-runner/_work/pytorch/pytorch/./.github/actions/setup-rocm
2025-07-24T04:32:59.8971139Z GITHUB_SERVER_URL=https://github.com
2025-07-24T04:32:59.8971729Z PYTORCH_ROCM_ARCH=gfx90a;gfx942
2025-07-24T04:32:59.8972303Z UCC_COMMIT=0c0fc21559835044ab107199e334f7157d6a0d3d
2025-07-24T04:32:59.8972899Z REENABLED_ISSUES=
2025-07-24T04:32:59.8973300Z SHLVL=1
2025-07-24T04:32:59.8973664Z MAX_JOBS=126
2025-07-24T04:32:59.8974069Z GITHUB_ACTOR_ID=54816060
2025-07-24T04:32:59.8974681Z GITHUB_WORKFLOW_SHA=27c8ef1839e0747b36769efb192befc9308ff93c
2025-07-24T04:32:59.8975388Z GITHUB_REF_NAME=ciflow/trunk/149961
2025-07-24T04:32:59.8975908Z ROCM_PATH=/opt/rocm
2025-07-24T04:32:59.8976336Z GITHUB_JOB=test
2025-07-24T04:32:59.8976740Z NO_TEST_TIMEOUT=False
2025-07-24T04:32:59.8977204Z GITHUB_REPOSITORY=pytorch/pytorch
2025-07-24T04:32:59.8977714Z LC_ALL=C.UTF-8
2025-07-24T04:32:59.8978128Z GITHUB_RETENTION_DAYS=90
2025-07-24T04:32:59.8978591Z OPENSSL_DIR=/opt/openssl
2025-07-24T04:32:59.8979061Z GITHUB_ACTION_REPOSITORY=
2025-07-24T04:32:59.8980727Z PATH=/opt/cache/bin:/opt/rocm/llvm/bin:/opt/rocm/opencl/bin:/opt/rocm/hip/bin:/opt/rocm/hcc/bin:/opt/rocm/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2025-07-24T04:32:59.8982405Z GITHUB_BASE_REF=
2025-07-24T04:32:59.8982792Z CI=true
2025-07-24T04:32:59.8983190Z GITHUB_REPOSITORY_OWNER=pytorch
2025-07-24T04:32:59.8983693Z JOB_ID=46615439132
2025-07-24T04:32:59.8984097Z GITHUB_HEAD_REF=
2025-07-24T04:32:59.8984499Z GITHUB_ACTION_REF=
2025-07-24T04:32:59.8984916Z TEST_SHOWLOCALS=False
2025-07-24T04:32:59.8985360Z GITHUB_WORKFLOW=trunk
2025-07-24T04:32:59.8985814Z DEBIAN_FRONTEND=noninteractive
2025-07-24T04:32:59.8986987Z GITHUB_OUTPUT=/var/home/pytorchci/actions-runner/_work/_temp/_runner_file_commands/set_output_b97dc80c-a4e1-4c3d-85a5-dddd5a801622
2025-07-24T04:32:59.8988131Z NO_TD=False
2025-07-24T04:32:59.8988522Z OLDPWD=/var/lib/jenkins
2025-07-24T04:32:59.8988952Z _=/usr/bin/env
2025-07-24T04:32:59.8989392Z + echo 'Testing pytorch'
2025-07-24T04:32:59.8989841Z Testing pytorch
2025-07-24T04:32:59.8990253Z + export LANG=C.UTF-8
2025-07-24T04:32:59.8990661Z + LANG=C.UTF-8
2025-07-24T04:32:59.8991041Z + PR_NUMBER=
2025-07-24T04:32:59.8991452Z + [[ default == \d\e\f\a\u\l\t ]]
2025-07-24T04:32:59.8991966Z + export CUDA_VISIBLE_DEVICES=0
2025-07-24T04:32:59.8992455Z + CUDA_VISIBLE_DEVICES=0
2025-07-24T04:32:59.8992924Z + export HIP_VISIBLE_DEVICES=0
2025-07-24T04:32:59.8993415Z + HIP_VISIBLE_DEVICES=0
2025-07-24T04:32:59.8994270Z + [[ default == \d\i\s\t\r\i\b\u\t\e\d ]]
2025-07-24T04:32:59.8995075Z + [[ default == \s\l\o\w ]]
2025-07-24T04:32:59.8995651Z + [[ linux-jammy-rocm-py3.10 == *slow-gradcheck* ]]
2025-07-24T04:32:59.8996313Z + [[ linux-jammy-rocm-py3.10 == *cuda* ]]
2025-07-24T04:32:59.8996882Z + [[ linux-jammy-rocm-py3.10 == *rocm* ]]
2025-07-24T04:32:59.8997478Z + export PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda
2025-07-24T04:32:59.8998085Z + PYTORCH_TESTING_DEVICE_ONLY_FOR=cuda
2025-07-24T04:32:59.8998623Z + [[ default == *crossref* ]]
2025-07-24T04:32:59.8999134Z + [[ linux-jammy-rocm-py3.10 == *rocm* ]]
2025-07-24T04:32:59.8999676Z + export VALGRIND=OFF
2025-07-24T04:32:59.9000099Z + VALGRIND=OFF
2025-07-24T04:32:59.9000473Z + rocminfo
2025-07-24T04:32:59.9097971Z ROCk module version 6.10.5 is loaded
2025-07-24T04:33:00.1108035Z =====================    
2025-07-24T04:33:00.1108696Z HSA System Attributes    
2025-07-24T04:33:00.1109185Z =====================    
2025-07-24T04:33:00.1109691Z Runtime Version:         1.15
2025-07-24T04:33:00.1110202Z Runtime Ext Version:     1.7
2025-07-24T04:33:00.1110713Z System Timestamp Freq.:  1000.000000MHz
2025-07-24T04:33:00.1112227Z Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
2025-07-24T04:33:00.1113132Z Machine Model:           LARGE                              
2025-07-24T04:33:00.1113861Z System Endianness:       LITTLE                             
2025-07-24T04:33:00.1114691Z Mwaitx:                  DISABLED
2025-07-24T04:33:00.1115220Z XNACK enabled:           NO
2025-07-24T04:33:00.1115714Z DMAbuf Support:          YES
2025-07-24T04:33:00.1116191Z VMM Support:             YES
2025-07-24T04:33:00.1116488Z 
2025-07-24T04:33:00.1116659Z ==========               
2025-07-24T04:33:00.1117101Z HSA Agents               
2025-07-24T04:33:00.1117520Z ==========               
2025-07-24T04:33:00.1117920Z *******                  
2025-07-24T04:33:00.1118335Z Agent 1                  
2025-07-24T04:33:00.1118756Z *******                  
2025-07-24T04:33:00.1119297Z   Name:                    AMD EPYC 7713 64-Core Processor    
2025-07-24T04:33:00.1119979Z   Uuid:                    CPU-XX                             
2025-07-24T04:33:00.1120676Z   Marketing Name:          AMD EPYC 7713 64-Core Processor    
2025-07-24T04:33:00.1121374Z   Vendor Name:             CPU                                
2025-07-24T04:33:00.1122047Z   Feature:                 None specified                     
2025-07-24T04:33:00.1122739Z   Profile:                 FULL_PROFILE                       
2025-07-24T04:33:00.1123466Z   Float Round Mode:        NEAR                               
2025-07-24T04:33:00.1124180Z   Max Queue Number:        0(0x0)                             
2025-07-24T04:33:00.1124869Z   Queue Min Size:          0(0x0)                             
2025-07-24T04:33:00.1125531Z   Queue Max Size:          0(0x0)                             
2025-07-24T04:33:00.1126191Z   Queue Type:              MULTI                              
2025-07-24T04:33:00.1126847Z   Node:                    0                                  
2025-07-24T04:33:00.1127505Z   Device Type:             CPU                                
2025-07-24T04:33:00.1128134Z   Cache Info:              
2025-07-24T04:33:00.1128650Z     L1:                      32768(0x8000) KB                   
2025-07-24T04:33:00.1129278Z   Chip ID:                 0(0x0)                             
2025-07-24T04:33:00.1129930Z   ASIC Revision:           0(0x0)                             
2025-07-24T04:33:00.1130611Z   Cacheline Size:          64(0x40)                           
2025-07-24T04:33:00.1131359Z   Max Clock Freq. (MHz):   2000                               
2025-07-24T04:33:00.1132001Z   BDFID:                   0                                  
2025-07-24T04:33:00.1132647Z   Internal Node ID:        0                                  
2025-07-24T04:33:00.1133338Z   Compute Unit:            64                                 
2025-07-24T04:33:00.1134353Z   SIMDs per CU:            0                                  
2025-07-24T04:33:00.1135066Z   Shader Engines:          0                                  
2025-07-24T04:33:00.1135768Z   Shader Arrs. per Eng.:   0                                  
2025-07-24T04:33:00.1136762Z   WatchPts on Addr. Ranges:1                                  
2025-07-24T04:33:00.1137401Z   Memory Properties:       
2025-07-24T04:33:00.1137881Z   Features:                None
2025-07-24T04:33:00.1138351Z   Pool Info:               
2025-07-24T04:33:00.1138796Z     Pool 1                   
2025-07-24T04:33:00.1139362Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:33:00.1140042Z       Size:                    528249788(0x1f7c73bc) KB           
2025-07-24T04:33:00.1140695Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1141376Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1142101Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:33:00.1142837Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1143539Z       Accessible by all:       TRUE                               
2025-07-24T04:33:00.1144483Z     Pool 2                   
2025-07-24T04:33:00.1145045Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:33:00.1145708Z       Size:                    528249788(0x1f7c73bc) KB           
2025-07-24T04:33:00.1146362Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1147035Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1147743Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:33:00.1148458Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1149154Z       Accessible by all:       TRUE                               
2025-07-24T04:33:00.1149757Z     Pool 3                   
2025-07-24T04:33:00.1150316Z       Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
2025-07-24T04:33:00.1150967Z       Size:                    528249788(0x1f7c73bc) KB           
2025-07-24T04:33:00.1151622Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1152294Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1152998Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:33:00.1153707Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1154538Z       Accessible by all:       TRUE                               
2025-07-24T04:33:00.1155153Z     Pool 4                   
2025-07-24T04:33:00.1155707Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:33:00.1156370Z       Size:                    528249788(0x1f7c73bc) KB           
2025-07-24T04:33:00.1157015Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1157698Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1158410Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:33:00.1159135Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1159829Z       Accessible by all:       TRUE                               
2025-07-24T04:33:00.1160434Z   ISA Info:                
2025-07-24T04:33:00.1160866Z *******                  
2025-07-24T04:33:00.1161288Z Agent 2                  
2025-07-24T04:33:00.1161694Z *******                  
2025-07-24T04:33:00.1162194Z   Name:                    AMD EPYC 7713 64-Core Processor    
2025-07-24T04:33:00.1162839Z   Uuid:                    CPU-XX                             
2025-07-24T04:33:00.1163516Z   Marketing Name:          AMD EPYC 7713 64-Core Processor    
2025-07-24T04:33:00.1164212Z   Vendor Name:             CPU                                
2025-07-24T04:33:00.1164868Z   Feature:                 None specified                     
2025-07-24T04:33:00.1165860Z   Profile:                 FULL_PROFILE                       
2025-07-24T04:33:00.1166543Z   Float Round Mode:        NEAR                               
2025-07-24T04:33:00.1167233Z   Max Queue Number:        0(0x0)                             
2025-07-24T04:33:00.1167898Z   Queue Min Size:          0(0x0)                             
2025-07-24T04:33:00.1168578Z   Queue Max Size:          0(0x0)                             
2025-07-24T04:33:00.1169232Z   Queue Type:              MULTI                              
2025-07-24T04:33:00.1169846Z   Node:                    1                                  
2025-07-24T04:33:00.1170462Z   Device Type:             CPU                                
2025-07-24T04:33:00.1171052Z   Cache Info:              
2025-07-24T04:33:00.1171549Z     L1:                      32768(0x8000) KB                   
2025-07-24T04:33:00.1172157Z   Chip ID:                 0(0x0)                             
2025-07-24T04:33:00.1172808Z   ASIC Revision:           0(0x0)                             
2025-07-24T04:33:00.1173505Z   Cacheline Size:          64(0x40)                           
2025-07-24T04:33:00.1174523Z   Max Clock Freq. (MHz):   2000                               
2025-07-24T04:33:00.1175155Z   BDFID:                   0                                  
2025-07-24T04:33:00.1175792Z   Internal Node ID:        1                                  
2025-07-24T04:33:00.1176466Z   Compute Unit:            64                                 
2025-07-24T04:33:00.1177114Z   SIMDs per CU:            0                                  
2025-07-24T04:33:00.1177776Z   Shader Engines:          0                                  
2025-07-24T04:33:00.1178472Z   Shader Arrs. per Eng.:   0                                  
2025-07-24T04:33:00.1179188Z   WatchPts on Addr. Ranges:1                                  
2025-07-24T04:33:00.1179803Z   Memory Properties:       
2025-07-24T04:33:00.1180273Z   Features:                None
2025-07-24T04:33:00.1180744Z   Pool Info:               
2025-07-24T04:33:00.1181188Z     Pool 1                   
2025-07-24T04:33:00.1181751Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:33:00.1182424Z       Size:                    528402452(0x1f7ec814) KB           
2025-07-24T04:33:00.1183082Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1183771Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1184504Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:33:00.1185278Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1186018Z       Accessible by all:       TRUE                               
2025-07-24T04:33:00.1186648Z     Pool 2                   
2025-07-24T04:33:00.1187217Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:33:00.1187884Z       Size:                    528402452(0x1f7ec814) KB           
2025-07-24T04:33:00.1188555Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1189253Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1189988Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:33:00.1190714Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1191412Z       Accessible by all:       TRUE                               
2025-07-24T04:33:00.1192016Z     Pool 3                   
2025-07-24T04:33:00.1192575Z       Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
2025-07-24T04:33:00.1193228Z       Size:                    528402452(0x1f7ec814) KB           
2025-07-24T04:33:00.1193874Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1194908Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1195641Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:33:00.1196707Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1197418Z       Accessible by all:       TRUE                               
2025-07-24T04:33:00.1198034Z     Pool 4                   
2025-07-24T04:33:00.1198587Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:33:00.1199242Z       Size:                    528402452(0x1f7ec814) KB           
2025-07-24T04:33:00.1199890Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1200572Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1201289Z       Alloc Recommended Granule:4KB                                
2025-07-24T04:33:00.1202007Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1202715Z       Accessible by all:       TRUE                               
2025-07-24T04:33:00.1203332Z   ISA Info:                
2025-07-24T04:33:00.1203778Z *******                  
2025-07-24T04:33:00.1204217Z Agent 3                  
2025-07-24T04:33:00.1204631Z *******                  
2025-07-24T04:33:00.1205107Z   Name:                    gfx90a                             
2025-07-24T04:33:00.1206091Z   Uuid:                    GPU-46d3423fd94a977d               
2025-07-24T04:33:00.1206777Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.1207473Z   Vendor Name:             AMD                                
2025-07-24T04:33:00.1208143Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:33:00.1208805Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:33:00.1209487Z   Float Round Mode:        NEAR                               
2025-07-24T04:33:00.1210172Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:33:00.1210844Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:33:00.1211511Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:33:00.1212186Z   Queue Type:              MULTI                              
2025-07-24T04:33:00.1212812Z   Node:                    2                                  
2025-07-24T04:33:00.1213460Z   Device Type:             GPU                                
2025-07-24T04:33:00.1214049Z   Cache Info:              
2025-07-24T04:33:00.1214540Z     L1:                      16(0x10) KB                        
2025-07-24T04:33:00.1215126Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:33:00.1215730Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:33:00.1216382Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:33:00.1217058Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:33:00.1217741Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:33:00.1218375Z   BDFID:                   12800                              
2025-07-24T04:33:00.1219030Z   Internal Node ID:        2                                  
2025-07-24T04:33:00.1219701Z   Compute Unit:            104                                
2025-07-24T04:33:00.1220361Z   SIMDs per CU:            4                                  
2025-07-24T04:33:00.1221030Z   Shader Engines:          8                                  
2025-07-24T04:33:00.1221714Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:33:00.1222425Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:33:00.1223134Z   Coherent Host Access:    FALSE                              
2025-07-24T04:33:00.1223766Z   Memory Properties:       
2025-07-24T04:33:00.1224277Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:33:00.1224913Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:33:00.1225605Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:33:00.1226305Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1227277Z   Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1227838Z     x                        1024(0x400)                        
2025-07-24T04:33:00.1228421Z     y                        1024(0x400)                        
2025-07-24T04:33:00.1228975Z     z                        1024(0x400)                        
2025-07-24T04:33:00.1229595Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:33:00.1230292Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:33:00.1230985Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1231598Z   Grid Max Size per Dimension:
2025-07-24T04:33:00.1232096Z     x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1232663Z     y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1233228Z     z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1233889Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:33:00.1234810Z   Packet Processor uCode:: 92                                 
2025-07-24T04:33:00.1235542Z   SDMA engine uCode::      9                                  
2025-07-24T04:33:00.1236583Z   IOMMU Support::          None                               
2025-07-24T04:33:00.1237189Z   Pool Info:               
2025-07-24T04:33:00.1237638Z     Pool 1                   
2025-07-24T04:33:00.1238202Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:33:00.1238876Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1239536Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1240217Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1240942Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1241679Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1242392Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1243005Z     Pool 2                   
2025-07-24T04:33:00.1243575Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:33:00.1244240Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1244890Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1245579Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1246289Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1247010Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1247711Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1248315Z     Pool 3                   
2025-07-24T04:33:00.1248853Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:33:00.1249507Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1250281Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1251086Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1251814Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1252535Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1253235Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1253853Z     Pool 4                   
2025-07-24T04:33:00.1254384Z       Segment:                 GROUP                              
2025-07-24T04:33:00.1255008Z       Size:                    64(0x40) KB                        
2025-07-24T04:33:00.1255651Z       Allocatable:             FALSE                              
2025-07-24T04:33:00.1256337Z       Alloc Granule:           0KB                                
2025-07-24T04:33:00.1257373Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:33:00.1258103Z       Alloc Alignment:         0KB                                
2025-07-24T04:33:00.1258796Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1259417Z   ISA Info:                
2025-07-24T04:33:00.1259859Z     ISA 1                    
2025-07-24T04:33:00.1260432Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:33:00.1261185Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:33:00.1261898Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:33:00.1262606Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1263329Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1264005Z       Fast f16:                TRUE                               
2025-07-24T04:33:00.1264679Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1265333Z       Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1265902Z         x                        1024(0x400)                        
2025-07-24T04:33:00.1266759Z         y                        1024(0x400)                        
2025-07-24T04:33:00.1267322Z         z                        1024(0x400)                        
2025-07-24T04:33:00.1267942Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1268568Z       Grid Max Size per Dimension:
2025-07-24T04:33:00.1269085Z         x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1269662Z         y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1270238Z         z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1270886Z       FBarrier Max Size:       32                                 
2025-07-24T04:33:00.1271487Z *******                  
2025-07-24T04:33:00.1271922Z Agent 4                  
2025-07-24T04:33:00.1272369Z *******                  
2025-07-24T04:33:00.1272861Z   Name:                    gfx90a                             
2025-07-24T04:33:00.1273519Z   Uuid:                    GPU-95cf8055be9ab54a               
2025-07-24T04:33:00.1274225Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.1275055Z   Vendor Name:             AMD                                
2025-07-24T04:33:00.1275721Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:33:00.1276515Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:33:00.1277319Z   Float Round Mode:        NEAR                               
2025-07-24T04:33:00.1278144Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:33:00.1278826Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:33:00.1279495Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:33:00.1280156Z   Queue Type:              MULTI                              
2025-07-24T04:33:00.1280837Z   Node:                    3                                  
2025-07-24T04:33:00.1281485Z   Device Type:             GPU                                
2025-07-24T04:33:00.1282094Z   Cache Info:              
2025-07-24T04:33:00.1282589Z     L1:                      16(0x10) KB                        
2025-07-24T04:33:00.1283174Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:33:00.1283777Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:33:00.1284439Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:33:00.1285118Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:33:00.1285811Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:33:00.1286449Z   BDFID:                   13568                              
2025-07-24T04:33:00.1287091Z   Internal Node ID:        3                                  
2025-07-24T04:33:00.1288100Z   Compute Unit:            104                                
2025-07-24T04:33:00.1288775Z   SIMDs per CU:            4                                  
2025-07-24T04:33:00.1289458Z   Shader Engines:          8                                  
2025-07-24T04:33:00.1290266Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:33:00.1291116Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:33:00.1291852Z   Coherent Host Access:    FALSE                              
2025-07-24T04:33:00.1292490Z   Memory Properties:       
2025-07-24T04:33:00.1292991Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:33:00.1293629Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:33:00.1294320Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:33:00.1295020Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1295661Z   Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1296207Z     x                        1024(0x400)                        
2025-07-24T04:33:00.1296771Z     y                        1024(0x400)                        
2025-07-24T04:33:00.1297667Z     z                        1024(0x400)                        
2025-07-24T04:33:00.1298290Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:33:00.1298985Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:33:00.1299672Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1300287Z   Grid Max Size per Dimension:
2025-07-24T04:33:00.1300788Z     x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1301369Z     y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1301941Z     z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1302603Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:33:00.1303399Z   Packet Processor uCode:: 92                                 
2025-07-24T04:33:00.1304151Z   SDMA engine uCode::      9                                  
2025-07-24T04:33:00.1304880Z   IOMMU Support::          None                               
2025-07-24T04:33:00.1305496Z   Pool Info:               
2025-07-24T04:33:00.1305968Z     Pool 1                   
2025-07-24T04:33:00.1306557Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:33:00.1307261Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1307946Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1308652Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1309400Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1310142Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1310857Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1311483Z     Pool 2                   
2025-07-24T04:33:00.1312057Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:33:00.1312749Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1313416Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1314118Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1315148Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1315917Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1316762Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1317496Z     Pool 3                   
2025-07-24T04:33:00.1318140Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:33:00.1318795Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1319797Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1320506Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1321244Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1321975Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1322678Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1323297Z     Pool 4                   
2025-07-24T04:33:00.1323834Z       Segment:                 GROUP                              
2025-07-24T04:33:00.1324472Z       Size:                    64(0x40) KB                        
2025-07-24T04:33:00.1325124Z       Allocatable:             FALSE                              
2025-07-24T04:33:00.1325830Z       Alloc Granule:           0KB                                
2025-07-24T04:33:00.1326558Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:33:00.1327299Z       Alloc Alignment:         0KB                                
2025-07-24T04:33:00.1328017Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1329016Z   ISA Info:                
2025-07-24T04:33:00.1329474Z     ISA 1                    
2025-07-24T04:33:00.1330131Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:33:00.1331048Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:33:00.1331843Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:33:00.1332568Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1333303Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1333990Z       Fast f16:                TRUE                               
2025-07-24T04:33:00.1334684Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1335344Z       Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1335928Z         x                        1024(0x400)                        
2025-07-24T04:33:00.1336524Z         y                        1024(0x400)                        
2025-07-24T04:33:00.1337110Z         z                        1024(0x400)                        
2025-07-24T04:33:00.1337753Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1338388Z       Grid Max Size per Dimension:
2025-07-24T04:33:00.1338921Z         x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1339508Z         y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1340093Z         z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1340731Z       FBarrier Max Size:       32                                 
2025-07-24T04:33:00.1341340Z *******                  
2025-07-24T04:33:00.1341782Z Agent 5                  
2025-07-24T04:33:00.1342211Z *******                  
2025-07-24T04:33:00.1342713Z   Name:                    gfx90a                             
2025-07-24T04:33:00.1343377Z   Uuid:                    GPU-e94c1f19ce37e5a1               
2025-07-24T04:33:00.1344111Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.1344960Z   Vendor Name:             AMD                                
2025-07-24T04:33:00.1345778Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:33:00.1346537Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:33:00.1347229Z   Float Round Mode:        NEAR                               
2025-07-24T04:33:00.1347928Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:33:00.1348623Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:33:00.1349301Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:33:00.1350064Z   Queue Type:              MULTI                              
2025-07-24T04:33:00.1350970Z   Node:                    4                                  
2025-07-24T04:33:00.1351625Z   Device Type:             GPU                                
2025-07-24T04:33:00.1352242Z   Cache Info:              
2025-07-24T04:33:00.1352762Z     L1:                      16(0x10) KB                        
2025-07-24T04:33:00.1353360Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:33:00.1353972Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:33:00.1354765Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:33:00.1355466Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:33:00.1356158Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:33:00.1356804Z   BDFID:                   4352                               
2025-07-24T04:33:00.1357463Z   Internal Node ID:        4                                  
2025-07-24T04:33:00.1358141Z   Compute Unit:            104                                
2025-07-24T04:33:00.1358810Z   SIMDs per CU:            4                                  
2025-07-24T04:33:00.1359486Z   Shader Engines:          8                                  
2025-07-24T04:33:00.1360539Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:33:00.1361275Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:33:00.1362009Z   Coherent Host Access:    FALSE                              
2025-07-24T04:33:00.1362663Z   Memory Properties:       
2025-07-24T04:33:00.1363187Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:33:00.1363843Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:33:00.1364555Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:33:00.1365280Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1365933Z   Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1366494Z     x                        1024(0x400)                        
2025-07-24T04:33:00.1367077Z     y                        1024(0x400)                        
2025-07-24T04:33:00.1367648Z     z                        1024(0x400)                        
2025-07-24T04:33:00.1368298Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:33:00.1369014Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:33:00.1369718Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1370456Z   Grid Max Size per Dimension:
2025-07-24T04:33:00.1371058Z     x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1371711Z     y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1372305Z     z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1372968Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:33:00.1373727Z   Packet Processor uCode:: 92                                 
2025-07-24T04:33:00.1374465Z   SDMA engine uCode::      9                                  
2025-07-24T04:33:00.1375175Z   IOMMU Support::          None                               
2025-07-24T04:33:00.1375809Z   Pool Info:               
2025-07-24T04:33:00.1376266Z     Pool 1                   
2025-07-24T04:33:00.1376845Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:33:00.1377533Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1378212Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1378917Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1379660Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1380415Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1381126Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1381745Z     Pool 2                   
2025-07-24T04:33:00.1382650Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:33:00.1383344Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1384025Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1384759Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1385493Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1386264Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1386983Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1387605Z     Pool 3                   
2025-07-24T04:33:00.1388157Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:33:00.1388840Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1389512Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1390285Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1391015Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1392036Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1392754Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1393376Z     Pool 4                   
2025-07-24T04:33:00.1393919Z       Segment:                 GROUP                              
2025-07-24T04:33:00.1394672Z       Size:                    64(0x40) KB                        
2025-07-24T04:33:00.1395332Z       Allocatable:             FALSE                              
2025-07-24T04:33:00.1396027Z       Alloc Granule:           0KB                                
2025-07-24T04:33:00.1396758Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:33:00.1397490Z       Alloc Alignment:         0KB                                
2025-07-24T04:33:00.1398213Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1398843Z   ISA Info:                
2025-07-24T04:33:00.1399306Z     ISA 1                    
2025-07-24T04:33:00.1399897Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:33:00.1400664Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:33:00.1401397Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:33:00.1402117Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1402858Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1403542Z       Fast f16:                TRUE                               
2025-07-24T04:33:00.1404234Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1404879Z       Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1405445Z         x                        1024(0x400)                        
2025-07-24T04:33:00.1406037Z         y                        1024(0x400)                        
2025-07-24T04:33:00.1406615Z         z                        1024(0x400)                        
2025-07-24T04:33:00.1407271Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1407902Z       Grid Max Size per Dimension:
2025-07-24T04:33:00.1408432Z         x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1409014Z         y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1409591Z         z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1410247Z       FBarrier Max Size:       32                                 
2025-07-24T04:33:00.1410858Z *******                  
2025-07-24T04:33:00.1411301Z Agent 6                  
2025-07-24T04:33:00.1411737Z *******                  
2025-07-24T04:33:00.1412222Z   Name:                    gfx90a                             
2025-07-24T04:33:00.1413198Z   Uuid:                    GPU-5159f7f6b913f996               
2025-07-24T04:33:00.1413931Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.1414658Z   Vendor Name:             AMD                                
2025-07-24T04:33:00.1415337Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:33:00.1416019Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:33:00.1416709Z   Float Round Mode:        NEAR                               
2025-07-24T04:33:00.1417406Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:33:00.1418093Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:33:00.1418765Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:33:00.1419437Z   Queue Type:              MULTI                              
2025-07-24T04:33:00.1420074Z   Node:                    5                                  
2025-07-24T04:33:00.1420720Z   Device Type:             GPU                                
2025-07-24T04:33:00.1421355Z   Cache Info:              
2025-07-24T04:33:00.1421857Z     L1:                      16(0x10) KB                        
2025-07-24T04:33:00.1422778Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:33:00.1423393Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:33:00.1424052Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:33:00.1424746Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:33:00.1425444Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:33:00.1426096Z   BDFID:                   5120                               
2025-07-24T04:33:00.1426756Z   Internal Node ID:        5                                  
2025-07-24T04:33:00.1427441Z   Compute Unit:            104                                
2025-07-24T04:33:00.1428114Z   SIMDs per CU:            4                                  
2025-07-24T04:33:00.1428784Z   Shader Engines:          8                                  
2025-07-24T04:33:00.1429482Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:33:00.1430211Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:33:00.1430920Z   Coherent Host Access:    FALSE                              
2025-07-24T04:33:00.1431557Z   Memory Properties:       
2025-07-24T04:33:00.1432068Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:33:00.1432706Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:33:00.1433405Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:33:00.1434110Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1435100Z   Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1435655Z     x                        1024(0x400)                        
2025-07-24T04:33:00.1436239Z     y                        1024(0x400)                        
2025-07-24T04:33:00.1436807Z     z                        1024(0x400)                        
2025-07-24T04:33:00.1437439Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:33:00.1438150Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:33:00.1438843Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1439453Z   Grid Max Size per Dimension:
2025-07-24T04:33:00.1439963Z     x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1440570Z     y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1441148Z     z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1441736Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:33:00.1442404Z   Packet Processor uCode:: 92                                 
2025-07-24T04:33:00.1443051Z   SDMA engine uCode::      9                                  
2025-07-24T04:33:00.1443970Z   IOMMU Support::          None                               
2025-07-24T04:33:00.1444521Z   Pool Info:               
2025-07-24T04:33:00.1444940Z     Pool 1                   
2025-07-24T04:33:00.1445453Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:33:00.1446067Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1446663Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1447280Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1447917Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1448574Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1449201Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1449754Z     Pool 2                   
2025-07-24T04:33:00.1450257Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:33:00.1450863Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1451451Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1452400Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1453042Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1453696Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1454323Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1454868Z     Pool 3                   
2025-07-24T04:33:00.1455354Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:33:00.1455921Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1456507Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1457124Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1457770Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1458414Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1459048Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1459591Z     Pool 4                   
2025-07-24T04:33:00.1460060Z       Segment:                 GROUP                              
2025-07-24T04:33:00.1460622Z       Size:                    64(0x40) KB                        
2025-07-24T04:33:00.1461204Z       Allocatable:             FALSE                              
2025-07-24T04:33:00.1461817Z       Alloc Granule:           0KB                                
2025-07-24T04:33:00.1462460Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:33:00.1463106Z       Alloc Alignment:         0KB                                
2025-07-24T04:33:00.1463734Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1464277Z   ISA Info:                
2025-07-24T04:33:00.1464686Z     ISA 1                    
2025-07-24T04:33:00.1465201Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:33:00.1465881Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:33:00.1466527Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:33:00.1467161Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1467812Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1468421Z       Fast f16:                TRUE                               
2025-07-24T04:33:00.1469030Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1469612Z       Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1470115Z         x                        1024(0x400)                        
2025-07-24T04:33:00.1470898Z         y                        1024(0x400)                        
2025-07-24T04:33:00.1471418Z         z                        1024(0x400)                        
2025-07-24T04:33:00.1471991Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1472541Z       Grid Max Size per Dimension:
2025-07-24T04:33:00.1473009Z         x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1473536Z         y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1474054Z         z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1474732Z       FBarrier Max Size:       32                                 
2025-07-24T04:33:00.1475271Z *******                  
2025-07-24T04:33:00.1475663Z Agent 7                  
2025-07-24T04:33:00.1476052Z *******                  
2025-07-24T04:33:00.1476509Z   Name:                    gfx90a                             
2025-07-24T04:33:00.1477072Z   Uuid:                    GPU-b7e9520488228d14               
2025-07-24T04:33:00.1477694Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.1478324Z   Vendor Name:             AMD                                
2025-07-24T04:33:00.1479219Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:33:00.1479824Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:33:00.1480435Z   Float Round Mode:        NEAR                               
2025-07-24T04:33:00.1481048Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:33:00.1481652Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:33:00.1482240Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:33:00.1482823Z   Queue Type:              MULTI                              
2025-07-24T04:33:00.1483379Z   Node:                    6                                  
2025-07-24T04:33:00.1483943Z   Device Type:             GPU                                
2025-07-24T04:33:00.1484477Z   Cache Info:              
2025-07-24T04:33:00.1484920Z     L1:                      16(0x10) KB                        
2025-07-24T04:33:00.1485451Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:33:00.1485988Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:33:00.1486573Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:33:00.1487169Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:33:00.1487781Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:33:00.1488350Z   BDFID:                   44544                              
2025-07-24T04:33:00.1488933Z   Internal Node ID:        6                                  
2025-07-24T04:33:00.1489549Z   Compute Unit:            104                                
2025-07-24T04:33:00.1490143Z   SIMDs per CU:            4                                  
2025-07-24T04:33:00.1490743Z   Shader Engines:          8                                  
2025-07-24T04:33:00.1491367Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:33:00.1492019Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:33:00.1492663Z   Coherent Host Access:    FALSE                              
2025-07-24T04:33:00.1493234Z   Memory Properties:       
2025-07-24T04:33:00.1493687Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:33:00.1494256Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:33:00.1494868Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:33:00.1495498Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1496078Z   Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1496562Z     x                        1024(0x400)                        
2025-07-24T04:33:00.1497060Z     y                        1024(0x400)                        
2025-07-24T04:33:00.1511410Z     z                        1024(0x400)                        
2025-07-24T04:33:00.1512162Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:33:00.1512843Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:33:00.1513504Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1514078Z   Grid Max Size per Dimension:
2025-07-24T04:33:00.1514911Z     x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1515486Z     y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1515994Z     z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1516574Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:33:00.1517263Z   Packet Processor uCode:: 92                                 
2025-07-24T04:33:00.1517916Z   SDMA engine uCode::      9                                  
2025-07-24T04:33:00.1518531Z   IOMMU Support::          None                               
2025-07-24T04:33:00.1519092Z   Pool Info:               
2025-07-24T04:33:00.1519523Z     Pool 1                   
2025-07-24T04:33:00.1520034Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:33:00.1520973Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1521561Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1522167Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1522812Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1523455Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1524082Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1524623Z     Pool 2                   
2025-07-24T04:33:00.1525112Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:33:00.1525723Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1526345Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1526980Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1527644Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1528296Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1528922Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1529458Z     Pool 3                   
2025-07-24T04:33:00.1529954Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:33:00.1530534Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1531120Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1531731Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1532376Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1533026Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1533650Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1534194Z     Pool 4                   
2025-07-24T04:33:00.1534667Z       Segment:                 GROUP                              
2025-07-24T04:33:00.1535226Z       Size:                    64(0x40) KB                        
2025-07-24T04:33:00.1535799Z       Allocatable:             FALSE                              
2025-07-24T04:33:00.1536413Z       Alloc Granule:           0KB                                
2025-07-24T04:33:00.1537052Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:33:00.1537690Z       Alloc Alignment:         0KB                                
2025-07-24T04:33:00.1538314Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1538855Z   ISA Info:                
2025-07-24T04:33:00.1539546Z     ISA 1                    
2025-07-24T04:33:00.1540073Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:33:00.1540750Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:33:00.1541389Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:33:00.1542025Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1542677Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1543272Z       Fast f16:                TRUE                               
2025-07-24T04:33:00.1543875Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1544457Z       Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1544977Z         x                        1024(0x400)                        
2025-07-24T04:33:00.1545491Z         y                        1024(0x400)                        
2025-07-24T04:33:00.1545999Z         z                        1024(0x400)                        
2025-07-24T04:33:00.1546558Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1547356Z       Grid Max Size per Dimension:
2025-07-24T04:33:00.1547828Z         x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1548337Z         y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1548839Z         z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1549403Z       FBarrier Max Size:       32                                 
2025-07-24T04:33:00.1549934Z *******                  
2025-07-24T04:33:00.1550316Z Agent 8                  
2025-07-24T04:33:00.1550692Z *******                  
2025-07-24T04:33:00.1551120Z   Name:                    gfx90a                             
2025-07-24T04:33:00.1551687Z   Uuid:                    GPU-9dc1bf8373e5d434               
2025-07-24T04:33:00.1552300Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.1552922Z   Vendor Name:             AMD                                
2025-07-24T04:33:00.1553509Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:33:00.1554114Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:33:00.1554850Z   Float Round Mode:        NEAR                               
2025-07-24T04:33:00.1555459Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:33:00.1556054Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:33:00.1556633Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:33:00.1557218Z   Queue Type:              MULTI                              
2025-07-24T04:33:00.1557769Z   Node:                    7                                  
2025-07-24T04:33:00.1558325Z   Device Type:             GPU                                
2025-07-24T04:33:00.1558864Z   Cache Info:              
2025-07-24T04:33:00.1559304Z     L1:                      16(0x10) KB                        
2025-07-24T04:33:00.1559824Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:33:00.1560372Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:33:00.1560948Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:33:00.1561552Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:33:00.1562148Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:33:00.1562713Z   BDFID:                   45824                              
2025-07-24T04:33:00.1563292Z   Internal Node ID:        7                                  
2025-07-24T04:33:00.1563879Z   Compute Unit:            104                                
2025-07-24T04:33:00.1564453Z   SIMDs per CU:            4                                  
2025-07-24T04:33:00.1565042Z   Shader Engines:          8                                  
2025-07-24T04:33:00.1566001Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:33:00.1566654Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:33:00.1567302Z   Coherent Host Access:    FALSE                              
2025-07-24T04:33:00.1567867Z   Memory Properties:       
2025-07-24T04:33:00.1568312Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:33:00.1568877Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:33:00.1569491Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:33:00.1570109Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1570681Z   Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1571161Z     x                        1024(0x400)                        
2025-07-24T04:33:00.1571663Z     y                        1024(0x400)                        
2025-07-24T04:33:00.1572148Z     z                        1024(0x400)                        
2025-07-24T04:33:00.1572705Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:33:00.1573324Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:33:00.1574223Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1574765Z   Grid Max Size per Dimension:
2025-07-24T04:33:00.1575236Z     x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1575781Z     y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1576303Z     z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1576911Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:33:00.1577597Z   Packet Processor uCode:: 92                                 
2025-07-24T04:33:00.1578262Z   SDMA engine uCode::      9                                  
2025-07-24T04:33:00.1578886Z   IOMMU Support::          None                               
2025-07-24T04:33:00.1579436Z   Pool Info:               
2025-07-24T04:33:00.1579857Z     Pool 1                   
2025-07-24T04:33:00.1580367Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:33:00.1580984Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1581579Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1582188Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1582837Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1583498Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1584129Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1584674Z     Pool 2                   
2025-07-24T04:33:00.1585177Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:33:00.1585768Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1586349Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1586961Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1587612Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1588255Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1588884Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1589429Z     Pool 3                   
2025-07-24T04:33:00.1589912Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:33:00.1590483Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1591052Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1591659Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1592292Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1593184Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1593821Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1594455Z     Pool 4                   
2025-07-24T04:33:00.1594928Z       Segment:                 GROUP                              
2025-07-24T04:33:00.1595500Z       Size:                    64(0x40) KB                        
2025-07-24T04:33:00.1596066Z       Allocatable:             FALSE                              
2025-07-24T04:33:00.1596677Z       Alloc Granule:           0KB                                
2025-07-24T04:33:00.1597333Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:33:00.1597971Z       Alloc Alignment:         0KB                                
2025-07-24T04:33:00.1598592Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1599133Z   ISA Info:                
2025-07-24T04:33:00.1599525Z     ISA 1                    
2025-07-24T04:33:00.1600042Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:33:00.1600705Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:33:00.1601647Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:33:00.1602276Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1602926Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1603522Z       Fast f16:                TRUE                               
2025-07-24T04:33:00.1604126Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1604706Z       Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1605209Z         x                        1024(0x400)                        
2025-07-24T04:33:00.1605718Z         y                        1024(0x400)                        
2025-07-24T04:33:00.1606215Z         z                        1024(0x400)                        
2025-07-24T04:33:00.1606777Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1607331Z       Grid Max Size per Dimension:
2025-07-24T04:33:00.1607806Z         x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1608317Z         y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1608838Z         z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1609411Z       FBarrier Max Size:       32                                 
2025-07-24T04:33:00.1609937Z *******                  
2025-07-24T04:33:00.1610317Z Agent 9                  
2025-07-24T04:33:00.1610689Z *******                  
2025-07-24T04:33:00.1611117Z   Name:                    gfx90a                             
2025-07-24T04:33:00.1611682Z   Uuid:                    GPU-2eced76570c2b642               
2025-07-24T04:33:00.1612298Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.1612923Z   Vendor Name:             AMD                                
2025-07-24T04:33:00.1613514Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:33:00.1614104Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:33:00.1614700Z   Float Round Mode:        NEAR                               
2025-07-24T04:33:00.1615308Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:33:00.1615906Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:33:00.1616488Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:33:00.1617069Z   Queue Type:              MULTI                              
2025-07-24T04:33:00.1617617Z   Node:                    8                                  
2025-07-24T04:33:00.1618176Z   Device Type:             GPU                                
2025-07-24T04:33:00.1618705Z   Cache Info:              
2025-07-24T04:33:00.1619148Z     L1:                      16(0x10) KB                        
2025-07-24T04:33:00.1619970Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:33:00.1620528Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:33:00.1621137Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:33:00.1621748Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:33:00.1622362Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:33:00.1622926Z   BDFID:                   36352                              
2025-07-24T04:33:00.1623516Z   Internal Node ID:        8                                  
2025-07-24T04:33:00.1624114Z   Compute Unit:            104                                
2025-07-24T04:33:00.1624703Z   SIMDs per CU:            4                                  
2025-07-24T04:33:00.1625309Z   Shader Engines:          8                                  
2025-07-24T04:33:00.1625938Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:33:00.1626594Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:33:00.1627245Z   Coherent Host Access:    FALSE                              
2025-07-24T04:33:00.1628064Z   Memory Properties:       
2025-07-24T04:33:00.1628523Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:33:00.1629088Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:33:00.1629715Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:33:00.1630335Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1630907Z   Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1631391Z     x                        1024(0x400)                        
2025-07-24T04:33:00.1631901Z     y                        1024(0x400)                        
2025-07-24T04:33:00.1632391Z     z                        1024(0x400)                        
2025-07-24T04:33:00.1632944Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:33:00.1633567Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:33:00.1634182Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1635003Z   Grid Max Size per Dimension:
2025-07-24T04:33:00.1635481Z     x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1636008Z     y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1636510Z     z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1637091Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:33:00.1637754Z   Packet Processor uCode:: 92                                 
2025-07-24T04:33:00.1638393Z   SDMA engine uCode::      9                                  
2025-07-24T04:33:00.1639014Z   IOMMU Support::          None                               
2025-07-24T04:33:00.1639553Z   Pool Info:               
2025-07-24T04:33:00.1639964Z     Pool 1                   
2025-07-24T04:33:00.1640474Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:33:00.1641079Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1641681Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1642296Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1642936Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1643587Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1644216Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1644757Z     Pool 2                   
2025-07-24T04:33:00.1645253Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:33:00.1645836Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1646415Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1647324Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1647978Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1648630Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1649254Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1649794Z     Pool 3                   
2025-07-24T04:33:00.1650277Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:33:00.1650849Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1651420Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1652025Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1652658Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1653297Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1653928Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1654463Z     Pool 4                   
2025-07-24T04:33:00.1654925Z       Segment:                 GROUP                              
2025-07-24T04:33:00.1655774Z       Size:                    64(0x40) KB                        
2025-07-24T04:33:00.1656347Z       Allocatable:             FALSE                              
2025-07-24T04:33:00.1656959Z       Alloc Granule:           0KB                                
2025-07-24T04:33:00.1657592Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:33:00.1658239Z       Alloc Alignment:         0KB                                
2025-07-24T04:33:00.1658866Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1659410Z   ISA Info:                
2025-07-24T04:33:00.1659801Z     ISA 1                    
2025-07-24T04:33:00.1660308Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:33:00.1660979Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:33:00.1661623Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:33:00.1662262Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1662914Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1663516Z       Fast f16:                TRUE                               
2025-07-24T04:33:00.1664117Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1664698Z       Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1665200Z         x                        1024(0x400)                        
2025-07-24T04:33:00.1665709Z         y                        1024(0x400)                        
2025-07-24T04:33:00.1666214Z         z                        1024(0x400)                        
2025-07-24T04:33:00.1666769Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1667331Z       Grid Max Size per Dimension:
2025-07-24T04:33:00.1667801Z         x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1668331Z         y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1668845Z         z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1669413Z       FBarrier Max Size:       32                                 
2025-07-24T04:33:00.1669938Z *******                  
2025-07-24T04:33:00.1670318Z Agent 10                 
2025-07-24T04:33:00.1670689Z *******                  
2025-07-24T04:33:00.1671112Z   Name:                    gfx90a                             
2025-07-24T04:33:00.1671682Z   Uuid:                    GPU-404705ee5bcc7f87               
2025-07-24T04:33:00.1672295Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.1672920Z   Vendor Name:             AMD                                
2025-07-24T04:33:00.1673794Z   Feature:                 KERNEL_DISPATCH                    
2025-07-24T04:33:00.1674523Z   Profile:                 BASE_PROFILE                       
2025-07-24T04:33:00.1675183Z   Float Round Mode:        NEAR                               
2025-07-24T04:33:00.1675797Z   Max Queue Number:        128(0x80)                          
2025-07-24T04:33:00.1676399Z   Queue Min Size:          64(0x40)                           
2025-07-24T04:33:00.1676994Z   Queue Max Size:          131072(0x20000)                    
2025-07-24T04:33:00.1677578Z   Queue Type:              MULTI                              
2025-07-24T04:33:00.1678130Z   Node:                    9                                  
2025-07-24T04:33:00.1678693Z   Device Type:             GPU                                
2025-07-24T04:33:00.1679218Z   Cache Info:              
2025-07-24T04:33:00.1679648Z     L1:                      16(0x10) KB                        
2025-07-24T04:33:00.1680167Z     L2:                      8192(0x2000) KB                    
2025-07-24T04:33:00.1680696Z   Chip ID:                 29708(0x740c)                      
2025-07-24T04:33:00.1681275Z   ASIC Revision:           1(0x1)                             
2025-07-24T04:33:00.1682193Z   Cacheline Size:          128(0x80)                          
2025-07-24T04:33:00.1682794Z   Max Clock Freq. (MHz):   1700                               
2025-07-24T04:33:00.1683363Z   BDFID:                   37632                              
2025-07-24T04:33:00.1683936Z   Internal Node ID:        9                                  
2025-07-24T04:33:00.1684538Z   Compute Unit:            104                                
2025-07-24T04:33:00.1685114Z   SIMDs per CU:            4                                  
2025-07-24T04:33:00.1685711Z   Shader Engines:          8                                  
2025-07-24T04:33:00.1686334Z   Shader Arrs. per Eng.:   1                                  
2025-07-24T04:33:00.1686967Z   WatchPts on Addr. Ranges:4                                  
2025-07-24T04:33:00.1687618Z   Coherent Host Access:    FALSE                              
2025-07-24T04:33:00.1688189Z   Memory Properties:       
2025-07-24T04:33:00.1688644Z   Features:                KERNEL_DISPATCH 
2025-07-24T04:33:00.1689207Z   Fast F16 Operation:      TRUE                               
2025-07-24T04:33:00.1689832Z   Wavefront Size:          64(0x40)                           
2025-07-24T04:33:00.1690450Z   Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1691024Z   Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1691520Z     x                        1024(0x400)                        
2025-07-24T04:33:00.1692027Z     y                        1024(0x400)                        
2025-07-24T04:33:00.1692510Z     z                        1024(0x400)                        
2025-07-24T04:33:00.1693056Z   Max Waves Per CU:        32(0x20)                           
2025-07-24T04:33:00.1693669Z   Max Work-item Per CU:    2048(0x800)                        
2025-07-24T04:33:00.1694289Z   Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1694834Z   Grid Max Size per Dimension:
2025-07-24T04:33:00.1695286Z     x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1695794Z     y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1696293Z     z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1696870Z   Max fbarriers/Workgrp:   32                                 
2025-07-24T04:33:00.1697537Z   Packet Processor uCode:: 92                                 
2025-07-24T04:33:00.1698179Z   SDMA engine uCode::      9                                  
2025-07-24T04:33:00.1698800Z   IOMMU Support::          None                               
2025-07-24T04:33:00.1699334Z   Pool Info:               
2025-07-24T04:33:00.1699736Z     Pool 1                   
2025-07-24T04:33:00.1700228Z       Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
2025-07-24T04:33:00.1701121Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1701728Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1702358Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1703002Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1703650Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1704274Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1704819Z     Pool 2                   
2025-07-24T04:33:00.1705320Z       Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
2025-07-24T04:33:00.1705907Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1706481Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1707089Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1707730Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1708371Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1709237Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1709775Z     Pool 3                   
2025-07-24T04:33:00.1710263Z       Segment:                 GLOBAL; FLAGS: FINE GRAINED        
2025-07-24T04:33:00.1710834Z       Size:                    67092480(0x3ffc000) KB             
2025-07-24T04:33:00.1711416Z       Allocatable:             TRUE                               
2025-07-24T04:33:00.1712027Z       Alloc Granule:           4KB                                
2025-07-24T04:33:00.1712661Z       Alloc Recommended Granule:2048KB                             
2025-07-24T04:33:00.1713305Z       Alloc Alignment:         4KB                                
2025-07-24T04:33:00.1713935Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1714592Z     Pool 4                   
2025-07-24T04:33:00.1715059Z       Segment:                 GROUP                              
2025-07-24T04:33:00.1715628Z       Size:                    64(0x40) KB                        
2025-07-24T04:33:00.1716203Z       Allocatable:             FALSE                              
2025-07-24T04:33:00.1716809Z       Alloc Granule:           0KB                                
2025-07-24T04:33:00.1717449Z       Alloc Recommended Granule:0KB                                
2025-07-24T04:33:00.1718093Z       Alloc Alignment:         0KB                                
2025-07-24T04:33:00.1718721Z       Accessible by all:       FALSE                              
2025-07-24T04:33:00.1719263Z   ISA Info:                
2025-07-24T04:33:00.1719657Z     ISA 1                    
2025-07-24T04:33:00.1720163Z       Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
2025-07-24T04:33:00.1720832Z       Machine Models:          HSA_MACHINE_MODEL_LARGE            
2025-07-24T04:33:00.1721478Z       Profiles:                HSA_PROFILE_BASE                   
2025-07-24T04:33:00.1722115Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1722769Z       Default Rounding Mode:   NEAR                               
2025-07-24T04:33:00.1723369Z       Fast f16:                TRUE                               
2025-07-24T04:33:00.1723971Z       Workgroup Max Size:      1024(0x400)                        
2025-07-24T04:33:00.1724547Z       Workgroup Max Size per Dimension:
2025-07-24T04:33:00.1725044Z         x                        1024(0x400)                        
2025-07-24T04:33:00.1725557Z         y                        1024(0x400)                        
2025-07-24T04:33:00.1726059Z         z                        1024(0x400)                        
2025-07-24T04:33:00.1726618Z       Grid Max Size:           4294967295(0xffffffff)             
2025-07-24T04:33:00.1727170Z       Grid Max Size per Dimension:
2025-07-24T04:33:00.1727937Z         x                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1728496Z         y                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1729026Z         z                        4294967295(0xffffffff)             
2025-07-24T04:33:00.1729608Z       FBarrier Max Size:       32                                 
2025-07-24T04:33:00.1730155Z *** Done ***             
2025-07-24T04:33:00.1730548Z + rocminfo
2025-07-24T04:33:00.1730916Z + grep -E 'Name:.*\sgfx|Marketing'
2025-07-24T04:33:00.3541496Z   Marketing Name:          AMD EPYC 7713 64-Core Processor    
2025-07-24T04:33:00.3542392Z   Marketing Name:          AMD EPYC 7713 64-Core Processor    
2025-07-24T04:33:00.3543149Z   Name:                    gfx90a                             
2025-07-24T04:33:00.3543839Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.3544528Z   Name:                    gfx90a                             
2025-07-24T04:33:00.3545205Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.3545856Z   Name:                    gfx90a                             
2025-07-24T04:33:00.3547056Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.3547700Z   Name:                    gfx90a                             
2025-07-24T04:33:00.3548353Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.3549011Z   Name:                    gfx90a                             
2025-07-24T04:33:00.3549663Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.3550315Z   Name:                    gfx90a                             
2025-07-24T04:33:00.3550960Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.3564720Z   Name:                    gfx90a                             
2025-07-24T04:33:00.3565530Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.3566270Z   Name:                    gfx90a                             
2025-07-24T04:33:00.3567020Z   Marketing Name:          AMD Instinct MI250X/MI250          
2025-07-24T04:33:00.3748537Z + MAYBE_ROCM=rocm/
2025-07-24T04:33:00.3749109Z + [[ linux-jammy-rocm-py3.10 == *xpu* ]]
2025-07-24T04:33:00.3749776Z + [[ linux-jammy-rocm-py3.10 != *-bazel-* ]]
2025-07-24T04:33:00.3750384Z + pip_install ninja==1.10.2
2025-07-24T04:33:00.3751076Z + pip_install_pkg='python3 -m pip install --progress-bar off'
2025-07-24T04:33:00.3751938Z + python3 -m pip install --progress-bar off ninja==1.10.2
2025-07-24T04:33:00.8628238Z Collecting ninja==1.10.2
2025-07-24T04:33:00.8840703Z   Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl.metadata (5.0 kB)
2025-07-24T04:33:00.8925365Z Downloading ninja-1.10.2-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (108 kB)
2025-07-24T04:33:01.3698728Z Installing collected packages: ninja
2025-07-24T04:33:01.3699157Z   Attempting uninstall: ninja
2025-07-24T04:33:01.3703940Z     Found existing installation: ninja 1.11.1.3
2025-07-24T04:33:01.3729260Z     Uninstalling ninja-1.11.1.3:
2025-07-24T04:33:01.3815417Z       Successfully uninstalled ninja-1.11.1.3
2025-07-24T04:33:01.4134442Z Successfully installed ninja-1.10.2
2025-07-24T04:33:01.4843816Z + export PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/rocm/llvm/bin:/opt/rocm/opencl/bin:/opt/rocm/hip/bin:/opt/rocm/hcc/bin:/opt/rocm/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2025-07-24T04:33:01.4845993Z + PATH=/var/lib/jenkins/.local/bin:/opt/cache/bin:/opt/rocm/llvm/bin:/opt/rocm/opencl/bin:/opt/rocm/hip/bin:/opt/rocm/hcc/bin:/opt/rocm/bin:/opt/conda/envs/py_3.10/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2025-07-24T04:33:01.4847199Z + [[ linux-jammy-rocm-py3.10 == *aarch64* ]]
2025-07-24T04:33:01.4847580Z + [[ linux-jammy-rocm-py3.10 == *asan* ]]
2025-07-24T04:33:01.4847941Z + [[ linux-jammy-rocm-py3.10 == *-debug* ]]
2025-07-24T04:33:01.4848797Z + [[ linux-jammy-rocm-py3.10 != *-bazel-* ]]
2025-07-24T04:33:01.4849328Z + echo 'We are not in debug mode: linux-jammy-rocm-py3.10. Expect the assertion to pass'
2025-07-24T04:33:01.4850013Z We are not in debug mode: linux-jammy-rocm-py3.10. Expect the assertion to pass
2025-07-24T04:33:01.4850546Z + cd test
2025-07-24T04:33:01.4850928Z + python -c 'import torch; torch._C._crash_if_debug_asserts_fail(424242)'
2025-07-24T04:33:03.1593847Z + [[ default == \n\o\g\p\u\_\N\O\_\A\V\X\2 ]]
2025-07-24T04:33:03.1594744Z + [[ default == \n\o\g\p\u\_\A\V\X\5\1\2 ]]
2025-07-24T04:33:03.1595171Z + [[ default == \l\e\g\a\c\y\_\n\v\i\d\i\a\_\d\r\i\v\e\r ]]
2025-07-24T04:33:03.1602685Z + DYNAMO_BENCHMARK_FLAGS=()
2025-07-24T04:33:03.1603325Z + [[ default == *pr_time_benchmarks* ]]
2025-07-24T04:33:03.1603907Z + [[ default == *dynamo_eager* ]]
2025-07-24T04:33:03.1604430Z + [[ default == *aot_eager* ]]
2025-07-24T04:33:03.1604972Z + [[ default == *aot_inductor* ]]
2025-07-24T04:33:03.1605536Z + [[ default == *max_autotune_inductor* ]]
2025-07-24T04:33:03.1606103Z + [[ default == *inductor* ]]
2025-07-24T04:33:03.1606625Z + [[ default == *dynamic* ]]
2025-07-24T04:33:03.1608120Z + [[ default == *cpu* ]]
2025-07-24T04:33:03.1608749Z + DYNAMO_BENCHMARK_FLAGS+=(--device cuda)
2025-07-24T04:33:03.1646386Z + [[ linux-jammy-rocm-py3.10 == *libtorch* ]]
2025-07-24T04:33:03.1647278Z + [[ linux-jammy-rocm-py3.10 == *-bazel-* ]]
2025-07-24T04:33:03.1656119Z + cd test
2025-07-24T04:33:03.1656518Z + python -c 'import torch; print(torch.__config__.show())'
2025-07-24T04:33:04.5617302Z PyTorch built with:
2025-07-24T04:33:04.5617738Z   - GCC 11.4
2025-07-24T04:33:04.5618046Z   - C++ Version: 201703
2025-07-24T04:33:04.5618703Z   - Intel(R) oneAPI Math Kernel Library Version 2024.2-Product Build 20240605 for Intel(R) 64 architecture applications
2025-07-24T04:33:04.5619479Z   - Intel(R) MKL-DNN v3.7.1 (Git Hash 8d263e693366ef8db40acc569cc7d8edf644556d)
2025-07-24T04:33:04.5619968Z   - OpenMP 201511 (a.k.a. OpenMP 4.5)
2025-07-24T04:33:04.5620382Z   - LAPACK is enabled (usually provided by MKL)
2025-07-24T04:33:04.5620744Z   - NNPACK is enabled
2025-07-24T04:33:04.5621051Z   - CPU capability usage: AVX2
2025-07-24T04:33:04.5621369Z   - HIP Runtime 6.4.43484
2025-07-24T04:33:04.5621662Z   - MIOpen 3.4.0
2025-07-24T04:33:04.5621916Z   - Magma 2.7.2
2025-07-24T04:33:04.5626514Z   - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, COMMIT_SHA=27c8ef1839e0747b36769efb192befc9308ff93c, CXX_COMPILER=/opt/cache/bin/c++, CXX_FLAGS= -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -DC10_NODEPRECATED -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=range-loop-construct -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -faligned-new -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, TORCH_VERSION=2.9.0, USE_CUDA=OFF, USE_CUDNN=OFF, USE_CUSPARSELT=OFF, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=ON, USE_ROCM_KERNEL_ASSERT=OFF, USE_XCCL=OFF, USE_XPU=OFF, 
2025-07-24T04:33:04.5631238Z 
2025-07-24T04:33:04.8695986Z + cd test
2025-07-24T04:33:04.8696753Z + python -c 'import torch; print(torch.__config__.parallel_info())'
2025-07-24T04:33:06.0414138Z ATen/Parallel:
2025-07-24T04:33:06.0414486Z 	at::get_num_threads() : 128
2025-07-24T04:33:06.0414846Z 	at::get_num_interop_threads() : 128
2025-07-24T04:33:06.0415154Z OpenMP 201511 (a.k.a. OpenMP 4.5)
2025-07-24T04:33:06.0415437Z 	omp_get_max_threads() : 128
2025-07-24T04:33:06.0416394Z Intel(R) oneAPI Math Kernel Library Version 2024.2-Product Build 20240605 for Intel(R) 64 architecture applications
2025-07-24T04:33:06.0416956Z 	mkl_get_max_threads() : 128
2025-07-24T04:33:06.0417372Z Intel(R) MKL-DNN v3.7.1 (Git Hash 8d263e693366ef8db40acc569cc7d8edf644556d)
2025-07-24T04:33:06.0417928Z std::thread::hardware_concurrency() : 128
2025-07-24T04:33:06.0418352Z Environment variables:
2025-07-24T04:33:06.0418706Z 	OMP_NUM_THREADS : [not set]
2025-07-24T04:33:06.0419058Z 	MKL_NUM_THREADS : [not set]
2025-07-24T04:33:06.0419370Z ATen parallel backend: OpenMP
2025-07-24T04:33:06.0419578Z 
2025-07-24T04:33:06.5286963Z + [[ default == *numpy_2* ]]
2025-07-24T04:33:06.5287416Z + [[ linux-jammy-rocm-py3.10 == *aarch64* ]]
2025-07-24T04:33:06.5287829Z + [[ default == *backward* ]]
2025-07-24T04:33:06.5288160Z + [[ default == *xla* ]]
2025-07-24T04:33:06.5288469Z + [[ default == *executorch* ]]
2025-07-24T04:33:06.5288779Z + [[ default == \j\i\t\_\l\e\g\a\c\y ]]
2025-07-24T04:33:06.5289156Z + [[ linux-jammy-rocm-py3.10 == *libtorch* ]]
2025-07-24T04:33:06.5289551Z + [[ default == distributed ]]
2025-07-24T04:33:06.5289872Z + [[ default == *operator_benchmark* ]]
2025-07-24T04:33:06.5290234Z + [[ default == *inductor_distributed* ]]
2025-07-24T04:33:06.5291455Z + [[ default == *inductor-halide* ]]
2025-07-24T04:33:06.5292114Z + [[ default == *inductor-triton-cpu* ]]
2025-07-24T04:33:06.5292831Z + [[ default == *inductor-micro-benchmark* ]]
2025-07-24T04:33:06.5293435Z + [[ default == *huggingface* ]]
2025-07-24T04:33:06.5293929Z + [[ default == *timm* ]]
2025-07-24T04:33:06.5294394Z + [[ default == cachebench ]]
2025-07-24T04:33:06.5294885Z + [[ default == verify_cachebench ]]
2025-07-24T04:33:06.5295415Z + [[ default == *torchbench* ]]
2025-07-24T04:33:06.5295933Z + [[ default == *inductor_cpp_wrapper* ]]
2025-07-24T04:33:06.5296492Z + [[ default == *inductor* ]]
2025-07-24T04:33:06.5296990Z + [[ default == *einops* ]]
2025-07-24T04:33:06.5297487Z + [[ default == *dynamo_wrapped* ]]
2025-07-24T04:33:06.5298040Z + [[ linux-jammy-rocm-py3.10 == *rocm* ]]
2025-07-24T04:33:06.5299514Z + [[ -n test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl ]]
2025-07-24T04:33:06.5301118Z + install_torchvision
2025-07-24T04:33:06.5301655Z + local orig_preload
2025-07-24T04:33:06.5302159Z + local commit
2025-07-24T04:33:06.5302676Z ++ get_pinned_commit vision
2025-07-24T04:33:06.5303284Z ++ cat .github/ci_commit_pins/vision.txt
2025-07-24T04:33:06.5334439Z + commit=966da7e46f65d6d49df3e31214470a4fe5cc8e66
2025-07-24T04:33:06.5334870Z + orig_preload=
2025-07-24T04:33:06.5335157Z + '[' -n '' ']'
2025-07-24T04:33:06.5335445Z + [[ linux-jammy-rocm-py3.10 == *cuda* ]]
2025-07-24T04:33:06.5336179Z + pip_build_and_install git+https://github.com/pytorch/vision.git@966da7e46f65d6d49df3e31214470a4fe5cc8e66 dist/vision
2025-07-24T04:33:06.5337120Z + local build_target=git+https://github.com/pytorch/vision.git@966da7e46f65d6d49df3e31214470a4fe5cc8e66
2025-07-24T04:33:06.5337735Z + local wheel_dir=dist/vision
2025-07-24T04:33:06.5338043Z + local found_whl=0
2025-07-24T04:33:06.5338319Z + for file in "${wheel_dir}"/*.whl
2025-07-24T04:33:06.5338662Z + [[ -f dist/vision/*.whl ]]
2025-07-24T04:33:06.5338946Z + '[' 0 == 0 ']'
2025-07-24T04:33:06.5339737Z + python3 -m pip wheel --no-build-isolation --no-deps --no-use-pep517 -w dist/vision git+https://github.com/pytorch/vision.git@966da7e46f65d6d49df3e31214470a4fe5cc8e66
2025-07-24T04:33:06.9156636Z Collecting git+https://github.com/pytorch/vision.git@966da7e46f65d6d49df3e31214470a4fe5cc8e66
2025-07-24T04:33:06.9160122Z   Cloning https://github.com/pytorch/vision.git (to revision 966da7e46f65d6d49df3e31214470a4fe5cc8e66) to /tmp/pip-req-build-y2d4a__i
2025-07-24T04:33:06.9223731Z   Running command git clone --filter=blob:none --quiet https://github.com/pytorch/vision.git /tmp/pip-req-build-y2d4a__i
2025-07-24T04:33:09.4602503Z   Running command git rev-parse -q --verify 'sha^966da7e46f65d6d49df3e31214470a4fe5cc8e66'
2025-07-24T04:33:09.4654484Z   Running command git fetch -q https://github.com/pytorch/vision.git 966da7e46f65d6d49df3e31214470a4fe5cc8e66
2025-07-24T04:33:09.9145522Z   Running command git checkout -q 966da7e46f65d6d49df3e31214470a4fe5cc8e66
2025-07-24T04:33:10.4712153Z   Resolved https://github.com/pytorch/vision.git to commit 966da7e46f65d6d49df3e31214470a4fe5cc8e66
2025-07-24T04:33:13.1362195Z   Preparing metadata (setup.py) ... [?25l- \ | / done
2025-07-24T04:33:13.1419488Z [?25hBuilding wheels for collected packages: torchvision
2025-07-24T04:33:13.1571920Z   DEPRECATION: Building 'torchvision' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'torchvision'. Discussion can be found at https://github.com/pypa/pip/issues/6334
2025-07-24T04:34:06.0001532Z   Building wheel for torchvision (setup.py) ... [?25l- \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | / - \ | done
2025-07-24T04:34:06.0033894Z [?25h  Created wheel for torchvision: filename=torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl size=1570180 sha256=dad34dd4434d50ee6c30cf732e34cad75a4dc03d9931549c5f6d84c81bb6ec87
2025-07-24T04:34:06.0036433Z   Stored in directory: /var/lib/jenkins/.cache/pip/wheels/9c/9d/3e/42fa2d5ac6ba44a90363f8fff0fa9e712e24d4f977637c81cb
2025-07-24T04:34:06.0081729Z Successfully built torchvision
2025-07-24T04:34:06.1263763Z + for file in "${wheel_dir}"/*.whl
2025-07-24T04:34:06.1264794Z + pip_install_whl dist/vision/torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl
2025-07-24T04:34:06.1266037Z + args=('dist/vision/torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl')
2025-07-24T04:34:06.1266835Z + local args
2025-07-24T04:34:06.1267578Z + [[ dist/vision/torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl == *\ * ]]
2025-07-24T04:34:06.1268427Z + for path in "${args[@]}"
2025-07-24T04:34:06.1269253Z + echo 'Installing dist/vision/torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl'
2025-07-24T04:34:06.1270409Z Installing dist/vision/torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl
2025-07-24T04:34:06.1271744Z + python3 -mpip install --no-index --no-deps dist/vision/torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl
2025-07-24T04:34:06.5304496Z Processing ./dist/vision/torchvision-0.22.0a0+966da7e-cp310-cp310-linux_x86_64.whl
2025-07-24T04:34:06.5388406Z Installing collected packages: torchvision
2025-07-24T04:34:06.9058057Z Successfully installed torchvision-0.22.0a0+966da7e
2025-07-24T04:34:06.9526625Z + '[' -n '' ']'
2025-07-24T04:34:06.9527138Z + test_python_shard 2
2025-07-24T04:34:06.9527604Z + [[ -z 2 ]]
2025-07-24T04:34:06.9529950Z + python test/run_test.py --exclude-jit-executor --exclude-distributed-tests --include test_nn test_torch test_cuda test_ops test_unary_ufuncs test_binary_ufuncs test_autograd inductor/test_torchinductor distributed/test_c10d_common distributed/test_c10d_nccl --shard 2 2 --verbose --upload-artifacts-while-running
2025-07-24T04:34:09.6707261Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:34:09.6709856Z   import pkg_resources
2025-07-24T04:34:11.5789363Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/pytorch/test/.pytorch-disabled-tests.json
2025-07-24T04:34:11.7606004Z Ignoring disabled issues:  ['']
2025-07-24T04:34:11.7864822Z Found test times from artifacts
2025-07-24T04:34:11.8725094Z Found test times from artifacts
2025-07-24T04:34:11.8763870Z Running 25% of tests based on TD
2025-07-24T04:34:11.8764814Z Running parallel tests on 8 processes
2025-07-24T04:34:11.8765553Z Name: tests to run (est. time: 9.66min)
2025-07-24T04:34:11.8766251Z   Serial tests (0):
2025-07-24T04:34:11.8766779Z   Parallel tests (2):
2025-07-24T04:34:11.8767360Z     inductor/test_torchinductor 1/2
2025-07-24T04:34:11.8767995Z     test_autograd 1/1
2025-07-24T04:34:11.8768464Z Name: excluded (est. time: 36.87min)
2025-07-24T04:34:11.8768982Z   Serial tests (0):
2025-07-24T04:34:11.8769401Z   Parallel tests (9):
2025-07-24T04:34:11.8769832Z     test_ops 1/10
2025-07-24T04:34:11.8770221Z     test_ops 3/10
2025-07-24T04:34:11.8770613Z     test_ops 4/10
2025-07-24T04:34:11.8771003Z     test_ops 7/10
2025-07-24T04:34:11.8771378Z     test_ops 8/10
2025-07-24T04:34:11.8771782Z     test_unary_ufuncs 1/3
2025-07-24T04:34:11.8772259Z     test_unary_ufuncs 2/3
2025-07-24T04:34:11.8772716Z     test_unary_ufuncs 3/3
2025-07-24T04:34:11.8773162Z     test_binary_ufuncs 1/1
2025-07-24T04:34:11.8773990Z Running inductor/test_torchinductor 1/2 ... [2025-07-24 04:34:11.876206]
2025-07-24T04:34:11.8774927Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set
2025-07-24T04:34:11.8777552Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_torchinductor.py', '-m', 'serial', '--shard-id=1', '--num-shards=2', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-07-24 04:34:11.876504]
2025-07-24T04:34:29.1809413Z 
2025-07-24T04:34:29.1811006Z inductor/test_torchinductor 1/2 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_torchinductor_1.2_c57275fecdd6ef95_.log
2025-07-24T04:34:29.1813312Z Running 1 items in this shard: test/inductor/test_torchinductor.py::GPUTests::test_large_block_sizes_cuda
2025-07-24T04:34:29.1814202Z 
2025-07-24T04:34:29.1814667Z GITHUB_RUN_ID, GITHUB_RUN_ATTEMPT, or ARTIFACTS_FILE_SUFFIX not set, not uploading
2025-07-24T04:34:29.1815565Z Uploading artifacts took 0.00 seconds
2025-07-24T04:34:29.1816818Z Running test_autograd 1/1 ... [2025-07-24 04:34:29.181228]
2025-07-24T04:34:29.1817518Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set
2025-07-24T04:34:29.1825174Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_autograd.py', '-m', 'serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-07-24 04:34:29.181818]
2025-07-24T04:34:34.3103213Z 
2025-07-24T04:34:34.3104975Z test_autograd 1/1 was successful, full logs can be found in artifacts with path test/test-reports/test_autograd_1.1_58149d13a14f9749_.log
2025-07-24T04:34:34.3106234Z Running 0 items in this shard:
2025-07-24T04:34:34.3106611Z 
2025-07-24T04:34:37.4561255Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:34:37.4563070Z   import pkg_resources
2025-07-24T04:34:37.4792691Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:34:37.4794259Z   import pkg_resources
2025-07-24T04:34:37.5500314Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:34:37.5502060Z   import pkg_resources
2025-07-24T04:34:37.5609261Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:34:37.5611007Z   import pkg_resources
2025-07-24T04:34:37.5991717Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:34:37.5993412Z   import pkg_resources
2025-07-24T04:34:37.6041827Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:34:37.6043533Z   import pkg_resources
2025-07-24T04:34:37.6046563Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:34:37.6048034Z   import pkg_resources
2025-07-24T04:34:37.6072128Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:34:37.6073809Z   import pkg_resources
2025-07-24T04:34:38.0107249Z Running inductor/test_torchinductor 1/2 ... [2025-07-24 04:34:38.010056]
2025-07-24T04:34:38.0108264Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set
2025-07-24T04:34:38.0111706Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'inductor/test_torchinductor.py', '-m', 'not serial', '--shard-id=1', '--num-shards=2', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-07-24 04:34:38.010666]
2025-07-24T04:34:38.0830381Z Running test_autograd 1/1 ... [2025-07-24 04:34:38.082295]
2025-07-24T04:34:38.0831234Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set
2025-07-24T04:34:38.0834041Z Executing ['/opt/conda/envs/py_3.10/bin/python', '-bb', 'test_autograd.py', '-m', 'not serial', '--shard-id=1', '--num-shards=1', '-v', '-vv', '-rfEX', '-p', 'no:xdist', '--use-pytest', '-x', '--reruns=2', '--import-slow-tests', '--import-disabled-tests'] ... [2025-07-24 04:34:38.082931]
2025-07-24T04:35:41.8464531Z 
2025-07-24T04:35:41.8478377Z test_autograd 1/1 was successful, full logs can be found in artifacts with path test/test-reports/test_autograd_1.1_c85d56a99a9e7eb0_.log
2025-07-24T04:35:41.8803583Z Running 648 items in this shard: test/test_autograd.py::TestAutograd::test_access_saved_tensor_twice_without_recomputation_works, test/test_autograd.py::TestAutograd::test_accumulate_grad, test/test_autograd.py::TestAutograd::test_accumulate_grad_posthooks_can_observe_tensor_prehook, test/test_autograd.py::TestAutograd::test_accumulate_grad_posthooks_should_not_execute, test/test_autograd.py::TestAutograd::test_accumulate_grad_tensor_reference, test/test_autograd.py::TestAutograd::test_accumulate_grad_with_zero_numel_grad, test/test_autograd.py::TestAutograd::test_anomaly_assign_parent_cleanup, test/test_autograd.py::TestAutograd::test_anomaly_detect_nan, test/test_autograd.py::TestAutograd::test_anomaly_grad_warnings, test/test_autograd.py::TestAutograd::test_anomaly_mode_no_check_nan, test/test_autograd.py::TestAutograd::test_attribute_deletion, test/test_autograd.py::TestAutograd::test_autograd_inplace_view_of_view, test/test_autograd.py::TestAutograd::test_autograd_inplace_views_creation_meta, test/test_autograd.py::TestAutograd::test_autograd_inplace_views_cross_dtype, test/test_autograd.py::TestAutograd::test_autograd_multiple_views_python, test/test_autograd.py::TestAutograd::test_autograd_node_isinstance, test/test_autograd.py::TestAutograd::test_autograd_print_tensor, test/test_autograd.py::TestAutograd::test_autograd_python_custom_function_inplace, test/test_autograd.py::TestAutograd::test_autograd_simple_views_python, test/test_autograd.py::TestAutograd::test_autograd_views_codegen, test/test_autograd.py::TestAutograd::test_backward, test/test_autograd.py::TestAutograd::test_backward_badcalls, test/test_autograd.py::TestAutograd::test_backward_copy, test/test_autograd.py::TestAutograd::test_backward_create_graph_warns, test/test_autograd.py::TestAutograd::test_backward_hook_relative_ordering, test/test_autograd.py::TestAutograd::test_backward_no_grad, test/test_autograd.py::TestAutograd::test_backward_to_node, test/test_autograd.py::TestAutograd::test_backward_twice_retained_graph_with_saved_values, test/test_autograd.py::TestAutograd::test_backward_twice_retained_graph_without_saved_values, test/test_autograd.py::TestAutograd::test_backward_twice_with_saved_values, test/test_autograd.py::TestAutograd::test_backward_twice_without_saved_values, test/test_autograd.py::TestAutograd::test_backward_with_inputs, test/test_autograd.py::TestAutograd::test_backward_with_nonleaf_inputs, test/test_autograd.py::TestAutograd::test_backward_with_scalar_input, test/test_autograd.py::TestAutograd::test_calculate_shape_util, test/test_autograd.py::TestAutograd::test_callback_adds_callback, test/test_autograd.py::TestAutograd::test_callback_propagates_errors_from_device_thread, test/test_autograd.py::TestAutograd::test_cant_create_saved_tensors, test/test_autograd.py::TestAutograd::test_checkpoint_detects_non_determinism, test/test_autograd.py::TestAutograd::test_checkpoint_sequential_warns_if_use_reentrant_not_passed_explcitly, test/test_autograd.py::TestAutograd::test_checkpoint_valid_reset_on_error, test/test_autograd.py::TestAutograd::test_checkpoint_warns_if_use_reentrant_not_passed_explcitly, test/test_autograd.py::TestAutograd::test_checkpointing, test/test_autograd.py::TestAutograd::test_checkpointing_non_reentrant_autocast_cpu, test/test_autograd.py::TestAutograd::test_checkpointing_non_reentrant_autocast_gpu, test/test_autograd.py::TestAutograd::test_checkpointing_without_reentrant_arbitrary_input_output, test/test_autograd.py::TestAutograd::test_checkpointing_without_reentrant_correct_grad, test/test_autograd.py::TestAutograd::test_checkpointing_without_reentrant_custom_function_works, test/test_autograd.py::TestAutograd::test_checkpointing_without_reentrant_dataparallel, test/test_autograd.py::TestAutograd::test_checkpointing_without_reentrant_detached_tensor_use_reentrant_False, test/test_autograd.py::TestAutograd::test_checkpointing_without_reentrant_detached_tensor_use_reentrant_True, test/test_autograd.py::TestAutograd::test_checkpointing_without_reentrant_input_requires_grad_False, test/test_autograd.py::TestAutograd::test_checkpointing_without_reentrant_input_requires_grad_True, test/test_autograd.py::TestAutograd::test_checkpointing_without_reentrant_memory_savings, test/test_autograd.py::TestAutograd::test_checkpointing_without_reentrant_parameter_used_in_an_out, test/test_autograd.py::TestAutograd::test_checkpointing_without_reentrant_saved_object_identity, test/test_autograd.py::TestAutograd::test_checkpointing_without_reentrant_with_context_fn, test/test_autograd.py::TestAutograd::test_copy_slices_graph_task_updates, test/test_autograd.py::TestAutograd::test_create_graph_and_full_backward_hook_cycle, test/test_autograd.py::TestAutograd::test_current_graph_task_execution_order, test/test_autograd.py::TestAutograd::test_current_graph_task_id, test/test_autograd.py::TestAutograd::test_current_node, test/test_autograd.py::TestAutograd::test_custom_autograd_no_early_free, test/test_autograd.py::TestAutograd::test_custom_autograd_repeated_grad_grad, test/test_autograd.py::TestAutograd::test_custom_function_cycle, test/test_autograd.py::TestAutograd::test_custom_function_error, test/test_autograd.py::TestAutograd::test_custom_function_exception, test/test_autograd.py::TestAutograd::test_custom_function_forward_mode_forward_is_no_op, test/test_autograd.py::TestAutograd::test_custom_function_forward_mode_inplace_checks, test/test_autograd.py::TestAutograd::test_custom_function_forward_mode_non_differentiable, test/test_autograd.py::TestAutograd::test_custom_function_forward_mode_non_tensor_before_tensor_args, test/test_autograd.py::TestAutograd::test_custom_function_forward_mode_view_checks, test/test_autograd.py::TestAutograd::test_custom_function_forward_mode_wrong_formula, test/test_autograd.py::TestAutograd::test_custom_function_inplace_on_non_default_view, test/test_autograd.py::TestAutograd::test_custom_function_inplace_on_view_of_leaf, test/test_autograd.py::TestAutograd::test_custom_function_local_inplace, test/test_autograd.py::TestAutograd::test_custom_function_mark_dirty_not_differentiable, test/test_autograd.py::TestAutograd::test_custom_function_mark_output_view_of_intermediate, test/test_autograd.py::TestAutograd::test_custom_function_no_tensors, test/test_autograd.py::TestAutograd::test_custom_function_non_tensor_inputs_outputs, test/test_autograd.py::TestAutograd::test_custom_function_preserve_torch_function_when_return_as_is, test/test_autograd.py::TestAutograd::test_custom_function_return_view_in_nograd, test/test_autograd.py::TestAutograd::test_custom_function_save_for_forward, test/test_autograd.py::TestAutograd::test_custom_function_saved_tensors, test/test_autograd.py::TestAutograd::test_custom_function_setup_context_multi_input, test/test_autograd.py::TestAutograd::test_custom_function_setup_context_multi_output, test/test_autograd.py::TestAutograd::test_custom_function_setup_context_simple, test/test_autograd.py::TestAutograd::test_custom_function_vmap_defaults, test/test_autograd.py::TestAutograd::test_deep_reentrant, test/test_autograd.py::TestAutograd::test_default_saved_tensors_hooks_double_backward, test/test_autograd.py::TestAutograd::test_dep_nograd, test/test_autograd.py::TestAutograd::test_dependent_backward, test/test_autograd.py::TestAutograd::test_detach, test/test_autograd.py::TestAutograd::test_detach_base, test/test_autograd.py::TestAutograd::test_detach_then_inplace_raises_in_autograd, test/test_autograd.py::TestAutograd::test_diagonal_expanded_v, test/test_autograd.py::TestAutograd::test_dir, test/test_autograd.py::TestAutograd::test_disabling_saved_tensor_hooks, test/test_autograd.py::TestAutograd::test_disabling_saved_tensor_hooks_nested, test/test_autograd.py::TestAutograd::test_dont_materialize_grads, test/test_autograd.py::TestAutograd::test_duplicate_backward_root, test/test_autograd.py::TestAutograd::test_enable_grad_decorator_no_paren, test/test_autograd.py::TestAutograd::test_first_grad_fn_access_in_no_grad_mode, test/test_autograd.py::TestAutograd::test_free_deep_graph, test/test_autograd.py::TestAutograd::test_free_deep_graph_complicated, test/test_autograd.py::TestAutograd::test_free_deep_graph_pyfunction, test/test_autograd.py::TestAutograd::test_full_backward_hook_double_backward, test/test_autograd.py::TestAutograd::test_function, test/test_autograd.py::TestAutograd::test_function_returns_input, test/test_autograd.py::TestAutograd::test_function_returns_undefined_tensor, test/test_autograd.py::TestAutograd::test_gc_in_destructor, test/test_autograd.py::TestAutograd::test_grad, test/test_autograd.py::TestAutograd::test_grad_badcalls, test/test_autograd.py::TestAutograd::test_grad_batched_grad, test/test_autograd.py::TestAutograd::test_grad_empty_inputs, test/test_autograd.py::TestAutograd::test_grad_fn_attr_bindings, test/test_autograd.py::TestAutograd::test_grad_fn_badcalls, test/test_autograd.py::TestAutograd::test_grad_fn_input_metadata, test/test_autograd.py::TestAutograd::test_grad_fn_prehooks, test/test_autograd.py::TestAutograd::test_grad_fn_prehooks_multiple_outputs, test/test_autograd.py::TestAutograd::test_grad_fn_prehooks_remove_hooks, test/test_autograd.py::TestAutograd::test_grad_materialize_grads, test/test_autograd.py::TestAutograd::test_grad_mode_class_decoration, test/test_autograd.py::TestAutograd::test_grad_mode_restored_reentrant, test/test_autograd.py::TestAutograd::test_grad_nonleaf, test/test_autograd.py::TestAutograd::test_grad_nonleaf_many_outputs, test/test_autograd.py::TestAutograd::test_grad_nonleaf_register_hook, test/test_autograd.py::TestAutograd::test_grad_to_node, test/test_autograd.py::TestAutograd::test_grad_to_node_inplace, test/test_autograd.py::TestAutograd::test_grad_to_node_materialize, test/test_autograd.py::TestAutograd::test_grad_to_node_multi, test/test_autograd.py::TestAutograd::test_grad_to_node_set, test/test_autograd.py::TestAutograd::test_grad_unreachable, test/test_autograd.py::TestAutograd::test_grad_unreachable_discovery, test/test_autograd.py::TestAutograd::test_gradcheck_backward_mul_by_grad_output, test/test_autograd.py::TestAutograd::test_gradcheck_check_batched_grad, test/test_autograd.py::TestAutograd::test_gradcheck_check_forward_or_backward_only, test/test_autograd.py::TestAutograd::test_gradcheck_check_no_differentiable_outputs, test/test_autograd.py::TestAutograd::test_gradcheck_complex_non_complex_outputs, test/test_autograd.py::TestAutograd::test_gradcheck_custom_error, test/test_autograd.py::TestAutograd::test_gradcheck_default_device_placement_context, test/test_autograd.py::TestAutograd::test_gradcheck_dense_and_sparse_inputs, test/test_autograd.py::TestAutograd::test_gradcheck_forward_ad, test/test_autograd.py::TestAutograd::test_gradcheck_forward_ad_batched_grad, test/test_autograd.py::TestAutograd::test_gradcheck_forward_ad_respects_requires_grad, test/test_autograd.py::TestAutograd::test_gradcheck_forward_ad_runs_with_no_requires_grad, test/test_autograd.py::TestAutograd::test_gradcheck_get_analytical_jacobian, test/test_autograd.py::TestAutograd::test_gradcheck_get_numerical_jacobian, test/test_autograd.py::TestAutograd::test_gradcheck_input_layout0, test/test_autograd.py::TestAutograd::test_gradcheck_input_layout1, test/test_autograd.py::TestAutograd::test_gradcheck_input_layout2, test/test_autograd.py::TestAutograd::test_gradcheck_input_layout3, test/test_autograd.py::TestAutograd::test_gradcheck_input_layout4, test/test_autograd.py::TestAutograd::test_gradcheck_jacobian_mismatch, test/test_autograd.py::TestAutograd::test_gradcheck_multiple_mkldnn_inputs, test/test_autograd.py::TestAutograd::test_gradcheck_nondeterministic, test/test_autograd.py::TestAutograd::test_gradcheck_output_shape_or_dtype_depend_on_values, test/test_autograd.py::TestAutograd::test_gradcheck_single_input, test/test_autograd.py::TestAutograd::test_gradcheck_test_outputs, test/test_autograd.py::TestAutograd::test_gradcheck_undefined_grad, test/test_autograd.py::TestAutograd::test_gradcheck_validates_input_mkldnn, test/test_autograd.py::TestAutograd::test_gradcheck_validates_inputs, test/test_autograd.py::TestAutograd::test_gradient_edge_output, test/test_autograd.py::TestAutograd::test_graph_save_on_cpu, test/test_autograd.py::TestAutograd::test_graph_save_on_cpu_cuda, test/test_autograd.py::TestAutograd::test_hessian_vector, test/test_autograd.py::TestAutograd::test_hook_closure_cycle_use_custom_function_False_use_tensor_hook_False, test/test_autograd.py::TestAutograd::test_hook_closure_cycle_use_custom_function_False_use_tensor_hook_True, test/test_autograd.py::TestAutograd::test_hook_closure_cycle_use_custom_function_True_use_tensor_hook_False, test/test_autograd.py::TestAutograd::test_hook_closure_cycle_use_custom_function_True_use_tensor_hook_True, test/test_autograd.py::TestAutograd::test_hook_edge_case_when_called_with_grad, test/test_autograd.py::TestAutograd::test_hook_none, test/test_autograd.py::TestAutograd::test_hook_with_no_name, test/test_autograd.py::TestAutograd::test_hooks, test/test_autograd.py::TestAutograd::test_hooks_cpp, test/test_autograd.py::TestAutograd::test_increment_version, test/test_autograd.py::TestAutograd::test_index_backward_does_not_save_tensor, test/test_autograd.py::TestAutograd::test_indexing, test/test_autograd.py::TestAutograd::test_indexing_duplicates, test/test_autograd.py::TestAutograd::test_inplace, test/test_autograd.py::TestAutograd::test_inplace_not_requires_grad, test/test_autograd.py::TestAutograd::test_inplace_on_view_backward, test/test_autograd.py::TestAutograd::test_inplace_on_view_leaf_errors, test/test_autograd.py::TestAutograd::test_inplace_on_view_saved_output, test/test_autograd.py::TestAutograd::test_inplace_on_view_weak_grad_fn, test/test_autograd.py::TestAutograd::test_input_buffer_accum, test/test_autograd.py::TestAutograd::test_integer_outputs, test/test_autograd.py::TestAutograd::test_invalid_gradients, test/test_autograd.py::TestAutograd::test_isolated_node, test/test_autograd.py::TestAutograd::test_leaf_assignment, test/test_autograd.py::TestAutograd::test_legacy_function_deprecation_exception, test/test_autograd.py::TestAutograd::test_lobpcg, test/test_autograd.py::TestAutograd::test_mark_non_differentiable, test/test_autograd.py::TestAutograd::test_mark_non_differentiable_mixed, test/test_autograd.py::TestAutograd::test_mark_non_differentiable_none, test/test_autograd.py::TestAutograd::test_materialize_grads, test/test_autograd.py::TestAutograd::test_multi_backward, test/test_autograd.py::TestAutograd::test_multi_backward_no_grad, test/test_autograd.py::TestAutograd::test_multi_grad_all_hooks, test/test_autograd.py::TestAutograd::test_multi_grad_any_hooks, test/test_autograd.py::TestAutograd::test_multi_grad_hooks_invalid_mode, test/test_autograd.py::TestAutograd::test_multiple_insert_removal_caching, test/test_autograd.py::TestAutograd::test_named_tensor_for_complex_views, test/test_autograd.py::TestAutograd::test_naughty_anomaly_access, test/test_autograd.py::TestAutograd::test_naughty_autograd_function_attribute_access, test/test_autograd.py::TestAutograd::test_naughty_autograd_function_stashing_ctx, test/test_autograd.py::TestAutograd::test_nested_anomaly_detect_nan, test/test_autograd.py::TestAutograd::test_nested_anomaly_printstack_cleanup, test/test_autograd.py::TestAutograd::test_next_functions, test/test_autograd.py::TestAutograd::test_no_grad, test/test_autograd.py::TestAutograd::test_no_grad_assignment, test/test_autograd.py::TestAutograd::test_no_grad_copy, test/test_autograd.py::TestAutograd::test_no_grad_copy_sparse, test/test_autograd.py::TestAutograd::test_no_grad_input, test/test_autograd.py::TestAutograd::test_no_grad_modifies_version, test/test_autograd.py::TestAutograd::test_no_grad_python_function, test/test_autograd.py::TestAutograd::test_no_requires_grad_inplace, test/test_autograd.py::TestAutograd::test_no_unnecessary_save, test/test_autograd.py::TestAutograd::test_no_unnecessary_unwrapping, test/test_autograd.py::TestAutograd::test_node_ordering_when_none_returned, test/test_autograd.py::TestAutograd::test_node_post_hook_registered_during_unpack_hook, test/test_autograd.py::TestAutograd::test_not_implemented_fwad, test/test_autograd.py::TestAutograd::test_not_implemented_grad, test/test_autograd.py::TestAutograd::test_numpy_requires_grad, test/test_autograd.py::TestAutograd::test_once_differentiable, test/test_autograd.py::TestAutograd::test_out_variant_raises_when_inputs_require_grad, test/test_autograd.py::TestAutograd::test_pack_hook_with_inplace_modification_should_fail, test/test_autograd.py::TestAutograd::test_pickle, test/test_autograd.py::TestAutograd::test_post_accumulate_grad_hook_e2e, test/test_autograd.py::TestAutograd::test_post_accumulate_grad_hook_gets_cleaned_up, test/test_autograd.py::TestAutograd::test_post_accumulate_grad_hook_multiple_hooks, test/test_autograd.py::TestAutograd::test_post_accumulate_grad_hook_multiple_tensors, test/test_autograd.py::TestAutograd::test_post_accumulate_grad_hook_on_non_leaf, test/test_autograd.py::TestAutograd::test_post_accumulate_grad_hook_ordering, test/test_autograd.py::TestAutograd::test_post_accumulate_grad_hook_returns_not_None, test/test_autograd.py::TestAutograd::test_pow_zero_tensor_gradient, test/test_autograd.py::TestAutograd::test_power_function, test/test_autograd.py::TestAutograd::test_prehook_ordering, test/test_autograd.py::TestAutograd::test_profiler, test/test_autograd.py::TestAutograd::test_profiler_aggregation_fake, test/test_autograd.py::TestAutograd::test_profiler_aggregation_lstm, test/test_autograd.py::TestAutograd::test_profiler_aggregation_table, test/test_autograd.py::TestAutograd::test_profiler_function_event_avg, test/test_autograd.py::TestAutograd::test_profiler_propagation, test/test_autograd.py::TestAutograd::test_profiler_seq_nr, test/test_autograd.py::TestAutograd::test_profiler_shapes, test/test_autograd.py::TestAutograd::test_profiler_unboxed_only, test/test_autograd.py::TestAutograd::test_pynode_destruction_deadlock, test/test_autograd.py::TestAutograd::test_record_function, test/test_autograd.py::TestAutograd::test_record_function_callbacks, test/test_autograd.py::TestAutograd::test_record_function_legacy, test/test_autograd.py::TestAutograd::test_record_function_multithreaded, test/test_autograd.py::TestAutograd::test_reentrant_child_error, test/test_autograd.py::TestAutograd::test_reentrant_priority, test/test_autograd.py::TestAutograd::test_reentrant_with_callbacks_both_depths, test/test_autograd.py::TestAutograd::test_reentrant_with_callbacks_depth_0, test/test_autograd.py::TestAutograd::test_reentrant_with_callbacks_depth_1, test/test_autograd.py::TestAutograd::test_reentrant_with_leaf_variable_hook, test/test_autograd.py::TestAutograd::test_reentrant_with_non_leaf_variable_hook, test/test_autograd.py::TestAutograd::test_requires_grad, test/test_autograd.py::TestAutograd::test_requires_grad_, test/test_autograd.py::TestAutograd::test_requires_grad_inplace, test/test_autograd.py::TestAutograd::test_retain_grad, test/test_autograd.py::TestAutograd::test_retain_grad_cycle, test/test_autograd.py::TestAutograd::test_retain_grad_inplace, test/test_autograd.py::TestAutograd::test_retain_grad_inplace_over_view, test/test_autograd.py::TestAutograd::test_retains_grad_can_always_observe_tensor_prehook, test/test_autograd.py::TestAutograd::test_retains_grad_inplace_multiple_outputs, test/test_autograd.py::TestAutograd::test_return_duplicate, test/test_autograd.py::TestAutograd::test_return_duplicate_inplace, test/test_autograd.py::TestAutograd::test_return_leaf, test/test_autograd.py::TestAutograd::test_return_leaf_inplace, test/test_autograd.py::TestAutograd::test_save_none_for_backward, test/test_autograd.py::TestAutograd::test_save_on_cpu_and_checkpoint, test/test_autograd.py::TestAutograd::test_save_output_nr, test/test_autograd.py::TestAutograd::test_saved_tensor_hooks_custom_error_propagation, test/test_autograd.py::TestAutograd::test_saved_tensor_hooks_custom_function_intermediates, test/test_autograd.py::TestAutograd::test_saved_tensor_hooks_extra_enter_during_bw_no_leak, test/test_autograd.py::TestAutograd::test_saved_tensor_hooks_extra_exit_during_bw_no_crash, test/test_autograd.py::TestAutograd::test_saved_tensors_hook_version_counter_not_shared, test/test_autograd.py::TestAutograd::test_saved_variable_packing_unpacking_did_not_save_original_with_default_hooks, test/test_autograd.py::TestAutograd::test_saved_variable_packing_unpacking_did_not_save_original_with_hooks, test/test_autograd.py::TestAutograd::test_saved_variable_packing_unpacking_saved_original_with_default_hooks, test/test_autograd.py::TestAutograd::test_saved_variable_packing_unpacking_saved_original_with_hooks, test/test_autograd.py::TestAutograd::test_saved_variable_saved_original_inplace_detach, test/test_autograd.py::TestAutograd::test_saved_variable_version_counter, test/test_autograd.py::TestAutograd::test_saved_variables_deprecated, test/test_autograd.py::TestAutograd::test_saving_variable_to_disk, test/test_autograd.py::TestAutograd::test_scalar_grad_mixed_device, test/test_autograd.py::TestAutograd::test_select_expanded_v, test/test_autograd.py::TestAutograd::test_select_sum, test/test_autograd.py::TestAutograd::test_set_data_preserve_pyobj, test/test_autograd.py::TestAutograd::test_set_data_self_requires_grad, test/test_autograd.py::TestAutograd::test_set_data_tensorimpl_type, test/test_autograd.py::TestAutograd::test_set_grad_coroutines, test/test_autograd.py::TestAutograd::test_set_grad_coroutines_benign_exceptions, test/test_autograd.py::TestAutograd::test_set_grad_coroutines_critical_exceptions, test/test_autograd.py::TestAutograd::test_set_grad_coroutines_exit, test/test_autograd.py::TestAutograd::test_set_grad_enabled, test/test_autograd.py::TestAutograd::test_set_grad_enabled_wraps, test/test_autograd.py::TestAutograd::test_set_grad_generator_functions, test/test_autograd.py::TestAutograd::test_set_grad_generator_functions_recursive, test/test_autograd.py::TestAutograd::test_set_materialize_non_diff_grads, test/test_autograd.py::TestAutograd::test_setitem, test/test_autograd.py::TestAutograd::test_setitem_mask, test/test_autograd.py::TestAutograd::test_setting_default_saved_variable_hooks_twice_should_not_fail, test/test_autograd.py::TestAutograd::test_setting_default_saved_variable_hooks_twice_should_use_inner, test/test_autograd.py::TestAutograd::test_setup_context_when_forward_has_default_args, test/test_autograd.py::TestAutograd::test_shape, test/test_autograd.py::TestAutograd::test_sharded_grad, test/test_autograd.py::TestAutograd::test_simple_reentrant, test/test_autograd.py::TestAutograd::test_slice_expanded_v, test/test_autograd.py::TestAutograd::test_sparse_gather_both_scalar, test/test_autograd.py::TestAutograd::test_sparse_gather_dim0, test/test_autograd.py::TestAutograd::test_sparse_gather_dim1, test/test_autograd.py::TestAutograd::test_sparse_gather_dim_neg, test/test_autograd.py::TestAutograd::test_sparse_gather_ind_scalar, test/test_autograd.py::TestAutograd::test_sparse_gather_x_scalar, test/test_autograd.py::TestAutograd::test_sparse_mm_backward, test/test_autograd.py::TestAutograd::test_tensor_grad_warnings, test/test_autograd.py::TestAutograd::test_tensor_hooks_inplace, test/test_autograd.py::TestAutograd::test_tensor_hooks_inplace_multiple_outputs, test/test_autograd.py::TestAutograd::test_tensor_hooks_inplace_over_view, test/test_autograd.py::TestAutograd::test_thread_shutdown, test/test_autograd.py::TestAutograd::test_to_sparse_backward, test/test_autograd.py::TestAutograd::test_too_many_grads, test/test_autograd.py::TestAutograd::test_type_conversions, test/test_autograd.py::TestAutograd::test_unpack_hooks_exec_count, test/test_autograd.py::TestAutograd::test_unrelated_inputs, test/test_autograd.py::TestAutograd::test_unsafe_set_version_counter, test/test_autograd.py::TestAutograd::test_unused_output, test/test_autograd.py::TestAutograd::test_var_mean_differentiable, test/test_autograd.py::TestAutograd::test_variable_traverse, test/test_autograd.py::TestAutograd::test_version_counter, test/test_autograd.py::TestAutograd::test_view_func_replay, test/test_autograd.py::TestAutograd::test_view_func_replay_with_modified_state, test/test_autograd.py::TestAutograd::test_view_replay_enabled, test/test_autograd.py::TestAutograd::test_volatile_deprecated, test/test_autograd.py::TestAutograd::test_will_engine_execute_node, test/test_autograd.py::TestAutograd::test_wrapped_number_saved_tensors_hooks, test/test_autograd.py::TestAutogradForwardModeBatchedGrad::test_inplace_on_view_not_same_layout, test/test_autograd.py::TestAutogradForwardModeBatchedGrad::test_inplace_on_view_same_layout, test/test_autograd.py::TestAutogradForwardModeBatchedGrad::test_metadata_check_for_storage_numel_skipped, test/test_autograd.py::TestAutogradForwardModeBatchedGrad::test_out_of_place_basic, test/test_autograd.py::TestAutogradForwardModeBatchedGrad::test_out_of_place_not_same_layout, test/test_autograd.py::TestAutogradForwardMode::test_advanced_packing_unpacking, test/test_autograd.py::TestAutogradForwardMode::test_backward_graph_destruction, test/test_autograd.py::TestAutogradForwardMode::test_basic_packing_unpacking, test/test_autograd.py::TestAutogradForwardMode::test_codegen_ignores_undefined_outputs, test/test_autograd.py::TestAutogradForwardMode::test_create_new_zeros_with_same_meta, test/test_autograd.py::TestAutogradForwardMode::test_default_level, test/test_autograd.py::TestAutogradForwardMode::test_detach_view_tracking, test/test_autograd.py::TestAutogradForwardMode::test_forward_level_cleanup, test/test_autograd.py::TestAutogradForwardMode::test_fwd_grad_enabled, test/test_autograd.py::TestAutogradForwardMode::test_grad_cleanup, test/test_autograd.py::TestAutogradForwardMode::test_make_dual_forbid_integral_dtype, test/test_autograd.py::TestAutogradForwardMode::test_make_dual_inference_tensor_in_inference_mode, test/test_autograd.py::TestAutogradForwardMode::test_make_dual_torch_dispatch, test/test_autograd.py::TestAutogradForwardMode::test_metadata_check_check_conj, test/test_autograd.py::TestAutogradForwardMode::test_metadata_check_checks_ignores_size_zero, test/test_autograd.py::TestAutogradForwardMode::test_metadata_check_checks_storage_numel, test/test_autograd.py::TestAutogradForwardMode::test_metadata_check_ignore_storage_offset_for_zero_numel_tensor, test/test_autograd.py::TestAutogradForwardMode::test_metadata_check_when_primal_has_conj_bit, test/test_autograd.py::TestAutogradForwardMode::test_metadata_check_when_primal_has_neg_bit, test/test_autograd.py::TestAutogradForwardMode::test_nested_level, test/test_autograd.py::TestAutogradForwardMode::test_non_differentiable, test/test_autograd.py::TestAutogradForwardMode::test_out_variant, test/test_autograd.py::TestAutogradForwardMode::test_print, test/test_autograd.py::TestAutogradForwardMode::test_set_fw_grad_having_own_fw_grad_at_same_level, test/test_autograd.py::TestAutogradForwardMode::test_set_fwd_grad_enabled, test/test_autograd.py::TestAutogradForwardMode::test_size_check, test/test_autograd.py::TestAutogradForwardMode::test_view_inplace_always_creates_a_view, test/test_autograd.py::TestAutogradForwardMode::test_view_inplace_differentiable_views, test/test_autograd.py::TestAutogradForwardMode::test_view_inplace_non_differentiable_views, test/test_autograd.py::TestAllowMutationOnSaved::test_backward_out_of_context, test/test_autograd.py::TestAllowMutationOnSaved::test_basic, test/test_autograd.py::TestAllowMutationOnSaved::test_disallow_nesting, test/test_autograd.py::TestAllowMutationOnSaved::test_double_backward, test/test_autograd.py::TestAllowMutationOnSaved::test_inplace_foreach, test/test_autograd.py::TestAllowMutationOnSaved::test_save_base_and_modify_view, test/test_autograd.py::TestAllowMutationOnSaved::test_save_view_modify_base, test/test_autograd.py::TestAllowMutationOnSaved::test_saved_but_not_anymore, test/test_autograd.py::TestAllowMutationOnSaved::test_saved_same_tensor_different_versions, test/test_autograd.py::TestAllowMutationOnSaved::test_saved_same_tensor_many_times, test/test_autograd.py::TestAllowMutationOnSaved::test_views, test/test_autograd.py::TestAllowMutationOnSaved::test_with_math_views, test/test_autograd.py::TestAllowMutationOnSaved::test_with_out_variant, test/test_autograd.py::TestAutogradInferenceMode::test_inference_mode_context_manager, test/test_autograd.py::TestAutogradInferenceMode::test_inference_mode_decorator, test/test_autograd.py::TestAutogradInferenceMode::test_inference_mode_existing_autograd_session, test/test_autograd.py::TestAutogradInferenceMode::test_inference_mode_handle_direct_view_on_rebase, test/test_autograd.py::TestAutogradInferenceMode::test_inference_mode_handle_indirect_view_on_rebase, test/test_autograd.py::TestAutogradInferenceMode::test_inference_mode_inf_tensor_in_inf_mode_functional_op, test/test_autograd.py::TestAutogradInferenceMode::test_inference_mode_inf_tensor_in_inf_mode_inplace_op, test/test_autograd.py::TestAutogradInferenceMode::test_inference_mode_inf_tensor_in_inf_mode_view_op, test/test_autograd.py::TestAutogradInferenceMode::test_inference_mode_inf_tensor_in_normal_mode_functional_op, test/test_autograd.py::TestAutogradInferenceMode::test_inference_mode_inf_tensor_in_normal_mode_inplace_op, test/test_autograd.py::TestAutogradInferenceMode::test_inference_mode_inf_tensor_in_normal_mode_view_op, test/test_autograd.py::TestAutogradInferenceMode::test_inference_mode_tensor_creation, test/test_autograd.py::TestAutogradInferenceMode::test_mix_inference_and_normal_tensor_functional_op, test/test_autograd.py::TestAutogradInferenceMode::test_mix_inference_and_normal_tensor_inplace_op, test/test_autograd.py::TestAutogradInferenceMode::test_mix_inference_and_normal_tensor_view_op, test/test_autograd.py::TestAutogradInferenceMode::test_normal_tensor_inplace_output_in_inference_mode, test/test_autograd.py::TestAutogradInferenceMode::test_normal_tensor_inplace_output_in_normal_mode, test/test_autograd.py::TestAutogradInferenceMode::test_normal_tensor_view_output_in_inference_mode, test/test_autograd.py::TestAutogradInferenceMode::test_normal_tensor_view_output_in_normal_mode, test/test_autograd.py::TestAutogradStreamSynchronization::test_consumer_to_multi_producer_case_4_correctness, test/test_autograd.py::TestAutogradStreamSynchronization::test_consumer_to_single_producer_case_2_correctness, test/test_autograd.py::TestAutogradStreamSynchronization::test_consumer_to_single_producer_case_3_correctness, test/test_autograd.py::TestAutogradStreamSynchronization::test_consumer_to_single_producer_case_3_correctness_non_default_ambient_stream, test/test_autograd.py::TestAutogradStreamSynchronization::test_consumer_to_single_producer_case_4_correctness, test/test_autograd.py::TestAutogradStreamSynchronization::test_side_stream_backward_overlap, test/test_autograd.py::TestMultithreadAutograd::test_cat_stack_r_to_c, test/test_autograd.py::TestMultithreadAutograd::test_custom_function_propagates_errors_from_device_thread, test/test_autograd.py::TestMultithreadAutograd::test_dataparallel_saved_tensors_hooks, test/test_autograd.py::TestMultithreadAutograd::test_fork_join_in_middle, test/test_autograd.py::TestMultithreadAutograd::test_multi_grad_all_hooks, test/test_autograd.py::TestMultithreadAutograd::test_multi_grad_any_hooks, test/test_autograd.py::TestMultithreadAutograd::test_multithreaded_exception_propagation, test/test_autograd.py::TestMultithreadAutograd::test_preserve_backtrace, test/test_autograd.py::TestMultithreadAutograd::test_python_thread_in_middle, test/test_autograd.py::TestMultithreadAutograd::test_set_multithreading_enabled_as_context_manager_and_function, test/test_autograd.py::TestMultithreadAutograd::test_simple_backward, test/test_autograd.py::TestMultithreadAutograd::test_simple_backward_same_input, test/test_autograd.py::TestNestedCheckpoint::test_nested_checkpoint_early_stop_False, test/test_autograd.py::TestNestedCheckpoint::test_nested_checkpoint_early_stop_True, test/test_autograd.py::TestNestedCheckpoint::test_nested_checkpoint_kwargs_early_stop_False, test/test_autograd.py::TestNestedCheckpoint::test_nested_checkpoint_kwargs_early_stop_True, test/test_autograd.py::TestNestedCheckpoint::test_nested_checkpoint_non_tensor_inputs_and_outputs_early_stop_False, test/test_autograd.py::TestNestedCheckpoint::test_nested_checkpoint_non_tensor_inputs_and_outputs_early_stop_True, test/test_autograd.py::TestNestedCheckpoint::test_nested_checkpoint_reentrant_backwards_early_stop_False, test/test_autograd.py::TestNestedCheckpoint::test_nested_checkpoint_reentrant_backwards_early_stop_True, test/test_autograd.py::TestNestedCheckpoint::test_nested_checkpoint_same_graph_early_stop_False, test/test_autograd.py::TestNestedCheckpoint::test_nested_checkpoint_same_graph_early_stop_True, test/test_autograd.py::TestNestedCheckpoint::test_nested_checkpoint_set_early_stop, test/test_autograd.py::TestNestedCheckpoint::test_nested_checkpoint_set_early_stop_no_recompution_needed, test/test_autograd.py::TestNestedCheckpoint::test_nested_checkpoint_two_children_early_stop_False, test/test_autograd.py::TestNestedCheckpoint::test_nested_checkpoint_two_children_early_stop_True, test/test_autograd.py::TestSelectiveActivationCheckpoint::test_bad_inputs, test/test_autograd.py::TestSelectiveActivationCheckpoint::test_can_only_trigger_recompute_once, test/test_autograd.py::TestSelectiveActivationCheckpoint::test_flops_and_mem, test/test_autograd.py::TestSelectiveActivationCheckpoint::test_function_with_more_than_one_output, test/test_autograd.py::TestSelectiveActivationCheckpoint::test_function_with_non_tensor_output, test/test_autograd.py::TestSelectiveActivationCheckpoint::test_output_already_has_autograd_meta, test/test_autograd.py::TestSelectiveActivationCheckpoint::test_policy_with_state, test/test_autograd.py::TestSelectiveActivationCheckpoint::test_storage_lifetime, test/test_autograd.py::TestSelectiveActivationCheckpoint::test_subclass_dispatching_sizes, test/test_autograd.py::TestSelectiveActivationCheckpoint::test_version_counter, test/test_autograd.py::TestAutogradComplex::test_view_func_for_complex_views, test/test_autograd.py::TestAutogradComplex::test_view_with_multi_output, test/test_autograd.py::TestAutogradFunctional::test_construct_standard_basis_for_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_construct_standard_basis_for_cuda_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_construct_standard_basis_for_cuda_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_construct_standard_basis_for_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_create_graph_vectorize_False_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_create_graph_vectorize_False_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_create_graph_vectorize_True_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_create_graph_vectorize_True_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_err_check_strict_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_err_check_strict_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_err_check_strict_vectorize_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_err_check_strict_vectorize_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_err_check_vectorize_False_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_err_check_vectorize_False_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_err_check_vectorize_True_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_err_check_vectorize_True_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_match_vhp_hvp_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_match_vhp_hvp_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_no_grad_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_no_grad_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_output_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_output_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_output_vectorized_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_output_vectorized_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_scalar_vectorize_False_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_scalar_vectorize_False_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_scalar_vectorize_True_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_scalar_vectorize_True_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_vectorize_correctness_multi_input_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_vectorize_correctness_multi_input_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_vectorize_correctness_simple_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_vectorize_correctness_simple_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_vectorize_correctness_unrelated_outputs_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_vectorize_correctness_unrelated_outputs_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_vectorize_raises_no_warnings_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hessian_vectorize_raises_no_warnings_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hvp_create_graph_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hvp_create_graph_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hvp_err_check_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hvp_err_check_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hvp_err_check_strict_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hvp_err_check_strict_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hvp_no_grad_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hvp_no_grad_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hvp_output_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hvp_output_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_hvp_scalar_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_hvp_scalar_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_create_graph_vectorize_False_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_create_graph_vectorize_False_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_create_graph_vectorize_True_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_create_graph_vectorize_True_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_err_check_strict_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_err_check_strict_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_err_check_strict_vectorize_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_err_check_strict_vectorize_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_err_check_vectorize_False_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_err_check_vectorize_False_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_err_check_vectorize_True_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_err_check_vectorize_True_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_match_vjp_jvp_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_match_vjp_jvp_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_no_grad_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_no_grad_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_output_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_output_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_output_vectorized_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_output_vectorized_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_scalar_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_scalar_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_scalar_vectorized_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_scalar_vectorized_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_correctness_different_devices_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_correctness_different_devices_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_correctness_different_dtype_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_correctness_different_dtype_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_correctness_multi_input_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_correctness_multi_input_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_correctness_multi_input_multi_output_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_correctness_multi_input_multi_output_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_correctness_simple_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_correctness_simple_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_correctness_unrelated_outputs_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_correctness_unrelated_outputs_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_correctness_zero_dim_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_correctness_zero_dim_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_raises_no_warnings_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jacobian_vectorize_raises_no_warnings_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jvp_create_graph_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jvp_create_graph_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jvp_err_check_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jvp_err_check_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jvp_err_check_strict_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jvp_err_check_strict_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jvp_no_grad_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jvp_no_grad_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jvp_output_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jvp_output_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_jvp_scalar_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_jvp_scalar_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_vhp_create_graph_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_vhp_create_graph_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_vhp_err_check_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_vhp_err_check_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_vhp_err_check_strict_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_vhp_err_check_strict_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_vhp_no_grad_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_vhp_no_grad_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_vhp_output_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_vhp_output_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_vhp_scalar_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_vhp_scalar_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_vjp_create_graph_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_vjp_create_graph_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_vjp_err_check_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_vjp_err_check_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_vjp_err_check_strict_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_vjp_err_check_strict_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_vjp_no_grad_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_vjp_no_grad_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_vjp_output_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_vjp_output_logging_tensor, test/test_autograd.py::TestAutogradFunctional::test_vjp_scalar_base_tensor, test/test_autograd.py::TestAutogradFunctional::test_vjp_scalar_logging_tensor, test/test_autograd.py::TestAutogradLogging::test_logging, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_advanced_indexing_backwards_large_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_advanced_indexing_backwards_memory_format_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_backward_device_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_complex_scalar_backward_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_copy__cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_copy_forward_ad_broadcasting_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_copy_forward_ad_same_layout_copies_grad_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_copy_r_to_c_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_cross_device_reentrant_autograd_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_free_unneeded_tensor_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_grad_assignment_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_gradcheck_input_output_different_device_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_multiple_output_view_of_view_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_on_view_backprop_base_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_on_view_backprop_view_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_on_view_backprop_view_of_view_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_on_view_gradcheck_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_on_view_makes_base_require_grad_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_on_view_modify_base_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_on_view_multi_output_safe_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_on_view_multi_output_unsafe_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_on_view_multiple_outputs_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_on_view_non_contig_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_on_view_of_multiple_output_view_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_on_view_of_view_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_on_view_python_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_on_view_then_no_grad_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inplace_on_view_undefined_grad_output_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_inputbuffer_add_multidevice_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_min_max_median_backprops_to_all_values_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_mv_grad_stride_0_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_non_differentiable_ops_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_parameter_resize_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_pin_memory_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_pow_real_negative_base_complex_exponent_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_profiler_emit_itt_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_profiler_emit_nvtx_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_pyscalar_conversions_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_reentrant_parent_error_on_cpu_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_requires_grad_factory_cuda_float32, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_requires_grad_factory_cuda_float64, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_resize_version_bump_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_rnn_backward_to_input_but_not_parameters_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_scatter_index_reduce_amin_amax_backprops_to_all_values_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_scatter_index_reduce_prod_gradgrad_error_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_set_requires_grad_only_for_floats_cuda_float16, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_set_requires_grad_only_for_floats_cuda_float32, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_set_requires_grad_only_for_floats_cuda_float64, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_set_requires_grad_only_for_floats_cuda_int16, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_set_requires_grad_only_for_floats_cuda_int32, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_set_requires_grad_only_for_floats_cuda_int64, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_set_requires_grad_only_for_floats_cuda_int8, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_simple_reentrant_cross_device_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_sparse_backward_cuda_complex128, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_sparse_backward_cuda_float64, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_sparse_ctor_getter_backward_cuda_complex128, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_sparse_ctor_getter_backward_cuda_float64, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_sparse_mask_autograd_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_strided_leaf_grad_layout_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_to_r_to_c_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_unused_output_device_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_warning_in_backward_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_where_functional_cuda, test/test_autograd.py::TestAutogradDeviceTypeCUDA::test_where_scalar_cuda, test/test_autograd.py::TestAutogradMultipleDispatchCUDA::test_autograd_composite_implicit_and_dispatch_registration_cuda, test/test_autograd.py::TestAutogradMultipleDispatchCUDA::test_autograd_multiple_dispatch_registrations_cuda, test/test_autograd.py::TestAutogradMultipleDispatchCUDA::test_backward_single_threaded_cuda, test/test_autograd.py::TestAutogradMultipleDispatchCUDA::test_backward_tls_stash_cuda, test/test_autograd.py::TestAutogradMultipleDispatchCUDA::test_foward_mode_AD_cuda, test/test_autograd.py::TestAutogradMultipleDispatchCUDA::test_is_retain_graph_cuda, test/test_autograd.py::TestAutogradMultipleDispatchCUDA::test_per_dispatch_key_input_saving_cuda, test/test_autograd.py::TestAutogradMultipleDispatchCUDA::test_set_sequence_nr_cuda, test/test_autograd.py::TestAutogradMultipleDispatchCUDA::test_view_copy_cuda
2025-07-24T04:35:41.9059904Z 
2025-07-24T04:43:35.9569593Z 
2025-07-24T04:43:35.9576648Z inductor/test_torchinductor 1/2 was successful, full logs can be found in artifacts with path test/test-reports/inductor.test_torchinductor_1.2_7058bcc364434905_.log
2025-07-24T04:43:35.9806656Z Running 443 items in this shard: test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast1_broadcast2, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast1_strided, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast1_transposed, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast2_broadcast2, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast2_double, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast2_int, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast2_transposed, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast3_dense, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_broadcast3_int, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_dense_broadcast1, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_dense_broadcast3, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_dense_dense, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_dense_double, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_dense_strided, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_double_broadcast2, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_double_double, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_double_int, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_double_strided, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_double_transposed, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_int_broadcast3, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_int_strided, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_int_transposed, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_strided_broadcast1, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_strided_broadcast3, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_strided_strided, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_strided_transposed, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_transposed_broadcast2, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_transposed_double, test/inductor/test_torchinductor.py::SweepInputsGPUTest::test_cuda_transposed_int, test/inductor/test_torchinductor.py::GPUTests::test_AllenaiLongformerBase_repro_cuda, test/inductor/test_torchinductor.py::GPUTests::test__dyn_quant_pack_4bit_weight_cuda, test/inductor/test_torchinductor.py::GPUTests::test__unsafe_masked_index_put_accumulate_cuda, test/inductor/test_torchinductor.py::GPUTests::test_abs_cuda, test/inductor/test_torchinductor.py::GPUTests::test_adaptive_avg_pool1d_argmax_cuda, test/inductor/test_torchinductor.py::GPUTests::test_adaptive_avg_pool2d1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_adaptive_avg_pool_with_output_size_0_cuda, test/inductor/test_torchinductor.py::GPUTests::test_adaptive_max_pool2d1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_adaptive_max_pool2d3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_adaptive_pool_errors_with_long_cuda, test/inductor/test_torchinductor.py::GPUTests::test_add_complex4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_add_complex5_cuda, test/inductor/test_torchinductor.py::GPUTests::test_add_complex6_cuda, test/inductor/test_torchinductor.py::GPUTests::test_add_const_float_cuda, test/inductor/test_torchinductor.py::GPUTests::test_add_inplace_permuted_cuda, test/inductor/test_torchinductor.py::GPUTests::test_addmm_cuda, test/inductor/test_torchinductor.py::GPUTests::test_alexnet_prefix_cuda, test/inductor/test_torchinductor.py::GPUTests::test_angle_cuda, test/inductor/test_torchinductor.py::GPUTests::test_any_cuda, test/inductor/test_torchinductor.py::GPUTests::test_aoti_eager_with_persistent_cache_cuda, test/inductor/test_torchinductor.py::GPUTests::test_aoti_eager_with_scalar_cuda, test/inductor/test_torchinductor.py::GPUTests::test_arange1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_arange2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_arange6_cuda, test/inductor/test_torchinductor.py::GPUTests::test_argmax_argmin1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_argmax_argmin2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_as_strided_cuda, test/inductor/test_torchinductor.py::GPUTests::test_assert_alignment_op_name_fail_cuda, test/inductor/test_torchinductor.py::GPUTests::test_assert_size_stride_op_name_fail_cuda, test/inductor/test_torchinductor.py::GPUTests::test_assert_size_stride_op_name_pass_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool2d4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool2d6_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool2d7_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool2d8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_avg_pool3d_backward4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_batch_norm_2d_2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_batch_norm_2d_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bitwise2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bmm2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_both_scalars_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_add_autotune_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_computed_offsets_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_default_kwargs_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int32_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int32_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int32_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int64_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int64_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int64_int64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int64_int8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int64_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int8_int8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_int8_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_uint8_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_uint8_int64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_bucketize_int_uint8_int8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_builtins_round_float_ndigits_neg_cuda, test/inductor/test_torchinductor.py::GPUTests::test_builtins_round_int_ndigits_pos_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cat_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cat_empty_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cat_empty_index_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cat_single_empty_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cat_unbacked_empty_1d_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cauchy_cuda, test/inductor/test_torchinductor.py::GPUTests::test_chunk_recompiles_cuda, test/inductor/test_torchinductor.py::GPUTests::test_clamp_type_promotion_cuda, test/inductor/test_torchinductor.py::GPUTests::test_clamp_type_promotion_non_tensor_cuda, test/inductor/test_torchinductor.py::GPUTests::test_clone_cuda, test/inductor/test_torchinductor.py::GPUTests::test_compar_cuda, test/inductor/test_torchinductor.py::GPUTests::test_complex_fallback_cuda, test/inductor/test_torchinductor.py::GPUTests::test_complex_memory_overlap_cuda, test/inductor/test_torchinductor.py::GPUTests::test_computed_buffer_inlining_cuda, test/inductor/test_torchinductor.py::GPUTests::test_concat_add_inplace_cuda, test/inductor/test_torchinductor.py::GPUTests::test_config_option_dont_assume_alignment_cudagraphs_cuda, test/inductor/test_torchinductor.py::GPUTests::test_config_option_dont_assume_alignment_recompiles_cuda, test/inductor/test_torchinductor.py::GPUTests::test_consecutive_split_cumprod_cuda, test/inductor/test_torchinductor.py::GPUTests::test_const_int32_to_float_cuda, test/inductor/test_torchinductor.py::GPUTests::test_constant_pad_3d_cuda, test/inductor/test_torchinductor.py::GPUTests::test_constant_pad_fill_dtype_cuda, test/inductor/test_torchinductor.py::GPUTests::test_constant_pad_nd_inplace_cuda, test/inductor/test_torchinductor.py::GPUTests::test_conv2d_backward_channels_last_cuda, test/inductor/test_torchinductor.py::GPUTests::test_conv2d_channels_last_cuda, test/inductor/test_torchinductor.py::GPUTests::test_conv3d_channels_last_use_block_ptr_False_cuda, test/inductor/test_torchinductor.py::GPUTests::test_conv3d_cuda, test/inductor/test_torchinductor.py::GPUTests::test_conv_shape_check_cuda, test/inductor/test_torchinductor.py::GPUTests::test_convolution1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_convolution3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cudnn_rnn_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cummin_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cumprod_zero_dim_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cumsum_inf_cuda, test/inductor/test_torchinductor.py::GPUTests::test_cumsum_pattern_matcher_issue_cuda, test/inductor/test_torchinductor.py::GPUTests::test_custom_op_3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_custom_op_unbacked_symints_cuda, test/inductor/test_torchinductor.py::GPUTests::test_data_type_propogation_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dense_mask_index_cuda, test/inductor/test_torchinductor.py::GPUTests::test_deterministic_codegen_on_graph_break_cuda, test/inductor/test_torchinductor.py::GPUTests::test_deterministic_codegen_with_suffix_cuda, test/inductor/test_torchinductor.py::GPUTests::test_device_assert_cuda, test/inductor/test_torchinductor.py::GPUTests::test_div1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_div2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_div3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_div4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_div7_cuda, test/inductor/test_torchinductor.py::GPUTests::test_div_by_zero_cuda, test/inductor/test_torchinductor.py::GPUTests::test_div_precision_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dont_constant_fold_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dropout3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dropout_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dropout_trivial_1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtype_mismatch_issue_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_bfloat16_float32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_bfloat16_float64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_bfloat16_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_bfloat16_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_bfloat16_int64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_bfloat16_int8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float16_bfloat16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float16_float32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float16_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float16_int8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float32_bfloat16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float32_float64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float32_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float32_int64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float64_bfloat16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float64_float32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float64_float64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float64_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float64_int64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float64_int8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_float64_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_fusion_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int16_bfloat16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int16_float64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int16_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int16_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int32_bfloat16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int32_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int64_bfloat16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int64_float16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int64_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int8_float16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int8_int8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_int8_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_uint8_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_uint8_int64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_uint8_int8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_dtypeview_uint8_uint8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_elu_cuda, test/inductor/test_torchinductor.py::GPUTests::test_embedding_bag_byte_unpack_cuda, test/inductor/test_torchinductor.py::GPUTests::test_embedding_bag_cuda, test/inductor/test_torchinductor.py::GPUTests::test_embedding_sparse_cuda, test/inductor/test_torchinductor.py::GPUTests::test_empty_strided_cuda, test/inductor/test_torchinductor.py::GPUTests::test_exp2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_exp_cuda, test/inductor/test_torchinductor.py::GPUTests::test_expand_as_cuda, test/inductor/test_torchinductor.py::GPUTests::test_expand_cuda, test/inductor/test_torchinductor.py::GPUTests::test_expanded_reduction_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fallback_mutable_op_basic_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fallback_mutable_op_list_tensor_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fallback_mutable_op_with_return_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fft_real_input_real_output_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fill1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_float16_to_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_float32_to_int32_cuda, test/inductor/test_torchinductor.py::GPUTests::test_float_index_expression_type_promotion_cuda, test/inductor/test_torchinductor.py::GPUTests::test_floordiv_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fmod_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fmod_zero_dim_cuda, test/inductor/test_torchinductor.py::GPUTests::test_full_boolean_cuda, test/inductor/test_torchinductor.py::GPUTests::test_full_like_cuda, test/inductor/test_torchinductor.py::GPUTests::test_full_truncation_cuda, test/inductor/test_torchinductor.py::GPUTests::test_functionalize_rng_wrappers_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fuse_large_params_cuda, test/inductor/test_torchinductor.py::GPUTests::test_fusing_write_into_disjoint_read_cuda, test/inductor/test_torchinductor.py::GPUTests::test_gather1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_gather3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_gelu_cuda, test/inductor/test_torchinductor.py::GPUTests::test_generate_rand_fp8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_generated_code_has_alignment_assert_cuda, test/inductor/test_torchinductor.py::GPUTests::test_generated_code_has_size_stride_assert_cuda, test/inductor/test_torchinductor.py::GPUTests::test_glu_cuda, test/inductor/test_torchinductor.py::GPUTests::test_graph_partition_arange2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_graph_partition_argmax_cuda, test/inductor/test_torchinductor.py::GPUTests::test_graph_partition_both_scalars_cuda, test/inductor/test_torchinductor.py::GPUTests::test_graph_partition_constant_tensor1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_graph_partition_mutation_real_name_cuda, test/inductor/test_torchinductor.py::GPUTests::test_graph_partition_pad_dynamic_cuda, test/inductor/test_torchinductor.py::GPUTests::test_graph_partition_refcount_cuda, test/inductor/test_torchinductor.py::GPUTests::test_graph_partition_scalar_inputs_cuda, test/inductor/test_torchinductor.py::GPUTests::test_grid_sampler_2d_cuda, test/inductor/test_torchinductor.py::GPUTests::test_hardtanh_cuda, test/inductor/test_torchinductor.py::GPUTests::test_horizonal_fusion1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_propagation_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_propagation_flip_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_propagation_floordiv_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_propagation_remainder_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_put3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_put_as_masked_fill_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_put_failed_reinplace_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_put_fallback1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_put_fallback2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_index_tensor_cuda, test/inductor/test_torchinductor.py::GPUTests::test_indirect_load_broadcast_cuda, test/inductor/test_torchinductor.py::GPUTests::test_inductor_assert_cuda, test/inductor/test_torchinductor.py::GPUTests::test_inductor_layout_optimization_input_mutations_cuda, test/inductor/test_torchinductor.py::GPUTests::test_inplace_mixed_dtype_ops_cuda, test/inductor/test_torchinductor.py::GPUTests::test_inplace_resize_as_cuda, test/inductor/test_torchinductor.py::GPUTests::test_inplace_where_pointwise_cuda, test/inductor/test_torchinductor.py::GPUTests::test_input_mutation2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_int8_weight_only_quant_cuda, test/inductor/test_torchinductor.py::GPUTests::test_int_input_dynamic_shapes_cuda, test/inductor/test_torchinductor.py::GPUTests::test_invalid_operand_issue1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_isin_tensor_scalar_cuda, test/inductor/test_torchinductor.py::GPUTests::test_isinf2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_large_grid_use_block_ptr_False_cuda, test/inductor/test_torchinductor.py::GPUTests::test_large_offset_pointwise_cuda, test/inductor/test_torchinductor.py::GPUTests::test_layer_norm_cuda, test/inductor/test_torchinductor.py::GPUTests::test_lerp_cuda, test/inductor/test_torchinductor.py::GPUTests::test_like_channels_last_cuda, test/inductor/test_torchinductor.py::GPUTests::test_like_rands2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_like_rands_cuda, test/inductor/test_torchinductor.py::GPUTests::test_linear_dynamic_maxautotune_cuda, test/inductor/test_torchinductor.py::GPUTests::test_linear_float64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_linspace3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_list_clearing_cuda, test/inductor/test_torchinductor.py::GPUTests::test_log1p_cuda, test/inductor/test_torchinductor.py::GPUTests::test_log2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_logsumexp_cuda, test/inductor/test_torchinductor.py::GPUTests::test_long_tensor_cuda, test/inductor/test_torchinductor.py::GPUTests::test_low_memory_max_pool_dilation_1_dim_2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_low_memory_max_pool_dilation_1_dim_3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_masked_fill_cuda, test/inductor/test_torchinductor.py::GPUTests::test_masked_fill_promotion_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_pool2d1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_pool2d2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_pool2d4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_pool2d6_dilation_1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_pool2d6_dilation_2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_pool2d7_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_pool2d8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_pool2d_with_indices_backward5_cuda, test/inductor/test_torchinductor.py::GPUTests::test_max_pool2d_with_indices_backward_cuda, test/inductor/test_torchinductor.py::GPUTests::test_mean_cuda, test/inductor/test_torchinductor.py::GPUTests::test_min_max_reduction_cuda, test/inductor/test_torchinductor.py::GPUTests::test_misaligned_address_issue1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_move_arange_cuda, test/inductor/test_torchinductor.py::GPUTests::test_multi_device_cuda, test/inductor/test_torchinductor.py::GPUTests::test_multi_threading_cuda, test/inductor/test_torchinductor.py::GPUTests::test_multilayer_any_cuda, test/inductor/test_torchinductor.py::GPUTests::test_multilayer_prime_size_cuda, test/inductor/test_torchinductor.py::GPUTests::test_multilayer_sum_low_prec_cuda, test/inductor/test_torchinductor.py::GPUTests::test_mutations_loop_fusion_cuda, test/inductor/test_torchinductor.py::GPUTests::test_narrow_cuda, test/inductor/test_torchinductor.py::GPUTests::test_needs_contiguous_strides_cuda, test/inductor/test_torchinductor.py::GPUTests::test_new_empty_cuda, test/inductor/test_torchinductor.py::GPUTests::test_new_empty_strided_cuda, test/inductor/test_torchinductor.py::GPUTests::test_new_ones_cuda, test/inductor/test_torchinductor.py::GPUTests::test_one_hot_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pad_cast_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pad_single_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pad_view_cuda, test/inductor/test_torchinductor.py::GPUTests::test_philox_rand_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pixel_shuffle_channels_last_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_airy_ai_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_bessel_y0_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_bessel_y1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_chebyshev_polynomial_t_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_erf_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_erfcx_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_erfinv_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_exp2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_gammaln_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_hermite_polynomial_h_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_i1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_laguerre_polynomial_l_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_legendre_polynomial_p_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_log1p_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_log_ndtr_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_modified_bessel_i0_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_modified_bessel_i1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_modified_bessel_k1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_multigammaln_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_ndtri_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_psi_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_round_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_scaled_modified_bessel_k0_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_shifted_chebyshev_polynomial_v_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_shifted_chebyshev_polynomial_w_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_spherical_bessel_j0_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pointwise_zeta_cuda, test/inductor/test_torchinductor.py::GPUTests::test_pow_by_natural_log2_dynamic_shapes_cuda, test/inductor/test_torchinductor.py::GPUTests::test_prod_cuda, test/inductor/test_torchinductor.py::GPUTests::test_rand_like_deterministic_cuda, test/inductor/test_torchinductor.py::GPUTests::test_randint_kernel_count_cuda, test/inductor/test_torchinductor.py::GPUTests::test_randn_with_dtype_and_device_cuda, test/inductor/test_torchinductor.py::GPUTests::test_reduction4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_reflection_pad2d_cuda, test/inductor/test_torchinductor.py::GPUTests::test_remainder_cuda, test/inductor/test_torchinductor.py::GPUTests::test_remove_noop_clone_cuda, test/inductor/test_torchinductor.py::GPUTests::test_remove_noop_copy_cuda, test/inductor/test_torchinductor.py::GPUTests::test_remove_noop_slice1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_repeat_as_strided_cuda, test/inductor/test_torchinductor.py::GPUTests::test_repeat_cuda, test/inductor/test_torchinductor.py::GPUTests::test_repeat_interleave_cuda, test/inductor/test_torchinductor.py::GPUTests::test_roi_align_cuda, test/inductor/test_torchinductor.py::GPUTests::test_round_correctness_cuda, test/inductor/test_torchinductor.py::GPUTests::test_round_cuda, test/inductor/test_torchinductor.py::GPUTests::test_rsqrt_dynamic_shapes_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scalar_cpu_tensor_arg_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scalar_input_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scaled_dot_product_efficient_attention_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scatter6_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scatter_add2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scatter_bf16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scatter_reduce2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scatter_reduce3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_scheduler_vertical_fusion1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sdpa_prefer_nd_tiling_False_use_block_ptr_False_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sdpa_prefer_nd_tiling_False_use_block_ptr_True_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sdpa_prefer_nd_tiling_True_use_block_ptr_True_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sdpa_unaligned_mask_cuda, test/inductor/test_torchinductor.py::GPUTests::test_searchsorted_broadcast_cuda, test/inductor/test_torchinductor.py::GPUTests::test_setitem_with_int_parameter_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sgn_cuda, test/inductor/test_torchinductor.py::GPUTests::test_shape_prop_torch_ones_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sign_dtype_cuda, test/inductor/test_torchinductor.py::GPUTests::test_simplify_loops_cuda, test/inductor/test_torchinductor.py::GPUTests::test_single_elem_indirect_cuda, test/inductor/test_torchinductor.py::GPUTests::test_size_asserts_for_multi_output_fallback_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sizehint_issue1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_slice2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_slice_scatter3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_slice_scatter_cuda, test/inductor/test_torchinductor.py::GPUTests::test_slice_view_with_graph_break_cuda, test/inductor/test_torchinductor.py::GPUTests::test_softmax_cuda, test/inductor/test_torchinductor.py::GPUTests::test_softmax_one_kernel_persist_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sort_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sort_stable_cuda, test/inductor/test_torchinductor.py::GPUTests::test_split_cumprod_cuda, test/inductor/test_torchinductor.py::GPUTests::test_split_failed_cuda, test/inductor/test_torchinductor.py::GPUTests::test_split_reduction_with_int64_size_cuda, test/inductor/test_torchinductor.py::GPUTests::test_split_with_list_cuda, test/inductor/test_torchinductor.py::GPUTests::test_split_with_sizes_with_unbacked_symints_cuda, test/inductor/test_torchinductor.py::GPUTests::test_split_with_unbacked_symints_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sqrt_dynamic_shapes_cuda, test/inductor/test_torchinductor.py::GPUTests::test_squeeze1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_stack_cuda, test/inductor/test_torchinductor.py::GPUTests::test_strided_inputs_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sum4_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sum_dtype_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sum_int_cuda, test/inductor/test_torchinductor.py::GPUTests::test_sum_keepdims_cuda, test/inductor/test_torchinductor.py::GPUTests::test_tan_cuda, test/inductor/test_torchinductor.py::GPUTests::test_tanh_cuda, test/inductor/test_torchinductor.py::GPUTests::test_tensor1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_tensor3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_tensor_index_put_slice_cuda, test/inductor/test_torchinductor.py::GPUTests::test_tmp_not_defined_issue3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_topk_cuda, test/inductor/test_torchinductor.py::GPUTests::test_transpose_add_cuda, test/inductor/test_torchinductor.py::GPUTests::test_transpose_cuda, test/inductor/test_torchinductor.py::GPUTests::test_uint_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unroll_small_reduction_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unspec_inputs_float16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unspec_inputs_int16_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unspec_inputs_int64_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unspec_inputs_int8_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unsqueeze_cuda, test/inductor/test_torchinductor.py::GPUTests::test_unsqueeze_inplace_cuda, test/inductor/test_torchinductor.py::GPUTests::test_upsample_bicubic2d_cuda, test/inductor/test_torchinductor.py::GPUTests::test_upsample_cat_conv_cuda, test/inductor/test_torchinductor.py::GPUTests::test_upsample_nearest2d_backward_cuda, test/inductor/test_torchinductor.py::GPUTests::test_var_mean_div_by_cuda, test/inductor/test_torchinductor.py::GPUTests::test_vdd_clamp_cuda, test/inductor/test_torchinductor.py::GPUTests::test_vertical_fusion1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_view_as_real_cuda, test/inductor/test_torchinductor.py::GPUTests::test_views1_cuda, test/inductor/test_torchinductor.py::GPUTests::test_views2_cuda, test/inductor/test_torchinductor.py::GPUTests::test_views3_cuda, test/inductor/test_torchinductor.py::GPUTests::test_where_broadcast_cuda, test/inductor/test_torchinductor.py::GPUTests::test_where_with_logical_op_cuda, test/inductor/test_torchinductor.py::GPUTests::test_zeros_cuda, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_bandwidth_profiler, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_codegen_config_option_dont_assume_alignment, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_comment_graph_fragment, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_computed_indirect_mask, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_constant_folding_deallocation, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_divisible_by_16_covers_numel_args, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_donated_buffer_inplace_gpt, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_evict_last_non_coalesced_loads, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_graph_partition_buffer_reuse, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_graph_partition_foreach_op, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_graph_partition_multiple_functions, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_graph_partition_symint_cat_backward, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_graph_partition_symint_from_mutation_index, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_graph_partition_symint_from_nested_indirect_indexing, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_graph_partition_unbacked_symint_multi_output_layout, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_indirect_device_assert, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_inductor_detach_view_backend_aot_eager, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_kernel_names_descriptive, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_layer_norm_inplaces_after_matmul, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_non_blocking_copy_codegen, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_numpy_autograd, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_optimize_indexing_dtype, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_red_followed_by_transposed_pointwise, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_sdpa_inference_mode_aot_compile, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_skip_l1_cache, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_split_op_with_sym, test/inductor/test_torchinductor.py::TritonCodeGenTests::test_triton_attrs_dict_constexpr_signature, test/inductor/test_torchinductor.py::NanCheckerTest::test_nan_checker_fail
2025-07-24T04:43:36.0013631Z 
2025-07-24T04:43:36.8055102Z Running test batch 'tests to run' cost 564.93 seconds
2025-07-24T04:43:37.6009296Z 
2025-07-24T04:43:37.6009955Z real	9m30.648s
2025-07-24T04:43:37.6010583Z user	16m6.152s
2025-07-24T04:43:37.6011109Z sys	5m42.639s
2025-07-24T04:43:37.6012022Z + assert_git_not_dirty
2025-07-24T04:43:37.6012641Z + [[ linux-jammy-rocm-py3.10 != *rocm* ]]
2025-07-24T04:43:37.6014033Z + test_aten
2025-07-24T04:43:37.6014503Z + echo 'Running ATen tests with pytorch lib'
2025-07-24T04:43:37.6015136Z Running ATen tests with pytorch lib
2025-07-24T04:43:37.6016066Z + [[ -n '' ]]
2025-07-24T04:43:37.6016700Z + echo 'Running test with the build folder'
2025-07-24T04:43:37.6017335Z Running test with the build folder
2025-07-24T04:43:37.6017875Z + TEST_BASE_DIR=build/bin
2025-07-24T04:43:37.6019285Z + ln -sf /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libc10.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libc10_hip.so build/bin
2025-07-24T04:43:37.6068336Z + ln -sf /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libcaffe2_nvrtc.so build/bin
2025-07-24T04:43:37.6104548Z + ln -sf '/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libmkldnn*' build/bin
2025-07-24T04:43:37.6139879Z + ln -sf '/opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libnccl*' build/bin
2025-07-24T04:43:37.6175175Z + ln -sf /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_cpu.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_global_deps.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_hip.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorch_python.so /opt/conda/envs/py_3.10/lib/python3.10/site-packages/torch/lib/libtorchbind_test.so build/bin
2025-07-24T04:43:37.6208651Z + ls build/bin
2025-07-24T04:43:37.6246632Z  BackoffTest
2025-07-24T04:43:37.6247135Z  CMakeFiles
2025-07-24T04:43:37.6247560Z  CTestTestfile.cmake
2025-07-24T04:43:37.6248051Z  CppSignature_test
2025-07-24T04:43:37.6248486Z  Dict_test
2025-07-24T04:43:37.6248866Z  Dimname_test
2025-07-24T04:43:37.6249253Z  FileStoreTest
2025-07-24T04:43:37.6249709Z  HashStoreTest
2025-07-24T04:43:37.6250169Z  IListRef_test
2025-07-24T04:43:37.6250672Z  KernelFunction_test
2025-07-24T04:43:37.6251186Z  List_test
2025-07-24T04:43:37.6251673Z  MaybeOwned_test
2025-07-24T04:43:37.6252122Z  NamedTensor_test
2025-07-24T04:43:37.6252557Z  ProcessGroupGlooTest
2025-07-24T04:43:37.6253021Z  StorageUtils_test
2025-07-24T04:43:37.6253433Z  TCPStoreTest
2025-07-24T04:43:37.6253845Z  aot_model_compiler_test
2025-07-24T04:43:37.6254306Z  apply_utils_test
2025-07-24T04:43:37.6254719Z  atest
2025-07-24T04:43:37.6255107Z  backend_fallback_test
2025-07-24T04:43:37.6255536Z  basic
2025-07-24T04:43:37.6255897Z  broadcast_test
2025-07-24T04:43:37.6256301Z  c10_ArrayRef_test
2025-07-24T04:43:37.6256698Z  c10_Bitset_test
2025-07-24T04:43:37.6257164Z  c10_CompileTimeFunctionPointer_test
2025-07-24T04:43:37.6257732Z  c10_ConstexprCrc_test
2025-07-24T04:43:37.6258180Z  c10_DeadlockDetection_test
2025-07-24T04:43:37.6258660Z  c10_DeviceGuard_test
2025-07-24T04:43:37.6259089Z  c10_Device_test
2025-07-24T04:43:37.6260164Z  c10_DispatchKeySet_test
2025-07-24T04:43:37.6260662Z  c10_Enumerate_test
2025-07-24T04:43:37.6261119Z  c10_Half_test
2025-07-24T04:43:37.6261553Z  c10_InlineDeviceGuard_test
2025-07-24T04:43:37.6262062Z  c10_InlineStreamGuard_test
2025-07-24T04:43:37.6262534Z  c10_IntrusiveList_test
2025-07-24T04:43:37.6263035Z  c10_LeftRight_test
2025-07-24T04:43:37.6263469Z  c10_Metaprogramming_test
2025-07-24T04:43:37.6263935Z  c10_NetworkFlow_test
2025-07-24T04:43:37.6264354Z  c10_Scalar_test
2025-07-24T04:43:37.6264752Z  c10_Semaphore_test
2025-07-24T04:43:37.6265170Z  c10_SizesAndStrides_test
2025-07-24T04:43:37.6265617Z  c10_StreamGuard_test
2025-07-24T04:43:37.6266034Z  c10_SymInt_test
2025-07-24T04:43:37.6266434Z  c10_Synchronized_test
2025-07-24T04:43:37.6266869Z  c10_ThreadLocal_test
2025-07-24T04:43:37.6267302Z  c10_TypeIndex_test
2025-07-24T04:43:37.6267712Z  c10_TypeList_test
2025-07-24T04:43:37.6268121Z  c10_TypeTraits_test
2025-07-24T04:43:37.6268546Z  c10_accumulate_test
2025-07-24T04:43:37.6268967Z  c10_bfloat16_test
2025-07-24T04:43:37.6269362Z  c10_bit_cast_test
2025-07-24T04:43:37.6269768Z  c10_complex_math_test
2025-07-24T04:43:37.6270636Z  c10_complex_test
2025-07-24T04:43:37.6271037Z  c10_cow_test
2025-07-24T04:43:37.6271420Z  c10_error_test
2025-07-24T04:43:37.6271817Z  c10_exception_test
2025-07-24T04:43:37.6272216Z  c10_flags_test
2025-07-24T04:43:37.6272611Z  c10_generic_math_test
2025-07-24T04:43:37.6273084Z  c10_hip_HIPAssertionsTest_1_var_test
2025-07-24T04:43:37.6273687Z  c10_hip_HIPAssertionsTest_catches_stream
2025-07-24T04:43:37.6274535Z  c10_hip_HIPAssertionsTest_catches_thread_and_block_and_device
2025-07-24T04:43:37.6275280Z  c10_hip_HIPAssertionsTest_from_2_processes
2025-07-24T04:43:37.6276070Z  c10_hip_HIPAssertionsTest_multiple_writes_from_blocks_and_threads
2025-07-24T04:43:37.6277121Z  c10_hip_HIPAssertionsTest_multiple_writes_from_multiple_blocks
2025-07-24T04:43:37.6278026Z  c10_hip_HIPAssertionsTest_multiple_writes_from_same_block
2025-07-24T04:43:37.6278671Z  c10_hip_HIPTest
2025-07-24T04:43:37.6279108Z  c10_intrusive_ptr_benchmark
2025-07-24T04:43:37.6279601Z  c10_intrusive_ptr_test
2025-07-24T04:43:37.6280055Z  c10_irange_test
2025-07-24T04:43:37.6280447Z  c10_lazy_test
2025-07-24T04:43:37.6280831Z  c10_logging_test
2025-07-24T04:43:37.6281225Z  c10_optional_test
2025-07-24T04:43:37.6281681Z  c10_ordered_preserving_dict_test
2025-07-24T04:43:37.6282188Z  c10_registry_test
2025-07-24T04:43:37.6282593Z  c10_small_vector_test
2025-07-24T04:43:37.6283014Z  c10_ssize_test
2025-07-24T04:43:37.6283406Z  c10_string_util_test
2025-07-24T04:43:37.6283838Z  c10_string_view_test
2025-07-24T04:43:37.6284258Z  c10_tempfile_test
2025-07-24T04:43:37.6284659Z  c10_typeid_test
2025-07-24T04:43:37.6285061Z  cmake_install.cmake
2025-07-24T04:43:37.6285476Z  cpu_allocator_test
2025-07-24T04:43:37.6285900Z  cpu_generator_test
2025-07-24T04:43:37.6286333Z  cpu_profiling_allocator_test
2025-07-24T04:43:37.6286806Z  cpu_rng_test
2025-07-24T04:43:37.6287198Z  dlconvertor_test
2025-07-24T04:43:37.6287621Z  example_allreduce
2025-07-24T04:43:37.6288050Z  extension_backend_test
2025-07-24T04:43:37.6288488Z  half_test
2025-07-24T04:43:37.6288876Z  hip_apply_test
2025-07-24T04:43:37.6289281Z  hip_complex_math_test
2025-07-24T04:43:37.6289712Z  hip_complex_test
2025-07-24T04:43:37.6290138Z  hip_distributions_test
2025-07-24T04:43:37.6290586Z  hip_dlconvertor_test
2025-07-24T04:43:37.6291015Z  hip_generator_test
2025-07-24T04:43:37.6291420Z  hip_half_test
2025-07-24T04:43:37.6291826Z  hip_integer_divider_test
2025-07-24T04:43:37.6292291Z  hip_optional_test
2025-07-24T04:43:37.6292747Z  hip_packedtensoraccessor_test
2025-07-24T04:43:37.6293240Z  hip_vectorized_test
2025-07-24T04:43:37.6293671Z  inline_container_test
2025-07-24T04:43:37.6294094Z  ivalue_test
2025-07-24T04:43:37.6294503Z  kernel_function_legacy_test
2025-07-24T04:43:37.6294999Z  kernel_function_test
2025-07-24T04:43:37.6295442Z  kernel_lambda_legacy_test
2025-07-24T04:43:37.6295905Z  kernel_lambda_test
2025-07-24T04:43:37.6296656Z  kernel_stackbased_test
2025-07-24T04:43:37.6297107Z  lazy_tensor_test
2025-07-24T04:43:37.6297509Z  legacy_vmap_test
2025-07-24T04:43:37.6297892Z  libc10.so
2025-07-24T04:43:37.6298258Z  libc10_hip.so
2025-07-24T04:43:37.6298650Z  libcaffe2_nvrtc.so
2025-07-24T04:43:37.6299047Z 'libmkldnn*'
2025-07-24T04:43:37.6299429Z 'libnccl*'
2025-07-24T04:43:37.6299793Z  libtorch.so
2025-07-24T04:43:37.6300174Z  libtorch_cpu.so
2025-07-24T04:43:37.6300592Z  libtorch_global_deps.so
2025-07-24T04:43:37.6301044Z  libtorch_hip.so
2025-07-24T04:43:37.6301444Z  libtorch_python.so
2025-07-24T04:43:37.6301853Z  libtorchbind_test.so
2025-07-24T04:43:37.6302323Z  make_boxed_from_unboxed_functor_test
2025-07-24T04:43:37.6302852Z  math_kernel_test
2025-07-24T04:43:37.6303249Z  memory_format_test
2025-07-24T04:43:37.6303748Z  memory_overlapping_test
2025-07-24T04:43:37.6304209Z  mobile_memory_cleanup
2025-07-24T04:43:37.6304629Z  native_test
2025-07-24T04:43:37.6305011Z  op_allowlist_test
2025-07-24T04:43:37.6305423Z  op_registration_test
2025-07-24T04:43:37.6305864Z  operator_name_test
2025-07-24T04:43:37.6306258Z  operators_test
2025-07-24T04:43:37.6306695Z  packedtensoraccessor_test
2025-07-24T04:43:37.6307503Z  parallel_benchmark
2025-07-24T04:43:37.6307904Z  pow_test
2025-07-24T04:43:37.6308270Z  protoc
2025-07-24T04:43:37.6308640Z  protoc-3.13.0.0
2025-07-24T04:43:37.6309048Z  quantized_test
2025-07-24T04:43:37.6309442Z  reduce_ops_test
2025-07-24T04:43:37.6309860Z  reportMemoryUsage_test
2025-07-24T04:43:37.6310297Z  scalar_tensor_test
2025-07-24T04:43:37.6310694Z  scalar_test
2025-07-24T04:43:37.6311084Z  static_runtime_bench
2025-07-24T04:43:37.6311513Z  static_runtime_test
2025-07-24T04:43:37.6311945Z  stride_properties_test
2025-07-24T04:43:37.6312383Z  tensor_iterator_test
2025-07-24T04:43:37.6312852Z  test_api
2025-07-24T04:43:37.6313218Z  test_cpp_rpc
2025-07-24T04:43:37.6313609Z  test_dist_autograd
2025-07-24T04:43:37.6313998Z  test_jit
2025-07-24T04:43:37.6314643Z  test_lazy
2025-07-24T04:43:37.6315033Z  test_mobile_nnc
2025-07-24T04:43:37.6315448Z  test_nativert
2025-07-24T04:43:37.6315835Z  test_parallel
2025-07-24T04:43:37.6316225Z  test_tensorexpr
2025-07-24T04:43:37.6316646Z  thread_init_test
2025-07-24T04:43:37.6317045Z  torch_shm_manager
2025-07-24T04:43:37.6317455Z  tutorial_tensorexpr
2025-07-24T04:43:37.6317869Z  type_ptr_test
2025-07-24T04:43:37.6318241Z  type_test
2025-07-24T04:43:37.6318623Z  undefined_tensor_test
2025-07-24T04:43:37.6319071Z  vec_test_all_types_AVX2
2025-07-24T04:43:37.6319532Z  vec_test_all_types_AVX512
2025-07-24T04:43:37.6320003Z  vec_test_all_types_DEFAULT
2025-07-24T04:43:37.6320473Z  verify_api_visibility
2025-07-24T04:43:37.6320898Z  weakref_test
2025-07-24T04:43:37.6321278Z  wrapdim_test
2025-07-24T04:43:37.6321657Z  xla_tensor_test
2025-07-24T04:43:37.6322087Z + aten/tools/run_tests.sh build/bin
2025-07-24T04:43:37.6322599Z + set -e
2025-07-24T04:43:37.6323013Z ++ dirname aten/tools/run_tests.sh
2025-07-24T04:43:37.6339939Z + VALGRIND_SUP=/var/lib/jenkins/pytorch/aten/tools/valgrind.sup
2025-07-24T04:43:37.6340974Z + export CPP_TESTS_DIR=build/bin
2025-07-24T04:43:37.6341665Z + CPP_TESTS_DIR=build/bin
2025-07-24T04:43:37.6342212Z + VALGRIND=OFF
2025-07-24T04:43:37.6345344Z + python test/run_test.py --cpp --verbose -i cpp/basic cpp/atest cpp/scalar_test cpp/broadcast_test cpp/wrapdim_test cpp/apply_utils_test cpp/dlconvertor_test cpp/native_test cpp/scalar_tensor_test cpp/undefined_tensor_test cpp/extension_backend_test cpp/lazy_tensor_test cpp/tensor_iterator_test cpp/Dimname_test cpp/Dict_test cpp/NamedTensor_test cpp/cpu_generator_test cpp/legacy_vmap_test cpp/operators_test
2025-07-24T04:43:40.3423839Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:43:40.3426475Z   import pkg_resources
2025-07-24T04:43:42.2075710Z Downloading https://ossci-metrics.s3.amazonaws.com/disabled-tests-condensed.json to /var/lib/jenkins/pytorch/test/.pytorch-disabled-tests.json
2025-07-24T04:43:42.2276199Z Found test times from artifacts
2025-07-24T04:43:42.3161080Z Found test times from artifacts
2025-07-24T04:43:42.3193440Z Running 25% of tests based on TD
2025-07-24T04:43:42.3198878Z Running parallel tests on 8 processes
2025-07-24T04:43:42.3203078Z Name: tests to run (est. time: 0.0min)
2025-07-24T04:43:42.3203708Z   Serial tests (0):
2025-07-24T04:43:42.3204154Z   Parallel tests (5):
2025-07-24T04:43:42.3204635Z     cpp/Dict_test 1/1
2025-07-24T04:43:42.3205105Z     cpp/Dimname_test 1/1
2025-07-24T04:43:42.3205608Z     cpp/NamedTensor_test 1/1
2025-07-24T04:43:42.3206117Z     cpp/apply_utils_test 1/1
2025-07-24T04:43:42.3206589Z     cpp/atest 1/1
2025-07-24T04:43:42.3207047Z Name: excluded (est. time: 0.0min)
2025-07-24T04:43:42.3207568Z   Serial tests (0):
2025-07-24T04:43:42.3208003Z   Parallel tests (14):
2025-07-24T04:43:42.3208471Z     cpp/basic 1/1
2025-07-24T04:43:42.3208889Z     cpp/broadcast_test 1/1
2025-07-24T04:43:42.3209381Z     cpp/cpu_generator_test 1/1
2025-07-24T04:43:42.3210452Z     cpp/dlconvertor_test 1/1
2025-07-24T04:43:42.3210953Z     cpp/extension_backend_test 1/1
2025-07-24T04:43:42.3211501Z     cpp/lazy_tensor_test 1/1
2025-07-24T04:43:42.3211981Z     cpp/legacy_vmap_test 1/1
2025-07-24T04:43:42.3212463Z     cpp/native_test 1/1
2025-07-24T04:43:42.3212924Z     cpp/operators_test 1/1
2025-07-24T04:43:42.3213402Z     cpp/scalar_tensor_test 1/1
2025-07-24T04:43:42.3213878Z     cpp/scalar_test 1/1
2025-07-24T04:43:42.3214330Z     cpp/tensor_iterator_test 1/1
2025-07-24T04:43:42.3214854Z     cpp/undefined_tensor_test 1/1
2025-07-24T04:43:42.3215371Z     cpp/wrapdim_test 1/1
2025-07-24T04:43:42.3215971Z Running cpp/Dict_test 1/1 ... [2025-07-24 04:43:42.320076]
2025-07-24T04:43:42.3216659Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set
2025-07-24T04:43:42.3218600Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/Dict_test', '-m', 'serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-3945a9b660ab6588.xml', '-x', '--reruns=2'] ... [2025-07-24 04:43:42.320645]
2025-07-24T04:43:43.5906434Z 
2025-07-24T04:43:43.5907727Z cpp/Dict_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.Dict_test_1.1_a87c421f3b5a71cb_.log
2025-07-24T04:43:43.5908596Z 
2025-07-24T04:43:43.5908993Z GITHUB_RUN_ID, GITHUB_RUN_ATTEMPT, or ARTIFACTS_FILE_SUFFIX not set, not uploading
2025-07-24T04:43:43.5909719Z Uploading artifacts took 0.00 seconds
2025-07-24T04:43:43.5910355Z Running cpp/Dimname_test 1/1 ... [2025-07-24 04:43:43.590405]
2025-07-24T04:43:43.5911004Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set
2025-07-24T04:43:43.5915219Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/Dimname_test', '-m', 'serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-2a18229fcb54253b.xml', '-x', '--reruns=2'] ... [2025-07-24 04:43:43.591029]
2025-07-24T04:43:44.8613369Z 
2025-07-24T04:43:44.8615227Z cpp/Dimname_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.Dimname_test_1.1_6b677b1081eea902_.log
2025-07-24T04:43:44.8616428Z 
2025-07-24T04:43:44.8616806Z Running cpp/NamedTensor_test 1/1 ... [2025-07-24 04:43:44.860821]
2025-07-24T04:43:44.8617560Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set
2025-07-24T04:43:44.8619612Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/NamedTensor_test', '-m', 'serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-405185a613f15a12.xml', '-x', '--reruns=2'] ... [2025-07-24 04:43:44.861248]
2025-07-24T04:43:46.0822723Z 
2025-07-24T04:43:46.0824226Z cpp/NamedTensor_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.NamedTensor_test_1.1_2855606c6f7bcf7f_.log
2025-07-24T04:43:46.0825400Z 
2025-07-24T04:43:46.0827048Z Running cpp/apply_utils_test 1/1 ... [2025-07-24 04:43:46.081885]
2025-07-24T04:43:46.0827864Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set
2025-07-24T04:43:46.0829931Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/apply_utils_test', '-m', 'serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-f489ef7db2f28923.xml', '-x', '--reruns=2'] ... [2025-07-24 04:43:46.082383]
2025-07-24T04:43:47.3526589Z 
2025-07-24T04:43:47.3528207Z cpp/apply_utils_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.apply_utils_test_1.1_af2f0dda54a493b2_.log
2025-07-24T04:43:47.3529233Z 
2025-07-24T04:43:47.3529511Z Running cpp/atest 1/1 ... [2025-07-24 04:43:47.352168]
2025-07-24T04:43:47.3530199Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set
2025-07-24T04:43:47.3533856Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/atest', '-m', 'serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-241054cb6d382686.xml', '-x', '--reruns=2'] ... [2025-07-24 04:43:47.352809]
2025-07-24T04:43:48.6227861Z 
2025-07-24T04:43:48.6229102Z cpp/atest 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.atest_1.1_747c43b02e064426_.log
2025-07-24T04:43:48.6230136Z 
2025-07-24T04:43:51.7763783Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:43:51.7765322Z   import pkg_resources
2025-07-24T04:43:51.9093653Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:43:51.9095242Z   import pkg_resources
2025-07-24T04:43:51.9096697Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:43:51.9098205Z   import pkg_resources
2025-07-24T04:43:51.9636973Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:43:51.9638614Z   import pkg_resources
2025-07-24T04:43:51.9640045Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:43:51.9641490Z   import pkg_resources
2025-07-24T04:43:51.9686544Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:43:51.9688298Z   import pkg_resources
2025-07-24T04:43:51.9712092Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:43:51.9713574Z   import pkg_resources
2025-07-24T04:43:51.9738696Z /opt/conda/envs/py_3.10/lib/python3.10/site-packages/hypothesis/entry_points.py:23: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
2025-07-24T04:43:51.9740094Z   import pkg_resources
2025-07-24T04:43:52.2961292Z Running cpp/Dict_test 1/1 ... [2025-07-24 04:43:52.294847]
2025-07-24T04:43:52.2962242Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set
2025-07-24T04:43:52.2964357Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/Dict_test', '-m', 'not serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-24ad6dec6dd69a3e.xml', '-x', '--reruns=2'] ... [2025-07-24 04:43:52.295754]
2025-07-24T04:43:52.5029441Z Running cpp/Dimname_test 1/1 ... [2025-07-24 04:43:52.502015]
2025-07-24T04:43:52.5030280Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set
2025-07-24T04:43:52.5042216Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/Dimname_test', '-m', 'not serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-17cae88540cc83db.xml', '-x', '--reruns=2'] ... [2025-07-24 04:43:52.502896]
2025-07-24T04:43:52.5195094Z Running cpp/NamedTensor_test 1/1 ... [2025-07-24 04:43:52.518080]
2025-07-24T04:43:52.5195963Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set
2025-07-24T04:43:52.5198186Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/NamedTensor_test', '-m', 'not serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-ae03b25313106943.xml', '-x', '--reruns=2'] ... [2025-07-24 04:43:52.518915]
2025-07-24T04:43:52.5998735Z Running cpp/apply_utils_test 1/1 ... [2025-07-24 04:43:52.598848]
2025-07-24T04:43:52.5999621Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set
2025-07-24T04:43:52.6004287Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/apply_utils_test', '-m', 'not serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-5f59f96ed5004216.xml', '-x', '--reruns=2'] ... [2025-07-24 04:43:52.599814]
2025-07-24T04:43:52.6147952Z Running cpp/atest 1/1 ... [2025-07-24 04:43:52.613838]
2025-07-24T04:43:52.6148735Z SCRIBE_GRAPHQL_ACCESS_TOKEN is NOT set
2025-07-24T04:43:52.6150848Z Executing ['pytest', '/var/lib/jenkins/pytorch/build/bin/atest', '-m', 'not serial', '-v', '-vv', '-rfEX', '-n', '8', '--junit-xml-reruns', 'test-reports/python-pytest/test.run_test/test.run_test-03c35a0dc91bb7ea.xml', '-x', '--reruns=2'] ... [2025-07-24 04:43:52.614463]
2025-07-24T04:43:53.6150058Z 
2025-07-24T04:43:53.6151521Z cpp/Dict_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.Dict_test_1.1_e6492ff42a7694cf_.log
2025-07-24T04:43:53.6152729Z 
2025-07-24T04:43:53.7730450Z 
2025-07-24T04:43:53.7731888Z cpp/Dimname_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.Dimname_test_1.1_8a151a07cf433d90_.log
2025-07-24T04:43:53.7732990Z 
2025-07-24T04:43:53.7888344Z 
2025-07-24T04:43:53.7889590Z cpp/NamedTensor_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.NamedTensor_test_1.1_d004de74f679eede_.log
2025-07-24T04:43:53.7890637Z 
2025-07-24T04:43:53.8191410Z 
2025-07-24T04:43:53.8192807Z cpp/apply_utils_test 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.apply_utils_test_1.1_fbc8fb2e2d444762_.log
2025-07-24T04:43:53.8194029Z 
2025-07-24T04:43:53.8340892Z 
2025-07-24T04:43:53.8341839Z cpp/atest 1/1 was successful, full logs can be found in artifacts with path test/test-reports/cpp.atest_1.1_fe387e291002bb02_.log
2025-07-24T04:43:53.8343226Z 
2025-07-24T04:43:54.6745538Z Running test batch 'tests to run' cost 12.35 seconds
2025-07-24T04:43:55.2715156Z + run_if_exists tensor_interop_test
2025-07-24T04:43:55.2716045Z + local test_name=tensor_interop_test
2025-07-24T04:43:55.2716676Z + [[ -x build/bin/tensor_interop_test ]]
2025-07-24T04:43:55.2717342Z + echo 'Warning: tensor_interop_test does not exist.'
2025-07-24T04:43:55.2718011Z Warning: tensor_interop_test does not exist.
2025-07-24T04:43:55.2719944Z + run_if_exists cudnn_test
2025-07-24T04:43:55.2720637Z + local test_name=cudnn_test
2025-07-24T04:43:55.2721215Z + [[ -x build/bin/cudnn_test ]]
2025-07-24T04:43:55.2721808Z + echo 'Warning: cudnn_test does not exist.'
2025-07-24T04:43:55.2722446Z Warning: cudnn_test does not exist.
2025-07-24T04:43:55.2723020Z + run_if_exists cuda_generator_test
2025-07-24T04:43:55.2723575Z + local test_name=cuda_generator_test
2025-07-24T04:43:55.2724185Z + [[ -x build/bin/cuda_generator_test ]]
2025-07-24T04:43:55.2724848Z + echo 'Warning: cuda_generator_test does not exist.'
2025-07-24T04:43:55.2725603Z Warning: cuda_generator_test does not exist.
2025-07-24T04:43:55.2726269Z + run_if_exists apply_test
2025-07-24T04:43:55.2726783Z + local test_name=apply_test
2025-07-24T04:43:55.2728172Z + [[ -x build/bin/apply_test ]]
2025-07-24T04:43:55.2728744Z + echo 'Warning: apply_test does not exist.'
2025-07-24T04:43:55.2729342Z Warning: apply_test does not exist.
2025-07-24T04:43:55.2729883Z + run_if_exists stream_test
2025-07-24T04:43:55.2730365Z + local test_name=stream_test
2025-07-24T04:43:55.2730877Z + [[ -x build/bin/stream_test ]]
2025-07-24T04:43:55.2731430Z + echo 'Warning: stream_test does not exist.'
2025-07-24T04:43:55.2732024Z Warning: stream_test does not exist.
2025-07-24T04:43:55.2732572Z + run_if_exists cuda_half_test
2025-07-24T04:43:55.2733075Z + local test_name=cuda_half_test
2025-07-24T04:43:55.2733595Z + [[ -x build/bin/cuda_half_test ]]
2025-07-24T04:43:55.2734179Z + echo 'Warning: cuda_half_test does not exist.'
2025-07-24T04:43:55.2734792Z Warning: cuda_half_test does not exist.
2025-07-24T04:43:55.2735376Z + run_if_exists cuda_vectorized_test
2025-07-24T04:43:55.2735931Z + local test_name=cuda_vectorized_test
2025-07-24T04:43:55.2736490Z + [[ -x build/bin/cuda_vectorized_test ]]
2025-07-24T04:43:55.2737144Z + echo 'Warning: cuda_vectorized_test does not exist.'
2025-07-24T04:43:55.2737823Z Warning: cuda_vectorized_test does not exist.
2025-07-24T04:43:55.2738432Z + run_if_exists cuda_distributions_test
2025-07-24T04:43:55.2739012Z + local test_name=cuda_distributions_test
2025-07-24T04:43:55.2739611Z + [[ -x build/bin/cuda_distributions_test ]]
2025-07-24T04:43:55.2740280Z + echo 'Warning: cuda_distributions_test does not exist.'
2025-07-24T04:43:55.2740997Z Warning: cuda_distributions_test does not exist.
2025-07-24T04:43:55.2741606Z + run_if_exists cuda_optional_test
2025-07-24T04:43:55.2742145Z + local test_name=cuda_optional_test
2025-07-24T04:43:55.2742696Z + [[ -x build/bin/cuda_optional_test ]]
2025-07-24T04:43:55.2743318Z + echo 'Warning: cuda_optional_test does not exist.'
2025-07-24T04:43:55.2743971Z Warning: cuda_optional_test does not exist.
2025-07-24T04:43:55.2744578Z + run_if_exists cuda_tensor_interop_test
2025-07-24T04:43:55.2745161Z + local test_name=cuda_tensor_interop_test
2025-07-24T04:43:55.2745772Z + [[ -x build/bin/cuda_tensor_interop_test ]]
2025-07-24T04:43:55.2746444Z + echo 'Warning: cuda_tensor_interop_test does not exist.'
2025-07-24T04:43:55.2747160Z Warning: cuda_tensor_interop_test does not exist.
2025-07-24T04:43:55.2747784Z + run_if_exists cuda_complex_test
2025-07-24T04:43:55.2748328Z + local test_name=cuda_complex_test
2025-07-24T04:43:55.2748878Z + [[ -x build/bin/cuda_complex_test ]]
2025-07-24T04:43:55.2749480Z + echo 'Warning: cuda_complex_test does not exist.'
2025-07-24T04:43:55.2750127Z Warning: cuda_complex_test does not exist.
2025-07-24T04:43:55.2750695Z + run_if_exists cuda_complex_math_test
2025-07-24T04:43:55.2751249Z + local test_name=cuda_complex_math_test
2025-07-24T04:43:55.2751831Z + [[ -x build/bin/cuda_complex_math_test ]]
2025-07-24T04:43:55.2752826Z + echo 'Warning: cuda_complex_math_test does not exist.'
2025-07-24T04:43:55.2753538Z Warning: cuda_complex_math_test does not exist.
2025-07-24T04:43:55.2754140Z + run_if_exists cuda_cub_test
2025-07-24T04:43:55.2754829Z + local test_name=cuda_cub_test
2025-07-24T04:43:55.2755353Z + [[ -x build/bin/cuda_cub_test ]]
2025-07-24T04:43:55.2755926Z + echo 'Warning: cuda_cub_test does not exist.'
2025-07-24T04:43:55.2756518Z Warning: cuda_cub_test does not exist.
2025-07-24T04:43:55.2757083Z + run_if_exists cuda_atomic_ops_test
2025-07-24T04:43:55.2757634Z + local test_name=cuda_atomic_ops_test
2025-07-24T04:43:55.2758185Z + [[ -x build/bin/cuda_atomic_ops_test ]]
2025-07-24T04:43:55.2758824Z + echo 'Warning: cuda_atomic_ops_test does not exist.'
2025-07-24T04:43:55.2759497Z Warning: cuda_atomic_ops_test does not exist.
2025-07-24T04:43:55.2760076Z + '[' OFF == ON ']'
2025-07-24T04:43:55.2760517Z + [[ -n '' ]]
2025-07-24T04:43:55.2760954Z + assert_git_not_dirty
2025-07-24T04:43:55.2761454Z + [[ linux-jammy-rocm-py3.10 != *rocm* ]]
2025-07-24T04:43:55.2762021Z + sccache_epilogue
2025-07-24T04:43:55.2762560Z + echo '::group::Sccache Compilation Log'
2025-07-24T04:43:55.2763679Z ##[group]Sccache Compilation Log
2025-07-24T04:43:55.2764686Z + echo '=================== sccache compilation log ==================='
2025-07-24T04:43:55.2765422Z =================== sccache compilation log ===================
2025-07-24T04:43:55.2766482Z + python /var/lib/jenkins/pytorch/.ci/pytorch/print_sccache_log.py /var/lib/jenkins/sccache_error.log
2025-07-24T04:43:55.3046934Z + echo '=========== If your build fails, please take a look at the log above for possible reasons ==========='
2025-07-24T04:43:55.3048233Z =========== If your build fails, please take a look at the log above for possible reasons ===========
2025-07-24T04:43:55.3049116Z + sccache --show-stats
2025-07-24T04:43:55.3106172Z Compile requests                    681
2025-07-24T04:43:55.3106893Z Compile requests executed            68
2025-07-24T04:43:55.3107451Z Cache hits                            4
2025-07-24T04:43:55.3108030Z Cache hits (C/C++)                    4
2025-07-24T04:43:55.3108624Z Cache misses                         64
2025-07-24T04:43:55.3109190Z Cache misses (C/C++)                 58
2025-07-24T04:43:55.3109726Z Cache misses (HIP)                    6
2025-07-24T04:43:55.3110269Z Cache hits rate                    5.88 %
2025-07-24T04:43:55.3110847Z Cache hits rate (C/C++)            6.45 %
2025-07-24T04:43:55.3111398Z Cache hits rate (HIP)              0.00 %
2025-07-24T04:43:55.3111942Z Cache timeouts                        0
2025-07-24T04:43:55.3112465Z Cache read errors                     0
2025-07-24T04:43:55.3112986Z Forced recaches                       0
2025-07-24T04:43:55.3113525Z Cache write errors                    0
2025-07-24T04:43:55.3114051Z Cache errors                          0
2025-07-24T04:43:55.3114732Z Compilations                         64
2025-07-24T04:43:55.3115265Z Compilation failures                  0
2025-07-24T04:43:55.3115830Z Non-cacheable compilations            0
2025-07-24T04:43:55.3116384Z Non-cacheable calls                  11
2025-07-24T04:43:55.3116924Z Non-compilation calls               602
2025-07-24T04:43:55.3117497Z Unsupported compiler calls            0
2025-07-24T04:43:55.3118072Z Average cache write               0.000 s
2025-07-24T04:43:55.3118653Z Average compiler                  8.334 s
2025-07-24T04:43:55.3119226Z Average cache read hit            0.000 s
2025-07-24T04:43:55.3119788Z Failed distributed compilations       0
2025-07-24T04:43:55.3120173Z 
2025-07-24T04:43:55.3120362Z Non-cacheable reasons:
2025-07-24T04:43:55.3120851Z unknown source language               9
2025-07-24T04:43:55.3121380Z -E                                    2
2025-07-24T04:43:55.3121746Z 
2025-07-24T04:43:55.3122106Z Cache location                  Local disk: "/var/lib/jenkins/.cache/sccache"
2025-07-24T04:43:55.3122861Z Use direct/preprocessor mode?   yes
2025-07-24T04:43:55.3123429Z Version (client)                0.10.0
2025-07-24T04:43:55.3124458Z Cache size                            3 MiB
2025-07-24T04:43:55.3125054Z Max cache size                       10 GiB
2025-07-24T04:43:55.3125642Z + sccache --stop-server
2025-07-24T04:43:55.3175355Z Stopping sccache server...
2025-07-24T04:43:55.3182167Z Compile requests                    681
2025-07-24T04:43:55.3182899Z Compile requests executed            68
2025-07-24T04:43:55.3183468Z Cache hits                            4
2025-07-24T04:43:55.3184018Z Cache hits (C/C++)                    4
2025-07-24T04:43:55.3184533Z Cache misses                         64
2025-07-24T04:43:55.3185044Z Cache misses (C/C++)                 58
2025-07-24T04:43:55.3185583Z Cache misses (HIP)                    6
2025-07-24T04:43:55.3186122Z Cache hits rate                    5.88 %
2025-07-24T04:43:55.3186676Z Cache hits rate (C/C++)            6.45 %
2025-07-24T04:43:55.3187240Z Cache hits rate (HIP)              0.00 %
2025-07-24T04:43:55.3187790Z Cache timeouts                        0
2025-07-24T04:43:55.3188348Z Cache read errors                     0
2025-07-24T04:43:55.3188885Z Forced recaches                       0
2025-07-24T04:43:55.3189404Z Cache write errors                    0
2025-07-24T04:43:55.3190415Z Cache errors                          0
2025-07-24T04:43:55.3190927Z Compilations                         64
2025-07-24T04:43:55.3191450Z Compilation failures                  0
2025-07-24T04:43:55.3191989Z Non-cacheable compilations            0
2025-07-24T04:43:55.3192537Z Non-cacheable calls                  11
2025-07-24T04:43:55.3193069Z Non-compilation calls               602
2025-07-24T04:43:55.3193615Z Unsupported compiler calls            0
2025-07-24T04:43:55.3194201Z Average cache write               0.000 s
2025-07-24T04:43:55.3194937Z Average compiler                  8.334 s
2025-07-24T04:43:55.3195493Z Average cache read hit            0.000 s
2025-07-24T04:43:55.3196062Z Failed distributed compilations       0
2025-07-24T04:43:55.3196429Z 
2025-07-24T04:43:55.3196624Z Non-cacheable reasons:
2025-07-24T04:43:55.3197121Z unknown source language               9
2025-07-24T04:43:55.3197634Z -E                                    2
2025-07-24T04:43:55.3198027Z 
2025-07-24T04:43:55.3198393Z Cache location                  Local disk: "/var/lib/jenkins/.cache/sccache"
2025-07-24T04:43:55.3199177Z Use direct/preprocessor mode?   yes
2025-07-24T04:43:55.3199771Z Version (client)                0.10.0
2025-07-24T04:43:55.3200354Z Cache size                            3 MiB
2025-07-24T04:43:55.3200918Z Max cache size                       10 GiB
2025-07-24T04:43:55.3201554Z + echo ::endgroup::
2025-07-24T04:43:55.3202322Z ##[endgroup]
2025-07-24T04:43:55.3364697Z ##[group]Run # copy test results back to the mounted workspace, needed sudo, resulting permissions were correct
2025-07-24T04:43:55.3366229Z # copy test results back to the mounted workspace, needed sudo, resulting permissions were correct
2025-07-24T04:43:55.3368022Z docker exec -t "51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c" sh -c "cd ../pytorch && sudo cp -R test/test-reports ../workspace/test"
2025-07-24T04:43:55.3431930Z shell: /usr/bin/bash -e {0}
2025-07-24T04:43:55.3432480Z env:
2025-07-24T04:43:55.3432884Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:43:55.3433665Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:43:55.3434885Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:43:55.3435917Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:43:55.3437690Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:43:55.3439220Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:43:55.3439746Z   AWS_REGION: us-east-1
2025-07-24T04:43:55.3440337Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:43:55.3441028Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:43:55.3451577Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:43:55.3452342Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:43:55.3453172Z ##[endgroup]
2025-07-24T04:43:55.4925860Z ##[group]Run cat test/**/*_toprint.log || true
2025-07-24T04:43:55.4926622Z cat test/**/*_toprint.log || true
2025-07-24T04:43:55.4993091Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:43:55.4993795Z env:
2025-07-24T04:43:55.4994214Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:43:55.4995105Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:43:55.4996271Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:43:55.4997331Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:43:55.4999066Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:43:55.5000618Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:43:55.5001595Z   AWS_REGION: us-east-1
2025-07-24T04:43:55.5002275Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:43:55.5002959Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:43:55.5013338Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:43:55.5014101Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:43:55.5014917Z ##[endgroup]
2025-07-24T04:43:55.5328872Z cat: 'test/**/*_toprint.log': No such file or directory
2025-07-24T04:43:55.5562081Z Prepare all required actions
2025-07-24T04:43:55.5563184Z Getting action download info
2025-07-24T04:43:55.8892759Z Download action repository 'seemethere/upload-artifact-s3@v5' (SHA:baba72d0712b404f646cebe0730933554ebce96a)
2025-07-24T04:43:56.7896133Z Download action repository 'actions/upload-artifact@v4' (SHA:ea165f8d65b6e75b540449e92b4886f43607fa02)
2025-07-24T04:43:57.4977715Z ##[group]Run ./.github/actions/upload-test-artifacts
2025-07-24T04:43:57.4978015Z with:
2025-07-24T04:43:57.4978184Z   use-gha: true
2025-07-24T04:43:57.4978453Z   file-suffix: test-default-2-2-linux.rocm.gpu.2_46615439132
2025-07-24T04:43:57.4978752Z   s3-bucket: gha-artifacts
2025-07-24T04:43:57.4978944Z env:
2025-07-24T04:43:57.4979114Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:43:57.4979440Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:43:57.4979908Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:43:57.4980406Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:43:57.4981125Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:43:57.4981745Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:43:57.4981952Z   AWS_REGION: us-east-1
2025-07-24T04:43:57.4982199Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:43:57.4982474Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:43:57.5012366Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:43:57.5012743Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:43:57.5013115Z ##[endgroup]
2025-07-24T04:43:57.5094945Z ##[group]Run actions/upload-artifact@v4
2025-07-24T04:43:57.5095186Z with:
2025-07-24T04:43:57.5095490Z   name: test-jsons-runattempt1-test-default-2-2-linux.rocm.gpu.2_46615439132.zip
2025-07-24T04:43:57.5095855Z   retention-days: 14
2025-07-24T04:43:57.5096058Z   if-no-files-found: warn
2025-07-24T04:43:57.5096265Z   path: test/**/*.json
2025-07-24T04:43:57.5096453Z   compression-level: 6
2025-07-24T04:43:57.5096644Z   overwrite: false
2025-07-24T04:43:57.5096834Z   include-hidden-files: false
2025-07-24T04:43:57.5097035Z env:
2025-07-24T04:43:57.5097192Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:43:57.5097491Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:43:57.5097951Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:43:57.5098385Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:43:57.5099081Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:43:57.5099704Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:43:57.5099918Z   AWS_REGION: us-east-1
2025-07-24T04:43:57.5100156Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:43:57.5100433Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:43:57.5104633Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:43:57.5104947Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:43:57.5105281Z ##[endgroup]
2025-07-24T04:43:58.3517024Z With the provided path, there will be 7 files uploaded
2025-07-24T04:43:58.3522125Z Artifact name is valid!
2025-07-24T04:43:58.3522976Z Root directory input is valid!
2025-07-24T04:43:58.5282186Z Beginning upload of artifact content to blob storage
2025-07-24T04:43:58.7715871Z Uploaded bytes 42778
2025-07-24T04:43:58.8194622Z Finished uploading artifact content to blob storage!
2025-07-24T04:43:58.8200212Z SHA256 digest of uploaded artifact zip is c700b34eba8bbce9260afcabeb0865a0fbcea200f22dd8c8d760e7b7c78c7970
2025-07-24T04:43:58.8203167Z Finalizing artifact upload
2025-07-24T04:43:58.9234198Z Artifact test-jsons-runattempt1-test-default-2-2-linux.rocm.gpu.2_46615439132.zip.zip successfully finalized. Artifact ID 3603118984
2025-07-24T04:43:58.9236815Z Artifact test-jsons-runattempt1-test-default-2-2-linux.rocm.gpu.2_46615439132.zip has been successfully uploaded! Final size is 42778 bytes. Artifact ID is 3603118984
2025-07-24T04:43:58.9250074Z Artifact download URL: https://github.com/pytorch/pytorch/actions/runs/16487210679/artifacts/3603118984
2025-07-24T04:43:58.9629743Z ##[group]Run actions/upload-artifact@v4
2025-07-24T04:43:58.9630331Z with:
2025-07-24T04:43:58.9631124Z   name: test-reports-runattempt1-test-default-2-2-linux.rocm.gpu.2_46615439132.zip
2025-07-24T04:43:58.9632058Z   retention-days: 14
2025-07-24T04:43:58.9632550Z   if-no-files-found: ignore
2025-07-24T04:43:58.9633095Z   path: test/**/*.xml
test/**/*.csv

2025-07-24T04:43:58.9633667Z   compression-level: 6
2025-07-24T04:43:58.9634134Z   overwrite: false
2025-07-24T04:43:58.9634764Z   include-hidden-files: false
2025-07-24T04:43:58.9635268Z env:
2025-07-24T04:43:58.9635703Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:43:58.9636482Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:43:58.9637639Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:43:58.9638699Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:43:58.9640448Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:43:58.9642024Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:43:58.9642560Z   AWS_REGION: us-east-1
2025-07-24T04:43:58.9643158Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:43:58.9643859Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:43:58.9654311Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:43:58.9655124Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:43:58.9655990Z ##[endgroup]
2025-07-24T04:43:59.8465632Z With the provided path, there will be 16 files uploaded
2025-07-24T04:43:59.8470509Z Artifact name is valid!
2025-07-24T04:43:59.8471110Z Root directory input is valid!
2025-07-24T04:44:00.0658676Z Beginning upload of artifact content to blob storage
2025-07-24T04:44:00.2938694Z Uploaded bytes 35902
2025-07-24T04:44:00.3366927Z Finished uploading artifact content to blob storage!
2025-07-24T04:44:00.3368594Z SHA256 digest of uploaded artifact zip is 394d01f3c01d8a11fb7d5e7c6b4819ea5e078a984d678e1b6872287462e9cd58
2025-07-24T04:44:00.3369789Z Finalizing artifact upload
2025-07-24T04:44:00.4708314Z Artifact test-reports-runattempt1-test-default-2-2-linux.rocm.gpu.2_46615439132.zip.zip successfully finalized. Artifact ID 3603119085
2025-07-24T04:44:00.4710774Z Artifact test-reports-runattempt1-test-default-2-2-linux.rocm.gpu.2_46615439132.zip has been successfully uploaded! Final size is 35902 bytes. Artifact ID is 3603119085
2025-07-24T04:44:00.4720615Z Artifact download URL: https://github.com/pytorch/pytorch/actions/runs/16487210679/artifacts/3603119085
2025-07-24T04:44:00.4975430Z ##[group]Run actions/upload-artifact@v4
2025-07-24T04:44:00.4975806Z with:
2025-07-24T04:44:00.4976232Z   name: logs-runattempt1-test-default-2-2-linux.rocm.gpu.2_46615439132.zip
2025-07-24T04:44:00.4976740Z   retention-days: 14
2025-07-24T04:44:00.4977039Z   if-no-files-found: ignore
2025-07-24T04:44:00.4977381Z   path: usage_log.txt
test/**/*.log

2025-07-24T04:44:00.4977766Z   compression-level: 6
2025-07-24T04:44:00.4978125Z   overwrite: false
2025-07-24T04:44:00.4978618Z   include-hidden-files: false
2025-07-24T04:44:00.4979732Z env:
2025-07-24T04:44:00.4980214Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:00.4981063Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:00.4982227Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:00.4983287Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:00.4985449Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:00.4987064Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:00.4987600Z   AWS_REGION: us-east-1
2025-07-24T04:44:00.4988203Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:00.4988933Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:00.4999927Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:00.5000742Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:00.5001646Z ##[endgroup]
2025-07-24T04:44:01.4069698Z Multiple search paths detected. Calculating the least common ancestor of all paths
2025-07-24T04:44:01.4072078Z The least common ancestor is /var/home/pytorchci/actions-runner/_work/pytorch/pytorch. This will be the root directory of the artifact
2025-07-24T04:44:01.4073592Z With the provided path, there will be 14 files uploaded
2025-07-24T04:44:01.4077371Z Artifact name is valid!
2025-07-24T04:44:01.4078573Z Root directory input is valid!
2025-07-24T04:44:01.5845399Z Beginning upload of artifact content to blob storage
2025-07-24T04:44:01.8068042Z Uploaded bytes 37534
2025-07-24T04:44:01.8504226Z Finished uploading artifact content to blob storage!
2025-07-24T04:44:01.8505814Z SHA256 digest of uploaded artifact zip is 8b40f0a26684d545a2cb2a79bf7a33306acaaecaa17e64423b5470b0c9b99b85
2025-07-24T04:44:01.8507031Z Finalizing artifact upload
2025-07-24T04:44:01.9551422Z Artifact logs-runattempt1-test-default-2-2-linux.rocm.gpu.2_46615439132.zip.zip successfully finalized. Artifact ID 3603119175
2025-07-24T04:44:01.9553653Z Artifact logs-runattempt1-test-default-2-2-linux.rocm.gpu.2_46615439132.zip has been successfully uploaded! Final size is 37534 bytes. Artifact ID is 3603119175
2025-07-24T04:44:01.9562530Z Artifact download URL: https://github.com/pytorch/pytorch/actions/runs/16487210679/artifacts/3603119175
2025-07-24T04:44:01.9818860Z ##[group]Run # shellcheck disable=SC2156
2025-07-24T04:44:01.9819626Z # shellcheck disable=SC2156
2025-07-24T04:44:01.9820717Z find . -iname "core.[1-9]*" -exec docker exec "${CONTAINER_NAME}" sh -c "gdb python {} -ex 'bt' -ex 'q'" \;
2025-07-24T04:44:01.9890075Z shell: /usr/bin/bash -e {0}
2025-07-24T04:44:01.9890695Z env:
2025-07-24T04:44:01.9891147Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:01.9891976Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:01.9893146Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:01.9894319Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:01.9896141Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:01.9898078Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:01.9898757Z   AWS_REGION: us-east-1
2025-07-24T04:44:01.9899381Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:01.9900149Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:01.9910638Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:01.9911468Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:01.9912320Z ##[endgroup]
2025-07-24T04:44:02.3879985Z ##[group]Run aws-actions/configure-aws-credentials@ececac1a45f3b08a01d2dd070d28d111c5fe6722
2025-07-24T04:44:02.3880884Z with:
2025-07-24T04:44:02.3881546Z   role-to-assume: arn:aws:iam::308535385114:role/gha_workflow_upload-benchmark-results
2025-07-24T04:44:02.3882784Z   role-duration-seconds: 18000
2025-07-24T04:44:02.3883300Z   aws-region: us-east-1
2025-07-24T04:44:02.3883852Z   audience: sts.amazonaws.com
2025-07-24T04:44:02.3884391Z env:
2025-07-24T04:44:02.3884838Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:02.3885706Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:02.3886957Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:02.3888090Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:02.3890013Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:02.3891449Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:02.3891938Z   AWS_REGION: us-east-1
2025-07-24T04:44:02.3892477Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:02.3893133Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:02.3902931Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:02.3903646Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:02.3904411Z ##[endgroup]
2025-07-24T04:44:02.6855590Z Assuming role with OIDC
2025-07-24T04:44:02.8580957Z Authenticated as assumedRoleId AROAUPVRELQNA5GQHA6IA:GitHubActions
2025-07-24T04:44:02.9361070Z ##[group]Run pytorch/test-infra/.github/actions/upload-benchmark-results@main
2025-07-24T04:44:02.9362028Z with:
2025-07-24T04:44:02.9362559Z   benchmark-results-dir: test/test-reports
2025-07-24T04:44:02.9363205Z   dry-run: false
2025-07-24T04:44:02.9363701Z   schema-version: v3
2025-07-24T04:44:02.9364657Z   github-token: ***
2025-07-24T04:44:02.9365109Z env:
2025-07-24T04:44:02.9365519Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:02.9366271Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:02.9367355Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:02.9368397Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:02.9369992Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:02.9371488Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:02.9372010Z   AWS_REGION: us-east-1
2025-07-24T04:44:02.9372558Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:02.9373228Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:02.9385292Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:02.9386031Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:02.9386824Z ##[endgroup]
2025-07-24T04:44:02.9418896Z ##[group]Run set -eux
2025-07-24T04:44:02.9419391Z set -eux
2025-07-24T04:44:02.9420026Z python3 -mpip install boto3==1.35.33 psutil==7.0.0 pynvml==12.0.0
2025-07-24T04:44:02.9420757Z 
2025-07-24T04:44:02.9421158Z DEVICE_NAME=""
2025-07-24T04:44:02.9421635Z DEVICE_TYPE=""
2025-07-24T04:44:02.9422069Z 
2025-07-24T04:44:02.9422505Z if command -v nvidia-smi; then
2025-07-24T04:44:02.9423282Z   # NB: I'm using PyTorch here to get the device name, however, it needs to
2025-07-24T04:44:02.9424229Z   # install the correct version of PyTorch manually for now. Any PyTorch
2025-07-24T04:44:02.9425119Z   # version is fine, I just use 2.7.1 to satify PYPIDEP linter
2025-07-24T04:44:02.9425849Z   python3 -mpip install torch==2.7.1
2025-07-24T04:44:02.9426448Z elif command -v rocminfo; then
2025-07-24T04:44:02.9427244Z   # NB: Installing torch on ROCm runner with pip here causes CI to fail
2025-07-24T04:44:02.9428202Z   # with a memoryview is too large error only on MI300 runners. Is pip
2025-07-24T04:44:02.9429546Z   # version on ROCm runner there too old? As a workaround, let's use the
2025-07-24T04:44:02.9430393Z   # GPU device name coming from rocminfo instead
2025-07-24T04:44:02.9431036Z   DEVICE_NAME=rocm
2025-07-24T04:44:02.9431856Z   DEVICE_TYPE=$(rocminfo | grep "Marketing Name" | tail -n1 | awk -F':' '{print $2}' | xargs)
2025-07-24T04:44:02.9432684Z fi
2025-07-24T04:44:02.9433086Z 
2025-07-24T04:44:02.9433587Z echo "DEVICE_NAME=$DEVICE_NAME" >> $GITHUB_ENV
2025-07-24T04:44:02.9434418Z echo "DEVICE_TYPE=$DEVICE_TYPE" >> $GITHUB_ENV
2025-07-24T04:44:02.9494291Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:44:02.9494938Z env:
2025-07-24T04:44:02.9495346Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:02.9496094Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:02.9497192Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:02.9498211Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:02.9499805Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:02.9501235Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:02.9502168Z   AWS_REGION: us-east-1
2025-07-24T04:44:02.9502747Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:02.9503431Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:02.9515341Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:02.9516116Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:02.9516906Z ##[endgroup]
2025-07-24T04:44:02.9614425Z + python3 -mpip install boto3==1.35.33 psutil==7.0.0 pynvml==12.0.0
2025-07-24T04:44:03.2487372Z Defaulting to user installation because normal site-packages is not writeable
2025-07-24T04:44:03.4210701Z Requirement already satisfied: boto3==1.35.33 in /var/home/pytorchci/.local/lib/python3.10/site-packages (1.35.33)
2025-07-24T04:44:03.4214851Z Requirement already satisfied: psutil==7.0.0 in /var/home/pytorchci/.local/lib/python3.10/site-packages (7.0.0)
2025-07-24T04:44:03.4220582Z Requirement already satisfied: pynvml==12.0.0 in /var/home/pytorchci/.local/lib/python3.10/site-packages (12.0.0)
2025-07-24T04:44:03.4255228Z Requirement already satisfied: botocore<1.36.0,>=1.35.33 in /var/home/pytorchci/.local/lib/python3.10/site-packages (from boto3==1.35.33) (1.35.99)
2025-07-24T04:44:03.4260592Z Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/lib/python3/dist-packages (from boto3==1.35.33) (0.10.0)
2025-07-24T04:44:03.4264564Z Requirement already satisfied: s3transfer<0.11.0,>=0.10.0 in /var/home/pytorchci/.local/lib/python3.10/site-packages (from boto3==1.35.33) (0.10.4)
2025-07-24T04:44:03.4417198Z Requirement already satisfied: nvidia-ml-py<13.0.0a0,>=12.0.0 in /var/home/pytorchci/.local/lib/python3.10/site-packages (from pynvml==12.0.0) (12.575.51)
2025-07-24T04:44:03.4465252Z Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in /usr/lib/python3/dist-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.26.5)
2025-07-24T04:44:03.4471114Z Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /var/home/pytorchci/.local/lib/python3.10/site-packages (from botocore<1.36.0,>=1.35.33->boto3==1.35.33) (2.9.0.post0)
2025-07-24T04:44:03.4514175Z Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.36.0,>=1.35.33->boto3==1.35.33) (1.16.0)
2025-07-24T04:44:03.7323000Z + DEVICE_NAME=
2025-07-24T04:44:03.7323585Z + DEVICE_TYPE=
2025-07-24T04:44:03.7324128Z + command -v nvidia-smi
2025-07-24T04:44:03.7324682Z + command -v rocminfo
2025-07-24T04:44:03.7325217Z + DEVICE_NAME=rocm
2025-07-24T04:44:03.7325728Z /opt/rocm/bin/rocminfo
2025-07-24T04:44:03.7341216Z ++ rocminfo
2025-07-24T04:44:03.7345516Z ++ grep 'Marketing Name'
2025-07-24T04:44:03.7348913Z ++ tail -n1
2025-07-24T04:44:03.7353958Z ++ awk -F: '{print $2}'
2025-07-24T04:44:03.7356634Z ++ xargs
2025-07-24T04:44:03.8951880Z + DEVICE_TYPE='AMD Instinct MI250X/MI250'
2025-07-24T04:44:03.8952629Z + echo DEVICE_NAME=rocm
2025-07-24T04:44:03.8953231Z + echo 'DEVICE_TYPE=AMD Instinct MI250X/MI250'
2025-07-24T04:44:03.9002204Z ##[group]Run set -eux
2025-07-24T04:44:03.9002780Z set -eux
2025-07-24T04:44:03.9003282Z 
2025-07-24T04:44:03.9003806Z if [[ -z "${GITHUB_TOKEN}" ]]; then
2025-07-24T04:44:03.9004524Z   echo "Missing github-token input"
2025-07-24T04:44:03.9005145Z   exit 1
2025-07-24T04:44:03.9005610Z fi
2025-07-24T04:44:03.9073515Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:44:03.9074291Z env:
2025-07-24T04:44:03.9075086Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:03.9075943Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:03.9077195Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:03.9078365Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:03.9080208Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:03.9082338Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:03.9082974Z   AWS_REGION: us-east-1
2025-07-24T04:44:03.9083647Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:03.9084414Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:03.9097722Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:03.9098580Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:03.9099478Z   DEVICE_NAME: rocm
2025-07-24T04:44:03.9100008Z   DEVICE_TYPE: AMD Instinct MI250X/MI250
2025-07-24T04:44:03.9100906Z   GITHUB_TOKEN: ***
2025-07-24T04:44:03.9101400Z ##[endgroup]
2025-07-24T04:44:03.9208460Z + [[ -z *** ]]
2025-07-24T04:44:03.9298830Z ##[group]Run pytorch/test-infra/.github/actions/get-workflow-job-id@main
2025-07-24T04:44:03.9299720Z with:
2025-07-24T04:44:03.9300478Z   github-token: ***
2025-07-24T04:44:03.9301002Z env:
2025-07-24T04:44:03.9301484Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:03.9302416Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:03.9303707Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:03.9304878Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:03.9306784Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:03.9308484Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:03.9309099Z   AWS_REGION: us-east-1
2025-07-24T04:44:03.9309737Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:03.9310562Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:03.9325036Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:03.9325932Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:03.9326889Z   DEVICE_NAME: rocm
2025-07-24T04:44:03.9327462Z   DEVICE_TYPE: AMD Instinct MI250X/MI250
2025-07-24T04:44:03.9328107Z ##[endgroup]
2025-07-24T04:44:03.9360592Z ##[group]Run set -eux
2025-07-24T04:44:03.9361163Z set -eux
2025-07-24T04:44:03.9361649Z 
2025-07-24T04:44:03.9362628Z python3 "${GITHUB_ACTION_PATH}/../../scripts/get_workflow_job_id.py" "${GITHUB_RUN_ID}" "${RUNNER_NAME}"
2025-07-24T04:44:03.9433047Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:44:03.9433847Z env:
2025-07-24T04:44:03.9434720Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:03.9435660Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:03.9436936Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:03.9438592Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:03.9440505Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:03.9442203Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:03.9442823Z   AWS_REGION: us-east-1
2025-07-24T04:44:03.9443476Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:03.9444169Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:03.9456078Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:03.9457386Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:03.9485419Z   DEVICE_NAME: rocm
2025-07-24T04:44:03.9485950Z   DEVICE_TYPE: AMD Instinct MI250X/MI250
2025-07-24T04:44:03.9486876Z   GITHUB_TOKEN: ***
2025-07-24T04:44:03.9487346Z ##[endgroup]
2025-07-24T04:44:03.9590940Z + python3 /var/home/pytorchci/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/get-workflow-job-id/../../scripts/get_workflow_job_id.py 16487210679 gpud553
2025-07-24T04:44:04.9201686Z setting job-id=46615439132
2025-07-24T04:44:04.9202756Z setting job-name=linux-jammy-rocm-py3.10 / test (default, 2, 2, linux.rocm.gpu.2)
2025-07-24T04:44:04.9451849Z ##[group]Run set -eux
2025-07-24T04:44:04.9452935Z set -eux
2025-07-24T04:44:04.9453462Z 
2025-07-24T04:44:04.9454285Z python3 "${GITHUB_ACTION_PATH}/../../scripts/benchmarks/gather_metadata.py" \
2025-07-24T04:44:04.9455341Z   --schema-version "${SCHEMA_VERSION}" \
2025-07-24T04:44:04.9456089Z   --repo "${REPO}" \
2025-07-24T04:44:04.9456773Z   --head-branch "${HEAD_BRANCH}" \
2025-07-24T04:44:04.9457496Z   --head-sha "${HEAD_SHA}" \
2025-07-24T04:44:04.9458218Z   --workflow-id "${WORKFLOW_RUN_ID}" \
2025-07-24T04:44:04.9458974Z   --run-attempt "${RUN_ATTEMPT}" \
2025-07-24T04:44:04.9459690Z   --job-id "${JOB_ID}" \
2025-07-24T04:44:04.9460340Z   --job-name "${JOB_NAME}"
2025-07-24T04:44:04.9527857Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:44:04.9528661Z env:
2025-07-24T04:44:04.9529162Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:04.9530074Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:04.9531365Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:04.9532561Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:04.9534492Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:04.9536219Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:04.9536848Z   AWS_REGION: us-east-1
2025-07-24T04:44:04.9537559Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:04.9538382Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:04.9552497Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:04.9553415Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:04.9554486Z   DEVICE_NAME: rocm
2025-07-24T04:44:04.9555080Z   DEVICE_TYPE: AMD Instinct MI250X/MI250
2025-07-24T04:44:04.9555764Z   SCHEMA_VERSION: v3
2025-07-24T04:44:04.9556309Z   REPO: pytorch/pytorch
2025-07-24T04:44:04.9556942Z   HEAD_BRANCH: refs/tags/ciflow/trunk/149961
2025-07-24T04:44:04.9557701Z   HEAD_SHA: 27c8ef1839e0747b36769efb192befc9308ff93c
2025-07-24T04:44:04.9558453Z   WORKFLOW_RUN_ID: 16487210679
2025-07-24T04:44:04.9559057Z   RUN_ATTEMPT: 1
2025-07-24T04:44:04.9559568Z   JOB_ID: 46615439132
2025-07-24T04:44:04.9560371Z   JOB_NAME: linux-jammy-rocm-py3.10 / test (default, 2, 2, linux.rocm.gpu.2)
2025-07-24T04:44:04.9561214Z ##[endgroup]
2025-07-24T04:44:04.9667169Z + python3 /var/home/pytorchci/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/benchmarks/gather_metadata.py --schema-version v3 --repo pytorch/pytorch --head-branch refs/tags/ciflow/trunk/149961 --head-sha 27c8ef1839e0747b36769efb192befc9308ff93c --workflow-id 16487210679 --run-attempt 1 --job-id 46615439132 --job-name 'linux-jammy-rocm-py3.10 / test (default, 2, 2, linux.rocm.gpu.2)'
2025-07-24T04:44:05.0262994Z ##[group]Run set -eux
2025-07-24T04:44:05.0263573Z set -eux
2025-07-24T04:44:05.0264049Z 
2025-07-24T04:44:05.0264832Z python3 "${GITHUB_ACTION_PATH}/../../scripts/benchmarks/gather_runners_info.py"
2025-07-24T04:44:05.0331650Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:44:05.0332413Z env:
2025-07-24T04:44:05.0332884Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:05.0333890Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:05.0335344Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:05.0336667Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:05.0338500Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:05.0340536Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:05.0341144Z   AWS_REGION: us-east-1
2025-07-24T04:44:05.0341805Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:05.0342596Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:05.0356182Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:05.0357035Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:05.0357943Z   DEVICE_NAME: rocm
2025-07-24T04:44:05.0358483Z   DEVICE_TYPE: AMD Instinct MI250X/MI250
2025-07-24T04:44:05.0359112Z ##[endgroup]
2025-07-24T04:44:05.0465406Z + python3 /var/home/pytorchci/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/benchmarks/gather_runners_info.py
2025-07-24T04:44:05.8164128Z /var/home/pytorchci/.local/lib/python3.10/site-packages/torch/_subclasses/functional_tensor.py:276: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:81.)
2025-07-24T04:44:05.8166166Z   cpu = _conversion_method_template(device=torch.device("cpu"))
2025-07-24T04:44:06.4266072Z ##[group]Run set -eux
2025-07-24T04:44:06.4266637Z set -eux
2025-07-24T04:44:06.4267099Z 
2025-07-24T04:44:06.4267627Z # TODO (huydhn): Implement this part
2025-07-24T04:44:06.4268388Z echo "dependencies={}" >> "${GITHUB_OUTPUT}"
2025-07-24T04:44:06.4337685Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:44:06.4338441Z env:
2025-07-24T04:44:06.4338922Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:06.4339755Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:06.4341018Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:06.4342133Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:06.4343932Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:06.4345550Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:06.4346129Z   AWS_REGION: us-east-1
2025-07-24T04:44:06.4346767Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:06.4347531Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:06.4361300Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:06.4362301Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:06.4363347Z   DEVICE_NAME: rocm
2025-07-24T04:44:06.4363989Z   DEVICE_TYPE: AMD Instinct MI250X/MI250
2025-07-24T04:44:06.4365262Z ##[endgroup]
2025-07-24T04:44:06.4460732Z + echo 'dependencies={}'
2025-07-24T04:44:06.4499738Z ##[group]Run set -eux
2025-07-24T04:44:06.4500300Z set -eux
2025-07-24T04:44:06.4500771Z 
2025-07-24T04:44:06.4501340Z if [[ ! -d "${BENCHMARK_RESULTS_DIR}" ]]; then
2025-07-24T04:44:06.4502192Z   echo "${BENCHMARK_RESULTS_DIR} does not exist, skipping"
2025-07-24T04:44:06.4503145Z   # We don't want the job to fail if the directory doesn't exist
2025-07-24T04:44:06.4503904Z   exit 0
2025-07-24T04:44:06.4504356Z fi
2025-07-24T04:44:06.4504790Z 
2025-07-24T04:44:06.4505274Z if [[ "${DRY_RUN}" == "true" ]]; then
2025-07-24T04:44:06.4506208Z   python3 "${GITHUB_ACTION_PATH}/../../scripts/upload_benchmark_results.py" \
2025-07-24T04:44:06.4507289Z     --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \
2025-07-24T04:44:06.4508114Z     --metadata "${BENCHMARK_METADATA}" \
2025-07-24T04:44:06.4508821Z     --runners "${RUNNER_INFO}" \
2025-07-24T04:44:06.4509494Z     --dependencies "${DEPENDENCIES}" \
2025-07-24T04:44:06.4510137Z     --dry-run
2025-07-24T04:44:06.4510644Z else
2025-07-24T04:44:06.4511379Z   python3 "${GITHUB_ACTION_PATH}/../../scripts/upload_benchmark_results.py" \
2025-07-24T04:44:06.4512788Z     --benchmark-results-dir "${BENCHMARK_RESULTS_DIR}" \
2025-07-24T04:44:06.4513634Z     --metadata "${BENCHMARK_METADATA}" \
2025-07-24T04:44:06.4514419Z     --runners "${RUNNER_INFO}" \
2025-07-24T04:44:06.4515089Z     --dependencies "${DEPENDENCIES}"
2025-07-24T04:44:06.4515717Z fi
2025-07-24T04:44:06.4577892Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:44:06.4578626Z env:
2025-07-24T04:44:06.4579089Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:06.4579922Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:06.4581118Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:06.4582248Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:06.4584043Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:06.4585664Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:06.4586269Z   AWS_REGION: us-east-1
2025-07-24T04:44:06.4586889Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:06.4587651Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:06.4601554Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:06.4602566Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:06.4603613Z   DEVICE_NAME: rocm
2025-07-24T04:44:06.4604249Z   DEVICE_TYPE: AMD Instinct MI250X/MI250
2025-07-24T04:44:06.4605059Z   BENCHMARK_RESULTS_DIR: test/test-reports
2025-07-24T04:44:06.4605811Z   DRY_RUN: false
2025-07-24T04:44:06.4608175Z   BENCHMARK_METADATA: {"timestamp": 1753332245, "schema_version": "v3", "name": "linux-jammy-rocm-py3.10 / test (default, 2, 2, linux.rocm.gpu.2)", "repo": "pytorch/pytorch", "head_branch": "refs/tags/ciflow/trunk/149961", "head_sha": "27c8ef1839e0747b36769efb192befc9308ff93c", "workflow_id": 16487210679, "run_attempt": 1, "job_id": 46615439132}
2025-07-24T04:44:06.4611362Z   RUNNER_INFO: [{"cpu_info": "x86_64", "cpu_count": 128, "avail_mem_in_gb": 1007, "extra_info": {"hostname": "gpud553.jax.cs.cpe.ice.amd.com"}, "name": "rocm", "type": "AMD Instinct MI250X/MI250"}]
2025-07-24T04:44:06.4612953Z   DEPENDENCIES: {}
2025-07-24T04:44:06.4613540Z ##[endgroup]
2025-07-24T04:44:06.4717518Z + [[ ! -d test/test-reports ]]
2025-07-24T04:44:06.4718273Z + [[ false == \t\r\u\e ]]
2025-07-24T04:44:06.4723764Z + python3 /var/home/pytorchci/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/upload_benchmark_results.py --benchmark-results-dir test/test-reports --metadata '{"timestamp": 1753332245, "schema_version": "v3", "name": "linux-jammy-rocm-py3.10 / test (default, 2, 2, linux.rocm.gpu.2)", "repo": "pytorch/pytorch", "head_branch": "refs/tags/ciflow/trunk/149961", "head_sha": "27c8ef1839e0747b36769efb192befc9308ff93c", "workflow_id": 16487210679, "run_attempt": 1, "job_id": 46615439132}' --runners '[{"cpu_info": "x86_64", "cpu_count": 128, "avail_mem_in_gb": 1007, "extra_info": {"hostname": "gpud553.jax.cs.cpe.ice.amd.com"}, "name": "rocm", "type": "AMD Instinct MI250X/MI250"}]' --dependencies '{}'
2025-07-24T04:44:06.6309045Z /var/home/pytorchci/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/upload_benchmark_results.py:236: UserWarning: {'included': [{'test_file': 'inductor/test_torchinductor'}, {'test_file': 'test_autograd'}, {'test_file': 'test_torch'}], 'excluded': [{'test_file': 'test_ops'}, {'test_file': 'test_binary_ufuncs'}, {'test_file': 'test_nn'}, {'test_file': 'test_unary_ufuncs'}, {'test_file': 'test_cuda'}]} from test/test-reports/td_exclusions-c4d78f1a178072c13d80.json is not a benchmark record, skipping
2025-07-24T04:44:06.6313448Z   warn(f"{result} from {filepath} is not a benchmark record, skipping")
2025-07-24T04:44:06.6321136Z /var/home/pytorchci/actions-runner/_work/_actions/pytorch/test-infra/main/.github/actions/upload-benchmark-results/../../scripts/upload_benchmark_results.py:236: UserWarning: {'included': [{'test_file': 'cpp/Dict_test'}, {'test_file': 'cpp/Dimname_test'}, {'test_file': 'cpp/NamedTensor_test'}, {'test_file': 'cpp/apply_utils_test'}, {'test_file': 'cpp/atest'}], 'excluded': [{'test_file': 'cpp/basic'}, {'test_file': 'cpp/broadcast_test'}, {'test_file': 'cpp/cpu_generator_test'}, {'test_file': 'cpp/dlconvertor_test'}, {'test_file': 'cpp/extension_backend_test'}, {'test_file': 'cpp/lazy_tensor_test'}, {'test_file': 'cpp/legacy_vmap_test'}, {'test_file': 'cpp/native_test'}, {'test_file': 'cpp/operators_test'}, {'test_file': 'cpp/scalar_tensor_test'}, {'test_file': 'cpp/scalar_test'}, {'test_file': 'cpp/tensor_iterator_test'}, {'test_file': 'cpp/undefined_tensor_test'}, {'test_file': 'cpp/wrapdim_test'}]} from test/test-reports/td_exclusions-dd839bb3c0c96df0a48b.json is not a benchmark record, skipping
2025-07-24T04:44:06.6327984Z   warn(f"{result} from {filepath} is not a benchmark record, skipping")
2025-07-24T04:44:06.6558528Z Prepare all required actions
2025-07-24T04:44:06.6559366Z Getting action download info
2025-07-24T04:44:06.6614587Z ##[group]Run ./.github/actions/teardown-rocm
2025-07-24T04:44:06.6615232Z env:
2025-07-24T04:44:06.6615680Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:06.6616645Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:06.6618100Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:06.6619393Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:06.6621238Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:06.6622874Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:06.6623449Z   AWS_REGION: us-east-1
2025-07-24T04:44:06.6624073Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:06.6624883Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:06.6638299Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:06.6639142Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:06.6640027Z   DEVICE_NAME: rocm
2025-07-24T04:44:06.6640561Z   DEVICE_TYPE: AMD Instinct MI250X/MI250
2025-07-24T04:44:06.6641168Z ##[endgroup]
2025-07-24T04:44:06.6672506Z ##[group]Run # ignore expansion of "docker ps -q" since it could be empty
2025-07-24T04:44:06.6673545Z # ignore expansion of "docker ps -q" since it could be empty
2025-07-24T04:44:06.6674726Z # shellcheck disable=SC2046
2025-07-24T04:44:06.6675952Z docker stop $(docker ps -q) || true
2025-07-24T04:44:06.6676646Z # Prune all stopped containers.
2025-07-24T04:44:06.6677302Z docker container prune -f
2025-07-24T04:44:06.6709612Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:44:06.6709924Z env:
2025-07-24T04:44:06.6710126Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:06.6710470Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:06.6710965Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:06.6711426Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:06.6712162Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:06.6712819Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:06.6713082Z   AWS_REGION: us-east-1
2025-07-24T04:44:06.6713335Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:06.6713648Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:06.6719123Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:06.6719477Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:06.6719847Z   DEVICE_NAME: rocm
2025-07-24T04:44:06.6720078Z   DEVICE_TYPE: AMD Instinct MI250X/MI250
2025-07-24T04:44:06.6720335Z ##[endgroup]
2025-07-24T04:44:17.4647265Z 51a08ea8590d
2025-07-24T04:44:24.5021711Z Deleted Containers:
2025-07-24T04:44:24.5023304Z 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:24.5024142Z 
2025-07-24T04:44:24.5024515Z Total reclaimed space: 9.562GB
2025-07-24T04:44:24.5166309Z Prepare all required actions
2025-07-24T04:44:24.5228710Z ##[group]Run ./.github/actions/diskspace-cleanup
2025-07-24T04:44:24.5229433Z with:
2025-07-24T04:44:24.5229930Z   diskspace-cutoff: 70
2025-07-24T04:44:24.5230505Z env:
2025-07-24T04:44:24.5230986Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:24.5231859Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:24.5233122Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:24.5234684Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:24.5237156Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:24.5238952Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:24.5239580Z   AWS_REGION: us-east-1
2025-07-24T04:44:24.5240263Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:24.5241074Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:24.5255209Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:24.5256123Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:24.5257073Z   DEVICE_NAME: rocm
2025-07-24T04:44:24.5257647Z   DEVICE_TYPE: AMD Instinct MI250X/MI250
2025-07-24T04:44:24.5258308Z ##[endgroup]
2025-07-24T04:44:24.5293127Z ##[group]Run set -ex
2025-07-24T04:44:24.5293739Z set -ex
2025-07-24T04:44:24.5294287Z diskspace_cutoff=70
2025-07-24T04:44:24.5295088Z docker_root_dir=$(docker info -f '{{.DockerRootDir}}')
2025-07-24T04:44:24.5295965Z if [ ! -d "$docker_root_dir" ]; then
2025-07-24T04:44:24.5297063Z     echo "Docker root directory ($docker_root_dir) does not exist. Skipping disk space check."
2025-07-24T04:44:24.5298113Z     exit 0
2025-07-24T04:44:24.5298636Z fi
2025-07-24T04:44:24.5299515Z diskspace=$(df -H --output=pcent ${docker_root_dir} | sed -n 2p | sed 's/%//' | sed 's/ //')
2025-07-24T04:44:24.5301282Z msg="Please file an issue on pytorch/pytorch reporting the faulty runner. Include a link to the runner logs so the runner can be identified"
2025-07-24T04:44:24.5303285Z if [[ "$diskspace" -ge "$diskspace_cutoff" ]] ; then
2025-07-24T04:44:24.5304103Z     docker system prune -af
2025-07-24T04:44:24.5305164Z     diskspace_new=$(df -H --output=pcent ${docker_root_dir} | sed -n 2p | sed 's/%//' | sed 's/ //')
2025-07-24T04:44:24.5306383Z     if [[ "$diskspace_new" -gt "$diskspace_cutoff" ]] ; then
2025-07-24T04:44:24.5307611Z         echo "Error: Available diskspace is less than $diskspace_cutoff percent. Not enough diskspace."
2025-07-24T04:44:24.5308710Z         echo "$msg"
2025-07-24T04:44:24.5309308Z         exit 1
2025-07-24T04:44:24.5309844Z     else
2025-07-24T04:44:24.5310488Z         difference=$((diskspace - diskspace_new))
2025-07-24T04:44:24.5311333Z         echo "Diskspace saved: $difference percent"
2025-07-24T04:44:24.5312054Z     fi
2025-07-24T04:44:24.5312552Z fi
2025-07-24T04:44:24.5376999Z shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
2025-07-24T04:44:24.5377711Z env:
2025-07-24T04:44:24.5378131Z   GIT_DEFAULT_BRANCH: main
2025-07-24T04:44:24.5378903Z   RUNNER_ARTIFACT_DIR: /var/home/pytorchci/actions-runner/_work/_temp/artifacts
2025-07-24T04:44:24.5379995Z   RUNNER_TEST_RESULTS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/test-results
2025-07-24T04:44:24.5380987Z   RUNNER_DOCS_DIR: /var/home/pytorchci/actions-runner/_work/_temp/docs
2025-07-24T04:44:24.5382605Z   GPU_FLAG: --device=/dev/mem --device=/dev/kfd --device /dev/dri --group-add video --group-add 110 --group-add daemon --group-add bin --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host
2025-07-24T04:44:24.5384051Z   AWS_DEFAULT_REGION: us-east-1
2025-07-24T04:44:24.5384581Z   AWS_REGION: us-east-1
2025-07-24T04:44:24.5385135Z   AWS_ACCESS_KEY_ID: ***
2025-07-24T04:44:24.5385815Z   AWS_SECRET_ACCESS_KEY: ***
2025-07-24T04:44:24.5397728Z   AWS_SESSION_TOKEN: ***
2025-07-24T04:44:24.5398480Z   CONTAINER_NAME: 51a08ea8590d2780137e33cda101c868d63c8b3a4972d514cc5c474768d25e3c
2025-07-24T04:44:24.5399288Z   DEVICE_NAME: rocm
2025-07-24T04:44:24.5399757Z   DEVICE_TYPE: AMD Instinct MI250X/MI250
2025-07-24T04:44:24.5400317Z ##[endgroup]
2025-07-24T04:44:24.5494787Z + diskspace_cutoff=70
2025-07-24T04:44:24.5503712Z ++ docker info -f '{{.DockerRootDir}}'
2025-07-24T04:44:24.6459539Z + docker_root_dir=/media/4TB/docker-rootless
2025-07-24T04:44:24.6460451Z + '[' '!' -d /media/4TB/docker-rootless ']'
2025-07-24T04:44:24.6474978Z ++ df -H --output=pcent /media/4TB/docker-rootless
2025-07-24T04:44:24.6477547Z ++ sed -n 2p
2025-07-24T04:44:24.6481044Z ++ sed s/%//
2025-07-24T04:44:24.6483268Z ++ sed 's/ //'
2025-07-24T04:44:24.6527951Z + diskspace=61
2025-07-24T04:44:24.6529312Z + msg='Please file an issue on pytorch/pytorch reporting the faulty runner. Include a link to the runner logs so the runner can be identified'
2025-07-24T04:44:24.6530597Z + [[ 61 -ge 70 ]]
2025-07-24T04:44:24.6619538Z Post job cleanup.
2025-07-24T04:44:24.6720094Z Post job cleanup.
2025-07-24T04:44:24.8185410Z Post job cleanup.
2025-07-24T04:44:24.8826618Z Logging out of registry 308535385114.dkr.ecr.us-east-1.amazonaws.com
2025-07-24T04:44:24.9390121Z Post job cleanup.
2025-07-24T04:44:25.0887574Z Post job cleanup.
2025-07-24T04:44:25.0968438Z Post job cleanup.
2025-07-24T04:44:25.2120667Z [command]/usr/bin/git version
2025-07-24T04:44:25.2177913Z git version 2.34.1
2025-07-24T04:44:25.2211798Z Copying '/var/home/pytorchci/.gitconfig' to '/var/home/pytorchci/actions-runner/_work/_temp/dfba5b1d-3818-41b7-bba9-b05042b07c5d/.gitconfig'
2025-07-24T04:44:25.2225374Z Temporarily overriding HOME='/var/home/pytorchci/actions-runner/_work/_temp/dfba5b1d-3818-41b7-bba9-b05042b07c5d' before making global git config changes
2025-07-24T04:44:25.2227368Z Adding repository directory to the temporary git global config as a safe directory
2025-07-24T04:44:25.2229017Z [command]/usr/bin/git config --global --add safe.directory /var/home/pytorchci/actions-runner/_work/pytorch/pytorch
2025-07-24T04:44:25.2321458Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2025-07-24T04:44:25.2408254Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :"
2025-07-24T04:44:25.3088857Z Entering 'android/libs/fbjni'
2025-07-24T04:44:25.3223170Z Entering 'third_party/FP16'
2025-07-24T04:44:25.3347563Z Entering 'third_party/FXdiv'
2025-07-24T04:44:25.3469848Z Entering 'third_party/NNPACK'
2025-07-24T04:44:25.3592546Z Entering 'third_party/NVTX'
2025-07-24T04:44:25.3721575Z Entering 'third_party/VulkanMemoryAllocator'
2025-07-24T04:44:25.3841166Z Entering 'third_party/XNNPACK'
2025-07-24T04:44:25.3999707Z Entering 'third_party/aiter'
2025-07-24T04:44:25.4127998Z Entering 'third_party/aiter/3rdparty/composable_kernel'
2025-07-24T04:44:25.4271369Z Entering 'third_party/benchmark'
2025-07-24T04:44:25.4395436Z Entering 'third_party/composable_kernel'
2025-07-24T04:44:25.4536468Z Entering 'third_party/cpp-httplib'
2025-07-24T04:44:25.4667899Z Entering 'third_party/cpuinfo'
2025-07-24T04:44:25.4794590Z Entering 'third_party/cudnn_frontend'
2025-07-24T04:44:25.4913542Z Entering 'third_party/cutlass'
2025-07-24T04:44:25.5063099Z Entering 'third_party/fbgemm'
2025-07-24T04:44:25.5183666Z Entering 'third_party/fbgemm/external/asmjit'
2025-07-24T04:44:25.5301510Z Entering 'third_party/fbgemm/external/composable_kernel'
2025-07-24T04:44:25.5423078Z Entering 'third_party/fbgemm/external/cpuinfo'
2025-07-24T04:44:25.5546159Z Entering 'third_party/fbgemm/external/cutlass'
2025-07-24T04:44:25.5687065Z Entering 'third_party/fbgemm/external/googletest'
2025-07-24T04:44:25.5804928Z Entering 'third_party/fbgemm/external/hipify_torch'
2025-07-24T04:44:25.5920893Z Entering 'third_party/fbgemm/external/json'
2025-07-24T04:44:25.6047126Z Entering 'third_party/flash-attention'
2025-07-24T04:44:25.6167670Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-07-24T04:44:25.6298985Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-07-24T04:44:25.6440237Z Entering 'third_party/flatbuffers'
2025-07-24T04:44:25.6566808Z Entering 'third_party/fmt'
2025-07-24T04:44:25.6689566Z Entering 'third_party/gemmlowp/gemmlowp'
2025-07-24T04:44:25.6811150Z Entering 'third_party/gloo'
2025-07-24T04:44:25.6932434Z Entering 'third_party/googletest'
2025-07-24T04:44:25.7058691Z Entering 'third_party/ideep'
2025-07-24T04:44:25.7174006Z Entering 'third_party/ideep/mkl-dnn'
2025-07-24T04:44:25.7312966Z Entering 'third_party/ittapi'
2025-07-24T04:44:25.7435230Z Entering 'third_party/kineto'
2025-07-24T04:44:25.7550418Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-07-24T04:44:25.7668660Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-07-24T04:44:25.7789388Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-07-24T04:44:25.7908243Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-07-24T04:44:25.8025821Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-07-24T04:44:25.8138179Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-07-24T04:44:25.8262016Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-07-24T04:44:25.8380340Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-07-24T04:44:25.8495995Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-07-24T04:44:25.8616941Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-07-24T04:44:25.8741263Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-07-24T04:44:25.8862344Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-07-24T04:44:25.8985461Z Entering 'third_party/kleidiai'
2025-07-24T04:44:25.9105094Z Entering 'third_party/mimalloc'
2025-07-24T04:44:25.9224637Z Entering 'third_party/nlohmann'
2025-07-24T04:44:25.9347641Z Entering 'third_party/onnx'
2025-07-24T04:44:25.9501008Z Entering 'third_party/onnx/third_party/pybind11'
2025-07-24T04:44:25.9632962Z Entering 'third_party/opentelemetry-cpp'
2025-07-24T04:44:25.9738348Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-07-24T04:44:25.9855013Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-07-24T04:44:25.9971809Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-07-24T04:44:26.0088284Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-07-24T04:44:26.0211418Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-07-24T04:44:26.0328264Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-07-24T04:44:26.0446597Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-07-24T04:44:26.0558901Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-07-24T04:44:26.0680606Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-07-24T04:44:26.0805733Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-07-24T04:44:26.0975045Z Entering 'third_party/pocketfft'
2025-07-24T04:44:26.1098465Z Entering 'third_party/protobuf'
2025-07-24T04:44:26.1226788Z Entering 'third_party/protobuf/third_party/benchmark'
2025-07-24T04:44:26.1346069Z Entering 'third_party/protobuf/third_party/googletest'
2025-07-24T04:44:26.1475071Z Entering 'third_party/psimd'
2025-07-24T04:44:26.1598838Z Entering 'third_party/pthreadpool'
2025-07-24T04:44:26.1721535Z Entering 'third_party/pybind11'
2025-07-24T04:44:26.1847661Z Entering 'third_party/python-peachpy'
2025-07-24T04:44:26.1973348Z Entering 'third_party/sleef'
2025-07-24T04:44:26.2096405Z Entering 'third_party/tensorpipe'
2025-07-24T04:44:26.2217119Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-07-24T04:44:26.2335276Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-07-24T04:44:26.2451946Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-07-24T04:44:26.2569874Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-07-24T04:44:26.2681756Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-07-24T04:44:26.2854559Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2025-07-24T04:44:26.2912911Z http.https://github.com/.extraheader
2025-07-24T04:44:26.2937540Z [command]/usr/bin/git config --local --unset-all http.https://github.com/.extraheader
2025-07-24T04:44:26.3020866Z [command]/usr/bin/git submodule foreach --recursive sh -c "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :"
2025-07-24T04:44:26.3698771Z Entering 'android/libs/fbjni'
2025-07-24T04:44:26.3765452Z http.https://github.com/.extraheader
2025-07-24T04:44:26.3862875Z Entering 'third_party/FP16'
2025-07-24T04:44:26.3928361Z http.https://github.com/.extraheader
2025-07-24T04:44:26.4022384Z Entering 'third_party/FXdiv'
2025-07-24T04:44:26.4088046Z http.https://github.com/.extraheader
2025-07-24T04:44:26.4184164Z Entering 'third_party/NNPACK'
2025-07-24T04:44:26.4250248Z http.https://github.com/.extraheader
2025-07-24T04:44:26.4344568Z Entering 'third_party/NVTX'
2025-07-24T04:44:26.4409691Z http.https://github.com/.extraheader
2025-07-24T04:44:26.4505370Z Entering 'third_party/VulkanMemoryAllocator'
2025-07-24T04:44:26.4569235Z http.https://github.com/.extraheader
2025-07-24T04:44:26.4666408Z Entering 'third_party/XNNPACK'
2025-07-24T04:44:26.4731986Z http.https://github.com/.extraheader
2025-07-24T04:44:26.4862795Z Entering 'third_party/aiter'
2025-07-24T04:44:26.4927811Z http.https://github.com/.extraheader
2025-07-24T04:44:26.5020227Z Entering 'third_party/aiter/3rdparty/composable_kernel'
2025-07-24T04:44:26.5085233Z http.https://github.com/.extraheader
2025-07-24T04:44:26.5203930Z Entering 'third_party/benchmark'
2025-07-24T04:44:26.5268905Z http.https://github.com/.extraheader
2025-07-24T04:44:26.5365672Z Entering 'third_party/composable_kernel'
2025-07-24T04:44:26.5430402Z http.https://github.com/.extraheader
2025-07-24T04:44:26.5541021Z Entering 'third_party/cpp-httplib'
2025-07-24T04:44:26.5607553Z http.https://github.com/.extraheader
2025-07-24T04:44:26.5703085Z Entering 'third_party/cpuinfo'
2025-07-24T04:44:26.5769001Z http.https://github.com/.extraheader
2025-07-24T04:44:26.5865347Z Entering 'third_party/cudnn_frontend'
2025-07-24T04:44:26.5932064Z http.https://github.com/.extraheader
2025-07-24T04:44:26.6027488Z Entering 'third_party/cutlass'
2025-07-24T04:44:26.6092828Z http.https://github.com/.extraheader
2025-07-24T04:44:26.6209337Z Entering 'third_party/fbgemm'
2025-07-24T04:44:26.6274761Z http.https://github.com/.extraheader
2025-07-24T04:44:26.6378191Z Entering 'third_party/fbgemm/external/asmjit'
2025-07-24T04:44:26.6441670Z http.https://github.com/.extraheader
2025-07-24T04:44:26.6533925Z Entering 'third_party/fbgemm/external/composable_kernel'
2025-07-24T04:44:26.6597372Z http.https://github.com/.extraheader
2025-07-24T04:44:26.6703642Z Entering 'third_party/fbgemm/external/cpuinfo'
2025-07-24T04:44:26.6769136Z http.https://github.com/.extraheader
2025-07-24T04:44:26.6865885Z Entering 'third_party/fbgemm/external/cutlass'
2025-07-24T04:44:26.6933367Z http.https://github.com/.extraheader
2025-07-24T04:44:26.7047599Z Entering 'third_party/fbgemm/external/googletest'
2025-07-24T04:44:26.7110539Z http.https://github.com/.extraheader
2025-07-24T04:44:26.7206708Z Entering 'third_party/fbgemm/external/hipify_torch'
2025-07-24T04:44:26.7269503Z http.https://github.com/.extraheader
2025-07-24T04:44:26.7361238Z Entering 'third_party/fbgemm/external/json'
2025-07-24T04:44:26.7424481Z http.https://github.com/.extraheader
2025-07-24T04:44:26.7525268Z Entering 'third_party/flash-attention'
2025-07-24T04:44:26.7589109Z http.https://github.com/.extraheader
2025-07-24T04:44:26.7683644Z Entering 'third_party/flash-attention/csrc/composable_kernel'
2025-07-24T04:44:26.7746928Z http.https://github.com/.extraheader
2025-07-24T04:44:26.7857358Z Entering 'third_party/flash-attention/csrc/cutlass'
2025-07-24T04:44:26.7921366Z http.https://github.com/.extraheader
2025-07-24T04:44:26.8040305Z Entering 'third_party/flatbuffers'
2025-07-24T04:44:26.8104446Z http.https://github.com/.extraheader
2025-07-24T04:44:26.8203870Z Entering 'third_party/fmt'
2025-07-24T04:44:26.8267415Z http.https://github.com/.extraheader
2025-07-24T04:44:26.8358374Z Entering 'third_party/gemmlowp/gemmlowp'
2025-07-24T04:44:26.8422931Z http.https://github.com/.extraheader
2025-07-24T04:44:26.8517869Z Entering 'third_party/gloo'
2025-07-24T04:44:26.8584320Z http.https://github.com/.extraheader
2025-07-24T04:44:26.8681989Z Entering 'third_party/googletest'
2025-07-24T04:44:26.8745894Z http.https://github.com/.extraheader
2025-07-24T04:44:26.8839235Z Entering 'third_party/ideep'
2025-07-24T04:44:26.8903625Z http.https://github.com/.extraheader
2025-07-24T04:44:26.8992551Z Entering 'third_party/ideep/mkl-dnn'
2025-07-24T04:44:26.9054378Z http.https://github.com/.extraheader
2025-07-24T04:44:26.9168408Z Entering 'third_party/ittapi'
2025-07-24T04:44:26.9233825Z http.https://github.com/.extraheader
2025-07-24T04:44:26.9328888Z Entering 'third_party/kineto'
2025-07-24T04:44:26.9393201Z http.https://github.com/.extraheader
2025-07-24T04:44:26.9483473Z Entering 'third_party/kineto/libkineto/third_party/dynolog'
2025-07-24T04:44:26.9546015Z http.https://github.com/.extraheader
2025-07-24T04:44:26.9641961Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/DCGM'
2025-07-24T04:44:26.9699888Z http.https://github.com/.extraheader
2025-07-24T04:44:26.9799124Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/cpr'
2025-07-24T04:44:26.9862895Z http.https://github.com/.extraheader
2025-07-24T04:44:26.9960059Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/fmt'
2025-07-24T04:44:27.0021613Z http.https://github.com/.extraheader
2025-07-24T04:44:27.0118433Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags'
2025-07-24T04:44:27.0180711Z http.https://github.com/.extraheader
2025-07-24T04:44:27.0267796Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/gflags/doc'
2025-07-24T04:44:27.0330763Z http.https://github.com/.extraheader
2025-07-24T04:44:27.0432672Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/glog'
2025-07-24T04:44:27.0496425Z http.https://github.com/.extraheader
2025-07-24T04:44:27.0590557Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/googletest'
2025-07-24T04:44:27.0653989Z http.https://github.com/.extraheader
2025-07-24T04:44:27.0748050Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/json'
2025-07-24T04:44:27.0811098Z http.https://github.com/.extraheader
2025-07-24T04:44:27.0909241Z Entering 'third_party/kineto/libkineto/third_party/dynolog/third_party/pfs'
2025-07-24T04:44:27.0971975Z http.https://github.com/.extraheader
2025-07-24T04:44:27.1070510Z Entering 'third_party/kineto/libkineto/third_party/fmt'
2025-07-24T04:44:27.1134043Z http.https://github.com/.extraheader
2025-07-24T04:44:27.1228030Z Entering 'third_party/kineto/libkineto/third_party/googletest'
2025-07-24T04:44:27.1290491Z http.https://github.com/.extraheader
2025-07-24T04:44:27.1389924Z Entering 'third_party/kleidiai'
2025-07-24T04:44:27.1455055Z http.https://github.com/.extraheader
2025-07-24T04:44:27.1550772Z Entering 'third_party/mimalloc'
2025-07-24T04:44:27.1615060Z http.https://github.com/.extraheader
2025-07-24T04:44:27.1708815Z Entering 'third_party/nlohmann'
2025-07-24T04:44:27.1772874Z http.https://github.com/.extraheader
2025-07-24T04:44:27.1868052Z Entering 'third_party/onnx'
2025-07-24T04:44:27.1932896Z http.https://github.com/.extraheader
2025-07-24T04:44:27.2059724Z Entering 'third_party/onnx/third_party/pybind11'
2025-07-24T04:44:27.2124449Z http.https://github.com/.extraheader
2025-07-24T04:44:27.2231625Z Entering 'third_party/opentelemetry-cpp'
2025-07-24T04:44:27.2295128Z http.https://github.com/.extraheader
2025-07-24T04:44:27.2388342Z Entering 'third_party/opentelemetry-cpp/third_party/benchmark'
2025-07-24T04:44:27.2451997Z http.https://github.com/.extraheader
2025-07-24T04:44:27.2543350Z Entering 'third_party/opentelemetry-cpp/third_party/googletest'
2025-07-24T04:44:27.2607425Z http.https://github.com/.extraheader
2025-07-24T04:44:27.2699646Z Entering 'third_party/opentelemetry-cpp/third_party/ms-gsl'
2025-07-24T04:44:27.2761668Z http.https://github.com/.extraheader
2025-07-24T04:44:27.2852221Z Entering 'third_party/opentelemetry-cpp/third_party/nlohmann-json'
2025-07-24T04:44:27.2915188Z http.https://github.com/.extraheader
2025-07-24T04:44:27.3009993Z Entering 'third_party/opentelemetry-cpp/third_party/opentelemetry-proto'
2025-07-24T04:44:27.3072142Z http.https://github.com/.extraheader
2025-07-24T04:44:27.3164909Z Entering 'third_party/opentelemetry-cpp/third_party/opentracing-cpp'
2025-07-24T04:44:27.3227495Z http.https://github.com/.extraheader
2025-07-24T04:44:27.3321283Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp'
2025-07-24T04:44:27.3384375Z http.https://github.com/.extraheader
2025-07-24T04:44:27.3472448Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/civetweb'
2025-07-24T04:44:27.3534264Z http.https://github.com/.extraheader
2025-07-24T04:44:27.3629151Z Entering 'third_party/opentelemetry-cpp/third_party/prometheus-cpp/3rdparty/googletest'
2025-07-24T04:44:27.3692838Z http.https://github.com/.extraheader
2025-07-24T04:44:27.3793135Z Entering 'third_party/opentelemetry-cpp/tools/vcpkg'
2025-07-24T04:44:27.3856360Z http.https://github.com/.extraheader
2025-07-24T04:44:27.3998241Z Entering 'third_party/pocketfft'
2025-07-24T04:44:27.4062643Z http.https://github.com/.extraheader
2025-07-24T04:44:27.4157914Z Entering 'third_party/protobuf'
2025-07-24T04:44:27.4224337Z http.https://github.com/.extraheader
2025-07-24T04:44:27.4325135Z Entering 'third_party/protobuf/third_party/benchmark'
2025-07-24T04:44:27.4387769Z http.https://github.com/.extraheader
2025-07-24T04:44:27.4479227Z Entering 'third_party/protobuf/third_party/googletest'
2025-07-24T04:44:27.4543073Z http.https://github.com/.extraheader
2025-07-24T04:44:27.4643512Z Entering 'third_party/psimd'
2025-07-24T04:44:27.4708504Z http.https://github.com/.extraheader
2025-07-24T04:44:27.4801229Z Entering 'third_party/pthreadpool'
2025-07-24T04:44:27.4866169Z http.https://github.com/.extraheader
2025-07-24T04:44:27.4957149Z Entering 'third_party/pybind11'
2025-07-24T04:44:27.5021473Z http.https://github.com/.extraheader
2025-07-24T04:44:27.5117329Z Entering 'third_party/python-peachpy'
2025-07-24T04:44:27.5184616Z http.https://github.com/.extraheader
2025-07-24T04:44:27.5281729Z Entering 'third_party/sleef'
2025-07-24T04:44:27.5344894Z http.https://github.com/.extraheader
2025-07-24T04:44:27.5438911Z Entering 'third_party/tensorpipe'
2025-07-24T04:44:27.5501614Z http.https://github.com/.extraheader
2025-07-24T04:44:27.5589288Z Entering 'third_party/tensorpipe/third_party/googletest'
2025-07-24T04:44:27.5651159Z http.https://github.com/.extraheader
2025-07-24T04:44:27.5741164Z Entering 'third_party/tensorpipe/third_party/libnop'
2025-07-24T04:44:27.5804736Z http.https://github.com/.extraheader
2025-07-24T04:44:27.5898834Z Entering 'third_party/tensorpipe/third_party/libuv'
2025-07-24T04:44:27.5961565Z http.https://github.com/.extraheader
2025-07-24T04:44:27.6054919Z Entering 'third_party/tensorpipe/third_party/pybind11'
2025-07-24T04:44:27.6117352Z http.https://github.com/.extraheader
2025-07-24T04:44:27.6205449Z Entering 'third_party/tensorpipe/third_party/pybind11/tools/clang'
2025-07-24T04:44:27.6268458Z http.https://github.com/.extraheader
2025-07-24T04:44:27.6688826Z Cleaning up orphan processes